The complexity of Neural Networks (available)

Starting Date: June 2023
Prerequisites: linear algebra, calculus, general machine learning and data analysis concepts, python
Will results be assigned to University: No

Neural Networks (NN) are powerful learning machines that allow you to extract meaningful information from historical data. Their strengths come from their flexibility and the possibility of training them through scalable methods, e.g. stochastic gradient descent. On the other hand, since NN depend on millions of real-valued parameters, using them has a high memory and energy footprint.

The recent literature contains examples of training procedures that address the resource problem by producing low-complexity NNs. Popular techniques include discretizing the model parameters [1] or enforcing sparsity [2]. In this project, you will explore these techniques and existing commercial tools (e.g. [3]) to understand the general relationship between computational costs and prediction accuracy and find a good trade-off.

[1] Qin, Haotong, et al. “Binary neural networks: A survey.” Pattern Recognition 105 (2020): 107281. [2] Blalock, Davis, et al. “What is the state of neural network pruning?.” Proceedings of machine learning and systems 2 (2020): 129-146. [3]