Bavarian AI Chair for Mathematical Foundations of Artificial Intelligence

Breadcrumb Navigation


Expressivity of Neural Networks


Master's thesis / Bachelor's thesis


  • Proficiency with Python and deep learning frameworks (TensorFlow or PyTorch)
  • Preferably Functional Analysis background


Deep Learning models have achieved tremendous success in many application areas. This also raises interesting theoretical questions such as: Why can Deep Learning models adapt to widely different tasks? Are we able to describe their expressive power? A longstanding result in this direction is the Universal Approximator Theorem, which states that neural networks with only two layers can, under weak conditions, can approximate any continuous function on a compact set up to arbitrary precision. However, this is only a starting point to the above questions. A common goal is to relate the approximation accuracy of neural networks to their complexity. It turns out that for many well-known classical function spaces, one can derive upper and lower complexity bounds in terms of the number of weights, neurons, and layers. Another exciting line of research tries to explain the benefit of deep neural networks over shallow ones. Yet another approach studies how neural architectural design choices influence expressivity. These examples are among the many interesting and open research questions within the field of expressivity of neural networks.