MH3520¶
MH3520: Mathematics of Deep Learning
Instructor: Zhongjian Wang (zhongjian dot wang at ntu dot edu dot sg).
Timing: Thur 10:30 - 12:20, Fri 15:30 - 17:20. Current at LT5.
Office Hour: Thur 15:30 - 16:20
At the moment, the course aims to cover:
Convergence of gradient descents on convex target: exponential convergence, Bakry–Émery theories.
Generic statistical machine learning theory: decomposition of error, PAC bounds.
Measure theoritic approaches in generative models: metrics in probablity spaces, the diffusion models.
Approximation of MLPs: universality, approximation with spline rates, exponential approximation by deep NNs.
Pre-requisites¶
An expected path to take it in Year 3 Sem 1:
PS0001 Intro to Computational Thinking - Sem 1 Year 1/2
MH2100 Calculus III - Sem 1 Year 2
MH2500 Probability - Sem 1 Year 2
MH2510 Statistics I - Sem 2 Year 2
MH3600 Topology and Manifolds - Sem 2 Year 2
The above requirement, in particular MH3600 or (MH3500 in some alternative paths), is waivable, but you will need to be prepared to learn a lot of new material in a short time (with help of modern LLM, it’s do-able!).
Note
More generally speaking, there may be a gap from the pre-requisites to the course contents, as a result of the teaching infrastructure. The course is not expected to fill it. The course is somehow self-consistent and aims to give you a preview of the fruits in the direction of Math for AI and therefore to get you prepared in the future learning or industial experience.
Course Policies¶
Homework (20%)¶
Weekly assigned, may involves mathematical derivation and coding in Python.
Presentation (20%)¶
You are require to present your solution to the homework or related material during the tutorial of each week.
Exams (60%)¶
There will be a final exam counts 60% which requires you to do some short answer questions. Calculator may be allowed but will not be useful.
Mock exam can be downloaded here.