Deep learning optimization is often treated as alchemy, yet its success relies on fundamental mathematical structures hidden within high-dimensional landscapes. This keynote deconstructs the training process from first principles, examining the “Big Brothers” that governs optimization: Convexity (geometry), and Smoothness (dynamics).

We demonstrate how physical intuitions link geometry to performance, present the rigorous connection between Optimization and Generalization, and explain essential tools for finding robust solutions in a shifting world.