[Math] How useful is differential geometry and topology to deep learning

dg.differential-geometryhomotopy-theorymachine learning

After seeing this article https://www.quantamagazine.org/an-idea-from-physics-helps-ai-see-in-higher-dimensions-20200109/ I wanted to ask myself how useful of an endeavor would it be if one goes through the process of learning differential geometry to deep learning? Perhaps there are other tools as well?

Can concepts such as homotopy be 'appropriately' encodable in deep learning? I feel notions of homotopy invariance if properly defined could be much more valuable than the simple attempt in quantamagazine article. A basic attempt (away from deep learning framework) is in https://arxiv.org/abs/1408.2584.

Best Answer

A "roadmap type" introduction is given by Roger Grosse in Differential geometry for machine learning.

Differential geometry is all about constructing things which are independent of the representation. You treat the space of objects (e.g. distributions) as a manifold, and describe your algorithm in terms of things that are intrinsic to the manifold itself. While you ultimately need to use some coordinate system to do the actual computations, the higher-level abstractions make it easier to check that the objects you're working with are intrinsically meaningful. This roadmap is intended to highlight some examples of models and algorithms from machine learning which can be interpreted in terms of differential geometry. Most of the content in this roadmap belongs to information geometry, the study of manifolds of probability distributions. The best reference on this topic is probably Amari and Nagaoka's Methods of Information Geometry.