Differentiate everything: a lesson from deep learning

Deep learning taught us a new way to play with computers: compose differentiable components into a computer program, then tune its parameters via gradient optimization until it achieves what we want. This is the key idea of differentiable programming. The rapid development of deep learning technology offers convenient tools for differentiable programming, and also opens a new frontier for computational physics. This article introduces the basic notion of differentiable programming and its physics applications including modeling, optimization, control, and inverse design.