SIAM Journal on Applied Mathematics· 2026Q1
Correcting Autodifferentiation in Neural ODE Training
- 1citations
- Q1SCImago
- 2026year
Short summary
Brute-force autodifferentiation in Neural ODEs using high-order methods (like leapfrog or ERK) can introduce gradient oscillations that hinder convergence; simple postprocessing techniques can correct these gradients.
AI-generated from the title and abstract; the full text is not read.
Key points
- Autodifferentiation in Neural ODEs with high-order methods can produce spurious gradient oscillations.
- These oscillations prevent convergence during model training.
- Simple postprocessing techniques are proposed to correct gradients for leapfrog and 2-stage ERK methods.
- Corrected gradients lead to accurate updates and enable convergence.
AI-generated from the title and abstract; the full text is not read.
Abstract
Abstract. Does the use of autodifferentiation yield reasonable updates for deep neural networks (DNNs)? Specifically, when DNNs are designed to adhere to neural ODE architectures, can we trust the gradients provided by autodifferentiation? Through mathematical analysis and numerical evidence, we demonstrate that when neural networks employ high-order methods, such as linear multistep methods or explicit Runge–Kutta Methods (ERK), to approximate the underlying ODE flows, brute-force autodifferentiation often introduces artificial oscillations in the gradients that prevent convergence. In the case of leapfrog and 2-stage ERK, we propose simple postprocessing techniques that effectively eliminate these oscillations, correct the gradient computation, and thus return the accurate updates.
The authors' abstract, as published at the source. SIAM Journal on Applied Mathematics, 2026 · DOI ↗
The rest is in the Pofolia app
Takeaways and questions to the paper; new summaries every day for your field. Free.
Sign in on the web to openField: Statistical and Nonlinear Physics
Statistical and Nonlinear PhysicsPhysics and Astronomy