Description
Numerical estimates of the derivative of a function are typically done using an approximation called "Finite Difference." However, the accuracy of this method is computationally bounded. Using Complex Analysis, we're able to take the step in the imaginary direction on the complex plane and achieve near perfect estimation of the derivative in only three lines of code!
Numerical estimates of the derivative of a function are typically done using an approximation called "Finite Difference." In a nutshell, the derivative is a function that tells you the slope of a function at a specific point. The Finite Difference approximation takes the rise over run formula (which works exactly for linear equations), and applies it to a non-linear with a very small step size. The derivative is defined as the limit of this process, as the step size goes to zero. Computationally, however, this problem is ill-formed: floating point numbers have gaps between them, and eventually our step size becomes smaller than those gaps. This puts a hard limit on the potential accuracy of finite difference approach. Fear not! Using some fancy mathematics (the Cauchy-Riemann equations, the crown jewel of Complex Analysis) we're able to take the step in the imaginary direction on the complex plane, reducing the problem to one evaluation of the target function and a division. This results in a near perfect estimation of the derivative in only three lines of code (with a few interesting caveats)!
Timeline: (5 min) The Derivative and Finite Differences (5 min) Scipy Implementation, with benchmarks (8 min) The Cauchy Riemann Equations Overview (5 min) Use the Cauchy Riemann Equations with Finite Difference (2 min) Implementing the Complex Step Derivative in Three Lines of Code! (5 min) Examples with benchmarks (5 min) Caveats and Fixes (5 min) Q&A