Tools › Numerical Methods › Equation Solvers › ODE and Nonlinear Equation Solver


ODE and Nonlinear Equation Solver

Type differential equations with their starting values, one equation f(x) = 0, or a system of equations, and get the numerical solution with the work shown: time histories and the phase plane for Euler, Heun, classical Runge-Kutta (RK4) or adaptive Dormand-Prince RK45 with every k of the first step written out, or the iteration table and convergence chart of bisection, false position, Newton and the secant method, or Newton's method for a system with its Jacobian. The typed quantities are plain numbers in whatever consistent units you choose.

  • Initial-value problems of up to 8 first-order equations: Euler, Heun, RK4 with a step-halving error estimate, or adaptive RK45 with a tolerance check; warnings for an unconverged or stiff problem

  • One equation by bisection, false position, Newton or secant, and systems of up to 6 equations by Newton with a finite-difference Jacobian: every iteration tabulated, the first one worked in full

  • Charts of the solution against time, the phase plane, the function with its iterates and the error against iteration; all numbers as copyable text

  • Checked against the SciPy initial-value, root and system solvers, closed-form solutions and the measured order of accuracy of each method

ODE and Nonlinear Equation Solver

Problem


Differential Equations: x' = ... on each line, values as name = number, time is t








One Equation: the unknown is the one name without a value





System of Equations: as many equations as names without a value



Stopping


×

ODE and Nonlinear Equation Solver

How to Type the Equations, the Four Integrators, Root Finding, What It Assumes, Common Mistakes

What the tool does

It solves three kinds of problem that have no answer in formulas: differential equations that start from known values (an initial-value problem), one equation in one unknown, and several equations in several unknowns. In each case it shows the work a numerical-methods course asks for: every slope of the first step, the first iteration with the numbers put in, the table of iterations, and an estimate of how far the answer can be trusted.

The idea

A differential equation y′ = f(t, y) gives the slope of the solution at any point but not the solution itself. A numerical method walks along it: from a known point it uses slopes to predict the value a short step h later, then repeats from there. The methods differ only in how many slopes they sample inside a step and how they average them (Figure 1). Euler uses the slope at the start; Heun averages the slope at the start with the slope at the predicted end; the classical Runge-Kutta method samples four. More slopes per step cost more arithmetic but cut the error much faster as the step shrinks, which is why RK4 is the default workhorse of hand and spreadsheet work.

One step of size 1 on the equation y prime equals y from y equals 1: Euler reaches 2.0, Heun 2.5, classical Runge-Kutta 2.7083, against the exact value 2.7183
Figure 1: One deliberately large step of three methods on y′ = y, against the exact solution

Root finding is the same kind of walk without time: start from a guess, use the function (and perhaps its slope) to make a better guess, repeat until the guesses stop moving.

How to type a problem

Differential equations. One first-order equation per line, written x' = expression (dx/dt = expression also works). The independent variable is always t. Every other name needs a value on its own line, k = 2.5; a value may be a formula, and a formula that uses the states (N = S + I + R) is worked out again at every step. Starting values go in their own box as x = 1, v = 0.

A higher-order equation becomes several first-order ones. Give each derivative below the highest its own name. The mass-spring-damper m x″ + c x′ + k x = 0 has the second derivative x″ = −(c x′ + k x)/m; call the velocity v and type

x' = v
v' = -(c*v + k*x)/m

A third-order equation needs three lines (x′ = v, v′ = a, a′ = …), and so on. Two coupled second-order equations become four lines.

One equation. Type it with an equals sign, both sides allowed (cos(x) = x), or as an expression that should be zero. Give every name but one a value; the name left over is the unknown. A system is the same with several lines: as many equations as names without a value.

Syntax. Write every multiplication sign (2*x, not 2x). Powers are ^; −x^2 means −(x²). Functions: sqrt, cbrt, exp, ln, log and log10 (both base 10), log2, sin, cos, tan, asin, acos, atan, atan2(y, x), sinh, cosh, tanh, abs, sign, min, max. Constants pi and e. Angles are in radians. Text after # is a comment, and a semicolon separates two statements on one line. The text is parsed into arithmetic, never executed: a name the tool does not know is refused in words.

Units. The typed quantities are plain numbers. A general typed equation cannot be checked for units, so the tool does not pretend to: use one consistent set (for example meters, kilograms and seconds throughout) and the answer is in that set. To convert values first, use the Unit Calculator Pad.

The equations: stepping methods

With y the list of states, f the list of right-hand sides and h the step:

Euler (first order)

k1 = f(tn, yn),   yn+1 = yn + h k1

Heun, or improved Euler (second order)

k1 = f(tn, yn),   k2 = f(tn + h, yn + h k1),   yn+1 = yn + (h/2)(k1 + k2)

Classical Runge-Kutta, RK4 (fourth order)

k1 = f(tn, yn)
k2 = f(tn + h/2, yn + (h/2) k1)
k3 = f(tn + h/2, yn + (h/2) k2)
k4 = f(tn + h, yn + h k3)
yn+1 = yn + (h/6)(k1 + 2k2 + 2k3 + k4)

Order p means the error at a fixed end time is proportional to hp: halve the step and the error of Euler halves, that of Heun falls 4 times and that of RK4 16 times.

Step-halving error estimate. The tool runs a fixed-step method twice, with N steps and with 2N. If yN and y2N are the two end values, the error of the first is about

E ≈ |yN − y2N| × 2p / (2p − 1)

because the second run has an error 2p times smaller. It is an estimate, valid once the step is small enough for the method to show its order; when it is more than 1 percent of the largest value a state reaches, the tool says that the answer has not converged.

Dormand-Prince RK45 (adaptive step). Seven slopes per step, ki = f(tn + ci h, yn + h Σ aij kj), with the coefficients

iciai1ai2ai3ai4ai5ai6
10
21/51/5
33/103/409/40
44/544/45−56/1532/9
58/919372/6561−25360/218764448/6561−212/729
619017/3168−355/3346732/524749/176−5103/18656
7135/3840500/1113125/192−2187/678411/84

The seventh row is also the set of weights of the fifth-order result, yn+1 = yn + h (35/384 k1 + 500/1113 k3 + 125/192 k4 − 2187/6784 k5 + 11/84 k6), so k7 of one step is k1 of the next. The same slopes with other weights give a fourth-order result; the difference of the two, h (71/57600 k1 − 71/16695 k3 + 71/1920 k4 − 17253/339200 k5 + 22/525 k6 − 1/40 k7), estimates the error of the step. Each component is divided by tol × (1 + the larger size of that state at the two ends of the step); the root mean square of these is the error norm. A norm below 1 accepts the step. Either way the next step is the old one times 0.9 × norm−1/5, kept between 0.2 and 10 (and not allowed to grow right after a rejection). The first step is chosen from the size of the starting values and slopes. This is the step control of the RK45 solver in SciPy, and the tool takes the same steps as that solver.

Tolerance check. The tolerance limits the error of each step, not of the end value. The tool repeats the run at a tenth of the tolerance and reports how far the end values move.

Stiffness. A problem is stiff when it contains a rate much faster than the one you are watching: a term that dies away in a millisecond inside a solution that takes minutes. An explicit method (all four here are explicit) must keep h × (fastest rate) below about 2 for Euler and Heun, 2.8 for RK4 and 3.3 for Dormand-Prince, or the numbers blow up, however smooth the true solution. The fixed-step methods warn when the step is beyond that limit; the adaptive method warns when its step is being held at the limit by stability on many steps in a row (the test of Hairer's DOPRI5 code). The cure is an implicit method, which this tool does not have.

The equations: one unknown

Write the equation as F(x) = 0 (the tool moves the right side to the left).

Bisection. Start with a and b where F has opposite signs. Take the midpoint c = (a + b)/2 and keep the half whose ends still have opposite signs. The bracket halves each time, so after n iterations the root is known to within (b − a)/2n: it cannot fail, and it gains one binary digit per iteration.

False position. The same bracket, but c is where the straight line between the two ends crosses zero: c = b − F(b)(b − a) / (F(b) − F(a)). Usually faster than bisection; one end often never moves, and then it crawls.

Newton. Follow the tangent to zero: xn+1 = xn − F(xn) / F′(xn). The tool finds the slope numerically, F′(x) ≈ [F(x + δ) − F(x − δ)] / (2δ) with δ = 10−6 × max(1, |x|), so no derivative has to be typed. Near a simple root the error is squared at each iteration (second order).

Secant. Newton with the tangent replaced by the line through the last two points: xn+1 = xn − F(xn)(xn − xn−1) / (F(xn) − F(xn−1)). Order about 1.62, one function evaluation per iteration.

All four stop when the last change in x (for bisection, half the bracket) is below tol × max(1, |x|), or after 100 iterations.

The equations: a system

For n equations Fi(x1, …, xn) = 0, Newton's method solves at each iteration the linear system

J Δ = −F,   then xnew = x + Δ

where J is the Jacobian matrix, Jij = ∂Fi/∂xj, found here by central differences with the same δ as above. The linear system is solved by Gaussian elimination with partial pivoting (for two equations the page shows Cramer's rule, which gives the same numbers). The iteration stops when the largest component of Δ is below tol × max(1, largest |x|), or after 50 iterations. The steps are plain Newton steps with no damping, so the table matches a hand calculation.

What it assumes

The right-hand sides are ordinary smooth functions of the states and of time: no delays, no events (a ball that bounces, a switch that trips), no algebraic constraints, no partial derivatives. The integration runs forward from the start time. A root finder assumes the function is continuous near the root, and Newton and secant assume a starting value close enough for the tangent to point the right way. Numbers are double precision (about 16 digits); the central-difference slopes are good to about 9 digits, enough that Newton's convergence is not visibly slowed.

When it misleads

Chaotic systems. For the Lorenz equations and their kind, two solutions that start a hair apart separate by a factor of ten every few time units. Every numerical solution is such a neighbor of the true one, so beyond some time the computed values are no longer those of the true solution, at any tolerance; the shape of the picture stays right, the numbers at a given late time do not. The tolerance check shows when this has happened.

Stiff systems. The methods here either blow up or take a very large number of tiny steps. The warnings say so; the answer needs an implicit solver.

Solutions that end. y′ = y² from y(0) = 1 reaches infinity at t = 1. No method can step across that; the tool stops there and says so.

Several roots, or none. A root finder returns one root, the one its starting values lead to, and gives no sign that others exist. A double root (the curve touches zero without crossing) has no sign change for bisection and slows Newton to first order. Where there is no real root, Newton and secant wander until the iteration limit.

A small residual is not a small error. Where the function is nearly flat, F can be tiny far from the root; where it is steep, the reverse. The tool stops on the change in x and reports F beside it.

An error estimate is not a bound. Step halving and the tolerance check both assume the error behaves smoothly. They are reliable when small and getting smaller as the step or tolerance is tightened; try it.

Limits of this page: 8 states, 6 unknowns, 20,000 steps, 100 iterations (50 for a system) and about six seconds of computing; when one is reached the page says which.

Common mistakes

  • Using the wrong point for a slope. In RK4, k3 is evaluated at yn + (h/2) k2, not at the point used for k2; k4 uses the full step h with k3. For a system, all the states must be advanced to the intermediate point before any k2 is computed: update x and v together, never x first and then v with the new x.
  • Forgetting the weights. The RK4 average is (k1 + 2k2 + 2k3 + k4)/6, not the plain mean of four slopes.
  • Forgetting that time moves too. When f depends on t, k2 and k3 use tn + h/2 and k4 uses tn + h.
  • Typing a second-order equation directly. x″ is not a state. Introduce v = x′ and write two lines; the starting values are then x and v.
  • Degrees in sin and cos. The functions take radians: 30 degrees is 30*pi/180. A pendulum released at "170" is released at 170 radians.
  • log for the natural logarithm. Here log is base 10 (as on a calculator and in the Colebrook equation) and ln is the natural logarithm. Many programming languages use log for the natural one.
  • Trusting one run. A solution that looks smooth can still be wrong. Halve the step (the tool does) and see whether the answer holds.
  • A bracket with no sign change. Bisection needs F(a) and F(b) of opposite sign. Tabulate or plot F first; an even number of roots between a and b also gives equal signs.
  • Stopping bisection on F instead of on the bracket. The bracket width is the guaranteed error; |F| is not.
  • Newton from a flat spot. Where the slope is near zero the tangent shoots far away. Start nearer the root, or take a few bisection steps first.
  • A sign slip in the Newton step for a system. The right-hand side is −F, and the Jacobian is evaluated at the current point, not the starting one.
  • Mixed units. g = 32.2 with a length in meters gives a confident wrong answer. The tool cannot see it.

Related tools

Linear systems with a transfer function are answered exactly, without stepping, by the Transfer Function Analyzer. The flight of a ball with drag and spin, an initial-value problem with its physics built in and its units handled, is in Ball Flight with Drag and Spin.




References

Dormand, J. R., and Prince, P. J. (1980). "A Family of Embedded Runge-Kutta Formulae." Journal of Computational and Applied Mathematics, Vol. 6, No. 1, pp. 19–26.

Hairer, E., Nørsett, S. P., and Wanner, G. (1993). Solving Ordinary Differential Equations I: Nonstiff Problems, 2nd ed. Springer, Berlin.

Shampine, L. F., and Reichelt, M. W. (1997). "The MATLAB ODE Suite." SIAM Journal on Scientific Computing, Vol. 18, No. 1, pp. 1–22.

Virtanen, P., Gommers, R., Oliphant, T. E., et al. (2020). "SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python." Nature Methods, Vol. 17, pp. 261–272.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P. (2007). Numerical Recipes: The Art of Scientific Computing, 3rd ed. Cambridge University Press, Cambridge.

Burden, R. L., and Faires, J. D. (2011). Numerical Analysis, 9th ed. Brooks/Cole, Boston, MA.

Lorenz, E. N. (1963). "Deterministic Nonperiodic Flow." Journal of the Atmospheric Sciences, Vol. 20, No. 2, pp. 130–141.

Colebrook, C. F. (1939). "Turbulent Flow in Pipes, with Particular Reference to the Transition Region between the Smooth and Rough Pipe Laws." Journal of the Institution of Civil Engineers, Vol. 11, No. 4, pp. 133–156.

Kermack, W. O., and McKendrick, A. G. (1927). "A Contribution to the Mathematical Theory of Epidemics." Proceedings of the Royal Society of London, Series A, Vol. 115, No. 772, pp. 700–721.


DISCLAIMER: The methods and calculation on aNOWysis.com are provided as means to educate the reader. There is no warranty, expessed or implied, as to the relevance or accuracy of the methods or calculations. All calculations should be checked by a qualified analyst for appropriateness and accuracy. We will not be held liable for any damages, injuries or deaths caused by using methods or calculations from this site.