Runge–Kutta methods

485 VIEWS

The Runge–Kutta method

Explicit Runge–Kutta methods

Examples

Second-order methods with two stages

Use

Adaptive Runge–Kutta methods

Nonconfluent Runge–Kutta methods

Runge–Kutta-Nyström methods

Implicit Runge–Kutta methods

Examples

Stability

B-stability

Derivation of the Runge–Kutta fourth-order method

Runge–Kutta methods

In numerical analysis, the Runge–Kutta methods are a family of implicit and explicit iterative methods, which include the well-known routine called the Euler Method, used in temporal discretization for the approximate solutions of ordinary differential equations.^[1] These methods were developed around 1900 by the German mathematicians Carl Runge and Wilhelm Kutta.

The Runge–Kutta method

Slopes used by the classical Runge-Kutta method

The most widely known member of the Runge–Kutta family is generally referred to as "RK4", the "classic Runge–Kutta method" or simply as "the Runge–Kutta method".

Let an initial value problem be specified as follows:

Hereis an unknown function (scalar or vector) of time, which we would like to approximate; we are told that, the rate at whichchanges, is a function ofand ofitself. At the initial timethe correspondingvalue is. The functionand the data,are given.

Now pick a step-size h > 0 and define

for n = 0, 1, 2, 3, ..., using^[2]

(Note: the above equations have different but equivalent definitions in different texts).^[3]

Hereis the RK4 approximation of, and the next value () is determined by the present value () plus theweighted averageof four increments, where each increment is the product of the size of the interval, h, and an estimated slope specified by function f on the right-hand side of the differential equation.

is the increment based on the slope at the beginning of the interval, using (Euler's method);
is the increment based on the slope at the midpoint of the interval, using and ;
is again the increment based on the slope at the midpoint, but now using and ;
is the increment based on the slope at the end of the interval, using and .

In averaging the four increments, greater weight is given to the increments at the midpoint. Ifis independent of, so that the differential equation is equivalent to a simple integral, then RK4 isSimpson's rule.^[4]

The RK4 method is a fourth-order method, meaning that thelocal truncation errorison the order of, while thetotal accumulated erroris on the order of.

In many practical applications the functionis independent of(so calledautonomous system, or time-invariant system, especially in physics), and their increments are not computed at all and not passed to function, with only the final formula forused.

Explicit Runge–Kutta methods

The family of explicit Runge–Kutta methods is a generalization of the RK4 method mentioned above. It is given by

where^[5]

(Note: the above equations may have different but equivalent definitions in some texts).^[3]

To specify a particular method, one needs to provide the integer s (the number of stages), and the coefficients aij (for 1 ≤ j < i ≤ s), bi (for i = 1, 2, ..., s) and ci (for i = 2, 3, ..., s). The matrix [aij] is called the Runge–Kutta matrix, while the bi and ci are known as the weights and the nodes.^[6] These data are usually arranged in a mnemonic device, known as a Butcher tableau (after John C. Butcher):

The Runge–Kutta method is consistent if

There are also accompanying requirements if one requires the method to have a certain order p, meaning that the local truncation error is O(hp+1). These can be derived from the definition of the truncation error itself. For example, a two-stage method has order 2 if b1 + b2 = 1, b2c2 = 1/2, and a21 = c2.^[7]

In general, if an explicit-stage Runge–Kutta method has order, then it can be proven that the number of stages must satisfy, and if, then.^[8] However, it is not known whether these bounds are sharp in all cases; for example, all known methods of order 8 have at least 11 stages, though it is possible that there are methods with fewer stages. (The bound above suggests that there could be a method with 9 stages; but it could also be that the bound is simply not sharp.) Indeed, it is an open problem what the precise minimum number of stagesis for an explicit Runge–Kutta method to have orderin those cases where no methods have yet been discovered that satisfy the bounds above with equality. Some values which are known are:^[9]

The provable bounds above then imply that we can not find methods of ordersthat require fewer stages than the methods we already know for these orders. However, it is conceivable that we might find a method of orderthat has only 8 stages, whereas the only ones known today have at least 9 stages as shown in the table.

Examples

The RK4 method falls in this framework. Its tableau is^[10]

0
1/2	1/2
1/2	0	1/2
1	0	0	1
	1/6	1/3	1/3	1/6

A slight variation of "the" Runge–Kutta method is also due to Kutta in 1901 and is called the 3/8-rule.^[11] The primary advantage this method has is that almost all of the error coefficients are smaller than in the popular method, but it requires slightly more FLOPs (floating-point operations) per time step. Its Butcher tableau is

0
1/3	1/3
2/3	-1/3	1
1	1	−1	1
	1/8	3/8	3/8	1/8

However, the simplest Runge–Kutta method is the (forward)Euler method, given by the formula. This is the only consistent explicit Runge–Kutta method with one stage. The corresponding tableau is

0
	1

Second-order methods with two stages

An example of a second-order method with two stages is provided by the midpoint method:

The corresponding tableau is

0
1/2	1/2
	0	1

The midpoint method is not the only second-order Runge–Kutta method with two stages; there is a family of such methods, parameterized by α and given by the formula^[12]

Its Butcher tableau is

0

In this family,gives the midpoint method, andisHeun's method.^[4]

Use

As an example, consider the two-stage second-order Runge–Kutta method with α = 2/3, also known as Ralston method. It is given by the tableau


0
2/3	2/3
	1/4	3/4

with the corresponding equations

This method is used to solve the initial-value problem

with step size h = 0.025, so the method needs to take four steps.

The method proceeds as follows:


$t_{0}=1\colon$
	$y_{0}=1$
$t_{1}=1.025\colon$
	$y_{0}=1$	$k_{1}=2.557407725$	$k_{2}=f(t_{0}+{\tfrac {2}{3}}h,\ y_{0}+{\tfrac {2}{3}}hk_{1})=2.7138981184$
	$y_{1}=y_{0}+h({\tfrac {1}{4}}k_{1}+{\tfrac {3}{4}}k_{2})={\underline {1.066869388}}$
$t_{2}=1.05\colon$
	$y_{1}=1.066869388$	$k_{1}=2.813524695$	$k_{2}=f(t_{1}+{\tfrac {2}{3}}h,\ y_{1}+{\tfrac {2}{3}}hk_{1})$
	$y_{2}=y_{1}+h({\tfrac {1}{4}}k_{1}+{\tfrac {3}{4}}k_{2})={\underline {1.141332181}}$
$t_{3}=1.075\colon$
	$y_{2}=1.141332181$	$k_{1}=3.183536647$	$k_{2}=f(t_{2}+{\tfrac {2}{3}}h,\ y_{2}+{\tfrac {2}{3}}hk_{1})$
	$y_{3}=y_{2}+h({\tfrac {1}{4}}k_{1}+{\tfrac {3}{4}}k_{2})={\underline {1.227417567}}$
$t_{4}=1.1\colon$
	$y_{3}=1.227417567$	$k_{1}=3.796866512$	$k_{2}=f(t_{3}+{\tfrac {2}{3}}h,\ y_{3}+{\tfrac {2}{3}}hk_{1})$
	$y_{4}=y_{3}+h({\tfrac {1}{4}}k_{1}+{\tfrac {3}{4}}k_{2})={\underline {1.335079087}}.$

The numerical solutions correspond to the underlined values.

Adaptive Runge–Kutta methods

Adaptive methods are designed to produce an estimate of the local truncation error of a single Runge–Kutta step. This is done by having two methods, one with orderand one with order. These methods are interwoven, i.e., they have common intermediate steps. Thanks to this, estimating the error has little or negligible computational cost compared to a step with the higher-order method.

During the integration, the step size is adapted such that the estimated error stays below a user-defined threshold: If the error is too high, a step is repeated with a lower step size; if the error is much smaller, the step size is increased to save time. This results in an (almost) optimal step size, which saves computation time. Moreover, the user does not have to spend time on finding an appropriate step size.

The lower-order step is given by

whereare the same as for the higher-order method. Then the error is

which is. The Butcher tableau for this kind of method is extended to give the values of:


0
$c_{2}$	$a_{21}$
$c_{3}$	$a_{31}$	$a_{32}$
$\vdots$	$\vdots$		$\ddots$
$c_{s}$	$a_{s1}$	$a_{s2}$	$\cdots$	$a_{s,s-1}$
	$b_{1}$	$b_{2}$	$\cdots$	$b_{s-1}$	$b_{s}$
	$b_{1}^{*}$	$b_{2}^{*}$	$\cdots$	$b_{s-1}^{*}$	$b_{s}^{*}$

The Runge–Kutta–Fehlberg method has two methods of orders 5 and 4. Its extended Butcher tableau is:


0
1/4	1/4
3/8	3/32	9/32
12/13	1932/2197	−7200/2197	7296/2197
1	439/216	−8	3680/513	-845/4104
1/2	−8/27	2	−3544/2565	1859/4104	−11/40
	16/135	0	6656/12825	28561/56430	−9/50	2/55
	25/216	0	1408/2565	2197/4104	−1/5	0

However, the simplest adaptive Runge–Kutta method involves combining Heun's method, which is order 2, with the Euler method, which is order 1. Its extended Butcher tableau is:


0
1	1
	1/2	1/2
	1	0

Other adaptive Runge–Kutta methods are the Bogacki–Shampine method (orders 3 and 2), the Cash–Karp method and the Dormand–Prince method (both with orders 5 and 4).

Nonconfluent Runge–Kutta methods

A Runge–Kutta method is said to be nonconfluent ^[13] if all theare distinct.

Runge–Kutta-Nyström methods

Runge-Kutta Nyström methods are specialized Runge-Kutta methods that are optimized for second-order differential equations of the form:^[14]

On the other hand, a general Runge-Kutta Nyström method is optimized for second-order differential equations of the form:^[15]

Implicit Runge–Kutta methods

All Runge–Kutta methods mentioned up to now are explicit methods. Explicit Runge–Kutta methods are generally unsuitable for the solution of stiff equations because their region of absolute stability is small; in particular, it is bounded.^[16] This issue is especially important in the solution of partial differential equations.

The instability of explicit Runge–Kutta methods motivates the development of implicit methods. An implicit Runge–Kutta method has the form

where

^[17]

The difference with an explicit method is that in an explicit method, the sum over j only goes up to i − 1. This also shows up in the Butcher tableau: the coefficient matrixof an explicit method is lower triangular. In an implicit method, the sum over j goes up to s and the coefficient matrix is not triangular, yielding a Butcher tableau of the form^[10]

SeeAdaptive Runge-Kutta methods abovefor the explanation of therow.

The consequence of this difference is that at every step, a system of algebraic equations has to be solved. This increases the computational cost considerably. If a method with s stages is used to solve a differential equation with m components, then the system of algebraic equations has ms components. This can be contrasted with implicit linear multistep methods (the other big family of methods for ODEs): an implicit s-step linear multistep method needs to solve a system of algebraic equations with only m components, so the size of the system does not increase as the number of steps increases.^[18]

Examples

The simplest example of an implicit Runge–Kutta method is the backward Euler method:

The Butcher tableau for this is simply:

This Butcher tableau corresponds to the formulae

which can be re-arranged to get the formula for the backward Euler method listed above.

Another example for an implicit Runge–Kutta method is the trapezoidal rule. Its Butcher tableau is:

The trapezoidal rule is a collocation method (as discussed in that article). All collocation methods are implicit Runge–Kutta methods, but not all implicit Runge–Kutta methods are collocation methods.^[19]

The Gauss–Legendre methods form a family of collocation methods based on Gauss quadrature. A Gauss–Legendre method with s stages has order 2s (thus, methods with arbitrarily high order can be constructed).^[20] The method with two stages (and thus order four) has Butcher tableau:

^[18]

Stability

The advantage of implicit Runge–Kutta methods over explicit ones is their greater stability, especially when applied tostiff equations. Consider the linear test equation y' = λy. A Runge–Kutta method applied to this equation reduces to the iteration, with r given by

^[21]

where e stands for the vector of ones. The function r is called the stability function.^[22] It follows from the formula that r is the quotient of two polynomials of degree s if the method has s stages. Explicit methods have a strictly lower triangular matrix A, which implies that det(I − zA) = 1 and that the stability function is a polynomial.^[23]

The numerical solution to the linear test equation decays to zero if | r(z) | < 1 with z = hλ. The set of such z is called the domain of absolute stability. In particular, the method is said to be absolute stable if all z with Re(z) < 0 are in the domain of absolute stability. The stability function of an explicit Runge–Kutta method is a polynomial, so explicit Runge–Kutta methods can never be A-stable.^[23]

If the method has order p, then the stability function satisfiesas. Thus, it is of interest to study quotients of polynomials of given degrees that approximate the exponential function the best. These are known asPadé approximants. A Padé approximant with numerator of degree m and denominator of degree n is A-stable if and only if m ≤ n ≤ m + 2.^[24]

The Gauss–Legendre method with s stages has order 2s, so its stability function is the Padé approximant with m = n = s. It follows that the method is A-stable.^[25] This shows that A-stable Runge–Kutta can have arbitrarily high order. In contrast, the order of A-stable linear multistep methods cannot exceed two.^[26]

B-stability

The A-stability concept for the solution of differential equations is related to the linear autonomous equation. Dahlquist proposed the investigation of stability of numerical schemes when applied to nonlinear systems that satisfy a monotonicity condition. The corresponding concepts were defined as G-stability for multistep methods (and the related one-leg methods) and B-stability (Butcher, 1975) for Runge–Kutta methods. A Runge–Kutta method applied to the non-linear system, which verifies, is called B-stable, if this condition impliesfor two numerical solutions.

Let,andbe threematrices defined by

A Runge–Kutta method is said to be algebraically stable ^[27] if the matricesandare both non-negative definite. A sufficient condition for B-stability ^[28] is:andare non-negative definite.

Derivation of the Runge–Kutta fourth-order method

In general a Runge–Kutta method of ordercan be written as:

where:

are increments obtained evaluating the derivatives ofat the-th order.

We develop the derivation^[29] for the Runge–Kutta fourth-order method using the general formula withevaluated, as explained above, at the starting point, the midpoint and the end point of any interval; thus, we choose:

andotherwise. We begin by defining the following quantities:

whereand. If we define:

and for the previous relations we can show that the following equalities hold up to:

where:

is the total derivative ofwith respect to time.

If we now express the general formula using what we just derived we obtain:

and comparing this with theTaylor seriesofaround:

we obtain a system of constraints on the coefficients:

which when solved givesas stated above.

References

[1]

Citation Linkopenlibrary.orgDEVRIES, Paul L. ; HASBUN, Javier E. A first course in computational physics. Second edition. Jones and Bartlett Publishers: 2011. p. 215.

Sep 18, 2019, 2:41 PM

[2]

Citation Linkopenlibrary.org, p. 908; Süli, Endre; Mayers, David (2003), An Introduction to Numerical Analysis, Cambridge University Press, ISBN 0-521-00794-1, p. 328

Sep 18, 2019, 2:41 PM

[3]

Citation Linkopenlibrary.orgAtkinson, Kendall A. (1989), An Introduction to Numerical Analysis (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-50023-0, p. 423), Hairer, Ernst; Nørsett, Syvert Paul; Wanner, Gerhard (1993), Solving ordinary differential equations I: Nonstiff problems, Berlin, New York: Springer-Verlag, ISBN 978-3-540-56670-0, p. 134), Kaw, Autar; Kalu, Egwu (2008), Numerical Methods with Applications (1st ed.), autarkaw.com, §8.4) and Stoer, Josef; Bulirsch, Roland (2002), Introduction to Numerical Analysis (3rd ed.), Berlin, New York: Springer-Verlag, ISBN 978-0-387-95452-3, p. 476) leave out the factor h in the definition of the stages. Ascher, Uri M.; Petzold, Linda R. (1998), Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations, Philadelphia: Society for Industrial and Applied Mathematics, ISBN 978-0-89871-412-8, p. 81), Butcher, John C. (2008), Numerical Methods for Ordinary Differential Equations, New York: John Wiley & Sons, ISBN 978-0-470-72335-7, p. 93) and Iserles, Arieh (1996), A First Course in the Numerical Analysis of Differential Equations, Cambridge University Press, ISBN 978-0-521-55655-2, p. 38) use the y values as stages.

Sep 18, 2019, 2:41 PM

[4]

Citation Linkopenlibrary.org, p. 328

Sep 18, 2019, 2:41 PM

[5]

Citation Linkopenlibrary.org, p. 907

Sep 18, 2019, 2:41 PM

[6]

Citation Linkopenlibrary.org, p. 38

Sep 18, 2019, 2:41 PM

[7]

Citation Linkopenlibrary.org, p. 39

Sep 18, 2019, 2:41 PM

[8]

Citation Linkopenlibrary.org, p. 187

Sep 18, 2019, 2:41 PM

[9]

Citation Linkopenlibrary.org, pp. 187–196

Sep 18, 2019, 2:41 PM

[10]

Citation Linkopenlibrary.org, p. 352

Sep 18, 2019, 2:41 PM

[11]

Citation Linkopenlibrary.org, p. 138) refer to Kutta, Martin (1901), Beitrag zur näherungweisen Integration totaler DifferentialgleichungenKutta, Martin (1901), "Beitrag zur näherungsweisen Integration totaler Differentialgleichungen", Zeitschrift für Mathematik und Physik, 46: 435–453.

Sep 18, 2019, 2:41 PM

[12]

Citation Linkopenlibrary.org, p. 327

Sep 18, 2019, 2:41 PM

[13]

Citation Linkopenlibrary.orgLambert, J.D (1991), Numerical Methods for Ordinary Differential Systems. The Initial Value Problem, John Wiley & Sons, ISBN 0-471-92990-5, p. 278

Sep 18, 2019, 2:41 PM

[14]

Citation Link//doi.org/10.1007%2FBF01230162Dormand, J. R.; Prince, P. J. (October 1978). "New Runge-Kutta Algorithms for Numerical Simulation in Dynamical Astronomy". Celestial Mechanics. 18 (3): 223–232. doi:10.1007/BF01230162.

Sep 18, 2019, 2:41 PM

[15]

Citation Linkopenlibrary.orgFehlberg, E. (October 1974). Classical seventh-, sixth-, and fifth-order Runge-Kutta-Nyström formulas with stepsize control for general second-order differential equations (Report) (NASA TR R-432 ed.). Marshall Space Flight Center, AL: National Aeronautics and Space Administration.

Sep 18, 2019, 2:41 PM

[16]

Citation Linkopenlibrary.org, pp. 349–351

Sep 18, 2019, 2:41 PM

[17]

Citation Linkopenlibrary.org, p. 41; , pp. 351–352

Sep 18, 2019, 2:41 PM

[18]

Citation Linkopenlibrary.org, p. 353

Sep 18, 2019, 2:41 PM

[19]

Citation Linkopenlibrary.org, pp. 43–44

Sep 18, 2019, 2:41 PM

[20]

Citation Linkopenlibrary.org, p. 47

Sep 18, 2019, 2:41 PM

Runge–Kutta methods

Runge–Kutta methods

The Runge–Kutta method

Explicit Runge–Kutta methods

Examples

Second-order methods with two stages

Use

Adaptive Runge–Kutta methods

Nonconfluent Runge–Kutta methods

Runge–Kutta-Nyström methods

Implicit Runge–Kutta methods

Examples

Stability

B-stability

Derivation of the Runge–Kutta fourth-order method

See also

References