2.3 Taylor's method

Taylor's formula for series expansion of a function $ f(x) $ around $ x_0 $ is given by $$ \begin{equation} f(x)=f(x_0)+(x-x_0)\cdot f'(x_0)+\frac{(x-x_0)^2}{2}f''(x_0)+\dots +\frac{(x-x_0)^n}{n !}f^{(n)}(x_0)+ \text{higher order terms} \tag{2.13} \end{equation} $$ Let's use this formula to find the first terms in the series expansion for $ \theta(t) $ around $ t=0 $ from the differential equation given in (2.9): $$ \begin{align} &\ddot{\theta} (t) +\theta (t) = 0 \tag{2.14}\\ &\theta (0) = \theta_0,\ \dot{\theta}(0)=0 \tag{2.15} \end{align} $$

First we observe that the solution to the ODE in (2.15) may be expressed as an Taylor expansion around the initial point: $$ \begin{align} \theta(t) \approx \theta(0)+t\cdot\dot{\theta}(0)+\frac{t^2}{2}\ddot\theta(0)+\frac{t^3}{6}\dddot\theta(0)+\frac{t^4}{24}\theta^{(4)}(0) \tag{2.16} \end{align} $$

By use of the initial conditions of the ODE in (2.15) $ \theta(0)=\theta_0,\ \dot\theta(0)=0 $ we get $$ \begin{equation} \theta(t)\approx\theta_0 +\frac{t^2}{2}\ddot\theta+\frac{t^3}{6}\dddot\theta(0)+\frac{t^4}{24}\theta^{(4)}(0) \tag{2.17} \end{equation} $$ From the ODE in (2.15) we obtain expressions for the differentials at the initial point: $$ \begin{equation} \ddot\theta(t)=-\theta(t)\to \ddot\theta(0)=-\theta(0)=-\theta_0 \tag{2.18} \end{equation} $$

Expressions for higher order differentials evaluated at the initial point may be obtained by further differentiation of the ODE (2.15) $$ \begin{equation} \dddot\theta(t)=-\dot\theta(t)\to \ddot\theta(0)=-\theta(0)=-\theta_0 \tag{2.19} \end{equation} $$ and $$ \begin{equation*} \theta^{(4)}(t)=-\ddot\theta(t) \to \theta^{(4)}(0)=-\ddot\theta(0)=\theta_0 \nonumber \end{equation*} $$

Substitution of these differentials into (2.17) yields $$ \begin{equation} \theta(t)\approx \theta_0\left(1-\frac{t^2}{2}+\frac{t^4}{24}\right)=\theta_0\left(1-\frac{t^2}{2!}+\frac{t^4}{4!}\right) \tag{2.20} \end{equation} $$

If we include $ n $ terms, we get $$ \begin{equation*} \theta(t)\approx \theta_0\cdot \left(1-\frac{t^2}{2!}+\frac{t^4}{4!}-\frac{t^6}{6!}+\dots+(-1)^n \frac{t^{2n}}{(2n)!}\right) \nonumber \end{equation*} $$

If we let $ n \to \infty $ we see that the parentheses give the series for $ \cos(t) $. In this case we have found the exact solution $ \theta(t)=\theta_0\cos(t) $ of the differential equation. Since this equation is linear we manage in this case to find a connection between the coefficients such that we recognize the series expansion of $ \cos(t) $.

Taylor's Method for solution of initial value ODE..

Use the ODE (e.g. (2.2) or (2.4)) to evaluate the differentials at the initial value.
Obtain an approximate solution of the ODE by substitution of the differentials in the Taylor expansion (2.13).
Differentiate the ODE as many times needed to obtain the wanted accuracy.

2.3.1 Example: Taylor's method for the non-linear mathematical pendulum

Let's try the same procedure on the non-linear version (2.7) $$ \begin{align} \nonumber \ddot{\theta}(t)& + \sin \left(\theta (t)\right) = 0 \\ \theta (0) &= \theta_0 ,\ \dot\theta (0) = 0 \nonumber \end{align} $$ We start in the same manner: $ \theta(t) \approx \theta(0)+\frac{t^2}{2}\ddot\theta(0)+\frac{t^3}{6}\dddot\theta(0)+\frac{t^4}{24}\theta^{(4)}(0) $. From the differential equation we have $ \ddot\theta=-\sin (\theta) \to \ddot\theta(0)=-\sin(\theta_0) $, which by consecutive differentiation gives $$ \begin{align} &\dddot\theta=-\cos(\theta)\cdot\dot\theta \to \dddot\theta(0)=0 \nonumber \\ &\theta^{(4)}=\sin(\theta)\cdot\dot\theta^2-\cos(\theta)\cdot\ddot\theta \to \theta^{(4)}(0)=-\ddot\theta(0)\cos(\theta(0))=\sin(\theta_0)\cos(\theta_0) \nonumber \end{align} $$ Inserted above: $ \theta(t) \approx \theta_0 -\frac{t^2}{2}\sin(\theta_0) + \frac{t^4}{24} \sin (\theta_0) \cos (\theta_0) $.

We may include more terms, but this complicates the differentiation and it is hard to find any connection between the coefficients. When we have found an approximation for $ \theta(t) $ we can get an approximation for $ \dot\theta(t) $ by differentiation: $ \dot\theta(t) \approx -t\sin(\theta_0) + \frac{t^3}{8} \sin (\theta_0) \cos (\theta_0) $.

Series expansions are often useful around the starting point when we solve initial value problems. The technique may also be used on non-linear equations.

Symbolic mathematical programs like Maple and Mathematica do this easily.

2.3.2 Example: Newton's first differential equation

We will end with one of the earliest known differential equations, which Newton solved with series expansion in 1671. $$ \begin{equation*} y'(x) =1-3x+y+x^2+xy,\ y(0)=0 \end{equation*} $$ Series expansion around $ x=0 $ gives $$ \begin{equation*} y(x) \approx x\cdot y'(0)+ \frac{x^2}{2} y''(0)+\frac{x^3}{6}y'''(0)+\frac{x^4}{24}y^{(4)}(0) \end{equation*} $$ From the differential equation we get $ y'(0)=1 $. By consecutive differentiation we get $$ \begin{equation*} \begin{array}{lclclcr} y''(x) &=&-3+y'+2x+xy'+y &\to& y''(0)&=&-2\\ y'''(x)&=& y''+2+xy''+2y' &\to& y'''(0)&=& 2\\ y^{(4)}(x)&=&y'''+xy'''+3y'' &\to &y^{(4)}(0)&=&-4 \end{array} \end{equation*} $$ Inserting above gives $ y(x) \approx x-x^2+\frac{x^3}{3}-\frac{x^4}{6} $.

Newton gave the following solution: $ y(x) \approx x-x^2+\frac{x^3}{3}-\frac{x^4}{6}+ \frac{x^5}{30}-\frac{x^6}{45} $.

Now you can check if Newton calculated correctly. Today it is possible to give the solution on closed form with known functions as follows, $$ \begin{align*} y(x)=&3\sqrt{2\pi e}\cdot \exp\left[x\left(1+\frac{x}{2}\right)\right]\cdot \left[\text{erf}\left(\frac{\sqrt{2}}{2}(1+x)\right)-\text{erf}\left(\frac{\sqrt{2}}{2}\right)\right]\\ +& 4\cdot\left[1-\exp[x\left(1+\frac{x}{2}\right)\right]-x \end{align*} $$

Note the combination $ \sqrt{2\pi e} $. See Hairer et al. [1] section 1.2 for more details on classical differential equations.