Phys 498CQM Lecture 2

Outline

• Root Finding
  • Bisection Method
  • Newton's Method
  • Secant Method
  • Hybrid Method

Root Finding

There are two kinds of roots to consider:

• Odd roots - sign changes across x₀. These are the roots that will be considered in this section.
• Even roots - no sign change. Numerically, these pose problems. If you add some numerical uncertainty to the function (f = f + constant), there root can split into two roots or disappear (become complex). Thus, if you are looking for an even root, it is best to solve for a local extremum instead.

Bisection Method

1. x_n+1 = ( a + b ) / 2
2. If f( x_n+1 × f(a) > 1 then let a = x_n+1
3. If f( x_n+1 × f(b) > 1 then let a = x_n+1 Thus the new [ a, b ] is a smaller interval that brackets the root.

   Comments:

   • Each step halves the interval size.
   • Convergence is guaranteed (you can't lose the root).
   • This method has "linear" convergence, that is, the log of the error decreases linearly with the number of iterations.

   When do you stop? This choice can can some difficulties:

   • Absolute error: | a - b | < delta. This will not work if x₀ is very large, since round off error in | a - b | may be larger than delta.
   • Relative error: | a - b | < delta × | a |. This runs into problems near x = 0.
   • Ill-conditioned root. If the slope of f(x) is very small near the root, then there may be a range of x values for which f(x) is nearly zero. Machine round-off error can prohibit determination of the correct value of x₀ in this case.

   Newton's Method

   Make a linear approximation to the function each iteration to get a better guess at the root.

   Start with an x₁ near the root. Iterate using the formula:

   x_n+1 = x_n - f ( x_n ) / f' ( x_n ) (This formula can be easily derived from a taylor expansion about x_n.)

   Example - Caluclate the Square Root of 5

   • f(x) = x² - 5
   • f'(x) = 2 x
   • so x_k+1 = x_k - ( x² - 5 ) / 2 x

   k	xk
   1	2
   2	2.25
   3	2.2361111
   4	2.2360798
   infinity	2.2360798

4. A nice description of the Newton method with an animated graphic is given here .

Comments:

• Much faster convergence than bisection (superlinear).
• But - this method can head to other roots, go to infinity, or get trapped in cycles. NOT GUARENTEED TO CONVERGE.

Secant Method

This is an example of a quasi-Newton method. If you don't know f'(x), you can approximate using a numerical derivative.

The obvious choice (but not a good choice) is

f'(x) - [ f(x+h) - f(x) ] / h There are a couple of problems with this:
• Extra evaluations of f are neccesary.
• The proper choice for h is not obvious.

The secant method uses the previous point in the sequence for the approximation of the derivative.

Start by choosing [ x₁, x₂ ] that is near the root. (This interval does not have to bracket the root.)

Iterate according to

x_n+1 = x_n - f(x_n) × ( x_n - x_n-1 ) / [ f(x_n) - f(x_n-1) ]

Comments:

• Much faster convergence than bisection (superlinear), but can be slower than Newton's method.
• You don't need to calculate f'.
• Just like Newton's method, this method can head to other roots, go to infinity, or get trapped in cycles. NOT GUARENTEED TO CONVERGE.

Hybrid Method

Combine speed of secant method with safe convergence of bisection.
1. Start with bracketing interval.
2. Perform one secant step to get x_k.
3. If the secant step goes outside of the bracketing interval, do bisection instead.
4. Refine the bracketing interval using the new x_k.
5. Repeat loop until converged.

The 'standard' (Brent's method) for calculating roots of a nonlinear equation without using derivatives is a hybrid method, combining root bracketing, bisection, and inverse quaderature interpolation (instead of the linear interpolation used in our example). For an example, see the function zeroin.f from netlib.

---

Last Modified Jan. 1
Email question/comments/corrections to rmartin@uiuc.edu