Practical considerations using floating point arithmetic Numerical differentiation

example showing difficulty of choosing



h


{\displaystyle h}

due both rounding error , formula error

an important consideration in practice when function calculated using floating point arithmetic how small value of h choose. if chosen small, subtraction yield large rounding error. in fact finite difference formulae ill-conditioned , due cancellation produce value of 0 if h small enough. if large, calculation of slope of secant line more accurately calculated, estimate of slope of tangent using secant worse.

a choice h small without producing large rounding error





ε


x


{\displaystyle {\sqrt {\varepsilon }}x}

(though not when x = 0!) machine epsilon ε typically of order 2.2×10. formula h balances rounding error against secant error optimum accuracy is

h
=
2


ε

|



f
(
x
)



f
″

(
x
)



|





{\displaystyle h=2{\sqrt {\varepsilon \left|{f(x) \over f (x)}\right|}}}

(though not when f (x) = 0) , employ require knowledge of function.

this epsilon double precision (64-bit) variables: such calculations in single precision useful. resulting value unlikely round number in binary, important realise although x machine-representable number, x + h not be. means x + h changed (via rounding or truncation) nearby machine-representable number, consequence (x + h) - x not equal h; 2 function evaluations not h apart. in regard, since decimal fractions recurring sequences in binary (just 1/3 in decimal) seemingly round step such h = 0.1 not round number in binary; 0.000110011001100... possible approach follows:

h:=sqrt(eps)*x;
xph:=x + h;
dx:=xph - x;
slope:=(f(xph) - f(x))/dx;

however, computers, compiler optimization facilities may fail attend details of actual computer arithmetic, , instead apply axioms of mathematics deduce dx , h same. c , similar languages, directive xph volatile variable prevent this.