Approximation and Errors

While solving a numerical problem, errors may occur due to several factors. Sometimes, there might be error while formulating the problem itself. The assumptions made for problems might be false or there might be errors while taking the data/measurement from the real world. In such cases, there will always be errors no matter how accurate the numerical method is. There could also be errors while implementing the algorithm/program for numerical methods. These kinds of errors can be eliminated with careful analysis and accurate measurements.

However, there are some kinds of errors that can't be completely eliminated. One example of such error is rounding error which occurs when we use a finite number of digits to represent a number.

Thus, we should expect there will always be an error in the solution of numerical problems. In other words, we can say the solution is an approximate of the exact solution.

Definition of Error

If $X_A$ is the approximate solution to the exact solution $X_E$, then the difference $X_E-X_A$ is called the error.

$$\text{Error}(E_i)=X_E-X_A$$

Types of Errors

1. Input Errors

Errors that occur due to invalid data/constants in the problem are called input errors. These errors occur due to invalid measurements or the use of incorrect tables for constants like $\pi, e$ etc.

2. Iteration Errors

Iteration count is especially important when dealing with indirect or iterative methods. Fewer iterations cause higher errors. The iteration count should be optimized according to how much error can be tolerated in the solution. Some problems might require faster computation while being able to cope with higher errors, in such cases it is ok to have fewer iterations. But for some problems accuracy can be crucial in such cases we will require more iterations.

3. Round off Errors

Numbers in computers are represented with a finite number of digits. Numbers such as $1/3, \pi, \sqrt{2}$ etc require an infinite number of digits to be represented exactly. Computers represent those digits approximately with finite memory. For example, if we represent 1/3 as 0.3333, the round-off error will be 1/3-0.3333=0.0000333... After just a few calculations we are likely to encounter an irrational number even if we start with rational numbers, in such case round-off becomes a necessity.

For example, if we use 2 digits with a moving decimal then we can represent numbers such as 0.00, 0.01, 0.02, ..., 0.99, 1.0, 1.1,1.2, ...,9.9 and 10. We can see that numbers such as 0.111 can't be represented exactly instead we should use 0.11 as an approximation for 0.111.

Similarly, if we perform 10-0.01 then the result won't be 0.999 as it can't be represented at all, instead, the result will be 10 obtained by rounding off 0.999.

If we consider multiplication then things get worse, multiplying 0.01 with 0.02 is 0.0002 which can't be represented and hence will be equal to 0 after rounding off. Similarly multiplying 10 with 5.0 is 50 which will result in an overflow.

To avoid round-off errors we can take the following approach:

1. If a small number is to be added to a large number iteratively then accumulate that number until it becomes comparable to the larger number and then add it.
2. Perform normalization so that all the numbers are of nearly equal magnitude.

4. Truncation Errors

Truncation errors occur when we make an infinite procedure finite. For example the series for $\sin$ function is:

$$\sin x = x-\frac{x^3}{3!}+\frac{x^5}{5!}-...$$

It is not reasonable to calculate the infinite series to find out the value of $\sin$ at a particular $x$. Instead, we might take, say, only the first 3 terms. In such case, the truncation error will be:

$$E_t=-\frac{x^7}{7!}+\frac{x^9}{9!}-...$$

5. Modelling Errors

Modeling errors occur due to incorrect assumptions about the real world or an incomplete mathematical model. For example, if we are calculating how long it takes for a rock to fall from a cliff, we might assume a constant acceleration but in real life the acceleration becomes zero after the rock attains a terminal velocity.