The following example illustrates the computation of MAD, MSE, and MAPE.
Example 1
Sales data of a microwave oven manufacturer and calculation of relevant errors are given in Table 24.1.
Table 24.1: Calculation of Errors
Using the figures,
| MAD | = | Σ |e| / n = 22/8 = 2.75 |
| MSE | = | Σ e2 /(n - 1) = 76/7 = 10.86 |
| MSE | = | Σ |e|/A / n = .0524/8 = .0066 |
One way these measures are used is to evaluate forecasting ability of alternative forecasting methods. For example, using either MAD or MSE, a forecaster could compare the results of exponential smoothing with
Figure 24.1: Monitoring forecast errors
How is it used and applied?
It is important to monitor forecast errors to insure that the forecast is performing well. If the model is performing poorly based on some criteria, the forecaster might reconsider the use of the existing model or switch to another forecasting model or technique. Forecasting control can be accomplished by comparing forecasting errors to predetermined values, or limits. Errors that fall within the limits would be judged acceptable while errors outside of the limits would signal that corrective action is desirable (See Figure 24.1).
Monitoring forecasts
Forecasts can be monitored using either tracking signals or control charts.
Tracking Signals
A tracking signal is based on the ratio of cumulative forecast error to the corresponding value of MAD.
Tracking signal = Σ(A - F) / MAD
The resulting tracking signal values are compared to predetermined limits. These are based on experience and judgment and often range from plus or minus 3 to plus or minus 8. Values within the limits suggest that the forecast is performing adequately. By the same token, when the signal goes beyond this range, corrective action is appropriate.
Example 2
Returning Example 1, the deviation and cumulative deviation have already been computed:
MAD= Σ |A - F| / n = 22 / 8 = 2.75
Tracking signal = Σ (A - F) / MAD = -2 / 2.75 = -0.73
A tracking signal is as low as - 0.73, which is substantially below the limit (-3 to -8). It would not suggest any action at this time.
Note: After an initial value of MAD has been computed, the estimate of the MAD can be continually updated using exponential smoothing.
MADt = α(A - F) + (1 - α) MADt-1
Control Charts
The control chart approach involves setting upper and lower limits for individual forecasting errors instead of cumulative errors. The limits are multiples of the estimated standard deviation of forecast, Sf, which is the square root of MSE. Frequently, control limits are set at 2 or 3 standard deviations.
± 2(or 3) Sf
Note: Plot the errors and see if all errors are within the limits, so that the forecaster can visualize the process and determine if the method being used is in control.
Example 3
For the sales data in Table 24.2, using the naive forecast, we will determine if the forecast is in control. For illustrative purposes, we will use 2 sigma control limits.
Table 24.2: Error Calculations
First, compute the standard deviation of forecast errors
Two sigma limits are then plus or minus 2(7.64) = -15.28 to +15.28
Note that the forecast error for year 3 is below the lower bound, so the forecast is not in control (See Figure 24.2). The use of other methods such as moving average, exponential smoothing, or regression might produce a better forecast.
Note: A system of monitoring forecasts needs to be developed. The computer may be programmed to print a report showing the past history when the tracking signal “trips” a limit. For example, when a type of exponential smoothing is used, the system may try a different value of α (so the forecast will be more responsive) and to continue forecasting.
Figure 24.2: Control charts for forecasting errors
25. Cost of Prediction Errors
Introduction
There is always a cost involved in a failure to predict a certain variable accurately. The cost of prediction errors associated with sales, expenses, and purchases can be significant.
How is it computed?
The cost of the prediction error is basically the contribution or profit lost because of an inaccurate prediction. It can be measured in terms of lost sales, disgruntled customers, or idle machines.
Example
Assume that a retail store has been selling a toy doll having a cost of $0.60 for $1.00 each. The fixed cost is $300. The store has no privilege to return any unsold dolls. It has predicted sales of 2,000 dolls. However, unforeseen competition has reduced sales to 1,500 dolls. The cost of its prediction error—that is, its failure to predict demand accurately, is calculated as follows:
1.Initial predicted sales = 2,000 dollars
Optimal decision: purchase 2,000 dollars
Expected net income = $500 [(2,000 dollars × $0.40 contribution) − $300 fixed cost]
2.Alternative parameter value = 1,500 dolls
Optimal decision: purchase 1500 dollars
Expected net income = $300 [(1,500 dollars × $0.40 contribution) − $300 fixed cost]
3.Results of original decision under alternative parameter value Expected net income:
Revenue (1,500 dolls × $1.00) – cost of dollars (2,000 dollars × $0.60) − $300 fixed cost = $1,500 – $1,200 – $300 = $0
4.Cost of prediction error = (2) – (3) = $300
How is it used and applied?
It is important to determine the cost of the prediction error in order to minimize the potential detrimental effect of future profitability. The cost of the prediction error can be substantial, depending on the circumstances. For example, failure to make an accurate projection of sales could result in poor production planning, too