(3.37)
which yields
(3.38)
Here, approximately 3M multiplications are required at each iteration of this update, which is the product of the orders of the two filters. This computational inefficiency corresponds to the original approach of unfolding a network in time to derive the gradient. However, we observe that at each iteration this weight update is repeated. Explicitly writing out the product terms for several iterations, we get
| Iteration | Calculation | |||||||
|---|---|---|---|---|---|---|---|---|
| k | e(k) | [ |
|
+ | b1x(k − 1) | + | b2x(k − 2) | ] |
| k + 1 | e(k + 1) | [ | box(k + 1) | + |
|
+ | b2x(k − 1) | ] |
| k + 2 | e(k + 2) | [ | box(k + 2) | + | b1x(k − 1) | + |
|
] |
| k + 3 | e(k + 3) | [ | box(k + 3) | + | b1x(k − 2) | + | b2x(k + 1) | ] |
Therefore, rather than grouping along the horizontal in the above equations, we may group along the diagonal (boxed terms). Gathering these terms, we get
(3.39)
where δ(k) is simply the error filtered backward through the second cascaded filter as illustrated in Figure 3.8. The alternative weight update is thus given by
Equation (3.40) represents temporal backpropagation. Each update now requires only M + 3 multiplications, the sum of the two filter orders. So, we can see that a simple reordering of terms results into a more efficient algorithm. This is the major advantage of the temporal backpropagation algorithm.
3.3 Time Series Prediction
In general, here we deal with the problem of predicting future samples of a time series using a number of samples from the past [6, 7]. Given M samples of the series, autoregression (AR) is fit to the data as
(3.41)
The model assumes that y(k) is obtained by summing up the past values of the sequence plus a modeling error e(k). This error represents the difference between the actual series y(k) and the single‐step prediction
(3.42)
In nonlinear prediction, the model is based on the following nonlinear AR:
(3.43)