Nonlinear Filters. Simon Haykin

Posterior Cramér–Rao Lower Bound

      To assess the performance of an estimator, a lower bound is always desirable. Such a bound is a measure of performance limitation that determines whether or not the design criterion is realistic and implementable. The Cramér–Rao lower bound (CRLB) is a lower bound that represents the lowest possible mean‐square error in the estimation of deterministic parameters for all unbiased estimators. It can be computed as the inverse of the Fisher information matrix. For random variables, a similar version of the CRLB, namely, the posterior Cramér–Rao lower bound (PCRLB) was derived in [52] as:

      (4.19)

      where denotes the inverse of Fisher information matrix at time instant . This bound is also referred to as the Bayesian CRLB [53, 54]. To compute it in an online manner, an iterative version of the PCRLB for nonlinear filtering using state‐space models was proposed in [55], where the posterior information matrix of the hidden state vector is decomposed for each discrete‐time instant by virtue of the factorization of the joint PDF of the state variables. In this way, an iterative structure is obtained for evolution of the information matrix. For a nonlinear system with the following state‐space model with zero‐mean additive Gaussian noise:

      (4.20)

      (4.21)

      the sequence of posterior information matrices, , for estimating state vectors, , can be computed as [55]:

      (4.22)

      where

      (4.23)

      (4.24)

      (4.25)

      (4.26)

      where and are the process and measurement noise covariance matrices, respectively.

4.6 Concluding Remarks

      The general formulation of the optimal nonlinear Bayesian filtering leads to a computationally intractable problem; hence, the Bayesian solution is a conceptual solution. Settling for computationally tractable suboptimal solutions through deploying different approximation methods has led to a wide range of classic as well as machine learning‐based filtering algorithms. Such algorithms have their own advantages, restrictions, and domains of applicability. To assess and compare such filtering algorithms, several performance metrics can be used including entropy, Fisher information, and PCRLB. Furthermore, the Fisher information matrix is used to define the natural gradient, which is helpful in machine learning.