12  State-space and switching models

The finite state space has been doing a lot of work. Dropping it — letting the hidden state be continuous, or letting discrete and continuous states coexist — produces the models in this chapter. The conditional independence structure of Definition 3.1 survives intact; what changes is that sums over states become integrals, and exact inference survives only in special cases.

The organising question is when the recursions of Chapter 5 remain computable. Two cases keep them: a finite state space, giving the HMM, and a linear-Gaussian model, giving the Kalman filter. Everything else requires approximation.

By the end of this chapter you will be able to

  • write the filtering recursion for a general state-space model and identify the predict and update steps;
  • derive the Kalman filter and match each of its pieces to the corresponding piece of the forward algorithm;
  • explain precisely why switching linear-Gaussian models are intractable, and what the standard approximations control;
  • place factorial, semi-Markov and autoregressive HMMs relative to the base model.

12.1 General state-space models

Definition 12.1 (State-space model) A state-space model consists of a hidden Markov process \((X_t)_{t\ge1}\) on a measurable space \(\mathcal{X}\) with initial density \(\mu\) and transition density \(p(x_t \mid x_{t-1})\), together with observations \((Y_t)\) satisfying \(Y_t \mid X_t = x_t \sim g(\cdot \mid x_t)\), conditionally independent across \(t\) given the states.

The joint density is \[ p(x_{1:T}, y_{1:T}) = \mu(x_1) g(y_1 \mid x_1) \prod_{t=2}^{T} p(x_t \mid x_{t-1}) \, g(y_t \mid x_t), \] identical in form to Theorem 3.1 with \(\delta_{s_1} \to \mu(x_1)\) and \(\gamma_{s_{t-1}s_t} \to p(x_t \mid x_{t-1})\). A finite-state HMM is the case \(\mathcal{X} = \{1,\dots,K\}\) with counting measure.

The filtering recursion carries over exactly, with integration in place of summation: \[ \underbrace{p(x_t \mid y_{1:t-1}) = \int p(x_t \mid x_{t-1}) \, p(x_{t-1} \mid y_{1:t-1}) \, dx_{t-1}}_{\text{predict}}, \qquad \underbrace{p(x_t \mid y_{1:t}) \propto g(y_t \mid x_t) \, p(x_t \mid y_{1:t-1})}_{\text{update}} . \] Prediction propagates the current belief through the dynamics; the update reweights it by the likelihood of the new observation. In the finite case these are a vector–matrix product and a diagonal reweighting, which is precisely \(\hat{\boldsymbol{\alpha}}_{t-1}^\top \boldsymbol{\Gamma} \mathbf{Q}(y_t)\) from Chapter 5.

The difficulty is that the integral is generally intractable, and the filtering distribution generally has no finite-dimensional representation. The finite-state case escapes because a distribution on \(K\) points is a vector of \(K\) numbers. The linear-Gaussian case escapes for a different reason.

12.2 The Kalman filter

Definition 12.2 (Linear-Gaussian state-space model) \[ X_t = F X_{t-1} + W_t, \quad W_t \sim \mathcal{N}(0, Q), \qquad Y_t = H X_t + V_t, \quad V_t \sim \mathcal{N}(0, R), \] with \(X_1 \sim \mathcal{N}(m_0, P_0)\) and all noise terms independent.

Theorem 12.1 (Kalman filter) For the model of Definition 12.2, \(\ X_t \mid Y_{1:t} \sim \mathcal{N}(m_t, P_t)\), where with \(m_{t|t-1} = F m_{t-1}\) and \(P_{t|t-1} = F P_{t-1} F^\top + Q\), \[ K_t = P_{t|t-1} H^\top \bigl( H P_{t|t-1} H^\top + R \bigr)^{-1}, \] \[ m_t = m_{t|t-1} + K_t \bigl( y_t - H m_{t|t-1} \bigr), \qquad P_t = (I - K_t H) P_{t|t-1} . \]

Proof. By induction. Suppose \(X_{t-1} \mid y_{1:t-1} \sim \mathcal{N}(m_{t-1}, P_{t-1})\). The prediction step applies a linear map plus independent Gaussian noise, so the predictive law is Gaussian with the stated moments. For the update, \((X_t, Y_t)\) given \(y_{1:t-1}\) is jointly Gaussian with \[ \operatorname{Cov}(X_t, Y_t) = P_{t|t-1}H^\top, \qquad \operatorname{Var}(Y_t) = HP_{t|t-1}H^\top + R, \] and the standard formula for the conditional distribution of one Gaussian block given another yields exactly the stated mean and covariance.

The filter is closed because the Gaussian family is closed under linear maps, convolution and conditioning — the same reason the finite-state filter is closed under its operations. The gain \(K_t\) interpolates between trusting the model (\(R\) large, \(K_t\) small) and trusting the observation (\(Q\) large, \(K_t\) large).

The parallels with Chapter 5 are exact and worth listing. Filtering is the forward recursion. The Rauch–Tung–Striebel smoother is forward–backward. The prediction error decomposition \(\ell_T = \sum_t \log \mathcal{N}(y_t; Hm_{t|t-1}, HP_{t|t-1}H^\top + R)\) is Proposition 5.1, with the scale factors \(c_t\) again equal to one-step predictive densities. And simulation smoothing is FFBS (Theorem 9.1). The algorithms are the same algorithm; only the representation of the filtering distribution differs. Kalman (1960) is the original; Durbin et al. (1998) and Douc et al. (2014) give modern treatments.

Nonlinear or non-Gaussian variants — the extended and unscented Kalman filters — retain a Gaussian representation by linearising or by propagating a deterministic set of points. They are cheap and often adequate, but they are approximations with no error guarantee, and they fail on genuinely multimodal filtering distributions.

12.3 Sequential Monte Carlo

When neither escape route is available, represent the filtering distribution by a weighted sample.

Definition 12.3 (Bootstrap particle filter) Maintain particles \(\{x_t^{(n)}\}_{n=1}^{N}\) with weights \(\{w_t^{(n)}\}\) approximating \(p(x_t \mid y_{1:t}) \approx \sum_n w_t^{(n)} \delta_{x_t^{(n)}}\). At each step: propagate each particle through the dynamics, \(x_t^{(n)} \sim p(\cdot \mid x_{t-1}^{(n)})\); reweight by \(w_t^{(n)} \propto g(y_t \mid x_t^{(n)})\); and resample when the effective sample size falls below a threshold.

The estimator of the likelihood, \(\hat L_T = \prod_t \bigl( N^{-1}\sum_n g(y_t \mid x_t^{(n)}) \bigr)\), is unbiased. That fact is more consequential than it appears: an unbiased likelihood estimator can be substituted into a Metropolis–Hastings acceptance ratio without disturbing the stationary distribution, which is the basis of particle MCMC and makes Bayesian inference feasible for models where the likelihood is unavailable. Chopin and Papaspiliopoulos (2020) is the modern reference.

For finite-state HMMs particle filters are pointless, since exact inference costs \(\mathcal{O}(K^2T)\). They matter when \(\mathcal{X}\) is continuous, or when \(K\) is so large that \(K^2\) is prohibitive.

12.4 Switching state-space models

Definition 12.4 (Switching linear-Gaussian model) A discrete regime \(S_t \in \{1,\dots,K\}\) follows a Markov chain with transition matrix \(\boldsymbol{\Gamma}\), and given the regime the continuous state evolves linearly: \[ X_t = F_{S_t} X_{t-1} + W_t, \quad W_t \sim \mathcal{N}(0, Q_{S_t}), \qquad Y_t = H_{S_t} X_t + V_t, \quad V_t \sim \mathcal{N}(0, R_{S_t}) . \]

The model combines the interpretability of discrete regimes with the flexibility of continuous dynamics, and it is the natural formalisation of a system that changes mode: an economy entering recession, a machine developing a fault, an aircraft changing flight regime.

Exact inference is intractable, and the reason is specific. Conditional on the regime sequence \(s_{1:T}\), the model is linear-Gaussian and the Kalman filter applies exactly. But the filtering distribution of \(X_t\) unconditionally is a mixture over all \(K^t\) regime histories, and the number of components multiplies by \(K\) at every step. This is exponential growth in the representation, not in the arithmetic, and no reorganisation of the recursion avoids it.

The standard responses all control that growth:

  • Collapsing. After each step, approximate the \(K\)-component mixture by a single Gaussian matching its first two moments, keeping one Gaussian per regime. This is generalised pseudo-Bayes, cheap and surprisingly effective when regimes are well separated.
  • Rao–Blackwellised particle filters. Sample the regime sequence and run an exact Kalman filter conditional on it. Only the discrete part is approximated, which is the right division of labour and gives far lower variance than sampling both.
  • Variational methods. Approximate the posterior by a product of a discrete chain and a linear-Gaussian chain, and optimise the bound of 6.1. Ghahramani and Hinton (2000) develop this in detail.

In econometrics the special case where \(X_t\) is absent and \(Y_t\) follows a regime-dependent autoregression is Hamilton’s Markov-switching model (Hamilton 1989), and there exact filtering is available: with \(Y_t = \mu_{S_t} + \varphi_{S_t} Y_{t-1} + \varepsilon_t\), the only latent variable is discrete, so the standard forward recursion applies with the emission density replaced by the conditional density given \(y_{t-1}\). The intractability of Definition 12.4 comes entirely from the continuous latent state.

12.5 Other extensions

Factorial HMMs. Several independent chains \(S_t^{(1)},\dots,S_t^{(M)}\) jointly generate one observation stream. The state space is the product, of size \(K^M\), so exact inference costs \(\mathcal{O}(K^{2M}T)\) and is infeasible beyond small \(M\). The representation is far more compact than a flat HMM with \(K^M\) states, which is what makes the model attractive and approximate inference worth developing (Ghahramani and Jordan 1997).

Hidden semi-Markov models. In an HMM the time spent in a state is geometric, since leaving is an independent coin flip at each step. That is often wrong — behaviours and regimes have characteristic durations with modes away from zero. A hidden semi-Markov model specifies the dwell-time distribution directly. It can be approximated arbitrarily well by an HMM with a block of states per semi-Markov state, which keeps the standard algorithms at the cost of a larger \(K\); Zucchini et al. (2016, ch. 12) give the construction.

Nonparametric emissions. Leaving the \(f_i\) unspecified and estimating them alongside the discrete structure is possible, and Gassiat et al. (2016) show identifiability holds under linear independence alone. Estimation uses spectral methods or nonparametric mixtures.

Autoregressive and covariate-dependent HMMs. Allowing \(Y_t\) to depend on \(Y_{t-1}\) as well as \(S_t\) weakens assumption (A2). All the algorithms survive: the emission factor \(f_{s_t}(y_t)\) simply becomes \(f_{s_t}(y_t \mid y_{t-1})\), and nothing in the derivations of Chapter 5 used the absence of that dependence.

12.6 Summary

  • State-space models keep the HMM conditional independence structure and replace the finite state space with a general one; the filtering recursion is unchanged apart from integration replacing summation (Definition 12.1).
  • Exact closed-form filtering survives in exactly two cases: finite state spaces, and linear-Gaussian models where the Kalman filter (Theorem 12.1) plays the role of the forward algorithm.
  • Sequential Monte Carlo handles the general case, and its unbiased likelihood estimator supports exact Bayesian inference through particle MCMC.
  • Switching models combining discrete regimes with continuous states are intractable because the filtering distribution is a mixture whose component count multiplies by \(K\) each step; collapsing, Rao–Blackwellisation and variational bounds all control that growth.

12.7 Further reading

Douc et al. (2014) covers state-space models, Kalman filtering and switching models with the same notation used here. Cappé et al. (2005, chs. 3–4) treat general state spaces rigorously. Chopin and Papaspiliopoulos (2020) is the standard modern reference for sequential Monte Carlo, and Doucet et al. (2001) remains a useful collection. Hamilton (1989) for regime-switching in econometrics, Ghahramani and Jordan (1997) and Ghahramani and Hinton (2000) for factorial and switching models with variational inference, and Zucchini et al. (2016, chs. 10, 12) for semi-Markov and covariate extensions in the finite-state setting.