8 Asymptotic theory
Does the maximum likelihood estimator of Chapter 6 converge to the truth, and if so how fast? For independent data the answers are classical. For HMMs they are not, because the observations are dependent, the latent states are never seen, and the parameter is identified only up to permutation. Each of these breaks a step of the standard argument, and repairing them took the field roughly thirty years — from Baum and Petrie (1966) to Bickel et al. (1998).
This chapter states the results and sketches the arguments. The technical machinery lives in Cappé et al. (2005, chs. 12–13) and Douc et al. (2014); what follows is the structure of the proofs and, more usefully, the conditions under which they hold.
By the end of this chapter you will be able to
- explain why the normalised log-likelihood is not an average of stationary terms, and how the infinite-past construction repairs that;
- state the consistency and asymptotic normality results, and identify which assumption each one leans on;
- say what changes under misspecification, and why the sandwich covariance replaces the inverse information;
- explain why AIC and BIC are heuristics for choosing the number of states rather than justified criteria.
8.1 The setting
Assume throughout:
(C1) The data are generated by an HMM with parameter \(\theta^\star\) in the interior of \(\Theta\).
(C2) The hidden chain under \(\theta^\star\) is irreducible and aperiodic, with stationary distribution \(\boldsymbol{\pi}^\star\) having all entries positive, and the process is started from stationarity, so \((S_t, Y_t)_{t \ge 1}\) is strictly stationary and ergodic.
(C3) \(\Theta\) is compact, and for every \(\theta \in \Theta\) the transition matrix satisfies \(\gamma_{ij}(\theta) \ge \epsilon > 0\) for all \(i,j\).
(C4) The emission densities \(f_i(y;\phi)\) are positive, continuous in \(\phi\), and satisfy the integrability condition \(\mathbb{E}_{\theta^\star} \bigl[ \sup_\theta |\log f_i(Y_1;\phi_i)| \bigr] < \infty\).
(C5) The model is identifiable up to permutation, in the sense of Theorem 4.1.
(C3) is the most restrictive. A uniform lower bound on every transition probability is a Doeblin condition, and it fails for exactly the sparse, structurally constrained transition matrices that motivate many applications — left-to-right models in speech, profile HMMs in bioinformatics. It is assumed because it delivers uniform forgetting via Theorem 2.3, which is what makes the log-likelihood behave like a sum of weakly dependent terms. Douc et al. (2004) relax it substantially at the cost of considerable technical work. Compactness of \(\Theta\) in (C1) also excludes the unbounded Gaussian likelihood of Chapter 4, and is typically enforced by bounding variances away from zero.
Because of the permutation symmetry, statements about consistency must be about the equivalence class \[ \mathcal{E}(\theta^\star) = \{ \theta^{\star\sigma} : \sigma \in \mathfrak{S}_K \}, \] and convergence means \(d(\hat\theta_T, \mathcal{E}(\theta^\star)) \to 0\) for a metric \(d\) on \(\Theta\).
8.2 The limiting log-likelihood
The standard consistency argument compares \(T^{-1}\ell_T(\theta)\) with its expectation and shows the limit is uniquely maximised at the truth. For independent data \(T^{-1}\ell_T(\theta)\) is an average of i.i.d. terms and the law of large numbers applies at once. Here the terms are dependent, and worse, the natural decomposition \[ \ell_T(\theta) = \sum_{t=1}^{T} \log p_\theta(y_t \mid y_{1:t-1}) \] has summands that are not stationary: the \(t\)-th term conditions on a history of growing length.
The fix is to compare with a stationary process that conditions on the infinite past.
Theorem 8.1 (Existence of the limit) Under (C1)–(C4), for each \(\theta \in \Theta\) there is a constant \(\ell_\infty(\theta)\) with \[ \bar\ell_T(\theta) := \frac{1}{T} \ell_T(\theta) \; \xrightarrow[T \to \infty]{\text{a.s. and } L^1} \; \ell_\infty(\theta), \] and the limit is \[ \ell_\infty(\theta) = \mathbb{E}_{\theta^\star} \bigl[ \log p_\theta(Y_0 \mid Y_{-\infty:-1}) \bigr] . \] Moreover \(\ell_\infty\) is continuous on \(\Theta\) and the convergence is uniform.
The argument has two halves. First, extend the observation process to a two-sided stationary sequence \((Y_t)_{t \in \mathbb{Z}}\) and show that the conditional density \(p_\theta(y_0 \mid y_{-m:-1})\) converges as \(m \to \infty\) to a limit \(p_\theta(y_0 \mid y_{-\infty:-1})\), uniformly and geometrically fast. This is precisely filter forgetting: by (C3) and Theorem 5.2, the predictive distribution at time \(0\) depends on observations far in the past only through an error decaying like \((1-\epsilon)^m\). Second, the resulting sequence \(\log p_\theta(Y_t \mid Y_{-\infty:t-1})\) is stationary and ergodic, so Birkhoff’s ergodic theorem applies, and the geometric approximation error is summable, so replacing the finite past by the infinite past changes the average by \(o(1)\).
The result is due to Leroux (1992), who obtained it via Kingman’s subadditive ergodic theorem, an elegant route that avoids the forgetting argument entirely: \(\ell_T\) is almost superadditive in \(T\).
The interpretation of \(\ell_\infty(\theta)\) is worth stating plainly. It is the expected log predictive density of one new observation given the infinite past, when the data come from \(\theta^\star\) but the prediction is made using \(\theta\). It is a scoring rule, evaluated in expectation.
8.3 Consistency
Theorem 8.2 (Strong consistency of the MLE) Under (C1)–(C5), \[ \ell_\infty(\theta) \le \ell_\infty(\theta^\star) \quad \text{for all } \theta \in \Theta, \] with equality if and only if \(\theta \in \mathcal{E}(\theta^\star)\). Consequently \(d(\hat\theta_T, \mathcal{E}(\theta^\star)) \to 0\) almost surely.
Proof. The inequality is an information inequality. Writing \(p^\star\) for the true conditional density, \[ \ell_\infty(\theta^\star) - \ell_\infty(\theta) = \mathbb{E}_{\theta^\star}\left[ \log \frac{p^\star(Y_0 \mid Y_{-\infty:-1})}{p_\theta(Y_0 \mid Y_{-\infty:-1})} \right] = \mathbb{E}_{\theta^\star}\bigl[ \mathrm{KL}\bigl( p^\star(\cdot \mid Y_{-\infty:-1}) \, \Vert \, p_\theta(\cdot \mid Y_{-\infty:-1}) \bigr) \bigr] , \] which is non-negative by Proposition 1.3, and zero only if the two predictive densities agree almost surely. Agreement for all conditioning histories means the two parameters induce the same law for the observed process, and (C5) then forces \(\theta \in \mathcal{E}(\theta^\star)\).
For the conclusion, note \(\bar\ell_T(\hat\theta_T) \ge \bar\ell_T(\theta^\star)\) by definition of the maximiser. Uniform convergence in Theorem 8.1 lets us pass to the limit on both sides, giving \(\limsup_T \ell_\infty(\hat\theta_T) \ge \ell_\infty(\theta^\star)\). Since \(\ell_\infty\) is continuous, \(\Theta\) compact, and the maximiser set of \(\ell_\infty\) is exactly \(\mathcal{E}(\theta^\star)\), any limit point of \(\hat\theta_T\) lies in \(\mathcal{E}(\theta^\star)\).
The shape of the argument is the classical Wald one; all the HMM-specific difficulty is buried in Theorem 8.1. Note also what the theorem does not say: it is a statement about the global maximiser, and Chapter 6 showed that EM returns a local one. Consistency of the global MLE is not consistency of what an algorithm produces.
8.4 Asymptotic normality
Definition 8.1 (Score and information) The score is \(U_T(\theta) = \nabla_\theta \ell_T(\theta)\) and the Fisher information per observation is \[ I(\theta^\star) = \lim_{T \to \infty} \frac{1}{T} \operatorname{Var}_{\theta^\star}\bigl( U_T(\theta^\star) \bigr) , \] assumed finite and non-singular.
Theorem 8.3 (Central limit theorem for the score) Under (C1)–(C5) and suitable smoothness, \(\ T^{-1/2} U_T(\theta^\star) \xrightarrow{d} \mathcal{N}\bigl(0, I(\theta^\star)\bigr)\).
The proof is a martingale argument. Decompose the score as a telescoping sum of increments, \[ U_T(\theta^\star) = \sum_{t=1}^{T} \nabla_\theta \log p_{\theta^\star}(Y_t \mid Y_{1:t-1}), \] and note each increment has conditional mean zero given \(\mathcal{Y}_{t-1}\), so the sum is a martingale with respect to the observation filtration. Replacing the finite conditioning history by the infinite one — legitimate by the same geometric forgetting used before — makes the increments stationary and ergodic, and the martingale central limit theorem applies. The information \(I(\theta^\star)\) is the variance of a single increment of the stationary version, which is why it can be written as an infinite sum over lags of score autocovariances. Bickel et al. (1998) give the argument for general state spaces; Douc et al. (2004) extend it to autoregressive regime-switching models.
Theorem 8.4 (Asymptotic normality of the MLE) Under the above conditions, with \(\hat\theta_T\) a consistent sequence of local maximisers converging to a fixed representative \(\theta^\star\) of its equivalence class, \[ \sqrt{T} \, \bigl( \hat\theta_T - \theta^\star \bigr) \; \xrightarrow{d} \; \mathcal{N}\bigl( 0, \, I(\theta^\star)^{-1} \bigr) . \]
Proof. Since \(\theta^\star\) is interior and \(\hat\theta_T \to \theta^\star\), eventually \(U_T(\hat\theta_T) = 0\). A first-order Taylor expansion about \(\theta^\star\) gives \[ 0 = U_T(\theta^\star) + \nabla^2 \ell_T(\tilde\theta_T)(\hat\theta_T - \theta^\star) \] for some \(\tilde\theta_T\) on the segment between them. Dividing by \(\sqrt{T}\), \[ \sqrt{T}(\hat\theta_T - \theta^\star) = -\Bigl( \tfrac{1}{T}\nabla^2 \ell_T(\tilde\theta_T) \Bigr)^{-1} \cdot \tfrac{1}{\sqrt{T}} U_T(\theta^\star) . \] A uniform law of large numbers for the Hessian gives \(T^{-1}\nabla^2 \ell_T(\tilde\theta_T) \to -I(\theta^\star)\) in probability, using consistency to handle \(\tilde\theta_T\). Slutsky’s theorem combined with Theorem 8.3 yields the stated limit.
Interiority is essential and is often violated in practice. If a transition probability is genuinely zero, \(\theta^\star\) sits on the boundary of \(\Theta\), the expansion is invalid, and the limiting distribution becomes a projection of a Gaussian onto a cone rather than a Gaussian. Confidence intervals computed from the inverse Hessian will then be wrong, typically too wide.
8.4.1 Computing the information
Three routes, in increasing order of reliability and cost.
The observed information \(-\nabla^2\ell_T(\hat\theta_T)\), computed by numerical differentiation of the forward recursion, is the usual choice and is what Zucchini et al. (2016) recommend. Louis’s identity expresses the same quantity in terms of complete-data quantities available from forward–backward, avoiding numerical differentiation. The parametric bootstrap sidesteps the asymptotics entirely and remains informative when the regularity conditions are doubtful, at the cost of refitting the model many times.
8.5 Misspecification
If the data-generating process is not an HMM in the assumed class, there is no \(\theta^\star\). The theory still says something.
Definition 8.2 (Pseudo-true parameter) Let \(\mathbb{P}^{\mathrm{true}}\) be the law of the observed process. A pseudo-true parameter is \[ \theta^\circ \in \arg\max_{\theta \in \Theta} \; \mathbb{E}^{\mathrm{true}}\bigl[ \log p_\theta(Y_0 \mid Y_{-\infty:-1}) \bigr] , \] equivalently a minimiser of the KL divergence rate from \(\mathbb{P}^{\mathrm{true}}\) to \(\mathbb{P}_\theta\).
Under conditions parallel to (C1)–(C4) — now imposed on the true process rather than on a member of the model class — Theorem 8.1 still holds and \(\hat\theta_T \to \theta^\circ\) almost surely. Asymptotic normality also survives, but with the sandwich covariance \[ \sqrt{T}(\hat\theta_T - \theta^\circ) \xrightarrow{d} \mathcal{N}\bigl( 0, \, H^{-1} V H^{-1} \bigr), \] where \(H\) is the limiting Hessian and \(V\) the limiting score variance at \(\theta^\circ\). Under correct specification the information equality \(H = V\) collapses this to \(I^{-1}\); under misspecification it does not, and using \(I^{-1}\) understates the uncertainty. This is White (1982) adapted to dependent data.
The practical message is that a fitted HMM should be read as the best available approximation in KL divergence, not as a recovered truth, and that reported standard errors should be of sandwich form unless there is reason to trust the specification.
8.6 Choosing the number of states
\(K\) is a discrete parameter and the theory above says nothing about it. It is also the hardest choice in practice.
The difficulty is that the models are nested in a degenerate way. A \(K\)-state model sits inside a \((K+1)\)-state model on a boundary — either one state has zero stationary probability, or two states have identical parameters — and in both cases the parameter is not identified at that point. The regularity conditions for likelihood ratio tests fail, and the \(\chi^2\) asymptotics do not apply. Nor is there a simple replacement: the limiting distribution depends on the geometry of the non-identified set.
Definition 8.3 (Information criteria) For a fitted model with \(d\) free parameters, \[ \mathrm{AIC} = -2\ell_T(\hat\theta_T) + 2d, \qquad \mathrm{BIC} = -2\ell_T(\hat\theta_T) + d \log T . \]
Both are used routinely and both should be treated as heuristics here, since the derivations of Akaike (1974) and Schwarz (1978) assume regularity conditions this problem violates. Empirically, AIC tends to select too many states and BIC fewer, and neither is reliable when the model is misspecified — which, for a model as stylised as an HMM, is the normal case. Pohle et al. (2017) document the pattern carefully and argue against relying on either in isolation.
More honest alternatives:
- Cross-validated likelihood, holding out blocks of the series and scoring one-step predictions (Celeux and Durand 2008). Slower, but it measures what is usually wanted.
- Model checking through pseudo-residuals and posterior predictive summaries, asking whether a fitted model reproduces the features of the data that matter, rather than which model has the best score.
- Treating \(K\) as a parameter with a prior and reporting its posterior. This is the Bayesian route, and it is the subject of Chapter 11, where the telescoping sampler makes it computationally routine. It also shifts the question in a useful way: from “how many states are in the model” to “how many states are occupied by the data”, a distinction that Chapter 11 takes seriously.
A modelling remark that carries more weight than any criterion: states are chosen to be interpretable as well as to fit. A three-state movement model whose states correspond to resting, foraging and travelling is more useful than a seven-state model with a better BIC and no interpretation. Pohle et al. (2017) make this argument at length.
8.7 Summary
- The normalised log-likelihood converges almost surely to a limit that is the expected log predictive density under the truth (Theorem 8.1), proved by combining filter forgetting with the ergodic theorem.
- That limit is maximised exactly on the permutation class of \(\theta^\star\), giving strong consistency of the global MLE (Theorem 8.2).
- The score is a martingale, so a martingale CLT gives asymptotic normality with the inverse Fisher information as covariance (Theorem 8.4) — provided \(\theta^\star\) is interior.
- Under misspecification the MLE converges to the KL-closest parameter and the covariance takes sandwich form.
- The number of states falls outside this theory. Information criteria are heuristics here; cross-validation, model checking, and putting a prior on \(K\) (Chapter 11) are the alternatives.
8.8 Further reading
Cappé et al. (2005, chs. 12–13) is the definitive treatment of consistency and asymptotic normality for HMMs. Leroux (1992) is the original consistency proof and remains readable; Bickel et al. (1998) established asymptotic normality for general state spaces and Douc et al. (2004) extended it to regime-switching autoregressions. Douc et al. (2014) place HMM asymptotics inside the broader theory of nonlinear time series. Vaart (1998) supplies the general machinery — Wald consistency, martingale CLTs, sandwich covariances — that these papers specialise. For the state-number problem, Pohle et al. (2017) and Celeux and Durand (2008).