10 Hierarchical models
“Hierarchical HMM” means at least three different things in the literature, and the three have almost nothing in common beyond the word. This chapter separates them, because conflating them causes real confusion:
- Hierarchical state spaces, where states are nested — a state can itself be an HMM, recursively. This is the hierarchical HMM of Fine et al. (1998).
- Multilevel models, where several series share structure and the parameters of each series are drawn from a population distribution. These are the mixed HMMs of Altman (2007).
- Hierarchical priors, where a Bayesian model places a prior on the hyperparameters governing the states of a single HMM. This is the structure that Chapter 11 builds on, and the same idea taken to its infinite limit gives the HDP-HMM.
The second and third are where most applied work happens, and they are the focus here.
By the end of this chapter you will be able to
- tell the three senses of “hierarchical” apart and say which one a given problem needs;
- specify a multilevel HMM for several series, and explain how it interpolates between pooling and not pooling;
- diagnose the funnel geometry that breaks samplers for hierarchical models, and fix it by reparameterising;
- explain how a shared base measure ties the rows of \(\boldsymbol{\Gamma}\) together, and what happens in the infinite limit.
All three add a layer of structure above the parameters of the basic model, and all three buy the same thing: a way to fit a model with many parameters without estimating each one independently. Sharing information across states, across subjects, or across levels is what makes the parameter count manageable. Gelman and Hill (2007) make the general argument; nothing about it is specific to HMMs.
10.1 Hierarchical state spaces
10.1.1 The model
Definition 10.1 (Hierarchical HMM) A hierarchical HMM of depth \(D\) has states arranged in a tree. Each internal state at level \(d < D\) is itself an HMM over its children at level \(d+1\). Production states, at level \(D\), emit observations. When an internal state is entered, control passes to its sub-HMM; when that sub-HMM reaches its designated end state, control returns to the parent, which then makes its own transition.
The recursive structure captures phenomena with genuine multi-scale organisation. In speech, a sentence is a sequence of words, each word a sequence of phones, each phone a sequence of acoustic sub-states. In behavioural ecology, an animal alternates between coarse activity modes, each of which decomposes into finer movement patterns. A flat HMM can represent any of these only by taking the product of all levels, which multiplies the number of states and, more damagingly, discards the constraint that transitions respect the hierarchy.
10.1.2 Inference
The original algorithm of Fine et al. (1998) is a generalisation of the inside–outside algorithm for probabilistic context-free grammars, and it costs \(\mathcal{O}(T^3)\) — cubic in the length of the series, because a sub-HMM may be entered and exited at any pair of times.
Murphy and Paskin (2001) observed that a hierarchical HMM can be flattened into an equivalent dynamic Bayesian network with one state variable per level plus indicator variables signalling when each level has finished. Inference in the flattened model is linear in \(T\), at the cost of a state space that is the product across levels. Their construction is the practical route: express the hierarchy, flatten it, and run the standard \(\mathcal{O}(K^2T)\) recursions of Chapter 5 on the product chain.
The reason this class sees less use than the other two is that the hierarchy has to be specified in advance. When the levels are known — as in speech, where the phonetic hierarchy is given — it is exactly right. When they are not, the multilevel models of the next section usually answer the real question better.
10.2 Multilevel and mixed HMMs
10.2.1 The setting
Suppose there are \(N\) series rather than one: \(N\) animals with movement tracks, \(N\) patients with symptom histories, \(N\) households with consumption records. Write \(y_{n,1:T_n}\) for series \(n\).
Two extreme analyses suggest themselves and both are bad.
Complete pooling fits one HMM to all series, treating them as replicates of a single process. It ignores between-series variation entirely. If one animal forages more than another, the pooled model attributes the difference to noise.
No pooling fits a separate HMM to each series. It respects heterogeneity but estimates \(N\) times as many parameters, and for short series the individual fits are unstable — some will have states that empty out, others will land on spurious local maxima.
The multilevel model interpolates. Each series gets its own parameters, but those parameters are themselves drawn from a population distribution to be estimated.
Definition 10.2 (Mixed HMM) For series \(n = 1,\dots,N\), let \((S_{n,t}, Y_{n,t})\) be an HMM with transition matrix \(\boldsymbol{\Gamma}_n\) and emission parameters \(\phi_{n,1:K}\), and let these be drawn from a population distribution: \[ \bigl( \boldsymbol{\Gamma}_n, \phi_{n,1:K} \bigr) \;\big|\; \psi \;\stackrel{\text{iid}}{\sim}\; H(\cdot \mid \psi), \qquad \psi \sim p(\psi), \] with the series conditionally independent given the individual parameters. The states are labelled consistently across series, so that state \(i\) means the same thing for every \(n\).
The last requirement is essential and easy to overlook. If state \(1\) means “resting” for animal \(n\) and “travelling” for animal \(n'\), then averaging over \(n\) is meaningless. In practice consistency is enforced through the population distribution itself: a tight \(H\) keeps the series’ state definitions close together, and identifiability constraints or relabelling (Chapter 9) handle what remains.
10.2.2 Random effects on transitions
The most common specification puts subject-level random effects on the multinomial-logit scale, which respects the simplex constraints automatically: \[ \gamma_{n,ij} = \frac{\exp\bigl( \beta_{ij} + b_{n,ij} + \lambda_{ij}^\top x_{n,t} \bigr)}{\sum_{l=1}^{K} \exp\bigl( \beta_{il} + b_{n,il} + \lambda_{il}^\top x_{n,t} \bigr)}, \qquad b_{n,i\cdot} \sim \mathcal{N}(0, \Sigma_i), \] with \(\beta_{ii} = 0\) and \(b_{n,ii} = 0\) fixed for identification. Here \(\beta\) carries the population-level transition structure, \(b_n\) the individual departure from it, and \(\lambda\) the effect of covariates. Setting \(\Sigma_i = 0\) recovers complete pooling; letting \(\Sigma_i \to \infty\) recovers no pooling. The data choose where between the extremes to sit, which is the entire point.
Random effects on the emission parameters work the same way — \(\phi_{n,i} = \phi_i + u_{n,i}\) with \(u_{n,i} \sim \mathcal{N}(0, \Omega_i)\) — and allow each subject its own baseline level within a shared state structure.
10.2.3 Fitting
The likelihood for series \(n\) conditional on its random effects is the usual \(\mathcal{O}(K^2 T_n)\) forward recursion. The marginal likelihood integrates over them: \[ L(\psi) = \prod_{n=1}^{N} \int \Bigl[ \boldsymbol{\delta}_n^\top \prod_{t=1}^{T_n} \mathbf{Q}_n(y_{n,t}) \boldsymbol{\Gamma}_n \, \mathbf{1} \Bigr] \; H(d\boldsymbol{\Gamma}_n, d\phi_n \mid \psi) . \] The integral has no closed form, and it is over a space whose dimension grows with the number of random effects. Three approaches:
- Numerical integration. Gauss–Hermite quadrature is feasible for one or two random effects per subject and hopeless beyond that, since the cost grows exponentially in the dimension.
- Monte Carlo EM. Treat the random effects as additional missing data and approximate the E-step by simulation. Straightforward to implement given a working HMM E-step, but slow and with a stopping rule that requires care.
- MCMC. Extend the Gibbs sampler of Definition 9.2 with steps for the random effects and hyperparameters. Given \(b_n\), the FFBS step is unchanged; given the states, the random effects update is a standard generalised linear mixed model step, typically requiring Metropolis or Hamiltonian moves since the logit link breaks conjugacy.
MCMC is the usual choice, and it is worth knowing one implementation detail. Hierarchical models with small group-level variances produce a posterior geometry — Neal’s funnel — in which the scale parameter and the random effects are strongly and non-linearly dependent, and samplers move through it badly. Reparameterising from the centred form \(b_{n} \sim \mathcal{N}(0,\Sigma)\) to the non-centred form \(b_n = \Sigma^{1/2} z_n\) with \(z_n \sim \mathcal{N}(0,I)\) decouples them and usually fixes it. Papaspiliopoulos et al. (2007) give the general theory of when to prefer each parameterisation, and Betancourt and Girolami (2015) analyse the geometry directly.
10.2.4 What multilevel HMMs buy
Shrinkage. Individual estimates are pulled towards the population mean by an amount that depends on how much information each series carries. Short series shrink more, long series less. The result is better prediction for individuals than no pooling gives, and honest heterogeneity that complete pooling suppresses.
Population-level inference. The hyperparameters \(\psi\) answer questions that no individual fit can: how much do subjects differ in their tendency to persist in a state, and does a covariate shift that tendency systematically?
Stability. Series too short to support their own \(K\)-state fit become fittable, because the population distribution supplies what the data cannot. This is what makes multilevel HMMs the default in longitudinal studies with many short records.
Altman (2007) introduced the framework and derived the likelihood carefully; Maruotti (2011) surveys the variants and their estimation.
10.3 Hierarchical priors within a single HMM
The third sense of “hierarchical” adds a level to the prior of a single-series model. Instead of fixing the hyperparameters of Chapter 9, give them a prior.
10.3.2 The infinite limit
Letting \(K \to \infty\) in the construction above, with \(\boldsymbol{\beta}\) replaced by a stick-breaking measure, gives the hierarchical Dirichlet process HMM.
Definition 10.3 (HDP-HMM) Let \(\boldsymbol{\beta} \sim \mathrm{GEM}(\alpha)\) be a stick-breaking distribution on the positive integers, and for each state \(j\) let \[ \boldsymbol{\Gamma}_{j,\cdot} \mid \boldsymbol{\beta} \sim \mathrm{DP}(\alpha_0, \boldsymbol{\beta}), \qquad \phi_j \stackrel{\text{iid}}{\sim} G_0 . \] The state space is countably infinite, but any finite data set occupies finitely many states.
The shared \(\boldsymbol{\beta}\) is what makes this work, and it is the reason the model is hierarchical rather than simply a collection of Dirichlet processes. Independent DP priors on the rows would, with probability one, give each row a disjoint set of destination states, and the chain could never return anywhere. The common base measure forces the rows to share atoms — the same states — while allowing different transition weights.
Two refinements are standard. The sticky variant of Fox et al. (2011) adds mass \(\varkappa\) to the self-transition, correcting a well-documented tendency of the plain HDP-HMM to create redundant, rapidly-switching states. Beam sampling (Van Gael et al. 2008) introduces auxiliary slice variables that truncate the infinite state space to a finite random set at each iteration, so that exact FFBS can be run; this avoids the fixed truncation that variational and blocked-Gibbs schemes require.
The construction originates with Beal et al. (2002) and was given its proper hierarchical formulation by Teh et al. (2006).
10.3.3 Finite or infinite?
The HDP-HMM sidesteps choosing \(K\) by making the state space infinite. Chapter 11 keeps it finite and puts a prior on \(K\). Which is preferable depends on what the states are meant to be.
If states are a device for flexible density estimation, and no particular number is scientifically meaningful, the nonparametric route is natural, and its behaviour is well understood: the number of occupied states grows logarithmically with \(T\). That growth is a feature when the model is a flexible approximation and a defect when the states are supposed to be a fixed set of real regimes, since it means the answer to “how many states are there” depends on how long you observed.
If the states correspond to something real and finite — a fixed set of behaviours, disease stages, market regimes — then a finite model with a prior on \(K\) asks the question you actually want answered, and gives a posterior on \(K\) that does not drift with sample size. Miller and Harrison (2018) and Argiento and De Iorio (2022) examine the relationship between the two frameworks carefully; the finite-mixture-with-random-\(K\) construction is not merely a truncation of the nonparametric one, and its posterior for the number of components behaves differently.
10.4 Summary
- Three unrelated things are called hierarchical: nested state spaces (Definition 10.1), multilevel models across series (Definition 10.2), and hierarchical priors within one model.
- Hierarchical state spaces capture genuine multi-scale structure and are best handled by flattening to a dynamic Bayesian network, restoring \(\mathcal{O}(T)\) inference (Murphy and Paskin 2001).
- Multilevel HMMs interpolate between complete pooling and no pooling, shrinking individual estimates towards a population distribution. Non-centred parameterisation is usually necessary for the sampler to work.
- Hierarchical priors tie the rows of \(\boldsymbol{\Gamma}\) through a shared base measure, which lets states share information about which destinations matter. Taking \(K \to \infty\) gives the HDP-HMM; keeping \(K\) finite with a prior gives the models of Chapter 11.
10.5 Further reading
Fine et al. (1998) and Murphy and Paskin (2001) for hierarchical state spaces. Altman (2007) introduced mixed HMMs and Maruotti (2011) surveys the field; Zucchini et al. (2016, ch. 13) covers random effects in the same notation used here. Gelman and Hill (2007) and Gelman et al. (2013, chs. 5, 15) are the general references for multilevel modelling, and Papaspiliopoulos et al. (2007) and Betancourt and Girolami (2015) for the parameterisation issues that decide whether the sampler works. For nonparametric versions, Teh et al. (2006) is the foundational paper, Fox et al. (2011) the practical refinement, and Van Gael et al. (2008) the sampler that makes them usable.