Econometrics Toolbox™ supports modeling and analysis of discrete-time Markov models. These models describe stochastic processes that assumestatesx_tin astate spaceX, subject to theMarkov property, which requires the distribution ofx_{t+ 1}to be independent of the history of the process before reaching statex_t.

Discrete-state Markov processes, orMarkov chains, are represented by a directed graph and described by a right-stochastic transition matrixP. The distribution of states at timet+1 is the distribution of states at timet, multiplied byP. The structure ofPdetermines the evolutionary trajectory of the chain, including asymptotics.

Continuous-state Markov processes, orstate-space models, allow for trajectories through a continuous state space. The underlying Markov process is typically unobserved. Supplemental observation equations describe the evolution of measurable characteristics of a system, dependent on the Markov process.