Linear Mixed-Effects Models

Linear mixed-effects models are extensions of linear regression models for data that are collected and summarized in groups. These models describe the relationship between a response variable and independent variables, with coefficients that can vary with respect to one or more grouping variables. A mixed-effects model consists of two parts, fixed effects and random effects. Fixed-effects terms are usually the conventional linear regression part, and the random effects are associated with individual experimental units drawn at random from a population. The random effects have prior distributions whereas fixed effects do not. Mixed-effects models can represent the covariance structure related to the grouping of data by associating the common random effects to observations that have the same level of a grouping variable. The standard form of a linear mixed-effects model is

$y = \underset{f i x e d}{\underset{︸}{X β}} + \underset{r a n d o m}{\underset{︸}{Z b}} + \underset{e r r o r}{\underset{︸}{ε}},$

where

yis then-by-1 response vector, andnis the number of observations.
Xis ann-by-pfixed-effects design matrix.
βis ap-by-1 fixed-effects vector.
Zis ann-by-qrandom-effects design matrix.
bis aq-by-1 random-effects vector.
εis then-by-1 observation error vector.

The assumptions for the linear mixed-effects model are:

Random-effects vector,b, and the error vector,ε, have the following prior distributions:

$\begin{array}{l} b ~ N (0, σ^{2} D (θ)), \\ ε ~ N (0, σ {}^{2}I), \end{array}$

whereDis a symmetric and positive semidefinite matrix, parameterized by a variance component vectorθ,Iis ann-by-n单位矩阵,σ²is the error variance.
Random-effects vector,b, and the error vector,ε, are independent from each other.

Mixed-effects models are also calledmultilevel modelsorhierarchical modelsdepending on the context. Mixed-effects models is a more general term than the latter two. Mixed-effects models might include factors that are not necessarily multilevel or hierarchical, for example crossed factors. That is why mixed-effects is the terminology preferred here. Sometimes mixed-effects models are expressed as multilevel regression models (first level and grouping level models) that are fit simultaneously. For example, a varying or random intercept model, with one continuous predictor variablexand one grouping variable withMlevels, can be expressed as

$\begin{array}{l} y_{i m} = β_{0 m} + β_{1} x_{i m} + ε_{i m}, i = 1, 2, .., n, m = 1, 2, ..., M, ε_{i m} ~ N (0, σ^{2}), \\ β_{0 m} = β_{00} + b_{0 m}, b_{0 m} ~ N (0, σ_{0}^{2}), \end{array}$

wherey_imcorresponds to data for observationiand groupm,nis the total number of observations, and b_0mand ε_im在dependent of each other. After substituting the group-level parameters in the first-level model, the model for the response vector becomes

$y_{i m} = \underset{f i x e d e f f e c t s}{\underset{︸}{β_{00} + β_{1} x_{i m}}} + \underset{r a n d o m e f f e c t s}{\underset{︸}{b_{0 m}}} + ε_{i m} .$

A random intercept and slope model with one continuous predictor variablex, where both the intercept and slope vary independently by a grouping variable withMlevels is

$\begin{array}{l} y_{i m} = β_{0 m} + β_{1 m} x_{i m} + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M, ε_{i m} ~ N (0, σ^{2}), \\ β_{0 m} = β_{00} + b_{0 m}, b_{0 m} ~ N (0, σ_{0}^{2}), \\ β_{1 m} = β_{10} + b_{1 m}, b_{1 m} ~ N (0, σ_{1}^{2}), \end{array}$

$b_{m} = (\begin{array}{l} b_{0 m} \\ b_{1 m} \end{array}) ~ N (0, (\begin{matrix} σ_{0}^{2} & 0 \\ 0 & σ_{1}^{2} \end{matrix})) .$

You might also have correlated random effects. In general, for a model with a random intercept and slope, the distribution of the random effects is

$b_{m} = (\begin{array}{l} b_{0 m} \\ b_{1 m} \end{array}) ~ N (0, σ {}^{2}D (θ)),$

whereDis a 2-by-2 symmetric and positive semidefinite matrix, parameterized by a variance component vectorθ.

After substituting the group-level parameters in the first-level model, the model for the response vector is

$y_{i m} = \underset{f i x e d e f f e c t s}{\underset{︸}{β_{00} + β_{10} x_{i m}}} + \underset{r a n d o m e f f e c t s}{\underset{︸}{b_{0 m} + b_{1 m} x_{i m}}} + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M .$

If you express the group-level variable,x_im, in the random-effects term byz_im, this model is

$y_{i m} = \underset{f i x e d e f f e c t s}{\underset{︸}{β_{00} + β_{10} x_{i m}}} + \underset{r a n d o m e f f e c t s}{\underset{︸}{b_{0 m} + b_{1 m} z_{i m}}} + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M .$

In this case, the same terms appear in both the fixed-effects design matrix and random-effects design matrix. Eachz_imandx_imcorrespond to the levelmof the grouping variable.

It is also possible to explain more of the group-level variations by adding more group-level predictor variables. A random-intercept and random-slope model with one continuous predictor variablex, where both the intercept and slope vary independently by a grouping variable withM水平,和一组级别的预测变量v_mis

$\begin{array}{l} y_{i m} = β_{0 i m} + β_{1 i m} x_{i m} + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M, ε_{i m} ~ N (0, σ^{2}), \\ β_{0 i m} = β_{00} + β_{01} v_{i m} + b_{0 m}, b_{0 m} ~ N (0, σ_{0}^{2}), \\ β_{1 i m} = β_{10} + β_{11} v_{i m} + b_{1 m}, b_{1 m} ~ N (0, σ_{1}^{2}) . \end{array}$

This model results in main effects of the group-level predictor and an interaction term between the first-level and group-level predictor variables in the model for the response variable as

$\begin{array}{l} y_{i m} = β_{00} + β_{01} v_{i m} + b_{0 m} + (β_{10} + β_{11} v_{i m} + b_{1 m}) x_{i m} + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M, \\ = \underset{f i x e d e f f e c t s}{\underset{︸}{β_{00} + β_{10} x_{i m} + β_{01} v_{i m} + β_{11} v_{i m} x_{i m}}} + \underset{r a n d o m e f f e c t s}{\underset{︸}{b_{0 m} + b_{1 m} x_{i m}}} + ε_{i m} . \end{array}$

The termβ₁₁v_mx_imis often called a cross-level interaction in many textbooks on multilevel models. The model for the response variableycan be expressed as

$\begin{array}{l} y_{i m} = [\begin{matrix} 1 & x_{1}_{i m} & v_{i m} & v_{i m} x_{1 i m} \end{matrix}] [\begin{matrix} β_{00} \\ β_{10} \\ β_{01} \\ β_{11} \end{matrix}] + [\begin{matrix} 1 & x_{1 i m} \end{matrix}] [\begin{matrix} b_{0 m} \\ b_{1 m} \end{matrix}] + ε_{i m}, i = 1, 2, ..., n, m = 1, 2, ..., M, \end{array}$

which corresponds to the standard form given earlier,

$y = X β + Z b + ε .$

In general, if there areRgrouping variables, andm(r,i) shows the level of grouping variabler, for observationi, then the model for the response variable for observationiis

$y_{i} = x_{i}^{T} β + \sum_{r = 1}^{R} z_{i r} b_{m (r, i)}^{(r)} + ε_{i}, i = 1, 2, ..., n,$

whereβis ap-by-1 fixed-effects vector,b^(r)_m(r,i)is aq(r)-by-1 random-effects vector for therth grouping variable and levelm(r,i), andε_iis a 1-by-1 error term for observationi.

References

[1] Pinherio, J. C., and D. M. Bates.Mixed-Effects Models in S and S-PLUS. Statistics and Computing Series, Springer, 2004.

[2] Hariharan, S. and J. H. Rogers. “Estimation Procedures for Hierarchical Linear Models.”Multilevel Modeling of Educational Data(A. A. Connell and D. B. McCoach, eds.). Charlotte, NC: Information Age Publishing, Inc., 2008.

[3] Hox, J.Multilevel Analysis, Techniques and Applications. Lawrence Erlbaum Associates, Inc., 2002

[4] Snidjers, T. and R. Bosker.Multilevel Analysis. Thousand Oaks, CA: Sage Publications, 1999.

[5] Gelman, A. and J. Hill.Data Analysis Using Regression and Multilevel/Hierarchical Models. New York, NY: Cambridge University Press, 2007.

Linear Mixed-Effects Models

References

See Also

Related Topics