Beyond SEM: General Latent Variable Modeling

Bengt O. Muthén

doi:10.2333/bhmk.29.81

Beyond SEM: General Latent Variable Modeling

Muthén, Bengt O. 2002-01-01 00:00:00 [kIIH, -i(lrlllN film V .. 1. 29. IIi". !. 2002. XI· I17 © The Author(s) 2002, corrected publication 2021 Bengt. O. Muthen* This article gives an overview of statistical analysis with latent variables. Us ing traditional tructural s equation modeling as startin a g point, it shows how the idea of latent variables captures a wide variety of statistical oncepts. c including random effects. missing data, sources of v ariation in hierarchical data. finite mix tures. latent classes. and clusters. These latent variable applications g o eyo b nd the traditional l atnt e variable se u age in psychometrics with its focus on measure ment error nd a hypothetical constructs measured by multiple indicators. The article argues for the value of integrating statistical and psychollletric modeling ideas. Different applications a re discussed in a unifying framework that brings together in one general model uch s different analysis types as factor models, growth curve models. lllultilevel models. latent class models and discrete-time survival models. Several possible combinations and extensions of these models are made clear due to the unifying framework. 1. Introduction This art icle gives a brief overview of statistical analysis wit h latent variables. A key feature is that well-known modeling with continuous latent variables is ex panded by adding new developments also including categorical latent variables. Taking traditional tructur s al qu e ation modeling as a st arting point, the article shows the generality of latent variables. being able t.o capture a wide variety of st atistical concepts, including random effects, missing data, sources o f variation in hierarchical data, finite mixtures, latent classes, and clusters. These latent variable applications go beyond the traditional latent variable use age in psycho metrics with its focus on measurement error and hypothetical onstruct c s measured by multiple indicators. The article does not discuss estimation nd a testing but focuses on m odeling ideas and connections between different modeling tradit ions. A few key applica tions will be discussed briefly. Although not going into details. the presentation is statistically-oriented . For ess l technical overviews and further applications of new d velopmen e ts using categorical latent variables, see. e.g .. 1IuthCn (2001a. b) and Muthen and :tvluthen (2000). All analyses are performed using the T\Iplus program (MutMn & MutMn, 1998-2001) and Mplus input, output. and data for these examples are available at www.statmodel.com/ mplus/examples/ penn.html. One aim of the rticl a e is tin o spire a b tt e er integration of psychometric mod eling ideas into mainstream statistics and a better use of statistical analysis ideas in latent variable modeling. Psychometrics and statistics have for too long been Key Words and Phrases : \Iaximum-likelihood estimation. factor anavs l is. tru s ctural eq ua t ion modeling, growt h urve c modeling, multilevel m odling e , finite mixture modeling, random effects. missing data, latent classes. * University o f California, Los Angeles. Graduate School of Education &: Information Studies. Moore Hall. Box 9.5 1521 , Los Angeles C A 90095-1.521, U.S.A. Email: hmut hen ©:ucla.edu 82 Bengt O. I\Iuthen developed too separately and both fields can benefit from input from the ot.her. Traditionally, psychometric models have been concerned with measurement error and latent variable constructs measured with multiple indicators as in factor anal ysis. Structural equation modeling (SE1\,1) took factor analysis one step further by relating the con8truct8 to each other and to covariates in a system of linear regressions thereby purging the "structural regressions" of biasing effects of mea surement error. The idea of using systems of linear regressions emanated from supply and demand modeling in econometrics and path analysis in biology. In this way. SEl'd consists of two ideas: latent variables and joint analysis of systems of equations. It is argued here that it is the latent variable idea that is more powerful and more generalizable. Despite its widespread use among applied re searchers. SEl\I has still not been fully accepted in mainstream statistics. Part of this is perhaps due to poor applications claiming the establishment of causal models and part is perhaps also due to strong reliance on latent variables that arc only indirectly defined. The skepticism about latent variables is unfortunate given that, as shown in this article, latent variables are widely used in statistics. although under different names and different forms. This article argues that by emphasizing the vehicle of latent variables. psy chometric modeling such as SEM can be brought into mainstream statistics. To accomplish this, it is necessary to dearly show how many statistical analyses implicitly utilize the idea of latent variables in the form of random effects. compo nents of variation. missing data. mixture components, and clusters. To this aim. a general model is discussed which integrates psychometric latent variable models with latent variable models presented in the statistical literature. The generality of the model is achieved by considering not only continuous latent variables but also categorical latent variables. This makes it possible to unify and to extend a \vide variety of common types of analyses, including SEM. growth curve modeling, multilevel modeling, missing data modeling. finite mixture modeling, latent class modeling. and survival modeling. The general model is shown schematically in Figure 1. The general framework (D) is represented by the square. while special cases (A. B. C) to be discussed in the article are shown in ellipses. The gen eraJ framework is drawn from ]\Iuthen and l'vIuthen (1998-2001; Appendix 8) as implemented in the Mplus computer program (www.statmodel.com). It should be noted that Figure 1 is a simplification. For example. the general framework includes direct effects from c to u. from c to y. and allows c to also influence regression and variance parameters in the u and y parts of the model. It is hoped that the use of a single modeling and software framework makes latent variable modeling more accessible to both statisticians and substantive researchers. Statis ticians can more easily see connections between latent variable uses that they are accustomed to and psychometric uses. Substantive researchers can more easily focus on the research problem at hand rather than learning a multitude of model specification systems and software languages. The article is structured as follows. Section 2 discusses framework A of the BEYOI\D SEM: VARIABLE 83 Figure 1: A general latent variable modeling framework general model. This framework corresponds to the more well-known case of con tinuous latent variables. Sub-sections discuss the modeling of measurement error and measurement invariance in convcntional SEM, random effects in growth mod eling, and variance components in multilevel modeling. Section 3 discusses frame work B introducing categorical latent variables, including latent class analysis and latent class growth analysis. A latent class analysis example is presented where individuals are classified based OIl their antisocial behavior. Section 4 discusses framework C, including latent profilc models and models that combine continuous and categorical latent variables such as growth mixture models. A growth mix ture example is presented where children are classified into a problematic class based on their reading development in Kindergarten and second grade. Section 5 discusses the general framework D, presenting new types of models, including modeling with missing data on a categorical latent variable in randomized trials. Section 6 concludes. 2. Modeling Framework A: Continuous Latent Variables Consider the special case A of the general modeling framework shown in Figure 1. Framework A is characterized by using continuous latent variables, denoted by the vector 77, shown as a circle in ellipse A in Figure 1. Here, latent variables are used to represent constructs that have fundamental substantive importance but are only measured indirectly through multiple indicators that capture different 84 Bengt O. Muthen aspects of the constructs. As a first step, a general SEM formulation of framework A is presented, fol lowed by the key analysis areas of random effects modeling and variance compo nent modeling. 1) The measurement part of the model is defined in terms of the p-dimensional continuous outcome vector y. (1) Yi = 1/ + A "1i + K Xi + ci· where "1 is an m-dimensional vector of latent variables, x is a q-dimensional vector of covariates. f. is a p-dimensional vector of residuals or measurement errors which are uncorrelated with other variables, 1/ is a p-dimensional paramet.er vector of measurement intercepts, A is a p x Tn parameter matrix of measurement slopes or factor loadings, and K is a p x q parameter matrix of regression slopes. Usually, only a few of the elements of K have nonzero elements, where a non-zero row corresponds to a y variable that is directly influenced by one or more x variables. The covariance matrix of f. is denoted e. The structural part of the model is defined in terms of the latent variables regressed on each other and the q dimensional vector x of independent variables, (2) Here, a is an Tn-dimensional parameter vector, B is an Tn x Tn parameter matrix of slopes for regressions of latent variables on other latent variables. B has zero diagonal elements and it is assumed that I - B is non-singular. Furthermore, r is an Tn x q slope parameter matrix for regressions of the latent variables on the independent variables. and ( is an m-dimensional vector of residuals. The covariance matrix of ( i s denoted q,.. In line with regression analysis, the marginal distribution of x is not modelled bllt is left unrestricted. This leads to the mean and covariance structures conditional on x. 1/ + A (I -B)-1 a + A (I - B)-1 r x + K x. (3) A (I - B )-1 q,. (I - B),-1 A' + e. (4) \Vith the customary normality assumption of y given x , the parameters of the model are estimated by fitting (3) and (4) to the corresponding sample quantities. This is the same as fitting the man e vector and covariance matrix for the vector (y,x), to the sample means, variances, and covariances for (y, x), (Joreskog & Goldberger, 1975). Here, the ma..ximum-likelihood estimates of J..L and :En are the corresponding sample quantities. examples of framework A models are given at www.statmodel.com/mplus/examples/ continuous.html. BEYOND SKTl.l: GENERAL LATENT VARIABLE MODELING 85 Joint analysis of independent samples from multiple groups is also possible, a,.'lsuming different degrees of parameter invariance across groups. In particular, full or partial invariance of the measurement parameters of v and A is of interest in order to study group differences with respect to 0: and \[I. From an application point of view, the modeling in (1), (2) is useful for purging regression relationships of detrimental effects of measurement error when multiple indicators of a construct are available. I\Ieasurement errors among the predictors are well-known to have particularly serious effects. but the modeling is also useful in examining a factor model where the measurement errors are among the outcome (indicator) variables as when using a factor analysis with covariates C'I\HI\IIC" modeling). In this special case, B = 0 in (2). A baseline MIMIC analysis assumes K = 0 in (1) and a sufficient number of restrictions on A and \[I to make the model identified (in an exploratory analysis, this amounts to using rn restrictions in line with exploratory factor analysis). The covariates strengthen the factor analysis in two ways (d. Muthen. 1989). First, by making the test of dimensionality stronger by using associations not only among the y variables but also between y and x. Second, by making it possible to examine the extent of measurement invariance across groups defined by different values on x. Measurement non-invariance across groups defined by Xk (e.g. Xki = 0/1 for individual i) with respect to an outcome Yj is captured by Kjk -I- 0, reflecting a group-varying intercept. Vj + Kjk Xk. The model of (1) - (4) is typically estimated by maximum-likelihood (ML) un der the assumption of multivariate normality. Browne and Arminger (1995) give an excellent summary of modeling and estimation issues for this model. This is the analysis framework used for the last 20 years by conventional SEM computer programs such as A1\10S, EQS, and LISREL. More recently, ML estimation as suming missing at random (MAR) in the sense of Little and Rubin (1987) has been introduced in SEM software. Browne and Arminger (1995) also discuss the case where some or all of the y outcomes are categorical. The case of categorical outcomes has been further treated in Muthen (1984. 1989) with an emphasis on weighted least-squares es timation, including a new approach presented in Muthen, DuToit. Spisic (1997). Mplus includes modeling with both continuous and categorical outcomes y.2) Drawing on I\Iuthen (1996) and Muthen and Christofferson (1981). I\Iplus pro vides a lIlore flexible parameterization than conventional SEM software in terms of its categorical outcome modeling for longitudinal data and multiple-group analysis using threshold measurement parameters that allow for partial invariance across time and group. For connections ''lith item response theory, sec. e.g .. I\Iuthen (1988). IvIuthen. Kao and Burstein (1991). and Takane and DeLeeuw (1987). For an overview of conventional SEM with continuous outcollles, see, e.g. Bollen (1989). For examples of SEI\I analysis in behavioral research. see. e.g .. examples are given at www.statmodel.com/mplus/examples/categorical.html. Sf) Bengt O . .\Iuthen .MacCallum and Austin (2000). 2.1 Random effects growth modeling The use of random effects is another example of modeling with continuous latent variables. In mainstream statistics, random effects are used to capture unobserved heterogeneity among subjects. That is, individuals differ in systematic ways that cannot be. or at least have not been. measured. Unlike the case of psychometric latent variable contexts. however. the random effects are typically not thought of as constructs of primary interest. and there is typically not an attempt at directly measuring the random effects. 2.1.1 A growth modeling example Growth modeling is an interesting example of random effect modeling where the heterogeneity concerns individual differences in trajectories. Consider an ex ample from reading research. The data are from a cohort-sequential reading study of 945 children in a sample of Texas schools, following them from Kindergarten through second grade. In Kindergarten a phonemic awareness score was mea sured as a reading precursor skill. In grades 1 and 2, word recognition scores were collected. All measures were collected at four times during the school year. At the end of grade 2, standardized reading and spelling scores were also recorded. These data will also be used to illustrate growth mixture modeling in framework C. Figure 2 shows observed individual trajectories on a phonemic awareness score for Kindergarten children divided into the upper and lower decile on the grade 2 spelling score. The individual variation in the trajectories is clearly seen with those in the lower spelling decile showing a lower initial and ending status 1Il Kindergarten and a lower growth rate than those in the upper spelling decile. 2.1.2 Alodel,tng issnes A modeling example shows the latent variable connections. Let the random variables 7)0, 7)1. and 7)2 represent an intercept, a linear. and a quadratic slope. respectively. These are coefficients in the regression of the outcome on time and the fact that they vary across individuals gives rise to the term random coeffi cients or random effects. The random effects capture the individual differences in development over time using the Laird and \Vare (1982) type of model Yit = Tloi + T)l; (a/ - a) + T)'2i (at - a)2 + ""t :rit + fit, (5) Tloi = eto + XiO + (Oi. (6) (7) Tili = (tl + :1::;0 + (ii, (8) '72i = et2 + {2 :1::iO + (21' where at is a time-related variable, a is centering constant, Xt is a time-varying covariate. and Xo is a time-invariant covariate. In multilevel terms (see, e.g .. Bryk BEYOND SEfl'l: GENERAL LATENT VARIABLE 87 Upper Decile 2nd Grade Spellers ........... :-: .... ........ :::. Lower Decile 2nd Grade Spellers ... ..... . ....... .......... . ...... __ ..... _-............ ::::::: . ..-.-::::::::::: . ..-.-_.- ':::::;:: Figure 2: Phonemic awareness development in Kindergarten & Raudenbush, 2002), (5) is referred to as the level 1 equation, while (6) - (8) are referred to as level 2 equations. In mixed linear modeling (see, e.g. Jennrich & Sluchter, 986; 1 Lindstrom & Bates, 1988; Goldstein, 1995), the model is expressed in terms of Yt related to at, X t , and :z:o, inserting (6) - (8) into (5) . It is clear that (5) - (8) can be expressed in SEM terms using ( 1) and (2) by let t.ing Yi = (Yi l ,Yi2, ... , YiT )', 'lJi = (r}Ol , l]li,T}2i)' , and X i = (xil, :ri.2: . . . :XiT,l:iO)' . 'While multilevel modeling views the analysis as a two-level analysis of a univariate outcome ;tI, the SEM approach is a single-level analysis of the multivariate vector y. This issue will be discussed further in Section 2.2. Furthermore, v = 0 while 0: contains three free parameters. Alternatively, the equivalent parameterization 88 Bengt O. VI = v2 = ... = VT with Q = (0.0:1,0'2)' may be used. Also. a1 - a (aj - aJ' ) a2 - a (a2 - a)2 , (9) h( ! aT - a (at - 0,)2 showing that growth modeling with random effects is a form of factor analysis with covariates. Time-invariant covariates have effects on the factors and time-varying covariates have direct ff eects on the outcomes. \Vhile not typically thought of as such. the repeated measures of YI , Y2 ,' .. ,YT can be seen as multiple indicators o f t he random effects, or growth factors as they are referred to in the latent variable literature. One may wonder why a latent variable model with such a restricted A matrix as in (9) , with no free parameters. would ever be realistic. But it has been found that this model often captures the essential feat ures of growth. In the latent variable framework however. it is easy to allow deviations from the functional growth form by estimating some of the oa l dings. Multilevel and mixed linear modeling traditions consider a more general form of (5), (10) Yit = 170i + 17li (ail - a) + 172i (ait - a) + " .j Xi I + f i l , where ail indicat.es the possibility of individually-varying times of observation and the slope Ki is yet another random effect. These traditions treat (ait - a) as data, whereas conventional SEM software treats (at - a) as parameters. This is the only way that random slopes can be handled in conventional SE:t'vI. In (10), 17li . 172i' and Ki are random slopes for individually-varying variables a and x. As pointed out in Raudenbush (2001) such modeling cannot be summarized in terms of mean nd a covariance structures. Unlike (4), the variance of y conditional on the a and x: variables varies as a function of these variables. In principle, however. this points to a hort s coming of conventional SEM software, not a shortcoming of latent variable modeling. Drawing on Asparouhov and 1Iuthen (2002), Mplus incorporates individuallY'-varying times of observations and random slopes for time-varying covariates as in (10) .:3) rvlainstream statistics aso l takes an interest in 'what psychometricians call fac tor scores, i.e. estimates of 'rJi values to be used for estimation of individual growth curves. Both fields favor empirical Bayes estimates. efe r rred to as the regression method in psychometrics. It follows that there are several advantages of placing the growth model in a latent variable context. For example. t.he psychometric idea of a latent variable construct is not ut.ilized in t he growth model of (5) - (8). Although the y out comes are manifestations o f growth. a psychometric approach ould c in principle 3)These features are included in Version 2.1 to be released Spri ng 2002 as a free upgrade for Version 2 users. BEYOND SEk1: GENERAL LATENT VARIABLE MODELING 89 seek specific indicators of the growth factors, for instance measuring indicators of growth potential at the outset of the study. all approach that does not seem to have been pursued. A more common situation is that a esea r rcher wants to study grO\\Tth in a latent variable construct measured with multiple indicators. The model specification is as follows, for simplicity shown for a linear model with a single latent variable construct 1}it. Let Yijt denote the outcome for individual i. indicator j. and timepoint t, and let Tht denote a latent variable construct, Level-1 a (measurement part) : (11 ) Yijt = Vj t + Aj t 1}i t + fijt , Level-1 b : llit = 1}Oi + 1]1; at + (it, (12) (13) Level-2a : 1]0; = no + :0 Xi + (Oi, (14) Level-2b : 'IIi = 0:1 + {I Xi + (Ii· In line with the second parameterization given above for a ingle s outcome, measurement invariance is specified by using time-invariant indicator intercepts and slopes: (15) Vjl = Vj2 = ... VjT = = Vj , (16) A 1 = Aj2 = ... = AjT = A]' setting the metric of the latent variable construct by Al = 1. The intercept of the level-2a equation is fixed at zero, 0:0 = O. V(Eijd and V'((it) may vary over time. Structural differences are captured by letting E(1]id and V(rJit ) vary over time. \Vith more than one population. across-population measurement invariance would be imposed and 0:0 fixed to zero only in the first population. Multiple-indicator growth modeling has the advantage t hat changes ill measurements can be made over time, assuming measurement invariance for a subset of indicators that are maintained between adjacent time points. Other advantages of growth modeling in a latent variable framework includes the ease with which to carry out analysis of multiple processes, both parallel in time and sequential, as well as multiple groups. Growth factors may bregressed e on each other using the B matrix in (2), for example studying growth while controlling for not only observed c:ovariates but also initial status. 110re generally, the growth model may be only a part of a larger modeL including for instance a factor analysis measurement part for covariates measured with errors. or including a m diational e path analysis part for variable influencing the growth factors, or includiug a set of variable that are influenced b y the grmvth process. For examples of growth modeling in a atent l variable framework, see, c.g., Muthen and Khoo (1998) and Muthen and Curran (1997). The ece r nt Collins and Sayer (2001) book gives applied contributions from several different traditions. 90 Hengt O. l\[ut.hen 2.2 Component8 of variation in hierarchical data Latent continuous variables are frequently used in statistical modeling of hi erarchical data. Here. latent variables are used to correctly reflect the sampling procedure with latent variables representing sources of variation at different levels of the hierarchy. It is instructive to consider a simple ANOVA model because it clearly shows re lationships between factor analysis. growth modeling. and more general multilevel latent variable models. Consider the nested. randolll-effects ANOVA, (17) Yij = V + 'Ii + fij :i = L 2 ..... n : j = L 2 .... , J. Here. i is the mode of variation for which an independent sample is obtained. while j is clustered within i. Typical examples are individuals observed \vithin households and students observed within classrooms. The different sources of variation are captured by the latent variables 1] and f. If instead j = 1,2, ... , ni, there is missing data on some of the J measures. Consider the covariance and variances for j = k and j = l, (18) COV(Yik: Yil) = 1'(1]). (19) V(Yik) = v(Yid = V(17) + VeE), resulting in the intraclass correlation (20) The intraclass correlation is frequently considered in the context of cluster sam pling. The intraclass correlation increases for increasing between-cluster variation VC'I) relative to total variation. Or. using equivalent homogeneity reasoning, the intraclass correlation increases when the within-cluster variation V(E) is small. In cluster samples. the intradass correlation is used to describe the lack of in dependence among observations and llsed when computing design effects. The simple model of (17) summarizes some key latent variable modeling issues in a nutshelL showing that factor analysis, growth modeling, and multilevel modeling are variations on the same theme. I t is clear that (17) can be seen as a special case of factor analysis in the SEl\I framework of (1). (2) with a single factor and A = (L 1. ... , I)'. Instead of thinking of the J units within each cluster as individuals as in (17), the J Y variables are now multiple indicators measured on the same individuaL Carrying this idea back to (17). this means that the individuals within a cluster can be seen as indicators measuring cluster characteristics. \Vhen Yj are repeated measures over time, j = t. (17) represents a growth model with random intercepts. The repeated measures take the role of multiple indicators measuring the random intercept growth factor. For example, t.he model BEYOND SEAl: GE:\EHAL LATE:'\[T VARIABLE 91 may represent blood pressure measurements on individuals where in a short time span there is no increasing or decreasing trend. The focus is on the construct of "long-term blood pressure level", e.g. for predicting later health outcomes. The E residuals represent measurement error as well &'3 time-specific variation, both of which may bc irrelevant for the prediction. It was noted earlier that growth modeling in the SEM framework leads to single-level analysis because a multivariate analysis of y is carried out. The non independence among repeated measures within an individual indicated by the intraclass correlations is modeled by the growth factors influencing the outcome at difl'erent time points. This i::; analogous to factor analy::;is. The same multi variate analysis approach may be used for more general multilevel modeling with latent variables, for example multilevel factor analysis and multilevel growth mod eling, referred to as 3-levelmodeling in the multilevel literature. The multivariate approach is suitable for situations where there are relatively few cluster members, such as with analysis of spouses. siblings. or analysis of twins in behavioral genet ics. For a recent application to growth modeling of alcohol use among siblings, see Khoo and Muthen (2000). The multivariate approach provides very flexible modeling where the relationships among units within a cluster can be modeled. In Khoo and t..Iuthen (2000) the growth factors of a younger sibling are regressed on those of an older sibling. The cluster units can also have different regressions on covariates. Different numbers of cluster units for different clusters can be han dled via missing data. although different models may be relevant for clusters of different size (i.e. two-sibling homes may have a different dynamics than homes with many siblings). With more than a couple of cluster units, however, the multivariate approach becomes computationally cumbersome. For instance with 10 measures per student with 15 students per cla.ssrooms, a multivariate vector of length 150 would have to be analyzed. As an alternative, multilevel modeling makes a simplifying assumption of cluster units being statistically equivalent as shown below. Assume c = L 2 ..... C independently observed clusters with i = L 2, ... , rlc indi vidual observations within duster c. Let z and y represent group- and individual level variables. Arrange the data vector for which independent observations are obtained as ' (' " ') c = Zc, Y cl , Y c2 , ... , Yen ' where we note that the length of de varies across clusters. The mean vector and covariance matrix are (21 ) [JLz', Inc' g JLy'] symmetric ] (22) Ineg + Inc g . The covariance matrix shows that the usual i.i.d assumption of simple random sampling is modified to allow for non-independent observations within clusters and that this non-independence is modeled by the matrix in line with the nested Bengt O. ]'viuthen ANOVA model of (17). In (21) and (22) the sizes of the arrays arc determined by the product nc x p where p is the number of observed variables. l\IcDonald and Goldstein (1989) pointed out that a great reduction in size is obtainable, reducing the ML expression {In I I + (de - J.LdJ' -1 (de - J.LdJ} c=1 to L Cd {In I I + tr[ -1 (SBd + nd (Vd - J.L)(Vd - J.L)')]} where d sums over clusters with distinct cluster sizes (for details, see Muthen, 1990). MutMn (1989, 1990. 1994) showed how SEM soft\'.rare can be used for analyzing models of this type. This is referred to ili'i 2-level modeling in a latent variable framework. Here, J.L, and are structured in terms of SEM parameter arrays based on (1) and (2). The analysis can be carried out using the Mplus program. In multilevel terms. this type of model may be viewed as a random intercept model ill line with (17) because of the additivity = + As mentioned earlier, the inclusion of random slopes leads to models that cannot be summarized in terms of mean and covariance structures as done above (Raudenbush. 2001). Nevertheless, random slopes can be incorporated into 2-levellatent variable mod eling (see Asparouhov &= l\Iuthen, 2002). 3-level modeling is also included in the framework of (21) and (22) when one of the levels can be handled by a multivariate representation. as in the case of growth modeling in line with Section 2.1. Latent variable growth modeling in cluster samples is discussed in MutMn (1997). For examples. see, e.g., Muthen (1991) with an application of multilevel factor analysis and MutMn (1989) with an application to SEl\1. Further examples are given in Hecht (20(H) and Kaplan and Elliott (1997). 3. Modeling Framework B Consider next the special case B of the general modeling framework shown in Figure 1. Framework B is characterized h;v using categorical latent variables. denoted by the circle c in Figure 1 (the circle denoted TJu will be discussed later on). The choice of using a categorical latent variable instead of a continuous latent variable is lllore fundamental than the corresponding choice of proper scale type for observed outcomes. The addition of categorical latent variables to the general framework in Figure 1 opens up a whole new set of modeling capabilities. BEYOND SEJ\I: GENERAL LAT ENT VARIABLE MODELING 93 In mainstream statistics, this type of modeling is referred to as finite mixture modeling. In the current article, the terms latent class and mixture modeling will be uoed interchangeably. As with continuouo latent variables, categorical latent variables are used for a variety of reasons as will now be shown. As a first step, a general modeling representation of framework B as used in I'vlplus (Muthen & l\Iutben, 1998-2001) is presented. This is followed by a discussion of four special cases: latent cl i::1SS analysis. latent class analysis with covariates, latent class growth analysis, latent transition analysis. and logistic re gression mixture analysis. :'Iethodological contributions to these areas have been made in separate fields often without sufficient connections and without sufficient connections to modeling in other frameworks. For example, until recently, model ing developments for continuous latent variables in framework A and categorical latent variables in framework B have been kept almost completely separate. ) Let C denote a latent categorical variable with K classes, Ci = (Cd, Cn , ... , ciK )'. where Cik = 1 if individual i belongs to class k and zero otherwise. Framework B has two parts: C related to x and u related to C and x. c is related to x by multinomial logistic regression using the K - I-dimensional parameter vector of logit intercepts etc and the (K - 1) x q parameter matrix of logit slopes r c : where for k = 1, 2, .. ., K X i e k P (Cik = llxi) = -----,-:""7, --:-::- , (23) "K eQc) +/c] Xi ' LJ=l where the last class is a reference class with coefficients standardized to zero, O:CK = 0, / Ck = O. For u , conditional independence is assumed given Ci and Xi, The categorical variable Uij (j = 1, 2, . .. , r) with Sj ordered categories follows an ordered polytomous logistic eg r ression (proport.ional odds model) , where for categories s = O. L 2, .. . , Sj - 1 and T),k,O = -00, Tj ,k,S = X , (25) Uij = s, i f Tj,k, s < ui :S Tj,k.s + l , (26) P(UiJ = SICi,xd = Fs+ 1('uT ) - Fs(ui ), j j F (u*) - __ 1,..-----c::7 (27) S - 1 + e-(Ts-U*)' where for ui = (uil,ui2,··· ,uir)', T7ui = C'/Ul i , ''lU2, , ···,T/Uj,)'' and conditional on class k, (28) u; = A Uk T7ui + KUk X i, (29) T7ui = Q + r X i, Uk Uk 4) Mplus examples of framework B m odels are given at www.statmodel.com/mplus/examples/mixture.html. 94 Bengt O. :'I'luthen where A ' is an l' x f logit parameter matrix varying across the K classes, K Uk uk is an r x q logit parameter matrix y-arying across the K classes, au,. is an f x 1 vector logit parameter vector varying across the J( classes, and r Uk is an f x q logit parameter matrix varying across the K classes. The thresholds may be stacked in the LJ=l (5 - 1) x 1 ec v tors Tk varying across the K classes. It should he noted that (28) does not include intercept terms given the presence of T parameters. Furthermore. T parameters haye opposite signs thanu* in (28) because of their interpretation as thresholds or cut points that a latent continuous response variable u' exceeds or falls below (see also Agresti. 1990. pp. For example. with a binary 1.[ scored Oj 1 (26) leads to . 1 (30) P(u.=llc.x )=l- ' )' . 1 + e-\T - U' (31) For example. the higher the T the higher 1.[* needs to be to exceed it, and the lower the probability of n = 1. Mixture modeling can involve Ilumerical and statistical problems. Mixture modeling is known to somet imes generate a likelihood function with several local maxima. The occurrence of this depends on the model and the data. It is therefore recommended that for a given dataset and a given model different optimizations are carried out using different sets of starting values. The numerical and statistical performance of mixture modeling benefits from confirmatory analysis. The same kind of confirmatory a n alysis as in regular mod eling is possible. using a priori restrictions on the parameters. \Vith mixture mod eling, however, there is also a second type of confirmatory analysis. A researcher may \'lant to incorporate the hypothesis that certain individuals are known to represent certain latent classes. Individuals with known class membership are referred to a!'; training data (see also ).Ic:Lachlan & Basford. 1988: Hosmer, IVIultiple-grouJ> modeling corresponds to the case of all sample units contributing training data so that c is in effect an observed categorical variable. In l\lplus. the training data can consists of 0 and 1 class membership values for all individuals. where 1 denotes which classes an individual may belong to. Known class membership for an individual corresponds to having training data value of 1 for the known class and 0 for all other classes. Unknown class membership for an individual is specified by the value 1 for all classes. \Vit h class membership training data. the cla..<;s probabilities are renormed for each individual to add to one over the admissible set of classes. Fractional training data is also allO\ved, corresponding to class probabilities adding to unity for each individual. \Vith fractional training data. the class probabilities are taken to be fixed quantities, which reduces the sampling variabilit:,,' accounted for in the standard error cal culations. Fractional training data where each individual has a probability of 1 for one class and ()'s for the other classes is equivalent to training data with cla..ss membership value 1 for only one class for each individual. Using training data BEYOND SET\£: GEKERAL LATENT VARIABLE MODELl KG 95 with a value of 1 for one class and (rs for the other classes makes it possible to perform multinomial logistic regression with an unordered, polytomous observed dependent variable using the l\lplus model part where c is related to x. S.l Latent class analysis In latent class analysis the categorical latent variable is used to represent unob served heterogeneity. Here. the particular aim is to find clusters (latent classes) of individuals who are similar. It is assumed that a sufficient number of latent classes for the categorical latent variable results in conditional independence among the observed outcomes. This may be viewed as heterogeneity among subjects such that the dependence among the outcomes is obtained in a spuriolls fashion by mixing the heterogeneous groups. Because the latent class variable is the only cause of dependence among the outcomes, the latent class model is similar in spirit to factor analysis with uncorrelated residuals. Latent class analysis typically considers categorical indicators u of the latent class variable c, using only a subset of modeling framework B. The variables of u are binary, ordered polytornous. or unordered polytomous. Due to the conditional independence specification, the joint probability of all u's is P(UI,U2, ... ,U ) = (32) L P(c = k) P(ullc = k) P(ll2lc = k) ... P(urlc = k). k=l The model has two types of parameters. The distribution of the categorical latent variable is represented by P( c = k) expressed in terms of the logit parameters O:q in (23). The conditional u probabilities are expressed via logit parameters in line with (31) where for a binary U logit = -Tk for class k, i.e. the u* part of (28) is not needed. Similar to factor analysis, the conditional U probabilities provide an interpretation of the latent cla.<;ses such that some activities represented by the different u's are more or less likely in some classes than others. The latent class counterpart of factor scores is obtained by posterior probabil ities for each individual belonging to all classes as computed by Bayes' formula P(c= klul,lL2 ..... Ur) = P(c = k) P(ullc = k) P(ll2lc = k) ... P(urlc = k) (33) PCllI, U2.···, u ) For an overview of latent class analysis. see Bartholomew (1987), Goodman (1974) and Clogg (1995). For examples. see, e.g., Muthen (2001b), Nestadt, Hanfelt. Liang, Lamacz, Wolyniec and Pulver (1994), Rindskopf and Rindskopf (1986), and Uebersax and Grove (1990). 96 Bengl. O. lvIuthen 3. 1.1 A latent class analysis example of antisocial behavior The National Longitudinal Survey of Youth (NLS) collected data on antisocial behavior among 16 - 23 year olds. The NLSY administers an instrument with 17 binary items. l\Iaximum-likelihood estimation by l\lplus was used. Preliminary latent class analysis of the 17 items pointed to 9 items that captured 4 differ ent latent classes of antisocial behavior. Class 4 is a normative class (no high probability of endorsing any item). Class is a drug involvement class (pot and drug items). Class 2 is a persona.] offense class (fight. threat items). Class 1 is a property offense class (shoplift, stealing less than 50. conning someone. stealing goods, breaking into property). The profile plot of Figure 3 shows the est.imated item probabilities for each of the 4 classes. It should be noted that the classes are not ordered in the sense of increasing item probabilities, but involves different kinds of antisocial activities. o --- Class 1. 11% Class 2. 27% x Class 3, 20% • Cia •• 4, 41% \, .. . ' , '. , '. i '. X--T:.. ,6 "'\./:-' pot drug thr.at shoplift goods property con Item Figure 3: Profiles of antisocial behavior Table 1 illustrates the use of t.he estimated posterior probabilities for each individual in each class. The rows correspond to individuals who have t.he highest probability for that class and the entries are the average probabilities in each class. High diagonal and low off-diagonal values are desirable for good classification. It is seen that class 2 and cla<;s 3 are the hardest to distinguish between with a relatively high average class 2 probability of 0.13 for those who have their highest probability in class 3. Class 2 is the person offense class (fight, threat) and class 3 BEYOND SEM: GENERAL LATENT VARIABLE 9i is the drug class (pot, drug). While Figure 3 shows that the two classes have rather different item probabilities on these 4 items. they arc similar on the remaining 5 items. This suggests that more items are needed to more clearly distinguish these two classes. Table 1: Classification table for antisocial behavior latent class analysis Mean Posterior Probabilities Most Likely Class Class 1 Class 2 Class 3 Class 4 Class 1 0.854 0.074 0.072 0.000 Class 2 0.042 0.810 0.082 0.066 Cla.;;s 3 0.052 0.134 0.754 0.061 Class 4 0.000 0.122 0.051 0.827 3.2 Latent class analysis with covariates Similar to factor analysis with covariates, it is useful to include covariates in the latent class analysis. The aim of the latent variable modeling is still to find homogeneous groups of individuals (latent classes), but now covariates x are included in order to both describe the formation of the latent classes and how they may be differently measured by the indicators u. The prediction of latent class membership is obtained by the multinomial re gression of c on x in (23). This gives information on the composition of the latent classes. It avoids biases in the commOll ad hoc 3-step procedure: (1) latent class analysis: (2) cla'lsification of individuals based on posterior probabilities: and (:3) logistic regression analysis relating classes to covariates. The variables of x may also have a direct influence on the variables of u. beyond the influence mediated by c. This is accommodated by estimating elements of KUk in (28). For example, with a binary u. the model forms the logistic regression of u on x for cla'>s k. . , I oglt = -Tie + ""Ie X. (34) so that the direct influence of x is allowed to vary across classes. It lllay be noted that all features of multiple-group analysis are included in the latent class analysis with covariates. with dUlllmy variable covariates representing the groups. Here. T parameters are the measurement parameters. (34) shows that conditional on class these can vary across the groups, representing for exam ple gender non-invariance. The multiple-group examples of Clogg and Goodman (1985) can all be analyzed in this way. For examples of latent class analysis with covariates, see, e.g., Bandeen-Roche, l\IigliorettL Zeger and Rathouz (1997). Fonnann (1992). Heijden. Dressens and Bockenholt (1996), fvIuthen and l'vIuthen (2000), and f..Iuthen (2001b). 98 Dengt O. 3.2.1 A latent class analysis e:rample continued Continuing the antisocial behavior latent class analysis example above, three covariatcs from the NLS are added: age, gender, and ethnicity. This example is drawn from Muthen and I\Iuthen (2000). These covariatcs are specified to influence the probability of class mmbership e using the multinomial regression part (23). I\ieasurement nOllinvariancc with respect to the three covariates can be studied by including a direct effect. from a covariate to an item but was not studied here. Maximum-likelihood estimation b.y Mplus was used. The estimates from the multinomial regression predicting class membership can be translated into the curves of Figure 4. The estimated item profiles remain approximately the same m; in Figure 3 and the class interpretation is t heefore r the same. For a given age, gender. and ethnicity. F igure 4 shows the probability of membership in each class (note t hat this is not a lonitudin g al study but the x axis correspond to ages represented in this cross-sectional sample). For example, it is seen that the normative class 4 is he tmost likely class for all ages among white women, whereas this is not true for the other three groups. White Males White Females ClaM' I c -- - c .... , I •. - CI_Z I " - - . C .... 2 C ... , " ...... Cia .. , . - CIaq.. : • - CIna ' . ! :1 --'- ----- ------- ..---. . -----. ····· x .. o 0 _ _ _ _ .o - - - - -o---__ o ___ __ o _ _ ,. 20 22 " " " " " AU Are Black Males Black Females "'-, I 0 0-, I ; """3 """". - • ••• •• )1,.-. . ' ...... .. .-•• >< -------.--- ........... " ... .... ... .. • .-----. ........... ...------• ....-- i ..... . ··· x . .... . ,..........--- . . .. ... ... .. . II , ...---- .... ----- ... .... ::.,-. . .... ...... -. . ---0 I - 6 • • • • " ••. • It 1 .-- ' ,.. ... .. .. I' o l o ____ o _ ___ _ o---- -0--- __ 0 j o ... . . . . -." . .. .. - , " " " " A.e Figure 4: Influence of covariates on antisocial behavior classes BEYOND SEl\I: GENERAL LATENT VARIABLE l\IODELINC 99 Table 2 shows the resulting c:lassification table based on estimated posterior probabilities. It is seen that the use of covariate information improves the c:lass 2, class ;) distinction relative to Table 1. Table 2: Classification table for antisocial behavior latent c:lass analysis with covariates .. Mean Posterior Probabilities Most Likely Class Class 1 Class 2 Class 3 Class 4 Class 1 0.859 0.065 0.076 0.000 Class 2 0.047 0.808 0.087 0.058 Class 3 0.033 0.067 0.816 0.084 0.048 0.846 Class 4 0.000 0.105 S.S Latent class gTOwth analysis Latent c:lass growth analysis again uses a categorical latent variable to repre sent unobserved heterogeneity, but this time in a form that connects the growth modeling discussed in Section 2.1 and the latent c:lass modeling just discussed. Here, latent c:lasses are sought that are homogeneous with respect to develop ment over time. The latent class growth analysis introduces the continuous latent variable 1711 of Figure 1. In latent class growth analysis the multiple indicators of the latent classes correspond to repeated measures over time. Individuals belong to different latent c:lasses characterized by different types of trajectories. Assume for simplicity a single outcome at each timepoinL ui = (ltil. lti2 . .... Uit, ... ,ltiT)" and the simple growth model corresponding to (28), LCl'el - 1 :uTt = T]Oi + T]li at, (35) where at are fixed time scores represented in A , l1 where T is the number of time periods. Here. 1711 in (28). (29) contains an intercept and a slope growth factor \vith differences across classes captured in a and llk r 11k Xi of (29). The effects of time-varying covariates can be captured in Kllk of (28) . It may be noted that the modeling does not incorporate continuous latent variables in the form of random effects. but that 17l! is non-stochastic conditional 100 Bengt O. Muthen on x. This implies that conditional on x there is zero within-class variation across individuals. This limitation can be relaxed in line with the growth modeling of Section 2.1. With an ordered categorical outcome variable Uit, let Tt.k.s be the sth threshold in class k at timepoint t, s = 0, L 2 .... ,St - 1, where Tt.k.O = -x, Tt.k.S = ()O. Across-time and across-class measurement invariance is imposed by the threshold specification Tl.l.s = T2,1,s = ... = TT.l,s = ... = Tl.K,s = ... = TT.K.s, (36) for each s value. In the level-2 equation corresponding to (29), the Q mean of the intercept growth factorl]oi is fixed at zero in the first class for identification purposes. The mean of the intercept growth factor is free to be estimated in the remaining classes. Latent class growth analysis has been proposed by Nagin and Land (1993); see also articles in the special issue of Land (2001). For further examples, see, e.g., Nagin (1999), Nagin and Tremblay (2001), and Muthen (2001b). 3.4 Latent transition analysis Latent transition analysis is a form of latent class analysis where the multi ple measures of the latent classes are repeated over time and where across-time transitions between classes are of particular interest. Here. latent categorical vari ables are used to capture fundamental latent variable constructs in a system of regression relations akin to SEJ\L The latent transition model is an example of the use of multiple latent class variables c and is therefore not directly incorporated in the framework specified above. Muthen (2001b) showed how multiple latent class variables can be ana lyzed using a confirmatory latent class analysis with a single latent class variable including all the possible latent class combinations. applying equality restrictions among the measurement parameters. Nevertheless. this does not handle multiple time points with parameter restrictions such as first-order Markov modeling for the latent class variables. Latent transition analysis incorporated in the general is a topic for future research. An overview of latent transition modeling issues is given in Collins and \Vugal ter (1992) and Rehoussin, Reboussin, Liang and Anthony (1998). For examples. see. e.g., Collins, Graham, Rousculp and Hansen (1997). Graham. Collins, \Vu gaIter, Chung and Hansen (1991). and Kandel, Yamaguchi and Chen (1992). 3.5 Logistic regression mixture analysis Logistic regression analysis with latent classes is interesting to consider as a special case of latent class analysis \vith c:ovariates. The model was proposed by Follman and Lambert (1989) and considers a single binary u. It may be expressed BEYOND SEM: GENERAL LATENT VARIABLE MODELING 101 for cla.'.;;s k as (37) loyit = -Tk + "'u x, which is a special case of (28) where the logit intercept, i.e. the negative of the threshold T, varies across cla..<;s but the slopes do not. In (23), , q = 0 so that the covariates are assumed to not influence the class membership. Follman and Lambert (1989) considered an application where two types of blood parasites were killed with various doses of poison. In this appli cation, the assumption of , q = 0 is natural because class membership existed before the poison was administered and was not influenced by it. Follman and Lambert (1989) discuss the identification status of the model. Even in this simple form. however, logistic regression mixture analysis is dif ficult to apply in practice, probably because of the limited information available with only a single binary 11 in addition to the covariates x. This is most likely why the analysis has not caught on in practice. In contrast. latent class analysis with covariates using multiple n variables is typically a well-behaved analysis method. 4. Modeling Framework C Consider next the special case C of the general modeling framework shown in Figure l. Framework C is characterized by adding categorical latent variables, denoted by the circle c in Figure 1, to framework A. Particular models include a variety of mainstream statistical and psychometric topics. To be discussed here are finite mixture modeling, latent profile analysis, growth mixture modeling, and mixture SEM. It is interesting to compare framework C with framework B. Framework B can be seen as containing models that use latent classes to explain relationships among observed variables. A more fundamental idea can. however. be extracted from latent class approaches. Different classes can have different parameter values and, unlike the latent cla..'is model. even different model types. In other words, the idea of unobserved heterogeneity can be taken a step further using categorical latent variables. This further step is taken in framework C, and also in the subsequent general framework D.5) In framework C, the SEl"I parameterization is generalized to multiple latent classes. adding a subscript k. This is analogous to the multiple-group situation, except that group is unobserved. In what follows, this generalization of (1) and (2) will be understood. Here, multivariate normality of y conditional on x and class is assumed. This implies that the resulting mixture distribution. not condi tioning on class, is allowed to be strongly non-normal. In the :rvIplus framework of l\luthen and Muthen (1998-2001: Appendix 8), the mixture modeling allows .olrvIplus examples of framework C models are given at www.statmodel.com/rnplus/examples/mixture.html. 102 Bengt O. every parameter of framework A to vary across the latent classes. 4.1 Finite rnixtuT'e rnodehng of multivariate normals A straightforward case of framework C is finite mixture modeling of multi variate distributions. Here, the continuous latent variables of 'r/ in Figure 1 are not used. It is assumed that for class k, y is distributed as N(llk, This is a special case of the latent class generalization of (]) where there are no fac tors. Ilk = VI", = e/.;. There are two different reasons why such a mixture model ,vould be of interest. (i) to fit a non-normal distribution and (ii) to study substantively meaningful mixture components (latent classes). The flexibility of the normal mixture model to fit highly skewed data was recognized already by Pearson (1895); for a review. see l'vIcLachlan and Peel (2000, pp. 14-17, 177-179). For example. a lognormal univariate distribution is very well fit by a 2-class mixture with equal variances. Figure 5 shows a 2-class example. At the top is shmvn the mixture distribution, that is the skewed distribution that would be seen in data. At the bottom are shown two normal mixture component distributions that when mixed together by the class probabilities r. and (1 - r.) perfectly describe the distribution at the top. If the interest is in fitting a model to data from the distribution at the top, the 2-class mixture model can be used to produce mixed mct..ximum-likelihood estimates, (38) iLm = r. p] + (1 - 7r) ih, am = r. (Pi + (1) + (1 - i.) + (2) (39) using the subscript Tn to denote the mixed estimates for the distribution at the top. The delta method can be used to compute standard errors. The idea of using mixed estimates has for example been discussed in missing data modeling using the pattern-mixture approach, see, e.g. Little and 'Wang (1996). Hogan and Laird (1997), and Hedeker and Rose (20()()). In many cases, however. the mixture components have a fundamental sub stantive meaning, ,,,,here there are theoretical rea.;;;ons for individuals to behave differently and have different antecedents and consequences. Here, mixed esti mates such as (38), (:19) are not of interest. but the focus is on the parameters of the different mixture component distributions. There may for example be bio logical/genetic reasons for the existence of different mixture components. such as ,,,,ith the two kinds of trypanosomes in Section 3.5. 11ixture modeling in applications where there are substantive reasons to in vestigate different latent classes relates to cluster analysis. Cluster analysis using finite mixture modeling has been proposed a.;;; a strong alternative to conventional clustering techniques. see, e.g .. McLachlan and Peel (2000). A classic example is the Fisher's iris data analyzed in Everitt and Hand (1981). Four measures corresponding to sepal and petal lengths and widths were used to classify 150 iris flowers. Here. there ,,,,'ere three known species of iris present and the interest BEYOKO SElII GENERAL LATENT VARIABLE !vIODELIKG 103 Mixture Distribution '" '" 0 - Component 1: NO , prcpotion .75 - .- Component 2: N(2, .a"'2), propotion .25 Mixture Components :: ,,; .................... ........ .. ' . ' . ... " ...... Figure 5: A mixture of two components was in how well the classification could be recovered. This particular example also illustrates the possible difficulty ill fitting mixture models with class-varying variances, with multiple maxima and possible non-convergence or convergence to singular covariance matrices depending on starting values. An excellent overview of finite mixture modeling is given in McLachlan and Peel 2000). ( This source abo gives a multitude of examples. The iris data example is available at the j,Iplus 'Areb ste i given above. 4.2 Latent profile analysis In contrast to the analysis of the iris example above, latent profile analysis applies a structure to the covariance matrices, assuming llllcorrelated outcomes 104 Bengt O. Muthen conditional on class. 0 0 0 ( e", ()22 0 0 ( 40) I:k 9 0 0 pPk \Vith class-varying means ILk' latent profile analysis is therefore analogous to la tent class analysis. In actual analyses. models with class-invariant variances in (40) are better behaved in terms of convergence. It is interesting to note that the latent class analysis does not face this choice given that means and variances of the categorical variables of u are not represented by separate parameters. Relation ships among latent class, latent profile. and factor analysis models are described in Bartholomew (1987), Gibson (1959), and Lazarsfeld and Henry (1968). 4.,'] Growth mixture modeling The growth modeling of Section 2.1 uses continuous latent variables in the form of random effects. The continuous latent variables capture unobserved het erogeneity in terms of individual differences in growth over time. In many appli cations. however. there are more fundamental forms of unobserved heterogeneity that cannot be well captured by continuous latent variables but require categor ical latent variables. The classes of the categorical latent variable can represent latent trajectory classes. Substantive theories motivating latent trajectory classes are common in many different fields, such as with normative and non-normative development in behavioral research and disease processes in medicine. As for latent profile analysis, growth mixture modeling imposes a structure on the covariance matrix for each class. Unlike latent profile analysis, however. growth mixture modeling does not assume uncorrelated outcomes given class. Instead, further heterogeneity within class is represented by random effects that influence the outcomes at all time points. causing them to be correlated. Assume for example the following quadratic growth model for individual 1 111 class Ie: (Ie: = L 2, ... ,K). (41 ) Yit = 170; +1]li 0kt +1]2i 0kt + fit· where Yit (i = L 2 .... . 71: t = L 2 ..... T) are outcomes influenced by the random effects r/Oi. 1/1i, and ']2i. In line with Section 2.L the time scores of ° enter into the Ak matrix. The residuals fit have a TxT covariance matrix e . possibly varying across the trajectory cla.'ises (k = L 2 ..... K). The random effects are related to the covariates x, ( 42) 170i = (Yak + Xi + (0;. ( 43) 171i = Glk + Xi + (li. ( 44) 7]2i = G2k + ';k x; + (2i' BEYOND SEAl: GENERAL LATENT VARIABLE tvlODELI:\G 105 The residuals (i have a 3 x covarianc 3 e matrix'll b possibly varying across classes k (k = L 2, ... . K). It is dear that thit; model fits into framework C in line with how the growth model fit into framework A. The growth mixture model offers great flexibility in across-class parameter differences. The different shapes of the latent trajectory classes are typically characterized by the class-varying ctk parameters holding Ak class-invariant. Cer tain classes may require class-specific variances'll k and e k. In addition, different cla.<;ses may have different relationt; to x orresponding c to cla.'ls-varying 'Y k coeffi cients. A special case of the growth mixture model is obtained as a continuous-outcome version of the latent class growth analysis presented in Section 3.3. This type of modeling, proposed by Nagin and introduced into PROC TRAJ in SAS specifies '11k = 0, ek = e I. In contrast, growth mixture modeling allows for individual variation within each class through'll k. The latent cla.<;s growth analysis typically requires many more classes to fit the same data and often several of the classes represent only minor variations in trajectories and not fundamentally different growth forms. Muthen et al. (in pres..,) present a growth mixture model suitable for random ized trials. In conventional growth modeling the treatment effects can be modeled as affecting the trajectories after the treatment has started. The Muthen et al. generalization addresses the common situation that treatment effects are often different for different kinds of individuals. It allows treatment effects to vary across latent trajectory classes for the repeated measures. For a technical description of growth mixture modeling, see Muthen and Shed den (1999) and :tvluthen and Muthen (1998-2001: Appendix 8). For examples, sec, e.g. IvIuthen and Shedden (1999), Muthen and :t-.luthen (2000). Muthen (2001a. b), Muthen. Brown. Masyn . .10, Khoo. Yallg. Wang. Kellam, Carlin and Liao (in press). and Li, Duncan, Duncan and Acock (2001). 4.3.1 A growth mixture modeling example of reading fa'iiure An example clarifies the analysis opportunities presented by growth mixture modeling. Section 2.1.1 introduced a r ading e data example with phonemic aware ness development in Kindergarten related to end of grade 2 spelling performance. Figure 2 suggests heterogeneity in the phonemic awareness development. with a group of children having a close to zero growth rate in Kindergarten, Reading research points to a subgroup of children who experience reading failure by third grade, It is therefore of interest to see if signs of a failing group can be found earlier. and perhaps as early as end of Kindergarten. Two analyses are presented here as illustration (see also :t-.lutlien. Khoo. Francis, & Boscardin. in press). First, a growth mixture analysis with 1 to 5 classes was lnade of the four phonemic awareness outcomes. Second, this growth mixture model was extended to include in the same analysis the spelling test outcome from the end of sec- 106 Bengt O. Muthen ond grade. l tting e the mean and variance of this out.come vary as a function of the lat.ent trajectory classes. Both models clearly fit in framework C. IVla...ximum likelihood estimat.ion by l\.Iplus was used. A conventional linear, single-dass random effects growth model fits well in this case (X (5) = 7.49, n = .582) and shows significant variation in the intercept and slope growth factors. Such a good mean and covariance structure fit can. however, he obtained even when the true model is a growth mixture model with more than one class (see. c.g. l\.Iuthen, 1989) . Fitting linear models with 2.3.4, and 5 latent cla<;ses pointed to a steady improvement of the Baye::;iall information criterion that rewaros a high log likelihood and a low number of parameters. Given the particular int.erest in a lo w. failing class, a choice does not have to be made between the 3-, 4 -. and 5-class solutions since they all re::;ulted in the same formation of a lowest cla::;::; of 56'/(. of the children. Figure 6 shows t he est.imated growth (solid li ne) and the corresponding o bserved trajectories, where the latter are obtained by using" pseudo-classes", i.e. t he selection of individuals are obtained by random draws from their estimated posterior probabilities a<; suggested in Bandeen-Roche et !. a (1977) andl\Jutlu§n et !. a (in press). Adding the second-grade spelling test to the growth mixture model shows the predictive pO'wer of the Kindergarten informatiO'n from two years earlier. The extended g rO'wth mixture model analysis showed that the means of the spelling test were significantly different across the 3 classes. Box plots of the spelling test ::;cores based on pseudo-class a'SSignments into the 3 classes are given in Figure 7. 4.4 Mixture SEM l\lixture SEM will be mentioned only briefly III this article. It follows from the dsc i ussion in Section 2 that mixture SEM and growth mixture modeling fit into the same modeling framework. l\1ixture SEM includes mixture linear regres sion. mixtnre path analysis. ac f tor mixture analysis, and general mixture SEM. Consider as an example factor mixture analysis. where for class k ( 45) E (y tJ = Vk + Ak O k, V( y d = Ak 'ltl,: + el,:. ( 46) AnalogO'us to multiple-group analysis. a major interest is in across-cla<;s variatiO'n in the factor means. variances. and covariances of Ok , 'It k. The model is similar to growth mixture analysis in that continuous latent variables, i.e. the factors, are llsed to describe correlations among the outcomes conditional O'n class as in (46) . Lubke. Muthen and Larsen (2001) studied the identifiability of the acto f r mixture mO'dei. The special case of measurement invariance fO'r all the O'utcomes, l.e. no class variation in v. A. is of particular interest becanse it places the factors in the same metric so that Ok: 'It k comparisons a re meaningful. However, Lubke, l-.luti1en and Larsen (2001 ) point to analysis difficulties with near-singular information matrix estimates when fitting such full invariance models. These BEYOJ\D SEM: GENERAL LATENT VARIABLE MODELING 107 difficulties are not shared by the growth mixture model, which typically imposes equality of v parameters across time and across class and has few if any free parameters in A. For overvie\vs and examples of fa.ctor mixture analysis and mixture SEl\L see, e.g., Arminger and Stein (1997), Arminger, Stein and Wittenberg (1998), Blafield (1980), Dolan and van der l\laas (1998), Hoshino (2001), Jedidi, Jagpal and De Sarbo(1997), Jedidi, Ramaswamy, DeSarbo and Wedel (1996) , McLachlan and Peel (2000), and Yung (1997). Class 3, 9% ' ! .,." Class 2, 34% Clas. 1, 56% Figure 6: 3-class growth mixture model for phonemic awareness 108 Bengt O. Muthen "! '" "ii Co r/l .. C! ." :! ." c:: <'oj '" Cia .. 1 Class 2 Cia •• 3 Figure 7: Box plots for second-grade spelling scores in three phonemic awareness classes 5. Framework D Consider next the most general case D of the modeling framework shown in Figure 1. Framework D is characterized by adding categorical latent variable in dicators u to framework C. Framework D clearly shows the modeling generality achieved by a combination of continuous and categorical latent variables. This unified framework is an example of the whole being more than the sum of its parts. It is powerful not only because it contains many special cases. but also because it suggests many new modeling combinations. Particular models include a wide variety of statistical and psychometric topics. To be discussed here are complier-average causal effect modeling, combined latent class and growth mix ture modeling, prediction of distal outcomes from growth shapes, discrete-time survival mixture analysis, non-ignorable missing data modeling, and modeling of semicontinuous outcomes. ) 5.1 Complier-average causal effect modeling Complier-average causal effect (CACE) modeling is used in randomized tri als where a portion of the individuals randomized to the treatment group choose filMplus examples of framework D models are given at www.statmodel.com/mplus/examples/mixture.htmlas well as at www.statmodel.com/mplus/examples/penn.html BEYOND SEM: GE,,-"ERAL LATENT VARIABLE l\lODELING 109 to not participate ("noncom pliers"). Although developed for this specialized ap plication, CACE modeling involves interesting general lat.ent variable modeling issues. In particular, CACE modeling illustrates how latent variables are used in mainstream statistics to capture missing data on categorical variables. Here, the mixture modeling focuses on estimating parameters for substanti vely meaningful mixture components. where these mixture components are inferred not only from the out.comes but also from auxiliary information. CACE modeling ep r resents a transition from framework C to framework D . where in addition to the frame work C observed dat.a information, a minimal amount of information 011 cla.;;;s membership is added in the form of a single 1l variable observed for part of the sample. In a randomized trial, it is common to have noncom pliers a mong those invited to treatment: t.hat is, some individuals do not show up for t reatment or do not take the medication. Because of randomization, a equal-sized group of noncom pliers is also present among control group individuals. although the non-compliance status does not manifest itself. The noncomplier and complier groups are typically not similar, but may differ with respect to several characteristics such as age, education, motivation, etc. The a.<;sessment of treatment effects with respect to say the mean of aoutcome n is t heefo r re complicated. Four main approaches are common. First, "intent-ta-treat" analysis makes a straightforward comparison of the treatment group to the control group. This may lead to a diluted teat r ment effect given that not everyone in this group has received treatment. Second, one may compare compliers in the treatment group with the controls. Third, compliers in the treatment group may be compared to the combined group of noncom pliers in the treatment group and everyone in the control group. Fourth, compliers in the treatment group may be compared to compliers in the control group. Only the last approach compares the same subset of people in the treatment and control groups, but presents the problem that this subset is not observed in the control group. This problem is solved by CACE mixture modeling. CACE modeling can b e expressed by t he framework D combination of (1), (2) generalized to include the lat.ent. cla.ss addition of (n) - (29). The probability of membership in the two latent classes as a function of covariates may be expressed by the logistic regression (23), while the outcome y is expressed by the mixed linear regretlsion. (47) where I denotes the 0/1 treatment/control dummy variable. Here, Ok captures the different y means for individuals in the absence of treatment. CACE modeling typically taketl rk = 0 for the noncomplier class. In statistical anays l is this situation is vewe i d as a missing data problem. Data are missing o n the binary compliance variable for individuals in the control group. while data on hi t s variable are present for the treatment group. The frame work D conceptualization is that non-compliance status is a ate l nt class variable, where this atent l class variable becomes observed for treatment group individu- 110 Bengt O. l\luthen also Hence. the latent class variable captures missing data on a categorical vari able. Although the choice between the two conceptualizations may seem as only a matter of semantics. as described below the lat.ent variable approach suggests extensions of CACE modeling using connections with psychometric modeling that have potential value in randomized trials. The fact that latent class status is knmvn for treatment group individuals can be handled in hvo equivalent \\lays in the Mplus analysis. First, training data may be used to indicate that membership in the non-compliance da.'is is impossible for complying individuals in t he treatment group and that rnembership in the compliance class is impossible for uon-complying individuals in the treatment group. Second. a binary latent class indicator 1t defined to be identical to the latent class variable Illay be introduced in line with the ate l nt class analysis of framework B. \\'itb 0 r pr e esenting noncompliance and 1 representing compliance, the v. variable has fixed parameter values, P(n = 11compliance dass) = 1. P(n = 1lnon-compliance class) = O. The variable 71. is observed for treatment group individuals and missing for control group individuals. This second approach shows t hat CACE modeling belongs in framework D and also suggests a generalization. In psychometrics. the typical approach is to seek observed indicators for latent variables. An attempt could be made to measure the variable u also among controls. for example by asking individuals before randomization how likely they are to participate in the treatment if chosen. Several different measures u could be designed and llsed as latent class analysis indicators in line with framework B modeling. For background on CACE modeling. see, e.g., AngrisL Imbens and Hubin (1996) and Frangakis and Baker (2001). For examples, see, e.g" Little and Yau (1998) , Jo (2001a, b, c), and Jo and Muthen (2001).7) 5 .;2 Combined lalent class and growth m.ixture analysis Figure 1 hows s clearly that framework D can combine the framework B latent class aualysis with the framework C growth mixture modeling. As an example, Tvluthen and I\luthen (2000) analy:t;ecl the NLSY data discussed in Section 3, where it was of interest to relate latent classes of individuals with respect to antisocial behavior at age 17 to latent trajectory clal:ises for heavy drinking ages IS-30. Here. a latent class variable was used for each of the two sets of variables, the latellt class measurement instrument for antisocial behavior and the repeated measures of heavy drinking. Csing the confirmatory latent class analysis t.echnique described in Muthen (2001b), these two latent class variables can be analyzed together. This gives estimates of the relationships between the t.\vo classifications. To the extent that the two classifications are highly correlated, a latent class 7lDat.a and 'Ilplus input for the Little and Yao (1998) example is available on the I\1plus web site. BEYOND SEA!: GENERAL LATEKT VARIABLE MODELING 111 analysis measurement instrument can improve the classification into the latent trajectory classes. This approach i s of potential importance, e.g. using the latent class measurement instrument as a screening device in a treatment study, where different treatments are matched to different kinds of trajectory classes. 5. 3 Prediction from growth shapes 11uthen and Shedden (1999) used the latent trajectory classes in a growth mixture model of heavy drinking as predictors of distal outcomes ill the form of binary 7L variables, such as indicators of alcohol dependence. Predicting from the heavy drinking growth factors faces the potential problem of a highly non linear r eat l ionship given that a growth factor aquires its meaning in conjunction \vith other growth factors. For example, in a study of problematic bhavior e such as heavy drinking. a low slope growth factor value has a different meaning if the intercept factor value is high ("chronic" development) than when it is low ("normal" development). Given that the latent trajectory classes can represent different shapes of development, prediction from the latent classes is a powerful approach. 5.4 Specia.l uses of u indicators The framework D addition of u to framework C not only adds latent class analysis type features but also provides several unexpected additional modeling possibilities. Muthen and IvIasyn (2001) show how u can be used as event history indicators in discrete-time survival analysis. This model corresponds to a single cla.'is latent class analysis. but Il/luthen and 11asyn (2001) also explore different types of mixture survival models. Muthen and Brown (2001) show how u can be used as missing data indicators for missingness on y. This leads to an approach to study non-ignorable missing data in mixture modeling, for example where missingness is rlated e to latent trajectory classes. Muthen (2001) shows how u can be used to indicate zero or " floor" values for y , that is values that represent absence of an activity. Such data are frequently seen in behavioral research given that time of onset varies across individuals. It is the strength of frame\vork D that these seemingly disparate models can be integrated and used in new combinations to provide answers to more probing research questions. 6. Conclusions This article has provided an overview of statistical analysis with latent vari ables. In psychometrics it is typical to use latent variables to represent theoretical constructs. The constructs themselves are of key int.erest and a focus is on mea suring different aspects of the constructs. In statistics, latent variables are more typically used to represent unobserved heterogeneity. sources of variation, and missing data. The latent variables are often not of key interest but are included 11 2 Bengt O. Muthen to more correctly model the data. Unobserved heterogeneity is typically rpr e e sented by random effects, i.e. continuolls latent variables. a common example being growth modeling in t he form of the mixed linear m odlin e g (multilevel mod eling) to capt.ure individual differences in growth. Continuous latent variables are also used to represent sources of variation in hierarchical cross-sectional data, to let the model properly reflect a cluster sampling scheme and to estimate variance components. Cluster analysis considers ullobserved heterogeneity in the form of categorical latent variables. i.e. latent cla..'3ses. in order t o find homogeneous of individuals. Finite mxt i ure modeling with categorical l atent variables is a rigorow; approach to such cluster analysis. :Missing data corresponds to latent varia bles that are either ontinu c ous or categorical. T he article discussed a general l atnt e variable modeling framework that uses a combination of continuous ann categorical latent variables to give a unifying view of psychometric and statistical latent variable applications. This framework shows connections between different modeling tradit ions and suggests interesting extensions. It is the hope that this general latent variable modeling framework stimulates a better integration of pychometric s and st atistical d evlopment. e Also, it is hoped hat t this framework provides substant ive researchers with an analysis tool that is both powerful and easy to uuderstand in order t o morreadily e respond to t he omplexit. c y of their research questions. Ongoing esea r rch b y t.he author aims at further xtension e s o f the modeling framework. Acknowledgements The research wa..<; supported by rant g K02 AA 00230-01 from NIAAA, by grant 40859 from and by a g rant from NIDA. The author thanks Tihomir As parouhov for valuable discussions and computational a..'isistance. Katherine Masyn provided helpful assistance with displaying the results. The work has also ben efitted from discussions in Bengt Muthen's Research Apprenticeship Course and Hendricks Brown's Prevention Science l\Iet.hodology group . REFERENCES Agresti, A. (1990). Categorical at d a analysis. New York: John Wiley &: Sons. Angrist . J. D .. Imbens. G. \Y .. &: Rubin. D. B. (1996). Idellt ification of causal effects us ing instrumental variables. Jo'umal of the American Statistical Association, 91 , 444-445. Arminger. G. &: Stein. P. (9fJi) 1 . Finite mixtures of covariance structure models with re gressors. Sociulogical A1etlwds £1 Research. 26, 148-182. Arl1linger, G .. Stein. P .. &: Wittenberg. J. (1998). of conditional mean- and covariance-structure models. Psychometrika. 64. 475-494. Asparouhov, T. &: B. (2002). Full-information maximum-likelihood estimation of general two-level latent v Clri Clble models. Draft. BEYOKD SEAl: GENERAL L ATENT VARIABLE MODELING 113 Bandeen-Roche, K. , Ivliglioretti. D. L., Zeger, S. L. , & Rathouz, P. J . (1997). Latent variable eg r ression for multiple discrete outcomes. Journal of the Amer ican Statistical Association, 92. 137-1 5 386. Bartholomew, D. J. (1987). Latent variable modls e and factoT analysis. New York: Ox ford University Press. Blafield, E. (1 980). Clustering of observations from finite mixtures with structural infor mation. Unpublished doctoral dissertation, Jyviiskyla studies in computer science, economies, and statistics, Jyvaskyla. Finland. Bollen, K. A. 989) ( 1 . Structural equations with l aen t t variables. New Y ork: John Wiley. Brow ne, ;"1. W . & Arrninger, G. (995 1 ). S pecification and estimation of m eaIl- and covariance-structure models. In G . Arminger. C. C. Clogg, & M . E . Sobel Ed ( s.), Handbook of statistical modeling for the s r:i o al and behavioral sciences (pp. 311- 350). New Y ork: Plenum Press. Bryk, A. S. & Raudenbush, S. W. (2002). Hierarchical linear m odels: A pplications and data analysis methods. 2nd ed. Newbury Park, CA: Sage Publications. Clogg, C . C. (190.5). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.) , Handbook of statistical m odeling fOT the social and behavioral sciences (pp, 311-359) . !'<ew York: Plenum Press. Clogg, C. C. & Goodman, L. A. (1985), Simultaneous latent structural anays l is in sev eral groups. In Tuma, N. B. (Ed .) , Sociological Methodology (pp. 81-110). San Francisco: J osey s -Bass Publishers. Collins, L. M& . Sayer. A. (Eds. ) (2001 ). New mthod e s for the analysis o f cha ge. n Wash ington, D.C.: APA. Collins, 1. M. & Wugalter, S. E. (1992). Latent class models for s age t -sequential d y namic latnt e variables. Multivariate Behavioral Research, 27, 131-157. Collins, L . M. , Graham, J. W., Rousculp, S. S., & Hansen, \V. B. (1007). Heavy caffeine use and the beginning of the substance use onset process: An illustra tion of l ant te transition analysis. In K. Bryant, M. \Vindle, & S. "Vest (Eds.) , The science of prevention: Methodological advances from alcohol and ubstance s use research (pp. 79-99). Washington DC: American Psychological Association. Dayton. C . 'vI. l & :'vlacready, G. B. ( 988). 1 Concomitant variable aten l t class models. JO'urnal of the meArican Statistical Association, 83, 173-178. Dolan, C. V. & van der Maas, H. L. J 99 . (81 ). Fitting multivariate normal mixtures subject to structural equation m odeling. Psychomet7ika, 63, 2 27-253. Everitt, B. S& . Hand, D. J. (1981). Finite mixture distributions. London: Chapman and Hll. a Follrnann, D. A. & Lambert. D. (1989). Generalizing logistic regression by nonparamct ric mixing. Journal of the American Statistical Association, 84, 295-300. Formann, A. K. (1992). Linear logistic latent class analysis for polytomous dta a . Jo 1177W I of the A mric e an Statistical Association, 87, 476-486. Frangakis C. E. &: Baker, S. G2001 . ( ). Compliance subsampling d esigns or fcomparative research: Estimation and opt imal planning. Biometrics, 57. 899-908. Gibson, W. A(1. 9.'59) . Three multivariate modl;;: e factor analysis, ate l nt structure anal ysis, and latent profile analysis. Psychom etrika. 24, 229-2S2. Goldstein, H. (1995) . Afultilevel statistical models. London: Edw,ll'd Arnold. Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 21S-23l. 114 Bengt O. Muthen Graham. J. \V., Collins, L. Wugalter. S. E., Chung. K., & HanselL W. B. (1991). I\Iodeling transitions in latent stage- sequential processes: A substance use prevention example. Journal of Consulting and Clinical Psychology, 59. 48-57. Hecht, R. H. (2001). :Vlultilevel modeling with SEM. In G. A. :tvIarcoulides & R. E. Schu macker (Eds.) . New developments and techniques in structural equation modeling (pp. 89-127). Mahaw. NJ: Lawrence Erlbaum Associates. Hedeker. D. & Rose. J. S. (2000). The natural history of smoking: A pattern-mixture random-effects regression model. In.1. Rose. 1. Chassin, C. Presson, & J. Sherman (Eds.). Multimriate applications in substance use research (pp. 79-112). Hillsdale, l\.J: Erlbaum. Heijden, P. G. M .. Dressens. J .. & Bockenholt. C. (1996). Estimating the concomitant variable latent-class model with the EM algorithm. Journal of Educational and Behavioral Statistics. 21. 215-229. Hogan. J. W. & Laird, M. (1997). Mixture models for the joint distribution of re peated measures and event times. Statistics in Medicine, 16. 239-258. Hoshino. T. (2001). Baycsian inference for finite mixtures in confirmatory factor analysis. Behaviormetrika. 28. 37-63. Hosmer, D. \V. (1973). A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761-770. Jedidi. K .. Jagpal, H. 5., & DeSarbo, W. S. (1997). Finite-mixture structural equation models for response-based segmentation and unobserved heterogeneity. AIarket ing Sc·ience, 16, 39-59. Jedidi, K .. Ramaswamy. V., DeSarbo, W. S., & Wedel, M. (1996). On estimating finite mixtures of multivariate regression and simultaneous equation models. Structural Equation AIodelinq, 3, 266-289. Jennrich. R. 1. & Schluchter, .\1. D. (1986). Unbalanced repeated-measures models with structured covariance matrices. Biometrics, 42, 805-820 . .10, B. (2001a). Estimation of intervention effects with noncompliance: Alternative model specifications. Accepted for publication in the Journal of Educational and Behav ioral Statistics. ,lo. B. (2001 b). ?vlodd misspecification sensitivity analysis in estimating causal effects of interventions "vith noncompliance. Accepted for publication in Statistics in Medicine. Jo, B. (2001c). Statistical power in randomized intervention studies with noncompliance. Forthcoming in Psychological Methods. Jo. B. & J\Iuthen, B. (2001). Modeling of intervention effects with noncompliance: A la tent variable modeling approach for randomized trials. In G. A. Marcoulides & R. E. Schumacker (Eds.), New df'l'eiopments and techniques in structural equation modeling (pp. 57-87). Mahaw, !'\.1: Lawrence Erlbaum Associates. Joreskog, K. G. & Goldberger, A. S. (1975). Estimation of a model with multiple indi cators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70, 631-6:39. Kandel. D. B .. Yamaguchi, K., f;,: Chen. K. (1992). Stages of progression in drug involve- ment from adolescence to adulthood: Further evidence for the gateway theory. Journal of Studies of Alcohol. 53, 447-457. Kaplan D. & Elliott. P. R. (1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation A10deling, 4, 1-23. BEYOND SEAI: GENERAL LATENT VARIABLE MODELING 115 Khoo, S. T. &. I\Iuthen. B. (2000). Longitudinal data on families: Growth modeling al ternatives. In J. Rose, L. Chassin, C. Presson. &. .1. Sherman (Eds.). A!ultivariate appl-ications in substance use research (pp. 43-78). Hillsdale, 1\ J: Erlbaum. Laird, N. M. &. Ware, .1. H. (1982). Random-effects models for longitudinal data. Bio metrics, 38, 963-974. Land, K. C. (2001). Introduction to the special issue on finite mixture models. Sociolog ical Methods &I Research, 29, 275-28l. Lazarsfeld, P. F. &: Henry. 1\. \V. (1968). Latent structure analysis. New York: Houghton Mifflin. Li, F., Duncan, T. E. Duncan, S. C., &. Acock, A. (2001). Latent growth modeling of longitudinal data: A finite growth mixture modeling approach. Structural Equation Modeling, 8 , 493-530. Little, R . .1. &: Wang, Y. (1996). Pattern-mixture models for multivariate incomplete data with covariates. Biometrics, 52, 98-111. Little, R. .I. &. Yau, L. H. Y. (1998). Statistical techniques for analyzing data from pre vention trials: Treatment of no-shows using Rubin's causal model. Psychological Methods. 3. 147-159. Lindstrom, M . .I., &. Bates, D. M. (1988). Newton-Raphson and K\l algorithms for lin ear mixed-effects models for repeated-measures data. Jo'urnal of the American Statistical 83, 1014-1022. Lubke, G., Muthen, B., & Larsen, K. (2001). Global and local identifiability of factor mixture models. Submitted for publication. University of California, Los Angeles. MacCallum, R. C. &: Austin . .I. T. (2000), Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201-226. McDonald, R. &. Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural relations in two-level data. British Journal of Mathematical and Sta tistical Psychology, 42, 215-232. McLachlan, G . .1, &. Krishnan, T. (1997). The EM algorithm and extensions. New York: .1ohn Wiley &. Sons. YlcLachlan, G. J. &. Peel, D. (2000). Finite mixture models. New York: Wiley &. Sons. }.iuthen, B. (1988). Some uses of structural equation modeling in validity studies: Ex tending IRT to external variables. In H. Wainer &. H. Braun (Eds.), Test validity (pp. 213-238), Hillsdale, N.1 : Erlbaum Associates. Muthen, B.(1989). Latent variable modeling in heterogeneous populations. Psychome trika, 54, 557-585. Muthen, B. (1990). }'lean and covariance structure analysis of hierarchical data. UCLA Statistics Series #62, August 1990. }'luthen, B. (1991). }'lultilevel factor analysis of class and student achievement compo nents. Journal of Educational Measurement, 28. 338-354. Muthen, B. (1994). covariance structure analysis. In .1. Hox &: I. Kreft (Eds.), Multilevel Modeling, a special issue of Sociological Methods E1 Research, 22, 376- I\luthen. B. (1997). Latent variable modeling with longitudinal and multilevel data. In A. Raftery (Ed.), Sociological Methodology (pp. 453-480). Boston: Blackwell Pub lishers. Muthen. B. (2001a). Second-generation structural equation modeling with a combina tion of categorical and continuous latent variables: New opportunities for latent class/latent growth modeling. In Collins, L. }I. & Sayer, A. (Eds.), New methods for the analysis of change (pp. 291-322). Washington, D.C.: APA. 116 Bengt O. Muthen Muthen, B. (2001b). Latent variable mixture modeling. In G. A. :\Iarcoulides & R. E. Schumacker (Eds.), New developments and techniq'ues in structural equation mod eling (pp. 1-33). :\1ahaw, KJ: Lawrence Erlbaum Associates. :\1 uthen, B. (2001 c). Two-part growth mixture modeling. Draft. University of California, Los Angeles. yluthen, B. & Brown. C. H. (2001). missing data in a general latent vari able framework. Draft. University of California, Los Angeles. :\luthen, B., Brown. C. H., :\lasyn. K, Jo, B .. Khoo, S. T .. Yang. C. c.. Wang. C. P., Kellam. S .. Carlin. J .. & Liao, J. (in press). General growth mixture modeling for randomized preventive interventions. Forthcoming in Biostatistics. :\luthen, B. & Curran. P. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framevmrk for analysis and power esti mation. Psychological Methods, 2, 371-402. :\luthen, B., Kao, C. F., & Burstein, L. (1991). Instructional sensitivity in mathematics achievement test items: Applications of a new IRT-based detection technique. Journal of Educational Measurement, 28, 1-22. :\1uthen. B. & Khoo. S. T. (1998). Longitudinal studies of achievement growth using la tent variable modeling. In John B. Cooney (Ed.), Latent Growth Curve Analysis, a special issue of Learning and Individual Differences. 10, 73-101. Muthen. B., Khoo, S. T., Francis, D .. & Kim Boscardin, C. (in press). Analysis of read ing skills development from Kindergarten through first grade: An application of growth mixture modeling to sequential processes. In S. R. Reise & N. Duan (Eds), Multilevel modeling: Methodoloqical advances, issues, and applications. Mahaw, K.1: Lawrence Erlbaum Associates. Muthen, B. & Masyn, K (2001). Discrete-time survival mixture analysis. University of California, Los Angeles. Muthen, B. & Muthen, L. (2000). Integrating person-centered and variable-centered analysis: Growth mixture modeling with latent trajectory classes. Alcoholism: Clinical and Experimental Research, 24, 882-891. Muthen, L. & I\luthen, B. (1998-2001). Mpz.ususer's guide. Los Angeles. CA: :\luthen & Nagin, D. S. (1999). Analyzing developmental trajectories: A semi-parametric, group based approach. Psychological Alethods, 4, 139-157. Nagin, D. S. & Land, K C. (1993). Age, criminal careers, and population heterogeneity: Specification and estimation of a nonparametric, mixed Poisson model. Crimino.l o.gy, 31. 327-362. Kagin, D. S. & Tremblay, R. E. (2001). Analyzing developmental trajectories of distinct but related behaviors: a group-based method. Psychological lv/ethods, 6, 18-34. Nestadt, G., Hanfelt. J.. Liang, K. Y., Lamacz. :\1., \Volyniec, P" & Pulver. A. E. (1994). An evaluation of the structure of schizophrenia spectrum personality dis orders. Journal o.f Personality Diso.rders. 8, 288-298. Pearson, K (1895). Contributions to the theory of mathematical evolution, II: skew vari ation. Philo.so.phical Tmnsactio.ns of the Royal Society o.f London A, 186. 343-414. Raudenbush, S. W. (2001). Toward a coherent framework for comparing trajectories of individual change. In Collins. L. :\1. &: Sayer. A. (Eels.), New methods for the analysis of change (pp. 35-64). vVashington. D.C.: APA. Reboussin, B. A .. Reboussin, D. ]'v1.. Liang. K. Y.. & Anthony, J. C. (1998). Latent transition modeling of progression of health-risk behavior. Multivariate Behavioml Research, 33, 457-478. BEYOND SEA!: GENERAL LATENT VARIABLE MODELING 117 Rindskopf, D. & Rindskopf, W. (1986). The value of latent class analysis in medical di agnosis. Statistics in Medicine, 5, 21-27. Takane, Y. & DeLeeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408. Uebersax, J. S., & Grove, W. I\f. (1990). Latent class analysis of diagnostic agreement. Statistics in Medicine, 9, 559-572. Yung, Y. F. (1997). Finite mixtures in confirmatory factor-analysis models. Psychome trika, 62. 297-330. Received December 2001. Final version accepted JanuarJ' 2002. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Com- mons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Behaviormetrika Springer Journals http://www.deepdyve.com/lp/springer-journals/beyond-sem-general-latent-variable-modeling-5ixO6rGn0I

Loading next page...

References (93)

J D Angrist, G W Imbens, D B Rubin (1996)
Identification of causal effects using instrumental variables
Journal of the American Statistical Association, 91
R. Bosker (1995)
Boekbespreking van "A.S. Bryk & S.W. Raudenbusch - Hierarchical linear models: Applications and data analysis methods" : Sage Publications, Newbury Parki, London/New Delhi 1992
, 62
B Muthén, A Raftery (1997)
Latent variable modeling with longitudinal and multilevel data
Sociological Methodology
C C Clogg, G Arminger, C C Clogg, M E Sobel (1995)
Latent class models
Handbook of statistical modeling for the social and behavioral sciences
(1991)
Instructional sensitivity in mathematics achievement test items: Applications of a new IRT-based detection technique
Kamel Jedidi, V. Ramaswamy, W. DeSarbo, M. Wedel (1996)
On Estimating Finite Mixtures of Multivariate Regression and Simultaneous Equation Models
Econometrics: Econometric & Statistical Methods - General eJournal
B. Muthén (2001)
Latent Variable Mixture Modeling
B. Muthén, S. Khoo, D. Francis, C. Boscardin (2002)
Analysis of reading skills development from kindergarten through first grade: An application of growth mixture modeling to sequential processes
(2001)
Non-ignorable missing data in a general latent variable framework
G. McLachlan, D. Peel (2000)
Finite Mixture Models
K. Jöreskog, A. Goldberger (1975)
Estimation of a Model with Multiple Indicators and Multiple Causes of a Single Latent Variable
Journal of the American Statistical Association, 70
L. Goodman (1974)
Exploratory latent structure analysis using both identifiable and unidentifiable models
Biometrika, 61
M. Browne, G. Arminger (1995)
Specification and Estimation of Mean- and Covariance-Structure Models
B. Muthén (1997)
10. Latent Variable Modeling of Longitudinal and Multilevel Data
Sociological Methodology, 27
K. Land (2001)
Introduction to the Special Issue on Finite Mixture Models
Sociological Methods & Research, 29
A. Formann (1992)
Linear Logistic Latent Class Analysis for Polytomous Data
Journal of the American Statistical Association, 87
G. Arminger, Petra Stein, J. Wittenberg (1999)
Mixtures of conditional mean- and covariance-structure models
Psychometrika, 64
S. Lipovetsky (2001)
Latent Variable Models and Factor Analysis
Technometrics, 43
B S Everitt, D J Hand (1981)
10.1007/978-94-009-5897-5
Finite mixture distributions
P. Heijden, J. Dessens, UIf Bockenholt (1996)
Estimating the Concomitant-Variable Latent-Class Model With the EM Algorithm
Journal of Educational Statistics, 21
E Blafield (1980)
Clustering of observations from finite mixtures with structural information
Unpublished doctoral dissertation, Jyväskyla studies in computer science, economics, and statistics, Jyväskyla, Finland
D. Nagin (1999)
Analyzing developmental trajectories: A semiparametric, group-based approach
Psychological Methods, 4
B. Muthén (2001)
Second-generation structural equation modeling with a combination of categorical and continuous latent variables: New opportunities for latent class–latent growth modeling.
D. Nagin, K. Land (1993)
AGE, CRIMINAL CAREERS, AND POPULATION HETEROGENEITY: SPECIFICATION AND ESTIMATION OF A NONPARAMETRIC, MIXED POISSON MODEL*
Criminology, 31
F. Krauss (1980)
Latent Structure Analysis
R. Little, Linda Yau (1998)
Statistical Techniques for Analyzing Data from Prevention Trials: Treatment of No-Shows Using Rubin's Causal Model
Psychological Methods, 3
C. Dolan, H.L.J. Maas (1998)
Fitting multivariage normal finite mixtures subject to structural equation modeling
Psychometrika, 63
B. Muthén, S. Khoo (1998)
LONGITUDINAL STUDIES OF ACHIEVEMENT GROWTH USING LATENT VARIABLE MODELING
Learning and Individual Differences, 10
Booil Jo (2002)
Statistical power in randomized intervention studies with noncompliance.
Psychological methods, 7 2
D. Follmann, D. Lambert (1989)
Generalizing Logistic Regression by Nonparametric Mixing
Journal of the American Statistical Association, 84
Bengt Muth¶en (2001)
Two-Part Growth Mixture Modeling
(2002)
Full-information maximum-likelihood estimation of general two-level latent variable models
T. Hoshino (2001)
BAYESIAN INFERENCE FOR FINITE MIXTURES IN CONFIRMATORY FACTOR ANALYSIS
Behaviormetrika, 28
(2001)
Final version accepted
D. Hedeker (1998)
The Natural History of Smoking : A Pattern-Mixture Random-e ® ects Regression Model
C. Clogg, L. Goodman (1985)
SIMULTANEOUS LATENT STRUCTURE ANALYSIS IN SEVERAL GROUPS
Sociological Methodology, 15
M. Lindstrom, D. Bates (1988)
Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data
Journal of the American Statistical Association, 83
Booil Jo (2002)
Model misspecification sensitivity analysis in estimating causal effects of interventions with non‐compliance
Statistics in Medicine, 21
S. Khoo, B. Muthén (2000)
Longitudinal data on families: Growth modeling alternatives
G. McLachlan, T. Krishnan (1996)
The EM algorithm and extensions
S. Raudenbush, A. Bryk (1992)
Hierarchical Linear Models: Applications and Data Analysis Methods
B. Muthén (1989)
Latent variable modeling in heterogeneous populations
Psychometrika, 54
Y. Takane, J. Leeuw (1987)
On the relationship between item response theory and factor analysis of discretized variables
Psychometrika, 52
Kamel Jedidi, Harsharanjeet Jagpal, W. DeSarbo (1997)
Finite-Mixture Structural Equation Models for Response-Based Segmentation and Unobserved Heterogeneity
ERN: Other Econometrics: Single Equation Models (Topic)
K. Widaman (2003)
New Methods for the Analysis of Change
Applied Psychological Measurement, 27
G Nestadt, J Hanfelt, K Y Liang, M Lamacz, P Wolyniec, A E Pulver (1994)
An evaluation of the structure of schizophrenia spectrum personality disorders
Journal of Personality Disorders, 8
B. Muthén (1986)
Some uses of structural equation modeling in validity studies: Extending IRT to external variables
S. Raudenbush (2001)
Toward a coherent framework for comparing trajectories of individual change.
(2001)
Global and local identihability of factor mixture models
Booil Jo (2002)
Estimation of Intervention Effects with Noncompliance: Alternative Model Specifications
Journal of Educational and Behavioral Statistics, 27
B. Muthén, L. Muthén (2000)
Integrating person-centered and variable-centered analyses: growth mixture modeling with latent trajectory classes.
Alcoholism, clinical and experimental research, 24 6
C. Dayton, G. Macready (1988)
Concomitant-Variable Latent-Class Models
Journal of the American Statistical Association, 83
J. Hogan, N. Laird (1997)
Mixture models for the joint distribution of repeated measures and event times.
Statistics in medicine, 16 1-3
N. Laird, J. Ware (1982)
Random-effects models for longitudinal data.
Biometrics, 38 4
R. Little, Yongxiao Wang (1996)
Pattern-mixture models for multivariate incomplete data with covariates.
Biometrics, 52 1
J. Angrist, G. Imbens, D. Rubin (1993)
Identification of Causal Effects Using Instrumental Variables
Fuzhong Li, Terry Duncan, S. Duncan, A. Acock (2001)
Latent Growth Modeling of Longitudinal Data: A Finite Growth Mixture Modeling Approach
Structural Equation Modeling: A Multidisciplinary Journal, 8
R. Heck (2001)
Multilevel Modeling With SEM
G. Marcoulides, R. Schumacker (2001)
New developments and techniques in structural equation modeling
W. Gibson (1959)
Three multivariate models: Factor analysis, latent structure analysis, and latent profile analysis
Psychometrika, 24
B Jo (2001)
Model misspecification sensitivity analysis in estimating causal effects of interventions with noncompliance
Accepted for publication in Statistics in Medicine
Beth Reboussin, D. Reboussin, Kung Liang, James Anthony (1998)
Latent Transition Modeling of Progression of Health-Risk Behavior.
Multivariate behavioral research, 33 4
G. Arminger, Petra Stein (1997)
Finite Mixtures of Covariance Structure Models with Regressors
Sociological Methods & Research, 26
John Graham, Linda Collins, S. Wugalter, N. Chung, William Hansen (1991)
Modeling transitions in latent stage-sequential processes: a substance use prevention example.
Journal of consulting and clinical psychology, 59 1
J. Vermunt (2004)
Latent Class Models
Psycho-oncology
Booil Jo, B. Muthén (2001)
Modeling of Intervention Effects With Noncompliance: A Latent Variable Approach for Randomized Trials
K. Bandeen-Roche, D. Miglioretti, S. Zeger, P. Rathouz (1997)
Latent Variable Regression for Multiple Discrete Outcomes
Journal of the American Statistical Association, 92
(1980)
Clustering of observations from finite mixtures with structural information . Unpublished doctoral dissertation, Jyväskyla studies in computer science, economics, and statistics
K A Bollen (1989)
10.1002/9781118619179
Structural equations with, latent variables
B Jo (2001)
Statistical power in randomized intervention studies with noncompliance
Forthcoming in Psychological Methods
B. Muthén (1991)
Multilevel Factor Analysis of Class and Student Achievement Components
(1998)
Mplus user's guide
Y. Yung (1997)
Finite mixtures in confirmatory factor-analysis models
Psychometrika, 62
D. Rindskopf, W. Rindskopf (1986)
The value of latent class analysis in medical diagnosis.
Statistics in medicine, 5 1
R. Jennrich, Mark Schluchter (1986)
Unbalanced repeated-measures models with structured covariance matrices.
Biometrics, 42 4
G. Nestadt, J. Hanfelt, K. Liang, M. Lamacz, P. Wolyniec, A. Pulver (1993)
An evaluation of the structure of schizophrenia spectrum personality disorders
Schizophrenia Research, 9
D. Kandel, K. Yamaguchi, Kevin Chen (1992)
Stages of progression in drug involvement from adolescence to adulthood: further evidence for the gateway theory.
Journal of studies on alcohol, 53 5
B. Muthén, Chih-Fen Kao, L. Burstein (1991)
Instructionally Sensitive Psychometrics: Application of a New IRT‐Based Detection Technique to Mathematics Achievement Test Items
Journal of Educational Measurement, 28
B. Muthén, P. Curran (1997)
General Longitudinal Modeling of Individual Differences in Experimental Designs: A Latent Variable Framework for Analysis and Power Estimation
Psychological Methods, 2
L. Collins, S. Wugalter (1992)
Latent Class Models for Stage-Sequential Dynamic Latent Variables
Multivariate Behavioral Research, 27
J. Uebersax, W. Grove (1990)
Latent class analysis of diagnostic agreement.
Statistics in medicine, 9 5
L. Collins, J. Graham, S. Rousculp, W. Hansen, K. Bryant, M. Windle, S. West (1997)
Heavy caffeine use and the beginning of the substance use onset process: An illustration of latent transition analysis.
K. Pearson
Contributions to the Mathematical Theory of Evolution. II. Skew Variation in Homogeneous Material
Philosophical Transactions of the Royal Society A, 186
C. Frangakis, S. Baker (2001)
Compliance Subsampling Designs for Comparative Research: Estimation and Optimal Planning
Biometrics, 57
A. Cohen, B. Everitt, D. Hand (1982)
Finite Mixture Distributions
Technometrics, 24
B. Muthén, C. Brown, Katherine Masyn, Booil Jo, S. Khoo, Chih-Chien Yang, Chen-Pin Wang, S. Kellam, J. Carlin, J. Liao (2001)
General growth mixture modeling for randomized preventive interventions.
Biostatistics, 3 4
D. Hosmer (1973)
A Comparison of Iterative Maximum Likelihood Estimates of the Parameters of a Mixture of Two Normal Distributions Under Three Different Types of Sample
Biometrics, 29
R. Stansfield (1983)
Sociological Methodology 1982
Journal of the Operational Research Society, 34
R. Maccallum, James Austin (2000)
Applications of structural equation modeling in psychological research.
Annual review of psychology, 51
B. Muthén (1994)
Multilevel Covariance Structure Analysis
Sociological Methods & Research, 22
D. Nagin, R. Tremblay (2001)
Analyzing developmental trajectories of distinct but related behaviors: a group-based method.
Psychological methods, 6 1
R. Mcdonald, H. Goldstein (1989)
Balanced versus unbalanced designs for linear structural relations in two‐level data
British Journal of Mathematical and Statistical Psychology, 42
D. Kaplan, P. Elliott (1997)
A didactic example of multilevel structural equation modeling applicable to the study of organizations
Structural Equation Modeling, 4

Publisher: Springer Journals
Copyright: Copyright © The Author(s) 2002. corrected publication 2021
Subject: Statistics; Statistical Theory and Methods; Statistics for Business, Management, Economics, Finance, Insurance; Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences
ISSN: 0385-7417
eISSN: 1349-6964
DOI: 10.2333/bhmk.29.81
Publisher site: See Article on Publisher Site

Abstract

[kIIH, -i(lrlllN film V .. 1. 29. IIi". !. 2002. XI· I17 © The Author(s) 2002, corrected publication 2021 Bengt. O. Muthen* This article gives an overview of statistical analysis with latent variables. Us ing traditional tructural s equation modeling as startin a g point, it shows how the idea of latent variables captures a wide variety of statistical oncepts. c including random effects. missing data, sources of v ariation in hierarchical data. finite mix tures. latent classes. and clusters. These latent variable applications g o eyo b nd the traditional l atnt e variable se u age in psychometrics with its focus on measure ment error nd a hypothetical constructs measured by multiple indicators. The article argues for the value of integrating statistical and psychollletric modeling ideas. Different applications a re discussed in a unifying framework that brings together in one general model uch s different analysis types as factor models, growth curve models. lllultilevel models. latent class models and discrete-time survival models. Several possible combinations and extensions of these models are made clear due to the unifying framework. 1. Introduction This art icle gives a brief overview of statistical analysis wit h latent variables. A key feature is that well-known modeling with continuous latent variables is ex panded by adding new developments also including categorical latent variables. Taking traditional tructur s al qu e ation modeling as a st arting point, the article shows the generality of latent variables. being able t.o capture a wide variety of st atistical concepts, including random effects, missing data, sources o f variation in hierarchical data, finite mixtures, latent classes, and clusters. These latent variable applications go beyond the traditional latent variable use age in psycho metrics with its focus on measurement error and hypothetical onstruct c s measured by multiple indicators. The article does not discuss estimation nd a testing but focuses on m odeling ideas and connections between different modeling tradit ions. A few key applica tions will be discussed briefly. Although not going into details. the presentation is statistically-oriented . For ess l technical overviews and further applications of new d velopmen e ts using categorical latent variables, see. e.g .. 1IuthCn (2001a. b) and Muthen and :tvluthen (2000). All analyses are performed using the T\Iplus program (MutMn & MutMn, 1998-2001) and Mplus input, output. and data for these examples are available at www.statmodel.com/ mplus/examples/ penn.html. One aim of the rticl a e is tin o spire a b tt e er integration of psychometric mod eling ideas into mainstream statistics and a better use of statistical analysis ideas in latent variable modeling. Psychometrics and statistics have for too long been Key Words and Phrases : \Iaximum-likelihood estimation. factor anavs l is. tru s ctural eq ua t ion modeling, growt h urve c modeling, multilevel m odling e , finite mixture modeling, random effects. missing data, latent classes. * University o f California, Los Angeles. Graduate School of Education &: Information Studies. Moore Hall. Box 9.5 1521 , Los Angeles C A 90095-1.521, U.S.A. Email: hmut hen ©:ucla.edu 82 Bengt O. I\Iuthen developed too separately and both fields can benefit from input from the ot.her. Traditionally, psychometric models have been concerned with measurement error and latent variable constructs measured with multiple indicators as in factor anal ysis. Structural equation modeling (SE1\,1) took factor analysis one step further by relating the con8truct8 to each other and to covariates in a system of linear regressions thereby purging the "structural regressions" of biasing effects of mea surement error. The idea of using systems of linear regressions emanated from supply and demand modeling in econometrics and path analysis in biology. In this way. SEl'd consists of two ideas: latent variables and joint analysis of systems of equations. It is argued here that it is the latent variable idea that is more powerful and more generalizable. Despite its widespread use among applied re searchers. SEl\I has still not been fully accepted in mainstream statistics. Part of this is perhaps due to poor applications claiming the establishment of causal models and part is perhaps also due to strong reliance on latent variables that arc only indirectly defined. The skepticism about latent variables is unfortunate given that, as shown in this article, latent variables are widely used in statistics. although under different names and different forms. This article argues that by emphasizing the vehicle of latent variables. psy chometric modeling such as SEM can be brought into mainstream statistics. To accomplish this, it is necessary to dearly show how many statistical analyses implicitly utilize the idea of latent variables in the form of random effects. compo nents of variation. missing data. mixture components, and clusters. To this aim. a general model is discussed which integrates psychometric latent variable models with latent variable models presented in the statistical literature. The generality of the model is achieved by considering not only continuous latent variables but also categorical latent variables. This makes it possible to unify and to extend a \vide variety of common types of analyses, including SEM. growth curve modeling, multilevel modeling, missing data modeling. finite mixture modeling, latent class modeling. and survival modeling. The general model is shown schematically in Figure 1. The general framework (D) is represented by the square. while special cases (A. B. C) to be discussed in the article are shown in ellipses. The gen eraJ framework is drawn from ]\Iuthen and l'vIuthen (1998-2001; Appendix 8) as implemented in the Mplus computer program (www.statmodel.com). It should be noted that Figure 1 is a simplification. For example. the general framework includes direct effects from c to u. from c to y. and allows c to also influence regression and variance parameters in the u and y parts of the model. It is hoped that the use of a single modeling and software framework makes latent variable modeling more accessible to both statisticians and substantive researchers. Statis ticians can more easily see connections between latent variable uses that they are accustomed to and psychometric uses. Substantive researchers can more easily focus on the research problem at hand rather than learning a multitude of model specification systems and software languages. The article is structured as follows. Section 2 discusses framework A of the BEYOI\D SEM: VARIABLE 83 Figure 1: A general latent variable modeling framework general model. This framework corresponds to the more well-known case of con tinuous latent variables. Sub-sections discuss the modeling of measurement error and measurement invariance in convcntional SEM, random effects in growth mod eling, and variance components in multilevel modeling. Section 3 discusses frame work B introducing categorical latent variables, including latent class analysis and latent class growth analysis. A latent class analysis example is presented where individuals are classified based OIl their antisocial behavior. Section 4 discusses framework C, including latent profilc models and models that combine continuous and categorical latent variables such as growth mixture models. A growth mix ture example is presented where children are classified into a problematic class based on their reading development in Kindergarten and second grade. Section 5 discusses the general framework D, presenting new types of models, including modeling with missing data on a categorical latent variable in randomized trials. Section 6 concludes. 2. Modeling Framework A: Continuous Latent Variables Consider the special case A of the general modeling framework shown in Figure 1. Framework A is characterized by using continuous latent variables, denoted by the vector 77, shown as a circle in ellipse A in Figure 1. Here, latent variables are used to represent constructs that have fundamental substantive importance but are only measured indirectly through multiple indicators that capture different 84 Bengt O. Muthen aspects of the constructs. As a first step, a general SEM formulation of framework A is presented, fol lowed by the key analysis areas of random effects modeling and variance compo nent modeling. 1) The measurement part of the model is defined in terms of the p-dimensional continuous outcome vector y. (1) Yi = 1/ + A "1i + K Xi + ci· where "1 is an m-dimensional vector of latent variables, x is a q-dimensional vector of covariates. f. is a p-dimensional vector of residuals or measurement errors which are uncorrelated with other variables, 1/ is a p-dimensional paramet.er vector of measurement intercepts, A is a p x Tn parameter matrix of measurement slopes or factor loadings, and K is a p x q parameter matrix of regression slopes. Usually, only a few of the elements of K have nonzero elements, where a non-zero row corresponds to a y variable that is directly influenced by one or more x variables. The covariance matrix of f. is denoted e. The structural part of the model is defined in terms of the latent variables regressed on each other and the q dimensional vector x of independent variables, (2) Here, a is an Tn-dimensional parameter vector, B is an Tn x Tn parameter matrix of slopes for regressions of latent variables on other latent variables. B has zero diagonal elements and it is assumed that I - B is non-singular. Furthermore, r is an Tn x q slope parameter matrix for regressions of the latent variables on the independent variables. and ( is an m-dimensional vector of residuals. The covariance matrix of ( i s denoted q,.. In line with regression analysis, the marginal distribution of x is not modelled bllt is left unrestricted. This leads to the mean and covariance structures conditional on x. 1/ + A (I -B)-1 a + A (I - B)-1 r x + K x. (3) A (I - B )-1 q,. (I - B),-1 A' + e. (4) \Vith the customary normality assumption of y given x , the parameters of the model are estimated by fitting (3) and (4) to the corresponding sample quantities. This is the same as fitting the man e vector and covariance matrix for the vector (y,x), to the sample means, variances, and covariances for (y, x), (Joreskog & Goldberger, 1975). Here, the ma..ximum-likelihood estimates of J..L and :En are the corresponding sample quantities. examples of framework A models are given at www.statmodel.com/mplus/examples/ continuous.html. BEYOND SKTl.l: GENERAL LATENT VARIABLE MODELING 85 Joint analysis of independent samples from multiple groups is also possible, a,.'lsuming different degrees of parameter invariance across groups. In particular, full or partial invariance of the measurement parameters of v and A is of interest in order to study group differences with respect to 0: and \[I. From an application point of view, the modeling in (1), (2) is useful for purging regression relationships of detrimental effects of measurement error when multiple indicators of a construct are available. I\Ieasurement errors among the predictors are well-known to have particularly serious effects. but the modeling is also useful in examining a factor model where the measurement errors are among the outcome (indicator) variables as when using a factor analysis with covariates C'I\HI\IIC" modeling). In this special case, B = 0 in (2). A baseline MIMIC analysis assumes K = 0 in (1) and a sufficient number of restrictions on A and \[I to make the model identified (in an exploratory analysis, this amounts to using rn restrictions in line with exploratory factor analysis). The covariates strengthen the factor analysis in two ways (d. Muthen. 1989). First, by making the test of dimensionality stronger by using associations not only among the y variables but also between y and x. Second, by making it possible to examine the extent of measurement invariance across groups defined by different values on x. Measurement non-invariance across groups defined by Xk (e.g. Xki = 0/1 for individual i) with respect to an outcome Yj is captured by Kjk -I- 0, reflecting a group-varying intercept. Vj + Kjk Xk. The model of (1) - (4) is typically estimated by maximum-likelihood (ML) un der the assumption of multivariate normality. Browne and Arminger (1995) give an excellent summary of modeling and estimation issues for this model. This is the analysis framework used for the last 20 years by conventional SEM computer programs such as A1\10S, EQS, and LISREL. More recently, ML estimation as suming missing at random (MAR) in the sense of Little and Rubin (1987) has been introduced in SEM software. Browne and Arminger (1995) also discuss the case where some or all of the y outcomes are categorical. The case of categorical outcomes has been further treated in Muthen (1984. 1989) with an emphasis on weighted least-squares es timation, including a new approach presented in Muthen, DuToit. Spisic (1997). Mplus includes modeling with both continuous and categorical outcomes y.2) Drawing on I\Iuthen (1996) and Muthen and Christofferson (1981). I\Iplus pro vides a lIlore flexible parameterization than conventional SEM software in terms of its categorical outcome modeling for longitudinal data and multiple-group analysis using threshold measurement parameters that allow for partial invariance across time and group. For connections ''lith item response theory, sec. e.g .. I\Iuthen (1988). IvIuthen. Kao and Burstein (1991). and Takane and DeLeeuw (1987). For an overview of conventional SEM with continuous outcollles, see, e.g. Bollen (1989). For examples of SEI\I analysis in behavioral research. see. e.g .. examples are given at www.statmodel.com/mplus/examples/categorical.html. Sf) Bengt O . .\Iuthen .MacCallum and Austin (2000). 2.1 Random effects growth modeling The use of random effects is another example of modeling with continuous latent variables. In mainstream statistics, random effects are used to capture unobserved heterogeneity among subjects. That is, individuals differ in systematic ways that cannot be. or at least have not been. measured. Unlike the case of psychometric latent variable contexts. however. the random effects are typically not thought of as constructs of primary interest. and there is typically not an attempt at directly measuring the random effects. 2.1.1 A growth modeling example Growth modeling is an interesting example of random effect modeling where the heterogeneity concerns individual differences in trajectories. Consider an ex ample from reading research. The data are from a cohort-sequential reading study of 945 children in a sample of Texas schools, following them from Kindergarten through second grade. In Kindergarten a phonemic awareness score was mea sured as a reading precursor skill. In grades 1 and 2, word recognition scores were collected. All measures were collected at four times during the school year. At the end of grade 2, standardized reading and spelling scores were also recorded. These data will also be used to illustrate growth mixture modeling in framework C. Figure 2 shows observed individual trajectories on a phonemic awareness score for Kindergarten children divided into the upper and lower decile on the grade 2 spelling score. The individual variation in the trajectories is clearly seen with those in the lower spelling decile showing a lower initial and ending status 1Il Kindergarten and a lower growth rate than those in the upper spelling decile. 2.1.2 Alodel,tng issnes A modeling example shows the latent variable connections. Let the random variables 7)0, 7)1. and 7)2 represent an intercept, a linear. and a quadratic slope. respectively. These are coefficients in the regression of the outcome on time and the fact that they vary across individuals gives rise to the term random coeffi cients or random effects. The random effects capture the individual differences in development over time using the Laird and \Vare (1982) type of model Yit = Tloi + T)l; (a/ - a) + T)'2i (at - a)2 + ""t :rit + fit, (5) Tloi = eto + XiO + (Oi. (6) (7) Tili = (tl + :1::;0 + (ii, (8) '72i = et2 + {2 :1::iO + (21' where at is a time-related variable, a is centering constant, Xt is a time-varying covariate. and Xo is a time-invariant covariate. In multilevel terms (see, e.g .. Bryk BEYOND SEfl'l: GENERAL LATENT VARIABLE 87 Upper Decile 2nd Grade Spellers ........... :-: .... ........ :::. Lower Decile 2nd Grade Spellers ... ..... . ....... .......... . ...... __ ..... _-............ ::::::: . ..-.-::::::::::: . ..-.-_.- ':::::;:: Figure 2: Phonemic awareness development in Kindergarten & Raudenbush, 2002), (5) is referred to as the level 1 equation, while (6) - (8) are referred to as level 2 equations. In mixed linear modeling (see, e.g. Jennrich & Sluchter, 986; 1 Lindstrom & Bates, 1988; Goldstein, 1995), the model is expressed in terms of Yt related to at, X t , and :z:o, inserting (6) - (8) into (5) . It is clear that (5) - (8) can be expressed in SEM terms using ( 1) and (2) by let t.ing Yi = (Yi l ,Yi2, ... , YiT )', 'lJi = (r}Ol , l]li,T}2i)' , and X i = (xil, :ri.2: . . . :XiT,l:iO)' . 'While multilevel modeling views the analysis as a two-level analysis of a univariate outcome ;tI, the SEM approach is a single-level analysis of the multivariate vector y. This issue will be discussed further in Section 2.2. Furthermore, v = 0 while 0: contains three free parameters. Alternatively, the equivalent parameterization 88 Bengt O. VI = v2 = ... = VT with Q = (0.0:1,0'2)' may be used. Also. a1 - a (aj - aJ' ) a2 - a (a2 - a)2 , (9) h( ! aT - a (at - 0,)2 showing that growth modeling with random effects is a form of factor analysis with covariates. Time-invariant covariates have effects on the factors and time-varying covariates have direct ff eects on the outcomes. \Vhile not typically thought of as such. the repeated measures of YI , Y2 ,' .. ,YT can be seen as multiple indicators o f t he random effects, or growth factors as they are referred to in the latent variable literature. One may wonder why a latent variable model with such a restricted A matrix as in (9) , with no free parameters. would ever be realistic. But it has been found that this model often captures the essential feat ures of growth. In the latent variable framework however. it is easy to allow deviations from the functional growth form by estimating some of the oa l dings. Multilevel and mixed linear modeling traditions consider a more general form of (5), (10) Yit = 170i + 17li (ail - a) + 172i (ait - a) + " .j Xi I + f i l , where ail indicat.es the possibility of individually-varying times of observation and the slope Ki is yet another random effect. These traditions treat (ait - a) as data, whereas conventional SEM software treats (at - a) as parameters. This is the only way that random slopes can be handled in conventional SE:t'vI. In (10), 17li . 172i' and Ki are random slopes for individually-varying variables a and x. As pointed out in Raudenbush (2001) such modeling cannot be summarized in terms of mean nd a covariance structures. Unlike (4), the variance of y conditional on the a and x: variables varies as a function of these variables. In principle, however. this points to a hort s coming of conventional SEM software, not a shortcoming of latent variable modeling. Drawing on Asparouhov and 1Iuthen (2002), Mplus incorporates individuallY'-varying times of observations and random slopes for time-varying covariates as in (10) .:3) rvlainstream statistics aso l takes an interest in 'what psychometricians call fac tor scores, i.e. estimates of 'rJi values to be used for estimation of individual growth curves. Both fields favor empirical Bayes estimates. efe r rred to as the regression method in psychometrics. It follows that there are several advantages of placing the growth model in a latent variable context. For example. t.he psychometric idea of a latent variable construct is not ut.ilized in t he growth model of (5) - (8). Although the y out comes are manifestations o f growth. a psychometric approach ould c in principle 3)These features are included in Version 2.1 to be released Spri ng 2002 as a free upgrade for Version 2 users. BEYOND SEk1: GENERAL LATENT VARIABLE MODELING 89 seek specific indicators of the growth factors, for instance measuring indicators of growth potential at the outset of the study. all approach that does not seem to have been pursued. A more common situation is that a esea r rcher wants to study grO\\Tth in a latent variable construct measured with multiple indicators. The model specification is as follows, for simplicity shown for a linear model with a single latent variable construct 1}it. Let Yijt denote the outcome for individual i. indicator j. and timepoint t, and let Tht denote a latent variable construct, Level-1 a (measurement part) : (11 ) Yijt = Vj t + Aj t 1}i t + fijt , Level-1 b : llit = 1}Oi + 1]1; at + (it, (12) (13) Level-2a : 1]0; = no + :0 Xi + (Oi, (14) Level-2b : 'IIi = 0:1 + {I Xi + (Ii· In line with the second parameterization given above for a ingle s outcome, measurement invariance is specified by using time-invariant indicator intercepts and slopes: (15) Vjl = Vj2 = ... VjT = = Vj , (16) A 1 = Aj2 = ... = AjT = A]' setting the metric of the latent variable construct by Al = 1. The intercept of the level-2a equation is fixed at zero, 0:0 = O. V(Eijd and V'((it) may vary over time. Structural differences are captured by letting E(1]id and V(rJit ) vary over time. \Vith more than one population. across-population measurement invariance would be imposed and 0:0 fixed to zero only in the first population. Multiple-indicator growth modeling has the advantage t hat changes ill measurements can be made over time, assuming measurement invariance for a subset of indicators that are maintained between adjacent time points. Other advantages of growth modeling in a latent variable framework includes the ease with which to carry out analysis of multiple processes, both parallel in time and sequential, as well as multiple groups. Growth factors may bregressed e on each other using the B matrix in (2), for example studying growth while controlling for not only observed c:ovariates but also initial status. 110re generally, the growth model may be only a part of a larger modeL including for instance a factor analysis measurement part for covariates measured with errors. or including a m diational e path analysis part for variable influencing the growth factors, or includiug a set of variable that are influenced b y the grmvth process. For examples of growth modeling in a atent l variable framework, see, c.g., Muthen and Khoo (1998) and Muthen and Curran (1997). The ece r nt Collins and Sayer (2001) book gives applied contributions from several different traditions. 90 Hengt O. l\[ut.hen 2.2 Component8 of variation in hierarchical data Latent continuous variables are frequently used in statistical modeling of hi erarchical data. Here. latent variables are used to correctly reflect the sampling procedure with latent variables representing sources of variation at different levels of the hierarchy. It is instructive to consider a simple ANOVA model because it clearly shows re lationships between factor analysis. growth modeling. and more general multilevel latent variable models. Consider the nested. randolll-effects ANOVA, (17) Yij = V + 'Ii + fij :i = L 2 ..... n : j = L 2 .... , J. Here. i is the mode of variation for which an independent sample is obtained. while j is clustered within i. Typical examples are individuals observed \vithin households and students observed within classrooms. The different sources of variation are captured by the latent variables 1] and f. If instead j = 1,2, ... , ni, there is missing data on some of the J measures. Consider the covariance and variances for j = k and j = l, (18) COV(Yik: Yil) = 1'(1]). (19) V(Yik) = v(Yid = V(17) + VeE), resulting in the intraclass correlation (20) The intraclass correlation is frequently considered in the context of cluster sam pling. The intraclass correlation increases for increasing between-cluster variation VC'I) relative to total variation. Or. using equivalent homogeneity reasoning, the intraclass correlation increases when the within-cluster variation V(E) is small. In cluster samples. the intradass correlation is used to describe the lack of in dependence among observations and llsed when computing design effects. The simple model of (17) summarizes some key latent variable modeling issues in a nutshelL showing that factor analysis, growth modeling, and multilevel modeling are variations on the same theme. I t is clear that (17) can be seen as a special case of factor analysis in the SEl\I framework of (1). (2) with a single factor and A = (L 1. ... , I)'. Instead of thinking of the J units within each cluster as individuals as in (17), the J Y variables are now multiple indicators measured on the same individuaL Carrying this idea back to (17). this means that the individuals within a cluster can be seen as indicators measuring cluster characteristics. \Vhen Yj are repeated measures over time, j = t. (17) represents a growth model with random intercepts. The repeated measures take the role of multiple indicators measuring the random intercept growth factor. For example, t.he model BEYOND SEAl: GE:\EHAL LATE:'\[T VARIABLE 91 may represent blood pressure measurements on individuals where in a short time span there is no increasing or decreasing trend. The focus is on the construct of "long-term blood pressure level", e.g. for predicting later health outcomes. The E residuals represent measurement error as well &'3 time-specific variation, both of which may bc irrelevant for the prediction. It was noted earlier that growth modeling in the SEM framework leads to single-level analysis because a multivariate analysis of y is carried out. The non independence among repeated measures within an individual indicated by the intraclass correlations is modeled by the growth factors influencing the outcome at difl'erent time points. This i::; analogous to factor analy::;is. The same multi variate analysis approach may be used for more general multilevel modeling with latent variables, for example multilevel factor analysis and multilevel growth mod eling, referred to as 3-levelmodeling in the multilevel literature. The multivariate approach is suitable for situations where there are relatively few cluster members, such as with analysis of spouses. siblings. or analysis of twins in behavioral genet ics. For a recent application to growth modeling of alcohol use among siblings, see Khoo and Muthen (2000). The multivariate approach provides very flexible modeling where the relationships among units within a cluster can be modeled. In Khoo and t..Iuthen (2000) the growth factors of a younger sibling are regressed on those of an older sibling. The cluster units can also have different regressions on covariates. Different numbers of cluster units for different clusters can be han dled via missing data. although different models may be relevant for clusters of different size (i.e. two-sibling homes may have a different dynamics than homes with many siblings). With more than a couple of cluster units, however, the multivariate approach becomes computationally cumbersome. For instance with 10 measures per student with 15 students per cla.ssrooms, a multivariate vector of length 150 would have to be analyzed. As an alternative, multilevel modeling makes a simplifying assumption of cluster units being statistically equivalent as shown below. Assume c = L 2 ..... C independently observed clusters with i = L 2, ... , rlc indi vidual observations within duster c. Let z and y represent group- and individual level variables. Arrange the data vector for which independent observations are obtained as ' (' " ') c = Zc, Y cl , Y c2 , ... , Yen ' where we note that the length of de varies across clusters. The mean vector and covariance matrix are (21 ) [JLz', Inc' g JLy'] symmetric ] (22) Ineg + Inc g . The covariance matrix shows that the usual i.i.d assumption of simple random sampling is modified to allow for non-independent observations within clusters and that this non-independence is modeled by the matrix in line with the nested Bengt O. ]'viuthen ANOVA model of (17). In (21) and (22) the sizes of the arrays arc determined by the product nc x p where p is the number of observed variables. l\IcDonald and Goldstein (1989) pointed out that a great reduction in size is obtainable, reducing the ML expression {In I I + (de - J.LdJ' -1 (de - J.LdJ} c=1 to L Cd {In I I + tr[ -1 (SBd + nd (Vd - J.L)(Vd - J.L)')]} where d sums over clusters with distinct cluster sizes (for details, see Muthen, 1990). MutMn (1989, 1990. 1994) showed how SEM soft\'.rare can be used for analyzing models of this type. This is referred to ili'i 2-level modeling in a latent variable framework. Here, J.L, and are structured in terms of SEM parameter arrays based on (1) and (2). The analysis can be carried out using the Mplus program. In multilevel terms. this type of model may be viewed as a random intercept model ill line with (17) because of the additivity = + As mentioned earlier, the inclusion of random slopes leads to models that cannot be summarized in terms of mean and covariance structures as done above (Raudenbush. 2001). Nevertheless, random slopes can be incorporated into 2-levellatent variable mod eling (see Asparouhov &= l\Iuthen, 2002). 3-level modeling is also included in the framework of (21) and (22) when one of the levels can be handled by a multivariate representation. as in the case of growth modeling in line with Section 2.1. Latent variable growth modeling in cluster samples is discussed in MutMn (1997). For examples. see, e.g., Muthen (1991) with an application of multilevel factor analysis and MutMn (1989) with an application to SEl\1. Further examples are given in Hecht (20(H) and Kaplan and Elliott (1997). 3. Modeling Framework B Consider next the special case B of the general modeling framework shown in Figure 1. Framework B is characterized h;v using categorical latent variables. denoted by the circle c in Figure 1 (the circle denoted TJu will be discussed later on). The choice of using a categorical latent variable instead of a continuous latent variable is lllore fundamental than the corresponding choice of proper scale type for observed outcomes. The addition of categorical latent variables to the general framework in Figure 1 opens up a whole new set of modeling capabilities. BEYOND SEJ\I: GENERAL LAT ENT VARIABLE MODELING 93 In mainstream statistics, this type of modeling is referred to as finite mixture modeling. In the current article, the terms latent class and mixture modeling will be uoed interchangeably. As with continuouo latent variables, categorical latent variables are used for a variety of reasons as will now be shown. As a first step, a general modeling representation of framework B as used in I'vlplus (Muthen & l\Iutben, 1998-2001) is presented. This is followed by a discussion of four special cases: latent cl i::1SS analysis. latent class analysis with covariates, latent class growth analysis, latent transition analysis. and logistic re gression mixture analysis. :'Iethodological contributions to these areas have been made in separate fields often without sufficient connections and without sufficient connections to modeling in other frameworks. For example, until recently, model ing developments for continuous latent variables in framework A and categorical latent variables in framework B have been kept almost completely separate. ) Let C denote a latent categorical variable with K classes, Ci = (Cd, Cn , ... , ciK )'. where Cik = 1 if individual i belongs to class k and zero otherwise. Framework B has two parts: C related to x and u related to C and x. c is related to x by multinomial logistic regression using the K - I-dimensional parameter vector of logit intercepts etc and the (K - 1) x q parameter matrix of logit slopes r c : where for k = 1, 2, .. ., K X i e k P (Cik = llxi) = -----,-:""7, --:-::- , (23) "K eQc) +/c] Xi ' LJ=l where the last class is a reference class with coefficients standardized to zero, O:CK = 0, / Ck = O. For u , conditional independence is assumed given Ci and Xi, The categorical variable Uij (j = 1, 2, . .. , r) with Sj ordered categories follows an ordered polytomous logistic eg r ression (proport.ional odds model) , where for categories s = O. L 2, .. . , Sj - 1 and T),k,O = -00, Tj ,k,S = X , (25) Uij = s, i f Tj,k, s < ui :S Tj,k.s + l , (26) P(UiJ = SICi,xd = Fs+ 1('uT ) - Fs(ui ), j j F (u*) - __ 1,..-----c::7 (27) S - 1 + e-(Ts-U*)' where for ui = (uil,ui2,··· ,uir)', T7ui = C'/Ul i , ''lU2, , ···,T/Uj,)'' and conditional on class k, (28) u; = A Uk T7ui + KUk X i, (29) T7ui = Q + r X i, Uk Uk 4) Mplus examples of framework B m odels are given at www.statmodel.com/mplus/examples/mixture.html. 94 Bengt O. :'I'luthen where A ' is an l' x f logit parameter matrix varying across the K classes, K Uk uk is an r x q logit parameter matrix y-arying across the K classes, au,. is an f x 1 vector logit parameter vector varying across the J( classes, and r Uk is an f x q logit parameter matrix varying across the K classes. The thresholds may be stacked in the LJ=l (5 - 1) x 1 ec v tors Tk varying across the K classes. It should he noted that (28) does not include intercept terms given the presence of T parameters. Furthermore. T parameters haye opposite signs thanu* in (28) because of their interpretation as thresholds or cut points that a latent continuous response variable u' exceeds or falls below (see also Agresti. 1990. pp. For example. with a binary 1.[ scored Oj 1 (26) leads to . 1 (30) P(u.=llc.x )=l- ' )' . 1 + e-\T - U' (31) For example. the higher the T the higher 1.[* needs to be to exceed it, and the lower the probability of n = 1. Mixture modeling can involve Ilumerical and statistical problems. Mixture modeling is known to somet imes generate a likelihood function with several local maxima. The occurrence of this depends on the model and the data. It is therefore recommended that for a given dataset and a given model different optimizations are carried out using different sets of starting values. The numerical and statistical performance of mixture modeling benefits from confirmatory analysis. The same kind of confirmatory a n alysis as in regular mod eling is possible. using a priori restrictions on the parameters. \Vith mixture mod eling, however, there is also a second type of confirmatory analysis. A researcher may \'lant to incorporate the hypothesis that certain individuals are known to represent certain latent classes. Individuals with known class membership are referred to a!'; training data (see also ).Ic:Lachlan & Basford. 1988: Hosmer, IVIultiple-grouJ> modeling corresponds to the case of all sample units contributing training data so that c is in effect an observed categorical variable. In l\lplus. the training data can consists of 0 and 1 class membership values for all individuals. where 1 denotes which classes an individual may belong to. Known class membership for an individual corresponds to having training data value of 1 for the known class and 0 for all other classes. Unknown class membership for an individual is specified by the value 1 for all classes. \Vit h class membership training data. the cla..<;s probabilities are renormed for each individual to add to one over the admissible set of classes. Fractional training data is also allO\ved, corresponding to class probabilities adding to unity for each individual. \Vith fractional training data. the class probabilities are taken to be fixed quantities, which reduces the sampling variabilit:,,' accounted for in the standard error cal culations. Fractional training data where each individual has a probability of 1 for one class and ()'s for the other classes is equivalent to training data with cla..ss membership value 1 for only one class for each individual. Using training data BEYOND SET\£: GEKERAL LATENT VARIABLE MODELl KG 95 with a value of 1 for one class and (rs for the other classes makes it possible to perform multinomial logistic regression with an unordered, polytomous observed dependent variable using the l\lplus model part where c is related to x. S.l Latent class analysis In latent class analysis the categorical latent variable is used to represent unob served heterogeneity. Here. the particular aim is to find clusters (latent classes) of individuals who are similar. It is assumed that a sufficient number of latent classes for the categorical latent variable results in conditional independence among the observed outcomes. This may be viewed as heterogeneity among subjects such that the dependence among the outcomes is obtained in a spuriolls fashion by mixing the heterogeneous groups. Because the latent class variable is the only cause of dependence among the outcomes, the latent class model is similar in spirit to factor analysis with uncorrelated residuals. Latent class analysis typically considers categorical indicators u of the latent class variable c, using only a subset of modeling framework B. The variables of u are binary, ordered polytornous. or unordered polytomous. Due to the conditional independence specification, the joint probability of all u's is P(UI,U2, ... ,U ) = (32) L P(c = k) P(ullc = k) P(ll2lc = k) ... P(urlc = k). k=l The model has two types of parameters. The distribution of the categorical latent variable is represented by P( c = k) expressed in terms of the logit parameters O:q in (23). The conditional u probabilities are expressed via logit parameters in line with (31) where for a binary U logit = -Tk for class k, i.e. the u* part of (28) is not needed. Similar to factor analysis, the conditional U probabilities provide an interpretation of the latent cla.<;ses such that some activities represented by the different u's are more or less likely in some classes than others. The latent class counterpart of factor scores is obtained by posterior probabil ities for each individual belonging to all classes as computed by Bayes' formula P(c= klul,lL2 ..... Ur) = P(c = k) P(ullc = k) P(ll2lc = k) ... P(urlc = k) (33) PCllI, U2.···, u ) For an overview of latent class analysis. see Bartholomew (1987), Goodman (1974) and Clogg (1995). For examples. see, e.g., Muthen (2001b), Nestadt, Hanfelt. Liang, Lamacz, Wolyniec and Pulver (1994), Rindskopf and Rindskopf (1986), and Uebersax and Grove (1990). 96 Bengl. O. lvIuthen 3. 1.1 A latent class analysis example of antisocial behavior The National Longitudinal Survey of Youth (NLS) collected data on antisocial behavior among 16 - 23 year olds. The NLSY administers an instrument with 17 binary items. l\Iaximum-likelihood estimation by l\lplus was used. Preliminary latent class analysis of the 17 items pointed to 9 items that captured 4 differ ent latent classes of antisocial behavior. Class 4 is a normative class (no high probability of endorsing any item). Class is a drug involvement class (pot and drug items). Class 2 is a persona.] offense class (fight. threat items). Class 1 is a property offense class (shoplift, stealing less than 50. conning someone. stealing goods, breaking into property). The profile plot of Figure 3 shows the est.imated item probabilities for each of the 4 classes. It should be noted that the classes are not ordered in the sense of increasing item probabilities, but involves different kinds of antisocial activities. o --- Class 1. 11% Class 2. 27% x Class 3, 20% • Cia •• 4, 41% \, .. . ' , '. , '. i '. X--T:.. ,6 "'\./:-' pot drug thr.at shoplift goods property con Item Figure 3: Profiles of antisocial behavior Table 1 illustrates the use of t.he estimated posterior probabilities for each individual in each class. The rows correspond to individuals who have t.he highest probability for that class and the entries are the average probabilities in each class. High diagonal and low off-diagonal values are desirable for good classification. It is seen that class 2 and cla<;s 3 are the hardest to distinguish between with a relatively high average class 2 probability of 0.13 for those who have their highest probability in class 3. Class 2 is the person offense class (fight, threat) and class 3 BEYOND SEM: GENERAL LATENT VARIABLE 9i is the drug class (pot, drug). While Figure 3 shows that the two classes have rather different item probabilities on these 4 items. they arc similar on the remaining 5 items. This suggests that more items are needed to more clearly distinguish these two classes. Table 1: Classification table for antisocial behavior latent class analysis Mean Posterior Probabilities Most Likely Class Class 1 Class 2 Class 3 Class 4 Class 1 0.854 0.074 0.072 0.000 Class 2 0.042 0.810 0.082 0.066 Cla.;;s 3 0.052 0.134 0.754 0.061 Class 4 0.000 0.122 0.051 0.827 3.2 Latent class analysis with covariates Similar to factor analysis with covariates, it is useful to include covariates in the latent class analysis. The aim of the latent variable modeling is still to find homogeneous groups of individuals (latent classes), but now covariates x are included in order to both describe the formation of the latent classes and how they may be differently measured by the indicators u. The prediction of latent class membership is obtained by the multinomial re gression of c on x in (23). This gives information on the composition of the latent classes. It avoids biases in the commOll ad hoc 3-step procedure: (1) latent class analysis: (2) cla'lsification of individuals based on posterior probabilities: and (:3) logistic regression analysis relating classes to covariates. The variables of x may also have a direct influence on the variables of u. beyond the influence mediated by c. This is accommodated by estimating elements of KUk in (28). For example, with a binary u. the model forms the logistic regression of u on x for cla'>s k. . , I oglt = -Tie + ""Ie X. (34) so that the direct influence of x is allowed to vary across classes. It lllay be noted that all features of multiple-group analysis are included in the latent class analysis with covariates. with dUlllmy variable covariates representing the groups. Here. T parameters are the measurement parameters. (34) shows that conditional on class these can vary across the groups, representing for exam ple gender non-invariance. The multiple-group examples of Clogg and Goodman (1985) can all be analyzed in this way. For examples of latent class analysis with covariates, see, e.g., Bandeen-Roche, l\IigliorettL Zeger and Rathouz (1997). Fonnann (1992). Heijden. Dressens and Bockenholt (1996), fvIuthen and l'vIuthen (2000), and f..Iuthen (2001b). 98 Dengt O. 3.2.1 A latent class analysis e:rample continued Continuing the antisocial behavior latent class analysis example above, three covariatcs from the NLS are added: age, gender, and ethnicity. This example is drawn from Muthen and I\Iuthen (2000). These covariatcs are specified to influence the probability of class mmbership e using the multinomial regression part (23). I\ieasurement nOllinvariancc with respect to the three covariates can be studied by including a direct effect. from a covariate to an item but was not studied here. Maximum-likelihood estimation b.y Mplus was used. The estimates from the multinomial regression predicting class membership can be translated into the curves of Figure 4. The estimated item profiles remain approximately the same m; in Figure 3 and the class interpretation is t heefore r the same. For a given age, gender. and ethnicity. F igure 4 shows the probability of membership in each class (note t hat this is not a lonitudin g al study but the x axis correspond to ages represented in this cross-sectional sample). For example, it is seen that the normative class 4 is he tmost likely class for all ages among white women, whereas this is not true for the other three groups. White Males White Females ClaM' I c -- - c .... , I •. - CI_Z I " - - . C .... 2 C ... , " ...... Cia .. , . - CIaq.. : • - CIna ' . ! :1 --'- ----- ------- ..---. . -----. ····· x .. o 0 _ _ _ _ .o - - - - -o---__ o ___ __ o _ _ ,. 20 22 " " " " " AU Are Black Males Black Females "'-, I 0 0-, I ; """3 """". - • ••• •• )1,.-. . ' ...... .. .-•• >< -------.--- ........... " ... .... ... .. • .-----. ........... ...------• ....-- i ..... . ··· x . .... . ,..........--- . . .. ... ... .. . II , ...---- .... ----- ... .... ::.,-. . .... ...... -. . ---0 I - 6 • • • • " ••. • It 1 .-- ' ,.. ... .. .. I' o l o ____ o _ ___ _ o---- -0--- __ 0 j o ... . . . . -." . .. .. - , " " " " A.e Figure 4: Influence of covariates on antisocial behavior classes BEYOND SEl\I: GENERAL LATENT VARIABLE l\IODELINC 99 Table 2 shows the resulting c:lassification table based on estimated posterior probabilities. It is seen that the use of covariate information improves the c:lass 2, class ;) distinction relative to Table 1. Table 2: Classification table for antisocial behavior latent c:lass analysis with covariates .. Mean Posterior Probabilities Most Likely Class Class 1 Class 2 Class 3 Class 4 Class 1 0.859 0.065 0.076 0.000 Class 2 0.047 0.808 0.087 0.058 Class 3 0.033 0.067 0.816 0.084 0.048 0.846 Class 4 0.000 0.105 S.S Latent class gTOwth analysis Latent c:lass growth analysis again uses a categorical latent variable to repre sent unobserved heterogeneity, but this time in a form that connects the growth modeling discussed in Section 2.1 and the latent c:lass modeling just discussed. Here, latent c:lasses are sought that are homogeneous with respect to develop ment over time. The latent class growth analysis introduces the continuous latent variable 1711 of Figure 1. In latent class growth analysis the multiple indicators of the latent classes correspond to repeated measures over time. Individuals belong to different latent c:lasses characterized by different types of trajectories. Assume for simplicity a single outcome at each timepoinL ui = (ltil. lti2 . .... Uit, ... ,ltiT)" and the simple growth model corresponding to (28), LCl'el - 1 :uTt = T]Oi + T]li at, (35) where at are fixed time scores represented in A , l1 where T is the number of time periods. Here. 1711 in (28). (29) contains an intercept and a slope growth factor \vith differences across classes captured in a and llk r 11k Xi of (29). The effects of time-varying covariates can be captured in Kllk of (28) . It may be noted that the modeling does not incorporate continuous latent variables in the form of random effects. but that 17l! is non-stochastic conditional 100 Bengt O. Muthen on x. This implies that conditional on x there is zero within-class variation across individuals. This limitation can be relaxed in line with the growth modeling of Section 2.1. With an ordered categorical outcome variable Uit, let Tt.k.s be the sth threshold in class k at timepoint t, s = 0, L 2 .... ,St - 1, where Tt.k.O = -x, Tt.k.S = ()O. Across-time and across-class measurement invariance is imposed by the threshold specification Tl.l.s = T2,1,s = ... = TT.l,s = ... = Tl.K,s = ... = TT.K.s, (36) for each s value. In the level-2 equation corresponding to (29), the Q mean of the intercept growth factorl]oi is fixed at zero in the first class for identification purposes. The mean of the intercept growth factor is free to be estimated in the remaining classes. Latent class growth analysis has been proposed by Nagin and Land (1993); see also articles in the special issue of Land (2001). For further examples, see, e.g., Nagin (1999), Nagin and Tremblay (2001), and Muthen (2001b). 3.4 Latent transition analysis Latent transition analysis is a form of latent class analysis where the multi ple measures of the latent classes are repeated over time and where across-time transitions between classes are of particular interest. Here. latent categorical vari ables are used to capture fundamental latent variable constructs in a system of regression relations akin to SEJ\L The latent transition model is an example of the use of multiple latent class variables c and is therefore not directly incorporated in the framework specified above. Muthen (2001b) showed how multiple latent class variables can be ana lyzed using a confirmatory latent class analysis with a single latent class variable including all the possible latent class combinations. applying equality restrictions among the measurement parameters. Nevertheless. this does not handle multiple time points with parameter restrictions such as first-order Markov modeling for the latent class variables. Latent transition analysis incorporated in the general is a topic for future research. An overview of latent transition modeling issues is given in Collins and \Vugal ter (1992) and Rehoussin, Reboussin, Liang and Anthony (1998). For examples. see. e.g., Collins, Graham, Rousculp and Hansen (1997). Graham. Collins, \Vu gaIter, Chung and Hansen (1991). and Kandel, Yamaguchi and Chen (1992). 3.5 Logistic regression mixture analysis Logistic regression analysis with latent classes is interesting to consider as a special case of latent class analysis \vith c:ovariates. The model was proposed by Follman and Lambert (1989) and considers a single binary u. It may be expressed BEYOND SEM: GENERAL LATENT VARIABLE MODELING 101 for cla.'.;;s k as (37) loyit = -Tk + "'u x, which is a special case of (28) where the logit intercept, i.e. the negative of the threshold T, varies across cla..<;s but the slopes do not. In (23), , q = 0 so that the covariates are assumed to not influence the class membership. Follman and Lambert (1989) considered an application where two types of blood parasites were killed with various doses of poison. In this appli cation, the assumption of , q = 0 is natural because class membership existed before the poison was administered and was not influenced by it. Follman and Lambert (1989) discuss the identification status of the model. Even in this simple form. however, logistic regression mixture analysis is dif ficult to apply in practice, probably because of the limited information available with only a single binary 11 in addition to the covariates x. This is most likely why the analysis has not caught on in practice. In contrast. latent class analysis with covariates using multiple n variables is typically a well-behaved analysis method. 4. Modeling Framework C Consider next the special case C of the general modeling framework shown in Figure l. Framework C is characterized by adding categorical latent variables, denoted by the circle c in Figure 1, to framework A. Particular models include a variety of mainstream statistical and psychometric topics. To be discussed here are finite mixture modeling, latent profile analysis, growth mixture modeling, and mixture SEM. It is interesting to compare framework C with framework B. Framework B can be seen as containing models that use latent classes to explain relationships among observed variables. A more fundamental idea can. however. be extracted from latent class approaches. Different classes can have different parameter values and, unlike the latent cla..'is model. even different model types. In other words, the idea of unobserved heterogeneity can be taken a step further using categorical latent variables. This further step is taken in framework C, and also in the subsequent general framework D.5) In framework C, the SEl"I parameterization is generalized to multiple latent classes. adding a subscript k. This is analogous to the multiple-group situation, except that group is unobserved. In what follows, this generalization of (1) and (2) will be understood. Here, multivariate normality of y conditional on x and class is assumed. This implies that the resulting mixture distribution. not condi tioning on class, is allowed to be strongly non-normal. In the :rvIplus framework of l\luthen and Muthen (1998-2001: Appendix 8), the mixture modeling allows .olrvIplus examples of framework C models are given at www.statmodel.com/rnplus/examples/mixture.html. 102 Bengt O. every parameter of framework A to vary across the latent classes. 4.1 Finite rnixtuT'e rnodehng of multivariate normals A straightforward case of framework C is finite mixture modeling of multi variate distributions. Here, the continuous latent variables of 'r/ in Figure 1 are not used. It is assumed that for class k, y is distributed as N(llk, This is a special case of the latent class generalization of (]) where there are no fac tors. Ilk = VI", = e/.;. There are two different reasons why such a mixture model ,vould be of interest. (i) to fit a non-normal distribution and (ii) to study substantively meaningful mixture components (latent classes). The flexibility of the normal mixture model to fit highly skewed data was recognized already by Pearson (1895); for a review. see l'vIcLachlan and Peel (2000, pp. 14-17, 177-179). For example. a lognormal univariate distribution is very well fit by a 2-class mixture with equal variances. Figure 5 shows a 2-class example. At the top is shmvn the mixture distribution, that is the skewed distribution that would be seen in data. At the bottom are shown two normal mixture component distributions that when mixed together by the class probabilities r. and (1 - r.) perfectly describe the distribution at the top. If the interest is in fitting a model to data from the distribution at the top, the 2-class mixture model can be used to produce mixed mct..ximum-likelihood estimates, (38) iLm = r. p] + (1 - 7r) ih, am = r. (Pi + (1) + (1 - i.) + (2) (39) using the subscript Tn to denote the mixed estimates for the distribution at the top. The delta method can be used to compute standard errors. The idea of using mixed estimates has for example been discussed in missing data modeling using the pattern-mixture approach, see, e.g. Little and 'Wang (1996). Hogan and Laird (1997), and Hedeker and Rose (20()()). In many cases, however. the mixture components have a fundamental sub stantive meaning, ,,,,here there are theoretical rea.;;;ons for individuals to behave differently and have different antecedents and consequences. Here, mixed esti mates such as (38), (:19) are not of interest. but the focus is on the parameters of the different mixture component distributions. There may for example be bio logical/genetic reasons for the existence of different mixture components. such as ,,,,ith the two kinds of trypanosomes in Section 3.5. 11ixture modeling in applications where there are substantive reasons to in vestigate different latent classes relates to cluster analysis. Cluster analysis using finite mixture modeling has been proposed a.;;; a strong alternative to conventional clustering techniques. see, e.g .. McLachlan and Peel (2000). A classic example is the Fisher's iris data analyzed in Everitt and Hand (1981). Four measures corresponding to sepal and petal lengths and widths were used to classify 150 iris flowers. Here. there ,,,,'ere three known species of iris present and the interest BEYOKO SElII GENERAL LATENT VARIABLE !vIODELIKG 103 Mixture Distribution '" '" 0 - Component 1: NO , prcpotion .75 - .- Component 2: N(2, .a"'2), propotion .25 Mixture Components :: ,,; .................... ........ .. ' . ' . ... " ...... Figure 5: A mixture of two components was in how well the classification could be recovered. This particular example also illustrates the possible difficulty ill fitting mixture models with class-varying variances, with multiple maxima and possible non-convergence or convergence to singular covariance matrices depending on starting values. An excellent overview of finite mixture modeling is given in McLachlan and Peel 2000). ( This source abo gives a multitude of examples. The iris data example is available at the j,Iplus 'Areb ste i given above. 4.2 Latent profile analysis In contrast to the analysis of the iris example above, latent profile analysis applies a structure to the covariance matrices, assuming llllcorrelated outcomes 104 Bengt O. Muthen conditional on class. 0 0 0 ( e", ()22 0 0 ( 40) I:k 9 0 0 pPk \Vith class-varying means ILk' latent profile analysis is therefore analogous to la tent class analysis. In actual analyses. models with class-invariant variances in (40) are better behaved in terms of convergence. It is interesting to note that the latent class analysis does not face this choice given that means and variances of the categorical variables of u are not represented by separate parameters. Relation ships among latent class, latent profile. and factor analysis models are described in Bartholomew (1987), Gibson (1959), and Lazarsfeld and Henry (1968). 4.,'] Growth mixture modeling The growth modeling of Section 2.1 uses continuous latent variables in the form of random effects. The continuous latent variables capture unobserved het erogeneity in terms of individual differences in growth over time. In many appli cations. however. there are more fundamental forms of unobserved heterogeneity that cannot be well captured by continuous latent variables but require categor ical latent variables. The classes of the categorical latent variable can represent latent trajectory classes. Substantive theories motivating latent trajectory classes are common in many different fields, such as with normative and non-normative development in behavioral research and disease processes in medicine. As for latent profile analysis, growth mixture modeling imposes a structure on the covariance matrix for each class. Unlike latent profile analysis, however. growth mixture modeling does not assume uncorrelated outcomes given class. Instead, further heterogeneity within class is represented by random effects that influence the outcomes at all time points. causing them to be correlated. Assume for example the following quadratic growth model for individual 1 111 class Ie: (Ie: = L 2, ... ,K). (41 ) Yit = 170; +1]li 0kt +1]2i 0kt + fit· where Yit (i = L 2 .... . 71: t = L 2 ..... T) are outcomes influenced by the random effects r/Oi. 1/1i, and ']2i. In line with Section 2.L the time scores of ° enter into the Ak matrix. The residuals fit have a TxT covariance matrix e . possibly varying across the trajectory cla.'ises (k = L 2 ..... K). The random effects are related to the covariates x, ( 42) 170i = (Yak + Xi + (0;. ( 43) 171i = Glk + Xi + (li. ( 44) 7]2i = G2k + ';k x; + (2i' BEYOND SEAl: GENERAL LATENT VARIABLE tvlODELI:\G 105 The residuals (i have a 3 x covarianc 3 e matrix'll b possibly varying across classes k (k = L 2, ... . K). It is dear that thit; model fits into framework C in line with how the growth model fit into framework A. The growth mixture model offers great flexibility in across-class parameter differences. The different shapes of the latent trajectory classes are typically characterized by the class-varying ctk parameters holding Ak class-invariant. Cer tain classes may require class-specific variances'll k and e k. In addition, different cla.<;ses may have different relationt; to x orresponding c to cla.'ls-varying 'Y k coeffi cients. A special case of the growth mixture model is obtained as a continuous-outcome version of the latent class growth analysis presented in Section 3.3. This type of modeling, proposed by Nagin and introduced into PROC TRAJ in SAS specifies '11k = 0, ek = e I. In contrast, growth mixture modeling allows for individual variation within each class through'll k. The latent cla.<;s growth analysis typically requires many more classes to fit the same data and often several of the classes represent only minor variations in trajectories and not fundamentally different growth forms. Muthen et al. (in pres..,) present a growth mixture model suitable for random ized trials. In conventional growth modeling the treatment effects can be modeled as affecting the trajectories after the treatment has started. The Muthen et al. generalization addresses the common situation that treatment effects are often different for different kinds of individuals. It allows treatment effects to vary across latent trajectory classes for the repeated measures. For a technical description of growth mixture modeling, see Muthen and Shed den (1999) and :tvluthen and Muthen (1998-2001: Appendix 8). For examples, sec, e.g. IvIuthen and Shedden (1999), Muthen and :t-.luthen (2000). Muthen (2001a. b), Muthen. Brown. Masyn . .10, Khoo. Yallg. Wang. Kellam, Carlin and Liao (in press). and Li, Duncan, Duncan and Acock (2001). 4.3.1 A growth mixture modeling example of reading fa'iiure An example clarifies the analysis opportunities presented by growth mixture modeling. Section 2.1.1 introduced a r ading e data example with phonemic aware ness development in Kindergarten related to end of grade 2 spelling performance. Figure 2 suggests heterogeneity in the phonemic awareness development. with a group of children having a close to zero growth rate in Kindergarten, Reading research points to a subgroup of children who experience reading failure by third grade, It is therefore of interest to see if signs of a failing group can be found earlier. and perhaps as early as end of Kindergarten. Two analyses are presented here as illustration (see also :t-.lutlien. Khoo. Francis, & Boscardin. in press). First, a growth mixture analysis with 1 to 5 classes was lnade of the four phonemic awareness outcomes. Second, this growth mixture model was extended to include in the same analysis the spelling test outcome from the end of sec- 106 Bengt O. Muthen ond grade. l tting e the mean and variance of this out.come vary as a function of the lat.ent trajectory classes. Both models clearly fit in framework C. IVla...ximum likelihood estimat.ion by l\.Iplus was used. A conventional linear, single-dass random effects growth model fits well in this case (X (5) = 7.49, n = .582) and shows significant variation in the intercept and slope growth factors. Such a good mean and covariance structure fit can. however, he obtained even when the true model is a growth mixture model with more than one class (see. c.g. l\.Iuthen, 1989) . Fitting linear models with 2.3.4, and 5 latent cla<;ses pointed to a steady improvement of the Baye::;iall information criterion that rewaros a high log likelihood and a low number of parameters. Given the particular int.erest in a lo w. failing class, a choice does not have to be made between the 3-, 4 -. and 5-class solutions since they all re::;ulted in the same formation of a lowest cla::;::; of 56'/(. of the children. Figure 6 shows t he est.imated growth (solid li ne) and the corresponding o bserved trajectories, where the latter are obtained by using" pseudo-classes", i.e. t he selection of individuals are obtained by random draws from their estimated posterior probabilities a<; suggested in Bandeen-Roche et !. a (1977) andl\Jutlu§n et !. a (in press). Adding the second-grade spelling test to the growth mixture model shows the predictive pO'wer of the Kindergarten informatiO'n from two years earlier. The extended g rO'wth mixture model analysis showed that the means of the spelling test were significantly different across the 3 classes. Box plots of the spelling test ::;cores based on pseudo-class a'SSignments into the 3 classes are given in Figure 7. 4.4 Mixture SEM l\lixture SEM will be mentioned only briefly III this article. It follows from the dsc i ussion in Section 2 that mixture SEM and growth mixture modeling fit into the same modeling framework. l\1ixture SEM includes mixture linear regres sion. mixtnre path analysis. ac f tor mixture analysis, and general mixture SEM. Consider as an example factor mixture analysis. where for class k ( 45) E (y tJ = Vk + Ak O k, V( y d = Ak 'ltl,: + el,:. ( 46) AnalogO'us to multiple-group analysis. a major interest is in across-cla<;s variatiO'n in the factor means. variances. and covariances of Ok , 'It k. The model is similar to growth mixture analysis in that continuous latent variables, i.e. the factors, are llsed to describe correlations among the outcomes conditional O'n class as in (46) . Lubke. Muthen and Larsen (2001) studied the identifiability of the acto f r mixture mO'dei. The special case of measurement invariance fO'r all the O'utcomes, l.e. no class variation in v. A. is of particular interest becanse it places the factors in the same metric so that Ok: 'It k comparisons a re meaningful. However, Lubke, l-.luti1en and Larsen (2001 ) point to analysis difficulties with near-singular information matrix estimates when fitting such full invariance models. These BEYOJ\D SEM: GENERAL LATENT VARIABLE MODELING 107 difficulties are not shared by the growth mixture model, which typically imposes equality of v parameters across time and across class and has few if any free parameters in A. For overvie\vs and examples of fa.ctor mixture analysis and mixture SEl\L see, e.g., Arminger and Stein (1997), Arminger, Stein and Wittenberg (1998), Blafield (1980), Dolan and van der l\laas (1998), Hoshino (2001), Jedidi, Jagpal and De Sarbo(1997), Jedidi, Ramaswamy, DeSarbo and Wedel (1996) , McLachlan and Peel (2000), and Yung (1997). Class 3, 9% ' ! .,." Class 2, 34% Clas. 1, 56% Figure 6: 3-class growth mixture model for phonemic awareness 108 Bengt O. Muthen "! '" "ii Co r/l .. C! ." :! ." c:: <'oj '" Cia .. 1 Class 2 Cia •• 3 Figure 7: Box plots for second-grade spelling scores in three phonemic awareness classes 5. Framework D Consider next the most general case D of the modeling framework shown in Figure 1. Framework D is characterized by adding categorical latent variable in dicators u to framework C. Framework D clearly shows the modeling generality achieved by a combination of continuous and categorical latent variables. This unified framework is an example of the whole being more than the sum of its parts. It is powerful not only because it contains many special cases. but also because it suggests many new modeling combinations. Particular models include a wide variety of statistical and psychometric topics. To be discussed here are complier-average causal effect modeling, combined latent class and growth mix ture modeling, prediction of distal outcomes from growth shapes, discrete-time survival mixture analysis, non-ignorable missing data modeling, and modeling of semicontinuous outcomes. ) 5.1 Complier-average causal effect modeling Complier-average causal effect (CACE) modeling is used in randomized tri als where a portion of the individuals randomized to the treatment group choose filMplus examples of framework D models are given at www.statmodel.com/mplus/examples/mixture.htmlas well as at www.statmodel.com/mplus/examples/penn.html BEYOND SEM: GE,,-"ERAL LATENT VARIABLE l\lODELING 109 to not participate ("noncom pliers"). Although developed for this specialized ap plication, CACE modeling involves interesting general lat.ent variable modeling issues. In particular, CACE modeling illustrates how latent variables are used in mainstream statistics to capture missing data on categorical variables. Here, the mixture modeling focuses on estimating parameters for substanti vely meaningful mixture components. where these mixture components are inferred not only from the out.comes but also from auxiliary information. CACE modeling ep r resents a transition from framework C to framework D . where in addition to the frame work C observed dat.a information, a minimal amount of information 011 cla.;;;s membership is added in the form of a single 1l variable observed for part of the sample. In a randomized trial, it is common to have noncom pliers a mong those invited to treatment: t.hat is, some individuals do not show up for t reatment or do not take the medication. Because of randomization, a equal-sized group of noncom pliers is also present among control group individuals. although the non-compliance status does not manifest itself. The noncomplier and complier groups are typically not similar, but may differ with respect to several characteristics such as age, education, motivation, etc. The a.<;sessment of treatment effects with respect to say the mean of aoutcome n is t heefo r re complicated. Four main approaches are common. First, "intent-ta-treat" analysis makes a straightforward comparison of the treatment group to the control group. This may lead to a diluted teat r ment effect given that not everyone in this group has received treatment. Second, one may compare compliers in the treatment group with the controls. Third, compliers in the treatment group may be compared to the combined group of noncom pliers in the treatment group and everyone in the control group. Fourth, compliers in the treatment group may be compared to compliers in the control group. Only the last approach compares the same subset of people in the treatment and control groups, but presents the problem that this subset is not observed in the control group. This problem is solved by CACE mixture modeling. CACE modeling can b e expressed by t he framework D combination of (1), (2) generalized to include the lat.ent. cla.ss addition of (n) - (29). The probability of membership in the two latent classes as a function of covariates may be expressed by the logistic regression (23), while the outcome y is expressed by the mixed linear regretlsion. (47) where I denotes the 0/1 treatment/control dummy variable. Here, Ok captures the different y means for individuals in the absence of treatment. CACE modeling typically taketl rk = 0 for the noncomplier class. In statistical anays l is this situation is vewe i d as a missing data problem. Data are missing o n the binary compliance variable for individuals in the control group. while data on hi t s variable are present for the treatment group. The frame work D conceptualization is that non-compliance status is a ate l nt class variable, where this atent l class variable becomes observed for treatment group individu- 110 Bengt O. l\luthen also Hence. the latent class variable captures missing data on a categorical vari able. Although the choice between the two conceptualizations may seem as only a matter of semantics. as described below the lat.ent variable approach suggests extensions of CACE modeling using connections with psychometric modeling that have potential value in randomized trials. The fact that latent class status is knmvn for treatment group individuals can be handled in hvo equivalent \\lays in the Mplus analysis. First, training data may be used to indicate that membership in the non-compliance da.'is is impossible for complying individuals in t he treatment group and that rnembership in the compliance class is impossible for uon-complying individuals in the treatment group. Second. a binary latent class indicator 1t defined to be identical to the latent class variable Illay be introduced in line with the ate l nt class analysis of framework B. \\'itb 0 r pr e esenting noncompliance and 1 representing compliance, the v. variable has fixed parameter values, P(n = 11compliance dass) = 1. P(n = 1lnon-compliance class) = O. The variable 71. is observed for treatment group individuals and missing for control group individuals. This second approach shows t hat CACE modeling belongs in framework D and also suggests a generalization. In psychometrics. the typical approach is to seek observed indicators for latent variables. An attempt could be made to measure the variable u also among controls. for example by asking individuals before randomization how likely they are to participate in the treatment if chosen. Several different measures u could be designed and llsed as latent class analysis indicators in line with framework B modeling. For background on CACE modeling. see, e.g., AngrisL Imbens and Hubin (1996) and Frangakis and Baker (2001). For examples, see, e.g" Little and Yau (1998) , Jo (2001a, b, c), and Jo and Muthen (2001).7) 5 .;2 Combined lalent class and growth m.ixture analysis Figure 1 hows s clearly that framework D can combine the framework B latent class aualysis with the framework C growth mixture modeling. As an example, Tvluthen and I\luthen (2000) analy:t;ecl the NLSY data discussed in Section 3, where it was of interest to relate latent classes of individuals with respect to antisocial behavior at age 17 to latent trajectory clal:ises for heavy drinking ages IS-30. Here. a latent class variable was used for each of the two sets of variables, the latellt class measurement instrument for antisocial behavior and the repeated measures of heavy drinking. Csing the confirmatory latent class analysis t.echnique described in Muthen (2001b), these two latent class variables can be analyzed together. This gives estimates of the relationships between the t.\vo classifications. To the extent that the two classifications are highly correlated, a latent class 7lDat.a and 'Ilplus input for the Little and Yao (1998) example is available on the I\1plus web site. BEYOND SEA!: GENERAL LATEKT VARIABLE MODELING 111 analysis measurement instrument can improve the classification into the latent trajectory classes. This approach i s of potential importance, e.g. using the latent class measurement instrument as a screening device in a treatment study, where different treatments are matched to different kinds of trajectory classes. 5. 3 Prediction from growth shapes 11uthen and Shedden (1999) used the latent trajectory classes in a growth mixture model of heavy drinking as predictors of distal outcomes ill the form of binary 7L variables, such as indicators of alcohol dependence. Predicting from the heavy drinking growth factors faces the potential problem of a highly non linear r eat l ionship given that a growth factor aquires its meaning in conjunction \vith other growth factors. For example, in a study of problematic bhavior e such as heavy drinking. a low slope growth factor value has a different meaning if the intercept factor value is high ("chronic" development) than when it is low ("normal" development). Given that the latent trajectory classes can represent different shapes of development, prediction from the latent classes is a powerful approach. 5.4 Specia.l uses of u indicators The framework D addition of u to framework C not only adds latent class analysis type features but also provides several unexpected additional modeling possibilities. Muthen and IvIasyn (2001) show how u can be used as event history indicators in discrete-time survival analysis. This model corresponds to a single cla.'is latent class analysis. but Il/luthen and 11asyn (2001) also explore different types of mixture survival models. Muthen and Brown (2001) show how u can be used as missing data indicators for missingness on y. This leads to an approach to study non-ignorable missing data in mixture modeling, for example where missingness is rlated e to latent trajectory classes. Muthen (2001) shows how u can be used to indicate zero or " floor" values for y , that is values that represent absence of an activity. Such data are frequently seen in behavioral research given that time of onset varies across individuals. It is the strength of frame\vork D that these seemingly disparate models can be integrated and used in new combinations to provide answers to more probing research questions. 6. Conclusions This article has provided an overview of statistical analysis with latent vari ables. In psychometrics it is typical to use latent variables to represent theoretical constructs. The constructs themselves are of key int.erest and a focus is on mea suring different aspects of the constructs. In statistics, latent variables are more typically used to represent unobserved heterogeneity. sources of variation, and missing data. The latent variables are often not of key interest but are included 11 2 Bengt O. Muthen to more correctly model the data. Unobserved heterogeneity is typically rpr e e sented by random effects, i.e. continuolls latent variables. a common example being growth modeling in t he form of the mixed linear m odlin e g (multilevel mod eling) to capt.ure individual differences in growth. Continuous latent variables are also used to represent sources of variation in hierarchical cross-sectional data, to let the model properly reflect a cluster sampling scheme and to estimate variance components. Cluster analysis considers ullobserved heterogeneity in the form of categorical latent variables. i.e. latent cla..'3ses. in order t o find homogeneous of individuals. Finite mxt i ure modeling with categorical l atent variables is a rigorow; approach to such cluster analysis. :Missing data corresponds to latent varia bles that are either ontinu c ous or categorical. T he article discussed a general l atnt e variable modeling framework that uses a combination of continuous ann categorical latent variables to give a unifying view of psychometric and statistical latent variable applications. This framework shows connections between different modeling tradit ions and suggests interesting extensions. It is the hope that this general latent variable modeling framework stimulates a better integration of pychometric s and st atistical d evlopment. e Also, it is hoped hat t this framework provides substant ive researchers with an analysis tool that is both powerful and easy to uuderstand in order t o morreadily e respond to t he omplexit. c y of their research questions. Ongoing esea r rch b y t.he author aims at further xtension e s o f the modeling framework. Acknowledgements The research wa..<; supported by rant g K02 AA 00230-01 from NIAAA, by grant 40859 from and by a g rant from NIDA. The author thanks Tihomir As parouhov for valuable discussions and computational a..'isistance. Katherine Masyn provided helpful assistance with displaying the results. The work has also ben efitted from discussions in Bengt Muthen's Research Apprenticeship Course and Hendricks Brown's Prevention Science l\Iet.hodology group . REFERENCES Agresti, A. (1990). Categorical at d a analysis. New York: John Wiley &: Sons. Angrist . J. D .. Imbens. G. \Y .. &: Rubin. D. B. (1996). Idellt ification of causal effects us ing instrumental variables. Jo'umal of the American Statistical Association, 91 , 444-445. Arminger. G. &: Stein. P. (9fJi) 1 . Finite mixtures of covariance structure models with re gressors. Sociulogical A1etlwds £1 Research. 26, 148-182. Arl1linger, G .. Stein. P .. &: Wittenberg. J. (1998). of conditional mean- and covariance-structure models. Psychometrika. 64. 475-494. Asparouhov, T. &: B. (2002). Full-information maximum-likelihood estimation of general two-level latent v Clri Clble models. Draft. BEYOKD SEAl: GENERAL L ATENT VARIABLE MODELING 113 Bandeen-Roche, K. , Ivliglioretti. D. L., Zeger, S. L. , & Rathouz, P. J . (1997). Latent variable eg r ression for multiple discrete outcomes. Journal of the Amer ican Statistical Association, 92. 137-1 5 386. Bartholomew, D. J. (1987). Latent variable modls e and factoT analysis. New York: Ox ford University Press. Blafield, E. (1 980). Clustering of observations from finite mixtures with structural infor mation. Unpublished doctoral dissertation, Jyviiskyla studies in computer science, economies, and statistics, Jyvaskyla. Finland. Bollen, K. A. 989) ( 1 . Structural equations with l aen t t variables. New Y ork: John Wiley. Brow ne, ;"1. W . & Arrninger, G. (995 1 ). S pecification and estimation of m eaIl- and covariance-structure models. In G . Arminger. C. C. Clogg, & M . E . Sobel Ed ( s.), Handbook of statistical modeling for the s r:i o al and behavioral sciences (pp. 311- 350). New Y ork: Plenum Press. Bryk, A. S. & Raudenbush, S. W. (2002). Hierarchical linear m odels: A pplications and data analysis methods. 2nd ed. Newbury Park, CA: Sage Publications. Clogg, C . C. (190.5). Latent class models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.) , Handbook of statistical m odeling fOT the social and behavioral sciences (pp, 311-359) . !'<ew York: Plenum Press. Clogg, C. C. & Goodman, L. A. (1985), Simultaneous latent structural anays l is in sev eral groups. In Tuma, N. B. (Ed .) , Sociological Methodology (pp. 81-110). San Francisco: J osey s -Bass Publishers. Collins, L. M& . Sayer. A. (Eds. ) (2001 ). New mthod e s for the analysis o f cha ge. n Wash ington, D.C.: APA. Collins, 1. M. & Wugalter, S. E. (1992). Latent class models for s age t -sequential d y namic latnt e variables. Multivariate Behavioral Research, 27, 131-157. Collins, L . M. , Graham, J. W., Rousculp, S. S., & Hansen, \V. B. (1007). Heavy caffeine use and the beginning of the substance use onset process: An illustra tion of l ant te transition analysis. In K. Bryant, M. \Vindle, & S. "Vest (Eds.) , The science of prevention: Methodological advances from alcohol and ubstance s use research (pp. 79-99). Washington DC: American Psychological Association. Dayton. C . 'vI. l & :'vlacready, G. B. ( 988). 1 Concomitant variable aten l t class models. JO'urnal of the meArican Statistical Association, 83, 173-178. Dolan, C. V. & van der Maas, H. L. J 99 . (81 ). Fitting multivariate normal mixtures subject to structural equation m odeling. Psychomet7ika, 63, 2 27-253. Everitt, B. S& . Hand, D. J. (1981). Finite mixture distributions. London: Chapman and Hll. a Follrnann, D. A. & Lambert. D. (1989). Generalizing logistic regression by nonparamct ric mixing. Journal of the American Statistical Association, 84, 295-300. Formann, A. K. (1992). Linear logistic latent class analysis for polytomous dta a . Jo 1177W I of the A mric e an Statistical Association, 87, 476-486. Frangakis C. E. &: Baker, S. G2001 . ( ). Compliance subsampling d esigns or fcomparative research: Estimation and opt imal planning. Biometrics, 57. 899-908. Gibson, W. A(1. 9.'59) . Three multivariate modl;;: e factor analysis, ate l nt structure anal ysis, and latent profile analysis. Psychom etrika. 24, 229-2S2. Goldstein, H. (1995) . Afultilevel statistical models. London: Edw,ll'd Arnold. Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 21S-23l. 114 Bengt O. Muthen Graham. J. \V., Collins, L. Wugalter. S. E., Chung. K., & HanselL W. B. (1991). I\Iodeling transitions in latent stage- sequential processes: A substance use prevention example. Journal of Consulting and Clinical Psychology, 59. 48-57. Hecht, R. H. (2001). :Vlultilevel modeling with SEM. In G. A. :tvIarcoulides & R. E. Schu macker (Eds.) . New developments and techniques in structural equation modeling (pp. 89-127). Mahaw. NJ: Lawrence Erlbaum Associates. Hedeker. D. & Rose. J. S. (2000). The natural history of smoking: A pattern-mixture random-effects regression model. In.1. Rose. 1. Chassin, C. Presson, & J. Sherman (Eds.). Multimriate applications in substance use research (pp. 79-112). Hillsdale, l\.J: Erlbaum. Heijden, P. G. M .. Dressens. J .. & Bockenholt. C. (1996). Estimating the concomitant variable latent-class model with the EM algorithm. Journal of Educational and Behavioral Statistics. 21. 215-229. Hogan. J. W. & Laird, M. (1997). Mixture models for the joint distribution of re peated measures and event times. Statistics in Medicine, 16. 239-258. Hoshino. T. (2001). Baycsian inference for finite mixtures in confirmatory factor analysis. Behaviormetrika. 28. 37-63. Hosmer, D. \V. (1973). A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761-770. Jedidi. K .. Jagpal, H. 5., & DeSarbo, W. S. (1997). Finite-mixture structural equation models for response-based segmentation and unobserved heterogeneity. AIarket ing Sc·ience, 16, 39-59. Jedidi, K .. Ramaswamy. V., DeSarbo, W. S., & Wedel, M. (1996). On estimating finite mixtures of multivariate regression and simultaneous equation models. Structural Equation AIodelinq, 3, 266-289. Jennrich. R. 1. & Schluchter, .\1. D. (1986). Unbalanced repeated-measures models with structured covariance matrices. Biometrics, 42, 805-820 . .10, B. (2001a). Estimation of intervention effects with noncompliance: Alternative model specifications. Accepted for publication in the Journal of Educational and Behav ioral Statistics. ,lo. B. (2001 b). ?vlodd misspecification sensitivity analysis in estimating causal effects of interventions "vith noncompliance. Accepted for publication in Statistics in Medicine. Jo, B. (2001c). Statistical power in randomized intervention studies with noncompliance. Forthcoming in Psychological Methods. Jo. B. & J\Iuthen, B. (2001). Modeling of intervention effects with noncompliance: A la tent variable modeling approach for randomized trials. In G. A. Marcoulides & R. E. Schumacker (Eds.), New df'l'eiopments and techniques in structural equation modeling (pp. 57-87). Mahaw, !'\.1: Lawrence Erlbaum Associates. Joreskog, K. G. & Goldberger, A. S. (1975). Estimation of a model with multiple indi cators and multiple causes of a single latent variable. Journal of the American Statistical Association, 70, 631-6:39. Kandel. D. B .. Yamaguchi, K., f;,: Chen. K. (1992). Stages of progression in drug involve- ment from adolescence to adulthood: Further evidence for the gateway theory. Journal of Studies of Alcohol. 53, 447-457. Kaplan D. & Elliott. P. R. (1997). A didactic example of multilevel structural equation modeling applicable to the study of organizations. Structural Equation A10deling, 4, 1-23. BEYOND SEAI: GENERAL LATENT VARIABLE MODELING 115 Khoo, S. T. &. I\Iuthen. B. (2000). Longitudinal data on families: Growth modeling al ternatives. In J. Rose, L. Chassin, C. Presson. &. .1. Sherman (Eds.). A!ultivariate appl-ications in substance use research (pp. 43-78). Hillsdale, 1\ J: Erlbaum. Laird, N. M. &. Ware, .1. H. (1982). Random-effects models for longitudinal data. Bio metrics, 38, 963-974. Land, K. C. (2001). Introduction to the special issue on finite mixture models. Sociolog ical Methods &I Research, 29, 275-28l. Lazarsfeld, P. F. &: Henry. 1\. \V. (1968). Latent structure analysis. New York: Houghton Mifflin. Li, F., Duncan, T. E. Duncan, S. C., &. Acock, A. (2001). Latent growth modeling of longitudinal data: A finite growth mixture modeling approach. Structural Equation Modeling, 8 , 493-530. Little, R . .1. &: Wang, Y. (1996). Pattern-mixture models for multivariate incomplete data with covariates. Biometrics, 52, 98-111. Little, R. .I. &. Yau, L. H. Y. (1998). Statistical techniques for analyzing data from pre vention trials: Treatment of no-shows using Rubin's causal model. Psychological Methods. 3. 147-159. Lindstrom, M . .I., &. Bates, D. M. (1988). Newton-Raphson and K\l algorithms for lin ear mixed-effects models for repeated-measures data. Jo'urnal of the American Statistical 83, 1014-1022. Lubke, G., Muthen, B., & Larsen, K. (2001). Global and local identifiability of factor mixture models. Submitted for publication. University of California, Los Angeles. MacCallum, R. C. &: Austin . .I. T. (2000), Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201-226. McDonald, R. &. Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural relations in two-level data. British Journal of Mathematical and Sta tistical Psychology, 42, 215-232. McLachlan, G . .1, &. Krishnan, T. (1997). The EM algorithm and extensions. New York: .1ohn Wiley &. Sons. YlcLachlan, G. J. &. Peel, D. (2000). Finite mixture models. New York: Wiley &. Sons. }.iuthen, B. (1988). Some uses of structural equation modeling in validity studies: Ex tending IRT to external variables. In H. Wainer &. H. Braun (Eds.), Test validity (pp. 213-238), Hillsdale, N.1 : Erlbaum Associates. Muthen, B.(1989). Latent variable modeling in heterogeneous populations. Psychome trika, 54, 557-585. Muthen, B. (1990). }'lean and covariance structure analysis of hierarchical data. UCLA Statistics Series #62, August 1990. }'luthen, B. (1991). }'lultilevel factor analysis of class and student achievement compo nents. Journal of Educational Measurement, 28. 338-354. Muthen, B. (1994). covariance structure analysis. In .1. Hox &: I. Kreft (Eds.), Multilevel Modeling, a special issue of Sociological Methods E1 Research, 22, 376- I\luthen. B. (1997). Latent variable modeling with longitudinal and multilevel data. In A. Raftery (Ed.), Sociological Methodology (pp. 453-480). Boston: Blackwell Pub lishers. Muthen. B. (2001a). Second-generation structural equation modeling with a combina tion of categorical and continuous latent variables: New opportunities for latent class/latent growth modeling. In Collins, L. }I. & Sayer, A. (Eds.), New methods for the analysis of change (pp. 291-322). Washington, D.C.: APA. 116 Bengt O. Muthen Muthen, B. (2001b). Latent variable mixture modeling. In G. A. :\Iarcoulides & R. E. Schumacker (Eds.), New developments and techniq'ues in structural equation mod eling (pp. 1-33). :\1ahaw, KJ: Lawrence Erlbaum Associates. :\1 uthen, B. (2001 c). Two-part growth mixture modeling. Draft. University of California, Los Angeles. yluthen, B. & Brown. C. H. (2001). missing data in a general latent vari able framework. Draft. University of California, Los Angeles. :\luthen, B., Brown. C. H., :\lasyn. K, Jo, B .. Khoo, S. T .. Yang. C. c.. Wang. C. P., Kellam. S .. Carlin. J .. & Liao, J. (in press). General growth mixture modeling for randomized preventive interventions. Forthcoming in Biostatistics. :\luthen, B. & Curran. P. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framevmrk for analysis and power esti mation. Psychological Methods, 2, 371-402. :\luthen, B., Kao, C. F., & Burstein, L. (1991). Instructional sensitivity in mathematics achievement test items: Applications of a new IRT-based detection technique. Journal of Educational Measurement, 28, 1-22. :\1uthen. B. & Khoo. S. T. (1998). Longitudinal studies of achievement growth using la tent variable modeling. In John B. Cooney (Ed.), Latent Growth Curve Analysis, a special issue of Learning and Individual Differences. 10, 73-101. Muthen. B., Khoo, S. T., Francis, D .. & Kim Boscardin, C. (in press). Analysis of read ing skills development from Kindergarten through first grade: An application of growth mixture modeling to sequential processes. In S. R. Reise & N. Duan (Eds), Multilevel modeling: Methodoloqical advances, issues, and applications. Mahaw, K.1: Lawrence Erlbaum Associates. Muthen, B. & Masyn, K (2001). Discrete-time survival mixture analysis. University of California, Los Angeles. Muthen, B. & Muthen, L. (2000). Integrating person-centered and variable-centered analysis: Growth mixture modeling with latent trajectory classes. Alcoholism: Clinical and Experimental Research, 24, 882-891. Muthen, L. & I\luthen, B. (1998-2001). Mpz.ususer's guide. Los Angeles. CA: :\luthen & Nagin, D. S. (1999). Analyzing developmental trajectories: A semi-parametric, group based approach. Psychological Alethods, 4, 139-157. Nagin, D. S. & Land, K C. (1993). Age, criminal careers, and population heterogeneity: Specification and estimation of a nonparametric, mixed Poisson model. Crimino.l o.gy, 31. 327-362. Kagin, D. S. & Tremblay, R. E. (2001). Analyzing developmental trajectories of distinct but related behaviors: a group-based method. Psychological lv/ethods, 6, 18-34. Nestadt, G., Hanfelt. J.. Liang, K. Y., Lamacz. :\1., \Volyniec, P" & Pulver. A. E. (1994). An evaluation of the structure of schizophrenia spectrum personality dis orders. Journal o.f Personality Diso.rders. 8, 288-298. Pearson, K (1895). Contributions to the theory of mathematical evolution, II: skew vari ation. Philo.so.phical Tmnsactio.ns of the Royal Society o.f London A, 186. 343-414. Raudenbush, S. W. (2001). Toward a coherent framework for comparing trajectories of individual change. In Collins. L. :\1. &: Sayer. A. (Eels.), New methods for the analysis of change (pp. 35-64). vVashington. D.C.: APA. Reboussin, B. A .. Reboussin, D. ]'v1.. Liang. K. Y.. & Anthony, J. C. (1998). Latent transition modeling of progression of health-risk behavior. Multivariate Behavioml Research, 33, 457-478. BEYOND SEA!: GENERAL LATENT VARIABLE MODELING 117 Rindskopf, D. & Rindskopf, W. (1986). The value of latent class analysis in medical di agnosis. Statistics in Medicine, 5, 21-27. Takane, Y. & DeLeeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408. Uebersax, J. S., & Grove, W. I\f. (1990). Latent class analysis of diagnostic agreement. Statistics in Medicine, 9, 559-572. Yung, Y. F. (1997). Finite mixtures in confirmatory factor-analysis models. Psychome trika, 62. 297-330. Received December 2001. Final version accepted JanuarJ' 2002. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Com- mons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.

Journal

Behaviormetrika – Springer Journals

Published: Jan 1, 2002

Keywords: Maximum-likelihood estimation; factor analysis; structural equation modeling; growth curve modeling; multilevel modeling; finite mixture modeling; random effects; missing data; latent classes

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Beyond SEM: General Latent Variable Modeling

Beyond SEM: General Latent Variable Modeling

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Beyond SEM: General Latent Variable Modeling

Beyond SEM: General Latent Variable Modeling

References (93)

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies