TIMSS MOTIVATION MEASURES
1
Factor Structure, Discriminant and Convergent Validity of TIMSS Math and Science Motivation Measures: A Comparison of Arab and Anglo-Saxon
Countries. Supplemental Material for External Website
<<Note: The external appendices are intended to appear only on a website associated with the journal that is hot-linked to the
article, not the printed article.>>
External Appendix 1
Table 1
Structure of Residual Variance/Covariance Matrix Representing Parallel Items (PI) and Negative Items (NI)
1 MVAL1
UNQ
2 MVAL2
0 UNQ
3 MVAL3
0
0 UNQ
4 MVAL4
0
0
0 UNQ
5 MSCP1
0
0
0
0 UNQ
6 MSCP2
0
0
0
0
0 UNQ
7 MSCN1
0
0
0
0
0
0 UNQ
8 MSCN2
0
0
0
0
0
0
NI1 UNQ
9 MAFFP1
0
0
0
0
0
0
0
0 UNQ
10 MAFFP2
0
0
0
0
0
0
0
0
0 UNQ
11 MAFFN1
0
0
0
0
0
0
NI
NI
0
0 UNQ
12 SVAL1
PI
0
0
0
0
0
0
0
0
0
0 UNQ
13 SVAL2
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
14 SVAL3
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
15 SVAL4
0
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
16 SSCP1
0
0
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
17 SSCP2
0
0
0
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
18 SSCN1
0
0
0
0
0
0 PI/NI NI
0
0
NI
0
0
0
0
0
0 UNQ
19 SSCN2
0
0
0
0
0
0
NI PI/NI 0
0
NI
0
0
0
0
0
0
NI
UNQ
20 SAFFP1
0
0
0
0
0
0
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
21 SAFFP2
0
0
0
0
0
0
0
0
0
PI
0
0
0
0
0
0
0
0
0
0 UNQ
22 SAFFN1
0
0
0
0
0
0
NI
NI
0
0 PI/NI 0
0
0
0
0
0
NI
NI
0
0 UNQ
23 MMORE
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0a
24 SMORE
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0a
Note. UNQ = uniqueness, the residual variance term for each item. PI = correlated uniqueness associated with the same item used to assess parallel math and verbal
construct. NI = correlated uniqueness associated with negatively-worded items (a negative item method effect)
aThese two items are used to define single-item factors and so the uniqueness term is fixed to zero and not estimated.
TIMSS MOTIVATION MEASURES
2
External Appendix 2
MPlus Syntax for Final Model 17 (see published article)
TITLE: Model 17 (Extended MIMIC Model);
DATA:FILE='D:\timss2007\data7jun\GRD8\TIMSS26May12Grd8imputation.txt';
TYPE=Imputation;
! Brings in the 5 multiple imputation files;
VARIABLE:
NAMES ARE
IDCNTRY IDSCHOOL IDCLASS IDGRADE IDSTUD ITSEX HmLang ITLANG
MALG MDAT MNUM MGEO SCHE SEAR SBIO SPHY MACHtot SACHtot MACH SACH
mscp1 mscn1 mscn2 mscp2 MaffP1 Maffn1 MaffP2
SSCp1 SSCn1 SSCn2 SSCp2 SaffP1 Saffn1 SaffP2
mval1 mval2 mval3 mval4 mmore sval1 sval2 sval3 sval4 smore
HOUWGT L2MACH L2SACH L2MACHmi L2SACHmi L2MACHmx L2SACHmx
FHIEd MHIEd EdAsp hied AGE L2AGE TchAge TchSEX TchYrs BSDMPATM;
! Variables in the files;
cluster is IDCLASS;
! complex design cluster by class
IDVARIABLE IS IDSTUD;
USEVARIABLES ARE
MVal1 MVal2 MVal3 MVal4 mscp1 mscp2 mscn1 mscn2 Maffp1 Maffp2 Maffn1
SVal1 SVal2 SVal3 SVal4 SSCp1 SSCp2 SSCn1 SSCn2 Saffp1 SafFp2 Saffn1
MMore SMore MACH SACH EdAsp ITSEX ;
! variables actually used in the analysis
CLUSTER is IDCLASS; ! Level 2 variable in Multilevel model;
weight is HOUWGT; !TIMSS weighting factor.
useobservations
IDCNTRY eq 840 or IDCNTRY eq 926 or IDCNTRY eq 36 or IDCNTRY eq 927 or
IDCNTRY eq 682 or IDCNTRY eq 400 or IDCNTRY eq 512 or IDCNTRY eq 818;
! select 8 countries used in the analysis
grouping is IDCNTRY
(840=usa 926=ENG 36=Aus 927=SCO 682=SAUD 400=JORD 512=Oman 818=EGYP);
! defines the grouping variable for multiple group analysis to be the 8 countries
define: edasp = (edasp - 3.77)/1.382;
define: ITSEX = (ITSEX - 1.500)/.500;
! standardize variables to facilitate interpretations of unstandardized estimates
ANALYSIS: type=complex; ESTIMATOR=MLR;
! uses complex design option and robust ML estimator
model:
!This is the general model specification that applies to all multiple groups but
can be qualified in relation to country specific models that follow
mval by MVal1-MVal4*(1-4);msc by mscp1-mscn2*(5-8);MAFF by MAFFp1-MAffn1*(911);
! The values in parentheses (1-4) refer to parameter estimates; Because these are
the same for the math and science factor loadings, they are estimated subject to
the constraint that they are invariant over the math and science domains;
sval by SVal1-SVal4*(1-4);ssc by sscp1-sscn2*(5-8);SAFF by SAFFp1-SAffn1*(911);
TIMSS MOTIVATION MEASURES
3
!metric defined by fixing variances
mval-saff@1; MASP by MMore@1; MMore@0;SASP by SMore@1; SMore@0;
!Intercepts Invariant over Domain
!CUs for matching items
MVal1-MAffn1 pwith SVal1-SAffn1;
!CUs for negative item effect
mscn1 mscn2 MAffn1 sscn1 sscn2 SAffn1 with mscn1 mscn2 MAffn1 sscn1 sscn2
SAffn1;
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX
! estimate relations with new variables added in the MIMIC Model
model usa :
!Country specific model for USA
! Factor Loadings: The values in parentheses [e.g. (A1-A4)] refer to parameter
estimates; Because these are the same for the math and science factor loadings,
they are estimated subject to the constraint that they are invariant over domain
(within-group invariance). Because they are the same for each country, factor
loading are estimated subject to the constraint that they are invariant over
country (between-group invariance). Six of the motivation factors are based on
multiple items, but two are single-item factors;
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-maff@1; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
! Item intercepts: Values in brackets are intercepts. The values in parentheses
e.g. (A101-A108) refer to parameter estimates of these intercepts; Because these
are the same for the math and science factor loadings, they are estimated subject
to the constraint that they are invariant over domain (within-group invariance).
Because they are the same for each country, factor loading are estimated subject
to the constraint that they are invariant over country (between-group
invariance). Six of the motivation factors are based on multiple items, but two
are single-item factors;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (C109-C111); [sscn1,SSCN2,sval3] (C109C111);[sval1,MVAL1*];
! In this model of partial invariance of intercepts, the constraint of invariance
over country is relaxed for scn1, SCN2 and val3 and val1 items in both domains;
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model eng :
!Country specific model for England (see discussion of country-specific
parameters in the detailed description of USA Model
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (B109-B111); [sscn1,SSCN2,sval3] (B109B111);[sval1,MVAL1*];
TIMSS MOTIVATION MEASURES
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model sco :
! Country-specific model for Scotland
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (C109-C111); [sscn1,SSCN2,sval3] (C109C111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model aus :
! Country-specific model for Australia
mval by MVal1-MVal4*(A1-A4);msc by MScp1-MScn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (C109-C111); [sscn1,SSCN2,sval3] (C109C111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model saud:
! Country-specific model for Saudi-Arabia
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-maff@0];[masp@0];[sval-saff*];[sasp*];masp-sasp*;MVal1-MAffn1 pwith
SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (E109-E111); [sscn1,SSCN2,sval3] (E109E111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model jord:
! Country-specific model for Jordan
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
4
TIMSS MOTIVATION MEASURES
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (F109-F111); [sscn1,SSCN2,sval3] (F109F111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
Model oman:
! Country-specific model for Oman
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (G109-G111); [sscn1,SSCN2,sval3] (G109G111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX
Model egyp:
! Country-specific model for Eqypt
mval by MVal1-MVal4*(A1-A4);msc by mscp1-mscn2*(A5-A8);maff by maffp1Maffn1*(A9-A11);
sval by SVal1-SVal4*(A1-A4);ssc by sscp1-sscn2*(A5-A8);
Saff by Saffp1-Saffn1*(A9-A11);
mval-saff* ; Masp by MMore@1; MMore@0;Sasp by SMore@1; SMore@0;
[MVal2,Mval4-mscp2,Maffp1-Maffn1,mmore] (A101-A108);
[sVal2,sval4-sscp2,Saffp1-Saffn1,smore] (A101-A108);
mscn1 mscn2 Maffn1 sscn1 sscn2 Saffn1 with mscn1 mscn2 Maffn1 sscn1 sscn2
Saffn1;
[mval-saff*]; [masp-sasp*]; masp-sasp*; MVal1-MAffn1 pwith SVal1-SAffn1;
[mscn1, MSCN2 ,mval3] (H109-H111); [sscn1,SSCN2,sval3] (H109H111);[sval1,MVAL1*];
mval-sasp,Mach-ITSEX with
mval-sasp,Mach-ITSEX;
OUTPUT: TECH1 TECH4 STDYX sampstat svalues MODINDICES(ALL) ;
! Note: Modification indices are not available with multiple imputation (and are
simply ignored) but are available for tests of individual imputation data sets
5
TIMSS MOTIVATION MEASURES
6