11/27/6
Estimating and Comparing Means
1
Estimating and Comparing Means
SEM has been primarily used as a set of tools to investigate relationships. As we know from our study of
regression analysis, relationship can be used to compare means. So, since SEM is a generalization of regression
techniques, SEM can also be used to compare means through relationships.
The covariates or group-coding-variables or relationships method of comparing means
Model: G:\MdbT\P595\AmosMeansEstimated\IndtExample
Data: G:\ MdbT\P595\AmosMeansEstimated\IndtExample.sav
Simplest Example: Comparing two independent means. Mean performance of Men vs. Women in an intro
statistics course. Men are Group=1, Women are Group=2.
Scatterplot of the relationship.
Data matrix:
Variance of
Group scores
The SEM
Raw regression
coefficient Difference
between means
when there are
only two groups.
Variance of
residuals
Raw coefficients
Standardized
regression
coefficient – r in
simple regression
Standardized coefficients
r-squared
FYI- The critical ratio for the regression coefficient is -.55, p = .583. So performance is not related to
Gender. That is, there is no significant difference between Group means.
This method is called the “covariates” method because the variable representing group membership is
treated as a predictor or “covariate” in SEM terminology.
11/27/6
Estimating and Comparing Means
The Separate Groups method of comparing group means.
2
Model: G:\MdbT\P595\AmosMeansEstimated\IndtExampleSeparateMeans
Data: G:\ MdbT\P595\AmosMeansEstimated\IndtExample.sav
Overview . . .
1) A multiple-group model is created, in which a path diagram is specified separately for each group.
2) A general model is applied in which a separate mean value for each group is estimated.
3) A special model, in which the same mean value for each group is estimated.
4) The chi-square difference test between the general and special models is used to test the significance of the
difference. If the chi-square is significant, that means that the general model allowing separate means fit
significantly better than the special model that required means to be equal, leading to the conclusions that the
means must differ significantly.
Independent Groups t Example: A Two group model comparing Performance between the two groups.
Separate groups comparisons can be performed in all popular CFA/SEM programs. Alas, the details of
performing the separate groups comparison are specific to the program. So, while you must learn the
specifics of Amos for this class, please focus on the big picture so that if you end up with a different
program, you can apply the big picture steps using the details of that different program.
1. This first part of the process tells Amos that we’re going to create a multi-group model and gives
some details of that process.
a. With a blank Amos Input window, double-click on “Group number 1”.
This opens a “Manage Groups” dialog box.
b. Type the name of the first group in the box.
c. For each subsequent group, click on “New” and type the name of the group.
d. When done specifying group names, click on the “Close” button.
11/27/6
Estimating and Comparing Means
3
2.
Connect Amos to the data and identify the groups in the data.
a. File -> Data Files . . . -> File Name. (Follow the steps below carefully.)
b. Click on [File Name] and identify the data file.
c. Click on [Grouping Variable] button and click on the name of the variable identifying the groups.
d. Click on the Group Value button and identify the value for the first group.
e. Highlight each subsequent group in the Data Files window, and repeat a, b, and c.
11/27/6
Estimating and Comparing Means
3. Draw the path diagram for each group.
4
By default, Amos will make the path diagram will be the same for each group, although the parameters
of the objects in the diagram, in this case, the means, will not be the same.
a. Highlight Males.
b. Draw the path diagram. For an independent samples t, the path diagram will simply be a single
rectangle. (Note that this is different from the covariates method, for which we had to tell Amos the
name of a covariate. In this method, the covariate is not specified, but is implied by the different group
models.)
When you draw a path diagram for one group, all characteristics of the path diagram are copied to
the other group(s). Highlight Females. Note that the path diagram is the same for both.
4. Make sure that the program is set up to estimate means.
a. View -> Analysis Properties -> Estimation -> Estimate Means and Intercepts.
Amos does NOT automatically estimate means. We’re telling to estimate them in this setp.
5. Specify that different means are to be estimated for each group.
a. Right-click on an object -> Object Properties.
As mentioned above, Amos automatically copies all characteristics of a path diagram to all groups.
Here were overriding that option for the means. We’ll allow the means to be different in each group.
b. click on the “Parameters” tab and uncheck the [All Groups] button. VERY IMPORTANT.
c. Highlight Males.
d. Enter a name for the mean of the Male group, e.g., malemean, the Mean field of the Object Properties
Parameters dialog.
e. Highlight Females, then enter a name for the mean of the Female group, e.g. femmean.
The path diagram should look like the following
:
6. Save the file.
11/27/6
Estimating and Comparing Means
5
7. Tell the program to apply two models – one with separate means and one with equal means.
a. Double-click on “Default Model”.
b. Type “Means Separate” in the Model Name field.
`Since the means have separate names, Amos will estimate them separately for each group by default.
c. Click on the New button, then type Means Equal in the Model Name button.
For this model, we want the two means to be estimated as identical values. So . . .
d. Type malemean = femmean in the Parameter Constraints field.
e. Click on the “Close” button.
We’ve now created two models – one in which the means are estimated separately and one in which the
means are estimated as having the same value.
8. Run the models.
11/27/6
Output from Amos
Estimating and Comparing Means
6
1. First, the mean separate model
2. Now the means equal model.
The chi-square is not significant, indicating that the there is no significant difference in goodness-of-fit between
the two models. From this we can infer that the means are NOT significantly different.
From the Text output, Chi-square = .260, p = .610.
Note that the test was conducted allowing the variances to be unequal.
Why, Why, Why? Why go to all this trouble to compare two means?
The Separate means method was much more difficult to carry out than the “Covariates” method.
Why would anyone do it?
1. The separate means method is much more flexible. It can be conducted assuming variances are equal or
assuming they’re unequal.
2. The separate means method extends quite easily (once you get the hang of specifying the models) to multiple
groups.
11/27/6
Estimating and Comparing Means
Example 2: Analysis of variance. (This is designed as an in-class exercise.)
7
Comparing mean performance in P511 for 3 years.
Boring example.
SPSS Output
Kept for completeness.
De scriptiv es
p5 11g
95 % Co nfide nce I nterva l for
Me an
20 03
N
18
Me an
.84 56
20 04
21
.85 29
20 05
36
.84 44
To tal
75
.84 71
Std . Deviatio n
.08 298
Std . Erro r
.01 956
Lo wer B ound
.80 43
Up per B ound
.88 68
Mi nimu m
.70
Ma ximu m
.98
.08 939
.01 951
.81 22
.89 35
.69
1.0 1
.07 696
.01 283
.81 84
.87 05
.65
.99
.08 097
.00 935
.82 84
.86 57
.65
1.0 1
Tes t of Homogene ity of Varia nces
p51 1g
Levene Statistic
.40 1
df1
df2
2
Sig .
.67 1
72
ANOVA
p5 11g
Su m of Squa res
df
Me an S quare
Be tween
Gro ups
.00 1
2
.00 0
Wi thin G roup s
.48 4
72
.00 7
To tal
.48 5
74
F
Sig .
.07 4
.92 9
So, there were no significant differences in mean performance across the 3 years.
The Amos output from a multigroup model, set up in the same way as the above independent groups t-test
model.
Amos chi-square for the model in which the three group means were constrained to be equal was .0.1 with
df=2, p > .05.
The nonsignificant of the chi-square says that the model with means constrained to equality fit essentially as
well as a model with means estimated separately.
From this we can infer that there are NOT significant differences between the means.
11/27/6
Estimating and Comparing Means
8
Example 3. Extending the multiple groups conceptualization to comparison of correlations.
This example is from research we’ve been conducting on respondent inconsistency – the tendency of persons to
give different self-reports to items from the same personality dimension. Some persons are quite consistent
from item to item within the same personality dimension. Others are more inconsistent, giving different
responses to items even though all the items represent the same personality dimension.
Although we’d expect some differences in responses because, after all, the items are different, we’ve found that
there are reliable differences in the amount of inconsistency shown by people. If a person is inconsistent in
responding to one personality questionnaire, he/she’ll be inconsistent in respond to other personality scales.
Inconsistency in self report appears to be a personality characteristic, one that cuts across different
questionnaires. (Need a MS thesis on whether or not it cuts across time periods.)
Comparing correlations across groups.
A. Comparing convergent validities between corresponding measures of the Big 5
From Biderman, M. D., & Reddock, C M. (2012). The relationship of scale reliability and
validity to respondent inconsistency. Personality and Individual Differences, 52,
647-651.
We formed three groups based on inconsistency of responses to the original IPIP 50-item
questionnaire.
We then created an Amos model which assessed the convergent validity between corresponding Big
Five dimensions as measured by two other Big Five questionnaires – the IPIP “Other” 50-item questionnaire
and the Thompson minimarkers.
We expected convergent validity to be greatest for the most consistent respondents and least for most
Inconsistent respondents.
We used the Amos models to assess the significance of the differences in inconsistency across the three
groups.
The Amos file is “MDBR\1BalancedScale\Inconsistency II\Inconsistency II Amos\Table 5 convergent validity
Revised 3.amw”
11/27/6
The Input model
Estimating and Comparing Means
9
The parameters circled in red are the convergent validities. It was assumed that they would be largest for
inconsistency group 1 (least inconsistent respondents) and smallest for group 3.
The names of the convergent validity parameters were different for each group.
Group 1: covee1
Group 2: covee2
Group 3: covee3
covaa1
covaa2
covaa3
The constrained model was
covcc1
covcc2
covcc3
covss1
covss2
covss3
covoo1
covoo2
covoo3
11/27/6
Estimating and Comparing Means
10
Group 1 results. Mean of convergent validities = .784.
Group 2 results. Mean of convergent validities = .720. (The red’d double-headed arrow is of no consequence.)
11/27/6
Estimating and Comparing Means
Group 3 results. Mean of convergent validities = .680.
11
11/27/6
Estimating and Comparing Means
12
Convergent Validities constrained model. Chi-square difference (10 df) = 20.81. p < .05.
Since the chi-square p-value was less than .05 (gotten from the text output), this indicates that the model which
constrained the convergent validities to be equal fit significantly worse than the model which let the
convergent validities be separate for each group.
From this we can infer that the convergent validities are less for the more inconsistent respondents.
So, if you have developed a new questionnaire and want to show that its measures exhibit high convergent
validity with existing measures of the same constructs, make sure you assess convergent validity using
consistent responders.
11/27/6
Estimating and Comparing Means
13
B. Comparing criterion-related validities between corresponding measures of the Big 5
From the article . . .
Skipped to end of
lecture in 2015 .
Criterion-related validity defined as correlations of GPA with measures of conscientiousness
from Big Five Questionnaire 2 and the Mini-Marker questionnaire were computed for each
group. Table 5 presents the validity coefficients. Inspection of the table shows that validity was
roughly the same for both the most consistent group and the middle group but fell off
dramatically for the most inconsistent group for both scales. To provide some evidence of the
significance of differences across inconsistency groups, two multigroup regression models
were formed using Amos. In the first, GPA was regressed simultaneously onto
Questionnaire 2 and Mini-Marker Conscientiousness domain scores allowing regression
coefficients and variances to be unique within each inconsistency group, creating a
saturated model. In the second, restricted model, variances and regression weights were
constrained to be equal across groups, yielding eight degrees of freedom. The chisquare difference statistic was 18.10 (p < .05), suggesting that criterion-related validity was
related to inconsistency.
For this analysis, the convergent validities of two measures of conscientiousness were compared across three
groups defined by inconsistency of respondents.
The input model is
The constraints are
Note that the variances of the predictors are constrained to be equal across inconsistency groups as are the
unstandardized regression coefficients (slopes).
11/27/6
Estimating and Comparing Means
14
The models
Group 1 (most consistent respondents) results (p values not printed because Chi-square = 0 and p is undefined.)
Group 2 results
Group 3 (most Inconsistent respondents) results. (Worst validity.)
11/27/6
Estimating and Comparing Means
15
The constrained parameters result
Constraining the parameters to equal across groups resulted in a significantly poorer fitting model, suggesting
that there were differences in criterion-related validity across groups. (Although I have to admit, the differences
were not huge.)