Measuring Group-Level
Psychological Properties
(A Tribute to Larry James)
Daniel A. Newman
University of Illinois
Daniel A. Newman, Ph.D.
1
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
2
Overview
Group-Level Psychological Properties?
1. Psychological Climate

James & Jones (1974), Jones & James (1979), James & Sells (1981),
James (1982), James et al. (1988), James & James (1989)
2. Aggregation Bias James (1982), James et al. (1980)
3. Why we need rWG (Within-group agreement)

James (1982), James, Demaree, & Wolf (1984; 1993), George &
James (1993)
4. rWG(J) for multi-item scales

3
James, Demaree, & Wolf (1984), LeBreton, James, & Lindell (2005)
Overview
Group-Level Psychological Properties?
1. Psychological Climate

James & Jones (1974), Jones & James (1979), James & Sells (1981),
James (1982), James et al. (1988), James & James (1989)
2. Aggregation Bias James (1982), James et al. (1980)
3. Why we need rWG (Within-group agreement)

James (1982), James, Demaree, & Wolf (1984; 1993), George &
James (1993)
4. rWG(J) for multi-item scales

4
James, Demaree, & Wolf (1984), LeBreton, James, & Lindell (2005)
Quotes & Equations
In summarizing Larry James’s contributions to
Multilevel Theory, I’ll use a two-pronged
approach:
1. Quotes
2. Equations
5
Quotes & Equations
In summarizing Larry James’s contributions to
Multilevel Theory, I’ll use a two-pronged
approach:
1. Quotes
2. Equations
rWG  1  (s x2j  E2 )
6
Levels of Analysis
• In social science, hypothetical constructs
reside at multiple levels of analysis (or levels
of aggregation):
– National Level: Culture
– Organizational Level: Organizational Climate,
CEO personality, Strategy
– Team Level: Team efficacy, Norms, Leader style
– Individual Level: Attitude, Personality, Job
Performance, Psychological Climate
7
Levels of Analysis
Organizational
Group
Individual
8
Levels of Analysis
• Individuals are nested within Groups
• Groups are nested within Organizations
• One level can influence another
– Group norms influence individual behavior
– Individual behaviors aggregate to produce
group/team performance
9
Psychological Climate
• Psychological Climate – ‘the meaning an
individual attaches to a work environment’
• Organizational Climate – ‘the aggregated
meaning; i.e., the typical, average, or usual
way people in a setting [work environment]
describe it’
• Schneider (1981, pp. 4-5), as cited by James (1982)
10
Psychological Climate
• Psychological Climate – individual level
construct
• Organizational Climate – group level
construct
11
Psychological Climate
• “… perceptual agreement implies a shared
assignment of psychological meaning, from which
it follows that an aggregate (mean) climate score
provides the opportunity to describe an
environment in psychological terms.”
• “Furthermore, given perceptual agreement, I submit
that a climate construct at the aggregate level is
defined in precisely the same manner as it is at the
individual level.”
• James (1982, p. 221)
12
Psychological Climate
Relationship between organizational climate and
psychological climate:
OC  average(PC)
• PC = psychological climate perception of
person in a group
• OC = organizational climate of the group
13
Psychological Climate
Relationship between organizational climate and
psychological climate:
ng
OC0 g   PCpg ng
p 1
• PCpg = psychological climate perception of
person p in group g
• OC0g = organizational climate in group g
14
Psychological Climate
Relationship between organizational climate and
psychological climate:
PCpg  OC0 g  u pg
• PCpg = psychological climate perception of
person p in group g
• OC0g = organizational climate in group g
• upg = deviation of person p’s individual psych.
climate perception from group g’s org. climate
15
Psychological Climate
James & Jones (1974), reviewed 3 approaches
to conceptualize & measure org. climate:
1) Org.-Level Attribute, Multiple Measures
2) Org.-Level Attribute, Perceptual Measures
3) Indiv.-Level Attribute, Perceptual Measures*
* Introduced the term, “Psychological Climate”
16
James & Jones (1974)
• “Returning to the perceptual definition of organizational
climate, it would seem that the reliance on perceptual
measurement may be interpreted as meaning that
organizational climate includes not only descriptions of
situational characteristics, but also individual differences
in perceptions and attitudes. This is somewhat confusing if
one wishes to employ organizational climate as an
organizational attribute or main effect, since the use of
perceptual measurement introduces variance which is a
function of differences between individuals and is not
necessarily descriptive of organizations or situations.
Therefore, the accuracy and/or consensus of perception
must be verified if accumulated perceptual organizational
climate measures are used to describe organizational
17
attributes (Guion, 1973).” (p. 1103)
Jones & James (1979)
• “The [conceptual] argument for aggregating perceptually
based climate scores (i.e., psychological climate scores)
appears to rest heavily on three basic assumptions: first, that
psychological climate scores describe perceived situations;
second, that individuals exposed to the same set of
situational conditions will describe these conditions in
similar ways; and third, that aggregation will emphasize
perceptual similarities and minimize individual
differences. Based on this logic, it is generally presumed that
empirically demonstrated agreement among different
perceivers implies that these perceivers have experienced
common situational conditions (Guion, 1973; Insel &
Moos, 1974; James & Jones, 1974; Schneider, 1975a),”
• (p. 206).
18
James & Jones (1974)
• “Although this school of thought [from Schneider and others]
assumes that situational and individual characteristics interact
to produce a third set of perceptual, intervening variables,
such an assumption does not mean that perceived climate is
different from an individual attribute. Rather, the intervening
variables are individual attributes which provide a bridge
between the situation and behavior.”
•
(p. 1107)
• So … “Psychological Climate” is born!
19
James (1982)
• “current thinking in climate suggests that the unit of theory
for climate, including organizational climate, is the
individual, and the appropriate unit to select for observation
is the individual. This thinking is based on the view that
climate involves a set of macro perceptions that reflect how
environments are cognitively represented in terms of their
psychological meaning and significance to the individual.”
•
(p. 219)
• So … measuring organizational climate (an
org.-level attribute) involves an individuallevel true score (i.e., psychological climate).
20
James et al. (1988)
• “Shared assignment of meaning justifies aggregation to a
higher level of analysis (e.g., groups, subsystems,
organizations) because it furnishes a way of relating a
construct (PC) that is defined and operationalized at one
level of analysis (the individual) to another form of the
construct at a different level of analysis (e.g., group
climate, subsystem climate, OC). Although the unit of
analysis for the aggregate psychological variable is the
situation (e.g., group, subsystem, organization), the definition
and basic unit of theory remains psychological.”
• (p. 130, from Organizations Do Not Cognize)
21
James & James (1989)
General PC
.85
Leader
Support
.86
Role Stress,
Conflict,
Ambiguity
.77
.81
Job
Autonomy,
Challenge
- PC = Cognitive evaluation of work environment
- See James & Sells (1981), Jones & James (1979)
Group
Warmth &
Cooperat.
22
Psychological Climate
Psych.
Climate
Job
Satisfaction
/Affect
- Reciprocal relationship between PC and Job Satis./Affect
23
- James & Tetrick (1986), James & James (1992)
Psychological Climate
Summary:
• There is a group-level organizational reality
(“the situation”)
• That reality is reflected in individual-level,
psychological perceptions
• The individual-level psychological climate
perceptions are a meaningful locus of theory
• The individual perceptions can be aggregated
to represent a group-level, psychological
24
property [if perceptions are shared]
Aggregation Bias
Aggregation – combining micro-level data so it
can represent the macro-level (typically, by
taking an average of micro-level responses)
• The aggregate of individuals’ scores
represents the group-level construct
25
Levels of Analysis
Organizational
Group
Individual
26
Aggregation
• Ecological fallacy – generalizing group-level
(aggregate) results to the individual level
– Because we know group collectivism is related to grouplevel cooperation, we inaccurately assume individual
collectivism is related to individual cooperativeness.
• Atomistic fallacy – generalizing individuallevel results to the group (aggregate) level
– Because we know indiv. IQ is strongly related to indiv.level job performance, we inaccurately assume group IQ
is strongly related to group performance.
27
Aggregation
The Truth about Aggregates:
• If the individual-level correlation between X and Y
is rindiv. = .3, this does not imply that the group-level
correlation between X and Y is rgroup = .3.
• Likewise, if the group-level correlation between X
and Y is rgroup = .3, this does not imply that the
individual-level correlation between X and Y is
rindiv. = .3.
28
Aggregation
Direction of a correlation (+ or -) can change when we
move from the individual level to the group level.
Within-Group
Correlation
Between-Group
Correlation
Y
X
29
Aggregation
Example) Foreign birth & Illiteracy (Robinson, 1950).
rindiv. = .12; rgroup(states) = -.53
Within-Group
Correlation
Between-Group
Correlation
Y
X
30
Aggregation
Total correlation is a combination of the individuallevel correlation and the group-level correlation.
Within-Group
Correlation
Between-Group
Correlation
rwithin
Y
Total
Correlation
X
rtotal
rbetween
31
Aggregation
• Total correlation is a combination of the
individual-level (within) correlation and the
group-level (between) correlation.
rtotal  f (rbetween, rwithin )
32
Aggregation
• Specifically,
rtotal  rbetween ( x y )  rwithin (1   )(1   )
2
x
2
y
• rtotal
= overall X-Y correlation, ignoring
group membership
• rbetween = between-groups X-Y correlation
• rwithin = within-groups X-Y correlation
• x2  SSbetween SStotal (from ANOVA; DV= X, IV= group)
[like R2; variance in X accounted for by group membership, then inflated by the
33
2
unreliability of group means; i.e.,  x  ICC(1) / ICC(2) .]
Aggregation
• For example, suppose
• rbetween = -.45 = between-groups X-Y correlation
• rwithin = .20 = within-groups X-Y correlation
2

• x
= .64 (from ANOVA; DV= X, IV= group)
2

• y
= .81 (from ANOVA; DV= Y, IV= group)
Then …
rtotal  rbetween ( x y )  rwithin (1   )(1   )
2
x
2
y
rtotal  .45( .64 .81)  .20 (1 .64)(1 .81)  .27
34
Aggregation
• For example, suppose
• rbetween = -.45 = between-groups X-Y correlation
• rwithin = .20 = within-groups X-Y correlation
2

• x
= .64 (from ANOVA; DV= X, IV= group)
2

• y
= .81 (from ANOVA; DV= Y, IV= group)
Then …
rtotal  rbetween ( x y )  rwithin (1   )(1   )
2
x
2
y
rtotal  .45( .64 .81)  .20 (1 .64)(1 .81)  .27
35
Aggregation
Total correlation is a combination of the individuallevel correlation and the group-level correlation.
Within-Group
Correlation
Between-Group
Correlation
rwithin
Y
Total
Correlation
X
rtotal
rbetween
36
Aggregation
Implications:
• Even if total correlation between X and Y
(rtotal) is statistically significant,
– rwithin might not be
– rbetween might not be
* Many studies in top journals report total relationships
between variables, while ignoring nesting/
nonindependence (e.g., different groups, different
jobs, different supervisors). Considering levels of
37
analysis could potentially change the results!
Aggregation
Implications:
• So-called “aggregation bias” – when rbetween
is larger than rtotal
– Only occurs if rbetween happens to be larger than
rwithin
rtotal  rbetween ( x y )  rwithin (1   )(1   )
2
x
2
y
38
Aggregation Bias
Implications:
• Don’t look at rtotal to draw inferences about
rwithin!
• Don’t look at rtotal to draw inferences about
rbetween!
rtotal  rbetween ( x y )  rwithin (1   )(1   )
2
x
2
y
• See James (1982) and James, Demaree, & Hater (1980),
who applied similar formulae to estimate bias in both 2 and
corr.’s between aggregated situational (OC) and individual
39
difference variables.
Aggregation Bias
Summary:
• When we aggregate individual-level
measures (e.g., psychological climate) to
represent organizational attributes (e.g.,
organizational climate), then all the
theoretical and empirical relationships can
change.
• Aggregation of the same measures can create
a different construct!
40
Why We Need rWG
• Justifying Aggregation
• “… organizational climate is the overall meaning derived
from the aggregation of individual perceptions of a work
environment (i.e., the typical or average way people in an
organization ascribe meaning to that organization) (James,
1982; Schneider, 1981). Thus, organizational climate can be
viewed as the outcome of aggregating individuals’
psychological climates. The important caveat is that these
psychological climates are shared in order to make the
inference that an organizational climate exists.”
•
James et al. (2008, pp. 15-16)
41
Why We Need rWG
Group-Level Consensus Constructs
• In measuring group consensus constructs,
agreement and reliability are tools used to
justify aggregation of individual-level
responses to the group level
• Agreement and reliability help us gauge how
well the average across individual responses
represents the group.
42
Why We Need rWG
Group-Level Consensus Constructs
Organizational
Climate
(average)
Psych. Climate,
Person #1
Psych. Climate,
Person #2
Psych. Climate,
Person #3
43
Why We Need rWG
Overview
• Aggregation/Composition Models
– Chan (1998)
– Kozlowski & Klein (2000)
• Agreement
– rWG family of indices
• Reliability
– ICC(1)
– ICC(2)
See Bliese, 2000
44
Why We Need rWG
• Aggregation/Composition Models
– Chan (1998)
– Kozlowski & Klein (2000)
• Both typologies include consensus models
– Use the mean of individual responses to
represent the group-level construct
– Assume isomorphism (James, 1982)
– Require high within-group agreement
45
Why We Need rWG
• Within-Group Agreement – degree to which
ratings from individuals are interchangeable
– Agreement-based tests reflect degree to which
raters provide essentially the same rating
– Three dominant indices designed to assess withingroup agreement:
• James et al.’s (1984) rWG(J)
*
r
• Lindell et al.’s (1999) WG ( J )
• Burke, Finkelstein, & Dusig’s (1999) AD index
46
George & James (1993)
• “The key statistical test of the appropriateness of
aggregation to the group level of analysis is that there is
within-group agreement on the variable in question. If there
is agreement within groups on the theorized group-level
variable, then the aggregate may be used in subsequent
analyses.”
• … agreement within a group is not conditional on betweengroups differences. For example, in a scenario that
Yammarino and Markham portray, in which all members in
each group have the same moderately high score, both
agreement and aggregation may be justified provided that
aggregation to the group level was theoretically based.
However, there would be no group effect inasmuch as the
group means do not vary under these conditions.”
47
• (p. 799)
Why We Need rWG
• Within-Group Agreement
– For single items:
rWG  1  (s x2j  E2 )
– s x2j = observed variance of single item
–  = theoretical null variance (represents “zero agreement”)
2
E
– rWG = “1 - observed variance over expected variance”
48
Why We Need rWG
Summary:
• Under consensus composition models (with
isomorphism across levels), within-group
agreement is needed to justify aggregation.
• Within-group agreement is even more essential
than ICC(1) and ICC(2), both of which depend
upon between-group variance.
• Within-group agreement = shared
psychological meaning!
• rWG is the key to measuring group-level
psychological properties!
49
rWG(J) for Multi-Item Scales
rWG(J) is NOT the same as rWG!
• rWG = for single items
rWG  1  (s x2j  E2 )
• rWG(J) = for multiple-item climate scale
rWG ( J ) 
J [1  (s
J [1  (s
2
xj
2
xj
 )]
2
E
 )]  (s
2
E
2
xj
 )
2
E
50
rWG(J) for Multi-Item Scales
• Within-Group Agreement (James et al., 1984)
– For multiple
items:
rWG ( J ) 
J [1  (s
J [1  (s
2
xj
2
xj
 )]
2
E
 )]  (s
2
E
2
xj
 )
2
E
– J = number of items
– s x2 = mean of observed item-level variances
j
2
–  E = theoretical null variance (represents “zero agreement”)
†
Can be derived without Spearman-Brown (LeBreton et al., 2005) 51
rWG(J) for Multi-Item Scales
• Three Issues with James et al.’s (1984) rWG(J) :
1) J = number of items
(is rWG(J) an index of agreement, reliability, or both?)
2) s
2
xj
= mean of observed item-level variances
2

3) E = theoretical null variance (represents “zero agreement”)
(addressed by LeBreton & Senter, 2008)
52
James et al. (1993)
• Describing whether rWG(J) is an index of agreement vs.
reliability:
• “Kozlowski and Hattrup are also correct in stating that our
intention was to suggest a measure of agreement, and not
consistency [reliability], and that rWG is an estimator of
agreement. However, what cannot be done, at least not the
way things are presently set up, is to follow Kozlowski and
Hattrup's recommendation to sever all ties between interrater
reliability and rWG and to treat rWG as strictly a measure of
agreement with, in effect, no ties to classic measurement
theory. It is not possible to follow this recommendation
because rWG is currently derived in terms of classic
measurement theory as an interchangeability (agreement)
index of interrater reliability.”
53
• (p. 306)
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
– J = number of items
• What happens to rWG(J) as number of items (J) increases?
rWG ( J ) 
J [1  (s
J [1  (s
2
Xj
2
Xj
 )]
2
E
 )]  (s
2
E
2
Xj
 )
2
E
54
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
– J = number of items
• What happens to rWG(J) as number of items (J) increases?
rWG ( J ) 
J [1  (s
J [1  (s
2
Xj
2
Xj
 )]
2
E
 )]  (s
2
E
2
Xj
 )
2
E
55
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
1
0.9
James et al.'s rWG(J)
0.8
0.7
0.6
0.5
0.4
mean_itemvar = 0.2
0.3
mean_itemvar = 0.6
0.2
mean_itemvar = 1
mean_itemvar = 1.4
0.1
mean_itemvar = 1.8
0
1
3
7
Number of Items (J)
11
20
56
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
1
rWG(J) = .7
0.9
James et al.'s rWG(J)
0.8
0.7
0.6
s x2 j
0.5
0.4
mean_itemvar = 0.2
0.3
mean_itemvar = 0.6
0.2
mean_itemvar = 1
mean_itemvar = 1.4
0.1
mean_itemvar = 1.8
J
0
1
3
7
Number of Items (J)
11
20
57
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
– To get a large rWG(J) (James et al., 1984), simply
add more items to your scale!!
– Even under near-maximal within-group variance,
2
s
[ x = 1.8] rWG(J) = .7 when the scale has J = 20 items!
j
58
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
– s x2 = mean of observed item-level variances
j
• What is it?
• First calculate the within-group variance of each item,
2
• Then average these variances across items, s x
j
59
s x2j
rWG(J) for Multi-Item Scales
• s x2 j = mean of observed item-level variances
Compare s
2
xj
vs.
2
x
s (scale score variance):
Scale score variance sx2 =>
• First calculate mean across items (i.e., scale score), x
2
• Then take the within-group variance of scale score, sx
s  s [ J   ( J  1)]
2
xj
•
s
2
xj
2
x
is almost always larger than s
2
x
60
rWG(J) for Multi-Item Scales
• Why is s x2 j almost always larger than scale
score variance sx2 ?
Psych. Climate
True Score
Variance
PC Item 1
d1
PC Item 2
d2
PC Item 3
d3
PC Item 4
d4
Item Unique
61
Variance
rWG(J) for Multi-Item Scales
• Why is s x2 j almost always larger than scale
score variance sx2 ?
Psych. Climate
True Score
Variance
s
2
x
s
PC Item 1
d1
PC Item 2
d2
PC Item 3
d3
PC Item 4
d4
2
xj
Item Unique
62
Variance
rWG(J) for Multi-Item Scales
• s x2 j = mean of observed item-level variances
Compare s
s
2
xj
vs.
2
x
s (scale score variance):
2
, Scale score variance => zooms in on true, construct-level
x
variance within-groups
vs.
s x2 j , Mean of observed item-level variances => includes true
construct-level variance + item-specific variance
s  s [ J   ( J  1)]
2
xj
2
x
63
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
– s x2 = mean of observed item-level variances
j
• It would be much clearer to just base withingroup agreement on the within-group variance
in scale scores, s x2j rather than on the average of
item-level within-group variances, s 2 .
xj
64
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
–  = theoretical null variance (represents “zero agreement”)
2
E
2
 EU
 ( A2 1) 12
– E.g., Uniform null distribution
– A = number of response options (e.g., A = 5 for a 5point Likert scale);
2
 EU
 (52 1) 12  2
65
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
 E2 = theoretical null variance
– Can alternatively use a non-uniform expected null
variance for rWG(J) (see James et al., 1984; LeBreton
& Senter, 2008)
• Normal null dist.
• Skewed null dist.
• Maximum null dist. (Brown & Hauenstein, 2005)
66
rWG(J) for Multi-Item Scales
• Issues with James et al.’s (1984) rWG(J) :
 E2 = theoretical null variance
– Can alternatively use an Average Deviation index
(AD; average absolute value deviation from mean
or median; Burke et al., 1999).
• Less vulnerable to outliers
• Still compared against arbitrary cutoff, AD < A/6
2
• Still includes item-specific variance (like s x j )
67
rWG(J) for Multi-Item Scales
Summary:
• Whereas rWG is a great index of standardized
within-group agreement,
rWG(J) reflects 3 sources of variance:
a) within-group variance in psych. climate/latent
construct true scores (“shared meaning”), plus
b) item-specific variance (in s x2 ), and
c) number of items (J).
j
• It would be better to use an agreement index
that homes in on (a) within-group variance in
psych. climate/latent construct true scores 68
(“shared psychological meaning”).
Within-Group Agreement
• So what is the alternative?
69
Within-Group Agreement
• What if we still want to assess within-group
agreement (“shared psychological meaning”)
with a multi-item climate scale?
• First, conceptualize the degree of “shared
psychological meaning” at the latent theoretical
level (James, 1982; James et al., 1988), but use a format
similar to rWG:
 (2psych.c limate)
 WG  1 
2
E
70
Within-Group Agreement
• WG does not increase as you add items to the
climate scale (i.e., it is a pure parameter of
within-group agreement, not reliability)
 WG  1 

2
( psych.c limate)
2
E

71
Within-Group Agreement
• How well does each of the following withingroup agreement indices estimate WG?
(“shared psychological meaning”)
1) James et al. (1984)
rWG ( J ) 
J [1  (s X2 j  E2 )]
J [1  (s X2 j  E2 )]  (s X2 j  E2 )
*
2
2

1

(
s

2) Lindell et al. (1999) rWG
(J )
X
E)
j
2
2
r

1

(
s

3) Simple index: WG( )
x
E)
72
Within-Group Agreement
• Comparison of rWG(J), rWG(J)*, and rWG()
1
0.9
J =5 items,
WG = .90
0.8
rWG index
0.7
0.6
PSI_WG
0.5
rWG(J)
0.4
rWG(J)*
0.3
rWG(alpha)
0.2
0.1
0
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
True Within-Group Agreement (WG)
1
Newman &
Sin, 2008
73
Within-Group Agreement
• Conclusions:
1) All within-group agreement indices are very
strongly correlated.
2) rWG(J) can notably overestimate within group
agreement, especially when rWG(J) > .7.
3) rWG() seems to offer a closer estimate of
within group agreement (slight underestimate)
4) One could also directly estimate WG .
74
Within-Group Agreement
• How well does each of the following withingroup agreement indices estimate WG?
(“shared psychological meaning”)
J =5 items,
WG = .90
1) When WG = .60:
rWG(J)= .75; rWG(J)*= .38, rWG() = .56
2) When WG = .65:
rWG(J)= .81; rWG(J)*= .46, rWG() = .61
3) When WG = .70:
rWG(J)= .85; rWG(J)*= .53, rWG() = .67
75
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
76
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
77
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
78
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
79
Overview
Group-Level Psychological Properties?
1. Psychological Climate
 Group-Level vs. Individual-Level Constructs
2. Aggregation Bias
3. Why we need rWG (Within-group agreement)
 Justifying Aggregation
4. rWG(J) for multi-item scales
 Agreement vs. Reliability
80
Thank You Larry!
OC
(average)
PC
Person #1
PC
Person #2
rWG  1  (s
2
xj
PC
Person #3
 )
81
2
E