Identifying suppression effect:
An application of meta-analytic technique
Introduction
Meta-analysis was proposed by Glass (1976) to investigate statistically the relationship
between two variables across studies. Suppressor is a possible cause in contradictory
relationship of two variables. Suppressor was first proposed by Horst (1941). Tzelgov and
Henik (1991) extended the definition of suppressor as: the variable that diminished correlation
between independent and dependent variable in three-variate case. Here I sketched an
example. Some scholars believe that team diversity will have positive impact on team
performance via effect of information diversity. However, affective conflict, which is also
positively related to team diversity, may have negative impact on performance. In summary,
the total effect of diversity on performance will be diminished or even become negative.
Affective conflict is suppressor between diversity and performance.
Traditionally, scholars test suppressor by conducting multiple regression. Holling (1983)
proposed a method by using general linear model. In this paper, I proposed a method by using
meta-analytical technique.
Suppression effect and correction for suppression
In this paper, I define suppression effect (SE) as “the amount of decline in linear
correlation of two variables by suppressor”. According to Cohen et al.’s (2003: 75-78) models
of partial redundancy in calculating direct and indirect effect, the relationship among
suppressor, independent, and dependent variables in this paper can be divided into two
category of models: (1) correlative model: independent variable and suppressor are mutually
correlated, and both are causes of dependent variable; and (2) mediative model: independent
variable is cause of suppressor, and both are causes of dependent variable (Fig. 1). The
suppression effects of these two models are different. The formulae of suppression effects and
effect sizes corrected by SE are as follow:
rcc = (rxy – rxs * rsy) / (1 – rxs2)
(1)
2
2
SEcc = rcc – rxy = (rxy * rxs – rxs * rsy) / (1 – rxs )
(2)
rcm = rxy – rxs * rsy
(3)
SEcm = rcm – rxy = – rxs * rsy
(4)
While rcc is corrected effect size of correlative model, SEcc is SE of corrective model, rcm is
corrected effect size of mediative model, SEcm is SE of mediative model, rxy, rxs, and rsy are
zero-order correlation coefficients among independent, suppressor, and dependent variables,
respectively. In this paper, we call these four formulae as CFS formulae (Correction for
suppression formulae).
Hold all others constant, rcc will be greater than rcm because (1 – rxs2) is lesser than 1.
According to the same reason, SEcc will be greater than SEcm.
Meta-analytic evidence of suppression effect
The evidence that there exist suppression effect is mainly based on: the effect sizes
corrected by SE are significantly greater than uncorrected effect sizes. I illustrate this
technique by showing the procedure.
Professor Frost wants to detect the suppressor SUP between independent variable IND
and dependent variable DEP by conducting meta-analysis. He had corrected 300 empirical
articles that included IND and DEP. Among these articles, 140 articles measured IND, DEP,
and SUP simultaneously, while the other 160 measured IND and DEP and did not measure
SUP. The 140 effect sizes corrected by CFS formulae were named as Group Ic, while the 140
uncorrected effect sizes as Group Iu. The other 160 effect sizes, which can’t be corrected
because that suppressor was not measured, were called Group N.
He can provide two kinds of evidence to support the existence of SE. The first is that the
paired t-test of SEs of Group Ic showed significantly different from zero. This evidence is
more robust than the other one and can be named as Prime evidence.
The second evidence is provided by testing 140 rcc (or rcm) of Group Ic and 160 ri of
Group N with two independent samples t-test. If the former is significantly greater than the
later, then the existence of SE was supported. In providing this evidence, the heterogeneity in
sample sizes and/or variances between two samples will elevate the possibility of type I error
and the variation in effect sizes themselves will elevate the possibility of type II error in two
sample t-test. Though the variation in effect sizes can be diminished after correction of
sampling error (see discussion), the degree of improvement in type II error is unknown.
Because of these shortages, we called this evidence as spurious evidence. Please see Table 1
for details.
Discussion
The testing of Prime evidence. Group Ic and Iu are highly homogeneous paired samples.
Suppose that we conduct two independent samples t-test to test the difference of these two
groups. The standard error will become larger than that of paired t-test because of variation in
effect sizes, and sample size, which reflected on degree of freedom, will become doubled. The
probability of type II error will be increased because that the advantage of the later always
can’t overcome the shortage of the former. Therefore, a paired t-test of Group Ic and Iu (i.e.,
SEcc or SEcm) is preferred in this case. As for Group N, the only way is to compare it with
Group Ic by conducting two samples t-test because of lack of suppressor-related data.
Sample size and alternative evidence. Prime evidence should be a better one. However,
the problem of low power and high type II error will emerge if sample sizes of Group Ic and
Iu are small. We proposed here an alternative method: comparing Group Ic with the
combination of Group Iu and N by using two independent samples t-test. This method will
increase heterogeneity in sample sizes but decrease heterogeneity in variances of two samples.
Thus, the impact on type I error is ambiguous. However, statistical power will be increased
and the possibility of type II error will be decreased because of increased sample sizes. This
method is, however, against the assumption to conduct two samples t-test because that these
two samples are non-independent, and is not recommended to be main evidence. We called
this evidence as alternative evidence.
Measurement and sampling error. Measurement error will diminish the correlation
between two variables. To restitute this correlation, some scholars that interested in
meta-analysis argued the necessity of correction for attenuation by reliabilities (cf. Hunter &
Schmidt, 1990). The general formula is as follow:
rcxy = rxy / (rxx * ryy)1/2
(5)
while rcxy is corrected effect size, rxy is zero-order correlation coefficient of these two variables,
and rxx and ryy are reliabilities of these two variables, respectively. According to the essence of
correction for attenuation, we also argued that the correlation coefficients in calculating
suppression effects and suppressor-corrected coefficients through four CFS formulae should
be corrected firstly by reliabilities.
It is worthy to mention that the impacts of reliabilities of different variables to these four
formulae are not the same. The reliability of suppressor is the most influential one.
Hunter, Schmidt, & Jackson (1982) argued that the variation of relationship between two
variables may come from substantial difference of populations and sampling error, and that to
correct sampling error in meta-analysis is necessary to estimate actual differences in
population correlations. According to this concern, we also argued that we should correct
sampling error before conducting t-test because that the correlations and SE among suppressor,
independent, and dependent variables may also vary because of sampling error and not actual
differences.
The probability of significant t-test of all the three evidence this paper proposed will be
increased by correcting of sampling error because that standard errors will be decreased.
Moderator in meta-analysis. In comparing with testing suppressor by regression or with
testing moderator by meta-analysis, the ways to approach moderator by the method proposed
in this paper is more attractive. We may investigate the moderation effect of “whether SUP
was measured or not” in comparison of Group Iu and N. We may seek for variables that
influenced on SEi of Group I directly if sample size is large enough. Further, we may also test
the interaction between certain moderator and “whether SUP was measured or not”.
Conclusion
This paper proposed a method that different from multiple regression and general linear
model to test the existence of suppressor, and expand the utility of meta-analysis. We wish
that scholars could pay more attention on the impact of possible suppressor.
References
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. 2003. Applied multiple regression/correlation
analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum
Associates.
Glass, G. V. 1976. Primary, secondary, and meta-analysis research. Educational Researcher, 5:
3-8.
Holling, H. 1983. Suppressor structures in the general linear model. Educational and
Psychological Measurement, 43: 1-9.
Horst, P. 1941. The role of the predictor variables which are independent of the criterion.
Social Science Research Council, 48: 431-436.
Hunter, J. E., Schmidt, F. L. 1990. Methods of meta-analysis: Correcting error and bias in
research findings. Newbury Park, CA: Sage.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. 1982. Meta-analysis: Cumulating research
findings across studies. Beverly Hills, CA: Sage.
Tzelgov, J., & Henik, A. 1991. Suppression situations in psychological research: Definitions,
implications, and applications. Psychological Bulletin, 109: 524-536.
Figure 1: left side are correlative models and right side are mediative models
X
X
+
Y
+
X
X
Y
S
-
S
+
-
Y
+
-
S
+
+
Y
-
+
+
S
Table 1:
Group I
Group Iu
Group Ic
effect size
effect size suppression
effect (rc - r)
study No.
Group N
Group N + Iu
effect size
effect size
study 1
r1
rc1
SE1
r1
study 2
：
r2
：
rc2
：
SE2
：
r2
：
：
：
：
：
：
study 140
r140
rc140
SE140
r140
study 141
r141
r141
study 142
：
r142
：
r142
：
：
：
：
study 300
r300
r300
Mean or weighted mean
MrIu
MrIc
MSE
MN
MA
Standard deviation or
corrected standard deviation
SDIu
SDIc
SDSE
SDN
SDA
Sample size
140
140
140
160
300
Prime evidence:
Conducting paired t-test with SEs of Group I
Spurious evidence:
Conducting two samples t-test with effect sizes of Group Ic and N
Alternative evidence:
Conducting two samples t-test with effect sizes of Group Ic and (N + Iu)