Testing the Causal Effects of
Social Capital: Design for a
Cluster-Randomized Field Trial
Adam Gamoran and Ruth N. López Turley
University of Wisconsin-Madison
Social Capital:
Conceptual and Causal Ambiguity

Social capital is one of the most popular terms
in social science today

Viewed as the source of many positive outcomes


Decline of social capital is seen as responsible for
many social ills


Test scores, school completion, social adjustment,
mental and physical health
Crime, apathy
Causal role of social capital is ambiguous
Social Capital:
Conceptual and Causal Ambiguity

Concept of social capital is also ambiguous





Relations of trust, mutual expectations, and
shared values embedded in social networks
Not possessed by individuals
Resides in the relationships individuals have with
one another
Individuals can draw upon social capital in their
networks
Social capital facilitates the flow of information
and the development and enforcement of norms
Concepts of Social Capital

Qualities of social networks that signify social
capital: How do we know if social capital is
present?

Intergenerational closure
Intergenerational Closure
Source: Coleman,
Am. J. Soc., 1988
Concepts of Social Capital

Qualities of social networks that signify social
capital: How do we know if social capital is
present?


Intergenerational closure
Trust



Network members rely on one another
Facilitates sharing of norms and information
Shared expectations

Also facilitates supporting norms and distributing
information
Concepts of Social Capital



Contrary to Coleman (1988), we do not define
social capital by its function
Contrary to Portes (1998), we view social
capital as a collective rather than as an
individual attribute
We follow Sampson et al. (1999):
“…social capital for children refers to the
resource potential of personal and
organizational networks…”
Domains of Social Capital



Parent-school relationships
Parent-parent relationships
Parent-child relationships
Parent-school Parent-parent Parent-child
Trust
Shared
expectations
Intergenerational
closure
Mechanisms of Social Capital


For young children, social capital operates
through their parents
Two primary mechanisms

Social support


Parents who feel more connected to others have better
access to information and are better able to establish
and enforce norms with their children
Social control

Parents’ positive social networks offer collective
socialization of children
Social Capital and Inequality



Unequal social capital contributes to unequal
child development
Among U.S. Latinos, social capital within
family networks is high, but parent-school
social capital is low
Building family-school social capital may
enhance child outcomes particularly for
Latinos – the focus of our empirical analysis
The Causal Role of Social Capital



Many studies have tested the relation between
social capital and child outcomes
Most rely on longitudinal data
Nonetheless, causal direction is ambiguous


Does social capital foster school success, or do
stronger social ties emerge in communities that
have more effective schools?
Some group norms can negatively affect child
outcomes!
The Causal Role of Social Capital

Survey research may overestimate effects of social
capital (Mouw, 2006)



Endogeneity: Group members influence one another at the
same time
Unobserved selectivity: unmeasured conditions lead to
both common memberships and common outcomes
Statistical efforts to resolve these causality issues
rely on questionable assumptions

E.g., effects are unbiased net of control variables
The Causal Role of Social Capital

An experimental design offers a more
rigorous approach to testing social capital’s
causal role


No unobserved selectivity: Assignment to
“treatment” is random
Avoid endogeneity problem through multilevel
assessment of social capital effects
The Causal Role of Social Capital

Conditions for an experimental assessment of social
capital effects:
1.
2.
3.
4.
5.
An intervention that manipulates social capital
Random assignment to treatment and control
Random assignment of groups of individuals (because
social capital is an attribute of groups, not individuals)
Tools for measuring social capital and outcomes
Statistical methods suitable for analysis of a clusterrandomized trial
1. An Intervention that Manipulates
Social Capital

FAST: Families and Schools Together


A multi-family group prevention program
Implemented in three stages



Outreach to parents
8 weeks of multi-family group meetings
2 years of monthly follow-up meetings led by parents
1. An Intervention that Manipulates
Social Capital

Elements of FAST



Led by a parent-professional partnership
Culturally representative and adapted
Research-based activities






Family meal
Group singing
Family games
Parent support/ children’s time
One-to-one responsive play
Closing circle
1. An Intervention that Manipulates
Social Capital

Prior research on FAST



4 previous randomized trials have documented
positive outcomes for children’s social and
academic outcomes
These studies have occurred at the individual or
classroom levels
School-wide, “multi-hub” FAST is likely to have
even more powerful effects
1. An Intervention that Manipulates
Social Capital

Prior research on FAST

FAST builds social capital




Parent-school: Reduces alienation from school
authorities, and increases comfort level
Parent-parent: Reduces isolation of parents by
creating a parent support group
Parent-child: Improve relationship through one-onone responsive play
Particularly valuable for immigrant communities
Conceptual Model
2. Random Assignment to Treatment
and Control: Experimental Design

Research Sites




San Antonio, TX: A large, long-standing Latino
populations (51% of students)
Milwaukee, WI: A rapidly growing Latino population
(21% of students)
Experience with FAST, community agencies
available to implement
Agreed to implement FAST in treatment schools, not
in control schools


Subject to agreement of principals and teachers
They love FAST, this won’t be a problem
3. Random Assignment of Groups of
Individuals: Experimental Design


26 schools from each district (13 treatment
and 13 control), total of 52 schools
All first-grade families will be invited to
participate


We anticipate 75% participation rate, 20%
attrition rate = 60% long-term follow-up
Three years of data collection (grades 1 to 3)
3. Random Assignment of Groups of
Individuals: Experimental Design

How did we decide on 52 schools?

Power analysis
Power Analysis: Assumptions






Power criterion (1 – β) = .80
Probability of Type I error () = .05
Within-school sample size (n) = 60
Effect size () =.25
Intraclass correlation () = .10
Covariate correlation (r) = .40-.60
Power Analysis: Software
http://sitemaker.umich.edu/group-based/optimal_design_software
Power Analysis
Power Analysis: Conclusion

Under reasonable assumptions, a sample of 52
schools will provide sufficient power to detect
the effects of social capital, if they exist.
4. Tools for measuring social capital
and outcomes

Outcomes



Parent and teacher ratings of child social skills
and problem behaviors (grades 1 and 3)
Teacher ratings of child academic competence
High-stakes standardized tests of reading and
mathematics
4. Tools for measuring social capital
and outcomes

Social capital

Parent social capital questionnaire

Our only pre-intervention measure



Not really needed for experimental design, but of interest in
its own right
Follow-up measures in the spring of grades 1 and 3
Key sources: Bryk and Schneider (2002);
McDonald and Moberg (2002)
Parent Social Capital Questionnaire

Parent-school trust, shared expectations
Parent Social Capital Questionnaire

Parent-parent closure, trust, shared expectations
Parent Social Capital Questionnaire

Parent-child trust, shared expectations
4. Tools for measuring social capital
and outcomes

Social capital





Parent Involvement in School Questionnaire
Indicators of trust and shared expectations in
parent-school and parent-child relationships
Separate forms with parallel questions from
parent and teacher perspectives
Source: Shumow, Vandell, and Kang (1996)
Completed by teachers and parents at the end of
grades 1 and 3
4. Tools to measure social capital and
outcomes: Other variables as indicated
5. Statistical methods suitable for
analysis of cluster-randomized trial

This study relies on place-based random
assignment




CRT: Cluster-randomized trial
Randomization is at the aggregate level
Well suited to contextual investigations
Must assess the intervention at the level at
which randomization occurs
5. Statistical methods suitable for
analysis of cluster-randomized trial

A multilevel model is the appropriate
statistical approach to analysis of CRT




Captures variability both at the level of the cluster
and within clusters
In our case: students within schools
Treatment is at the level of the school
Theoretically, social capital is also at the level of
the school

We allow for individual-level variation
5. Statistical methods suitable for
analysis of cluster-randomized trial


School-level control variables reduce
variation between schools, permit more
precise treatment effects
Individual-level background controls also
increase precision

More importantly, multilevel interactions permit
estimation of differential treatment effects
Multilevel Models: Linear Outcomes
Level 1.
Yij = ß0j + ß1j(SEX)ij + ß2j(LATINO)ij +
ß3j(BLACK)ij + ß4j(POVERTY)ij + rij
Level 2.
ß0j = γ00 + γ01(MEAN PRIOR ACH)j +
γ02(PERCENT POVERTY)j + γ03(PERCENT LATINO)j +
γ04(PERCENT BLACK)j + γ05(FAST)j + γ06(CITY)j +
γ07(PERCENT LATINO x FAST)j + γ08(PERCENT
BLACK x FAST)j + u0j
Level 2.
ß2j = γ20 + γ21(FAST)j + γ22(CITY)j + u2j
ß3j = γ20 + γ21(FAST)j + γ22(CITY)j + u3j
ß4j = γ20 + γ21(FAST)j + γ22(CITY)j + u4j
5. Statistical methods suitable for
analysis of cluster-randomized trial


By adding social capital to the model, we test
whether social capital accounts for the effects
of FAST on child outcomes
Main focus is on school-level effects
Multilevel Models: Linear Outcomes
Level 1.
Yij = ß0j + ß1j(SEX)ij + ß2j(LATINO)ij +
ß3j(BLACK)ij + ß4j(POVERTY)ij +
ß5j(SOCIAL CAPITAL)ij + rij
Level 2.
ß0j = γ00 + γ01(MEAN PRIOR ACH)j +
γ02(PERCENT POVERTY)j + γ03(PERCENT LATINO)j +
γ04(FAST)j + γ05(MEAN SOCIAL CAPITAL)j +
γ06(CITY)j + u0j
5. Statistical methods suitable for
analysis of cluster-randomized trial

Additional challenges

Uncommon measures: Different tests in Texas
and Wisconsin


Linking strategy, corrected for unreliability
Examine probability of reaching the proficiency
threshold rather than test score
5. Statistical methods suitable for
analysis of cluster-randomized trial

Additional challenges

Bias in social capital effects


FAST effects will be estimated without selectivity bias
Social capital effects will also be estimated without selectivity
bias if they derive only from FAST


This is probably not the case
If social capital occurs independently of FAST, an omitted
variable may affect social capital and child outcomes



Use pre-FAST measure to check
Use FAST as an instrument for social capital
Control for pre-FAST social capital
5. Statistical methods suitable for
analysis of cluster-randomized trial

Additional challenges

Bias in social capital effects

Differential non-response by treatment and control
parents


Consent will be obtained prior to randomization
Follow up a random subsample of non-respondents with
home visits
5. Statistical methods suitable for
analysis of cluster-randomized trial

Additional challenges

Fidelity of implementation

Implementation study




Implementation checklist
Interviews, focus groups with parents and teachers
Including interviews with 2 non-participating parents in each
treatment school
Qualitative data will provide more nuanced insights
on the mechanisms through which FAST affects (or
does not affect) child outcomes
Conclusions



The term “social capital” has reflected many
different ideas in different writings
Causal ambiguity has been a consistent
limitation of social capital research
By manipulating social capital experimentally,
we aim to provide a more persuasive test of
social capital effects
References







Bryk, A. S., & Schneider, B. L. (2002). Trust in schools: A core resource for
improvement. New York: Russell Sage Foundation.
Coleman, J. S. (1988). Social capital in the creation of human capital. American
Journal of Sociology, 94(Suppl.), S95–S120.
McDonald, L., & Moberg, D. P. (2002). Social relationships questionnaire.
Madison, WI: FAST National Training and Evaluation Center.
Mouw, T. (2006). Estimating the causal effects of social capital: A review of recent
research. Annual Review of Sociology, 32, 79–102.
Portes, A. (1998). Social capital: Its origins and applications in modern sociology.
Annual Review of Sociology, 24, 1–24.
Sampson, R. J., Morenoff, J. D., & Earls, F. (1999). Beyond social capital: Spatial
dynamics of collective efficacy for children. American Sociological Review, 64(5),
633–660.
Shumow, L., Vandell, D. L., & Kang, K. (1996). School choice, family
characteristics, and home-school relations: Contributors to school achievement?
Journal of Educational Psychology, 88, 451–460.
Further Reading on ClusterRandomized Trials





Bloom, H. S. (2006). Learning more from social experiments: Evolving
analytic approaches. New York: Russell Sage Foundation.
Bloom, H. S., Bos, J. M., & Lee, S. W. (1999). Using cluster random
assignment to measure program impacts: Statistical implications for the
evaluation of education programs. Evaluation Review, 23, 445–469.
Borman, G. D., Slavin, R. E., Cheung, A., Chamberlain, A., Madden, N., &
Chambers, B. (2005). Success for All: First-year results from the national
randomized field trial. Educational Evaluation and Policy Analysis, 27(1),
1–22.
Boruch, R., May, H., Turner, H., Lavenberg, J., Petrosino, A., & de Moya,
D. (2004). Estimating the effects of interventions that are deployed in many
places: Place-randomized trials. American Behavioral Scientist, 47, 608–
633.
Raudenbush, S. W. (1997). Statistical analysis and optimal design for
cluster randomized trials. Psychological Methods, 2, 173–185.
Further Reading on FAST





Abt Associates. (2001). National evaluation of family support programs: Vol. B. Research
studies: Final report. Cambridge, MA: Author. Retrieved February 12, 2007, from
http://www.abtassoc.com/reports/NEFSP-VolB.pdf
Kratochwill, T. R., McDonald, L., Levin, J. R., Young Bear-Tibbetts, H., & Demaray, M. K.
(2004). Families and Schools Together: An experimental analysis of a parent-mediated multifamily group intervention program for American Indian children. Journal of School
Psychology, 42, 359–383.
McDonald, L., Moberg, D. P., Brown, R., Rodriguez-Espiricueta, I., Flores, N., Burke, M. P.,
et al. (2006). After-school multifamily groups: A randomized controlled trial involving lowincome, urban, Latino children. Children and Schools, 18, 25–34.
U.S. Office of Juvenile Justice and Delinquency Prevention. (2006). Families and Schools
Together (FAST). In U.S. Office of Juvenile Justice and Delinquency Prevention, OJJDP
model programs guide. Retrieved February 11, 2007, from
http://www.dsgonline.com/mpg2.5/TitleV_MPG_Table_Ind_Rec.asp?ID=459
U.S. Substance Abuse and Mental Health Services Administration. (2005). Families and
Schools Together (FAST). In U.S. Substance Abuse and Mental Health Services
Administration, SAMHSA model programs: Effective substance abuse and mental health
programs for every community. Washington, DC: Author. Retrieved February 11, 2007, from
http://www.modelprograms.samhsa.gov/pdfs/Details/FAST.pdf