Social Settings and
Randomized Cluster Trials:
Journey of the William T. Grant
Foundation
Edward Seidman
Event: National and International Perspectives on Place-Randomized Trials in Education
Location: Institute for Human Development and Social Change, New York University
Date: Friday, October 3rd, 2008
William T. Grant Foundation www.wtgrantfoundation.org
Overview of the Presentation
• Shifting Paradigms
• WTGF’ RFPs and Research Interests in the 21st
Century
• Social Settings Framework
• Evolving Research Design and Analysis
Challenges
• Evolving Setting-Level Measurement
Challenges
• Future Challenges and Directions
William T. Grant Foundation www.wtgrantfoundation.org
2
Shifting Paradigms
• Individualism (or person-centered) foci
• Preoccupation of western style thought
• Well developed instrumentation
• Population-centered prevention/promotion
• Policy perspective
• Context
• Seemingly all inclusive
• Hard to get a handle on for:
• Research, including measurement
• Intervention
• The proximal Social Settings of daily experience
• A “place to stand”
William T. Grant Foundation www.wtgrantfoundation.org
Chronology of RFPs Re. Social Setting
Research and Lessons Learned
• 2002 RFP: Intervention Research
• Naïve conceptions of intervention, research design and analysis,
limited understanding of power when “program” was the unit of
analysis > need for capacity-building
• Need for further specification of our interests in
contexts/settings
•2005-7 RFP: Intervention Research to Improve YouthServing Organizations
•Need for the development and improvement of the measurement
of social setting processes
• 2007-? RFP: Development and Improvement of the
Measurement of Classroom Quality (in collaboration with
the Spencer Foundation)
William T. Grant Foundation www.wtgrantfoundation.org
Why Social Settings?
• Need to understand the proximal settings of daily
experience – families, peers, classrooms and
schools, youth-serving organizations,
neighborhoods
• Need to understand how policies work through
proximal settings of daily experience to effect
youth
• Social settings as loci of intervention > more
“bang for the buck,” especially in classrooms and
schools and other youth-serving organizations in
terms of prevention and promotion
William T. Grant Foundation www.wtgrantfoundation.org
Social Setting Theory and
Measurement References
• WTG Annual Report Essays:
• 2005 Social Setting Theory and Measurement
(website)
• 2006 Why try to understand and improve social
settings? (website)
• 2006 Doing social setting research (website)
• 2007 Measuring social settings (website)
William T. Grant Foundation www.wtgrantfoundation.org
Social Setting Theory and
Measurement References (continued)
Tseng, V., & Seidman, E. (2007). A systems framework
for understanding social settings. American Journal of
Community Psychology, 39(3-4), 217-228
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003).
Resources, instruction, and research. Educational
Evaluation and Policy Analysis, 25, 119-142.
Pianta, R.M. (in press). Classroom management and
relationships between children and teachers: Implications
for research and practice. In Good, T. & Evertson, (Eds.),
Handbook of Classroom Management.
Shinn, M., & Yoshikawa, H. (Eds.) (2008). Towards Positive
Youth Development: Transforming Schools and Community
Programs. New York: Oxford Press.
William T. Grant Foundation www.wtgrantfoundation.org
Macro
Forces
Schools
(Classrooms)
Macro
Forces
Youth
Outcomes
Families
Neighborhoods
Youth Programs
O.S.T.
Macro
Forces
Macro forces are policies, the economy, the environment,
secular trends in demography, and the like.
2
Resources
Macro
Forces
Social
Social Processes
Settings
Organization
of
Resources
William T. Grant Foundation www.wtgrantfoundation.org
Macro
Forces
Youth
Outcomes
Setting Norms
Relationships/
Practices
Social
Processes
Participation
in Activities
Human
Physical
Resources
Economic
Temporal
Social
Organization
Physical
Organization
Temporal
Organization
Economic
Organization
Organization of
Resources
Evolving Research Design & Analysis
Challenges
• Designing Research for Place-Randomized Trials
to increase precision (Raudenbush, Bloom, and
colleagues): Old issues in search of new answers
• N of nested unit – asymptotes quickly
• N of randomly assigned units
• Baseline covariates, pretests at individualand school-level
• Intra-class correlation coefficients
• Matching of covariates
– Products:
• Optimal Design Software
• Consultation Service
William T. Grant Foundation www.wtgrantfoundation.org
Evolving Setting Measurement
Challenges
•How can we effectively measure setting-level
processes?
• Observations, logs, artifacts, etc?
• Multi-method, multi-source, multi-construct?
• Norms: consensus measures?
• Reliability of measuring setting-level processes
= Generalizability studies, e.g., CLASS, YPQA, & others
•Linking setting-level change to individual-level
change
William T. Grant Foundation www.wtgrantfoundation.org
Future Challenges and
Directions
• Using new knowledge accrued to better plan
and reduce costs of future place randomized
trials
•Disentangling dynamic nature of setting-level
constructs
•Discovery of new or improvement of analytic
methods to establish causality between settinglevel and individual-level change
William T. Grant Foundation www.wtgrantfoundation.org