University of Iowa
Iowa Research Online
Theses and Dissertations
Spring 2011
The effect of study abroad on intercultural
competence among undergraduate college
students
Mark Hungerford Salisbury
University of Iowa
Copyright 2011 Mark Hungerford Salisbury
This dissertation is available at Iowa Research Online: https://ir.uiowa.edu/etd/1073
Recommended Citation
Salisbury, Mark Hungerford. "The effect of study abroad on intercultural competence among undergraduate college students." PhD
(Doctor of Philosophy) thesis, University of Iowa, 2011.
https://ir.uiowa.edu/etd/1073. https://doi.org/10.17077/etd.9rvi4i3z
Follow this and additional works at: https://ir.uiowa.edu/etd
Part of the Educational Administration and Supervision Commons
THE EFFECT OF STUDY ABROAD
ON INTERCULTURAL COMPETENCE
AMONG UNDERGRADUATE COLLEGE STUDENTS
by
Mark Hungerford Salisbury
An Abstract
Of a thesis submitted in partial fulfillment
of the requirements for the Doctor of Philosophy
degree in Educational Policy and Leadership Studies
in the Graduate College of
The University of Iowa
May 2011
Thesis Supervisor: Professor Ernest T. Pascarella
1
ABSTRACT
During the last decade higher education organizations and educational policy
makers have substantially increased efforts to incentivize study abroad participation.
These efforts are grounded in the longstanding belief that study abroad participation
improves intercultural competence – an educational outcome critical in a globalized 21st
century economy. Yet decades of evidence that appear to support this claim are
repeatedly limited by a series of methodological weaknesses including small homogenous
samples, an absence of longitudinal study design, no accounting for potential selection
bias, and the lack of controls for potentially confounding demographic and college
experience variables. Thus, a major competing explanation for differences found between
students who do and do not study abroad continues to be the possibility that these
differences existed prior to participation.
The current study sought to determine the effect of study abroad on intercultural
competence among 1,593 participants of the 2006 cohort of the Wabash National Study
on Liberal Arts Education. The Wabash National Study is a longitudinal study of
undergraduates that gathered pre- and post-test measures on numerous educational
outcomes, an array of institutional and self-reported pre-college characteristics, and a
host of college experiences. The current study employed both propensity score matching
and covariate adjustment methods to account for pre-college characteristics, college
experiences, the selection effect, and the clustered nature of the data to both crossvalidate findings and provide guidance for future research.
Under such rigorous analytic conditions, this study found that study abroad
generated a statistically significant positive effect on intercultural competence; an effect
2
that appears to be general rather than conditional. Moreover, both covariate adjustment
and propensity score matching methods generated similar results. In examining the effect
of study abroad across the three constituent subscales of the overall measure of
intercultural competence, this study found that study abroad influences students’ diversity
of contact but has no statistically significant effect on relativistic appreciation of cultural
differences or comfort with diversity. Finally, the results of this study suggest that the
relationship between study abroad and intercultural competence is one of selection and
accentuation, holding important implications for postsecondary policy makers, higher
education institutions, and college impact scholars.
Abstract Approved: ____________________________________
Ernest T. Pascarella
____________________________________
Professor, Educational Policy and Leadership
Studies
____________________________________
Date
THE EFFECT OF STUDY ABROAD
ON INTERCULTURAL COMPETENCE
AMONG UNDERGRADUATE COLLEGE STUDENTS
by
Mark Hungerford Salisbury
A thesis submitted in partial fulfillment
of the requirements for the Doctor of Philosophy
degree in Educational Leadership and Policy Studies
in the Graduate College of
The University of Iowa
May 2011
Thesis Supervisor: Professor Ernest T. Pascarella
Copyright by
Mark Hungerford Salisbury
2011
All Rights Reserved
Graduate College
The University of Iowa
Iowa City, Iowa
CERTIFICATE OF APPROVAL
_______________________
DOCTORAL THESIS
_______________
This is to certify that the Doctoral thesis of
Mark Hungerford Salisbury
has been approved by the Examining Committee
for the thesis requirement for the Doctor of Philosophy
degree in Educational Leadership and Policy Studies
at the May 2011 graduation.
Thesis Committee: ___________________________________
Ernest T. Pascarella, Thesis Supervisor
___________________________________
Michael B. Paulsen
___________________________________
Brian P. An
___________________________________
Scott F. McNabb
___________________________________
Downing A. Thomas
To my parents, Anne and Lee Salisbury, who exemplify a lifelong pursuit of learning.
ii
What we obtain too cheap, we esteem too lightly; it is dearness only that gives everything
its value.
Thomas Paine
The Crisis, December 23, 1776
iii
ACKNOWLEDGMENTS
For any substantial recognition bestowed upon an individual, there are in reality
many who deserve credit for the accomplishment. To my long-time friends and family,
your constant encouragement and support carried me when my confidence waned. To the
faculty in the Higher Education program at the University of Iowa, your wealth of
expertise and perspective provided a foundation of knowledge for which I will always be
grateful. To Ernie Pascarella, Mike Paulsen, and Paul Umbach, I cannot thank you
enough for the opportunity you gave me to test the deep waters of educational research. I
look forward to many more years of fruitful collaboration and friendship.
It is hard to put into words my appreciation for the community of graduate
students with whom I had the extreme fortune of sharing this experience. You created an
atmosphere of support and camaraderie that made all of the hours spent together
absolutely worthwhile. In particular, Kathy Goodman and Georgianna Martin, thank you
for willingly engaging in the deep, albeit sometimes meandering, dialogue that
challenged me to think more deeply and consider more broadly.
Finally, I share this accomplishment fully with my wife, Lynn, and my two boys,
Keaghan and Reid. Every day you keep me centered, motivated, and focused. I love you
more than you know.
iv
ABSTRACT
During the last decade higher education organizations and educational policy
makers have substantially increased efforts to incentivize study abroad participation.
These efforts are grounded in the longstanding belief that study abroad participation
improves intercultural competence – an educational outcome critical in a globalized 21st
century economy. Yet decades of evidence that appear to support this claim are
repeatedly limited by a series of methodological weaknesses including small homogenous
samples, an absence of longitudinal study design, no accounting for potential selection
bias, and the lack of controls for potentially confounding demographic and college
experience variables. Thus, a major competing explanation for differences found between
students who do and do not study abroad continues to be the possibility that these
differences existed prior to participation.
The current study sought to determine the effect of study abroad on intercultural
competence among 1,593 participants of the 2006 cohort of the Wabash National Study
on Liberal Arts Education. The Wabash National Study is a longitudinal study of
undergraduates that gathered pre- and post-test measures on numerous educational
outcomes, an array of institutional and self-reported pre-college characteristics, and a
host of college experiences. The current study employed both propensity score matching
and covariate adjustment methods to account for pre-college characteristics, college
experiences, the selection effect, and the clustered nature of the data to both crossvalidate findings and provide guidance for future research.
Under such rigorous analytic conditions, this study found that study abroad
generated a statistically significant positive effect on intercultural competence; an effect
v
that appears to be general rather than conditional. Moreover, both covariate adjustment
and propensity score matching methods generated similar results. In examining the effect
of study abroad across the three constituent subscales of the overall measure of
intercultural competence, this study found that study abroad influences students’ diversity
of contact but has no statistically significant effect on relativistic appreciation of cultural
differences or comfort with diversity. Finally, the results of this study suggest that the
relationship between study abroad and intercultural competence is one of selection and
accentuation, holding important implications for postsecondary policy makers, higher
education institutions, and college impact scholars.
vi
TABLE OF CONTENTS
LIST OF TABLES............................................................................................................. iv
LIST OF FIGURES .............................................................................................................v
CHAPTER
I.
INTRODUCTION ............................................................................................1
Statement of the Problem..................................................................................9
Purpose of the Study.......................................................................................10
Research Questions.........................................................................................11
Definitions ......................................................................................................12
Significance of the Study................................................................................13
II.
LITERATURE REVIEW ...............................................................................17
Theoretical Background of College Impact Research ....................................19
Intercultural Competence................................................................................23
Study Abroad and Intercultural Competence .................................................33
Predicting Participation in Study Abroad .......................................................41
Summary.........................................................................................................44
III.
CONCEPTUAL FRAMEWORK AND METHODOLOGY .........................46
Conceptual Framework...................................................................................46
Methodology...................................................................................................48
Sample.....................................................................................................48
Variables .................................................................................................50
Analysis...................................................................................................54
Limitations ..............................................................................................59
IV.
RESULTS .......................................................................................................77
Descriptive Statistics ......................................................................................77
Analytic Results..............................................................................................78
Covariate Adjustment .............................................................................78
Propensity Score Matching .....................................................................80
Including a Propensity Score in a Covariate Adjustment Model............82
V.
DISCUSSION.................................................................................................91
The Effect of Study Abroad on Intercultural Competence.............................92
Estimating Treatment Effects on College Impact Research ...........................96
Conclusion ......................................................................................................99
VI.
REFERENCES .............................................................................................101
vii
LIST OF TABLES
Table
1.
Names, Descriptions, and Response Options for Variables Used in this Study.......68
2.
Descriptive Statistics for Variables Included in the Analysis of General Effects ...............74
3.
Correlation Matrix for All Variables Included in this Study ....................................75
4.
The Effect of Study Abroad on Intercultural Competence Using Covariate
Adjustment while Accounting for the Clustering Effect. .........................................86
5.
A Comparison of Results derived from Covariate Adjustment and Propensity
Score Matching Analyzing the Effect of Study Abroad on Intercultural
Competence ..............................................................................................................87
6.
A Comparison of Propensity Score Matching Results using Various
Combinations of Independent Variable Blocks to Generate the Propensity
Score (listed in order from least to most biased) ......................................................88
7.
The Effect of Study Abroad on Intercultural Competence Comparing Two
Covariate Adjustment Regression Equations to Explore the Impact of a
Propensity Score vs. a Direct Measure of Study Abroad Intent on the Overall
Model ........................................................................................................................89
8.
The Effect of Study Abroad on each MGUDS Subscale..........................................90
viii
LIST OF FIGURES
Figure
1.
Conceptual Model to Examine the Effect of Study Abroad Participation on
Intercultural Competence..........................................................................................63
2.
Miville-Guzman Universality-Diversity Scale – Short Form (MGUDS-S) .............64
3.
MGUDS-Short Form Scoring Key ...........................................................................66
ix
1
CHAPTER I
INTRODUCTION
In the wake of the 9/11 attacks on the World Trade Center and the Pentagon, the
American Council on Education (ACE) published Beyond September 11: A
Comprehensive National Policy on International Education (2002), urgently calling on
the federal government to reverse decades of declining emphasis on international
education and foreign language study and instill new national policy that would prioritize
international education.
Like the challenge of Sputnik in 1957, the attacks of September 11 have
brought America’s international preparedness to a crossroads. The global
transformations of the last decade have created an unparalleled need in the
United States for expanded international knowledge and skills.
But the nation is unready. And our future success or failure in
international endeavors will rely almost entirely on the global competence
of our people. (p. 7)
ACE advocated three policy objectives for the United States during the next
century: 1) produce more international experts that align with strategic needs, 2) improve
the nation’s ability to address and resolve global challenges, and 3) “develop a globally
competent citizenry and workforce” (p. 9-10). ACE argued that the United States simply
could not achieve any of these three goals without substantially increasing the number of
Americans students who annually engage in international study abroad.
Beyond September 11 was not the first clarion call for increased investment and
participation in postsecondary study abroad. Over twenty years earlier, the President’s
Commission on Foreign Languages and International Study (1979) warned in its final
report Strength Through Wisdom: A Critique of U.S. Capability that the shrinking number
and deteriorating quality of international study opportunities in postsecondary education
2
would undermine the United States’ future ability to succeed in effective global
diplomacy, international crisis resolution, and preventative national security. In the
intervening decades, advocacy reports from the Council on International Educational
Exchange (National Task Force on Undergraduate Education Abroad, 1990) and the
Association of American Colleges & Universities (AAC&U) (Cornwell & Stoddard,
1999; Johnston, Jr. & Edelstein, 1993) urged postsecondary institutions to increase study
abroad participation by more clearly and overtly providing support for and promotion of
international study. In addition, study abroad program providers – colleges, universities,
and private providers – were encouraged to carefully reconsider whether their programs
legitimately improved their participants’ intercultural competency skills.
During the same period, the Office of Research in the Department of Education
(Adelman, 1994) and the Rand Corporation (Bikson & Law, 1994) initiated studies
examining the skills most desired by multi-national employers of their future work force.
These studies hypothesized that the emerging trends toward economic globalization and
corporate multi-nationalism might affect the way companies evaluated potential future
employees. In both cases, the researchers found a growing belief among senior executives
at large corporations that college graduates with internationally compatible skills such as
second language fluency and knowledge of international cultures and customs would be
vital to the continued profitability of the firm and, by extension, the nation. The authors
suggested that unless the public and private sector intentionally collaborated to increase
the number of college graduates entering the job market with these skills, the United
States would struggle to maintain a trajectory of economic growth and might well lose its
position as a dominant economic power.
3
This growing emphasis on study abroad seems to have influenced an increase in
participation. The compound annual rate of growth between 1993 and 1997 (10.55%) was
more than three times the size of the compound annual growth rate between 1987 and
1993 (3.43%)(NCES, 2008a, 2008b). However, though the number of study abroad
participants increased throughout the 1990s, the proportion of all college students who
embarked on an international study experience changed little, remaining below one
percent (NCES, 2008a, 2008b). Prior to 2001, while higher education institutions seemed
willing to exalt the importance of international educational experiences, they seemed less
willing or able to institute substantial curricular changes or reshuffle budget priorities to
boldly put those words into action (ACE, 2002; Altbach & Peterson, 1998; Siaya &
Hayward, 2003).
Since 2001, increasing the number of college graduates with the ability to
effectively interact internationally has emerged as a national policy priority (APLU,
2004; Lincoln Commission, 2005; S. Res. 2005). The final report of the National
Commission on Terrorist Attacks Upon the United States, commonly referred to as The
9/11 Commission (2003) appointed by President George W. Bush to identify the steps
and missteps of the U.S. intelligence community leading up to the 9/11 terrorist attacks,
noted that one of the weaknesses in effective intelligence gathering and diplomacy in the
Arab world was the infinitesimal number of postsecondary graduates with degrees in
Arabic studies. Upon the recommendations of NAFSA’s Strategic Task Force on
Education Abroad (2003) and the efforts of its honorary co-chair Senator Paul Simon,
Congress appropriated funds in 2003 to create The Commission on the Abraham Lincoln
Study Abroad Fellowship Program (Consolidated Appropriations Act, 2004) and explore
4
ways in which the federal government might facilitate a substantial increase in annual
study abroad participation. President George W. Bush greeted the 2005 publication of the
Lincoln Commission’s final report Global Competence and National Needs: One Million
Americans Studying Abroad by signing Senate Resolution 308 (2005) to declare 2006
“The Year of Study Abroad.” Soon thereafter the Higher Education Reconciliation Act of
2005 created the National Sciences and Mathematics Access to Retain Talent (SMART)
Grants that included scholarship funds for students majoring in foreign languages deemed
critical to national security interests (Deficit Reduction Act of 2005).
The efforts of the Lincoln Commission represented bold aspirations to make study
abroad a more central piece of the undergraduate experience. The Commission’s report
(2005) established a goal of one million American students studying abroad annually by
2016-17. The commission proposed that Congress appropriate annual funding of $50
million initially with incremental increases to reach $125 million annually by 2011-12 to
fund non-renewable scholarships of up to $5,000 to pay the costs of studying abroad. The
Lincoln Commission proposed funding protocols that would especially encourage
participation among traditionally underrepresented populations and criteria that would
ensure quality control in the study abroad programs supported by this effort. Finally, the
Commission proposed that higher education institutions partner with the federal
government by increasing their own investment in study abroad scholarships in response
to this federal initiative so that federal dollars could be most efficiently used to encourage
study abroad growth across all types of postsecondary institutions.
The resulting legislation, titled The Paul Simon Study Abroad Foundation Act
(H.R. 1469/S. 991) in honor of the late Senator Simon who died just before the Lincoln
5
Commission was established, passed the House of Representatives unanimously in June
2007 but stalled in the Senate despite broad bi-partisan support. Although it has been reintroduced in subsequent sessions of Congress and continues to garner near unanimous
support (H. R. 2410, 2009; S. 473, 2009), it has struggled to survive the lawmaking
process in the face of recent grim economic realities. Nonetheless, the broad public
support for study abroad generated by the Lincoln Commission helped to further
galvanize higher education organizations on behalf of international education generally
and study abroad in particular (Blum, 2006; Chmela, 2005; Farrell, 2007). Throughout
the last decade, ACE has sponsored an extensive series of programs and publications to
promote internationalization among higher education institutions through its Center for
International Initiatives, emphasizing the investment institutions should make in
supporting study abroad participation (ACE 2008; Field, 2009; Green, 2005; Green, Luu,
& Burris, 2008; Hill & Green, 2008; Olson, Green, & Hill, 2005, 2006; Siaya &
Hayward, 2003).
At the same time, AAC&U’s Liberal Learning and America’s Promise (LEAP)
(2007) initiative further situated study abroad as an ideal means of developing one of the
central educational outcomes of a liberal arts education – intercultural knowledge and
competence. This collaborative effort, funded generously by a host of influential
educational philanthropies, sought to articulate the specific types of knowledge and skills
that all undergraduates must obtain from a twenty-first century college education in order
to be successful in a new global century. The authors asserted, “Student success in
college cannot be documented – as it usually is – only in terms of enrollment, persistence,
and degree attainment” (p. 1). AAC&U detailed four broad educational outcomes that
6
would “provide a framework to guide students’ cumulative progress from school through
college” (p. 2): 1) knowledge of human cultures and the physical and natural world, 2)
intellectual and practical skills, 3) personal and social responsibility, and 4) integrative
learning. Study abroad participation was highlighted as an example of an educational
experience that would positively influence gains in each category. In 2008, the AAC&U
further emphasized the value of study abroad participation in High Impact Educational
Practices: What They Are, Who Has Access to Them, and Why They Matter (Kuh). This
publication used National Survey of Student Engagement (NSSE) data to demonstrate
that certain types of educational activities are particularly influential in producing gains
across a variety of the educational outcomes outlined in the LEAP initiative. Study
abroad was prominently featured as one such activity. Institutions were further
encouraged to support and promote study abroad participation, especially for students
with few prior cross-cultural experiences.
Again, although it might be difficult to demonstrate a direct causal relationship,
the growing chorus of public and private support for the importance of study abroad
seems to have influenced growth in institutional support and student participation.
Reporting data from an ACE survey of internationalization efforts at higher education
institutions, Siaya and Hayward (2003) noted comparative increases over the previous
decade in the proportion of institutions that administered study abroad programs and the
proportion of institutions that provided financial support for students participating in
those programs. ACE followed up these findings five years later and found that the
proportion of institutions administering study abroad programs had increased again –
85% in 2003 to 91% in 2008 (Green, Luu, & Burris). Moreover, despite the economic
7
recession that followed 9/11 and the accompanying American fears of travel or studying
abroad (Borcover, 2002; McKeown, 2003; Siaya & Hayward), the compound annual
growth rate of study abroad participation between 2002 and 2007 was 8.48% – only
minimally less than the 8.91% compound annual growth rate between 1997 and 2002
(NCES, 2008a, 2008b) – with the absolute number of study abroad participants growing
from 174,629 in 2002-03 to 262,416 in 2007-08.
The growing investment and emphasis at both the federal and institutional level
on increasing study abroad participation turns on the popular and long accepted belief
that a study abroad experience uniquely improves the intercultural skills of all
participants (Lincoln Commission, 2005; NAFSA, 2003; National Task Force on
Undergraduate Education Abroad, 1990). Yet some study abroad scholars have recently
questioned whether the successful push to drastically increase participation rates has
drowned out efforts to ensure the educational effectiveness of the experience (Vande
Berg, 2003, 2007; Woolf, 2007). Describing the originating principles behind The
Fulbright Program, established in 1946 as the first and arguably most prominent federally
supported study abroad scholarship program (Jeffrey, 1987), Senator J. William Fulbright
(1989) declared,
That ethos, in sum, is the belief that international relations can be
improved, and the danger of war significantly reduced, by producing
generations of leaders, especially in the big countries, who through the
experience of educational exchange, will have acquired some feeling and
understanding of other peoples' cultures – why they operate as they do,
why they think as they do, why they react as they do – and of the
differences among these cultures. It is possible – not very probable, but
possible – that people can find in themselves, through intercultural
education, the ways and means of living together in peace. (p. 193-194)
8
Although decades of studies seem to suggest that study abroad participation can
improve a range of intercultural attitudes and skills (Pascarella & Terenzini, 2005; Sell,
1983), Pascarella and Terenzini (2005) summarized the primary methodological
weaknesses that undercut the validity of these claims. “All of these studies are smallscale and leave uncontrolled numerous factors on which students who study abroad are
known to differ from those who do not” (p. 316). This leaves open the distinct possibility
that the alleged positive effects of participating in study abroad are not due to the
experience itself, but rather because of specific characteristics of the participants who
select to embark on an international educational experience.
Spurred by the growing emphasis on assessment of student learning in higher
education and calls among international educators to document student learning in study
abroad (Ingraham, 2003; Vande Berg, 2007), several recent large scale studies have
sought to empirically demonstrate the unique educational benefit of study abroad
participation on intercultural growth (Braskamp, Braskamp, & Merrill, 2009; Sutton &
Rubin, 2004, 2010; Vande Berg, Connor-Linton, & Paige, 2009). However, in each case
critical weaknesses in research design or analytical methodology undermine the validity
of their findings. The GLOSSARI study (Sutton & Rubin, 2004, 2010) compared
students from the University of Georgia System who did and did not study abroad on
seven dimensions of intercultural knowledge and skills. However, in addition to a lack of
demographic similarities between treatment and control groups, the researchers used a
cross-sectional design and were therefore unable to rule out the possibility that the
differences revealed in their study existed prior to the study abroad students’ departure.
9
Both Braskamp, Braskamp, and Merrill’s (2009) Global Perspectives Inventory
(GPI) study and Vande Berg, Connor-Linton, and Paige’s (2009) Georgetown
Consortium Project employed a pretest-posttest design and could therefore explore the
changes in intercultural competence over time. However, neither study accounted for predeparture demographic or attitudinal differences that might 1) influence the decision to
select to study abroad or 2) predispose an individual to benefit from the experience.
Furthermore, in addition to substantial problems in creating control groups
demographically comparable to their treatment groups, neither study employed an
analytical methodology designed to account for the selection effect when examining the
effect of a given treatment in quasi-experimental studies (Rosenbaum & Rubin, 1983;
Rubin, 2008; Schneider, Carnoy, Kilpatrick, Schmidt, & Shavelson, 2007). Although
these studies demonstrate a welcomed increased effort toward the assessment of learning
through study abroad, the lack of a rigorous research design or appropriate analytical
method does little to provide empirical evidence of study abroad’s effect on intercultural
competence among undergraduate college students.
Statement of the Problem
Over the course of several decades, postsecondary study abroad has evolved from
a selective niche educational endeavor to a national educational priority. Substantial
increases in institutional support for study abroad programs and potential federal funding
for study abroad scholarships have helped to boost participation. International
educational opportunities, once awarded to an elite few after an exhaustive process of
application and interview prior to selection (Gardner, 1968; McEvoy, 1967; Wallace,
1965), are now marketed to all students with the hope of attracting as many participants
10
as possible (Lincoln Commission, 2005; NAFSA, 2003). Some institutions have
instituted, or are seriously considering, substantial curricular changes to require study
abroad as a prerequisite for graduation (Chmela, 2005; Fischer, 2007). Although a wide
range of governmental and institutional stakeholders agree that increasing the number of
students who study abroad will increase the intercultural competency of the participants
and prepare more college graduates to succeed in a 21st century globally interconnected
knowledge economy, evidence to support the belief that study abroad participation
guarantees the acquisition of intercultural competence is far from compelling. In fact, no
conclusive evidence exists to demonstrate that the benefits of study abroad participation
suggested by prior research are attributable to the study abroad experience itself and not
to differences between the students who choose to participate and the students who do
not. In an era of shrinking fiscal resources, higher education institutions and governing
bodies need to know whether the substantial investment to increase study abroad
participation can be expected to produce the anticipated educational gains for all
participants. College students and their families need to know whether the personal
investment of time and finances necessary to participate in a study abroad experience will
produce the educational advantages the study abroad advocates claim.
Purpose of the Study
The purpose of this study is to estimate the unique effect of study abroad
participation on intercultural competence. To ensure the methodological rigor of this
study, however, this effort requires multiple stages of analysis. First, this study will
examine the differences between students who participate in study abroad and students
who do not and develop a model to predict the likelihood of study abroad participation.
11
Second, this study will estimate the effects of participating in study abroad on the
development of intercultural competence in the presence of the potentially confounding
pre-college characteristics, attitudes, and interests, institutional differences, and college
experience variables identified by the predictive model for participation. In addition, this
study will investigate whether those effects are conditional or general, i.e. whether
differences in the effects of participating in study abroad exist based upon differences in
pre-college characteristics, academic major, institutional type, and first-year experiences.
Third, this study will implement and then compare two analytical procedures –
propensity score matching methods and covariate adjustment – for estimating effects of
study abroad participation using observational data in order to strengthen the robustness
of the study as well as provide a deeper understanding of the value of propensity score
matching methods for college impact research.
Research Questions
1. What are the factors that previous research might indicate predicts participation in
study abroad?
2. What are the net total and direct effects of study abroad participation on
intercultural competence?
3. Are the net effects of study abroad participation on intercultural competence
conditional or general?
4. How do estimates generated by propensity score matching methods compare with
those generated by covariate adjustment methods?
12
Definitions
1. Study Abroad: Study abroad has evolved significantly over the last century (Hoffa,
2007; Hoffa & DePaul, 2010). Traditionally, study abroad participants traveled to a
university in a western European country at the beginning of their junior year,
attended courses for credit at the host institutions for a full academic year, and fully
immersed themselves in the local culture by either living with a host family or living
in a typical university residence with other native students. Today, study abroad
participants can travel to virtually any corner of the globe, enroll in programs that last
as little as three weeks or as long as a year, and experience widely divergent degrees
of immersion from living with a host family to spending the entire experience with a
group of like-minded American students under the close tutelage of a faculty
member. For the purposes of this study, a study abroad experience will be defined to
include the full range of credit-earning experiences that require international travel.
2. Intercultural Competence: An extensive body of empirical research and theoretical
writing has sought to describe, define, and measure intercultural competence under a
wide variety of definitions and constructs. Although no full agreement exists on a
definition of intercultural competence, a recent study applied both survey and Delphi
methods to bring a range of intercultural relations scholars and international
education administrators together and encapsulate the many perspectives on
intercultural competence into a single consensus definition that could be used as a
starting point for future attempts to measure the construct (Deardorff, 2004, 2006).
The resulting definition used to guide the present study describes intercultural
competence as a process orientation that is organized into two levels or stages – an
13
individual level and an interactional level, each containing two separate steps. Within
the individual level, the first step requires that one possess attitudes of respect or
value for other cultures, openness and the ability to withhold judgment, and curiosity
to discover while tolerating ambiguity. The second step requires that one develop
specific knowledge and comprehension that would include cultural self-awareness,
deep cultural knowledge, and sociolinguistic awareness. Consequently, to continually
acquire and comprehend this kind of knowledge, one must possess the skills to listen,
observe, evaluate, analyze, interpret, and relate. At the interactional level, this
definition of intercultural competence differentiates between two types of expected
outcomes: internal and external. The internal outcomes demonstrating intercultural
competence are an informed frame of reference shift that would come through
increased adaptability, flexibility, ethnorelativism, and empathy. The external
outcome expected of this process orientation is that all of these developmental gains
are integrated holistically so that the individual demonstrates effective and
appropriate communication and behavior in an intercultural situation (Deardorff,
2006, p. 256).
Significance of the Study
This study holds potential importance for a variety of practical, policy, theoretical,
and methodological considerations. Although institutional and public policy makers,
intercultural relations scholars, and international education administrators seem to firmly
believe in the positive effects of study abroad for most – if not all – students, the lack of
rigorously conducted research supporting this claim suggests the possibility that this
belief may qualify as one of higher education’s closely held “rational myths” (Pascarella,
14
2006, p. 513). This study will be the first large scale, multi-institutional study to
rigorously examine the effect of study abroad participation on intercultural competence in
the presence of the potential confounding effects of pre-college characteristics, academic
interests and career goals, institutional type differences, and college experiences.
Moreover, this study will be the first to employ the analytical procedures necessary to
fully account for the “selection effect,” i.e. the nonrandom selection of participants into
studying abroad. Although this study does not account for the wide variability in study
abroad location, program length, or depth of cultural immersion, the findings of this study
will provide a foundation for future research that might more precisely investigate the
potential existence of differential effects among these programmatic variations.
The findings of this study will be of considerable importance for public higher
education policy. If the findings support the longstanding belief in the positive
educational value of study abroad, they will provide the first methodologically sound
evidence to date regarding study abroad’s unique educational importance and embolden
advocates of international education to argue for continued increases in postsecondary
investment in study abroad infrastructure, programs, and scholarships as well as final
passage of the Senator Paul Simon Study Abroad Foundation Act. On the other hand, if
the findings suggest that study abroad participation, after accounting for potentially
confounding variables, does not uniquely improve intercultural competence – or if the
findings suggest that study abroad benefits only certain types of students because of preexisting traits, this study, at the very least, will emphasize the dangers of making policy
decisions in the absence of methodologically sound evidence.
15
Efforts to identify conditional effects could hold important implications for
practice as well as policy. If study abroad is particularly beneficial for certain types of
students, institutions might craft targeted programs to ensure that those students
participate in study abroad and maximize the benefits of the experience. Moreover, by
examining the effects of differences in college experiences on the development of
intercultural competence of study abroad participants, institutions may be able to
construct new initiatives, or improve upon existing ones, to ensure that study abroad
students are fully primed to successfully benefit from an international educational
opportunity.
The findings from this study will be useful for theory development as well.
Although not often cited in recent research on the positive effects of study abroad, earlier
studies grounded their research in the application of the contact hypothesis (Allport,
1954; Hofmann & Zak, 1969; Salter & Teger, 1975), which argued that interaction
between two groups would reduce the prejudicial feelings among the individuals in both
groups. Despite Allport’s and other scholar’s subsequent work demonstrating that
important conditions must be met before intergroup contact can be expected to reduce
prejudice and increase intercultural awareness, sensitivity, or communication (Amir,
1969; Hewstone & Brown, 1986; Pettigrew & Tropp, 2006), much of the popular
argument for study abroad participation seems to stems from the supposition that
sustained contact by itself will produce positive growth (Fulbright, 1989; Lincoln
Commission, 2005; NAFSA, 2003). The findings from this study may provide evidence
through which scholars can reconsider the application of the contact hypothesis as a
rationale for promoting study abroad participation and examining its effect. More
16
broadly, these findings could provide additional evidence to show that merely
participating in a high impact activity (Kuh, 2008) such as study abroad does not
necessarily ensure improved educational quality (Salisbury & Goodman, 2009). Rather,
as Kuh notes, high impact activities are only effective when they are purposefully
conducted to elicit learning.
Finally, because of the thorough design of the Wabash National Study on Liberal
Arts Education (WNS) (the source of the data for this study), this study will be able to
compare two analytical methods for analyzing observational data when the variable of
interest is participation in a non-randomly selected treatment. Statisticians have
empirically demonstrated that research exploring the effect of a non-randomly selected
treatment using observational data rather than data gathered through a true experimental
design can be vulnerable to type I error (i.e., finding a significant effect when, in fact,
there is none) because the lack of random selection leaves open the possibility that
individuals who choose to participate in the treatment are systematically different from
those who do not (Dehajia & Wahba, 1999; Rosenbaum & Rubin, 1983). The findings
from this section of the study will provide additional insight into determining the most
appropriate research design and analytical method for accurately estimating the effect of
a non-randomly assigned treatment.
17
CHAPTER II
LITERATURE REVIEW
American educational policy makers, employers, and higher education leaders
have recognized that American college graduates must acquire the ability to
communicate and collaborate across racial, ethnic, and cultural differences if they are to
successfully engage, compete, and contribute in “the new global century” (AAC&U,
2007; APLU, 2004; Bikson, Treverton, Moini, & Lindstrom, 2003; Lincoln Commission,
2005). To this end, postsecondary institutions and organizations have invested substantial
resources to increase study abroad participation with the expectation that students who
live and learn in the midst of another culture for an extended period of time will develop
intercultural awareness, sensitivity, and communication skills that that they could not
acquire through other educational mechanisms available on campus (ACE, 2002; Green,
Luu, & Burris, 2008; NAFSA, 2003).
While these efforts seem to have produced growth in study abroad participation
rates, the goals set forth in the Lincoln Commission’s final report – one million students
studying abroad annually by 2016-17 – or the even more ambitious goals outlined by
NAFSA in Securing America’s Future: Global Education for a Global Age of 20% of all
undergraduates studying abroad by 2010 – are still far from being achieved. Moreover,
the beneficial effects of study abroad participation on intercultural competence may be
more a popular narrative than an empirically grounded claim due to methodological
weaknesses in prior research (Pascarella & Terenzini, 2005), the subtle conversion of
fervently advocated beliefs into presumed fact (Wolf, 2007), and policy priorities among
18
international education administrators and study abroad programs that have emphasized
boosting participation rates over ensuring educational quality (Vande Berg, 2003, 2007).
As higher education institutions continue to invest more heavily in
internationalizing undergraduates through study abroad, and the federal government
continues to contemplate a significant increase in funding for study abroad scholarships
(H. R. 1496/S. 991, 2007; S. 473, 2009; S. Res. 208, 2005), it is critical to determine
whether these investments can be expected to produce their intended results. The purpose
of this study is to rigorously examine whether participation in study abroad uniquely
improves intercultural competence even after accounting for the pre-existing
characteristics or prior educational experiences of the participants.
This chapter reviews the literature related to study abroad and its impact on
college students’ intercultural competence. It is divided into four sections: 1) the
theoretical underpinnings of college impact research, 2) the construct of intercultural
competence, 3) prior research on the effects of study abroad, and 4) the factors that might
influence a “selection effect,” i.e., the decision to participate in study abroad. First, this
chapter reviews the theory and research that grounds the field of college impact study and
guides scholars’ understanding of how the college experience might influence students’
intellectual, affective, and interpersonal growth. Second, this chapter examines the theory
and literature that defines and details the construct of intercultural competence. The third
section of this chapter explores the research on the effects of study abroad, particularly in
the area of intercultural competence. Finally, this chapter examines the theory and
research examining the factors that influence study abroad participation.
19
Theoretical Background of College Impact Research
A series of seminal works now trace a long and increasingly diverse evolution of
research into the impact of college on students (Bloom & Webster, 1960; Boyer &
Michael, 1965; Feldman & Newcomb, 1969; Freedman, 1960; Jacob, 1957; Pascarella &
Terenzini, 1991, 2005; Strang, 1937). Summing the expansive research reviewed by
Feldman and Newcomb in 1969 with the breadth of studies examined by Pascarella and
Terenzini for their 1991 publication How College Affects Students and its equally
comprehensive 2005 sequel, these authors examined approximately 6,500 college impact
studies published over a seventy-year period. Moreover, the number of studies reviewed
in each publication indicates an increasing interest in the field, with Feldman and
Newcomb’s work examining 1,500 studies published over a forty-year period, Pascarella
and Terenzini’s 1991 book covering about 2,600 studies published within a twenty-year
period, and How College Affects Students, Vol. 2: A Third Decade of Research
(Pascarella & Terenzini, 2005) reviewing almost 2,500 studies produced over a period of
ten years. Collectively, this body of research has repeatedly demonstrated that the college
experience or particular aspects of the college experience can significantly influence the
development of college students across a range of developmental outcomes beyond the
inevitable effects of maturation. This ever-broadening field of study can be roughly
organized into two categories – studies that endeavor to identify a particular aspect (or
aspects) of the college experience that influences change in students and studies that
focus on the nature of the change or development attributed to the impact of college
(Pascarella & Terenzini, 2005, p. 19). Both families of theories and corresponding
20
research on college student development play an important role in framing the present
study.
Astin’s Input-Environment-Output (I-E-O) Model (1970a, 1970b) and
Pascarella’s General Model for Assessing the Effects of Differential Environments on
Student Learning and Cognitive Development (1985) lay out two theoretical models for
understanding college impact that have been repeatedly and successfully utilized to
identify aspects of the college experience that influence student growth (Astin, 1977,
1993; Pascarella & Terenzini, 1991, 2005). The I-E-O Model posits that the effects of
college can be isolated and measured by creating an analytical model composed of three
blocks: 1) inputs – the pre-college characteristics of the student including demographic
traits, attitudes, values, academic preparation, personal or social attributes, and family
background characteristics, 2) the environment – all the features that shape the college
experience including faculty, staff, peers, courses, curricula, co-curricular programs, and
experiences with which the student must contend during college, and 3) outputs – the
knowledge, skills, attitudes, values, beliefs, interests, and behavioral traits of the students
upon graduation, or departure, from college. As Pascarella and Terenzini (2005) note, this
model has been especially important for research examining the effects of discrete
programs that are organized or coordinated by institutional faculty or staff (p. 53). In the
context of research on study abroad, scholars have applied this model to examine study
abroad as both an outcome (Rust, Dhanatya, Furuto, & Kheiltash, 2007) and as an
element of the collegiate environment influencing subsequent educationally important
activity (Gonyea, 2008).
21
Pascarella’s General Model for Assessing the Effects of Differential
Environments on Student Learning and Cognitive Outcomes (1985) expands upon the
I-E-O Model to isolate and differentiate several critical sets of potentially confounding
variables previously absent from Astin’s college impact model. In addition to accounting
separately for inputs and outputs, Pascarella’s model divided Astin’s conception of the
environment into four separate blocks that more specifically account for the different
influences that might shape the effect of a given college experience. Pascarella
conceptualized Astin’s environment as a function of four potentially influential factors:
1) the structural/organizational characteristics of the institution, 2) the institutional
environment, 3) the interactions with agents of socialization, and 4) the quality of the
student effort (Pascarella & Terenzini, 2005, p. 57). This model allows researchers to
account more precisely for the separate elements of the symbiotic relationship between
the traits, attitudes, and interests that can shape the individual student’s involvement or
engagement in their own educational endeavor and the institution’s array of curricular
offerings, co-curricular programs, and informal experiences that can both affect the
individual student’s likelihood of engagement as well as shape the broader institutional
context. Equally important, this model is particularly critical for appropriately analyzing
multi-institutional data for individual level effects (p. 56).
Applied to better understand college impact, the second category of theories
focuses on the nature of the change or development observed in college students. While
these theories attempt to explain a range of developmental constructs such as
psychosocial development (Chickering, 1969; Erikson, 1950), cognitive development
(Belenky, Clinchy, Goldberger, & Tarule, 1986; King & Kitchener, 1994; Perry, 1970),
22
moral development (Kohlberg, 1971; Gilligan, 1982), identity development (Cass, 1979;
D’Augelli, 1994; Helms, 1990; Josselson, 1987; Phinney, 1990), or more holistic
conceptions of development (Baxter Magolda, 2001; Kegan, 1982), they emanate from
the pioneering developmental psychology work of Erikson’s theory of psychosocial
development (1950, 1959, 1968) and Piaget’s theory of cognitive development in
children (1952, 1964) and share several commonalities. They often describe a series of
stages through which an individual moves over time. Each stage requires a more complex
construction of understanding than the previous stage, and membership within each
subsequent stage depends upon the adoption of a new construction of knowing
accumulated during a previous stage. In most cases, stages function in a sequence with
transitional phases between them that are often difficult to negotiate smoothly, requiring
some sort of developmental effort, and in some cases the experiences within these
transitional stages can operate as impediments to development (Miller and Winston,
1990). Pascarella and Terenzini (2005) also note that, in order to move into higher
developmental stages, these theories require a cognitive readiness to progress, a
recognition of increased complexity insufficiently explained by the present stage, an
appropriately constituted challenge to the epistemological equilibrium of the current
stage, and – to move into the latter stages of development – the individual needs to gain
the ability to experience empathy and to detach from self (pp. 49-50). In each case, these
developmental theories either focus specifically on changes that occur over the course of
four years in college (e.g., Perry) or include a stage that is particularly germane to
explaining the development of individuals as they move through the ages of a traditional
college student (e.g., Erikson, Kohlberg, Baxter Magolda).
23
Examining the effect of study abroad on the development of intercultural
competence necessitates grounding in both of these theoretical families. Study abroad is a
distinct program provided by higher education institutions for the educational benefit of
its students and thereby fits into Astin’s environment block or, as a representation of the
student’s choice to engage in an educationally productive experience, fits into
Pascarella’s quality of student effort block. Furthermore, participating in study abroad
has long been associated with development across a range of cognitive, psychosocial, and
interpersonal domains. Bearing numerous similarities to the latter stages of Kegan’s
orders of consciousness (1982, 1994), the educational goals of study abroad participation
are often described as a combination of intercultural awareness, sensitivity, knowledge,
and communication skills that, when taken together, clearly rely on an interdependence
between multiple developmental domains (Deardorff, 2004). King and Baxter Magolda
(1996, 2005) have also argued that these various domains of development – psychosocial,
cognitive, identity, moral – are best understood as an integrated whole when considering
the development of college students. The interwoven nature of these domains is
poignantly exemplified by intercultural competence – a primary intended educational
outcome of study abroad participation.
Intercultural Competence
Higher education institutions have increasingly emphasized internationalization as
a core element of their strategic goals for the next century (Green, Luu, & Burris, 2008;
Siaya & Hayward, 2003). When institutional strategic plans and mission statements
articulate the intended educational outcomes of internationalization, they often claim that
the institution will instill in its students the intercultural competency skills necessary to
24
successfully engage, compete, and thrive in an increasingly diverse and globally
interconnected world (Deardorff, 2004; Knight, 2004). Unfortunately, clearly defining
and measuring intercultural competence has proven more difficult. But as the construct
has become an essential capacity for personal and professional success both domestically
and internationally, a consensus definition has begun to emerge that provides an
opportunity to assess elements of intercultural competence as an educational outcome of
college (Deardorff, 2009). First, this section briefly examines the concept of competence
as an educational goal. Second, it describes an emerging comprehensive definition of
intercultural competence that resulted from a Delphi study of the construct (Deardorff,
2004). Third, this section examines a developmental model of intercultural maturity
(King & Baxter Magolda, 2005) that applied a holistic developmental model grounded in
Kegan’s orders of consciousness (1994) to the notion of educating college students
toward intercultural competence. Finally, this section examines the similarities between
the multidimensional attributes necessary to exhibit intercultural competence and the
construct of universal-diverse orientation (UDO) and its accompanying measurement
instrument, the Miville-Guzman Universality-Diversity Scale (Miville et al, 1999).
The concept of competence has itself been the subject of some debate among
educational scholars. Emerging from a movement amongst educational policy makers in
the 1960s and 70s toward establishing observable and measurable minimum standards of
personal and professional skills and behaviors, elementary, secondary, and postsecondary
educational organizations and scholars described this approach as competency-based
education (Bowden and Marton, 1998; Grant & Associates, 1979; Spady, 1977). Efforts
to clearly define the intended outcome of this movement soon lagged behind its
25
increasing popularity – a challenge snidely articulated in the title of William Spady’s
article (1977) “Competency Based Education: A Bandwagon in Search of a Definition.”
Bowden and Marton (1998) describe four increasingly complex stages through
which definitions of competency have evolved (p. 106). Early definitions focused on
behavior and suggested that competence could be identified primarily through observing
and evaluating workplace performance. Scholars then added the importance of previously
acquired knowledge and suggested that competence was an additive construct of both
knowledge and performance that should be measured separately. In its third stage,
Bowden and Marton suggest that scholars integrated the measurement of knowledge and
performance to accommodate the situational nature of demonstrated competence. Lastly,
Bowden and Marton argue that competence is best conceptualized as a holistic construct
that perpetually integrates 1) the way one sees one’s role in a given situation, 2) the
capacity one has to undertake that role, and 3) the integration of previously acquired
knowledge and current performance within the given context to effectively accomplish a
given task.
Stoof, Marten, van Marrienboer, and Bastiaens (2002) argue that the difficulty in
defining competency originates from efforts to define it within the framework of an
objectivist paradigm that presumes competence can exist outside of a unique situational
context. Stoof et al. suggest that any demonstration of competence is fundamentally
influenced by three variables – people, goals, and context. Furthermore, they argue that
competence is most effectively defined when conceived within a constructivist paradigm
that recognizes its iterative formation process and contextualized manifestation. This
approach to defining competence closely resembles the epistemological underpinnings of
26
the holistic development models proposed by Baxter Magolda (2001) and Kegan (1982,
1994) that integrate multiple aspects of the cognitive, intrapersonal, and interpersonal
domains and recognize that, while development along all three trajectories is necessary to
move to higher “orders of consciousness” (Kegan) or “mature levels of development”
(King & Baxter Magolda, 2005), the interdependence of those domains can affect the
nature and process of development for each individual. Bowden and Marton (1998)
proposed that educators focus on developing students toward competency construct
curricula that include a myriad of ill-structured problems and require simultaneous use of
all three domains as students engage an assigned educational experience. In applying a
holistic perspective to propose the stages through which one would progress to achieve
intercultural maturity, King and Baxter Magolda’s (2005) student interviews show that
study abroad participation can present students with the intercultural interactions that
require them to engage the kinds of ill-structured problems that foster holistic
development.
Intercultural competence applies the notion of competence to describe the
successful engagement or collaboration toward a single or shared set of goals between
individuals or groups who do not share the same cultural origins or background. These
cultural differences can arise from any combination of factors including racial, ethnic,
socio-economic, religious, and national differences. Individuals or groups demonstrating
intercultural competence are able in a given situation to find “common purpose through
mutually coordinated communication across cultures and languages” (Spitzberg &
Changnon, 2009, p. 2). But while intercultural competence may seem like a manageable
27
if not identifiable construct, attempts to define it have generated increasingly divergent
approaches that have made it difficult to reach a consensus among scholars.
In a recent meta-analysis that reviewed both definitions and developmental
models of intercultural competence, Spitzberg and Changnon (2009) identified over 300
conceptual frameworks of the construct. Moreover, for almost every conceptualizations
of intercultural competence identified, Spitzberg and Changnon found that a unique
measurement instrument was devised to assess the validity of the particular version of the
construct. While most of these instruments seem to have been employed only once,
several have been more broadly utilized to measure intercultural competence (Dunn,
Smith, & Montoya, 2006; Sinicrope, Norris, & Watanabe, 2003) either to compare scores
across diverse populations or to test the effect of a particular treatment on intercultural
competence growth. Some of the more common instruments include the Behavioral
Assessment Scale for Intercultural Competence (BASIC) (Koeser & Olebe 1988; Ruben
& Kealey, 1979), the Intercultural Sensitivity Inventory (ICSI) (Bhawuk & Brislin,
1992), the Intercultural Development Inventory (IDI) (Hammer, Bennett, & Wiseman,
2003), the Cross-Cultural Adaptability Inventory (CCAI) (Kelley & Meyers, 1995), the
Intercultural Sensitivity Index (ISI) (Olson & Kroeger, 2001), the Assessment of
Intercultural Competence (AIC) (Fantini, 2000, 2006), the Miville-Guzman UniversalityDiversity Scale (M-GUDS) (Miville, Gelso, Pannu, Lui, Touradji, Holloway, & Fuertes,
1999), and the Global Perspective Inventory (GPI) (Braskamp, Braskamp, & Merrill,
2008). Several other measures have been developed for assessment and training in the
counseling profession such as the Multicultural Counseling Inventory (MCI) (Sodowsky,
Taffe, Gutkin, & Wise, 1994), the Cross-Cultural Counseling Inventory-Revised (CCCI-
28
R) (LaFromboise, Coleman, & Hernancez, 1991), the Multicultural Counseling
Awareness Scale: Form B (MCAS:B) (Ponterotto & Alexander, 1996; Ponterotto,
Gretchen, Utsey, Rieger, & Austin, 2002), and the Multicultural Awareness Knowledge
Skills Scale (MAKSS) (D’Andrea, Daniels, & Heck, 1991). An instrument was also
developed specifically for multicultural competence development in a higher education
context called the Multicultural Competence in Student Affairs-Preliminary 2 (MCSAP2) (Pope & Mueller, 2000).
After reviewing the diversity of approaches to modeling the development of, or
manifestation of, intercultural competence, Spitzberg and Changnon (2009) suggest that
although this range of conceptualizations provides “a rich conceptual and theoretical
landscape” (p. 44), many of these definitions or frameworks identify similar ingredients
that can be categorized into five groups: motivation (affective, emotion), knowledge
(cognitive), skills (behavior, actional), context (situation, environment, culture,
relationship, function), and outcomes (perceived appropriateness or effectiveness,
satisfaction, understanding, attraction, intimacy, assimilation, task achievement) (p. 7).
Spitzberg and Changnon found that in most cases these conceptualizations differed more
in terminology than in substance. They conclude that “many conceptual wheels are being
reinvented at the expense of legitimate progress” (p. 45) toward a common definition that
would better facilitate development of instruments or methodologies to measure the
construct and test its validity.
In an effort to move toward a broadly accepted definition of intercultural
competence and thereby allow institutions to better assess educational efforts toward
intercultural competence, Deardorff (2004) conducted a two-pronged study that included
29
a survey of international education administrators regarding institutional
internationalization efforts and a Delphi approach to developing a consensus definition
among an international panel of 23 intercultural relations experts. Deardorff found
substantial overlap among scholars and international education administrators in
conceptualizing intercultural competence and organized those findings into a four-part
process model (2006, p. 256). These four parts are divided into two stages – individual
and interaction. The individual stage encompasses two sets of attitudes or attributes. The
first includes a set of attitudes including respect for or valuing other cultures, openness
without judgment to intercultural learning and to people from other cultures, and a
curiosity for discovery that can tolerate ambiguity during the process of exploration. The
second includes an interrelated set of knowledge comprehension and interactive skills.
The critical aspects of knowledge and comprehension include cultural self-awareness, a
deep understanding and knowledge of culture that includes contexts, the role and impact
of culture on differing worldviews, culture-specific information, and sociolinguistic
awareness. The interactive skills include the ability to listen, observe, and interpret as
well as analyze, evaluate, and relate knowledge gained in one setting to circumstances in
a new setting.
The interaction stage is also composed of two stages – internal outcomes and
external outcomes. The internal outcomes describe the development of several
psychological traits including adaptability to different communication styles and
behaviors or new cultural environments, both cognitive and emotional flexibility in
selecting and using appropriate communication styles and behaviors, the development of
a increasingly nuanced ethnorelative view, and an increased sense of empathy across
30
cultural differences. Finally, the desired external outcome was defined as “behaving and
communicating effectively and appropriately based upon one’s intercultural knowledge,
skills, and attitudes to achieve one’s goals to some degree” (Deardorff, 2006, p. 254).
While the external outcome described in this model is undoubtedly influenced by
the social context within which the interchange occurs (and therefore outside the control
of the individual), each of the three prior stages taken together (as situated in the
Deardorff’s process model) represent the attitudes, values, and attributes necessary to
demonstrate intercultural competence. These capacities closely match the culminating
traits of each of the three multidimensional domains proposed in King and Baxter
Magolda’s (2005) developmental model of intercultural maturity. Constructed within a
college student development paradigm, this model articulates the developmental
trajectories necessary to demonstrate intercultural competence. King and Baxter Magolda
build their model upon the theoretical framework for holistic development proposed by
Kegan (1982, 1994), which proposes three domains of development – cognitive,
intrapersonal, and interpersonal (King & Baxter Magolda, p. 576). The cognitive domain
addresses knowledge acquisition, comprehension, and understanding and describes
cognitive development as a move from conceptualizing knowledge as universal and
dichotomous (i.e. answers to dilemmas are frames simply as right or wrong) to
understanding the existence and effects of multiple dimensions of nuance across all
manner of individual, situational, and cultural differences. The intrapersonal domain
addresses psychosocial and personal identity development within which individuals move
from lacking awareness of one’s own values and identity apart from external associations
to embracing a dynamic sense of an internal self that willingly considers differing beliefs
31
and values through an ethnorelative lens that understands and respects difference. The
interpersonal domain describes the capacity for successful interaction with diverse others.
Individuals who exhibit a mature level of development in the interpersonal domain
demonstrate a “capacity to engage in meaningful, interdependent relationships with
diverse others that are grounded in an understanding and appreciation for human
differences” (p. 576).
The traits described within the mature stages of each domain in King and Baxter
Magolda’s (2005) model closely mirror the culmination of the requisite attitudes,
knowledge, and skills that manifest themselves in the internal outcomes of Deardorff’s
(2004) consensus definition of intercultural competence. Where King and Baxter
Magolda suggest that individuals in the mature stage of cognitive development have the
“ability to consciously shift perspectives and behaviors into an alternative cultural
worldview and to use multiple cultural frames” (p. 576), Deardorff’s (2004) consensus
definition identifies the ability to adapt to different cultural environments and the
cognitive flexibility to utilize culturally-specific and appropriate communication
techniques among the internal outcomes that set the stage for demonstrating intercultural
competence. Where the Developmental Model of Intercultural Maturity describes the
mature stage of development in the intrapersonal domain as the “capacity to create an
internal self that . . . considers social identities (race, class, gender, etc.) in a global and
national context” (p. 576), Deardorff’s (2004) consensus definition describes
interculturally competent individuals as possessing an ethnorelative view, a cultural and
sociolinguistic self-awareness, and an openness to intercultural learning while
withholding judgment. Finally, where King and Baxter Magolda describe the mature
32
stage of interpersonal development as the “capacity to engage in meaningful,
interdependent relationships with diverse others that are grounded in an understanding
and appreciation for human differences” (p. 576), Deardorff’s consensus definition
describes intercultural competence as a multi-pronged capacity requiring respect and a
valuing of cultural diversity, the skills to listen and observe among other important
elements of successful social interaction, and empathy for another’s cultural context.
Although both scholars use differing terminology, clearly these models share substantial
overlap in describing the cognitive, intrapersonal, and interpersonal capacities required to
demonstrate intercultural competence. In summary, the developmental process toward
intercultural competence might be understood along three vectors: 1) cognitive
development that allows for a relativistic appreciation of similarities and differences
among diverse individuals, 2) psychosocial (or intrapersonal) development that facilitates
increasing comfort when engaged in interactions with diverse others, and 3) interpersonal
development that empowers one to seek out diverse interactions through experiences that
highlight, celebrate, or examine differences among diverse individuals or groups.
As student populations have diversified and both policy makers and employers
have recognized the importance of intercultural competent among college graduates,
college impact scholars have focused on identifying the educational experiences that
enhance student attitudes toward diversity and develop intercultural competence skills
(Pascarella & Terenzini, 2005). Although typically described in the context of domestic
interracial interactions, this body of research has employed dependent measures that
closely mirror the cognitive, intrapersonal, and interpersonal capacities required of
intercultural competence, including openness to diversity, racial tolerance, respect for
33
racial and ethnic differences, cultural knowledge, multicultural understanding, increased
interracial comfort, international understanding, and pluralistic orientation (Astin, 1993;
Engberg, 2007; Gurin, Peng, Lopez, & Nagda, 1999; Hurtado, Engberg, & Ponjuan,
2003; Milem, 1994; Myers-Lipton, 1996; Pascarella, Edison, Nora, Hagedorn, &
Terenzini, 1996). Researchers have found that a variety of college experiences and
activities can positively influence intercultural competence, including living on campus
(Pike, 2002), interracial friendship groups (Antonio, 2001), diversity-related course
taking (Chang, 2002), racial awareness workshops (Whitt, Edison, Pascarella, Terenzini,
& Nora, 2001), leadership training (Cress, Astin, Zimmerman-Oster, & Burkhardt, 2001),
volunteer or service learning experiences (Gray, Ondaatjie, Fricker, & Geschwind, 2000;
Sax & Astin, 1997), and participation in a study abroad program (Carlson, Burn, Useem,
& Yachimovicz, 1990; Williams, 2005).
Study Abroad and Intercultural Competence
As study abroad participation rates have increased and diversified across
academic disciplines, researchers from a wide range of academic fields have increasingly
endeavored to demonstrate the educational benefits of studying abroad (Lewin, 2009;
McKeown, 2009; Savicki, 2008). International education scholars have linked study
abroad participation with several indicators of general academic success including
graduation rates (Posey, Jr., 2003; Sutton & Rubin, 2010), time to degree (Ingraham &
Peterson, 2004; Flash, 1999), retention (Kasravi, 2009; Metzger, 2006; Young, 2003),
and grade point average (Posey, Jr.; Sutton & Rubin; Thomas & McMahon, 1998). More
specifically, foreign language scholars have repeatedly found study abroad to positively
affect second language acquisition (Brecht, Davidson, & Ginsberg, 1993; DuFon &
34
Churchill, 2006; Freed, 1995; Magnan & Back, 2006), one of the original motivations for
the creation of formal international educational experiences (Hoffa, 2007). Moreover,
researchers have also found study abroad to be correlated with increased engagement in
other educationally beneficial activities during college (Gonyea, 2008) as well as specific
career choices after college (Armstrong, 1984; Mistretta, 2008; Norris & Gillespie, 2009;
Wallace, 1999).
In addition, study abroad advocates have long argued that the experience of living
and learning abroad has a unique effect on many of the broad educational outcomes
historically associated with a liberal arts education. Study abroad participation has been
linked with aspects of psychosocial – often called personal – development (Gmelch,
1997; Gullahorn & Gullahorn, 1966; Kauffmann, 1983; Milstein, 2005), identity
development (Dolby, 2004; Talburt & Stewart, 1999), moral or values development
(Jurgens & McAuliffe, 2004; Lindsey, 2005; Ryan & Twibell, 2000), intellectual
development (Barrutia, 1971; McKeown, 2009; Zhai & Scheer, 2002), and holistic
development conceptualized as self-authorship (Braskamp, Braskamp, & Merrill, 2009;
Du, 2007). Most prominent among the educational benefits claimed of an international
educational experience, study abroad has long been considered a powerful mechanism for
developing intercultural competence (Fulbright, 1989; Hoffa, 2007; Hoffa & DePaul,
2010; Lincoln Commission, 2005; NAFSA, 2003; National Task Force on Undergraduate
Education Abroad, 1990).
The assertion that studying abroad improves intercultural competence is grounded
in the contact hypothesis. Allport (1954) proposed that prejudice held by one group
toward another group could be reduced if individuals from both groups participated in
35
sustained interpersonal contact. Subsequent research testing this hypothesis under a
variety of conditions identified several caveats under which intergroup relations were
most likely to reduce prejudice (Hewstone & Brown, 1986; Nesdale & Todd, 2000).
These conditions include equality of status during contact, a social context that supports
equality between groups, collaborative engagement toward a shared goal, opportunity to
develop the level of intimacy necessary to contradict previously held stereotypes, and the
support of applicable authority figures. In a meta-analysis of 713 independent samples
from 515 studies distributed across almost 50 years of research, Pettigrew and Tropp
(2006) found strong support for the contact hypothesis. In addition, their findings suggest
that although the conditions identified by prior research might facilitate a greater
reduction of prejudice, the existence of those conditions were not a necessary prerequisite
for prejudice to be reduced.
While the contact hypothesis originated within the context of improving relations
between domestic racial groups, Amir (1969) reviewed and synthesized efforts to apply
the contact hypothesis to improve inter-ethnic and international relations. He argued that
it was crucial to specifically examine the validity of the contact hypothesis in the context
of improving ethnic relations because the assumptions underlying this theory are
exemplified in the explicit or tacit objectives of various international
exchange programs: student exchanges or those of professional people,
organized tours and visits to foreign countries, the sending of foreign
students to visit or live with native families, etc. . . . International
seminars, international conferences and exhibitions, the Olympic games –
all these – are often thought to be effective because of the opportunities
for contact that they afford. The basic premise is typically that personal
contact can overcome difficulties where tons of paper work and
memoranda have not succeeded (p. 320).
36
Amir’s review of the literature found that, although the majority of findings
support the assertion that contact between groups of differing ethnic origins or
nationalities was likely to produce change in the attitudes of both groups toward the
other, the conditions under which this contact occurs are significantly influential in
determining the direction, or intensity, of the attitudinal change. In addition to the
conditions outlined by Allport (1954) that might encourage positive attitudinal change,
Amir noted several conditions specific to inter-ethnic or international interactions that
might inhibit positive change or even increase prejudice. These unfavorable conditions
include when the contact is “unpleasant, involuntary or tension laden,” when one group is
“in a state of frustration (i.e., inadequate personality structure, recent defeat or failure,
economic depression, etc.)” potentially leading to ethnic “scapegoating,” and when the
two groups find each other’s moral or ethical values objectionable (p. 339). In the
presence of one or more unfavorable conditions, sustained contact could increase or
intensify prejudice rather than reduce it.
Clearly, the cognitive, intrapersonal, and interpersonal capacities necessary to
exhibit intercultural competence – sensitivity to cultural differences, awareness of sociohistorical cultural contexts, adaptability and flexibility to view cultural differences and
contextual circumstances through an informed ethnorelative lens, and the empathy to
seek deeper understanding while withholding judgment – are all attributes that could
mitigate the presence of the unfavorable conditions listed above and thereby increase the
likelihood of a positive cross-cultural outcome. Not surprisingly, study abroad advocates
and international education scholars have repeatedly sought to demonstrate the positive
effect of studying abroad on intercultural competence under the presumption that the
37
conditions for intergroup contact during a study abroad experience are ideal for reducing
prejudice, developing intercultural competency skills, and improving relations across
cultural, ethnic, and/or national differences (Sell, 1983; Pascarella & Terenzini, 2005).
A few exceptions notwithstanding (Kalunian, 1997, as cited in Pascarella &
Terenzini, 2005; Patterson, 2006; Wilkinson, 1998), researchers have repeatedly found
that students who study abroad demonstrate positive change on several aspects of
intercultural competence upon return to their home campuses. Although the language
describing the outcome of interest varies across this body of work, numerous singleinstitution, small sample studies have investigated the effect of studying abroad on three
broad aspects of intercultural competence: 1) the respondent’s view of the host culture or
country, 2) the respondent’s global perspective or world mindedness, and 3) the
respondent’s intercultural awareness or sensitivity. Findings from these studies seem to
suggest that students who study abroad develop a more positive view of the host culture
(Bicknese, 1974; Carlson & Widaman, 1988; Cushner & Karim, 2004; Nash, 1976),
expand their global perspective or world mindedness (Chieffo & Griffiths, 2004; Cushner
& Mahon, 2002; Douglas & Jones-Rikker, 2001; Golay, 2006), and increase their
intercultural awareness and sensitivity (Anderson, Lawton, Rexeisen, & Hubbard, 2006;
Black & Duhon, 2006; Pedersen, 2009; Shaheen, 2004; Williams, 2005).
But while all of these studies assert evidence to support the claim that studying
abroad uniquely influences the development of intercultural competence, several
methodological weaknesses undercut the validity and generalizability of their findings.
First, each of these studies examines a small group of students at a particular institution,
comparing them to an equally small and homogenous non-randomly assigned control
38
group. In most cases, these studies are also limited to students who participated in a
particular study abroad program. Furthermore, some of these studies are even restricted to
students in a specific major. Since each of the samples in these studies is far from
nationally representative of all undergraduates, it is exceedingly difficult to generalize
any of these findings to all college students who study abroad. Second, as Pascarella and
Terenzini noted in their review of college impact research (2005), none of these studies
adequately accounts for the potentially confounding demographic, attitudinal, or
aspirational characteristics that might systematically differentiate between students who
do and do not study abroad.
Even the most extensive efforts to demonstrate study abroad’s effect on its
participants’ intercultural attitudes and skills suffer from notable weaknesses in design or
analysis. Although the 1984-85 Study Abroad Evaluation Project (SAEP) (Carlson,
Burns, Useem, and Yachimovicz, 1990) collected data from students at multiple
American institutions (N=4), applied a longitudinal design, and controlled for a host of
potentially confounding variables, the analysis did not account for the possibility that
effects observed may have resulted from differences between the students who selected to
participate in study abroad and the students who did not. In addition, the sample of
student respondents came primarily from large research institutions (the control group
coming almost entirely from a single large public university), thus making it difficult to
generalize the findings to students at other types of institutions, especially small, private
liberal arts colleges where study abroad originated and remains more deeply
institutionalized. Although the study found that students who studied abroad grew in
ways that the students who stayed home did not, it was impossible to know whether that
39
growth was uniquely a function of studying abroad or rather a by-product of other
observed or unobserved characteristics.
The Georgetown Consortium Project (Vande Berg, Connor-Litton, & Paige,
2009) examined gains on several outcomes regarding second language acquisition and
intercultural awareness and competency development across a range of study abroad
program differences using a pre-test/post-test design. However this study only analyzed
mean differences between the pre- and post-test scores for the treatment and control
groups without controlling for any other potentially confounding characteristics.
Furthermore, the control group differed substantially from the treatment group in size and
distribution across institutions, making it difficult to make confident assertions about the
validity of the findings.
Finally, The GLOSSARI (Georgia Learning Outcomes of Students Studying
Abroad Research Initiative) Project (Sutton & Rubin, 2004) endeavored to assess the
effects of study abroad participation across a variety of intercultural and academic
progress measures using data gathered from students at institutions throughout the
University System of Georgia. However, this study failed to employ a pre-test/post-test
design and only controlled for GPA and academic major in its analysis, omitting any
other pre-college characteristics or attitudinal differences between student groups.
Furthermore, the study’s means of convenience sampling left open the significant
possibility of sample selection bias. Although this study is currently implementing a
second phase that attempts to correct for previous methodological and design
weaknesses, the sampling frame continues to be students within the University System of
40
Georgia, again making it difficult to generalize the findings to students at private
institutions.
Two additional and potentially more fundamental problems undermine the
validity of the existing body of research on study abroad. After reviewing the state of
study abroad research almost twenty-five years ago, Church (1982) and Sell (1983) noted
that the vast majority of the study abroad research to date had been constructed absent
grounding in a plausible theoretical framework. Black and Mendenhall (1991) found the
same to be true of the research on cross-cultural adjustment among study abroad
participants. Although Pascarella and Terenzini (2005) did not assert such a stark
assessment of the study abroad research involving college students, they clearly noted
that the existing body of findings had failed to account for differences between students
who choose to study abroad and students who choose not to study abroad. In the context
of Pascarella’s model for examining the impact of specific college experiences on college
student educational outcomes (1985), Pascarella and Terenzini’s critique of research on
undergraduate study abroad seems to mirror the conclusions of these earlier reviews.
Undergraduate study abroad does not occur, nor does it function, in isolation from the
larger postsecondary educational enterprise. As such, any effort to examine its impact
without accounting for the potential confounding effects of this context would seem
incomplete. Moreover, despite an extensive body of research demonstrating that college
student development across a range of cognitive, affective, or holistic domains cannot be
expected to occur at a uniform pace, in a linear fashion, or within a quantifiable
timeframe (Baxter Magolda, 2001; Kegan, 1982, 1994; King & Kitchener, 1994;
Pascarella & Terenzini, 1991, 2005; Perry, 1970), research on the effects of study abroad
41
has consistently operated under the assumption that any change resulting from the study
abroad experience should be evident immediately upon its conclusion.
The second methodological weakness found both in the single institution studies
as well as the larger scale studies is the lack of adequate accounting for factors that might
systematically differentiate students who choose to study abroad from students who
choose not to study abroad. Yet both descriptive data on study abroad participants as well
as the few studies examining the factors that predict study abroad intent or participation
have demonstrated the degree to which students who study abroad differ from those who
remain on campus (Booker, 2001; Carlson et al, 1990; Goldstein & Kim, 2006; IIE, 2009;
Koester, 1985; Lincoln Commission, 2005; NAFSA, 2003; Salisbury, Paulsen, &
Pascarella, 2010, in press; Salisbury, Umbach, Paulsen, & Pascarella, 2009; Spiering &
Erickson, 2006). Because any effort to isolate an effect of study abroad participation on
intercultural competence needs to account for the full array of potentially confounding
variables, the last section of this chapter reviews the research regarding systemic
differences between students who do and do not study abroad and the factors that might
predict participation.
Predicting Participation in Study Abroad
Study abroad advocates have long lamented the disproportionate nature of study
abroad participation rates among undergraduates (Council on International Educational
Exchange [CIEE], 1991; Dessoff, 2006; Koester, 1985; Lambert, 1989; Redden, 2008).
Students who study abroad have historically been overwhelmingly white, more female
than male by a ratio of almost 2:1, and primarily majoring in the humanities, social
sciences, and fine arts (IIE, 2009; Lincoln Commission, 2005; NAFSA, 2003). Salisbury,
42
Paulsen, & Pascarella (2010, in press) found that the disparities in participation by gender
and race were reflected in rates of study abroad intent. Likewise, Booker (2001) and
Stroud (2010) both found that choice of major influences the decision to participate in
study abroad. The findings of the Lincoln Commission (2005) echoed this disparity and
pointed specifically at the more prescriptive curricular requirements of the natural
sciences, pre-med, and engineering as barriers to larger participation rates among those
majors. In a single institution study, Goldstein and Kim (2006) found that heightened
concerns about completing a major on time were a significant negative predictor of study
abroad participation. Although not specifically tied to curricular complexities, Spiering
and Erickson (2006) employed a diffusion of innovation theory to examine why students
who attended an informational session on study abroad chose to participate or not
participate and found that students who did not participate tended to rank the complexity
of the issues involved in making it possible to study abroad as the number one reason
they chose not to participate.
Historically, small liberal arts colleges tend to send larger percentages of their
students on study abroad programs than large public institutions (Hoffa, 2007; IIE;
NAFSA). Salisbury, Umbach, Paulsen, and Pascarella (2009) found that students
attending research universities, regional institutions, and community colleges were
substantially less likely to intend to study abroad. Potentially reflecting differences across
institutional types, study abroad advocates have long argued that the cost of participation
has disproportionately favored students of higher socioeconomic status (National Task
Force on Undergraduate Education Abroad, 1990; Lincoln Commission). Salisbury,
43
Umbach, Paulsen, and Pascarella (2009) also found that students who intend to study
abroad tend to originate from a higher socioeconomic status.
In addition to longstanding demographic differences between American
undergraduates who do and do not study abroad, a small subset of findings suggests that
study abroad participants may differ from those who do not participate in their attitudes
toward intercultural interaction, their interest in intercultural experiences, and the degree
to which they value intercultural interaction as a part of their college education. As a part
of the SAEP study, Carlson et al. (1990) asked both the study abroad students and the
control group (i.e. students who remained on campus) a series of questions about
international knowledge and interest and intercultural aptitudes at the beginning of their
study, prior to departure. The researchers identified clear differences in the degree to
which the two groups of students viewed American foreign policy (the study abroad
students were more critical of American foreign policy), the quality of postsecondary
education in Western European countries (the study abroad students held more positive
opinions), and degree of interest in experiencing other cultures (the study abroad students
were much more interested in experiencing other cultures than the students who were not
about to study abroad). Goldstein and Kim (2006) conducted a single-institution
longitudinal study to identify variables that might predict study abroad participation and
found that the students who studied abroad had on average differed significantly on
measures of ethnocentrism, prejudice, and intercultural attitudes.
Goldstein and Kim (2006) replicated earlier findings of King and Young (1994),
findings that students who studied abroad tended to be more interested in learning a
foreign language. Salisbury et al. (2009) and Stroud (2010) both found that the degree to
44
which students’ value developing cross-cultural understanding during college
significantly increased the likelihood of intending to study abroad. Salisbury and his
colleagues also found that diverse experiences during the first year of college could
improve the likelihood of intent to study abroad, suggesting that systemic attitudinal
differences between students who intend and students do not intend to study abroad may
not be entirely solidified prior to matriculation.
Summary
Despite a growing emphasis on postsecondary study abroad participation in
response to the increasing realities of living and working in a “new global century”
((AAC&U, 2007; APLU, 2004; Green, Luu, & Burris, 2008; NAFSA, 2003),
international education advocates have yet to fully demonstrate that participating in a
study abroad program will improve intercultural competence (Vande Berg, 2007). As the
above review of literature suggests, this challenge is highlighted by the lack of
theoretically grounded, methodologically rigorous research on the effects of study abroad
participation that fully accounts for the array of potentially confounding factors. This
includes a pre-test/post-test data collection, a large sample gathered from a range of
institutional types, data accounting for a wide array of pre-college characteristics,
educational interests and attitudes, college experiences, and a conceptual model grounded
in an empirically-vetted theoretical framework addressing the impact of college and
college experiences on student development.
In the absence of such a study, the present body of research examining the effect
of study abroad on intercultural competence has not adequately addressed 1) whether the
findings can be attributed to the study abroad experience itself or are in fact the result of
45
pre-treatment differences between those who participate and those who do not, and 2)
whether findings identified immediately after the conclusion of the study abroad
experience are accentuated, mitigated, or otherwise affected by the overall college
experience within which participation in a study abroad program exists. The present
study endeavors to address all of these design factors by 1) estimating the effects of study
abroad participation on intercultural competence while accounting for the range of
demographic and affective characteristics on which prior research has shown that
students who do and students do not study abroad differ and 2) examining whether the
effects of studying abroad on intercultural competence are conditional and differ based
upon the existence of specific pre-college characteristics.
46
CHAPTER III
CONCEPTUAL FRAMEWORK AND METHODOLOGY
Conceptual Framework
The conceptual frameworks grounding this study are Pascarella’s (1985) General
Model for Assessing the Effects of Differential Environments on Student Learning and
Cognitive Outcomes and Astin’s (1970a, 1970b) input - environment - output (I-E-O)
model. Astin’s I-E-O model asserts that the impact on a given outcome of the college
experience generally or a specific activity within the college experience is a function of
1) the pre-college characteristics of the student and 2) the college environment within
which the student engages in a given program, activity, or curriculum. Pascarella’s
General Model for Assessing the Effects of Differential Environments of Student
Learning and Cognitive Outcomes expands and extends the I-E-O model to more
precisely represent the symbiotic relationship between the postsecondary institution and
the student’s educational experience within it. Pascarella portrayed his model as a
function of five categories of variables that can directly or indirectly affect student
learning toward educational outcomes. These categories include: 1)
structural/organizational characteristics of institutions such as enrollment size,
faculty/student ratio, selectivity, and the percentage of students living on campus, 2)
student background/pre-college traits such as aptitude, achievement, personality,
aspiration, and ethnicity, 3) interaction with agents of socialization such as interaction
with faculty or peers, 4) institutional environment, and 5) quality of student effort. In this
model Pascarella asserted that the structural/organizational characteristics of the
institution and the student background/pre-college traits influenced each other. In
47
addition, they both shaped the institutional environment. Furthermore, the characteristics
of the institution and the background of the students as well as the institutional
environment influenced interactions with agents of socialization. The quality of the
student effort was influenced by the student background traits, the institutional
environment, and interactions with agents of socialization. Finally, student learning and
cognitive development could be understood as a function of student background/precollege traits, interactions with agents of socialization, and the quality of the student
effort.
The goal of the present study is to examine the effect of study abroad participation
on intercultural competence after accounting for the factors that predict participation in
study abroad or might otherwise account for change in the outcome of interest.
Furthermore, this study endeavors to examine whether the effects of studying abroad on
intercultural competence are conditional or general, i.e., whether the effect of studying
abroad differs based upon differences in the pre-college characteristics of the student,
differences in structural or environmental characteristics of the institution, or differences
in the college experiences of the student. Consequently, the conceptual model for this
study is derived from Pascarella’s (1985) model and is populated by variables identified
through prior research as predictors of study abroad participation among college students
and the variables that previous research has identified as predictors of change in
intercultural competence among students who study abroad.
Figure 1 portrays the hypothesized conceptual model for this study. As in
Pascarella’s (1985) model, institutional characteristics and pre-college student
characteristics influence each other. Institutional characteristics and pre-college student
48
characteristics both influence agents of socialization. Institutional characteristics precollege student characteristics, and agents of socialization influence the quality of student
effort. The treatment of interest, i.e., participation in study abroad, is influenced by precollege student characteristics and quality of student effort. Finally, the outcome of
interest, in this case intercultural competence, is influenced by the pre-college student
characteristics, the agents of socializations, and the treatment of interest – study abroad
participation.
Methodology
This study analyzes data from the 2006 cohort of the Wabash National Study of
Liberal Arts Education (WNS). Funded by the Center of Inquiry in the Liberal Arts at
Wabash College, the WNS is a multi-institutional longitudinal study focused on
identifying the curricular and co-curricular experiences that impact student gains on the
educational outcomes traditionally associated with a liberal arts education. The
educational outcomes examined by the WNS are effective reasoning and problem
solving, well-being, inclination to inquire and lifelong learning, intercultural
effectiveness (i.e., intercultural competence), leadership, moral character, orientation
toward life and career, political orientation, and academic motivation (Pascarella &
Colleagues, 2007).
Sample
Responding to a national call for institutions to participate in the WNS, over 60
colleges and universities applied to be selected for the 2006 cohort. 19 four- and two-year
colleges and universities were purposefully selected in an effort to create a diverse
institutional sample. As a result, the institutional members of the 2006 cohort represent
49
the full range of institutional types and vary widely in selectivity, size, location, religious
affiliation, and public/private control. Three of the institutions are research universities,
three are regional, non-doctoral granting universities, two are community colleges, and
11 are small liberal arts colleges. Because the goal of the WNS was to examine the
effects of a liberal arts education, liberal arts colleges were intentionally over-sampled.
Because postsecondary study abroad participation predominantly occurs during the third
or fourth year of college, the two community colleges were eliminated from the current
study, resulting in a final institutional sample of 17 schools.
Since the WNS was designed as a longitudinal study and focused on identifying
factors that influenced change over time, data was collected from students at three points
during their college careers: at the beginning of the freshman year, at the end of the
freshman year, and in the second semester of the senior year. Each institution invited
first-time, full-time undergraduates to participate in one of two ways. For the larger
institutions, students from the incoming freshmen class were randomly invited to
participate. For the smaller institutions, the entire incoming freshmen class was invited to
participate. Students were promised confidentiality and a $50 stipend for participating in
each data collection. 4,193 students completed pencil and paper surveys during the first
data collection in the fall of 2006. These students were asked to provide extensive
information regarding demographic characteristics, family background, high school
experiences, aspirations for college and career, and personal attitudes, values, and beliefs.
Students were also asked to complete standardized instruments for each of the
educational outcomes associated with the WNS. In addition, the students provided
50
permission for the institutions to share institutional admissions data involving ACT, SAT,
or COMPASS scores and awarded financial aid.
2,953 students returned in the spring of 2007 to provide data at the end of the
freshman year. These students completed the same standardized instruments for each of
the educational outcome measures as well as a host of data regarding curricular and cocurricular experiences during the first year. In the spring of 2010 (the fourth year of the
study), 2,212 students returned to provide data during the third data collection. Once
again students were asked to complete each of the standardized instruments used to
measure each of the educational outcomes of interest to the WNS. The students also
provided data regarding their curricular and co-curricular experiences. In addition, the
institutions provided institutional data regarding enrollment and academic performance.
In total, 1,865 students provided data at all three data collection points.
Variables
The variables utilized in this study are conceptually portrayed in Figure 1. The
variables within each block were chosen for this study based upon prior research on study
abroad participation and its effect on intercultural competence. Variable selection was
constrained by the need to account for the “clustered” or “nested” nature of the data
(Groves, Fowler, Couper, Lepkowski, Singer, & Tourangeau, 2004). Properly accounting
for this issue requires that the total number of variables in the model not exceed the
number of institutions, or “clusters.” Thus, the selection of variables was limited to a total
of 17 variables – 1 dependent variable and 16 independent variables. Each of the
variables utilized in this analysis are further described in Tables 1 with the means,
51
standard deviations, and range of responses provided in Table 2. A correlation matrix of
all the variables included in this study is portrayed in Table 3.
A single variable addresses systemic differences in institutional characteristics. A
dichotomous variable was employed to indicate whether or not the student attended a
small liberal arts college or a large regional or research university based upon Carnegie
classification (liberal arts college=1, other=0). Potential differences in institutional
characteristics are also addressed by the statistical adjustment for the nested nature of the
data.
Variables chosen to represent pre-college student characteristics include sex
(male=1, female=0), race (white=1, non-white=0), ACT score (SAT and COMPASS
scores were converted to an ACT metric), a pretest score for intercultural competence
(MGUDS), an attitude toward literacy scale score (PATL), and the average of the
parents’ highest educational attainment. All of these variables were derived from data
collected at the beginning of the freshman year.
Variables in the block representing agents of socialization address a range of
curricular and co-curricular influences. Since study abroad participants have traditionally
come primarily from among the humanities, social science, and fine arts majors, a
dichotomous variable was created from data gathered at the end of the senior year to
identify student in these majors. Data addressing the three final potentially confounding
factors was gathered twice – once at the end of the freshman year and once at the end of
the senior year. Two categorical variables account for the number of hours per week the
student was involved in co-curricular activities during the freshman year and during the
senior year, respectively. Four additional variables address the two educational
52
experiences identified through the review of literature as potentially influential in
intercultural competency development. Like co-curricular involvement, these items were
measured once at the end of the freshman year and once at the end of the senior year. The
diverse experiences scale is a continuous variable derived from nine items that ask about
the frequency of various interactions across diverse differences. Finally, the integrative
learning scale is a continuous variable derived from seven items that ask about frequency
or degree that the student has integrated knowledge gained in one setting into an activity
or project set in a different context.
A single item asking whether the student intended to study abroad represents the
quality of the student effort. This item was converted to a dichotomous variable to
account for those students who intended to study abroad compared with those who did
not intend to study abroad or didn’t know (1, 0). This item was asked during the second
data collection at the end of the freshmen year. The treatment of interest for this study –
study abroad participation – was determined by a single item asked at the end of the
senior year.
Finally, the outcome of interest is measured by the MGUDS – one of the
standardized instruments chosen by the WNS to assess intercultural competence. The
Miville-Guzman Universality-Diversity Scale [MGUDS] (1999) was developed and
tested to measure an individual’s degree of universal-diverse orientation (UDO), i.e., the
degree to which an individual possesses an “attitude of awareness and acceptance of both
similarities and differences that exist among people” (Miville, Gelso, Pannu, Liu,
Touradji, Hollowa, & Fuertes, 1999). The MGUDS total scale is composed of three
subscales that contribute equally to the overarching construct. The Diversity of Contact
53
subscale measures the degree to which the respondent is interested in and values
engaging in diverse social and cultural activities. The Relativistic Appreciation subscale
measures the degree to which the respondent appreciates the importance of recognizing
both similarities and differences among diverse groups and the connection between that
recognition and their own growth. The Comfort with Difference subscale measures one’s
degree of comfort with the idea of diverse interactions. The authors found that this
instrument correlated in theoretically expected ways with a variety of other instruments
measuring empathy, racial identity, homophobia, and dogmatism (Miville et al). The
external validity of the MGUDS has been supported by studies that used the instrument to
successfully predict attitudes among first year college students toward diversity in a
college setting (Fuertes, Sedlacek, Rogers, & Mohr, 2000) and majority students’
perceptions of minority mental health counselors (Fuertes & Gelso, 2000; Fuertes &
Brobst, 2002).
While the original MGUDS included 45 items with 15 items in each subscale,
Fuertes, Miville, Mohr, Sedlacek, and Gretchen (2000) conducted a multi-stage factor
analysis study and derived an equally predictive short form of the MGUDS instrument
that reduced the length of the instrument to 15 questions with each subscale composed of
five questions. Lau and Finney (2006) replicated this factor analysis and found the
MGUDS short form to be comparably predictive of the underlying Universality-Diversity
construct and its component subscales. Figures 2 and 3 provide copies of the MGUDS
short form and its scoring key. Because of the exhaustive nature of the WNS and the
length of many of the instruments utilized by the study, the architects of the WNS chose
to use the MGUDS short form instead of the long form.
54
Alpha reliabilities for the subscales among first year students in the WNS ranged
from .77 to .78, while the alpha reliability for the total MGUDS score among first year
students was .85 (Pascarella & Colleagues, 2007). The predictive validity of the MGUDS
is exemplified by the numerous significant and theoretically expected correlations
between the MGUDS pretest scores and a variety of other WNS measures of attitudes and
activities that represent attitudinal elements or behavioral manifestations of intercultural
competence. Among WNS participants, the MGUDS pretest score correlates significantly
(p<.05) with the respondents’ openness to diversity as an important element of a college
education (.616), belief in the importance of cultural understanding (.511), and belief in
the importance of racial understanding (.459). Furthermore, the MGUDS pretest score
correlates significantly with how often respondents’ made friends with international
students during the first year (.324), made friends with students of another race during the
first year (.330), had serious conversations with students of a different race or ethnicity
(.335), and tried to better understand someone else’s views by imagining how an issue
looks from his or her perspective (.304).
Analysis
This study employs two methods of analysis to examine the effect of study abroad
on intercultural competence in order to ensure the robustness of the overall findings as
well as provide an opportunity to compare the results of two different approaches to
identifying the effects of participation in a non-randomly assigned treatment. First, this
study employs a stepwise, covariate adjustment method to examine the effect of study
abroad participation on intercultural competence while accounting for a variety of
potentially confounding variables (Gall, Gall, & Borg, 2003; McClendon, 1994). All
55
continuous variables were standardized prior to analysis in order to produce comparable
effect sizes. A series of four equations regresses the end of senior year MGUDS score on
the variables from the first five blocks of the model portrayed in Figure 1. The first
equation regresses the dependent variable on the pretest and study abroad participation.
The immediate inclusion of the pretest allows each resultant effect of all independent
variables to represent its respective effect on a change in intercultural competence. The
second equation adds all of remaining pre-college characteristics variables (gender, race,
pre-college tested academic preparation, parents’ educational attainment, socioeconomic
status, and positive attitude toward literacy scale) to the first equation. The third equation
adds the college experience variables (institutional type, major choice, co-curricular
involvement, diverse experiences and interactions scale, and integrative learning scale) to
the variables included in equation two. The fourth and final equation adds the potentially
confounding variable of intent to study abroad. The specific order of these equations
allows for the identification of a potential total effect followed by a potential direct effect,
which in turn allows for the identification of any moderating effects as additional
variables were added to the model. Importantly, because of the nested nature of the data,
analytic procedures are included to account for the clustering effect (Groves et al., 2004).
In addition to testing for a general effect, this study explores for the possibility of
conditional effects of study abroad, i.e. the possibility that the effect of studying abroad
on intercultural competence might differ based upon other characteristics. Normally,
potential interaction variables are created, added to the model, and tested to determine
whether their addition to the model significantly increases the model’s explanatory power
as measured by the R-square. However, because of the need to account for the clustering
56
effect, the maximum number of variables was limited to 16, the number of institutions
from which data in this study was gathered. To maintain the theoretical integrity of the
study, an intermediate step is employed to determine the plausibility of any conditional
effects prior to creating an interaction variable and introducing it into the full model.
This study explores for the possibility of conditional effects along seven different
variables - institutional type, sex, race, academic ability, pretest score, degree of diverse
experiences, and degree of integrative learning experiences. Thus, the procedure
described below was repeated seven times. First, the full sample is divided into two
groups. For the three dichotomous variables (race, gender, and institutional type), this
division point is self-evident. For the four continuous variables (academic ability, pretest
score, degree of diverse experiences, degree of integrative learning experiences), the
sample is divided at the mean. Second, the full regression equation is applied to both
subsets of the full sample. Third, a Wald Chi-Square test is employed to determine if the
difference between the coefficients generated by the study abroad participation variable is
significant and can therefore be attributed to the conditional trait by which the sample
was divided.
Identifying the existence of a conditional effect based upon gender presents a
unique challenge because some of the institutional samples do not include both men and
women who had studied abroad. Therefore, the clustering adjustment procedure cannot
be applied since the analysis of the male sample no longer included data from 16
institutions (i.e., aggregate units). Instead, an interaction variable that generated the cross
product of studying abroad and gender (male=1) is created and introduced directly to the
57
model. A global F-test is also employed to determine whether or not the introduction of
the new cross product variable significantly adds to the explanatory power of the model.
The second analytic method utilized to estimate the effect of studying abroad on
intercultural competence is propensity score matching. In recent decades social scientists
have expressed increasing concern about the validity and reliability of research
examining the effect of a particular program, activity, or experience – often termed
“treatment” – in observational data where participation in the treatment is not randomly
assigned (Dehejia & Wahba, 1999; LaLonde, 1986; Reynolds & DesJardins, 2009;
Rosenbaum & Rubin, 1983; Rubin, 2001, 2008; Schneider, Carnoy, Kilpatrick, Schmidt,
& Shavelson, 2007). This concern arises primarily from the inability of traditional
covariate adjustment methods to account for unobserved characteristics upon which those
who select to participate in the treatment of interest might systematically differ from
those who chose not to participate. Rosenbaum and Rubin (1983) devised an analytic
method by which a logistic regression equation is employed to create a propensity score
representing the likelihood of participation in the treatment for each member of the
sample. This new variable is then used to examine the effect of the treatment across cases
that share a similar propensity for selecting into the treatment, thus minimizing the
potential bias introduced by the observed “selection effect.” Although some
postsecondary education researchers have attempted to show the importance of this
method for examining a range of important research questions using observational data
(Brand & Halaby, 2006; Reynolds & DesJardins, 2009; Titus, 2007), other scholars have
questioned the perceived uniform effectiveness of propensity score matching to produce
more accurate estimates and have shown that this approach may not always improve the
58
accuracy of the findings and is vulnerable to a range of research design and analysis
decisions (Padgett, Salisbury, An, & Pascarella, in press; Peikes, Moreno, & Orzol, 2008;
Shadish, Clark, & Steiner, 2008; Steiner, Cook, Shadish, and Clark, in press).
Utilizing a propensity score matching method in the current study requires two
stages of analysis. First, a logistic regression model is constructed to predict participation
in study abroad. This model is grounded in the research reviewed in chapter 2 and mirrors
the predictive model for intent to study abroad created by Salisbury, Umbach, Paulson,
and Pascarella (2009). This model included all of the variables from Figure 1 with the
only difference being the operationalization of the academic major. Instead of two
dichotomous variables to isolate the effects of being a STEM and applied science major
or an applied pre-professional major, a single dichotomous variable is employed denoting
humanities, social science, and fine arts majors (1, 0). This logistic regression equation
produced a coefficient for each case in the sample that was converted to a probability of
participation in study abroad between zero and one and converted into a new variable
titled propensity score.
The second stage of this analytic method requires that cases are matched with
others bearing a similar propensity for study abroad participation. This matching
procedure allows the creation of several strata within which cases shared similar, but not
necessarily exact, propensity scores. Cases for which there were no comparable matches
were withdrawn from the analysis. Then, within each stratum, the senior year MGUDS
score means are compared between those who had and those had not studied abroad. A
weighted average of these results is then derived based on proportion of observations
within each stratum (Epstein, Allen, & Satten, 2007; Lunceford & Davidian, 2004).
59
The current study presents a unique opportunity to directly compare the findings
from covariate adjustment and propensity score matching. In particular, the existence of a
variable asking whether the student intends to study abroad tacitly functions as a
plausible representation of non-randomized selection. Furthermore, one analysis of
propensity score methods suggests that the inclusion of a pretest may eliminate the need
to employ propensity score matching (Steiner, Cook, Shadish, and Clark, in press). Thus,
the final stage of analysis for the current study re-ran the analytic techniques described
above with and without the MGUDS pretest measure and with and without the study
abroad intent variable to identify comparative reductions in bias.
Limitations
This study is not without limitations. First, the construct of intercultural
competence as defined by Deardorff (2006) clearly includes a behavioral component that
is difficult if not impossible to fully capture in a survey instrument. In the absence of a
behavioral component, some might argue that the MGUDS - the dependent measure
utilized by the Wabash National Study and the present investigation – only addresses the
capacity for intercultural competence rather than the situationally-specific demonstration
of intercultural competence.
Second, there are clear limitations resulting from the sample. Although extensive
efforts were made by the Wabash National Study to generate a diverse sample of
institutions and students, it is not nationally representative and therefore the result of this
study may be limited in their generalizability. The WNS purposefully oversampled liberal
arts colleges in an effort to remain true to its primary mission as an investigation of a
liberal arts education. In addition, the larger institutions included in the sample are not
60
intended to be representative of all Research I or Regional institutions as defined by
Carnegie classifications. Because of the variables necessary to represent the theoretical
framework of the study, the sample only includes students who participated in each of the
three WNS data collections. This may have unfairly limited the sample to those who did
not have competing obligations at the times that data was collected. The students in this
study were paid for their participation, which may have influenced their responses.
Furthermore, the students in this sample do not represent the overall racial and ethnic
diversity enrolled in American postsecondary education. While weighting is sometimes
used in an attempt to more accurately represent the population from which the sample
was derived, it is unclear how to employ weighting procedures within the propensity
score matching method chosen for this study. Since one of the goals of this study was to
compare the results of covariate adjustment and propensity score matching methods, the
decision was made to forego any attempt to weight findings to the sample’s originating
population. Finally, it is not possible to adjust for non-response bias.
Third, the analytic methodology chosen for this study required decisions based on
competing issues. Accounting for the clustered nature of the data limited the number of
variables allowed in the model. Thus, this study may have omitted important confounding
factors. Moreover, although the WNS collected an extensive range of data from its
respondents, it is possible that key variables for this study were not collected. The WNS
did not collect data regarding the various types of study abroad programs on which
student embarked. As such, this study could not account for differences in study abroad
program duration, destination, or degree of cultural immersion.
61
Fourth, study abroad programs vary widely by location, duration, and degree of
cultural immersion. Although these variations may well influence the effect of studying
abroad on intercultural competence, those details of the respondents’ study abroad
participation are not available in the WNS dataset. However, since further examination of
the kinds of programs available to students at all of the institutions in the WNS indicates
a comparable range of programmatic differences, any statistically significant findings
arising from this study can be considered foundational as an average effect across
programmatic variations and additional grounds for future research. To complicate
measures further, students who participate in study abroad do not do so at the same point
in their college careers. Although the WNS includes institutional measures that indicate if
students studied off-campus during a particular semester, it became clear through a series
of validity checks that the number of students indicating that they had studied abroad
substantially exceeded the institutional statistics of students who had studied off campus.
Since study abroad experiences do not always occur during the academic year and do not
always produce academic credit, this study chose to use student self-reports of studying
abroad as the independent variable of interest and, in light of this decision, was unable to
control for the potentially confounding factor of when the student chose to study abroad.
Finally, the measure of intent to study abroad is a dichotomous variable collected
at the end of the first year and may not fully represent or account for the degree to which
respondents have initiated actions that would move them from intent to participation.
Some students indicating that they intend to study abroad may have already met with a
study abroad advisor and be in the final stages of preparing to study abroad during the
sophomore year while others may have indicated intent to study abroad but only in the
62
most abstract sense. The degree to which the intent to study abroad variable fails to
capture these differences may also affect the results of this analysis.
(1) Institutional
Characteristics
Attends a Liberal Arts
College
(2) Pre-College
Characteristics
Gender
Race
ACT Score
Parents’ Education
Socioeconomic Status
Attitude toward Literacy
Pretest - Intercultural
Competence
(3) Agents of Socialization
Academic Major
Co-Curricular Involvement
Diverse Experiences
Integrative Learning Experiences
(6) Outcome of
Interest
(4) Quality of Student Effort
Intent to Study Abroad
Posttest –
Intercultural
Competence
(5) Treatment of Interest
Study Abroad Participation
Figure 1.
Conceptual Model to Examine the Effect of Study Abroad Participation on Intercultural Competence
63
64
Figure 2.
Miville-Guzman Universality-Diversity Scale – Short Form (MGUDS-S)
The following items are made up of statements using several terms which are
defined below for you. Please refer to them throughout the rest of the questionnaire.
Culture refers to the beliefs, values, traditions, ways of behaving, language of any social
group. A social group may be racial, ethnic, religious, etc.
Race or racial background refers to a sub-group of people possessing common physical
or genetic characteristics. Examples include White, Black, American Indian.
Ethnicity or ethnic group refers to specific social group sharing a unique cultural
heritage (i.e., customs, beliefs, language, etc.). Two people can be of the same race
(e.g., White), but be from different ethnic groups (e.g., Irish-American, Italian
American).
Country refers to groups that have been politically defined; people from these groups
belong to the same government (e.g., France, Ethiopia, United States). People of
different races (White, Black, Asian) or ethnicities (Italian, Japanese) can be from the
same country (United States).
Instructions: Please indicate how descriptive each statement is of you by filling in the
number corresponding to your response. This is not a test, so there are no right or wrong,
good or bad, answers. All responses are anonymous and confidential.
1
Strongly
Disagree
1._____
2
Disagree
3
Disagree
a little bit
4
Agree
a little bit
5
Agree
6
Strongly
Agree
I would like to join an organization that emphasizes getting to know people from different
countries.
2._____
Persons with disabilities can teach me things I could not learn elsewhere.
3._____
Getting to know someone of another race is generally an uncomfortable experience for me.
4._____
I would like to go to dances that feature music from other countries.
65
Figure 2 – continued.
5._____
I can best understand someone after I get to know how he/she is both similar and different
from me.
6._____
I am only at ease with people of my race.
7._____
I often listen to music of other cultures.
8._____
Knowing how a person differs from me greatly enhances our friendship.
9._____
It’s really hard for me to feel close to a person from another race.
10.____
I am interested in learning about the many cultures that have existed in this world.
11.____
In getting to know someone, I like knowing both how he/she differs from me and is similar
to me.
12.____
It is very important that a friend agrees with me on most issues.
13.____
I attend events where I might get to know people from different racial backgrounds.
14.____
Knowing about the different experiences of other people helps me understand my own
problems better.
15.____
I often feel irritated by persons of a different race.
© 1992 Marie L. Miville
Permission is granted for research and clinical use of the scale. Further permission must
be obtained before any modification or revision of the scale can be made.
66
Figure 3.
MGUDS-Short Form Scoring Key
Please write the Likert responses for each of the following items. Find the sum of the
responses to determine the subscale and total scale scores.
Subscale 1: Diversity of Contact
Item
1)
4)
7)
10)
13)
Likert Score
_____
_____
_____
_____
_____
Subtotal _______
Subscale 2: Relativistic Appreciation
Item
2)
5)
8)
11)
14)
Likert Score
_____
_____
_____
_____
_____
Subtotal ______
Subscale 3: Comfort with Differences
Item
3)
6)
9)
12)
15)
Likert Score
_____
_____
_____
_____
_____
Subtotal ______
67
Figure 3 – continued.
Items may or may not be reversed scored on Subscale 3. If you want to use total MGUDS-S scale
score (see below), then reverse score these items for scoring consistency.
TOTAL MGUDS-S SCORE –Sum Subscales 1, 2, and 3 subtotals
Subscale 1__________
Subscale 2__________ +
Subscale 3__________ +
TOTAL
=
__________
© 1992 Marie L. Miville
Permission is granted for research and clinical use of the scale and scoring key. Further
permission must be obtained before any modification or revision of the scale can be
made.
Table 1.
Names, Descriptions, and Response Options for Variables Used in this Study
Variable Name
- Intercultural
Competence Pretest
Description
The Miville-Guzman Universality Diversity Scale (MGUDS) short form
completed during the first data gathering at the beginning of the
freshmen year
The Miville-Guzman Universality Diversity Scale (MGUDS) short form
completed during the third data gathering at the end of the senior year
Response Options
See Figures 2 and
3
- Study Abroad
Respondent's self-report of participation in an undergraduate study
abroad experience gathered at the end of the senior year
Have not decided,
Do not plan to do,
Plan to do, Done
(Recoded into
Done=1, Other=0)
- Gender
Gender of respondent provided by institution during the first data
collection
Race of respondent provided by institution during the first data
collection
ACT, SAT, or COMPASS score provided by the institution during the
first data collection then converted to an ACT metric
Male=1, Female=0
- Intercultural
Competence Posttest
- Race
- Pre-College Tested
Academic
Preparation
See Figures 2 and
3
White=1, Other=0
scores range from
16 to 36
68
Table 1 – continued.
- Parents' Educational
Attainment
Average of the respondent's reported parents' educational attainment
provided during the first data collection at the beginning of the freshmen
year (If only one parent's score was provided, this score was used in
place of an average)
scores range from
11 (did not finish
high school) to 20
(both parents have
earned a doctorate)
- Socioeconomic
Status
Institutional data indicating if the respondent met the threshold for
federal financial aid and received a Pell Grant
Yes=1, No=0
- Positive Attitude
Toward Literacy
Scale
A six item scale measuring the degree to which the respondent enjoys
and benefits from reading and writing (WNS Alpha=.69). Constituent
items include:
- Institutional Type
- Humanities, Fine
Arts, or Social
Science Major
- I enjoy reading poetry and literature.
- I enjoy reading about science.
- I enjoy reading about history.
- I enjoy expressing ideas in writing.
- After writing about something, I see that subject differently.
- If I have something good to read, I am never bored.
Carnegie Classification of each institution participating in the WNS
based on Carnegie Foundation data
Respondent majored in a humanities, fine arts, or social science field
strongly disagree,
disagree, neutral,
agree, strongly
agree
Liberal Arts
College=1,
Other=0
Yes=1, No=0
69
Table 1 – continued.
- Co-Curricular
Involvement *
Respondent's self-report of participation in co-curricular activities such
as student organizations, campus publications, student government,
greek organizations, athletics, etc.
- Diverse Experiences
Scale *
A nine item scale measuring the extent to which respondents' engaged in
conversations, interactions, and experiences with diverse individuals
involving issues of difference (WNS Alpha=.80). Constituent items
include:
- How often did you attend a debate or lecture on a current
political/social issue during this academic year?
- How often did you have serious discussions with staff whose
political, social, or religious opinions were different than your
own?
- How often did you participate in a racial or cultural awareness
workshop during this academic year?
0 hours, 1-5 hours,
6-10 hours, 11-15
hours, 16-20
hours, 21-25
hours, 26-30
hours, more than
30 hours
Very often, Often,
Sometimes,
Rarely, Never
- Indicate the extent to which your institution emphasizes
encouraging contact among students from different economic,
social, and racial or ethnic backgrounds.
Very little, Some,
Quite a bit, Very
much
- During the current school year, how often have you had serious
conversations with students of a different race or ethnicity than
your own?
Never, Sometimes,
Often, Very often
70
Table 1 – continued.
- During the current school year, how often have you had serious
conversations with students who are very different from you in
terms of their religious beliefs, political opinions, or personal
values?
- How often have you had discussions regarding intergroup relations
with diverse students while attending this college?
- How often have you had meaningful and honest discussions about
issues related to social justice with diverse students while attending
this college?
Very often, Often,
Sometimes,
Rarely, Never
- How often have you shared personal feelings and problems with
diverse students while attending this college?
- Integrative Learning
Experiences Scale *
A nine item scale measuring the extent to which the respondent has
integrated knowledge, information, or skills gained in one setting into
learning experiences engaged in a different environment (WNS
Alpha=.76). Constituent items include:
- My courses have helped me understand the historical, political, and strongly disagree,
social connections of past events.
disagree, neutral,
agree, strongly
- My courses have helped me see the connections between my
agree
intended career and how it affects society.
- Out of class experiences have helped me connect what was learned
in the classroom with life events.
- Out of class experiences have helped me translate knowledge and
understanding from the classroom into action.
Very often, Often,
Sometimes,
Rarely, Never
71
Table 1 – continued.
- During the current school year, how often have you worked on a
paper or project that required integrating ideas or information from
various sources?
Never, Sometimes,
Often, Very often
- During the current school year, how often have you put together
ideas or concepts from different courses when completing
assignments or during class discussions?
- During the current school year, how often have you discussed
ideas from readings or classes with other outside of class (students,
family members, co-workers, etc.)?
- During the current school year, how much time have your spent
synthesizing and organizing ideas, information, or experiences into
new, more complex interpretations and relationships?
Very little, Some,
Quite a bit, Very
much
- During the current school year, how much time have you spent
making judgments about the value of information, arguments, or
methods, such as examining how others gathered and interpreted
data, and assessing the soundness of their conclusions?
72
Table 1 – continued.
- Intent to Study
Abroad
Respondent's self-report of intent to participate in an undergraduate
study abroad experience gathered during the second data collection at
the end of the freshmen year
Have not decided,
Do not plan to do,
Plan to do, Done
(Recoded into Plan
to do=1, Other=0;
38 respondents
who indicated
"Done" at this data
gathering point
were eliminated
from the analysis)
* Two versions of these variables are used in this analysis, one from data collected at the end of the first year and one from data
collected at the end of the fourth year.
73
74
Table 2.
Descriptive Statistics for Variables Included in the Analysis of General Effects a (N=1,593)
Variable
MGUDS Total Score - Pretest
- Diverse Contact Subscale Score - Pretest
- Comfort with Diversity Subscale Score - Pretest
- Relativistic Appreciation Subscale Score - Pretest
MGUDS Total Score - Posttest
- Diverse Contact Subscale Score - Posttest
- Comfort with Diversity Subscale Score - Posttest
- Relativistic Appreciation Subscale Score - Posttest
Study Abroad
Gender (Male=1)
Race (White=1)
Pre-College Tested Academic Preparation
Parents' Educational Attainment
Socioeconomic Status (Pell Grant Recipient=1)
Positive Attitude Toward Literacy Scale
Institutional Type (Liberal Arts College=1)
Humanities, Fine Arts, or Social Science Major
Co-Curricular Involvement - First Year
Diverse Experiences - First Year
Integrative Learning Experiences - First Year
Co-Curricular Involvement - Fourth Year
Diverse Experiences - Fourth Year
Integrative Learning Experiences - Fourth Year
Intent to Study Abroad
a
all continuous variables are standardized
Mean
0
0
0
0
0
0
0
0
.438
.347
.814
0
0
.129
0
.537
.433
0
0
0
0
0
0
.660
St. Dev.
1
1
1
1
1
1
1
1
.496
.477
.390
1
1
.335
1
.499
.496
1
1
1
1
1
1
.474
Min.
-5.347
-3.407
-4.743
-5.583
-4.195
-3.441
-5.377
-5.253
0
0
0
-2.738
-2.175
0
-3.258
0
0
-1.137
-2.291
-3.428
-1.119
-2.475
-3.759
0
Max
2.237
1.879
1.558
1.791
2.107
1.879
1.440
1.690
1
1
1
2.074
2.063
1
2.338
1
1
3.486
2.867
2.554
3.316
3.052
1.955
1
Table 3.
Correlation Matrix for All Variables Included in this Study
1
1
2
3
4
5
6
7
8
MGUDS Total Score - Pretest
2
3
4
5
6
7
9
10
11
12
1.000
- Diverse Contact Score - Pretest
0.866
1.000
- Comfort with Diversity Score - Pretest
0.727
0.422
1.000
- Relativistic Appreciation Score - Pretest
0.726
0.508
0.276
1.000
0.624
0.573
0.421
0.443
1.000
- Diverse Contact Score - Posttest
0.567
0.628
0.307
0.335
0.855
1.000
- Comfort with Diversity Score - Posttest
0.441
0.310
0.501
0.212
0.713
0.407
1.000
- Relativistic Appreciation Score - Posttest
MGUDS Total Score - Posttest
8
0.409
0.327
0.166
0.492
0.724
0.458
0.280
1.000
9
10
Study Abroad
0.144
0.185
0.062
0.067
0.204
0.275
0.088
0.070
1.000
Gender (Male=1)
-0.184
-0.174
-0.103
-0.147
-0.134
-0.137
-0.048
-0.115
-0.067
1.000
11
12
Race (White=1)
-0.163
-0.154
-0.101
-0.120
-0.146
-0.151
-0.084
-0.090
0.094
0.038
1.000
Pre-College Tested Academic Preparation
0.036
0.031
0.066
-0.021
0.018
0.018
0.034
-0.014
0.208
0.072
0.249
1.000
13
14
Parents' Educational Attainment
0.059
0.085
0.051
-0.014
0.049
0.083
0.029
-0.016
0.153
0.052
0.211
0.356
SES (Pell Grant Recipient=1)
0.031
0.033
-0.008
0.047
0.061
0.065
0.027
0.044
-0.076
-0.057
-0.202
-0.260
15
16
17
Positive Attitude Toward Literacy Scale
0.415
0.427
0.218
0.300
0.346
0.037
0.180
0.221
0.124
-0.113
0.012
0.225
Institutional Type (Liberal Arts College=1)
-0.002
0.019
-0.038
0.013
0.035
0.074
-0.010
0.000
0.146
0.042
0.036
-0.215
Humanities, Fine Arts, or Social Science Major
0.196
0.192
0.139
0.113
0.181
0.223
0.093
0.075
0.141
-0.008
-0.072
0.009
18
19
Co-Curricular Involvement - First Year
-0.023
-0.029
-0.023
0.005
-0.021
-0.021
-0.041
0.016
0.058
0.140
0.130
0.010
Diverse Experiences - First Year
0.392
0.371
0.270
0.256
0.383
0.360
0.269
0.237
0.092
0.043
-0.218
0.048
20
21
Integrative Learning Experiences - First Year
0.266
0.206
0.167
0.263
0.293
0.222
0.198
0.263
0.099
0.009
-0.034
0.012
Co-Curricular Involvement - Fourth Year
-0.008
-0.023
0.006
0.006
0.023
0.032
0.008
0.009
0.069
0.105
0.045
-0.006
22
23
Diverse Experiences - Fourth Year
0.345
0.329
0.235
0.225
0.515
0.490
0.360
0.310
0.102
0.045
-0.206
-0.028
Integrative Learning Experiences - Fourth Year
0.215
0.165
0.126
0.222
0.379
0.310
0.230
0.337
0.144
-0.075
-0.025
-0.037
24
Intent to Study Abroad
0.207
0.259
0.096
0.098
0.242
0.295
0.109
0.121
0.415
-0.118
0.008
0.079
75
Table 3 – continued.
13
1
2
3
4
5
14
15
16
17
18
19
20
21
22
23
24
MGUDS Total Score - Pretest
- Diverse Contact Score - Pretest
- Comfort with Diversity Score - Pretest
- Relativistic Appreciation Score - Pretest
MGUDS Total Score - Posttest
6
7
- Diverse Contact Score - Posttest
8
9
- Relativistic Appreciation Score - Posttest
- Comfort with Diversity Score - Posttest
Study Abroad
10
11
Gender (Male=1)
12
13
14
Pre-College Tested Academic Preparation
Race (White=1)
Parents' Educational Attainment
1.000
SES (Pell Grant Recipient=1)
-0.278
1.000
15
16
Positive Attitude Toward Literacy Scale
0.112
-0.008
1.000
Institutional Type (Liberal Arts College=1)
-0.076
0.150
0.050
1.000
17
18
Humanities, Fine Arts, or Social Science Major
0.060
0.054
0.184
0.195
1.000
Co-Curricular Involvement - First Year
0.044
-0.012
-0.031
0.090
0.000
1.000
19
20
Diverse Experiences - First Year
0.004
0.082
0.267
0.014
0.182
0.042
1.000
Integrative Learning Experiences - First Year
0.017
0.035
0.263
0.106
0.130
0.095
0.476
1.000
21
22
Co-Curricular Involvement - Fourth Year
0.059
-0.005
-0.041
0.133
0.023
0.497
0.098
0.110
1.000
Diverse Experiences - Fourth Year
-0.009
0.110
0.263
0.095
0.222
0.083
0.581
0.371
0.177
1.000
23
24
Integrative Learning Experiences - Fourth Year
-0.017
0.042
0.183
0.134
0.139
0.077
0.310
0.531
0.141
0.470
1.000
Intent to Study Abroad
0.095
-0.069
0.176
0.103
0.190
0.051
0.164
0.067
0.031
0.165
0.111
1.000
76
77
CHAPTER IV
RESULTS
The primary goals of this study were 1) to examine the effect of study abroad on
intercultural competence in the presence of pre-college characteristics, educational
aspirations, and college experiences, and 2) to explore whether the effect of studying
abroad is general or conditional (i.e., whether the intercultural competence of diverse
student types who study abroad is systematically effected differently). To address these
questions, this study utilized two analytical methods – covariate adjustment and
propensity score matching. This allowed the study to compare and contrast specific
findings derived from each method and strengthened the overall validity of the study.
Descriptive Statistics
Of the 1,865 students who provided data at all three WNS data collection points,
234 cases were removed due to missing data. In addition, 38 cases were removed because
these respondents indicated at the end of the first year that they had already participated
in study abroad (thus eliminating the opportunity to account for study abroad intent since
this particular variable was gathered at the end of the freshmen year). The final analytic
sample for this study included 1,593 students. Table 2 portrays the descriptive statistics
for all of the variables included in this study.
Among the students in the final sample, 693 students (43.5%) indicated that they
had studied abroad. While this far exceeds the national proportion of students who study
abroad, it allowed for a robust comparison of students who did and did not study abroad.
Of those who studied abroad, 477 were female and 216 were male. This distribution
slightly exceeds the national female/male ratio of study abroad participation of 2:1 (IIE,
78
2009). The proportion of study abroad participants in our sample identifying as white
(85.57%) is slightly higher the national percentage of study abroad students who identify
as white (IIE, 2009).
Analytic Results
Covariate Adjustment
The results of the stepwise covariate adjustment analyses are depicted in Table 4.
Since all continuous variables were standardized prior to analysis, the coefficients
generated by each independent variable can also be understood to represent the size of
their explanatory effects. The first stage of analysis examined the total effect of studying
abroad on the MGUDS measure of intercultural competence after accounting for the
parallel MGUDS pretest score. This equation accounts for a substantial portion of the
variance (R2 = .403; p < .001). Both the pretest and study abroad participation generated
significant effects (p < .001) of .607 and .236, respectively.
The second stage of the analysis adds the pre-college characteristic variables of
gender, race, pre-college tested academic preparation, parents’ educational attainment,
socioeconomic status, and the positive attitude toward literacy scale score to the variables
included in first equation. The addition of these variables marginally increases the
explanatory power of the model (R2 = .418; p < .001), with the race variable (White=1)
generating a significant negative effect (-.145, p < .01) and the positive attitude toward
literacy scale generating a significant positive effect (.109, p < .001). The effect of study
abroad participation remained significant (p < .001), its effect size increasing slightly to
.250. The pretest continued to generate a large effect of .550 (p < .001), albeit somewhat
smaller than its effect size in the first equation.
79
The third stage of the analysis adds a series of college experience variables
(institutional type, major choice, and senior year co-curricular involvement, diverse
experiences, and integrative learning experiences) to the list of variables included in the
second stage. The addition of these variables substantially increased the explanatory
power of the model (R2 = .521; p < .001). Moreover, two of the five variables added to
the model in this stage generated significant effects. A one standard deviation increase in
the diverse experiences scale predicts .280 of a standard deviation increase in the
dependent variable (p < .001). In addition, a one standard deviation increase in the
integrative learning scale predicts a .133 increase in the dependent variable (p < .001).
The addition of these variables reduced the effect of studying abroad to .187, and the
effect remains significant at p < .001. As might be expected, the pretest continues to
generate a large and significant effect of .458 (p < .001).
The fourth and final stage of the covariate adjustment analysis adds a single
variable to those utilized in the third stage. This variable represents quality of student
effort in the conceptual framework (see block 4 in Figure 1) and is operationalized for
this study by intent to study abroad. While the addition of this variable produces a
marginal increase in the model’s variance explained (R2 = .523; p < .001), an adjusted
Wald test reveals that the addition of the intent to study abroad variable significantly
improves the explanatory power of the model [F (1, 15) = 6.87; Prob > F = .0193]. Study
abroad participation continues to generate a significant positive effect of .147 in the final
model (p < .001). Again, the effect of the pretest score is comparatively large (.454) and
significantly predictive of the post-test score of intercultural competence (p < .001).
80
Propensity Score Matching
In recent years, scholars of educational research have strongly encouraged higher
education researchers to apply increasingly rigorous and more precisely appropriate
analytical methods in postsecondary education research, particularly in examining the
effect of programs or “treatments” where selection occurs nonrandomly, i.e., students
choose whether or not to participate (Reynolds & Desjardins, 2009; Schneider et al,
2007). Therefore, in addition to covariate adjustment methods, the current study also
examined the effect of study abroad on intercultural competence using a propensity score
matching method available through the psmatch2 command in the STATA statistical
software package (Leuven & Sianesi, 2003). The results of this analysis are portrayed in
Table 5. This method reduced the analytic sample to 1,576, eliminating cases that did not
fall within boundaries of common support, i.e., groupings of study abroad participants
and non-participants who had a similar propensity to participate in study abroad. In
addition, because propensity score matching is designed to specifically account for the
selection effect occurring pre-treatment, the only independent variables included in the
model had to have occurred pre-treatment. Therefore, the end of freshman year measures
of co-curricular involvement, diverse experiences, and integrative learning experiences
were used in place of the equivalent senior year measures. As a result, this model did not
account for any potentially confounding factors occurring simultaneous to the treatment
of study abroad.
Given these limitations, it may not be surprising that the result of this
methodological approach overestimates the effect of study abroad when compared to the
findings of the covariate adjustment model (Table 5). While study abroad participation
81
generated a significant positive effect on intercultural competence (p < .01) using the
psmatch2 command, the effect size generated by the propensity score matching method
(.176) is 19.7% larger than the effect generated by covariate adjustment (.147). Not
surprisingly, the standard error derived from the propensity score matching model (.061)
was substantially higher than that resulting from covariate adjustment (.040) and, as a
consequence, the t-statistic derived from the propensity score matching method (2.19)
was smaller than the t-statistic that emerged from the covariate adjustment model (3.55).
A benefit of the stepwise covariate adjustment model in this study (Table 4) is the
opportunity it provides to see how the inclusion of each additional group of potentially
confounding independent variables influences the effect size of study abroad on
intercultural competence. In an attempt to both generate similar comparative data using a
propensity score matching method and to better understand how different combinations
of variables might affect the results of propensity score matching methods, the propensity
score matching method was repeated fifteen times using each potential combination of
independent variable blocks (including each block individually) as they were utilized in
the stepwise covariate adjustment model. Simply described, these blocks include the
pretest, pre-college characteristics, college experiences, and study abroad intent.
The results of these analyses are portrayed in Table 6. Interestingly, the least
biased effect results from a model that includes only the pretest, the college experiences,
and the intent to study abroad variable (.175; t = 3.08). By contrast, the full model –
which adds the pre-college characteristics in addition to the variable blocks listed above –
generates a marginally larger coefficient, albeit a small t - statistic (.176; t = 2.19). The
study abroad intent variable appears to be of primary importance in generating an
82
accurate propensity score matching equation with the pretest also playing an important
role. By comparison, the model that utilized only the pre-college characteristics produced
the most biased effect (.371; t = 7.27). Moreover, in several instances it appears that the
pre-college characteristics block did nothing to reduce estimation bias, and in some cases
it appears that this block of variables actually added bias to the estimation of an effect.
Including a Propensity Score in a Covariate Adjustment Model
Comparing these two analytic approaches reveals a set of competing
methodological benefits and limitations. Although the covariate adjustment model
includes a measure of study abroad intent that might function as a measure of selection,
the nonrandom nature of the study design remains, leaving open the possibility of
existing selection effect bias. Furthermore, the need to account for the clustered nature of
the data limited the number of variables in the model to the number of aggregates (i.e.
postsecondary institutions) included in the study, ultimately requiring that this model
omit the three freshman year college experience variables (co-curricular involvement,
diverse experiences, and integrative learning experiences) that prior research has found to
be influential in predicting study abroad intent (Salisbury, Umbach, Paulsen, &
Pascarella, 2008). Conversely, since the propensity score matching method is designed to
account for selection but not for potentially confounding variables that occur
simultaneous to the treatment, that model did not include the three senior year college
experience variables accounting for co-curricular involvement, diverse experiences, and
integrative learning which the covariate adjustment model suggests are important in
accurately estimating the effect of study abroad on intercultural competence.
83
Another way to use a propensity score in estimating the effect of a given
treatment is to include it in a traditional covariate adjustment equation (Shadish, Clark, &
Steiner, 2008). This allows the model to account for both potentially influential factors
that occur simultaneous to the treatment while at the same time statistically accounting
for potential selection bias. To this end, a regression equation was constructed using all of
the variables included in this study (pretest, pre-college characteristics, major,
institutional type, and both freshman year and senior year college experience variables)
and the propensity score created by the psmatch2 command during the previous stage of
this analysis. As a comparison, the regression equation was repeated using the intent to
study abroad variable in place of the propensity score. Finally, because the number of
variables in these models exceeds the number of institutions from which the data was
gathered, the command to account for the clustered nature of the data could no longer be
used, leaving open the possibility that the equation would generate underestimated
standard errors and increasing the possibility of Type I error. In response, the threshold
for reporting significance was reduced to p<.01 from the traditionally accepted p<.05.
Table 7 portrays the results of the full covariate adjustment model including the
propensity score (Model 1) as well as the comparative findings of the full covariate
adjustment model including a direct measure of intent to study abroad (Model 2). In both
cases, study abroad participation produced the identical significant positive effect (p <
.01) of .147; the same effect size generated by the clustered covariate adjustment model
employed in the first stage of this study. While slight differences in effect sizes appear
throughout the two models and in comparison with the prior clustered covariate
adjustment model, these differences appear to be minimal. As might be expected, when
84
the propensity score is included in the model, the intent to study abroad variable is no
longer significant, likely due to substantial collinearity with the propensity score.
In addition to examining the effect of study abroad on the MGUDS total score,
this study sought to better understand the nature of this effect by examining the effect of
study abroad on each of the subscales of the MGUDS. The three subscales measure the
respondents’ inclination to engage in diverse contact, the respondents’ comfort with
diverse interactions, and the respondents’ relativistic appreciation of differences between
cultures in values, attitudes, and perspectives. The descriptive statistics of the pretest and
posttest measures of each subscale are portrayed in Table 2.
Based on the findings of the full covariate adjustment model portrayed in Table 7,
analysis of study abroad participation’s effects on each of the MGUDS subscales utilized
the full covariate adjustment model above including the direct measure of study abroad
intent. The results of these analyses are portrayed in Table 8. Study abroad participation
generated a significant (p < .001) and comparatively large positive effect on the diverse
contact subscale. However, the effect of study abroad on both the comfort with diversity
and relativistic appreciation subscales was nonsignificant.
Finally, this study endeavored to explore whether the effect of study abroad was
general or conditional, i.e. whether the effect of study abroad on intercultural competence
differed based upon variations of other independent variables in the model. Examination
of potential differences based upon institutional type, race, pre-college tested preparation,
pretest score, degree of diverse experiences, and degree of integrative learning
experiences produced no evidence of conditional effects. Although the exploration of
conditional effects by gender produced some indication that study abroad might affect
85
men and women differently, the results of this analysis did not meet the significance
threshold of p<.01 set to address the possibility of underestimated standard errors and
subsequent Type 1 error.
86
Table 4.
The Effect of Study Abroad on Intercultural Competence Using Covariate
Adjustment while Accounting for the Clustering Effect (N=1,593)
Variable
Intercultural Competence Pretest
Study Abroad
Model 1
.607 ***
(.024)
.236 ***
(.041)
Gender (Male=1)
Race (White=1)
Pre-College Tested Academic Preparation
Parents' Educational Attainment
Socioeconomic Status (Pell Grant=1)
Positive Attitude Toward Literacy
Model 2
.550 ***
(.019)
.250 ***
(.034)
-.012
(.051)
-.145 **
(.045)
-.035
(.025)
.021
(.022)
.118
(.070)
.109 ***
(.020)
Institutional Type (Liberal Arts
College=1)
Humanities, Fine Arts, or Social Science
Major
Co-Curricular Involvement - Fourth Year
Diverse Experiences - Fourth Year
Integrative Learning Experiences - Fourth
Year
Model 3
.458 ***
(.018)
.187 ***
(.034)
-.071
(.037)
-.033
(.042)
-.018
(.031)
.025
(.024)
.057
(.064)
.045
(.018)
Model 4
.454 ***
(.017)
.147 **
(.040)
-.062
(.038)
-.033
(.043)
-.017
(.032)
.024
(.024)
.068
(.067)
.042
(.019)
-.041
(.057)
-.045
(.059)
-.019
(.046)
-.040
(.019)
.280
(.024)
-.029
(.043)
-.040
(.019)
.275
(.023)
.133
(.027)
***
***
Intent to Study Abroad
R-squared
* p<.05, ** p<.01, *** p<.001
.403
***
.418
***
.521
***
.135
(.028)
.114
(.043)
.523
***
***
*
**
87
Table 5.
A Comparison of Results derived from Covariate Adjustment and Propensity
Score Matching Analyzing the Effect of Study Abroad on Intercultural
Competence
Method of Analysis
N
Effect
St. Error
t - Statistic
Covariate Adjustment a
1,593
.147**
.040
3.69
Propensity Score
Matching b
1,576
.176**
.061
2.89
a
This model accounts for the intercultural competence pretest score, gender, race, pre-college
tested academic preparation, parents' educational attainment, socioeconomic status, positive
attitude toward literacy scale, institutional type, major, senior year co-curricular involvement,
senior year diverse experiences, senior year integrative learning experiences, intent to study
abroad, and the clustering effect.
b
This model accounts for all equivalent pre-treatment variables: intercultural competence pretest
score, gender, race, pre-college tested academic preparation, parents' educational attainment,
socioeconomic status, positive attitude toward literacy scale, institutional type, major, first year
co-curricular involvement, first year diverse experiences, first year integrative learning
experiences, and intent to study abroad.
* p<.05, ** p<01, *** p<.001
Table 6.
A Comparison of Propensity Score Matching Results using Various Combinations of Independent Variable Blocks to Generate the Propensity
Score (listed in order from least to most biased) a
Combination of Included Variables
Pretest, college experiences, and study abroad intent
Pretest, pre-college characteristics, college experiences, and study abroad intent
Pretest, pre-college characteristics, and study abroad intent
College experiences and study abroad intent
Pretest and study abroad intent
Pre-college characteristics, college experiences, and study abroad intent
Pretest, pre-college characteristics, and college experiences
Study abroad intent only
Pretest and college experiences
Pre-college characteristics and study abroad intent
Pretest and pre-college characteristics
Pretest only
Pre-college characteristics and college experiences
College experiences only
Pre-college characteristics only
a
N
1,577
1,576
1,557
1,587
1,577
1,579
1,565
1,593
1,586
1,567
1,576
1,592
1,560
1,584
1,590
Coef.
.175**
.176**
.187***
.200***
.211***
.212***
.221***
.231***
.235***
.238***
.261***
.289***
.300***
.326***
.371***
St. Error
.057
.058
.058
.057
.055
.061
.054
.056
.050
.058
.051
.049
.054
.050
.051
t - Statistic
3.08
2.19
3.21
3.52
3.85
3.49
4.08
4.14
4.69
4.05
5.08
5.87
5.55
6.49
7.27
These equations includes blocks of variables organized by the pretest (MGUDS measure of intercultural competence gathered at the beginning of the
freshmen year), pre-college characteristics (gender, race, pre-college tested academic preparation, parents' educational attainment, socioeconomic status,
positive attitude toward literacy scale), college experiences (institutional type, major, first year co-curricular involvement, first year diverse experiences, first
year integrative learning experiences) and intent to study abroad.
* p<.05, ** p<01, *** p<.001
88
89
Table 7.
The Effect of Study Abroad on Intercultural Competence Comparing Two Covariate Adjustment
Regression Equations to Explore the Impact of a Propensity Score vs. a Direct Measure of Study
Abroad Intent on the Overall Model (N=1,593).
Variable
Intercultural Competence Pretest
Study Abroad
Gender (Male=1)
Race (White=1)
Pre-College Tested Academic Preparation
Parents' Educational Attainment
Socioeconomic Status (Received Pell Grant=1)
Positive Attitude Toward Literacy
Institutional Type (Liberal Arts College=1)
Humanities, Fine Arts, or Social Science Major
Co-Curricular Involvement - First Year
Diverse Experiences - First Year
Integrative Learning Experiences - First Year
Co-Curricular Involvement - Fourth Year
Diverse Experiences - Fourth Year
Integrative Learning Experiences - Fourth Year
Propensity Score
Intent to Study Abroad
R-squared
** p<.01, *** p<.001
Model 1
.462
(.025)
.147
(.041)
-.051
(.044)
-.030
(.054)
-.014
(.046)
.024
(.022)
.071
(.056)
.045
(.022)
-.043
(.073)
-.028
(.040)
-.030
(.021)
-.027
(.024)
-.013
(.025)
-.025
(.021)
.289
(.024)
.143
(.022)
-.022
(.438)
0.126
(.169)
.524
***
**
***
***
***
Model 2
.462 ***
(.021)
.147 ***
(.040)
-.050
(.039)
-.031
(.050)
-.016
(.021)
.023
(.019)
.072
(.056)
.045
(.020)
-.046
(.038)
-.028
(.038)
-.030
(.020)
-.027
(.024)
-.014
(.023)
-.025
(.021)
.289 ***
(.024)
.143 ***
(.022)
.118
(.042)
.524
**
**
90
Table 8.
The Effect of Study Abroad on each MGUDS Subscale (1,593) a
MGUDS Subscale
Diverse Contact
Coefficient
.251***
St. Error
.040
t - Statistic
6.22
Comfort with Diversity
.058
.048
1.20
Relativistic Appreciation
-.017
.048
-.34
a
Each equation accounts for the equivalent subscale pretest score, gender, race, pre-college
tested academic preparation, parents' educational attainment, socioeconomic status, positive
attitude toward literacy scale, institutional type, major, first year co-curricular involvement,
first year diverse experiences, first year integrative learning experiences, senior year cocurricular involvement, senior year diverse experiences, senior year integrative learning
experiences, and intent to study abroad.
*** p<.001
91
CHAPTER V
DISCUSSION
Despite widespread agreement among policy makers and study abroad advocates
regarding the educational role of study abroad in developing intercultural competence, no
prior methodologically rigorous, theoretically grounded research could be found to
empirically support this claim. This study endeavored to determine the effect of study
abroad participation on intercultural competence after accounting for the influence of an
equivalent pretest measure, pre-college characteristics, college experiences, and study
abroad intent. In addition, it sought to determine whether any identified effect of studying
abroad was general or differed systematically based on the condition of a third
independent variable. The current study appears to be the first to attempt such a fully
specified analysis utilizing appropriately rigorous methodological analyses.
An additional purpose of this study was to compare and contrast results derived
from covariate adjustment and propensity score matching methods in the hope of
providing additional guidance for future scholars interested in estimating the effect of a
given treatment in a quasi-experimental study where assignment to the treatment is not
random. Although research methodologists have implored educational scholars to utilize
increasingly rigorous analytic methods, little comparative scholarship exists to compare
analytic methods and guide selection of the most appropriate – rather than merely the
most complicated or arcane – means of analysis.
The current study utilized data from the 2006 cohort of the Wabash National
Study of Liberal Arts Education. This large-scale multi-institution research project of the
factors that influence development on the historically articulated educational outcomes of
92
a liberal arts education gathered data at the beginning of the freshman year, the end of the
freshman year, and the end of the senior year from students at 19 institutions. These
colleges and universities differed across a variety of domains including size, selectivity,
geographic location, public/private control, and Carnegie classification.
The Effect of Study Abroad on Intercultural Competence
The results of the current study suggest that, even in the presence of a pretest
score, a variety of pre-college characteristics and college experiences, and either a direct
measure of intent to study abroad or a statistical adjustment for the selection effect, on
average studying abroad significantly affects the positive development of intercultural
competence. Furthermore, this effect appears to be general rather than conditional. This
analysis found no evidence to indicate that the effect of studying abroad varies
systematically by gender, race, SES, institutional type, pre-college tested academic
preparation, pretest score, or college experiences. In recent years, a growing number of
study abroad advocates and scholars have recognized that in an age of increased calls for
accountability in higher education the field of international education can no longer rely
solely on a belief in study abroad’s unique educational value (Vande Berg, 2007). This
study provides clear evidence that studying abroad provides an educational benefit
regardless of the student’s pre-college background, educational aspirations, or college
experiences.
This finding holds potentially powerful implications for educational policy
makers, higher education institutions, and study abroad providers. Although international
education advocates have long asserted the educational benefits of study abroad, the lack
of rigorously obtained evidence supporting this claim has hindered the ability of
93
postsecondary institutions and governmental bodies to enact large-scale financial
legislation to increase study abroad participation. Based on these findings, the federal
government can enact the Senator Paul Simon Study Abroad Act (Lincoln Commission,
2005) backed by empirically derived evidence that the passage of this legislation might
produce its intended objectives. Moreover, in an era of shrinking public resources, these
findings provide some evidence for postsecondary education policy makers at both the
state and federal level who are faced with the unenviable responsibility of choosing
which programs or scholarships to fund from an array of educational funding needs.
Likewise, these findings should provide higher education institutions with even
more incentive to invest in study abroad and to find ways to increase student
participation. Despite extensive efforts at many institutions, diversifying the student
population is often a challenge fraught with complex obstacles that often exceed
institutional resources. The results of this study suggest that if institutions make the
concerted effort to increase study abroad participation, not only can these opportunities
provide the cross-cultural experiences less available to students on their home campuses;
these experiences can be expected to develop intercultural competence, an educational
outcome widely recognized as vital to the success of college graduates in the 21st century.
Yet the findings of this study regarding the nature of study abroad’s effect suggest
that study abroad participation does not influence the multiple domains that encapsulate
intercultural competence equally. In fact, this study suggests that while study abroad
participation increases one’s inclination toward diverse contact, it has no statistically
significant effect on one’s comfort with diverse interactions or one’s relativistic
appreciation of cultural differences. While study abroad professionals and advocates
94
might be encouraged by the overarching effect identified in this study, the underlying
findings regarding the nature of this effect should present reasons for concern. According
to Deardorff’s work toward a consensus definition of intercultural competence (2004,
2006), individuals demonstrating intercultural competence appreciate the ethnorelativity
of cultural differences and demonstrate comfort when engaged in diverse interactions. If
studying abroad only increases the inclination toward diverse contact but does not
contribute to growth along other domains as the present research suggests, then study
abroad may not be contributing to the kind of holistic transformative effect that it claims.
Even though study abroad appears to significantly contribute to growth on an overall
measure of intercultural competence, the underlying findings leave study abroad
susceptible to being little more than educational tourism. Study abroad providers focused
on influencing student growth along each of these domains might consider introducing
more intentionally developed educational experiences prior to departure, throughout the
experience abroad, and upon return to ensure that students engage opportunities for
growth across all three domains.
The findings of this study present a number of inviting opportunities for future
research. Since this study examines study abroad generally and does not parse study
abroad experiences by length of program, type of immersion, or location of study, these
findings merely scratch the surface in seeking to understand the relationship between
study abroad and intercultural competence development. Given the vast differences in
study abroad experiences based on various potential combinations of location, length of
stay, and depth of immersion, it is entirely plausible to suspect that variation along each
of those vectors might uniquely influence the degree and nature of it’s effect.
95
Furthermore, such potential differences might be additionally influenced by individual
student characteristics or even differences in institutional type or culture. Recognizing the
degree to which this series of factors might impact the effect of the experience further
emphasizes the need for extensive additional research to tease out the significant features
that might make specific program designs ideally suited, or specifically unsuitable, for
certain student types. Instead of focusing primarily on overall participation rates,
international education – and postsecondary education generally – might better meet
increasing standards of accountability for learning by research that reveals whether
certain types of study abroad programs are better suited for certain types of students. This
kind of research would allow institutions to align specific students with specific programs
and better meet diverse student needs and goals.
In addition, the findings of the covariate adjustment models in this study
demonstrate that other college experiences contribute to intercultural competence
development. The experience scales included in this study – diverse experiences and
integrative learning experiences – largely assess experiences that are not specific to
discrete programs. Instead, they describe opportunities for interaction, either between
individuals or between learning contexts. By its very nature, study abroad can often
provide many such experiences. College impact scholars might explore ways in which
campus experiences included in this study might accentuate, or minimize, the educational
benefits of study abroad.
The findings of this study appear to confirm the core suppositions of Allport’s
contact hypothesis (1954). Participation in study abroad – an experience that by its very
nature sets the stage for a period of increased contact across a given cultural difference –
96
seems to produce a statistically significant increase in gains on a measure of intercultural
competence not experienced by those who do not study abroad. However, after
considering the disproportionate effects across the three domains of intercultural
competence unveiled by this analysis, these findings may suggest no more than a partial
confirmation of the contact hypothesis. On the other hand, it may be important not to
overemphasize the import of these findings in the context of an educational outcome as
multi-faceted as intercultural competence. As Baxter Magolda’s research on the
development of self-authorship has demonstrated (2001), an individual’s cognitive,
affective, and interpersonal development does not stop upon graduation from college.
The study abroad participants in this study may indeed grow in their comfort with
diversity and relativistic appreciation as a result of their study abroad experiences – just
not within the confines of the college experience.
Estimating Treatment Effects in College Impact Research
This study also holds some important implications for educational researchers
focused on the impact of the college experience on learning. Although experimental
design remains the gold standard in social science research for obtaining unbiased
estimates, the ethical obligations and practicalities of engaging in educational research
often require a quasi-experimental study design. Given this reality, educational scholars
are regularly faced with the necessity of accounting for the non-random assignment of
participants to the treatment of interest. This study used an examination of study abroad,
a college experience in which students choose to participate, as an opportunity to
compare two analytic methods of estimating the effect of a given treatment – covariate
adjustment and propensity score matching.
97
Although recommendations to use propensity score matching methods instead of
traditional covariate adjustment methods in educational research have increased in recent
years, the results of this study suggest that this dichotomy may at the very least be
misleading. Propensity score matching methods are designed to statistically adjust for the
possible existence of an observed selection effect. However, most matching procedures –
including the method used in this study – are limited to accounting for pre-treatment
differences that might, when taken together, approximate a selection effect. Any effort to
isolate the unique effect of a given treatment in a quasi-experimentally designed study of
college experiences will most likely also need to account for potentially confounding
factors that occur simultaneous to the treatment. In such cases, most propensity score
matching methods would be conceptually inappropriate.
This is not to say that a propensity score cannot function as a useful or potentially
important variable. In the present study, when the effect of studying abroad was
measured with the full covariate adjustment model (Table 7), the propensity score and the
direct measure of study abroad intent functioned similarly in impacting the estimated
effect of study abroad participation. Additional comparative analysis using different
combinations of variables to generate the propensity score produced similar results. But
in each case, results were derived through covariate adjustment with the inclusion of a
propensity score as an independent variable.
The more critical issue identified by this study appears to be the study design and
the array of variables gathered by researchers and available for inclusion in the analysis.
The longitudinal design of the Wabash National Study of Liberal Arts Education required
that pretest scores of each outcome measure were gathered at the beginning of the study.
98
The analysis portrayed in Table 6 demonstrates the important role of the pretest in
reducing bias when using propensity score matching methods. Without a pretest, this
study would have likely reported a substantial overestimation of study abroad’s effect.
Likewise, Tables 4 and 7 portray the impact of the pretest in predicting intercultural
competence at the end of the senior year using a covariate adjustment approach. It
appears that, no matter the method of analysis, the absence of a pretest would
substantially undermine the accuracy of the finding.
Likewise, the direct measure of study abroad intent significantly reduced bias in
estimating study abroad’s effect on intercultural competence in both analytic approaches.
As portrayed in Tables 4, 6, and 7, the direct measure of study abroad intent played an
important role in reducing estimation bias using either analytic method. Again, the
longitudinal design of the Wabash National Study allowed researchers to gather this
measure of intent. In the context of recent research examining the complex combination
of factors that predict intent to study abroad (Salisbury, Umbach, Paulsen, & Pascarella,
2008), a cross-sectional study endeavoring to answer the research questions of this study
would have provided an insufficient range of variables to rigorously obtain results.
Ultimately, based on the comparative findings of this study, covariate adjustment
appears to be an entirely sufficient method for estimating the effect of a treatment where
selection into the treatment is non-random. However, to generate a least biased estimate,
the choice of study design and the variables gathered during the course of the data
collection seem more important than the method of analysis.
99
Conclusion
The increasingly compelling argument for internationalization in higher education
and for study abroad in particular is rooted in the undeniable realities of globalization.
College students graduate into an economic, technological, and socio-cultural
environment that is globally intertwined and interdependent; where a financial crisis in
one country can unhinge a another nation’s economy and where the online support for
ordering flowers occurs on the edge of a desert on the other side of the world (Friedman,
2005). At the same time, in an era of alarming increases in the costs of postsecondary
education, the public has demanded greater accountability from higher education
institutions in documenting and demonstrating educational value. Moreover, education
policy makers have emphasized the use of rigorous analytic methods before making the
causal claims on which governmental bodies might base funding decisions. In this
context, international education advocates have recognized that impassioned arguments
and anecdotal vignettes are no longer enough to obtain the public funding required to
increase and diversify participation in study abroad.
This research is the first to document the effect of study abroad on a measure of
intercultural competence using conceptually and statistically appropriate analytic
methods to make causal claims about a non-randomly selected treatment in large-scale
data gathered through a quasi-experimental study. Furthermore, this study provides
important methodological guidance for future educational researchers seeking to make
causal statements about the effects of participation in a particular curricular or cocurricular experience on postsecondary educational outcomes. As postsecondary
education continues to diversify and grow in the face of an increasingly complex and
100
interculturally-dependent post-graduate reality, it is critical for educational researchers to
expand on these substantive and methodological findings to broaden understanding of the
potentially important educational experiences that will prepare all students to succeed in a
twenty-first century globally interconnected world.
101
REFERENCES
Adelman, C. (1994). What employers expect of college graduates: International
knowledge and second language skills. (Department of Education publication
#OR-94-3215). Washington DC: U.S. Government Printing Office.
Allport, G. W. (1954). The nature of prejudice. Cambridge, MA: Addison-Wesley.
American Council on Education. (2002). Beyond September 11: A comprehensive
national policy on international education. Washington DC: American Council
on Education.
American Council on Education. (2008). College-bound students’ interests in study
abroad and other international learning activities. Washington DC: American
Council on Education.
Amir, Y. (1969). Contact hypothesis in ethnic relations. Psychological Bulletin, 71, 319342.
Anderson, P. H., Lawton, L., Rexeisen, R. J., & Hubbard, A. C. (2006). Short-term study
abroad and intercultural sensitivity: A pilot study. International Journal of
Intercultural Relations, 30, 457-469.
Antonio, A. (2001). Diversity and the influence of friendship groups in college. Review of
Higher Education, 25, 63-89.
Armstrong, G. K. (1984). Life after study abroad: A survey of undergraduate academic
and career choices. The Modern Language Journal, 68, 1-6.
Association of Public and Land-Grant Universities. (2004). A call to leadership: The
presidential role in internationalizing the university. Final report from the Task
Force on International Education to the National Association of State Universities
and Land Grant Colleges. Retrieved from
http://www.aplu.org/NetCommunity/Document.Doc?id=340
Association of American Colleges and Universities. (2007). College learning for the new
global century. A report from the National Leadership Council for Liberal
Education & America’s Promise. Retrieved from
http://www.aacu.org/leap/documents/GlobalCentury_final.pdf
Astin, A. (1970a). The methodology of research on college impact (I). Sociology of
Education, 43, 223-254.
Astin, A. (1970b). The methodology of research on college impact (II). Sociology of
Education, 43, 437-450.
102
Astin, A. (1977). Four critical years: Effects of college on beliefs, attitudes, and
knowledge. San Francisco: Jossey-Bass.
Astin, A. (1993). What matters in college? Four critical years revisited. San Francisco:
Jossey-Bass.
Altbach, P. G., & Peterson, P. M. (1998). Internationalize American higher education?
Not exactly. Change, 30(4), 36-39.
Barrutia, R. (1971). Study abroad. The Modern Language Journal, 55, 232-234.
Baxter Magolda, M. (2001). Making their own way: Narratives for transforming higher
education to promote self-development. Sterling, VA: Stylus.
Belenky, M., Clinchy, B., Goldberger, N., & Tarule, J. (1986). Women’s ways of
knowing: The development of self, voice, and mind. New York: Basic Books.
Bhawuk, D. P. S., & Brislin, R. (1992). The measurement of intercultural sensitivity
using the concepts of individualism and collectivism. Intercultural Journal of
Intercultural Relations, 16, 413-436.
Bicknese, G. (1974). Study abroad Part I: A comparative test of attitudes and opinions.
Foreign Language Annals, 7, 325-336.
Bikson, T. K., & Law, S. A. (1994). Global preparedness and human resources: College
and corporate perspectives. Santa Monica, CA: RAND Corporation Institute on
Education and Training.
Bikson, T. K., Treverton, G. T., Moini, J., & Lindstrom, G. (2003). New challenges for
international leadership: Lessons from organizations with global missions. Santa
Monica, CA: RAND Corporation National Security Research Division.
Back, H. T., & Duhon, D. L. (2006). Assessing the impact of business study abroad
programs on cultural awareness and personal development. Journal of Education
for Business, 81(3), 140-144.
Black, J. S., & Mendenhall, M. (1991). The u-curve adjustment hypothesis revisited: A
review and theoretical framework. Journal of International Business Studies, 22,
225-247.
Bloom, B. S., & Webster, H. (1960). The outcomes of college. Review of Educational
Research, 20, 321-333.
Blum, D. E. (2006). Seeking to prepare global citizens, colleges push more students to
study abroad. Chronicle of Higher Education (October 27). Retrieved from
http://chronicle.com/article/Seeking-to-Prepare-Global-C/23025/
103
Booker, R. W. (2001). Differences between applicants and non-applicants relevant to the
decision to apply to study abroad. (Ph.D., University of Missouri-Columbia).
Bowden, J., & Marton, F. (1998). The university of learning: Beyond quality and
competence. London: Kogan Page.
Borcover, A. (2002, September 8). Travel industry still feeling 9/11. Chicago Tribune
(Chicagoland Final ed.), 2. Retrieved from http://infoweb.newsbank.com
Boyer, E. L., & Michael, W. B. (1965). Outcomes of college. Review of Educational
Research, 35, 277-291.
Brand, J. E., & Halaby, C. N. (2006). Regression and matching estimates of the effects of
elite college attendance on educational and career achievement. Social Science
Research, 35, 749-770.
Braskamp, L. A., Braskamp, D. C., & Merrill, K. C. (2008). Global perspective inventory
(GPI): Its purpose, construction, potential uses, and psychometric characteristics.
Chicago: Global Perspective Institute Inc. Retrieved from
https://gpi.central.edu/supportDocs/manual.pdf
Braskamp, L. A., Braskamp, D. C., & Merrill, K. C. (2009). Assessing progress in global
learning and development of students with education abroad experiences.
Frontiers: The Interdisciplinary Journal of Study Abroad, 18, 101-118.
Brecht, R., Davidson, D., & Ginsberg, R. (1993). Predictors of foreign language gain
during study abroad. Washington DC: National Foreign Language Center.
Carlson, J., Burn, B., Useem, J., & Yachimovicz, D. (1990). Study abroad: The
experience of American undergraduates. Westport, CT: Greenwood Press.
Carlson, J. S., & Widaman, K. F. (1988). The effect of study abroad during college on
attitudes toward other cultures. International Journal of Intercultural Relations,
12, 1-17.
Cass, V. (1979). Homosexual identity formation: A theoretical model. Journal of
Homosexuality, 9, 219-235.
Chieffo, L., & Griffiths, L. (2004). Large-scale assessment of student attitudes after a
short-term study abroad program. Frontiers: The Interdisciplinary Journal of
Study Abroad, 10, 165-177.
Chang, M. J. (2002). The impact of an undergraduate diversity course requirement on
students’ racial views and attitudes. Journal of General Education, 51, 21-42.
104
Chickering, A. W. & Reisser, L. (1969). Education and identity (Second Edition). San
Francisco: Jossey-Bass.
Chmela, H. (2005). Foreign detour en route to a college degree. New York Times,
(October 19). Retrieved from
http://travel.nytimes.com/2005/10/19/education/19goucher.html?_r=1&scp=1&sq
=%22lincoln+commission%22+and+%22study+abroad%22&st=nyt
Church, A. T. (1982). Sojourner adjustment. Psychological Bulletin, 91, 540-572.
Consolidated Appropriations Act, 2004, Pub. L. No. 108-199, 118 Stat. 3 (2004).
Cook, S. W., & Selltiz, C. (1955). Some factors which influence the attitudinal outcomes
of personal contact. International Social Science Bulletin, 7, 51-58.
Cornwell, G., & Stoddard, E. (1999). Globalizing knowledge: Connecting international
and intercultural studies. Washington DC: Association of American Colleges and
Universities.
Council on International Educational Exchange [CIEE]. (1991). Black students and
overseas programs: Broadening the base of participation. Papers and speeches
presented at the 43rd CIEE International Conference on Educational Exchange/
New York: Council on International Educational Exchange.
Cress, C., Astin, H., Zimmerman-Oster, K., & Burkhardt, J. (2001). Developmental
outcomes of college students’ involvement in leadership activities. Journal of
College Student Development, 42, 15-27.
Cushner, K., & Karim, A. U. (2004). Study abroad at the university level. In D. Landis, J.
M. Bennett, & M. J. Bennett (Eds.), Handbook of intercultural training, 3rd
Edition (pp. 289-308). Thousand Oaks, CA: Sage Publications.
Cushner, K., & Mahon, J. (2002). Overseas student teaching: Affecting personal,
professional, and global competencies in an age of globalization. Journal of
Studies in International Education, 6, 44-58.
D’Andrea, M., Daniels, J., & Heck, R. (1991). Evaluating the impact of multicultural
counseling training. Journal of Counseling & Development, 70, 143-150.
D’Augelli, A. R. (1994). Identity development and sexual orientation: Toward a model of
lesbian, gay, and bisexual development. In E. Trickett, R. Watts, & D. Birman
(Eds.), Human diversity: Perspectives on people in context (pp. 312-333). San
Francisco: Jossey-Bass.
105
Deardorff, D. K. (2004). The identification and assessment of intercultural competence as
a student outcome of internationalization at institutions of higher education in the
United States. (Ph.D., North Carolina State University).
Deardorff, D. K. (2006). Identification and assessment of intercultural competence as a
student outcome of internationalization. Journal of Studies in International
Education, 10(3), 241-266.
Deardorff, D. K. (2009). Understanding the challenges of assessing global citizenship. In
R. Lewin (Ed.), The handbook of practice and research in study abroad: Higher
education’s quest for global citizenship (pp. 346-364). New York: Routledge.
Deficit Reduction Act of 2005, Pub. L. No. 109-171, 120 Stat. 4 (2005).
Dehejia, R. H., & Wahba, S. (1999). Causal effects in nonexperimental studies:
Reevaluating the evaluation of training programs. Journal of the American
Statistical Association, 94(448), 1053-1062.
Designating 2006 as the “Year of Study Abroad.” S. Res. 208, 109th Cong., 1st. Sess.
(2005). (enacted).
Dessoff, A. (2006). Who’s not going abroad? International Educator, 15(2), 20-27.
Dolby, N. (2004). Encountering an American self: Study abroad and national identity.
Comparative Education Review, 48(2), 150-173.
Douglas, C., & Jones-Rikkers, C. G. (2001). Study abroad programs and American
student worldmindedness: An empirical analysis. Journal of Teaching in
International Business, 13, 55-66.
Du, F. (2007). Self-authorship as a learning outcome of study abroad: Towards a new
approach for examining learning and learning conditions. (Ph.D., University of
Minnesota).
DuFon, M. A., & Churchill, E. (2006). Evolving threads in study abroad research. In M.
A. DuFon & E. Churchill (Eds.), Language learners in study abroad contexts (pp.
1-29). Clevedon, UK: Multilingual Matters, Ltd.
Dunn, T. W., Smith, T. B., & Montoya, J. A. (2006). Multicultural competency
instrumentation: A review and analysis of reliability generalization. Journal of
Counseling and Development, 84, 471-482.
Engberg, M. E. (2007). Educating the workforce for the 21st century: A cross-disciplinary
analysis of the impact of the undergraduate experience on the students’
development of a pluralistic orientation. Research in Higher Education, 48, 283317.
106
Epstein, M. P., Allen, A. S., & Satten, G. A. (2007). A simple and improved correction
for population stratification in case-control studies. American Journal of Human
Genetics, 8, 921-930.
Erikson, E. H. (1950). Childhood and society. New York: Norton.
Erikson, E. H. (1959). Identity and the life cycle. New York: International Universities
Press.
Erikson, E. H. (1968). Identity, youth, and crisis. New York: Norton.
Fantini, A. E. (2000). A central concern: Developing intercultural competence. SIT
Occasional Papers Series, I. 25-42. Retrieved from
http://www.sit.edu/SITOccasionalPapers/sitops01.pdf
Fantini. A. E. (2006). Exploring and assessing intercultural competence. Final Report of a
Research Project conducted by the Federation of The Experiment in International
Living. Retrieved from
http://www.sit.edu/SITOccasionalPapers/feil_research_report.pdf
Farrell, E. F. (2007). Study abroad blossoms into big business. Chronicle of Higher
Education, (September 7). Retrieved from http://chronicle.com/article/StudyAbroad-Blossoms-Into-/19935/
Field, R. (Producer & Director). (2009). Engaging the world: U.S. global competence in
the 21st century. (DVD). (Available from The American Council on Education,
One Dupont Circle NW, Washington DC, 20036). Retrieved from
http://www.usglobalcompetence.org/videos/download.html
Fischer, K. (2007, June 20). All abroad! Overseas study required. The Chronicle of
Higher Education. Retrieved from http://chronicle.com/article/All-AbroadOverseas-Study-/13923/
Flash, S. J. (1999). Study abroad program participation effects on academic progress.
(Ph.D., State University of New York at Buffalo).
Foreign Relations Authorization Act, Fiscal Years 2010 and 2011, H. R. 2410, 111th
Cong. (2009).
Freed, B. F. (1995). Second language acquisition in a study abroad context. Philadelphia,
PA: Benjamins Publishing Company.
Freedman, M. B. (1960). Impact of college. New Dimensions in Higher Education, No. 4.
U.S Department of Health, Education, and Welfare. Washington DC: U.S.
Government Printing Office.
107
Friedman, T. L. (2005). The world is flat: A brief history of the twenty-first century. New
York: Farra, Strauss, & Giroux.
Fuertes, J. N., & Brobst, K. (2002). Clients’ ratings of counselor multicultural
competency. Cultural Diversity and Ethnic Minority Psychology, 8, 214-223.
Fuertes, J. N., & Gelso, C. J. (2000). Hispanic counselors’ race and accent and EuroAmericans’ universal-diverse orientation: A study of initial perceptions. Cultural
Diversity and Ethnic Minority Psychology, 6, 211-219.
Fuertes, J. N., Miville, M. L., Mohr, J. J., Sedlacek, W. E., & Gretchen, D. (2000). Factor
structure and short form of the Miville-Guzman Universality-Diversity Scale.
Measurement and Evaluation in Counseling and Development, 33, 157-169.
Fuertes, J. N., Sedlacek, W. E., Roger, P. R., & Mohr, J. J. (2000). Correlates of
universal-diverse orientation among first-year university students. Journal of
First-Year Experience and Students in Transition, 12, 45-59.
Fulbright, J. W. (1989). The price of empire. New York: Pantheon.
Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction. 7th
Edition. Boston: Allyn and Bacon.
Gardner, E. G. (1968). Selection of American undergraduate students for participation in
study programs overseas. . Occasional Paper on International Educational
Exchange No. 13. Council on International Educational Exchange. Retrieved from
http://www.ciee.org/images/uploaded/pdf/occasional13.pdf
Gilligan, C. (1982). In a different voice: Psychological theory and women’s development.
Cambridge, MA: Harvard University Press.
Gmelch, G. (1997). Crossing cultures: Student travel and personal development.
International Journal of Intercultural Relations, 21, 475-490.
Golay, P. (2006). The effects of study abroad on the development of global-mindedness
among students enrolled in international programs at Florida State University.
(Ed.D., The Florida State University).
Goldstein, S. B., & Kim, R. I. (2006). Predictors of US college students’ participation in
study abroad programs: A longitudinal study. International Journal of
Intercultural Relations, 30, 507-521.
Gonyea, R. M. (2008, November). The impact of study abroad on senior year
engagement. Paper presented at the Annual Meeting of the Association for the
Study of Higher Education, Jacksonville, FL.
108
Grant, G., & Associates. (1979). On competence: A critical analysis of competence-based
reforms in higher education. San Francisco: Jossey-Bass.
Gray, M., Ondaatjie, E., Fricker, R., & Geschwind, S. (2000). Assessing service learning:
Results from a survey of “Learn and Serve American, Higher Education.”
Change, 32, 30-39.
Green, M. (2005). Internationalization in U.S. higher education: The student perspective.
Washington DC: American Council on Education.
Green, M., Luu, D. T., & Burris, B. (2008). Mapping internationalization on U.S.
campuses: 2008 edition. Washington DC: American Council on Education.
Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau,
R. (2004). Survey methodology. Hoboken, NJ: Wiley-Interscience.
Gullahorn, J. E., & Gullahorn, J. T. (1966). American students abroad: Professional
versus personal development. Annals of the American Academy of Political and
Social Science, 368, 43-59.
Gurin, P., Peng, T., Lopez, G., & Nagda, B. (1999). Context, identity, and intergroup
relations. In D. Prentice & D Miller (Eds.), Cultural divides: Understanding and
overcoming group conflict (pp. 133-170). New York: Russell Sage Foundation.
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural
sensitivity: The intercultural development inventory. International Journal of
Intercultural Relations, 27, 421-443.
Helms, J. (1990). Toward a model of White racial identity development. In J. Helms
(Ed.), Black and White racial identity Theory, research, and practice (pp. 49-66).
Westport, CT: Greenwood Press.
Hewstone, M. & Brown, R. (1986). Contact is not enough: An intergroup perspective on
the ‘contact hypothesis.’ In M. Hewstone & R. Brown (Eds.), Contact and
conflict in intergroup encounters (pp.1-44). Cambridge, MA: Addison-Wesley.
Hill, B., & Green, M. (2008). A guide to internationalization for chief academic officers.
Washington DC: American Council on Education.
Hoffa, W. W. (2007). A history of U.S. study abroad: Beginnings to 1965. Carlisle, PA:
The Forum on Education Abroad.
Hoffa, W. W., & DePaul, S. C. (2010). A history of U.S. study abroad: 1965-present.
Carlisle, PA: The Forum on Education Abroad.
109
Hofman, J. E., & Zak, I. (1969). Interpersonal contact and attitude change in a crosscultural situation. The Journal of Social Psychology, 78, 165-171.
Hurtado, S., Engberg, M. E., & Ponjuan, L. (2003). The impact of diversity experience on
student’s learning for a diverse democracy. Paper presented at Annual Meeting of
the Association of the Study of Higher Education, Portland, OR.
Ingraham, E. C. (2003). Documentation and assessment of the impact of study abroad. In
G. T. M. Hult & E. C. Lashbrooke, Jr. (Eds.), Study abroad: Perspectives and
experiences from business schools (pp. 1-22). Oxford, UK: Elsevier Science Ltd.
Ingraham, E. C., & Peterson, D. L. (2004). Assessing the impact of study abroad on
student learning at Michigan State University. Frontiers: The Interdisciplinary
Journal of Study Abroad, 10, 83-100.
Institute of International Education [IIE]. (2009). Open doors report 2009. Retrieved
from http://opendoors.iienetwork.org/
Jacob, P. E. (1957). Changing values in college: An exploratory study of the impact of
college teaching. New York: Harper.
Jeffrey, H. P. (1987). Legislative origins of the Fulbright program. Annals of the
American Academy of Political and Social Science, 491(May), 36-47.
Johnston, Jr., J. S., & Edelstein, R. J. (1993). Beyond borders: Profiles in international
education. Washington DC: Association of American Colleges and Universities.
Josselson, R. (1987). Finding herself: Pathways to identity development in women. San
Francisco: Jossey-Bass.
Jurgens, J. C., McAuliffe, G. (2004). Short-term study-abroad experience in Ireland: An
exercise in cross-cultural counseling. International Journal for the Advancement
of Counseling, 26, 147-161.
Kalunian, J. (1997, Fall). Correlations between global-mindedness and study abroad.
International Education Forum, 17, 131-143.
Kasravi, J. (2009). Factors influencing the decision to study abroad for students of color:
Moving beyond the barriers. (Ph.D., University of Minnesota).
Kauffmann, N. L. (1983). The impact of study abroad on personality change. (Ph.D.,
Indiana University).
Kegan, R. (1982). The evolving self: Problems and process in human development.
Cambridge, MA: Harvard University Press.
110
Kegan, R. (1994). In over our heads: The mental demands of modern life. Cambridge,
MA: Harvard University Press.
Kelley, C., & Meyers, J. (1995). Cross-Cultural Adaptability Inventory. Minneapolis,
MN: National Computer Systems.
King, P. M., & Baxter Magolda, M. (1996). A developmental perspective on learning.
Journal of College Student Development, 37, 163-173.
King, P.M., & Baxter Magolda, M. (2005). A developmental model of intercultural
maturity. Journal of College Student Development, 46, 571-592.
King, P. M., & Kitchener, K. (1994). Developing reflective judgment: Understanding and
promoting intellectual growth and critical thinking in adolescents and adults. San
Francisco: Jossey-Bass.
King, L. J., & Young, J. A. (1994). Study abroad: Education for the 21st century. Die
Unterrichtspaxis / Teaching German, 27, 77-87.
Kitsantas, A., & Meyers, J. (2004). The role of college students’ goals on the
development of cross-cultural skills and global understanding. College Student
Journal, 38, 441-452.
Knight, J. (2004). Internationalization remodeled: Definitions, approaches, and rationales.
Journal of Studies in International Education, 8, 5-31.
Koeser, J., & Olebe, M. (1988). The behavioral assessment scale for intercultural
communication effectiveness. International Journal of Intercultural Relations, 12,
233-246.
Koester, J. (1985). A profile of the U.S. student abroad. New York: Council on
International Educational Exchange.
Kohlberg, L. (1971). Stages of moral development as a basis of moral education. In C.
Beck, B. Crittenden, E. Sullivan (Eds.), Moral education: Interdisciplinary
approaches. Toronto: University of Toronto Press.
Kuh, G. D. (2008). High impact educational practices: What they are, who has access to
them, and why they matter. Washington DC: Association of American Colleges
and Universities.
LaFromboise, T. D., Coleman, H. L., & Hernandez, A. (1991). Development and factor
structure of the Cross-Cultural Counseling Inventory-Revised. Professional
Psychology: Research and Practice, 22, 380-388.
111
LaLonde, R. (1986). Evaluating the econometric evaluations of training programs with
experimental data. American Economic Review, 76, 604-620.
Lambert, R. D. (1989). International studies and the undergraduate. Washington, DC:
American Council on Education.
Lau A., & Finney, S. (2006). Measuring diversity orientation: An examination of the
Miville-Guzman universality-diversity scale short form. Paper presented at the
annual meeting of the Association of Psychological Sciences, New York.
Leuven, E., & Sianesi, B. (2003). PSMATCH2: Stata module to perform full
Mahalanobis and propensity score matching, common support graphing and
covariate imbalance testing. Retrieved from
http://ideas.repec.org/c/boc/bocode/s432001.html. Version 1.2.3.
Lewin, R. (ed.) (2009). The handbook of practice and research in study abroad: Higher
education and the quest for global citizenship. New York: Routledge.
Lincoln Commission. (2005). Global competence and national needs: One million
Americans studying abroad. Final Report from the Commission on the Abraham
Lincoln Fellowship Program, Washington, DC.
Lindsey, E. W. (2005). Study abroad and values development in social work students.
Journal of Social Work Education, 41, 229-249.
Lunceford, J. K., & Davidian, M. (2004). Stratification and weighting via the propensity
score in estimation of causal treatment effects: A comparative study. Statistics in
Medicine, 23, 2937-2960.
Magnan, S. S., & Back, M. (2007). Social interaction and linguistic gain during study
abroad. Foreign Language Annals, 40, 43-61.
McClendon, M. J. (1994). Multiple regression and causal analysis. Long Grove, IL:
Waveland Press, Inc.
McEvoy, T. L. (1967). Psychological growth, cosmopolitanism, and selection of students
for cross-cultural educational programs. Occasional Paper on International
Educational Exchange No. 7. Council on International Educational Exchange.
Retrieved from http://www.ciee.org/images/uploaded/pdf/occasional07.pdf
McKeown, J. S. (2003). The impact of September 11 on study abroad student interest and
concern: An exploratory study. International Education, 32(2), 85-95.
McKeown, J. S. (2009). The first time effect: The impact of study abroad on college
student intellectual development. Albany, NY: State University of New York
Press.
112
Metzger, C. (2006). Study abroad programming: A 21st century retention strategy?
College Student Affairs Journal, 25, 164-175.
Milem, J. (1994). College, students, and racial understanding. Thought and Action, 9, 5194.
Milstein, T. (2005). Transformation abroad: Sojourning and the perceived enhancement
of self-efficacy. International Journal of Intercultural Relations, 29, 217-239.
Mistretta, W. (2008). Life-enhancing: An exploration of the long-term effects of study
abroad. (Ph.D., State University of New York at Buffalo).
Miville, M. L., Gelso, C. J., Pannu, R., Liu, W., Touradji, P., Hollowa, P., & Fuertes, J.
N. (1999). Appreciating similarities and valuing differences: The Miville-Guzman
Universality Diversity Scale. Journal of Counseling Psychology, 46, 291-307.
Myers-Lipton, S. (1996). Effect of service-learning on college students’ attitudes toward
international understanding. Journal of College Student Development, 37, 659668.
NAFSA: Association of International Educators. (2003). Securing America’s future:
Global education for a global age. Report of the Strategic Task Force on
Education Abroad. Retrieved from
http://www.nafsa.org/public_policy.sec/public_policy_document/study_abroad_1/
securing_america_s_future/
NAFSA: Association of International Educators. (2008). Strengthening study abroad:
Recommendations for effective institutional management for presidents, senior
administrators, and study abroad professionals. Report of NAFSA’s Task Force of
Institutional Management of Study Abroad. Retrieved from
http://www.nafsa.org/_/File/_/final_imsa_taskforce.pdf
Nash, D. (1976). The personal consequences of a year of study abroad. The Journal of
Higher Education, 47, 191-203.
National Center for Education Statistics. (2008a). Table A-40-1. Number and percentage
distribution of U.S. study abroad students, by host region: Selected academic
years, 1987-88 through 2007-08. Retrieved from
http://nces.ed.gov/programs/coe/2010/section5/table-ssa-1.asp
National Center for Education Statistics. (2008b). Table 189. Total fall enrollment in
degree-granting institutions, by attendance status, sex of student, and control of
institution: Selected years, 1947 through 2008. Retrieved from
http://nces.ed.gov/programs/digest/d09/tables/dt09_189.asp
113
National Commission on Terrorist Attacks upon the United States. (2003). The 9/11
commission report. Washington DC: U.S. Government Printing Office. Retrieved
from http://govinfo.library.unt.edu/911/report/911Report.pdf
National Task Force on Undergraduate Education Abroad. (1990). A national mandate
for education abroad: Getting on with the task. New York: Council on
International Educational Exchange.
Norris, M. E., & Gillespie, J. (2009). How study abroad shapes global careers: Evidences
from the United States. Journal of Studies in International Education, 13, 382397.
Olson, C., Green, M., & Hill, B. (2005). Building a strategic framework for
comprehensive internationalization. Washington DC: American Council on
Education.
Olson, C., Green, M., & Hill, B. (2006). A handbook for advancing comprehensive
internationalization: What institutions can do and what students should learn.
Washington DC: American Council on Education.
Olson, C. L., & Kroeger, K. R. (2001). Global competency and intercultural sensitivity.
Journal of Studies in International Education, 5(2), 116-137.
Padgett, R. D., Salisbury, M. H., An, B. P., & Pascarella, E. T. (in press). Required,
practical, or unnecessary? An examination and demonstration of propensity score
matching using longitudinal secondary data. Assessment Issue of New Directions
in Institutional Research. San Francisco: Jossey-Bass.
Pascarella, E. T. (1985). College environmental influences on learning and cognitive
development: A critical review and synthesis. In J. Smart (Ed.). Higher
education: Handbook of theory and research (Vol.1, pp. 1-64). New York:
Agathon.
Pascarella, E. T. (2006). How college affects students: Ten directions for future research.
Journal of College Student Development, 47(5), 508-520.
Pascarella, E. T., Edison, M., Nora, A., Hagedorn, L., & Terenzini, P. T. (1996).
Influences on students’ openness to diversity and challenge in the first year of
college. Journal of Higher Education, 67, 174-195.
Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students. San Francisco:
Jossey-Bass.
Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students (Vol. 2): A third
decade of research. San Francisco: Jossey-Bass.
114
Patterson, P. K. (2006). Effect of study abroad on intercultural sensitivity. (Ph.D.,
University of Missouri-Columbia).
Pedersen, P. J. (2009). Teaching towards an ethnorelative worldview through psychology
study abroad. Intercultural Education, 20, 73-86.
Peikes, D. N., Moreno, L., & Orzol, S. M. (2008). Propensity score matching: A note of
caution for evaluators of social programs. The American Statistician, 62, 222-231.
Perry, W. (1970). Forms of intellectual and ethical development in the college years: A
scheme. San Francisco: Jossey-Bass.
Pettigrew, T. F., & Tropp, L. R. (2006). A meta-analytic test of intergroup contact theory.
Journal of Personality and Social Psychology, 90, 751-783.
Phinney, J. (1990). Ethnic identity in adolescents and adults: Review of research.
Psychological Bulletin, 108, 499-514.
Piaget, J. (1928). Judgment and reasoning in the child. New York: Harcourt, Brace, and
Company.
Piaget, J. (1952). The origins of intelligence in children. New York: International
Universities Press.
Pike, G. (2002). The differential effect of on- and off- campus living arrangements on
students’ openness to diversity. NASPA Journal, 39, 283-299.
Ponterotto, J. G., & Alexander, C. M. (1996). Assessing the multicultural competence of
counselors and clinicians. In L. A., Suzuki, P. J. Meller, & J. G. Ponterotto (Eds.),
Handbook of multicultural assessment (pp. 651-672). Thousand Oaks, CA: Sage.
Ponterotto, J. G., Gretchen, D., Utsey, S. O., Rieger, B. P., & Austin, R. (2002). A
revision of the Multicultural Counseling Awareness Scale. Journal of
Multicultural Counseling and Development, 30, 153-180.
Pope, R. L., & Mueller, J. A. (2000). Development and initial validation of Multicultural
Competence in Student Affairs-Preliminary 2 Scale. Journal of College Student
Development, 41, 599-608.
Posey, Jr., J. T. (2003). Study abroad: Educational and employment outcomes of
participants and non participants. (Ph.D., Florida State University).
President’s Commission on Foreign Languages and International Study. (1980). Strength
through wisdom: A critique of U.S. capability. Modern Language Journal, 64(1),
9-57.
115
Redden, E. (2008). Women abroad and men at home. Inside Higher Ed. Retrieved from
www.insidehighered.com/news/2008/12/4/genderabroad
Reynolds, C. L., & DesJardins, S. L. (2009). The use of matching methods in higher
education research: Answering whether attendance at a 2-year institution results
in differences in educational attainment. In J. C. Smart (Ed.), Higher education:
Handbook of theory and research, (pp. 47-104). New York: Springer Science +
Business Media.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in
observational studies for causal effects. Biometrika, 70, 41-55.
Ruben, B. D., & Kealey, D. (1979). Behavioral assessment of communication
competency and the prediction of cross-cultural adaptation. International Journal
of Intercultural Relations, 3, 15-48.
Rubin, D. B. (2008). For objective causal inference, design trumps analysis. Annals of
Applied Statistics, 2, 808-840.
Rubin, D. B. (2001). Using propensity scores to help design observational studies:
Application to the tobacco litigation. Health Services and Outcomes Research
Methodology, 2, 169-188.
Rubin, D. L., & Sutton, R. C. (2001). Study abroad learning outcomes assessment. Paper
presented at the American Association for Higher Education Assessment Forum,
Denver, Colorado, June, 25.
Rust, V., Dhanatya, C., Furuto, L. H. L., & Kheiltash, O. (2007). Student involvement as
predictive of college freshmen plans to study abroad. Frontiers: The
Interdisciplinary Journal of Study Abroad, 15(fall), 1-16.
Ryan, M., & Twibell, R. (2000). Concerns, values, stress, coping, health, and educational
outcomes of college students who studied abroad. International Journal of
Intercultural Relations, 24, 409-435.
Salisbury, M. H., & Goodman, K. (2009). Educational practices that foster intercultural
competence. Diversity and Democracy, 12(2), 12-13.
Salisbury, M. H., Paulsen, M. B., & Pascarella, E. T. (2010). To see the world or stay at
home: Applying an integrated student choice model to explore the gender gap in
the intent to study abroad. Research in Higher Education, 51. Retrieved from
http://www.springerlink.com/content/713786l56010450n/
116
Salisbury, M. H., Paulsen, M. B., & Pascarella, E. T. (in press). Why do all the study
abroad students look alike? Using an integrated student choice model to explore
differences in the development of white and minority students’ intent to study
abroad. Research in Higher Education.
Salisbury, M. H., Umbach, P. D., Paulsen, M. B., & Pascarella, E. T. (2009). Going
global: Understanding the choice process of the intent to study abroad. Research
in Higher Education, 50, 119-143.
Savicki, V. (ed.) (2008). Developing intercultural competence and transformation:
Theory, research, and application in international education. Sterling, VA:
Stylus.
Sax, L. & Astin, A. (1997). The benefits of service: Evidence from undergraduates.
Educational Record, 78, 25-32.
Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W. H., & Shavelson, R. J. (2007).
Estimating causal effects using experimental and observational designs.
Washington, DC: American Educational Research Association.
Sell, D. K. (1983). Research on attitude change in U.S. students who participate in
foreign study experiences: Past findings and suggestions for future research.
International Journal of Intercultural Relations, 7, 131-147.
Senator Paul Simon Study Abroad Foundation Act of 2007, H. R. 1469/S. 991, 110th
Cong. (2007).
Senator Paul Simon Study Abroad Foundation Act of 2009, S. 473, 110th Cong. (2009).
Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments
yield accurate answers? A randomized experiment comparing random and
nonrandom assignments. Journal of the American Statistical Association, 103,
1334-1356.
Shaheen, S. (2004). The effect of pre-departure preparation on student intercultural
development during study abroad programs. (PhD., The Ohio State University).
Siaya, L., & Hayward, F. M. (2003). Mapping internationalization on U.S. campuses.
Washington DC: American Council on Education.
Sinicrope, C., Norris, J., & Watanabe, Y. (2003). Understanding and assessing
intercultural competence: A summary of theory, research, and practice (Technical
report for the Foreign Language Program Evaluation Project). Second Language
Studies, 26, 1-58. Retrieved from
http://www.hawaii.edu/sls/uhwpesl/26(1)/Norris.pdf
117
Sodowsky, G. R., Taffe, R. C., Gutkin, T. B., & Wise, S. L. (1994). Development of the
Multicultural Counseling Inventory: A self-report measure of multicultural
competencies. Journal of Counseling Psychology, 41, 137-148.
Spady, W. G. (1977). Competency based education: A bandwagon in search of a
definition. Educational Researcher, 6, 9-14.
Spiering, K., & Erickson, S. (2006). Study abroad as innovation: Applying the diffusion
model to international education. International Education Journal, 7, 314-322.
Spitzberg, B. H., & Changnon, G. (2009). Conceptualizing intercultural competence. In
D. K. Deardorff (Ed.), The SAGE handbook of intercultural competence (p. 2-52).
Thousand Oaks, CA: SAGE Publications, Inc.
Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (in press). The importance of
covariate selection in controlling for selection bias in observational studies.
Psychological Methods.
Strang, R. (1937). Behavior and background of student in college and secondary school.
New York: Harper.
Stroud, A. H. (2010). Who plans (not) to study abroad? An examination of U.S. student
intent. Journal of Studies in International Education, 20(10), 1-18.
Sutton, R. C., & Rubin, D. L. (2004). The GLOSSARI Project: Initial findings from a
system-wide research initiative on study abroad learning outcomes. Frontiers:
The Interdisciplinary Journal of Study Abroad, 10, 65-82
Sutton, R. C., & Rubin, D. L. (2010). Documenting the academic impact of study abroad.
Final report of the GLOSSARI project. Paper presented at the NAFSA Annual
Conference. Kansas City, MO.
Talburt, S., & Stewart, M. A. (1999). What’s the subject of study abroad?: Race, gender,
and “living culture.” The Modern Language Journal, 83, 163-175.
Thomas, S. L., & McMahon, M. E. (1998). Americans abroad: Student characteristics,
pre-departure qualifications and performance. International Journal of
Educational Management, 12(2), 57-64.
Titus, M. A. (2007). Detecting selection bias, using propensity score matching, and
estimating treatment effects: An application to the private returns to a master’s
degree. Research in Higher Education, 48, 487-521.
Vande Berg, M. (2003). The case for assessing educational outcomes in study abroad. In
G. T. M. Hult & E. C. Lashbrooke, Jr. (Eds.), Study abroad: Perspectives and
experiences from business schools (pp. 23-36). Oxford, UK: Elsevier Science Ltd.
118
Vande Berg, M. (2007). Intervening in the learning of U.S. students abroad. Journal of
Studies in International Education, 11(3/4), 392-399.
Vande Berg, M., Connor-Linton, J., & Paige, M. R. (2009). The Georgetown Consortium
Project: Interventions for student learning abroad. Frontiers: The
Interdisciplinary Journal of Study Abroad, 18, 1-75.
Wallace, D. H. (1999). Academic study abroad: The long-term impact on alumni careers,
volunteer activities, world, and personal perspectives. (Ph.D., Claremont Graduate
University).
Wallace, J. A. (1965). Selection of participants. Occasional Paper on International
Educational Exchange No. 3. Council on International Educational Exchange.
Retrieved from http://www.ciee.org/images/uploaded/pdf/occasional03.pdf
Whitt, E., Edison, M., Pascarella, E. T., Terenzini, P. T., & Nora, A. (2001). Influences
on students’ openness to diversity and challenge in the second and third years of
college. Journal of Higher Education, 72, 172-204.
Wilkinson, S. (1998). Study abroad from the participants’ perspective: A challenge to
common beliefs. Foreign Language Annals, 31, 23-49.
Wilkinson, S. (2000). Emerging questions about study abroad. ADFL Bulletin, 32, 36-41.
Williams, T. R. (2005). Exploring the impact of study abroad on students’ intercultural
communications skills: Adaptability and sensitivity. Journal of Studies in
International Education, 9, 356-371.
Woolf, M. (2007). Impossible things before breakfast, Myths in education abroad.
Journal of Studies in International Education, 11(3/4), 496-509.
Young, D. Y. (2003). Participation in a study-abroad program and persistence at a liberal
arts university. (Ph.D., University of North Texas).
Zhai, L., & Scheer, S. D. (2002). Influence of international study abroad programs
on agricultural students. Journal of International Agricultural and
Extension Education, 9(3), 23-29.