FLORIDA INTERNATIONAL UNIVERSITY
Miami, Florida
PROPOSAL FOR DISSERTATION
DOCTOR OF EDUCATION IN EDUCATIONAL ADMINISTRATION AND
SUPERVISION
COLLEGE OF EDUCATION
DEPARTMENT OF LEADERSHIP AND PROFESSIONAL STUDIES
Stephanie Paul Doscher
2011
I propose to the Major Professor and to the Committee Members a study of the following
topic to be conducted in partial fulfillment of the requirements for the degree of Doctor of
Education in Educational Administration and Supervision: THE DEVELOPMENT OF
RUBRICS TO MEASURE UNDERGRADUATE STUDENTS’ GLOBAL
AWARENESS AND GLOBAL PERSPECTIVE: A VALIDITY STUDY
TABLE OF CONTENTS
CHAPTER
I.
INTRODUCTION ...................................................................................................1
Background of the Study .........................................................................................1
Research Problem ....................................................................................................6
Setting for the Study ................................................................................................9
Purpose of the Study ..............................................................................................14
Research Questions ................................................................................................14
Theoretical Framework ..........................................................................................14
Significance of the Study .......................................................................................16
Delimitations ..........................................................................................................18
Assumptions...........................................................................................................18
Definition of Terms................................................................................................19
Overview of Succeeding Chapters .........................................................................22
II.
REVIEW OF LITERATURE ................................................................................23
Global Awareness and Global Perspective: Defining Outcomes of
Global Education .......................................................................................23
Two Approaches to the Development of International Education ........................24
Education for International Understanding Approach ...............................25
Education for World Citizenship Approach ..............................................28
Global Awareness ..................................................................................................32
Global Perspective .................................................................................................35
Global Learning as an Educational Process ...........................................................39
Assessing Global Awareness and Global Perspective ...........................................43
Global Understanding Survey ....................................................................44
Worldmindedness Survey ..........................................................................48
Global-mindedness Scale ...........................................................................50
Intercultural Development Inventory .........................................................52
Global Perspective Inventory.....................................................................54
Summary ................................................................................................................58
III.
METHODS ............................................................................................................60
Research Questions and Hypotheses .....................................................................60
Research Design.....................................................................................................63
Variables ................................................................................................................64
Population ..............................................................................................................66
Sample....................................................................................................................67
Instrumentation ......................................................................................................67
Data Collection Procedures....................................................................................69
Data Analysis Procedures ......................................................................................70
Limitations .............................................................................................................71
ii
REFERENCES ..................................................................................................................72
APPENDICES ...................................................................................................................85
iii
LIST OF ACRONYMS
AAC&U
American Association of Colleges and Universities
AERA
American Educational Research Association
ANCOVA
Analysis of Covariance
APA
American Psychological Association
CMI
Case Method of Instruction
CFT
Cognitive Flexibility Theory
DMIS
Developmental Model of Intercultural Sensitivity
DV
Dependent Variable
GPI
Global Perspective Inventory
IDI
Intercultural Development Inventory
ISD
Ill-structured Domain
IV
Independent Variable
NCME
National Council on Measurement in Education
SLO
Student Learning Outcome
UNDP
United Nations Development Programme
UNESCO
United Nations Educational, Scientific and Cultural Organization
UNICEF
United Nations International Children’s Emergency Fund
iv
CHAPTER I
Introduction
This proposal outlines a study to examine the development of rubrics to measure
undergraduate students’ global awareness and global perspective. Specifically, it presents
an approach to estimating the validity and reliability of scores yielded from these rubrics.
Chapter 1 provides the background of the proposed study, the research problem and
purpose, research questions, the theoretical framework, and the study’s significance,
assumptions, and delimitations. The chapter concludes with definitions of terms and an
overview of succeeding chapters.
Background of the Study
Today’s young adults are citizens in a diverse and interconnected world. The
issues and problems they face—whether national, international, or global in scope—are
complex, ill-structured, and shaped by shifting dynamics. In order to think critically and
make responsible decisions concerning these challenges, undergraduates must understand
how local, global, international, and intercultural issues, trends, and systems are
interrelated and be able to analyze problems from multiple perspectives (Adams &
Carfagna, 2006; American Council on International Intercultural Education Conference,
1996; Hunter, White, & Godbey, 2006). Knowledge of interrelationships among issues,
trends, and systems across the globe has been called global awareness (Lemke, 2002).
The ability to examine the world via diverse cultural, intellectual, and spiritual points of
view has been called global perspective (Braskamp et al., 2009). Increased global
awareness and global perspective are often cited as among the intended student learning
outcomes (SLOs) of global education (Hovland, 2009; Loveland, 2010; Skelton, 2010).
1
The Association for Curriculum Supervision and Development (Tye, 1990)
defines global education in terms of its learning outcomes, knowledge of interrelatedness,
and the ability to view the world through multiple perspectives:
Global education involves learning about those problems and issues which cut
across national boundaries and about the interconnectedness of systems—cultural,
ecological, economic, political, and technological. Global education also involves
learning to understand and appreciate our neighbors who have different cultural
backgrounds from ours; to see the world through the eyes and minds of others;
and to realize that other peoples of the world need and want much the same
things. (p. 5)
Global awareness and global perspective have also been identified as fundamental SLOs
for global citizenship (Florida International University, 2010). Increasingly, students
view themselves as citizens of not only local and national communities, but also the
global community (Education Development Center, 2006; Our World Alliance, 2006).
Students’ sense of affiliation with interconnected civic spheres has significant
implications for education in the 21st century. To address this broader understanding of
affiliation, colleges and universities across the country are initiating a variety of global
education initiatives—many of these involve global learning (Grudzinski-Hall, 2007).
Global learning is the process by which students are prepared to fulfill their civic
responsibilities in a diverse and interconnected world (Hovland, 2006). Global learning is
also the term used to describe the specific curricular, pedagogical, and assessment
strategies that enable students to develop SLOs associated with global education and
global citizenship, such as global awareness and global perspective.
2
There is growing consensus that global learning should be part of the educational
mission of all American colleges and universities (American Council on International
Intercultural Education Conference, 1996; International Association of Universities,
2003; Hovland, 2006). Global learning is an educational process that was developed in
response to the ways in which globalization has transformed everyday life. Many of these
changes were driven by an unprecedented acceleration in the pace, volume, and scale of
information sharing during the 20th century (Castells, 1999; Thompson, 2003). Thick
information networks have opened individuals’ eyes to diverse problems and perspectives
and enabled them to develop an understanding of the interconnectivity of people, the
institutions they create, and the environment in which they live. Globalization is often
described in terms of the Information Revolution’s macro-level impact on economies,
markets, supply chains, human resource flows, consumption patterns, and cultural
transfer (Chase-Dunn, 1999; Cole, 2003; Keohane & Nye, 2000; Tomlinson, 1999), but
globalization has also affected the way individuals view their relationship with other
individuals and societies (Drucker, 1999). Increasingly, American universities are
initiating global learning initiatives in order to prepare students for success as global
citizens.
The idea of global citizenship was born in classical Greece, but it has taken on
st
new relevance in the era of 21 century globalization (Appiah, 2006). Global citizenship
is a distinctly different notion than that of national citizenship. Whereas national
citizenship is defined as a set of rights and responsibilities granted by the nation-state,
global citizenship is a disposition that guides individuals to take on responsibilities within
interconnected local, global, intercultural, and international contexts (Steenburgen, 1994).
3
Implicit in this concept is the idea that people are members of a larger community than
that of the nation-state. Whereas national citizenship is granted by virtue of place of birth,
parentage, or naturalization, global citizenship is an outlook developed through
education. Nussbaum (2004) has asserted that these perceptions of global citizenship
dictate the need for global learning, even in a time of cost-cutting and increasing preprofessional specialization:
Cultivating our humanity in a complex interlocking world involves understanding
the ways in which common needs and aims are differently realized in different
circumstances. This requires a great deal of knowledge that American college
students rarely got in previous eras….We must become more curious and more
humble about our role in the world, and we will do this only if undergraduate
education is reformed in this direction. (p. 45)
Nussbaum (2004) has also argued that global citizens cannot function on the basis
of factual knowledge alone. Global citizens need to be familiar with prevailing world
conditions, but they must also understand how issues, trends, and systems are
interrelated. Adams and Carfagna echoed this position in Coming of Age in a Globalized
World: The Next Generation (2006), wherein they argued that global citizens must
understand contemporary interconnected local and global dynamics. Likening knowledge
of interrelatedness to a connect-the-dot puzzle, the authors warned of the danger of
focusing on the isolated dots, rather than the connections between and among them:
As a society, we are flooded with information. It can be overwhelming, but it is
critically important to find meaning…Without understanding relationships and
connections, we are forced only to react to isolated events. We can never make
4
decisions or act in a way that anticipates or takes advantage of trends or events.
We must each therefore develop the ability to connect the dots. (p. 2)
Global citizens also need to be able to view the world from multiple perspectives.
One’s perspective consists of ordinarily unexamined assumptions, evaluations,
explanations, and conceptions of time, space, and causality (Hanvey, 1975). A person
needs to develop a sense of his or her own perspective and recognize that it is shaped by
multiple influences (e.g., culture, religion, gender, socioeconomic status, education), in
order to develop an understanding of others’ perspectives and discern their distinctive and
common qualities. Perspectives determine the ways people create meaning from
experience (Tomlinson, 1999). The ability to understand issues from multiple
perspectives is critically important to problem solving in a diverse and interconnected
world.
As a result of their global awareness and global perspective, global citizens
perceive themselves as shaping the conditions of the world rather than merely navigating
them. National citizenship carries with it rights and responsibilities, but global citizens
are driven to define rights and take on responsibilities in multiple contexts.
Understanding that they are members of interrelated communities and that others’ wellbeing impacts their own, global citizens accept shared responsibility for solving problems
(Hanvey, 1975). What’s more, global citizens are willing to take action to solve these
problems (Falk, 1994). In essence, global citizens view themselves as change agents.
Their actions are grounded in their understanding of the interrelatedness of world
conditions and their ability to approach issues from multiple perspectives.
5
Global learning prepares students to manage the complexity, diversity, and
change that define contemporary life (College Learning for a New Global Century,
2007). In the past, the knowledge supply remained relatively constant. Knowledge and
skills formed through a traditional liberal arts education were adequate over the long term
(Brunold, 2005). However, a traditional liberal education, once deemed global because it
provided a breadth of exposure to a variety of disciplines, no longer suffices. Institutions
of higher learning across the United States are adopting global learning initiatives in
order to prepare students to meet the challenges and opportunities of citizenship in the
21st century (Grudzinski-Hall, 2007). These initiatives may involve different components,
(e.g., general education reforms, certificate programs, foreign language requirements,
study abroad programs, and service learning programs), but they share a common
purpose: to enable students to acquire the knowledge, skills, and attitudes necessary to
succeed in an increasingly globalized world.
Research Problem
Global awareness and global perspective are widely acknowledged to be
important SLOs of higher education. A variety of American institutions of higher
learning have adopted these outcomes (Grudzinski-Hall, 2007). Institutions have
developed different kinds of global learning initiatives to support student achievement of
these outcomes, but when it comes to learning outcome assessment, the options are
limited.
In a review of global learning assessment instruments, the Global Understanding
Survey (Barrows, Ager, Bennett, Braun, Clark, Harris, & Klein, 1981 ), the
Worldmindedness Scale (Sampson & Smith, 1957), the Global-mindedness Scale (Hett,
6
1993), the Intercultural Development Inventory (Hammer, Bennett, & Wiseman, 2006),
and the Global Perspective Inventory (Braskamp, Braskamp, & Merrill, 2009) were each
found to have been used to assess students’ global awareness and perspective. Although
these instruments attempt to capture students’ knowledge of the world and how students
develop openness to and appreciation of multiple perspectives, they do not assess global
awareness and perspective as defined in this study. Moreover, all of the above
instruments address the knowledge and skills they purport to measure in isolation. These
instruments assess global learning and global awareness as attributes that are
disconnected from one another and from the real-world problem-solving context in which
they are to be applied. These issues call into question the effectiveness, authenticity, and
validity of these instruments as measures of global awareness and global perspective.
Effective assessment is based on the premise that learning is complex and
integrative and that it involves not only what students know and can do, but also how
they apply knowledge and skills to authentic tasks (American Association for Higher
Education, 1991; Greater Expectations Project on Accreditation and Assessment, 2004).
Authentic assessments present students with real-world tasks that require use of their
collective capabilities, that is, their wide-ranging knowledge and skill base. Authentic
assessments also require students to inductively develop responses based on evidence
rather than allowing them to select answers from a pre-determined set, regardless of
reason. Generally speaking, authentic assessments involve ill-structured challenges that
allow students to rehearse problem-solving skills tailored to the uncertainty of personal,
civic, and professional tasks (Wiggins, 1990). Performance assessment is often used as a
synonym for authentic assessment (Palm, 2008).
7
The above-mentioned assessments do not require students to apply their global
awareness and global perspective to authentic real-world tasks, therefore scores yielded
from these measures may not be valid indicators of these SLOs. According to Wiggins
(1990), validity should depend in part on the premise that the activity mirrors a realworld test of knowledge and skill. Validity is “an evaluative judgment of the degree to
which empirical evidence and theoretical rationales support the adequacy and
appropriateness of interpretations and actions based on test scores or other modes of
assessment” (Messick, 1996, p. 1). Validity is not a characteristic of the instrument itself;
rather, validity represents the degree to which evidence supports interpretations of
assessment data and actions based on those interpretations (American Educational
Research Association [AERA], American Psychological Association [APA] & National
Council on Measurement in Education [NCME], 1999; Messick, 1996; Moskal &
Leydens, 2000). To estimate validity, researchers engage in an ongoing process of
accruing evidence to support and/or refute the use of assessment results for making
various types of decisions (Messick, 1996; Steen, 1999). In short, validation addresses
“the appropriateness, meaningfulness, and usefulness of the specific inferences made
from test scores” (AERA, APA & NCME, 1999, p. 9).
Validity and reliability of authentic assessments should be estimated by
standardizing appropriate criteria for a variety of possible student responses. Reliability is
the consistency or agreement of scores across raters and testing situations (Moskal &
Leydens, 2000). Validity and reliability are interrelated concepts in that estimates of
reliability affect estimates of validity. An instrument may be considered highly reliable
yet yield low estimates of validity. For example, a scale may be highly reliable—the scale
8
registers the same weight for a five-pound bag of potatoes consistently—yet be an invalid
measure of the number of potatoes in the bag. However, if an instrument is to be
considered valid, it must also yield high estimates of reliability. Evidence must indicate
that the test measures what it is designed to measure and that the test yields consistent
results. To extend the example, if a scale registers a different weight each time a bag of
potatoes is weighed, the scale would yield low estimates of validity as an instrument to
measure weight. Estimates of assessment score reliability hold the same high-stakes
consequences as estimates of validity because they indicate the extent to which decisions
made on the basis of those scores are fair and accurate. This is particularly true with
large-scale assessments, where decisions based on student learning data are often
significant and irreversible (Jonsson & Svingby, 2007).
Given the proliferation of global learning initiatives across the United States and
the dearth of appropriate assessments, there is a need for authentic instruments for the
measurement of students’ global awareness and global perspective. Moreover, there is a
need for studies that allow researchers and educational decision-makers to estimate the
extent to which data yielded from these instruments represent valid and reliable
measurements of students’ global awareness and global perspective.
Setting for the Study
Global awareness and global perspective are two SLOs for an institution-wide
global learning initiative at a large, public, urban, research university in South Florida.
The purpose of this particular initiative is to provide all undergraduate students with
curricular and co-curricular opportunities to develop these outcomes through global
learning. All undergraduates, both native and transfer, take a minimum of two global
9
learning courses—one as part of the general education curriculum and one as part of their
major program of study—and participate in co-curricular activities designed to increase
students’ global awareness and global perspective. Global learning courses are designed
to enhance students’ global awareness and global perspective through components such
as international and global content, active learning strategies, team teaching, integrated
co-curricular activities, and interdisciplinary and problem-based curricula.
The university has developed a process for developing and approving global
learning courses. Faculty and Student Affairs staff members who are developing new
courses and activities, or who are revising existing courses and activities for global
learning, participate in interdisciplinary and interdepartmental workshops, for which they
receive a stipend. Workshop participants learn how to develop course and activity
outcomes aligned with the initiative’s SLOs and they also learn how to implement active
learning strategies, interdisciplinary and global content, and appropriate authentic
assessments for global learning. Following the workshop, faculty and staff members
design global learning course syllabi and comprehensive assessment plans (see Appendix
A) for submission to Faculty Senate global learning curriculum committees. These
committees assess new and revised courses for adherence to course approval guidelines
(see Appendix B).
The university’s provost established an administrative office to coordinate all
aspects of the initiative, including the development of global learning SLOs, faculty and
staff development, and pre/post assessment of the SLOs. The provost appointed an
Associate Professor of Teaching and Learning in the university’s College of Education as
director of the global learning office. The director appointed this researcher, a doctoral
10
candidate in Educational Administration and Supervision, as associate director. Both the
director and associate director possessed experience and expertise in the field of global
education and assessment.
The director and associate director led a year-long series of focus groups and
discussions to determine SLOs for the university’s global learning initiative. In these
talks, faculty, staff, students, and other institutional stakeholders consistently cited global
awareness and global perspective as among the most important learning outcomes for 21st
century undergraduates. At the end of this process, the university’s faculty senate and
Board of Trustees approved the following wording for the initiative’s SLOs:
Global Awareness
Knowledge of the interrelatedness of local, global, international, and intercultural
issues, trends, and systems.
Global perspective
The ability to conduct a multi-perspective analysis of local, global, international,
and intercultural problems.
Global engagement
The willingness to engage in local, global, international, and intercultural problem
solving.
The director sought instrument that would enable direct assessment of the
knowledge (global awareness) and skill (global perspective) SLOs and indirect
assessment of the attitude (global engagement) SLO. These instruments would be
delivered to samples of incoming freshmen and transfer students as well as to samples of
graduating seniors. This would allow stakeholders to evaluate the effect of the entire
11
global learning program on student achievement of the global awareness and global
perspective learning outcomes. A review of existing instruments yielded several that
could be used to assess global engagement. Following a thorough item analysis, the
Global Perspective Inventory (Braskamp et al., 2009) was chosen for this purpose.
However, a dearth of instruments was found that directly measure global awareness and
global perspective as interrelated knowledge and skill outcomes in an authentic problemsolving context. Additionally, the review did not reveal any instruments designed to
measure these outcomes across the curriculum.
The director sought to develop an instrument in-house in order to fulfill the
university’s assessment needs. Based on their knowledge of global education and
educational assessment, the director and associate director determined that global
awareness and global perspective would be best measured by a performance assessment.
Performance assessments require students “to actively accomplish complex and
significant tasks, while bringing to bear prior knowledge, recent learning, and relevant
skills to solve realistic or authentic problems” (Herman, Aschbacher, & Winters, 1992, p.
2). Performance assessments are often evaluated using rubrics—scoring guidelines that
include detailed qualitative descriptions of performance standards (Andrade, 2005;
Popham, 1997). Rubrics are commonly used to measure student achievement on
performance assessments (Moskal, 2000; Popp, Ryan, & Thompson, 2009; Stellmack,
Konheim-Kalkstein, Manor, Massey, & Schmitz, 2009; Thaler, Kazemi, & Huscher,
2009).
The director initiated the development of two rubrics—one for assessing students’
global awareness and another for assessing global perspective—that could be used to
12
evaluate a variety of performance tasks across the curriculum. For pre/post assessment of
the outcomes, this researcher developed a performance task in which students are
required to respond to two open-ended questions concerning complex case studies. This
researcher also developed two rubrics to evaluate students’ written responses to the
questions.
In some disciplines, case studies are used to provide students with necessary
background knowledge and practice applying critical thinking skills to complex, often illstructured problems. The Case Method of Instruction (CMI) is used in professional
education for many fields. CMI is used to encourage student engagement and interest in
addressing real-world situations in a classroom setting. Faculty and staff use case studies
to help students learn to think logically and systematically. Through discussion and
analysis, students also learn to view problems in context and identify multiple theoretical
and analytical perspectives pertinent to each case. Viewed through the lens of a teachercentered paradigm, CMI is considered a pedagogical strategy that faculty and staff use to
transmit knowledge and model critical thinking processes. However, when viewed from
the perspective of a learner-centered paradigm, case study analysis presents students with
an opportunity to demonstrate their ability to integrate general and discipline-specific
knowledge and skills in the construction of an evaluative argument (Golich, Boyer,
Franko, & Lamy, 2000; Hunter & Freed, 2000; Wraga, 2008).
The analysis of case studies can thus be considered a type of performance
assessment. As is typical of performance assessments, written analyses of case studies
allow the assessor to gain insight into the student’s development of specific knowledge
and skills. Performance tasks must be sufficiently complex, and take sufficient time to
13
complete, in order to permit assessment of the knowledge and skills employed in bringing
the task to completion (Messick, 1996; Wiggins, 1993).
Purpose of the Study
In a review of instruments designed to measure undergraduate students’ global
awareness and global perspective, none were found to use rubrics to assess these
outcomes in the context of authentic performance tasks. The purpose of this study is to
demonstrate the extent to which evidence supports the validity and reliability of scores
yielded from rubrics designed to measure undergraduate students’ global awareness and
global perspective.
Research Questions
This study will address four research questions. These research questions are as
follows:
1. To what extent does evidence support the reliability of scores yielded from a
rubric measuring students’ global awareness?
2. To what extent does evidence support the reliability of scores yielded from a
rubric measuring students’ global perspective?
3. To what extent does evidence support the validity of scores yielded from a
rubric measuring students’ global awareness?
4. To what extent does evidence support the validity of scores yielded from a
rubric measuring students’ global perspective?
Theoretical Framework
Constructivism is the guiding theoretical framework of this study, specifically
Cognitive Flexibility Theory (CFT), which focuses on learning and cognition in ill-
14
structured content domains (ISDs). ISDs require the individual to flexibly apply
background knowledge and skills to unique cases (Spiro & Deschryver, 2006; Spiro,
Vispoel, Schmitz, Samarapungavan, & Boerger, 1987). Whereas well-structured domains
such as basic math and sciences involve algorithmic, generalizable, and predictable
content knowledge and processes, ISDs are characterized by a high degree of irregularity,
multi-dimensionality, and interconnectedness. Examples of ISDs include the humanities,
social sciences, arts, and advanced and applied sciences. According to Spiro and
Deschryver (2006), all professional domains contain aspects of ill-structuredness.
Unbounded, real-world situational problem solving also presents cognitive challenges
associated with ill-structuredness. CFT addresses how knowledge and skills should be
organized and acquired to facilitate transfer to a range of new, unanticipated situations,
(i.e., cognitive flexibility).
According to CFT, individuals must consider multiple unanticipated contextual
variables in order to think critically in ISDs. CFT theorists advocate for instructional
approaches that begin with multiple perspectives on subject matter rather than narrow,
reductionist representations. Constructivist teaching strategies that activate
interdisciplinary connections and adaptive knowledge assembly encourage intellectual
independence. Pedagogical approaches aligned with CFT should also facilitate the use of
context-dependent protocols. In ISDs, cases, (i.e., “examples, occurrences, events,
occasions of use of conceptual knowledge”; Spiro, Coulson, Feltovich, & Anderson,
1988, p. 7), are often complex and variable from one instance to the next. Exposure to
and evaluation of multiple case representations is necessary to overcome overreliance on
reductive cognitive schema. Given that logical reasoning based on a variety of sometimes
15
unrelated precedent cases is required for critical thinking in ISDs, case analysis is the
foundation of cognitive flexibility (Spiro & Deschryver, 2006; Spiro, Vispoel, Schmitz,
Samarapungavan, & Boerger, 1987).
Significance of Study
This study describes a process for estimating the extent to which scores yielded
from two rubrics represent valid and reliable measures of students’ global awareness and
global perspective. Validity and reliability are critically important to institutions when
selecting and using instruments to assess learning outcomes, particularly for pre/post
assessments. Colleges and universities use student learning assessment data to make a
variety of decisions that directly affect curriculum, faculty development and placement,
planning, and budgeting. Institutions must provide evidence to accrediting agencies,
students, and other stakeholders that demonstrates the extent to which decisions made
over time on the basis of assessment data are valid, (i.e., meaningful, useful, and
appropriate; Messick, 1998). This must be done because these decisions have ethical,
instructional, and practical implications that influence the efficacy of the educational
endeavor.
The Code of Fair Testing Practices in Education was developed to establish
ethical guidelines for professionals “in fulfilling their obligation to provide and use tests
that are fair to all test takers regardless of age, gender, disability, race, ethnicity, national
origin, religion, sexual orientation, linguistic background, or other personal
characteristics” (Joint Committee on Testing Practices, 2004, p. 2). The first part of the
code, “Developing and Selecting Appropriate Tests,” recommends that developers
provide information that users need in order to choose suitable tests. This includes an
16
ample explanation of how test content was chosen and how the test was developed, as
well as data concerning validity and reliability. Additionally, developers are urged to
provide guidance concerning the interpretation and appropriate use of test results. Test
users need all of this information to carry out their own ethical practice. After defining
their needs in terms of test purpose, test sample, and content and skills to be tested, users
should identify the most appropriate instrument following a detailed review of available
test information. This study offers an example of ethical reporting for test developers and
users.
This study also has instructional implications. Global awareness and global
perspective are learning outcomes that are operationally defined in this study. However,
these outcomes and definitions are not universally adopted by all colleges and
universities engaged in global learning initiatives. The university described in this study
adopted these terms and definitions based on feedback gained from student, faculty, and
staff focus groups, open forums, and college and departmental discussions. Assessment
instruments were designed to specifically measure student achievement of the outcomes
as described in these talks. The process of identifying SLOs and developing assessment
instruments in-house has been found to positively impact instruction. The process has
been found to encourage faculty buy-in to assessment, resulting in increased use of
student learning data to improve content and pedagogy. It has also been found to lead to
increased coherence in the organization of curriculum and increased understanding of the
value, as well as the limits, of assessment data (Crossley & Wang, 2010).
Practically speaking, this study presents an example of how an institution can
leverage internal resources in order to balance the need for self-improvement with the
17
exigencies of external accountability. Over the past decade, accrediting agencies have
increased pressure on institutions to assess SLOs and present evidence that assessment
data are used to improve such areas as curriculum, instruction, resource allocation, and
student services (Volkwein, 2009). The process described in this study may be found to
be useful for the development of assessment instruments for SLOs at the classroom,
program, or graduation levels. As faculty learn and implement the process, they can serve
as mentors to others, empowering and capitalizing on in-house expertise (Rivas, Jones, &
Pena, 2010).
Delimitations
This study describes the development and validation of rubrics used to measure
student responses to a performance assessment, namely open-ended questions concerning
complex case studies. It may not be possible to generalize the findings of this study for
the development of measurement criteria or the establishment of validity and reliability
evidence for other types of assessment instruments. The activities described in this study
relate specifically to two SLOs, global awareness and global perspective, both of which
are operationally defined herein. It may not be possible to generalize this study’s findings
to other SLOs. In addition, this study is delimited to an ethnically and racially diverse
population of students attending a large, urban, multi-campus public research institution
in South Florida. Learning gains achieved by these students may not be generalizable to
other student populations in other types of institutions of higher learning.
Assumptions
The underlying premise of this study is that global learning is a substantively
different educational process than that which is typically present in American higher
18
education. This study is based on the idea that specific learning strategies constitute
global learning and that these strategies enable the development of global learning SLOs.
Whether inside or outside of the classroom, whether studying, serving, or working at
home or abroad, students need to engage with others in diverse, interdisciplinary, and
problem-based learning environments in order to develop global awareness and a global
perspective. Therefore, this study rests on the assumption that when global learning
strategies as described in this study are implemented with fidelity, students will make
learning gains in global awareness and global perspective.
Definitions of Terms
Authentic Assessment. Assessment, which involves tasks that “are either replicas of or
analogous to the kinds of problems faced by adults citizens and consumers or
professionals in the field” (Archbald & Newmann, 1993, p. 206)
Case Method of Instruction. A pedagogical approach that emphasizes the teaching of
problem solving and decision making skills through the analysis of real-life situations.
Cognitive Flexibility Theory. A constructivist theory based on the idea that learning in
ill-structured domains is best achieved in an environment that emphasizes “the use of
multiple mental and pedagogical representations; the promotion of multiple alternative
systems of linkage among knowledge elements; the promotion of schema assembly (as
opposed to the retrieval of prepackaged schemas); the centrality of “cases of application
as a vehicle for engendering functional conceptual understanding; and the need for
participatory learning, tutorial guidance, and adjunct support for aiding the management
of complexity” (Spiro, Coulson, Feltovich, & Anderson, 1987, p. 1).
19
Constructivism. An epistemology based on the idea that human beings generate
knowledge and meaning by reflecting on experience.
Global Awareness. Knowledge of the interrelatedness of local, global, international, and
intercultural issues, trends, and systems (Florida International University, 2010, p. 23).
Global Citizenship. The willingness of individuals to apply their knowledge of
interrelated issues, trends, and systems and multiperspective analytical skills to local,
global, international, and intercultural problem solving (Florida International University,
2010, p. 58).
Global Education. Global education involves learning about those problems and issues
which cut across national boundaries and about the interconnectedness of systems—
cultural, ecological, economic, political, and technological. Global education also
involves learning to understand and appreciate our neighbors who have different cultural
backgrounds from ours; to see the world through the eyes and minds of others; and to
realize that other peoples of the world need and want much the same things (Tye, 1990,
p. 5).
Global Learning. The process by which students are prepared to fulfill their civic
responsibilities in a diverse and interconnected world (Hovland, 2006).
Global Perspective. The ability to conduct a multi-perspective analysis of local, global,
international, and intercultural problems (Florida International University, 2010, p. 23).
Ill-structured Domain. A knowledge domain in which the following two properties
hold: (a) each case or example of knowledge application typically involves the
simultaneous interactive involvement of multiple, wide-application conceptual structures
(multiple schemas, perspectives, organizational principles, and so on) . . . and (b) the
20
pattern of conceptual incidence and interaction varies substantially across cases
nominally of the same type, that is, the domain involves across-case irregularity (Spiro,
Feltovich, Jacobson, & Coulson, 1995, p. 92).
Inter-rater Agreement. Occurs when different raters assign the same score to the same
subject, (i.e., raters are interchangeable; Kozolowski & Hattrup, as cited in Fleenor,
Fleenor, & Grossnickle, 1996).
Norming. Coming to consensus regarding the meaning of rubric scores and the
characteristics of student responses that warrant each score.
Performance Assessment. Product- and behavior-based measurements based on settings
designed to emulate real-life contexts or conditions in which specific knowledge or skills
are actually applied (American Educational Research Association, American
Psychological Association, & National Council on Measurement in Education, 1999, p.
179).
Perspective. Ordinarily unexamined assumptions, evaluations, explanations, and
conceptions of time, space, causality, etc. (Hanvey, 1975)
Power. The probability that a false null hypothesis will be rejected.
Quasi-experimental. An experimental research design that does not include random
assignment of subjects to comparison groups. This design allows researchers to look at
relationships between variables rather than causes and effects.
Reliability. The consistency or agreement of scores across raters and testing situations.
Reliability in this study is operationally defined as the percentage of inter-rater
agreement.
21
Rubric. A systematic scoring guideline to evaluate behaviors, documents or performance
through the use of detailed performance standards.
Student Learning Outcome. A measurable outcome focused on students’ knowledge,
skills, or attitudes after completing a degree program with the graduating student as the
unit of analysis.
Validity. An evaluation of the degree to which evidence supports interpretations of
assessment data and actions based on those interpretations (Messick, 1996).
Well-structured Domain. A knowledge domain that involves algorithmic, generalizable,
and predictable content knowledge and processes.
Overview of Succeeding Chapters
This research proposal consists of two additional chapters. Chapter 2 presents a
review of related literature. The chapter includes an overview of global awareness and
global perspective as defining SLOs of global education and a review of literature
defining global learning as a process for developing these SLOs. Chapter 2 continues
with a review of instruments used to assess global awareness and global perspective and
the validity and reliability studies conducted for these instruments. Chapter 3 describes
the methods that will be used in this study. The chapter reviews the study’s hypotheses,
research design, variables, population, sample, instrumentation, data collection and
analysis procedures, and limitations.
22
CHAPTER II
Review of Literature
This chapter begins with an overview of global awareness and global perspective
as defining SLOs of global education. It continues with a review of literature placing
global learning within the context of global education and defining it as a process for
developing global awareness and global perspective. It is important to firmly establish the
theoretical consistency of these components for several reasons. Most importantly,
effective student learning is based upon coherent alignment of learning outcomes,
content, pedagogical strategies, and assessment (Biggs, 1999; McTighe & Thomas,
2003). In addition, the generalizability of this study’s results depends in part on the
clarity of terms used. Coherence and clarity also have implications for the internal
consistency of the study and the further research it prompts. Following this discussion,
Chapter 2 continues with a review of instruments that have been used to assess global
awareness and global perspective and an exploration of validity and reliability studies
conducted for these instruments.
Global Awareness and Global Perspective:
Defining Outcomes of Global Education
Scholars often cite the late 1960’s as the period of inception for global education
(Gaudelli, 2003; Tucker, 1996; Tye, 2009). Some attribute the beginning of a global view
of education to the release of Apollo 8 images of Earth as a big blue marble, floating in
space, devoid of arbitrary divisions such as national borders (Becker, 1969; Gaudelli,
2003). However, a close reading of the development of the concept of international
education reveals the hidden origins of global education as far back as the 17th century
23
(Stomfay-Stitz, 1993; Sylvester, 2002). Through such an analysis it is also possible to see
how the concepts of interrelatedness and multiple perspectives—the core features of
global awareness and global perspective—help to define global education as a unified
conceptual framework.
Two Approaches to the Development of International Education
The history of the development of international education is marked by two
conceptual approaches: education for international understanding and education for world
citizenship (Sylvester, 2002; Sylvester, 2003; Sylvester, 2005). This is evidenced by the
Library of Congress’ definition for the subject heading international education as “works
on education for international understanding; world citizenship, etc.” (Library of
Congress Authorities, 2011).
Although these two approaches have been used interchangeably to describe the
nature and purposes of international education, they differ significantly in terms of their
professed goals, SLOs, unit of analysis, curricular and co-curricular content, and program
and pedagogical planning strategies. A review of literature reveals that prior to the late
1960’s, the education for international understanding approach dominated the dialogue
on the nature of international education. However, throughout that period, and
increasingly after the late 1960s, theorists around the world explored a different
philosophy, one that was based on a holistic view of humanity and the environment—
namely the world citizenship approach to education. It is argued here that this nascent
world citizenship approach ultimately gave rise to a unique framework that is now
commonly referred to as global education. This framework can be viewed as distinct
24
from that of international education, which is still firmly rooted in the traditional
practices associated with the education for international understanding approach.
Whereas the global education framework includes many of the ideas and practices
associated with international education, international education as a conceptual
framework does not address the defining characteristics of global education. Apropos to
this study, knowledge of the world’s interrelatedness (global awareness) and the ability to
view the world from multiple perspectives (global perspective) are defining SLOs of
global education, yet these are seldom or cursorily addressed by proponents of
international education. The education for international understanding framework
provides a generalized rationale for all international activities at educational institutions.
In contrast, global education is primarily concerned with educational activities and
processes that lead to specific global learning outcomes, (e.g., global awareness and
global perspective). This is explained in more detail in the following sections.
Education for international understanding approach. Prior to the late 1960’s,
the education for international understanding approach strongly influenced the
development of international education. Calls for this type of education emerged with the
rise of the nation-state and increased nationalistic wars in the late 19th century (Butts,
1970). It was claimed that peaceful relations among nations could be achieved through
knowledge of the “other” (Bonney, 1894; Buell, 1925; Good, 1960; Prescott, 1930). Butts
(1970) described three elements of education for international understanding (a) objective
study of other societies in K-16 curricula, (b) student and faculty research and learning
abroad, and, (c) educational development aid. These elements fall within Arum and Van
de Water’s (1992) definition of international education, which was based on their
25
historical review of the term: “’International education’…refers to the multiple activities,
programs, and services that fall within international studies, international educational
exchange and technical cooperation” (p. 202).
Proponents of the international understanding approach view the world as a
composite of nations in existence and world peace as an aggregation of congenial
relations between nations (Faure, Herrera, Kaddoura, Lopes, Petrovsky, Rahnema, &
Ward, 1972; Scanlon, 1960; Stoker, 1933; Wooten, 1929). Based on this worldview,
educational strategies for international understanding are often carried out through
bilateral and multilateral agreements between and among educational institutions,
nations, NGOs, and intergovernmental agencies such as the United Nations Educational,
Scientific and Cultural Organization (UNESCO), the United Nations International
Children’s Emergency Fund (UNICEF), and the United Nations Development
Programme (UNDP). The content of this approach centers on the study of transactional
issues through such disciplines as political science, economics, history, international
relations, intercultural communication, and foreign languages (Butts, 1969; Kandel,
1937; Tewksbury, 1945; Torney-Purta, 1989). Fraser and Brickman (1968) refer
specifically to the transactional nature of this approach in their definition of international
education:
International education connotes the various kinds of relationships—intellectual,
cultural, and educational—among individuals and groups from two or more
nations. It is a dynamic concept in that it involves a movement across frontiers,
whether by person, book or idea. International education refers to the various
methods of international cooperation, understanding and exchange. (p. 1)
26
This definition substantiates the assertion that the education for international
understanding approach is predicated on the nation-state as the basic unit of analysis
(Kandel, 1937; Torney-Purta, 1989). The strong national political interests that undergird
this approach are balanced with the idea that well-being among nations is interrelated—
but only to a limited extent. Nations acknowledge interrelatedness and educate for world
peace insofar as it serves their self-interests to do so (Kandel, 1955; Scanlon, 1960;
Wilson & Collings, 1963). This approach is consistent with national adherence to what
Becker (1969) calls the myth of self-sufficiency, which has led to international
unwillingness to fully commit to the idea of interrelated global well-being. Referring to
the tension between national interests and calls for education for world peace in the early
to mid-20th century, Woody (as cited in Sylvester, 2002) commented that, “Educational
leaders and idealists in many lands, East and West, saw the vision and were ready to
work to realize it; but politicians and men of affairs were reluctant” (p. 119). Kenworthy
(1951) concurred with this observation, noting that,
Everywhere nationalism is a potent force, and there is still fear lest too much
emphasis on education for a world society result in minimizing education for
national citizenship…The one phrase which various nations seem to be willing to
use is ‘education for international understanding’ as attested to by the adoption of
this phraseology by UNESCO after long and heated debates. These words imply a
less ambitious approach and one which most governments are willing to approve.
(p. 200)
The concept of interrelatedness is present in the literature on education for
international understanding, but exploration of the concept of perspective is
27
conspicuously absent. There are some calls for students to acquire “universal respect for
diversity” (Kenworthy, et al., as cited in Sylvester, 2003, p. 192) and “appreciation for
the character, attainments, and traditions of other peoples” (Smith & Crayton, as cited in
Sylvester, 2002), but there is no discussion of the need to view the world through others’
eyes nor of the role perspective plays in students’ ability to understand how the world
works. Theorists who adhere to this approach refer primarily to SLOs describing
students’ knowledge of world affairs, communication competencies, and attitudes
consistent with peaceful international relations (Anderson, 1954; Kirkwood, 2001). Calls
for students to think critically about the world’s interrelatedness through diverse
perspectives emerge in the literature on education for world citizenship.
Education for world citizenship approach. Whereas education for international
understanding is based on a view of the world as the sum of nations in existence,
education for world citizenship is concerned with the world as a totality. This is an
important distinction, for as Becker (1969) has observed, “an aggregation of knowledge
about the parts that make up the world is not equivalent to an understanding of the world
as a whole” (p. 26).
The world citizenship approach to defining international education emerged from
the idea that the planet is a single integrated entity that has been divided into individual
nation-states. This view was not born with space travel. The concept of world citizenship
pre-dates that of international understanding. As Fraser and Brickman (1968) noted in
their documentary history of international and comparative education, “Prior to the
nineteenth century, the terms cosmopolitanism and universalism were accepted and
understood, but the idea of internationalism was virtually unknown” (pp. 18-19).
28
Cosmopolitanism, or world citizenship, dates back to ancient Greece, but
Sylvester (2002) credits John Amos Comenius (1592-1670), the Moravian bishop
considered the father of universal education (Piaget, 1993; Scanlon, 1960), with
pioneering the intersection between education and world citizenship. In his work,
Comenius was concerned with the whole. He viewed education as the “art of teaching all
things to all men” (Comenius, as cited in Piaget, 1993, p. 2). In The Way of Light, he
envisioned an international university that would serve all men, regardless of race,
religion, or nationality, and educate them for peace and the well-being of all humanity
(Sylvester, 2002). He believed in the power of the newly invented printing press to
support his pansophic ideal of a universal body of knowledge that could be applied to the
solution of universally relevant problems. He even went so far as to suggest that for the
good of peace and communication, all men should become bi-lingual—conversant in
their home language and in an international lingua franca that would allow the
propagation of ideas beyond borders. These ideas and others lead to the conclusion that
Comenius saw himself as a world citizen in the most contemporary meaning of the term
(Sadler, 1970).
Comenius was a unique thinker in an age in which divisions of age, class,
religion, and community were much more restrictive than they are today. His belief in
universal education was rooted in his holistic vision of the human community. In his
book The Pampaedia, Comenius stated forthrightly his belief that:
It is essential that we should wish that even utterly barbarous peoples should be
enlightened and liberated from the darkness due to lack of knowledge, for they are
a part of the human race and the part should be like the whole; and further, the
29
whole is not the whole if any part is lacking…whoever then does not wish to
appear a half-wit or evil-minded, must wish good to all men, and not only to
himself, or only to his own near ones, or only to his own nation. (as cited in Auba,
1970, p. 55)
These ideas had far reaching effect, both geographically—according to Cotton Mather,
John Winthrop invited Comenius to become president of Harvard—and over the long
term—in international education forums such as the World Congress of Education of the
Columbian Exposition in 1893 and the founding of UNESCO in 1945. Comenius’ work
established the world citizenship approach to international education. He considered
education to be an ethical imperative that transcended national, gender, religious,
cultural, or socio-economic borders. This view is threaded throughout the development of
the concept of international education. A prime example is found in the Russian Prince
Serge Wolkonsky’s (1894) welcoming comments to the World Congress of Education, in
which he exhorted that the union of the words international and educational would:
…loudly proclaim that every one of us belongs, first, to humanity, and secondly,
to one or another nation; may it teach that there is more honour for any one of us
in being a man than in being an American, or a Russian, or a German, or an
Italian, or a Greek, or a Japanese, or whatever else it may be. (p. 38-9)
The basic unit of analysis in the world citizenship approach is the individual. In
this approach, human well-being is completely interrelated. This is consistent with the
view that Earth is an integrated system. This system is influenced by interrelated global
dynamics involving humans, the institutions they create, and the environment in which
they live (Becker, 1969; Burn, 1980; Hicks, 2003; Kenworthy, 1951). In order to educate
30
individuals towards this kind of global or “worldminded” view, proponents of this
approach urge the study of issues, trends, and dynamics that transcend national
boundaries and affect all human and environmental well-being (Gilliom, 2001;
Kirkwood, 2001; Tye & Tye, 1992). Since these global dynamics affect well-being in a
variety of ways, this approach calls for interdisciplinary content and problem-centered
teaching and learning strategies (Anderson, Nicklas, & Broadford, 1994; Kerr, 1979;
Lamy, 1982). Such study helps students to understand how interrelatedness works and
forms the basis of a cognitive disposition towards world citizenship (Anderson, 1979;
Davies, 2006; Murray, 1929).
By the late 20th century, proponents of the world citizenship approach had begun
to use the term global education to describe an emerging conceptual framework, yet
definitional ambiguities remained. This was largely due to the fact that the world citizen
approach had long been subsumed within the term international education. In an attempt
to clarify, Kirkwood (2001) explained that global education had emerged from the “need
for a ‘civic culture’ of individuals across the Earth [who] actively engage in meeting
human needs” (p. 11). She claimed that global education had become a more full-bodied
conceptual framework than that of international education, particularly in terms of its
vision for content and its impact on individual learning. According to Kirkwood, all
people need access to a global education that centers on four themes: multiple
perspectives, comprehension and appreciation of cultures, knowledge of global issues,
and the world as an interrelated system (2001, p. 12). These themes can be integrated
across the curriculum and into all subject areas and educational activities. This is an
extension of Anderson’s (as cited in Bragaw, 2001) assertion that:
31
A student need not be studying things foreign or international, as we have
conventionally thought of these terms, in order to be involved in global education.
There are ways in which a student can study his or her own community and be as
much involved in global education as when he or she is studying a community in
another part of the world. (p. 2)
The weak national political interests that underlie education for world citizenship
allow for a stronger focus on individual learning and individual well-being within the
global system. This is based on mastery of the knowledge and skills individuals need in
order to carry out their rights and responsibilities—particularly those that arise from their
sense of affiliation with interrelated human and environmental communities. Thus it is
that knowledge of interrelatedness and the ability to analyze the world from multiple
perspectives that are defining learning outcomes for global education. Arising from the
need to educate for world citizenship and giving rise to learning activities across the
curriculum, global awareness and global perspective differentiate global education from
international education.
Global Awareness
Global awareness has been called foundational for global citizenship (Gibson,
Rimmington, & Landwehr-Brown, 2008; Merryfield, 2008; Von Karolyi, 2008). While
some have conceived of global awareness as a general familiarity with world events and
information (Tucker, 1982), others describe this outcome as a kind of holistic outlook on
diversity and interdependence (Clark, 2002; Davis & Robinson, 2006; Howard, 2002;
Simonson, 1977). A number of global education theorists combine these two concepts—
knowledge of global issues and the world as an interrelated system—which are akin to
32
two of the themes identified by Kirkwood (2001), to define global awareness as
knowledge of the interrelatedness of global dynamics.
In her work with gifted children, Roeper (2008) often observed an enhanced
cognitive and emotional response to the world’s complexity. She observed gifted
children’s propensity for grasping themselves as part of the world’s interrelated system
and for understanding how they are affected as individuals by global events and trends.
Roeper (1992) combined these ideas in her description of global awareness as “a mind
set, a way of seeing ourselves as an integral part of every aspect of the world. It is a
conscious and unconscious realization that we are completely intertwined and
interdependent with all things on earth” (p. 52). Roeper contended that although gifted
children are born with this kind of global cognitive acuity, global awareness may be
enhanced through education.
Clarke (2004) echoed Roeper’s description of global awareness as a worldview,
but went further, defining it as the “cognitive or knowledge aspect of students’
perceptions of interrelatedness” (p. 56). Gibson, et al. (2008) further refined this focus,
defining global awareness as “knowledge of globalization and the resulting issues and
problems that affect everyone’s lives. It refers to an understanding of interconnectedness
and interdependence of the world” (p. 15). These definitions specify the kind of
knowledge global citizens need in order to think critically about the world. However,
global awareness is seldom associated with lower order cognitive development, (i.e.,
simple recall and comprehension of facts; Bloom, 1956). Global awareness is generally
discussed as knowledge that results from critical analysis of global dynamics and enables
33
the individual to make evaluative decisions in new, ill-defined, and unstructured
situations.
Kniep (1989) did not use the term global awareness, but he did also delineate
specific content that should be taught in order to discern interconnectedness. He claimed
that young people need to be involved in the following domains of inquiry in order to be
prepared to tackle local, national, and global problems:
Human values. Study of universal human values that transcend group identity
(e.g., equality, justice, and liberty) and diverse cultural values that define group
membership and contribute to differing worldviews (e.g., values related to aesthetics,
life-style, and the environment).
Global systems. Study of the workings of the four major interactive global
systems: economic, political, ecological, and technological.
Global issues and problems. Study of the persistent, transnational,
interconnected concerns of our age: peace and security, development, the environment,
and human rights.
Global history. Study of the evolution of human values, the historical
development of contemporary global systems, and the origins of current global issues and
problems.
Merryfield (2008) folded this kind of focus on content into her description of
global awareness as a mindset students need in order to survive in a world “increasingly
characterized by economic, political, cultural, environmental, and technological
interconnectedness” (p. 383). Merryfield contended that it is the duty of social studies
teachers, in particular, to prepare students for this mindset by helping them to understand
34
how the world affects them and how they, their community, and their nation influence
others. She has presented three assumptions teachers hold concerning the characteristics
of global awareness (a) openmindedness; (b) mastery of a multi-disciplinary, global body
of knowledge about how the world works; and, (c) the ability to apply knowledge to
authentic, relevant problems. Merryfield suggested that global awareness could be taught
through specific pedagogical strategies. These include (a) authentic engagement in
projects that solve real-world global problems, (b) reflection on one’s own cultural lens,
and (c) exposure to primary sources from voices around the world.
Merryfield’s discussion of the way in which students acquire global awareness—
through examination of issues from multiple perspectives—is indicative of the
interrelatedness of these two outcomes. Although they can be described as discrete
cognitive characteristics, they are each necessary for the other’s development. Nowhere
in the literature is this more evident than in the work of Robert Hanvey, considered the
father of global education.
Global Perspective
Hanvey’s An Attainable Global Perspective (1975) is seminal in the literature on
global education. For Hanvey, an individual’s perspective is composed of their ordinarily
unexamined assumptions of time, space, causality, etc. A global perspective is a
characteristic of a given group, and it is composed of the differentiated cognitive
attributes of the individual members of that group. In order to move the group toward a
global perspective, individuals should be educated towards development of the following
capacities:
35
Perspective consciousness. Recognition that one has a unique perspective not
necessarily shared by others.
State of the planet awareness. Alertness to the distortions associated with
various means of attaining knowledge of world conditions.
Cross-cultural awareness. Understanding of the diversity of practices and beliefs
in human societies.
Knowledge of global dynamics. Comprehension of key traits and mechanisms
associated with change on a global scale.
Awareness of human choices. Understanding of the problems of choice
associated with an expanded global perspective.
Hanvey (1975) asserted that individuals develop this suite of capacities to a
greater or lesser degree throughout the course of their lives. He stated that the ultimate
goal of education was to prompt students to respond to information about the world with,
at minimum, the thought that, “There may be more there than meets the eye” or “Other
eyes might see it differently” (p. 2). In Hanvey’s list of attributes, knowledge of global
dynamics is foundational to the effort to teach people to view the world from multiple
perspectives.
Case (1993) also viewed knowledge as one of the cognitive building blocks of a
broader perspective. He expanded upon Hanvey’s work by presenting what he perceived
as two interconnected dimensions of a global perspective: the substantive (i.e., the object
being viewed) and the perceptual (i.e., the lens through which the object is viewed).
Case’s substantive dimension is akin to what is defined here as global awareness. Case
defined this dimension as “knowledge of various features of the world and how it
36
works…Included in this dimension is knowledge of interconnected global systems,
international events, world cultures, global geography, and so on” (p. 318). The
perceptual element, however, “describes an orientation or outlook” (p. 318). Case
summarized his view of a global perspective as the “capacity to see the ‘whole picture’
whether focusing on a local or an international matter” (p. 318). He went on to explain
that this picture is constructed by making sense of the world through multiple
perspectives and that one must possess certain characteristics in order to see the whole
picture. These characteristics are:
Openmindedness. Willingness to base our beliefs on the impartial consideration
of available evidence (Hare, as cited in Case, 1993, p. 321)
Anticipation of complexity. Skepticism of explanations that fail to consider with
sufficient imagination the range of interacting global factors and the breadth of plausible
consequences (Case, 1993, p. 322).
Resistance to stereotypes. Skepticism about the adequacy of accounts of people,
cultures, or nations that either are limited to a narrow range of characteristics (i.e.,
important features of the group are ignored) or depict little or no diversity within them
(i.e., group heterogeneity is ignored) (p. 322).
Inclination towards empathy. Willingness and capacity to place ourselves in the
role or predicament of others or at least to imagine issues from other individuals' or
groups' perspectives (p. 323).
Non-chauvinism. The inclination neither to prejudice our judgments of others
because we are not affiliated with them, nor to discount unfairly the interests of others
even if, on occasion, they are incompatible with our own interests (p. 323).
37
Case was careful to point out that although it is possible to define separate
dimensions and components of a global perspective, these elements are so intertwined as
to make it impossible to teach them individually. Rather, his effort in deconstructing the
concept of global perspective was meant to lend credence to the idea that a global
perspective is a complex notion and is a skill, as Hanvey asserted, that individuals
possess in varying degrees for different reasons. In addition, Case claimed that as an
SLO, global perspective is not unique to global education. All quality education is
intended to move students away from “naïve, often mistaken views of the world” (p. 324)
and towards more nuanced understandings based on their ability to critically analyze
content from multiple perspectives.
While not using the term global perspective, Wicklund (1999), also discussed
multiple perspectives as a form of perception in which:
…people can recognize that an event may be viewed, defined or perceived in
more than one manner, through several social focal points. This is in contrast to
the perception of an event as univocal, as having only one meaning, anchored in
the presumed objective, physical nature of that event. (p. 2)
Pike and Selby (2006) also did not use the term global perspective, but the issues
dimension of their four-dimension theory of global education did address multiperspective thinking. This was composed of knowledge of how perspectives are formed;
the ability to analyze, organize, and evaluate new information; and, critical consideration
of other points of view. Also included in the issues dimension was knowledge of
interconnections among issues, trends, and events and research and inquiry skills. The
Selby and Pike (2000) model of global education was rooted in critical thinking and was
38
based on the interrelationship between knowledge of interconnectedness and
multiperspective analysis. To educate for this knowledge and skill base, these authors
advocated for an active, transformative approach to learning that “entails a dynamic
interaction between teachers, learners and multiple sources of information. Thus, the
functions of ‘teaching’ and ‘learning’ are frequently interchangeable” (p. 140). Selby and
Pike’s learner-centered approach to global education pointed the way toward the
formulation of global learning as an educational process designed to enable learners to
construct meaning for active participation in a global society.
Global Learning as an Educational Process
Since 2001, the American Association of Colleges and Schools (AAC&U) has led
the national conversation on global learning through its Shared Futures: Global Learning
and Social Responsibility initiative. Colleges and universities widely utilize AAC&U’s
definition of global learning as “the process by which students are prepared to fulfill their
civic responsibilities in a diverse and interconnected world” (Hovland, 2006). Although
global awareness and a global perspective are commonly cited as the knowledge and
skills that should result from global learning (Hovland, 2009; Loveland, 2010; Skelton,
2010), there is far less agreement upon what constitutes the process that should be
implemented to encourage the development of these outcomes.
When the concept of global learning was first introduced, educators discussed it
in terms of the strategies associated with a traditional international education conceptual
framework: study abroad, language instruction, and area studies programs (Cornwell &
Stoddard, 1999; Hovland, 2009). Increasingly, global learning theorists are attempting to
discern the cognitive behaviors inherent in the outcomes of global learning, identifying
39
effective teaching and learning strategies for these behaviors, and applying these
strategies across the curriculum and co-curriculum (Gibson, et al., 2008; Loveland,
2010). This current focus is more aligned with global education as a conceptual
framework that deals with education as a process. The strategies identified by these
researchers have been grouped under the term global learning. In this process, SLOs
drive the way faculty design curriculum, teach courses, and assess learning, as well as the
ways institutions integrate curriculum and co-curriculum. According to AAC&U, global
learning features “a close alignment between professed goals and actions taken to achieve
those goals” (Musil 2006, p. 4).
Global learning theorists consider global awareness and global perspective to be
interrelated SLOs, (i.e., experiences that bolster one outcome also serve to support the
development of the other.) Gibson, et al (2008) asserted that the cognitive behaviors
underlying these outcomes are those associated with critical thinking. These researchers
identified specific global learning conditions that enable students to gain an
understanding of the world’s interconnectedness through activities that require them to
view the world from multiple perspectives. These conditions are:
Cultural contrast. Also known as culture shock, cultural contrast, prompts
students to examine the extent of difference between values and beliefs on issues. The
greater the frequency and intensity of contrast, the more memorable and meaningful the
global learning experience.
Modern communication technologies. Methods such as email,
videoconferencing, and web-based threaded discussions can enable students to
experience cultural contrast without leaving their home environment.
40
Substantive and authentic goals. Effective global learning experiences engage
students in real-world endeavors that are too complex and extensive for an individual to
complete independently. Ideally, these projects should have intercultural ramifications
and require a wide range of expertise to plan, design, and implement.
Teamwork. Global learning teams should be composed of members from
different countries. The challenges associated with intercultural communication,
particularly when different languages and communications technology are involved,
sensitize students to the need to improve communication for the sake of collaboration.
Gibson, et al. (2008) claimed that global learning is actually enhanced when
learners remain situated in their home environment, as enculturation may inhibit students’
ability to discern their own perspective. Merryfield (2008) also called for students to
reflect on their own and others’ perceptual lenses within the classroom setting. She
recommended the exploration of themes, i.e. civil rights, through source materials from
various cultural, political, and historical perspectives. Echoing the claim by Gibson, et al.
(2008) that global learning is enhanced through teamwork, Merryfield wrote that global
awareness becomes significant when students collaborate to address authentic problems
for the common good.
Interdisciplinary investigation of problems has also become a hallmark of global
learning. Multiple institutions have established interdisciplinary centers and programs in
order to teach for global awareness and perspective (American Association of Colleges
and Universities, 2005). Hovland (2009) asserted that interdisciplinary problems are
particularly important for global learning because they demand that the student bring
multiple disciplinary frameworks to bear on a particular issue. In so doing,
41
undergraduates learn to compare different disciplinary tools and perspectives. This is
consistent with AAC&U’s recommendation in College Learning for the New Global
Century (2007) that students engage with big, complex questions that are both
contemporary and enduring.
In Learning for Democracy: From World Studies to Global Citizenship, Holden
(2000) explored how global learning requires active, rather than passive learning
strategies. Bonwell and Eison (1991) defined active learning strategies as those in which
students go beyond simply listening, they “read, write, discuss, or be engaged in solving
problems. Most important, to be actively involved, students must engage in such higher
order thinking tasks as analysis, synthesis, and evaluation” (p. iii). Typical active learning
strategies used in higher education include debates, peer teaching, case analysis, service
learning, role playing and simulations, computer-based instruction, and cooperative and
team-based learning. Lantis, Kille, and Krain (2010) asserted that active learning
strategies increase comprehension, help students make connections between theory and
practice, and increase knowledge retention. In particular, case study analysis and
problem-based learning encourage these outcomes, as well as critical thinking and
multiperspective analysis (Lamy, 2007).
Given that the ultimate goal of global learning as an educational process is for
students to acquire the knowledge and skills necessary to function successfully as global
citizens, responsibility for global learning lies not only with faculty but also with student
affairs professionals. With a common focus on global learning SLOs, academic and
student affairs professionals can work together to provide a wide variety of venues for
students to rehearse and reinforce their global awareness and global perspective (Bowen,
42
2005; Braskamp, 2010; Hovland, 2009; Temple-Thurston, 2005). This integrated
curricular and co-curricular global learning approach aligns with the “Principles of
Excellence” outlined in AAC&U’s report, College Learning for the New Global Century
(2007), which recommended making SLOs a framework for the entire educational
experience. It also adheres to the Learning Reconsidered (Keeling 2004) paradigm under
which many university student affairs divisions operate. Learning Reconsidered was an
argument for the integrated use of all of higher education’s resources in the education and
preparation of the whole student. It defined learning as a comprehensive, holistic,
transformative activity that integrates academic learning and student development,
processes that have often been considered separate and even independent of each other.
Assessing Global Awareness and Global Perspective
In College Learning for the New Global Century, AAC&U (2007) recommended
that essential SLOs form the basis of both educational intentionality and accountability
across the institution. Additionally, the report’s authors asserted that, “The right standard
for both assessment and accountability at the college level is students’ demonstrated
ability to apply their learning to complex, unscripted problems in the context of their
advanced studies” (p. 40). In its recommendation, AAC&U suggested that essential SLOs
be assessed to this standard at milestones within students’ major program of study, and
that such assessments be incorporated into graduation requirements. This suggestion
called forth two exigencies: one, assessments for common SLOs must address
differentiated content across fields of study, and two, these assessments must be held to
common criteria in order to comparatively analyze and strengthen student achievement as
part of the cycle of continuous improvement.
43
Assessments of global awareness and global perspective must reliably
accommodate both of these needs—differentiated content and common criteria—in order
to be useful for global learning initiatives in higher education. Instruments must be tightly
aligned to the constructs upon which these SLOs are based in order for student scores to
be valid and meaningful. Interpretation of test results must enable faculty, staff, and other
institutional stakeholders to make practical and effective improvements to content and
teaching strategies in order to improve global learning. Institutions have made use of a
variety of instruments to assess global learning, and these will be explored in terms of
their validity and reliability, as well as their meaningfulness and utility, as measurements
of global awareness and global perspective.
Following is a review of instruments that have been used to assess college and
university students’ global awareness and global perspective. These assessments were
chosen for review because they are most closely aligned with this study’s definitions of
global awareness and global perspective. Instruments that purported to assess these
outcomes, even if they included the terms global awareness or global perspective in their
titles, yet were not based on similar constructs were not reviewed herein. The
assessments below are reviewed for the conceptual framework upon which they are
based; explanations of how the tests were developed and the content chosen; the types of
studies used to establish their validity and reliability; and, their construct validity as
assessments of global awareness as defined in this study.
Global Understanding Survey
The Global Understanding Survey (Barrows, et al., 1981) was a landmark survey
developed to assess “what college students actually know and perceive about global
44
relationships and to measure their comprehension of current global complexities”
(Barrows, Clark, & Klein, 1980, p. 10). The survey was a response to a request made by
the Education and the World View project of the Council on Learning for an assessment
of global understanding. The purpose of the study was to identify ways that higher
education professionals could remedy what was then perceived as a deficit in college
students’ global understanding. The authors of the instrument acknowledged from the
outset that a significant goal of the initiative was to determine the meaning of global
understanding, an oft-used but previously undefined term.
Barrows, et al. (1981) based their work on the premise that global understanding
is a complex construct consisting of both cognitive and affective domains. In order to
develop items for the instrument, the authors operationally defined global understanding
as a function of four components (a) knowledge, (b) attitudes and perceptions, (c) general
background correlates, and (d) language proficiency. Empirical data gathered from
student responses on the instrument were then used to determine a reliable and valid
predictive model for the construct.
The knowledge domain of the Global Understanding Survey is most closely
aligned with global awareness as defined in this study. The developers discussed two
primary approaches to determining the content of test items. One approach centered on
traditional international relations and area studies curricula and the other approach dealt
with global issues that transcend nations and regions. The committee decided that the
latter approach would facilitate development of items that were indicative of their
impression of global understanding, as respondents would be able to apply knowledge
from multiple disciplines and identify ramifications of issues across time, space, and
45
social institutions. On the advice of a consulting faculty committee from Eisenhower
College, items were developed to address 13 global themes (a) environment, (b) food, (c)
health, (d) energy, (e) religion, (f) arts and culture, (g) distribution of national
characteristics, (h) relations among states, (i) war and armaments, (j) international
monetary and trade arrangements, (k) human rights, (l) racial and ethnic issues, (m) and,
(n) population. Test items addressed real-world issues and most could be answered
correctly on the basis of knowledge gained from regular reading of newspapers with good
international coverage. As would be noted from analysis of background correlates,
students who reported regular news consumption scored higher on the test. Some items
required background instruction or reading in geography, world history, economics, and
international relations (Torney-Purta, 1982).
The knowledge domain consisted of 101 multiple-choice questions. Items
addressed international institutions, major historical events and trends, and legal and
policy frameworks associated with the 13 global themes. The authors assessed the
domain’s reliability by analyzing the internal consistency of respondent scores. The
authors contended that internal consistency was demonstrated through high reliability
quotients (.84 for freshmen, .86 for seniors, and .87 for two-year college students),
meaning that performance on each item was consistent with performance on others. The
authors did not expressly discuss evidence concerning the domain’s validity. However,
the authors did contend that low correlations between scores on the knowledge, affect,
and language portions of the survey indicated that these domains were indeed distinct,
which was consistent with their proposed construct for global understanding.
46
In his summary of the findings of the study, Barrows admitted that the project fell
short of its purpose. Data analysis did not yield a strongly predictive model for global
understanding and score interpretations did not provide any indications of methods for
improving students’ level of global understanding based on the four-component
construct. In particular, the authors found no relationship between students’ educational
experiences—coursework, language study, or study abroad—and their levels of
knowledge. Barrows reported that the authors’ disappointment in this finding was
mitigated by their perception of the college experience as being deficient, (e.g., fewer
than 20 percent of students reported discussing global issues on a daily basis and more
than ten percent claimed they had never had such discussions). Nevertheless, the lack of
correlation between any sort of experience and performance on the knowledge section of
the test sheds considerable doubt concerning the utility of scores yielded, a major threat
to construct validity (Messick, 1996).
Another likely threat to the validity of these scores was construct
underrepresentation. Construct representation is traditionally achieved through cognitive
task analysis and identification of constituent processes. It is substantiated by evidence
that assessment tasks rely on knowledge and skills that are consistent with those required
by the construct (Messick, 1996). Apart from the expert judge opinions of the
Eisenhower Faculty committee for the knowledge domain, Barrows, et al. (1981) did not
provide any theoretical or empirical evidence that would assist in the determination of
construct representation. In fact, the opposite appears to be true; the authors stated in the
introduction that it was hoped that through the exploration of potential components and
possible correlates, the study might yield some suggestion of the nature of global
47
understanding and how it is formed. On the basis of the data presented by Barrows, et al.
(1981), it appears that the knowledge component of the Global Understanding Survey
was too narrow and failed to address important aspects of the construct of global
understanding that could be empirically linked to formal and/or informal educational
experiences.
In addition, the items in this domain do not align with the construct of global
awareness as defined in this study. The items require students to demonstrate that they
have knowledge of institutions and events that represent international and global
connections, however they do not require students to demonstrate understanding of how
issues, trends, and systems influence each other to produce the globalized context in
which citizens live and work. On the basis of the evidence provided by Barrows, et al.
(1981), the Global Understanding Survey was a test of lower order recall of then-current
world events, institutions, and legal and policy frameworks associated with global
interconnectedness. Data indicated a strong correlation between news consumption and
test performance; however, the assessment did not present students with an authentic task
such as the analysis of news articles. Although the assessment held students to a common
criterion, it did not allow for differentiation for content across the curriculum, nor did it
allow students to bring their diverse background knowledge to bear in their understanding
of the items. For these reasons, the Global Understanding Survey is not a valid
assessment or assessment model for global awareness as defined in this study.
Worldmindedness Scale
The Worldmindedness Scale (Sampson & Smith, 1957) is one of the most
frequently-used instruments to measure a global awareness (Hett, 1993). It is composed
48
of 32 items—16 worldminded statements and 16 non-worldminded statements—
addressing eight thematic dimensions: religion, immigration, government, economics,
patriotism, race, education, and war. Participants express agreement or disagreement with
each statement along a six-point Likert scale.
In defining the construct upon which the instrument is based, Sampson and Smith
(1957) distinguish between “international-mindedness,” which refers to factual
knowledge about international affairs, and “worldmindedness,” which they define as a
“value orientation, or frame of reference, apart from knowledge about, or interest in,
international relations” (p. 99). The authors describe the worldminded individual as one
who is concerned with a global rather than a nationalistic view of problems, whose
primary affiliation is with all of humanity rather than a single national or cultural group.
This orientation certainly places the Worldmindedness Scale within the global education
conceptual framework. However, Sampson and Smith explicitly identify worldmindedness as an attitude, rather than a knowledge set. The items on the scale are worded
in terms of the affective component of attitudes, (i.e., “It would be a dangerous
procedure if every person in the world had equal rights which were guaranteed by an
international charter” and “Our country is probably no better than many others”; p. 100).
Furthermore, the authors’ definition of worldmindedness is not at all based in knowledge
of interrelatedness. These issues both call into question the construct validity of the
Worldmindedness Scale as a measure of global awareness as defined in this study.
Sampson and Smith (1957) established the reliability of the scale through both
odd-even and test-retest methods. They argue that the internal consistency of the frame of
reference of the scale’s items provides evidence of construct validity, as does correlation
49
with a similar instrument, the Ethnocentrism Scale of the California Public Opinion
Scale. Finally, Sampson and Smith employ the known-group technique to establish
validity, by comparing the pretest/posttest difference in Worldmindedness Scale mean
scores of students who travelled to Europe as tourists or with student organizations in the
summer of 1950 with those of students who travelled to Europe with the Quaker
International Voluntary Service. The authors established non-equivalency primarily
through two methods. First, selection for the Quaker service program was based on a
requirement that participants possess a worldminded attitude, whereas this requirement
did not exist for students travelling to Europe for other reasons. Second, pretest
Worldminded Scale means differed significantly between the two groups, with Quaker
service program students scoring higher than students travelling for other reasons.
Global-mindedness Scale
In the words of the late Jane Hett (1993), her Global-mindedness Scale was
developed in order to “measure attitudes of students related to their sense of connection
to, interest in, and responsibility for, the global community and the behaviors associated
with this perspective” (p. 4). Hett intended for her instrument to be used in both
curricular and co-curricular settings. She also placed the term global-mindedness within
Muessig and Gilliom’s (1981, as cited in Hett, 1993) six-component theory of global
education. Hett developed the concept of global-mindedness and her associated survey in
order to overcome several issues that called into question the construct validity of the
concept of worldmindedness and its associated survey. First, Hett claimed that
worldmindedness did not incorporate a contemporary notion of diversity and that several
of the items in the Sampson and Smith (1957) scale were overtly racist. She also noted
50
that worldmindedness was not a truly global concept, in that it was placed in opposition
to nationalism rather than subsuming it within its framework. As Hett noted, research
does not indicate that a person cannot be both patriotic and worldminded. The multitude
of definitions in the literature for the concept of worldmindedness also concerned Hett,
particularly in light of the contextual changes that had occurred in the years since the
development of the term. Hett developed the concept of global-mindedness in order to
address all of these concerns with the construct validity of the term worldmindedness.
The final version of Hett’s (1993) Global-mindedness Scale contains 30 items
which participants score for level of agreement on a five-point Likert scale. Similarly to
the Worldmindedness Scale, Hett’s items also address themes, although she argued that
hers were more valid given her contemporary globalized context. These themes were (a)
interconnectedness of humanity, (b) cultural pluralism, (c) ethic of responsibility/care, (d)
futurist orientation and, (e) behaviors. These themes form the basis of Hett’s definition
for global-mindedness:
People who are global-minded possess an ecological world view, believe in the
unity of humankind and the interdependence of humanity, support universal
human rights, have loyalties that extend beyond national borders, and are
futurists. (p. 9)
The items in Hett’s scale address beliefs and behaviors associated with the themes
in her definition of global-mindedness. Some of these items address a global perspective
as defined in this study, but only as indirect assessments of an individual’s affective or
behavioral inclination towards analyzing the world via multiple perspectives. Examples
of indirect assessment of these skills include the items, “Americans can learn something
51
of value from all different cultures;” “I enjoy trying to understand people’s behavior in
the context of their culture;” and, “I sometimes try to imagine how a person who is
always hungry must feel” (pp. 193-195). There are no items in the scale that directly
assess the individual’s ability to view the world from multiple perspectives.
Hett established validity through both qualitative and quantitative methods. Scale
items were induced from both a review of literature and interviews with subjects who had
demonstrated a commitment to understanding global-mindedness. Expert judges assessed
the content validity of instrument drafts and validity was enhanced through revisions that
addressed their comments and concerns. Hett also assessed content validity through
instrument administration and scoring in a pilot study. Convergent validity was tested by
establishing correlations between the Global-mindedness Scale and instruments that
assessed related concepts (i.e., the Chauvinism and International Understanding subscales
of the Global Understanding Project instrument; Barrows, Ager, Bennett, Braun, Clark,
Harris, & Klein, as cited in Hett, 1993). In addition, Hett explored predictive validity
among the demographic variables collected and mean Global-mindedness Scale scores.
Reliability was established through tests of internal consistency.
Intercultural Development Inventory
This instrument is based on Bennett’s (1986) Developmental Model of
Intercultural Sensitivity (DMIS). The DMIS identifies six orientations that explain how
people construe cultural difference. As people move through these orientations, they
develop greater degrees of intercultural competence, which is defined as the ability to
think and act in interculturally appropriate ways (Hammer, Bennett, & Wiseman, 2003).
Bennett grounds his model in a constructivist view of meaning, asserting that the more
52
perceptual and conceptual discriminations that can be brought to bear on one’s
experience of cultural difference, the more complex and nuanced one’s understanding
and behavioral reaction to the event can become.
Bennett (1986) posits that the following six cognitive schemata can be used to
describe how cultural difference is understood:
Denial. Belief that one’s own culture is the only real one. Other cultures are not
understood or may be understood vaguely.
Defense. Belief that one’s own culture is the only viable one. The world is
divided into “us” and “them” and other cultures are deemed inferior.
Minimization. Belief that one’s own culture is universal. Similarities may be
played up and differences denied or deemed unimportant.
Acceptance. Understanding that one’s culture is one of a variety of cultures. One
experiences difference but does not judge others as unequal.
Adaptation. Experience of other cultures’ changes perceptions and behaviors
appropriate to that culture. One’s view of the world is expanded to include knowledge of
other worldviews.
Integration. State in which one’s experience of self includes movement in and
out of different worldviews. Describes the experience of individuals who describe
themselves as bicultural or multicultural.
The final version of the Intercultural Development Inventory (IDI) includes 50
items, ten of which address demographic variables. Items are composed of statements
clustered around the six DMIS orientations, to which respondents express degrees of
agreement or disagreement along a seven-point Likert scale. As does the Global-
53
mindedness Scale, the IDI also presents items as statements of affect and behavioral
inclination. Sample items such as “People from other cultures are generally lazier
compared to people from my culture,” “People are the same despite outward differences
in appearances,” and “I often act as a cultural mediator in disagreements between people
from different cultures” (Landis, Bennett, & Bennett, p. 100) are indirect assessments of
respondents’ ability to actually view the world from others’ perspectives.
Content validity for the IDI items was established by using statements gleaned
from interviews with student volunteers from a private university in the United States.
Expert judges provided feedback on the items in the draft pool. Revisions to the
instrument were made on the basis of qualitative responses and agreement among expert
judges of .60 or above. To establish construct validity, the developers correlated the IDI
with two comparable instruments: the Worldmindedness Scale (Sampson & Smith, 1957)
and the Intercultural Anxiety Scale, a modified version of the Social Anxiety Scale (Gao
& Gudykunst, as cited in Hammer, Bennett, & Wiseman, 2003). The developers explored
predictive validity by exploring correlations between demographic variables and IDI
scores. Reliability was established by both expert judge agreement as to the validity of
the items and through a confirmatory factor analysis which established the goodness of fit
between the items and the discrete dimensions of the DMIS model.
Global Perspective Inventory
Braskamp, Braskamp, and Merrill (2009) developed the Global Perspective
Inventory (GPI) to assess the holistic development of a global perspective. The
developers base their instrument on two theoretical constructs: intercultural maturity and
intercultural communication. They use the work of Kegan (as cited in Braskamp,
54
Braskamp, and Merrill, 2009) and King and Magolda (as cited in Braskamp, Braskamp,
and Merrill, 2009) to identify three major domains of human development—cognitive,
intrapersonal, and interpersonal—and divide each domain into two scales. In each
domain one scale is based on intercultural development theory and the other is based on
intercultural communication theory. These domains and scales are described as follows:
Cognitive domain. Knowledge and understanding of what one knows and is
important to know. Includes viewing knowledge as complex and taking into account
multiple perspectives.
Knowing scale. Degree to which one views cultural context as important in
judging what one knows and is important to know.
Knowledge scale. Degree of understanding of other cultures and their influence
on global society, as well as level of proficiency in more than one language.
Intrapersonal domain. Sense of self-direction, purpose, and awareness of values
and strengths.
Identity scale. Degree of acceptance of the gender, racial, and ethnic components
of one’s identity.
Affect scale. Degree of acceptance of differing perspectives and emotional
tolerance for complex situations.
Interpersonal domain. Willingness to interact with people who are different.
Includes being able to view others differently and move from dependence to
independence and interdependence.
Social responsibility scale. Degree of sense of interdependence and concern for
others’ well-being.
55
Social interactions scale. Degree of engagement with others who are different
and sensitivity to difference.
The latest version of the GPI (Version 6a) features 69 items, 21 of which address
demographic variables. Respondents express degree of agreement with item statements
on a five-point Likert scale. Of the assessments reviewed for this study, the GPI features
items most closely aligned with the definition of global perspective. The majority of these
items are part of the Cognitive Knowing scale, the most straightforward being, “I can
evaluate issues from several different perspectives.” This statement specifically requires
students to evaluate whether or not they are able to analyze problems from multiple
perspectives. Other pertinent items include, “Cultural differences make me question what
is really true” and a reverse-coded item, “I tend to judge the values of others based on my
own value system” (Braskamp et al., 2009). Although these items speak to respondents’
multiperspective analytical ability, they are not direct assessments of the extent to which
respondents are able to do so. This instrument would make an excellent companion to a
direct assessment of this outcome, in that it could provide insight into respondents’
attitudinal disposition towards exercising their global perspective in complex and diverse
settings.
In their discussion in the GPI manual of the psychometric characteristics of the
instrument, the developers address the issue of trustworthiness of self-report responses.
They write that certain characteristics of the testing situation make it less likely that a
respondent will choose socially desirable answers, thus throwing into question the
credibility of data gathered from the assessment. They claim that respondents are more
likely to be honest when the items are clearly understood and non-threatening and the
56
results are not used for selection purposes.
The developers report that they have used multiple methods to explore the valid
use of scores yielded from the GPI. They established face validity for the survey by
gathering feedback from college students and experts in study abroad as to the extent to
which items were deemed fair and reasonable. The developers claim that this feedback
led to revisions in the instrument, including reduction of items from the 69 administered
in the pilot test to the 40 administered in Version 5 and beyond. The focus of their
exploration of face validity concerned the perceived utility of items for making decisions
that affected campus climate and programs.
The developers also conducted several studies to explore construct validity.
Intercorrelations between the two scales in each domain yielded what was deemed a
“reasonable” relationship between them: .18, .46, and .42. They report that factor
analyses support the assertion that the survey provides a “reasonable” structure of the
three domains of the construct. One study tested convergent validity by correlating the
GPI with the 16 scales of the Inventory on Learning Climate and Student Well-Being
(Walker, as cited in Braskamp, Braskamp, & Merrill, 2010). Twelve of the GPI items
accounted for 47% of the variability of students’ well being and five of the items
accounted for 20% of the variance. This was based on a sample of 185 students attending
a Catholic university in the east.
Further analyses revealed that male and female average responses varied among
the scales, with the greatest difference being on the social responsibility scale. Mean
scores trended upward as students rose in class rank, with the greatest difference being
between freshman and sophomore years. Students in private institutions tend to score
57
higher than students in public institutions, and in a pre/post study of students in semesterlong study abroad programs, differences in mean scores on five of the six scales were
significantly different. The developers contended that these findings are consistent with
those that would be expected within the instrument’s theoretical construct.
The GPIs test-retest reliability and internal consistency were also reported. A
study was conducted to explore the stability of pretest and posttest scores for two groups
of students who spent time studying abroad: those who spent three weeks abroad and
those who spent one semester abroad. In five of the six scales, scores were more stable in
the group of students who spent only three weeks abroad, but the developers do not report
the statistical significance of the difference between these groups. The developers
reported the coefficient alpha reliabilities for each of the subscales, but they did not
present a discussion of what these statistics reveal concerning the GPI’s internal
consistency.
The developers contend that these studies support the valid use of GPI scores
within the following educational contexts: (a) program or institutional interventions, (b)
study abroad, (c) international student orientation, (d) service learning, (e) freshman-tosenior gains, (f) faculty perspectives, and/or (g) the accreditation and quality
improvement process of Forum on Education Abroad. They emphasize that the GPI is not
a criterion-based instrument and that its greatest utility is as a means of focusing
discussion concerning students’ holistic development as globally-oriented citizens.
Summary
This review established the theoretical consistency of the major educational
components of this study—global awareness, global perspective, and global learning—
58
within the context of global education as a conceptual framework. Valid assessments of
global awareness and global perspective must sample knowledge and cognitive processes
that are theoretically consistent with these components (Messick, 1996). Based on this
review, in order to be theoretically valid, assessments of these outcomes must require that
students engage in authentic problem-solving activities concerning issues that transcend
national borders and affect humans and their environment (Gibson, et al., 2008; Kniep,
1989; Roeper, 2008). The content of these real-world problems must be sufficiently
complex as to require the application of multiple disciplinary perspectives and broadbased cultural, socio-political, scientific, and/or historical knowledge in their analysis and
evaluation (Clarke, 2004; Kniep, 1989; Merryfield, 2008). Such assessments must also
require that students demonstrate an ability to discern differentiated viewpoints that
might potentially be brought to bear in the problem’s analysis and solution, as well as the
underlying influences of these viewpoints (Case, 1993; Hanvey, 1975). In summary,
valid assessments of global awareness and global perspective enable faculty and other
stakeholders to gain insight into the knowledge and cognitive processes students utilize to
determine responsible choices in a diverse and interconnected world.
This researcher was not able to find any existing assessment instruments that meet
these theoretical requirements. However, as a result of the review of development,
reliability, and validity data provided by the instruments’ developers, this researcher was
able to identify methods that could provide useful data for those evaluating the
meaningfulness and usefulness of scores yielded from the instrument designed for this
study. These methods will be described in detail in Chapter 3.
59
CHAPTER III
Methods
This chapter outlines the methods that will be used in this study. This is a quasiexperimental study estimating the validity and reliability of scores yielded from rubrics
measuring students’ global awareness and global perspective. Chapter 3 describes the
study’s research questions and hypotheses, research design, variables, population,
sample, instrumentation, data collection procedures, and data analysis procedures.
Research Questions and Hypotheses
This study will address four research questions concerning the extent to which
evidence supports the validity and reliability of scores yielded from rubrics measuring
students’ global awareness and global perspective. Hypotheses have been developed for
each of the research questions:
Research Question 1
To what extent does evidence support the reliability of scores yielded from a
rubric measuring students’ global awareness?
Hypothesis 1. There will be significant inter-rater agreement among scorers using
the global awareness rubric to measure students enrolled in a global learning course and
among scorers using the rubric to measure students who are not enrolled in a global
learning course.
Research Question 2
To what extent does evidence support the reliability of scores yielded from a
rubric measuring students’ global perspective?
60
Hypothesis 2. There will be significant inter-rater agreement among scorers using
the global perspective rubric to measure students enrolled in a global learning course and
among scorers using the rubric to measure students who are not enrolled in a global
learning course.
Research Question 3
To what extent does evidence support the validity of scores yielded from a rubric
measuring students’ global awareness?
Hypothesis 3a. There will be a significant difference on global awareness rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and class status, independent of global
perspective rubric scores.
Hypothesis 3b. There will be a significant difference on global awareness rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and race/ethnicity, independent of global
perspective rubric scores.
Hypothesis 3c. There will be a significant difference on global awareness rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and fluency in more than one language,
independent of global perspective rubric scores.
Hypothesis 3d. There will be a significant difference on global awareness rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and time spent abroad, independent of global
perspective rubric scores.
61
Hypothesis 3e. There will be a significant difference on global awareness rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and previous global learning course
completion, independent of global perspective rubric scores.
Hypothesis 3f. There will be a significant difference on global awareness rubric
scores between students who earn a “B” and above in a global learning course and
students who earn a “D” and below in a global learning course.
Research Question 4
To what extent does evidence support the validity of scores yielded from a rubric
measuring students’ global perspective?
Hypothesis 4a. There will be a significant difference on global perspective rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and class status, independent of global
awareness rubric scores.
Hypothesis 4b. There will be a significant difference on global perspective
rubric scores between those students who are enrolled in a global learning course and
those who are not when controlling for pre-test scores and race/ethnicity, independent of
global awareness rubric scores.
Hypothesis 4c. There will be a significant difference on global perspective rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and fluency in more than one language,
independent of global awareness rubric scores.
62
Hypothesis 4d. There will be a significant difference on global perspective
rubric scores between those students who are enrolled in a global learning course and
those who are not when controlling for pre-test scores and time spent abroad, independent
of global awareness rubric scores.
Hypothesis 4e. There will be a significant difference on global perspective rubric
scores between those students who are enrolled in a global learning course and those who
are not when controlling for pre-test scores and previous global learning course
completion, independent of global awareness rubric scores.
Hypothesis 4f. There will be a significant difference on global awareness rubric
scores between students who earn a “B” and above in a global learning course and
students who earn a “D” and below in a global learning course.
Research Design
The research design that will be used in this study is quasi-experimental (Cook &
Campbell, 1979). Specifically, the study will utilize a pretest/posttest nonequivalent
group design to compare the average learning gains of students enrolled in global
learning courses and students enrolled in non-global learning courses. This design will be
utilized in order to determine the extent to which evidence supports the construct validity
and reliability of the rubrics. The study is designed to enable comparison of known
groups—students who have completed a global learning course and students who have
not completed a global learning course. An assumption of this study is that if students are
exposed to global learning, they are more likely to develop global awareness and a global
perspective. If this assumption is true, then the rubrics should detect differences in
learning gains pertaining to these outcomes between groups. Protocol for evaluating the
63
fidelity of implementation of global learning courses is described below (see “Data
Collection Procedures”).
Participants will not be randomly assigned to comparison groups because the
study will make use of existing classes of students. Selection bias is one of the primary
threats to the internal validity of non-randomized quasi-experimental studies (Campbell
& Stanley, 1963; Cook & Campbell, 1979). This study will control for selection threat in
two ways. The pretest will be used to provide the researcher with some information
concerning the equivalence of groups. The study will also make use of statistical methods
for controlling variation and isolating the independent variable (IV) and extraneous
variables. These methods are explained below.
Variables
The primary IV in this study will be completion of a global learning course. The
dependent variables (DVs) in this study will be student scores on a rubric measuring
global awareness and student scores on a rubric measuring global perspective.
The study will control for several extraneous variables. This will be done in order
to minimize within group variance and maximize experimental variance. Controlling for
these variables will also minimize threats to the study’s internal validity, as explained
below.
Class Status
Students’ class status will be collected (freshman, sophomore, junior, senior) in
order to control for maturation effects. Class status will be coded both as categorical and
continuous variables to facilitate different types of analysis.
64
Race/Ethnicity
Students will be asked to report their ethnicity (American Indian/Alaskan Native,
African-American, Asian/Pacific Islander, Hispanic, White, Unknown/Other). In addition
to these racial/ethnic categories students, will be able to identify themselves as multiethnic. Multi-ethnic students may have a greater propensity for developing global
awareness and global perspective than those who claim a single ethnicity (Banks, 2008;
Moore, 2008). This data will be used to control for selection bias. Race/ethnicity will be
coded as a categorical variable.
Fluency In More Than One Language
Students will be asked to report the number of languages they speak fluently (1, 2,
3 or more). Multi-lingual students may have a greater propensity for developing global
awareness and global perspective than those who speak a single language (Clark, 1981;
Tochon, 2009). These data will be used to control for selection bias. Fluency in more than
one language will be coded as a continuous variable.
Time Spent Abroad
Students will be asked to report the length of time spent abroad (none, less than
two weeks, more than 2 weeks) and the reason for their travel (academic, work, service,
residence). Students who have travelled or lived abroad may have a greater propensity for
developing global awareness and global perspective than those who have not (DonnellySmith, 2009; Sobania & Braskamp, 2009). This data will be used to control for selection
bias. Time spent abroad will be coded as categorical and continuous variables to facilitate
different types of analysis.
65
Previous Global Learning Course Completion
Students will be asked if they have previously completed a global learning course
at the university. It is possible that some students may have previously completed a
global learning course, which could affect the variability of pretest and posttest scores.
Previous global learning course completion will be coded as a categorical variable.
Previous Case Response Assessment Completion
Students will be asked if they have previously taken the case response assessment
pretest. It is possible that some students may have previously taken the pretest as part of
the university assessment program. Students who have previously taken the pretest will
be removed from the sample. This will be done to control for testing bias.
Population
The population for this study is composed of ethnically and racially diverse
undergraduate students enrolled in a large, urban, multi-campus public research
institution in South Florida. Of the approximately 32,901 undergraduates enrolled in this
institution, approximately 65% are of Hispanic/Latino origin, 12% are of White origin,
12% are of Black or African American origin, 3% are of Asian origin, 1% are of
American Indian or Alaskan Native origin, and .05% are of Native Hawaiian or other
Pacific Islander origin. Approximately 93% of undergraduates are classified as in-state
residents and approximately 3% of undergraduates are classified as international students.
Approximately 64% of the undergraduate student body is composed of students
who transfer from another institution to complete their baccalaureate degree. Annually
approximately 616 undergraduates engage in credit-bearing travel abroad programs and
247 engage in non-credit bearing travel abroad, including service and internships. As a
66
public institution, the university has a foreign language requirement for entrance. All
students must complete at least two years of the same foreign language for admission.
The College of Arts and Sciences has an additional foreign language requirement. This
entrance requirement states that students must have proficiency equivalent to the second
semester university level in a foreign language. The university has not collected data on
the number of undergraduates who report fluency in more than one language.
Sample
The study will involve a purposive sample of students enrolled in two global
learning courses and students enrolled in two non-global learning courses. Students
enrolled in global learning courses will be considered the experimental group. Students
enrolled in non-global learning courses will be considered the comparison group. The
sample sizes for each group will be sought on the basis of a power analysis. Power refers
to the probability that a false null hypothesis will be rejected. One of the ways the
researcher can increase power is to increase sample size. Large sample sizes more
accurately reflect the characteristics of the population, thus increasing power and
reducing Type II error. Since it is often impractical to obtain large samples in real-world
settings, researchers rely on sample size rules of thumb in order to achieve maximum
power in the context of available resources (Van Voorhis & Morgan, 2007).
Based upon the maximum number of linearly independent vectors in the model, a
power analysis was conducted. The analysis indicated that for an α of .05 and a medium
effect size, f² = .15 (Cohen, 1988), the power will be at least .99 for an N of at least 400.
Instrumentation
67
This researcher initially drafted a case response assessment instrument in spring
2009. The instrument was composed of two parallel case studies (see Appendix C and D),
open-ended questions for each of the case studies, and draft scoring rubrics. At the
beginning of the spring 2009 semester, the university was working with six global
learning SLOs, therefore the initial scoring rubric addressed each of these outcomes (see
Appendix E). Later in the semester the university narrowed the knowledge and skill
outcomes to two SLOs, and a separate rubric was developed for each of the case studies
(see Appendix F and G).
Six faculty members who were using the global learning SLOs to revise their
existing core curriculum courses field-tested the instruments and rubrics during spring
2009. To evaluate the construct validity of the parallel case narratives, questions, and
scoring rubrics, this researcher conducted a survey of the faculty members (see Appendix
H). The instrument was also sent to three outside global learning experts for feedback
(see Appendix I). The survey results were used to make adjustments to the case
narratives, questions, and rubrics.
A field test of the revised parallel forms of the instrument was conducted in fall
and spring 2009. Feedback from scorers was used to make further revisions to the
wording of the scoring rubrics to enhance construct validity and inter-rater agreement and
reliability. In fall 2010, the university began delivering the final draft of the instrument to
a 10% sample of incoming freshman, transfer students, and graduating seniors. In the
final form of the instrument, one question addresses the global awareness SLO and the
other addresses the global perspective SLO. Two five-level (0 – 4) holistic rubrics are
used to score students’ responses according to their level of achievement of the SLOs
68
(see Appendix J and K). The levels of the rubric correspond to the levels of Bloom's
Taxonomy of Cognitive Development (Bloom, 1956). A score of “3” represents the
cognitive level of analysis. The pilot faculty determined this score to be a meaningful
academic criterion and the minimum criteria for success.
Data Collection Procedures
The researcher will identify global learning and non-global learning courses being
taught in the same areas of the core curriculum and that cover similar subject matter.
Emails describing the research study will be sent to faculty teaching select global
learning foundations courses to introduce the study and to request their cooperation.
Emails will also be sent to chairs of departments which oversee select non-global
learning courses to introduce the study and to request their cooperation in identifying
faculty teaching course sections in which the pre/posttests could be administered (see
Appendix L).
The pretest will be administered in class during the first two weeks of the
semester (see Appendix M). The posttest will be administered in class during the last two
weeks of the semester (see Appendix N). Faculty will be informed that the administration
will take 45 minutes. To encourage students to give the assessment their best effort,
faculty will be asked to remain present during the administration and assign some sort of
grade credit for completion of the task. The researcher will attend class to introduce and
administer the assessments. To protect anonymity, only student IDs will be placed on
assessment forms. The completed assessments will be kept in a locked file in the
researcher's office; only the researcher, scorers, and participating faculty members will
have access to the completed assessments.
69
Trained faculty raters will score the completed assessments. Faculty raters will
receive a stipend to score the assessments involved in this study as well as those collected
as part of the university’s pre/post assessment program. Two raters will read and score
student responses to each question. Each rater will assign each response a score (0 - 4). If
the two raters’ scores are not discrepant, the final response score will be an average of the
two raters’ scores. Discrepancy is defined as a difference of more than one between the
two raters’ scores. For instance, if one rater gives a response a score of “1” and a second
rater gives the same response a score of “3,” the scores are considered discrepant. In the
case of discrepancy, a third rater will read the response and the final score will be an
average of the three raters’ scores.
Prior to the scoring sessions, raters will be sent a letter outlining the scoring
procedures, as well as copies of the case, questions, and rubrics for their review (see
Appendix O). At the beginning of the scoring sessions, the team of raters will participate
in a 60-75 minute training. The training consists of a review of the rubrics, a norming
session in which all raters score and discuss anchor papers, and a sample scoring session
(10% of the total papers to be scored) to establish an inter-rater agreement rate of at least
.80. If this rate is not established, norming will continue with group review of additional
anchor papers.
At the end of the semester, faculty will be asked to provide course grades for
students who have completed the global learning courses. This data will be collected in
order to perform a within-group analysis, described below.
Data Analysis Procedures
Reliability in this study is operationally defined as percentage of inter-rater
70
agreement. Inter-rater reliability of at least .80 agreement between raters will be required
to meet the minimum reliability requirement.
Multiple linear regression will be used to test the hypotheses (Hinkle, Wiersma, &
Jurs, 2003). Multiple linear regression is the general case of the least square solution.
ANOVA and ANCOVA are subsets of the general linear model. Since this study will
determine if rubric scores predict known groups, independent of the number of
covariates, ANCOVA is an appropriate approach to use. These covariates may be
categorical and/or continuous. The general linear model allows for this. The Bonferroni
correction will be used to control for Type I error build-up (Newman, Newman, Brown,
& McNeely, 2006).
One-tailed tests of significance at the .05 level will be used to test hypotheses
because there is no reason to believe that the non-treatment group would do better than
the treatment group. Performing a directional test increases the study’s power (i.e.,
detecting if a relationship exists). Similarly, a within group analysis will determine if a
positive relationship exists between rubric scores and students’ grades in global learning
courses. This is comparable to an ANCOVA holding pre-test scores constant.
Limitations
This study will be limited by the faculty who agree to participate in this study.
Many faculty members will be invited to participate. Those faculty members who choose
to participate may possess characteristics or employ strategies that skew student learning
gains. The study will also be limited by those students who enrolled in the participating
classes and are present in class on the days that the pretest and posttest are administered.
These student samples may not be representative of the entire population of
71
undergraduates attending this university. Other limitations may be evident after data are
collected.
REFERENCES
Adams, J. M., & Carfagna, A. (2006). Coming of age in a globalized world. Bloomfield,
IL: Kumarian Press.
American Association of Colleges and Universities. (2005). Globalizing the curriculum.
Diversity Digest, 8(3), 12.
American Association for Higher Education. (1991). Nine principles of good practice for
assessing student learning. Sterling, VA: Stylus.
American Council on International Intercultural Education Conference. (1996,
November). Educating for the global community, a framework for community
colleges. Paper presented at the Stanley Foundation and the American Council on
International Intercultural Education Conference, Warrenton, VA. Retrieved from
www.stanleyfoundation.org/publications/archive/CC2.pdf
American Educational Research Association, American Psychological Association &
National Council on Measurement in Education. (1999). Standards for
educational and psychological testing. Washington, DC: American
Educational Research Association.
Anderson, C. C., Nicklas, S. K., & Broadford, A. R. (1994). Global understandings: A
framework for teaching and learning. Retrieved from ERIC database. (ED381462)
Anderson, H. R. (1954). Developing an understanding of world affairs. In H. R.
Anderson (Ed.)., Approaches to an understanding of world affairs (pp. 1-10).
Menasha, WI: George Banta Publishing.
Andrade, H. G. (2005). Teaching with rubrics: The good, the bad, and the ugly. College
Teaching, 53(1), 27-30.
Appiah, K. A. (2006). Cosmopolitanism: Ethics in a world of strangers. New York, NY:
W. W. Norton & Company.
Archbald, D.A., & Newmann, F.M. (1988). Beyond standardized testing: Assessing
authentic academic achievement in the secondary school. Reston, VA: National
Association of Secondary School Principals.
Arum, S., & Van de Water, J. (1992). The need for definition of international education
in U.S. universities. In C. B. Klasek (Ed.), Bridges to the future: Strategies for
72
internationalizing higher education (pp. 191-203). Carbondale, IL: Association of
International Education Administrators.
Auba, J. (1970). Comenius and the organisation of education. In C. H. Dobinson (Ed.),
Comenius and contemporary education (pp. 60-75). Retrieved from ERIC
database. (ED079212)
Banks, J. A. (2008). Diversity, group identity, and citizenship education in a global age.
Educational researcher, 37(3), 129-139.
Barrows, T. S., Ager, S. M., Bennett, M. F., Braun, H. I., Clark, J. L. D., Harris, L. G., &
Klein, S. F. (1981). College students’ knowledge and beliefs: A survey of global
understanding. New Rochelle, NY: Change Magazine Press.
Barrows, T. S., Clark, J., & Klein, S. What students know about their world. Change,
12(4), 10-17.
Becker, J. M. (1969). An examination of objectives, needs and priorities in international
education in US secondary and elementary schools. New York, NY: Foreign
Policy Association.
Bennett, M. J. (1986). A developmental approach to training for intercultural sensitivity.
International journal of intercultural relations, 10(2), 179-196.
Biggs, J. (1999). Enhancing teaching through constructive alignment. Higher education,
32(3), 347-364.
Bloom B. S. (1956). Taxonomy of educational objectives, Handbook I: The Cognitive
Domain. New York: David McKay.
Bonney, C. G. (1894). Address of welcome. Proceedings of the international congress of
education of the Columbian Exposition, Chicago, July 25-28, 1893. New York:
National Education Association.
Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the
classroom. Retrieved from ERIC database. (ED336049)
Bowen, L. (2005). Engaging diversity on the homogenous campus: The power of
immersion experiences. Diversity digest, 8(3), 18-19.
Bragaw, D. (2001). Schooling and citizenship in a global age. Retrieved from ERIC
database. (ED460055)
Braskamp, L. (2010). Being effective interventionists to foster global citizenship. Journal
of college and character, 11(1), 1-6.
73
Braskamp, L., Braskamp, D. C., & Merrill, K. C. (2009). Global perspectives inventory
(GPI): Its purpose, construction, potential uses, and psychometric characteristics.
Retrieved from https://gpi.central.edu/supportDocs/manual.pdf
Brunold, A. O. (2005). Global learning and education for sustainable development.
Higher Education in Europe, 30(3 & 4), 295-306.
Buell, R. L. (1925). International relations. New York: Henry Holt and Company.
Burn, B. B. (1980). Expanding the international dimension of higher education. London:
Jossey-Bass Publishers.
Butts, R. F. (1969). America’s role in international education: A perspective on thirty
years. In H. G. Shane (Ed.), The United States and international education: The
sixty-eighth yearbook of the National Society for the Study of Education (pp. 345). Chicago, IL: University of Chicago Press.
Butts, R. F. (1971). International education: overview. In L. C. Deighton (Ed.), The
encyclopedia of education (pp. 164-171). New York: Macmillan.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs
for research. Chicago: Rand McNally.
Case, R. (1993). Key elements of a global perspective. Social education, 57(6), 318-325.
Castells, M. (1999). Information technology, globalization, and social development [PDF
document]. Discussion paper, United Nations Research Institute for Social
Development. Retrieved from
http://vega.soi.city.ac.uk/~abct353/SM1061N/Castells_1999.pdf
Chase-Dunn, C. (1999). Globalization: A world-systems perspective. Journal of WorldSystems Research, 2(3), 187-215.
Clark, J. L. D. (1981). Language. In T. S. Barrows, S. M. Ager, M. F. Bennett, H. I.
Braun, J. L. D. Clark, L. G. Harris, & S. F. Klein, College students’ knowledge
and beliefs: A survey of global understanding. New Rochelle, NY: Change
Magazine Press.
Clark, R. P. (2002). Global awareness: Thinking systematically about the world.
Lanham, MD: Rowman and Littlefield Publishers.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition).
Hillsdale, NJ: Erlbaum.
74
Cole, K. (2003). Globalization: Understanding complexity. Progress in Development
Studies, 3(4), 323.
Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation: design and analysis issues
for field settings. Chicago: Rand McNally.
Corbitt, J. N. (1998). Global awareness profile. Yarmouth, ME: Intercultural Press.
Cornwell, G. H., & Stoddard, E. W. (1999). Globalizing knowledge: Connecting
international and intercultural studies. Washington, DC: American Association of
Colleges and Schools.
Crossley, M., & Wang, L. (2010). Learning by doing: An experience with outcomes
assessment. The University of Toledo Law Review, 41(2). Retrieved from
http://law.utoledo.edu/students/lawreview/volumes/v41n2/index.htm
Davies, L. (2006). Global citizenship: Abstraction or framework for action? Educational
review, 58(1), 5-25.
Davis, N. J., & Robinson, R. V. (2006). Using a research article to foster moral reflection
and global awareness in teaching about religion and politics, theory testing, and
democracy in the Muslim world. Teaching Sociology, 34(3), 296-312.
Donnelly-Smith, L. (2009). Global learning through short-term study abroad. Peer
Review, 11(4), 12-15.
Drucker, P. (1999). Beyond the information revolution. Atlantic Monthly, 284(3), 47-48.
Education Development Center. (2006, September 19). Global youth survey explores
perspectives on social, cultural identity. Retrieved from
http://www.edc.org/newsroom/press_releases/global_youth_survey_explores_pers
pectives_social_cultural_identity
Faure, E., Herrera, F., Kaddoura, A-R., Lopes, H., Petrovsky, A. V., Rahnema, M., &
Ward, F. C. (1972). Learning to be: The world of education today and
tomorrow.
Paris: United Nations Educational, Scientific and Cultural Organization.
Falk, R. (1994). The making of global citizenship. In B. V. Steenbergen (Ed.), The
condition of citizenship (pp. 127-140). London: Sage Publications.
Fleenor, J. W., Fleenor, J. B., & Grossnickle, W. F. (1996). Interrater reliability and
agreement of performance ratings: A methodological comparison. Journal of
Business and Psychology, 10(3), 367-380.
75
Florida International University. (2010, September). Florida International University’s
quality enhancement plan: Global learning for global citizenship. Retrieved from
http://goglobal.fiu.edu/QEP_Report-Final.pdf
Gaudelli, W. (2003) World class: Teaching and learning in global times. Mahwah, NJ:
Lawrence Erlbaum Associates.
Gibson, K. L., Rimmington, G. M., & Landwehr-Brown, M. (2008). Developing global
awareness and responsible world citizenship with global learning. Roeper Review,
30(1), 11-23.
Gilliom, E. M. (2001). Global education and the social studies. Theory into practice,
20(3), 169-173.
Golich, V. L., Boyer, M., Franko, P., & Lamy, S. (2000). The ABCs of case teaching.
Retrieved from http://ecase.georgetown.edu/abcs.pdf
Good, H. G. (1960). A history of western education. New York: Macmillan.
Greater Expectations Project on Accreditation and Assessment. (2004). Taking
responsibility for the quality of the baccalaureate degree. Retrieved from
http://www.mus.edu/transfer/councils/TakingResponsibility.pdf
Green, S. B. (1991). How many subjects does it take to do a regression analysis?
Multivariate Behavioral Research, 26, 499‐510.
Grudzinski-Hall, M. N. (2007). How do college and university undergraduate level
global citizenship programs advance the development and experiences of global
competencies? Retrieved from Dissertations & Theses: Full Text. (AAT
3261868).
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). The Intercultural Development
Inventory: A measure of intercultural sensitivity. International Journal of
Intercultural Relations, 27(4), 421-443.
Hanvey, R. G. (1975). An attainable global perspective. Retrieved from ERIC database.
(ED116993)
Herman, J. L., Aschbacher, P. R., & Winters, L. (1992). Using alternative assessment for
decision making. A practical guide to alternative assessment. Alexandria, VA:
Association for Supervision and Curriculum Development.
Hett, E. J. (1993). The development of an instrument to measure global-mindedness.
(Doctoral dissertation). Retrieved from ProQuest dissertations & theses.
(Publication No. AAT 9408210).
76
Hicks, D. (2003). Thirty years of global education: A reminder of key principles and
precedents. Educational Review, 55(3), 265-275.
Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics for the behavioral
sciences. Boston, MA: Houghton Mifflin.
Holden, C. (2000). Learning for democracy: From world studies to global citizenship.
Theory into practice, 39(2), 74-80.
Hovland, K. (2006). Shared futures: Global learning and liberal education. Washington,
DC: Association of American Colleges and Universities.
Hovland, K. (2009). Global learning: What is it? Who is responsible for it? Peer Review,
11(4), 4-7.
Howard, E. R. (2002). Two-way immersion: A key to global awareness. Educational
Leadership, 60(2), 62-64.
Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on college campuses:
Shifting the focus from teaching to learning. Boston, MA: Allyn and Bacon.
Hunter, B., White, G. P., & Godbey, G. C. (2006). What does it mean to be globally
competent? Journal of Studies in International Education, 10(3), 267-285.
Hunter, C., Singh, P., & Latib, M. (n.d.). About global competence aptitude assessment.
Retrieved from http://www.globalcompetence.org/aboutGCAA/
International Association of Universities. (2003). Internationalization of higher
education: Trends and developments since 1998. Retrieved from
http://portal.unesco.org/education/en/files/20030/10561413171IAU_rev.pdf/IAU_
rev.pdf
Joint Committee on Testing Practices. (2004). Code of fair testing practices in education.
Retrieved from http://www.ncme.org/pubs/pdf/CodeofFairTestingPractices.pdf
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and
educational consequence. Educational Research Review, 2, 130-44.
Kandel, I. L. (1937). Intelligent nationalism in the school curriculum. In I. L. Kandel &
G. M. Whipple (Eds.), Thirty-sixth yearbook of the National Society for the Study
of Education, committee on understanding, Part II, international understanding
through the public school curriculum (pp. 35-42). Bloomington, IN: Public
School Publishing, Inc.
77
Kandel, I. L. (1955). National and international aspects of education. International review
of education, 1, 5-15.
Keeling, R. P. (2004). Learning reconsidered. Washington, D.C.: National Association of
Student Personnel Administrators.
Kenworthy, L. S. (1951). The schools of the world an education for a world society. In C.
O. Arndt and S. Everett (Eds.), Education for a world society (pp. 199-230). New
York: Harper & Brothers.
Keohane, R. O., & Nye, J. S., Jr. (2000). Globalization: What's new? what's not? (and so
what?). Foreign Policy, 118, 104.
Kerr, C. (1979). Education for global perspectives. The annals of the American academy
of political and social science, 442, 109-116.
Kirkwood, T. F. (2001). Our global age requires global education: Clarifying definitional
ambiguities. The social studies, 92(1), 10-15.
Lantis, J. S., Kille, K. J., & Krain, M. (2010). The state of the active teaching and l
earning literature. In R. A. Denemark (Ed.), The international studies
encyclopedia. Chichester, West Sussex, U.K.: Wiley-Blackwell.
Lamy, S. (1982). Teacher training in global perspectives education: The center for
teaching international relations. Theory into practice, 21(3), 206-211.
Lamy, S. (2007). Challenging hegemonic paradigms and practices: Critical thinking and
active learning strategies for international relations. PS: Political science and
politics, 40(1), 112-116.
Lemke, C. (2002). EnGauge 21st century skills: Digital literacies for a digital age.
Retrieved from ERIC database. (ED463753)
Library of Congress Online Catalog. International education. Retrieved from
http://authorities.loc.gov/cgibin/Pwebrecon.cgi?GetScopeNotes=1&SEQ=20110327173119&PID=AV0fuFZlQI3up2Msw7hJQyGUGF4
Loveland, E. (2010). Global learning for all. International Educator, 19(6), 12-23.
McNeil, K. A., Newman, I., & Kelly, F. J. (1996). Testing research hypotheses with the
general linear model. Carbondale, IL: Southern Illinois University Press.
McTighe, J., & Thomas, R. S. (2003). Backward design for forward action. Educational
leadership, 60(5), 52-55.
78
Merryfield, M. (2008). Scaffolding social studies for global awareness. Social Education,
72(7), 363-366.
Messick, S. (1998). Test validity: A matter of consequence. Social Indicators Research,
45, 35-44.
Messick, S. (1996). Validity of performance assessments. In G.W. Phillips (Ed.),
Technical issues in large scale performance assessment (pp. 1-18). Washington,
DC: National Center for Educational Statistics.
Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002).
Psychometric properties and correlates of the Civic Attitudes and Skills
Questionnaire (CASQ): A measure of students’ attitudes related to servicelearning. Michigan Journal of Community Service Learning, 8(2), 15-26.
Moore, J. R. (2008). The role of ethnicity in social studies education: Identity and conflict
in a global age. Social studies research and practice, 3(1), 42-54.
Moskal, B. M. (2000). Scoring rubrics: what, when and how? Practical assessment,
research & evaluation. 7(3). Retrieved from
http://PAREonline.net/getvn.asp?v=7&n=3
Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: validity and
reliability. Practical assessment, research & evaluation, 7(10). Retrieved from
http://PAREonline.net/getvn.asp?v=7&n=1
Murray, S. G. (1929). The international aspect of education. School and Society, 30(764),
213-215.
Musil, C. M. (2006). Assessing global learning: Matching good intentions with good
practice. Washington, DC: Association of American Colleges and Universities.
National Geographic Education Foundation. (2006). Final report: National GeographicRoper Public Affairs 2006 geographic literacy study. Retrieved from
http://www.nationalgeographic.com/roper2006/pdf/FINALReport2006GeogLitsur
vey.pdf
National Leadership Council for Liberal Education & America’s Promise (LEAP).
(2007). College Learning for the New Global Century [PDF document].
Retrieved from http://www.aacu.org/leap/documents/GlobalCentury_final.pdf
Newman, I., Newman, C, Brown, R., & McNeely, S. (2006). Conceptual statistics for
beginners (3rd ed.). Lanham, MD: University Press of America.
79
Nussbaum, M. (2004). Liberal education and global community. Liberal Education,
90(4), 42-48.
Our World Alliance. (2006). Survey of youth identity and citizenship. Retrieved from
http://www.jugendhilfeportal.de/db/adminbin/getfile.php?c_fileid=DE0260000006
Paige, R. M. (2004). Instrumentation in intercultural training. In D. Landis, J. M. Bennett,
& M. J. Bennett (Eds.), Handbook of intercultural training. Thousand Oaks, CA:
Sage Publications, Inc.
Palm, T. (2008). Performance assessment and authentic assessment: A conceptual
analysis of the literature. Practical assessment, research & evaluation, 13(4).
Retrieved from http://pareonline.net/genpare.asp?wh=0&abt=13
Piaget, J. (1993). John Amos Comenius. Prospects, 23(1/2), 173-196.
Pike, G., & Selby, D. (2001). In the global classroom, 2. Toronto, Canada: Pippin
Publishing.
Popham, W. J. (1997). What’s wrong—and what’s right—with rubrics. Educational
Leadership, 55(2), 72-75.
Popp, S. E. O., Ryan, J. M., & Thompson, M. S. (2009). The critical role of anchor paper
selection in writing assessment. Applied Measurement in Education, 22(3), 255271.
Prescott, D. (1930). Education and international relations: A study of the social forces
that determine the influence of education. Cambridge, MA: Harvard University
Press.
Read, G. R. (1966). The international education act of 1966. The Phi Delta Kappan,
47(8), 406-409.
Rivas, O., Jones, I., & Pena, E. (2010). Lessons learned while conducting educational
program assessment. Journal of Case Studies in Accreditation and Assessment, 1,
1-8. Retrieved from http://www.aabri.com/jcsaa.html
Roeper, A. (1992). Global awareness and the young child. Roeper Review, 15(1), 52-53.
Roeper, A. (2008). Global awareness and gifted children. Roeper Review, 30(1), 8-10.
Sadler, F. E. (1970). Comenius as an international citizen. In C. H. Dobinson (Ed.),
Comenius and contemporary education (pp. 60-75). Retrieved from ERIC
database. (ED079212)
80
Sampson, D. & Smith, H. (1957). A scale to measure world-minded attitudes. Journal of
social psychology, 45(1), 99-106.
Scanlon, D. G. (1960). International education: An introduction. In D. G. Scanlon (Ed.),
International education: A documentary history. New York: Columbia
University, Bureau of Publications, Teachers College.
Selby, D., & Pike, G. (2000). Civil global education: Relevant learning for the twentyfirst century. Convergence, 33(1/2), 138-149.
Simonson, M. R. (1977). Global awareness: A curriculum plan for world study. National
association of secondary school principals bulletin, 61, 75-80.
Skelton, M. (2010). Global learning. Education journal, 124, 39.
Sobania, N., & Braskamp, L. (2009). Study abroad or study away: It's not merely
semantics. Peer Review, 11(4), 17-20.
Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. (1988). Cognitive flexibility
theory: Advanced knowledge acquisition in ill-structured domains. Retrieved
from ERIC database. (ED302821)
Spiro, R. J., Feltovich, P. J., Jacobson, M. I., & Coulson, R. L. (1995). Cognitive
flexibility, constructivism, and hypertext: Random access instruction for advanced
knowledge acquisition in ill-structured domains. In L. P. Steffe & J. E. Gale
(Eds.), Constructivism in education (pp. 85-107). Mahwah, NJ: Lawrence
Erlbaum Associates.
Spiro, R. J., Vispoel, W. L., Schmitz, J., Samarapungavan, A., & Boerger, A. (1987).
Knowledge acquisition for application: Cognitive flexibility and transfer in
complex content domains. Retrieved from ERIC database. (ED287155).
Steen, L. (1999). Assessing assessment. In B. Gold, S. Z. Keith & W. A. Marion (Eds.),
Assessment practices in undergraduate mathematics (pp. 1-5). Washington, DC:
Mathematical Association of America.
Steenburgen, B. V. (1994). The condition of citizenship. London, England: Sage.
Stellmack, M. A., Konheim-Kalkstein, Y. L., Manor, J. E., Massey, A. R., & Schmitz, J.
A. P. (2009). An assessment of reliability and validity of a rubric for grading
APA-style introductions. Teaching of Psychology, 36(2), 102-107.
Stoker, S. (1933). The schools and international understanding. Chapel Hill, NC:
University of North Carolina Press.
81
Stomfay-Stitz, A. M. (1993). Peace education in America 1828–1990: Sourcebook for
education and research. London, England: Scarecrow Press.
Sylvester, R. (2002). Mapping international education: A historical survey 1893-1944.
Journal of research in international education, 1(1), 90-125.
Sylvester, R. (2003). Further mapping of the territory of international education in the
20th century (1944-1969). Journal of research in international education, 2(2),
185-204.
Sylvester, R. (2005). Framing the map of international education (1969-1998). Journal of
research in international education, 4(2), 123-151.
Temple-Thurston, B. (2005). Student civic engagement at home and abroad. Diversity
digest, 8(3), 8-9.
Tewksbury, D. G. (1945). New directions in international education. Teachers college
record, 46(5), 293-301.
Thaler, N., Kazemi, E., & Huscher, C. (2009). Developing a rubric to assess student
learning outcomes using a class assignment. Teaching of psychology, 36(2), 113116.
Thompson, J. B. (2003). The globalization of communication. In D. Held and A.
McGrew (Eds.), The global transformations reader (pp. 246-259). Cambridge,
England: Polity Press.
Tochon, F. V. (2009). The key to global understanding: World languages education—
why schools need to adapt. Review of educational research, 79(2), 650-681.
Tomlinson, J. (1999). Globalization and culture. Chicago, IL: University of Chicago
Press.
Torney-Purta, J. V. (1982). The global awareness survey: Implications for teacher
education. Theory into practice, 21(3), 200-205.
Torney-Purta, J. V. (1989). A research agenda for the study of global/international
education in the United States. Retrieved from ERIC database. (ED313295)
Tucker, J. L. (1982). Developing a global dimension in teacher education: The Florida
International University experience. Theory into practice, 21(3), 212-217.
Tye, K. A. (1990). Introduction: The world at a crossroads. In K. A. Tye (Ed.), Global
education: From thought to action (pp. 1-9). Alexandria, VA: The Association for
Supervision and Curriculum Development.
82
Tye, K. A. (2003). Global education as a worldwide movement. Phi delta Kappan, 85(2),
165-168.
Tye, K. A. (2009). A history of the global education movement in the United States. In T.
F. Kirkwood-Tucker (Ed.), Visions in global education: The globalization of
curriculum and pedagogy in teacher education and schools (pp. 3-24). New
York: Peter Lang Publishing, Inc.
Tye, B. B., & Tye, K. A. (1992). Global education: A study of school change. Albany,
NY: State University of New York Press.
Van Voorhis, C. R. W., & Morgan, B. L. (2007). Understanding power and rules of
thumb for determining sample sizes. Tutorials in Quantitative Methods for
Psychology, 3(2), 43‐50.
Volkwein, J. F. (2009). The assessment context: Accreditation, accountability, and
performance. New directions for institutional research, 2010(S1), 3-12. doi:
10.1002/ir.327
Von Karolyi, C. (2008). Introduction to special issue on global awareness and giftedness.
Roeper Review, 30(1), 6-7.
Wicklund, R. A. Multiple perspectives in person perception and theorizing. Theory &
psychology, 9(5), 668-678.
Wiggins, G. (1990). The case for authentic assessment. Practical assessment, research
& evaluation, 2(2). Retrieved from http://PAREonline.net/getvn.asp?v=2&n=2
Wiggins, G. (1993). Assessment: Authenticity, context, and validity. Phi Delta Kappan,
75(3), 200-214.
Wilson, H. E., & Collings, M. R. (1963). Education for international understanding.
National elementary principal, 42(6), 11-17.
Wittgenstein, L. (1953). Philosophical investigations. New York, NY: Macmillan.
Wolkonsky, S. (1894). Address of welcome by the delegate of the ministry of public
education in Russia. Proceedings of the international congress of education of the
world’s Columbian exposition in Chicago, July 25-28, 1893 (pp. 37-9). New
York: National Education Association.
Wooten, F. (1929). The international school of Geneva. School and society, 30(758), 2325.
83
Wraga, W. G. (2008). ‘”Trying our own medicine’”: The case method of instruction in
the United States in historical perspective. International Journal of Case Method
Research & Application, 20(3), 299-312.
84
APPENDICES
85
APPENDIX A
Blank Global Learning Course Assessment Plan
86
87
88
89
APPENDIX B
Global Learning Foundations Course Approval Guidelines
90
91
APPENDIX C
Draft Case Study, “The Case of Hoodia”
Spring 2009
92
Multinational pharmaceutical firms commonly explore, extract, develop, and
distribute drugs from traditional medicinal plants. Please read “The Case of
Hoodia,” a fictional account of a real-life dilemma that involves multiple
stakeholders, including the San, the oldest continual human inhabitants of Africa,
a multinational pharmaceutical firm, the health concerns of obese people around
the world, a large pan-African government research organization, and a South
African non-governmental organization.
After completing the reading, answer questions 1 and 2.
----------------------------------The Case of Hoodia
“So, what do you do?”
Angela Bingham turned to her seatmate and tried to muster a genuine smile.
Although she was proud of her work, Angela disliked being asked such a personal,
invasive question by a stranger. Nevertheless, she was stuck sitting next to this man for
the remainder of her 11-hour flight to Cape Town, so she decided to open up a little.
“I work for a company called Pharmedics. It’s a British pharmaceuticals concern
that specializes in developing drugs from traditional medicinal plants. The medicines we
work on are used to treat asthma, Parkinson’s disease, Alzheimer’s disease, AIDS…you
name it. My latest project is development of an extract from a plant called Hoodia
Gordonii. It grows in the wild all over southern Africa and has been used by the San, or
the Bushmen of the Kalahari, for thousands of years. The San are the first human
inhabitants of Africa. They take Hoodia to stave off hunger and thirst on long hunting and
gathering expeditions and during times of drought. The extract, P57, may turn out to be
an anti-obesity wonder drug.”
“Wow, that sounds interesting and like really good work. Are you a scientist?”
“No, I’m an account director. Actually, Pharmedics is a virtual company—there
are very few of us who are employed directly by the company itself. I work with
outsourced field researchers, lab scientists, clinicians, and manufacturers. I’m a
middleman; I develop a communications strategy between the stakeholders and I
coordinate feasibility studies for research and production. Pharmedics works on initial
isolation of extracts. We leave the commercialization up to the big boys.”
“The ‘big boys’?”
“Yeah, Phizer, Unilever—big multinational pharmaceutical firms. They’ve got the
money and the power to push drugs through the Food and Drug Administration and
such. But tell me, what do you do, um...I can’t believe I already forgot your name…”
Angela’s seatmate smiled graciously. “Roger. Don’t worry about it—I’m an artist,
a sculptor, so I’m a little flighty myself. I’m bringing a commissioned work to Cape Town
to be placed in front of the headquarters of a big shipbuilding company. I work with
metal. The pieces of the sculpture are down in the baggage compartment. I’m going to
South Africa to put them all together.”
“Well, well,” beamed Angela, “that’s basically what I’m going to Cape Town to do,
put together pieces. But I’m no artist. This is more like a sales job, although I’m not quite
sure what I’m selling or to whom.”
Temporarily saved from having to explain further by the arrival of the dinner cart,
Angela leaned back in her seat and closed her eyes. She recalled the conversation
she’d had the previous week with her company’s president, David Campbell, when she
was initially dispatched on this mission.
93
“Angela, I just want you to know that you’ve done incredible work on the clinical
trials of P57. It has enormous commercial potential and Phizer is very interested in
taking it to the next level. But Angela, nothing can happen at all until we work things out
with the San. I’ve got their lawyer, reporters from the Observer, a bunch of NGOs, and
the governments of Namibia, Botswana, and South Africa breathing down my neck…it’s
unbelievable. I didn’t even know the San existed anymore. I need you to go over there
and make everybody happy.”
Angela’s heart pounded. She was used to bringing people together to work as a
team, but this sounded much more complicated than what she usually did. “David, I’m
not sure I understand what you want me to do. Why do we have a problem with the
San? They don’t have the development license on the patent for Hoodia, we do.”
Taking off his glasses, David Campbell stood and began pacing the room. “We
purchased the development rights for Hoodia from the initial patent holder, the South
African-based Council for Scientific and Industrial Research (CSIR), one of the largest
research organizations in Africa. Although it was a government-sponsored institution, it
did not consult with the San, the original holders of the knowledge of Hoodia, before
applying for the patent. Even if they had approached them, the San may have had little
trust for an apartheid-era institution. They may not have even understood what was at
stake for them. The San’s way of life has been undermined by development in southern
Africa. The San are poverty-stricken and they lack education and access to information,
so they have little power to negotiate or profit from developing their indigenous
knowledge…anyway, a South African NGO called BioWatch got wind of the CSIR
agreement with us and leaked it to the press.”
Angela was starting to catch on. “So do the San believe they are the true owners
of Hoodia? Do they want some sort of monetary compensation for their knowledge of
Hoodia?”
“To tell you the truth, the San find the very idea that anyone should pay them for
their knowledge morally abhorrent. The San culture values knowledge as a collective
resource. What’s more, the whole patent process makes little sense to them. They don’t
see how life—even plant life—can be ‘owned.’”
Sitting back down at his desk, Campbell went on to explain how matters were
made even more complicated by the fact that the San were not a single community, but
a group of multiple far-flung communities that lived and travelled throughout South
Africa, Namibia, and Botswana. An advocacy organization had been formed in 1996 to
lobby for the interests of the San communities, the Working Group of Indigenous
Minorities in Southern Africa (WIMSA). Through their lawyer, WIMSA had recently
informed Pharmedics and the CSIR of their decision not to pursue the San’s ‘no patents
on life’ policy in court, as it was too expensive. Instead, the San wanted to negotiate a
benefits-sharing agreement, with Hoodia royalties being used to alleviate poverty and
sustain endangered aspects of San culture. The distribution of such benefits was,
however, potentially problematic. Even if an agreement could be reached between the
CSIR, Pharmedics, and WIMSA, how could a system be created to fairly compensate
multiple nomadic San groups across three countries?
Angela was overwhelmed but determined. “David, I can’t believe what a puzzle
you’ve placed in front of me. I’ll go to Cape Town. I can’t promise I’ll make everyone
happy, but I’ll try to help everyone recognize all the many moving parts and how they
can best fit together.”
94
“Ma’am, would you like eggs or French toast?”
Angela’s attention snapped back into the present.
“Oh, uh, thank you. French toast, please.” She looked away from the steward
and over towards Roger. His dinner tray had been replaced with one featuring eggs and
toast, and the sun was shining brightly through the window.
“Well good morning, sleepyhead, just in time for breakfast! You passed out
without even taking a bite of dinner. I didn’t want to wake you—I hope that’s o.k. We’ve
only got a few more hours before landing.”
“Oh yes, of course. Roger, can I ask you something? You said you are going to
Cape Town to put the pieces of your metal sculpture together. How exactly are you
going to do that?”
“Well, you choose your method depending on the types of metals you are
working with. If the metals are the same, you can weld them together. It takes a lot of
heat and it’s dangerous, but if you are careful the joining will last a long time. If the
metals are different, it’s very difficult to force them together with welding. You generally
have to use some sort of fastener like bolts or rivets. You pick the process to match the
parts.”
“Thank you, Roger. I’m starting to think I should conceive of my task in Cape
Town more in terms of sculpting than selling. You’ve helped me a lot.”
Angela leaned back in her seat. She was grateful Roger had asked her what she
did for a living; moreover, she was glad she’d chosen to open up to him. She smiled to
herself, and this time it was genuine.
________________
QUESTION 1:
Imagine yourself as a reporter for the International Herald Tribune writing a
comprehensive article on Hoodia Gordonii. Who would you interview? What issues
would you make sure to cover in your story?
QUESTION 2:
“The Case of Hoodia” concerns bioprospecting. Bioprospecting refers to the centuriesold practice of collecting and screening plant and other biological material for
commercial purposes, such as the development of new drugs, seeds and cosmetics.
Biopiracy is a negative term referring to the claiming of legal rights over indigenous
knowledge, usually by means of patents, without compensation to the groups who
originated the knowledge. Graham Dutfield has described fundamentally different views
on biopiracy as follows:
In countries like India, the predominant view is that the nation itself is the “victim”
of biopiracy. For Africa, the perception seems to be that the continent as a whole
is prey to the biopiracies. But in the Americas, Australia and New Zealand, the
victims are seen generally as indigenous peoples who usually—though not
always—represent minority populations.
Comment on Dutfield’s possible reasons for drawing these conclusions.
95
APPENDIX D
Draft Case Study, “A Monumental Dilemma”
Spring 2009
96
According to Dr. Martha Honey, co-founder and co-director of the Center on
Ecotourism and Sustainable Development, ecotourism is travel to fragile,
pristine, and usually protected areas that strives to be low impact and is often
small-scale. It helps educate the traveler, provides funds for conservation,
directly benefits the economic development and political empowerment of local
communities, and fosters respect for different cultures and for human rights.
Please read “A Monumental Dilemma,” a fictional account of a travel reporter’s
experience conducting research for a travel magazine article about ecotourism in
Cambodia. After completing the reading, answer questions 1 and 2.
----------------------------------A Monumental Dilemma
It is 4:30 a.m. and as promised, my guide and driver, Kim San, is waiting for me
at the hotel entrance. We had met the previous day to work out a sightseeing schedule
for the week, and he insisted that the first thing I do on my tour of the Angkor
Archaeological Park was witness the sunrise over Angkor Wat, the largest religious
monument in the world.
I climb aboard Kim San’s motorbike, and we’re off. My heart races as we weave
in and out of the streets of Siem Reap, the boomtown launching point for millions of
yearly visitors to Angkor. In the darkness, the motorbike headlights reveal shadowy
forms of men and women bustling to set up shops and restaurants that will serve the
waking hordes of tourists.
It’s a seven-kilometer drive to the main ticket booth to Angkor Wat. Kim San
stops in front of a large, modern complex, built to move large crowds quickly through the
concession.
Climbing off the bike, I look around. “Kim San, you said this place would be
packed, but there’s hardly anyone here.”
Kim San smiles. “Many people wait to come until just before the sun rises. They
are lazy. I have guided journalists before. I know you want to have the best view, and
that is why I brought you here early. You will see. Believe me. Here, you must take a
flashlight or you will trip and fall. You must purchase your ticket at the booth,” says Kim
San, ”I will bring water. Follow me.”
Looming in the distance, I sense the presence of Angkor Wat, though it lies
nearly 2 kilometers away. Designated in 1993 as a United Nations Educational,
Scientific, and Cultural Organization (UNESCO) World Heritage Site, Cambodia’s
Angkor Wat temple was also a finalist in the New Seven Wonders of the World
competition in 2007. It is the best-preserved structure in the complex of over 1000
temples known collectively as Angkor, the Sanskrit word for city. Angkor flourished
between the 9th and 15th centuries A.D. as the seat of the Khmer empire, which ruled
over parts of present-day Laos, Thailand, Vietnam, Myanmar, and Malaysia. It was the
largest preindustrial metropolis in the world, with a population of nearly one million and
an urban footprint roughly the size of modern Los Angeles. Since its founding in the 12th
century, the temple complex of Angkor Wat has remained an active religious center, first
dedicated to the Hindu god Vishnu, then re-dedicated to Theravada Buddhist use in the
14th or 15th century. It is a source of great national pride and has been depicted on every
version of the Cambodian flag since 1863.
97
Kim San leads me to a ticket window. At this hour, there are more employees
and guards lingering about than tourists. I pay my $60 US fee for a week’s entrance to
the park and am taken to a side room to have my photo taken for the pass. While waiting
I do some quick mental calculations. In my background research, I read that there were
nearly three million yearly visitors to Angkor. That’s $180 million US—a huge revenue
source for a country with a Gross Domestic Product of only about $10 billion US.
“Tickets are expensive, aren’t they?” I comment to Kim San as we make our way
back to his bike for the remaining 2-kilometer ride. “Angkor Wat brings in a huge amount
of money to Cambodia.”
“I guess so,” he responds. “Cambodians get to enter for free, which is good, but
no one really knows exactly where the money goes that is collected from foreigners. In
1999 the government gave a 10-year lease to a private company called Sokimex to
handle all of the ticket sales in Angkor. A man named Sok Kong owns Sokimex, and he
is a personal friend and creditor to Prime Minister Hun Sen and his family. Sokimex is
supposed to give $10 million US per year to Aspara, the government agency that
oversees and manages the archaeological park. People think that most of that money
actually ends up in the hands of corrupt government officials, because hardly any of it is
spent to conserve the sites in the park.”
“Is Angkor falling into disrepair?”
“Yes,” agrees Kim San, “three million pairs of hands and feet brushing up against
the sandstone bricks of the temples does a lot of damage, not to mention looting and
vandalism, all of the waste produced, and the water used. Overuse of water destroyed
the original city of Angkor, and now overuse is undermining the temples’ sand
foundation—the ground is literally sinking.”
As we speed towards Angkor Wat, I realize I have a problem. The magazine
dispatched me on this assignment to cover Angkor as an ecotourism site—to describe
how tourism has helped revive Cambodia’s ailing economy and preserve the local
culture and environment. This information about ticket sales, temple destruction, and
pollution seems to go against the ecotourism focus of my story.
Kim San stops along the long moat we’ll have to cross to enter the main temple
complex. As we walk, Kim San continues his commentary. “Most Cambodians are happy
with the tourism, though, Joseph. Even the anchovy paste sellers in Siem Reap are
making money. We are safe—the Khmer Rouge is gone—so most Cambodians feel that
letting Sok Kong, Hun Sen, and their cronies keep the money is a small price to pay.”
From my research, I know that Khmer Rouge is the name given to Cambodia’s
ruling party between 1975 and 1979. When the Khmer Rouge came to power in 1975,
they declared that year the Year Zero. All Cambodian history and culture prior to Year
Zero was to be destroyed and replaced by the new revolutionary culture, starting from
scratch. Foreigners weren’t allowed in the country; essentially, Cambodia was cut off
from the rest of the world until 1992, when the United Nations began its peacekeeping
mission.
When we reach the top of the tower, Kim San instructs me to find a place to sit
comfortably. There’s nothing to do now but wait for the sun to rise and reveal the view. In
the stillness, I slowly become conscious of the sound of water buffalo moving through
the waters of the moat and muffled chants of nearby monks. Over 100,000 people live
within the boundaries of the archaeological park, making Angkor a living, breathing
model of Cambodia’s cultural heritage.
At last, the dawn breaks, the sun bathes the temple towers in a golden light, and
thousands of intricate sculptures, carvings, and stone reliefs emerge from the shadows.
98
I’m shocked out of my reverie by a group of tourists huffing and puffing up the steps
behind us and fussing to their guide that they’re late and they’ve missed the sunrise.
“I’m sorry, Madame. I’m afraid the sun waits for no one, not even someone who
forgot her camera in the room.”
I laugh to myself at the clever retort. I’d been impressed to learn through my
background research that official guides like Kim San are certified by the National
Tourism Agency of Cambodia. They all speak exceptional English, hold university
degrees, and are steeped in the culture and history of the area. All this work earns them
a daily rate of between $10 and $20 US per day—a king’s ransom compared to the
average Siem Reap salary of approximately $40 US per month. Even off-duty
policemen, paid approximately $30 US per month, hang out around the temples, ready
to guide those who decided against hiring someone in town.
The arrivals are increasing with the light. Busses are lining up on the other side
of the moat and the souvenir sellers are beginning their steady sales pitch.
“You were right, Kim San, it’s getting crowded around here. Shall we explore?”
A group of monks walk past, chanting and holding flowers, incense, and candles.
The cameras click away. Kim San explains, “They are celebrating Magha Puja, a day of
veneration for Buddha and his teachings. The ceremony is supposed to take place at
night, but they perform it during the day, too, when the tourists are here. When they are
done they will accept tips to have their pictures taken with the tourists. They use the
money to fund a school; the monks teach local people the old crafts, and then people
make things to sell here and in Siem Reap.”
I turn and notice three little girls, bracelets and bamboo flutes in hand, standing in
the middle of a group of shouting tourists.
“Canada! What’s the capitol of Canada?”
“Ottawa!” responds one girl eagerly. “Ottawa in Ontario. Canada have 10
provinces. Ontario, Quebec, Nova Scotia, New Brunswick, Manitoba, British
Columbia…” The child goes on to rattle off the rest of the provinces, plus their capitals
and relative populations.
The crowd loves it. Video cameras whirr away, recording the scene.
“These kids are going to be on YouTube next week, aren’t they?” I quip.
Kim San smiles. “They already are. They are earning money to pay their
teachers, probably. The Khmer Rouge are gone, but we still have a big enemy in
Cambodia: corruption. It is everywhere. Teachers charge children to enter the
classroom, and even white-haired old women must pay off the army or police for the
right to beg in the temples. We pay under the table for everything—birth certificates,
travel visas, fair rulings from judges, everything. Everyone needs the money and
everyone pays.”
A little girl is tugging at my shirttail. “Handsome mister, where you from?”
“America,” I respond.
“America, very good country. Capitol Washington, D.C. You buy flutes for your
children? 2 flutes 2000 riels.”
“I’ll buy your flutes if you answer some questions for me,” I bargain. “Tell me, do
you go to school?”
“No. My brothers go to school. I earn money so they go to school.”
“Why do your brothers go to school? What do they want to do when they grow
up?”
“My brothers want to have a hotel. Make lots of money. They don’t want to work
on farm. Too hard work. No money. Now you buy flutes?”
99
“Yes, now I’ll buy your flutes.” Digging deep in my pockets for the 2000 riels, I
glance at Kim San, who, with his university degree, observes these interactions with
detached amusement. I look back at the determined face of this little salesgirl, who, at 8
or 9-years-old, probably knows more geography than I do. I hand her the 2000 riels and
turn around to look at Angkor Wat. With the sun rising behind it, it glows like a beacon of
hope and casts a wide shadow below. At that moment, I know what the title of my article
will be—Angkor Wat: A Monumental Dilemma.
-----------------------------------QUESTION 1:
What is the “dilemma” that you think Joseph is writing about? What perspectives
need to be taken into account in order to arrive at a solution to this dilemma?
QUESTION 2:
Donald O’Reilly, archeological advocate, has said, “We see tourism as the best
way to preserve Cambodia’s rich archaeological heritage.” In contrast, John Stubbs of
the World Monuments Fund has said, “Tourism is already out of control, and unless the
Cambodian government takes some pretty radical action to reign it in now much of
Angkor’s magic and heritage could be lost forever.” Given your knowledge of the forces
currently affecting our world (historical, economic, political, social, environmental, etc.),
do you think it is possible for Cambodia to preserve its cultural heritage through tourism?
Please support your opinion with evidence of your knowledge of the forces affecting this
issue.
100
APPENDIX E
Draft Case Response Scoring Rubric
January 2009
101
102
103
APPENDIX F
Draft Case Response Scoring Rubric, “The Case of Hoodia”
March 2009
104
105
106
APPENDIX G
Draft Case Response Scoring Rubric, “A Monumental Dilemma”
April 2009
107
108
109
APPENDIX H
Case Response Assessment Pilot Faculty Survey
Spring 2009
110
Case Response Assessment Pilot Faculty Survey, Spring 2009
Pilot Courses Taught
ENG 2012, Section A, Approaches to Literature
BOT 1010, Introductory Botany
WHO 2001, World Civilizations
ENG 2012, Section B, Approaches to Literature
COM 3461, Intercultural Communication
1. In your estimation, what percent of your total class time did you spend explicitly
addressing the following:
ENG 2012, Section A: Perspective Consciousness – 5, Knowledge of Global Dynamics – 5
BOT 1010: Perspective Consciousness – 10, Knowledge of Global Dynamics – 10
WHO 2001: Perspective Consciousness – 20, Knowledge of Global Dynamics – 70
ENG 2012, Section B: Perspective Consciousness – 20, Knowledge of Global Dynamics – 10
COM 3461: Perspective Consciousness – 40, Knowledge of Global Dynamics – 25
2. Using your knowledge of both the "The Case of Hoodia" and perspective consciousness,
do you believe there is enough content presented in the case to enable the student to answer
this question?
ENG 2012, Section A: Yes - Students responded to role playing aspect of the prompt. The facts
of the case were less important to the response than the POV of the reporter herself. Internally,
the case depicted the cognitive process it intended to measure.
BOT 1010: Yes
WHO 2001: Yes - Students can see many different perspectives from the reading -- San,
corporations, government agencies, NGO aiding the San.
ENG 2012, Section B: Yes. The information is certainly available, but I even find myself
struggling with the complexity and unfamiliarity of the "characters" involved. By this I mean that
there are numerous entities involved in this story-- a group of people called the San, Pharmedics,
Hoodia, WIMSA, BioWatch, CSIR, etc. One sentence on p. 5 says, "WIMSA had recently
informed Pharmedics and the CSIR of their decision. . ." There's a communication or information
absorption issue that arises here. At least according to the writing training I've received, people
can only generally comprehend so many new "characters" (i.e. actors in a narrative) at once and
there's a hierarchy of comprehensibility. People are the most comprehensible characters, new
acronyms are a lot more difficult for readers to comprehend, assimilate into their own vocabulary,
and then redeploy. So while the information is definitely available, the students' ability to
comprehend and then use the different actors to answer this question is an concern. The students
may have only been able to talk about the characters they had "met" before in other contexts-drug companies and maybe indigenous people like the San-- while they simply might not have
been able to "upload" and "download" all the other actors into an answer for this question.
COM 3461: Yes. The story discussed the issue of the San being nomadic--covering several
different countries, Botswana, Namibia and South Africa. So not only do you have the
perspective of the San, but each of the governments. The story discusses the issue of Biowatch,
CSIR, WIMSA. A bit more detail about these organizations would have made it more obvious to
the student that these agencies have a stake in the outcome. I would say this if this case was the
first case at the start of the semester. If this case is used at the end of the semester, the students
should be able to pick up on the interest of these agencies. Obviously, you have the pharmedics
and the individuals in the story--Angela.
111
3. Using your knowledge of perspective consciousness, do you believe this question measures
the student's ability to assemble a multi-perspective analysis of a problem?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes I think that this question is successful in eliciting student answers at different
levels which accurately represent the understanding that students have of multiple perspectives.
ENG 2012, Section B: Yes. Again, the information is there, but with all the different and
unfamiliar actors, some of which are acronyms, in the story, I think it's difficult to attach the
information about perspectives to these unfamiliar characters. In addition, because there are
multiple actors in this very short story, it's a bit hard for the story to give thorough evidence of
each actor's perspective. There probably needs to be some investigation into how we can present
multiple perspectives in a story without overwhelming the student and ourselves with too many
perspectives.
COM 3461: Yes. I feel the question is well written so as not to be too obvious. At the start of the
semester, I would not expect that too many students would pick up on wanting to interview
multiple stakeholders, but by the end of the semester they should be able to.
4. Using your knowledge of perspective consciousness, do you believe the rubric is a valid
measure of the student's ability to assemble a multi-perspective analysis? In other words, do
the levels of the rubric enable the scorer to distinguish between levels of achievement of this
skill?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. Quite successfully. The variance in student answers shows this well.
ENG 2012, Section B: Yes.
COM 3461: Yes. Although catagory 4 (solutions) the question does not ask them to provide
solutions.
5. Using your knowledge of both "The Case of Hoodia" and global dynamics, do you believe
there is enough content presented in the case to enable the student to answer this question?
ENG 2012, Section A: No - The tag line, 'Comment on Dutfield's reasons. . . ' is a distraction
from the global dynamics variable itself. The prompt asks for an analysis of a perspective on
another variable; thus 'global dynamics' is de-centered. Dutfield's response to the issues is thrust
ahead of the student's evaluation of the issues.
BOT 1010: Yes
WHO 2001: No - The problem here is the quote from Dutfield. Students have to know a lot
already about the different parts of the world in order to make sense of this.
ENG 2012, Section B: No. For better or for worse, this question will mostly demonstrate that the
majority of our students have no understanding of the histories of the different continents and
countries mentioned in the question. But I think this just ends up making our students (and us)
look bad for the things they didn't learn in high school. I think we'll have to give a good deal of
attention to delivering historical background in the cases themselves because we can't assume the
kids have it.
COM 3461: No. I do not know how you could word it so as not to get off point, but I do not think
the students understand what you were getting at.
112
6. Using your knowledge of global dynamics, do you believe this question measures the
student's ability to analyze the forces influencing current global dynamics?
ENG 2012, Section A: No
BOT 1010: Yes
WHO 2001: No. Some of the students need more information than they have in order to answer
the question successfully.
ENG 2012, Section B: No. For same reasons as mentioned above.
COM 3461: No. Same as above.
7. Using your knowledge of global dynamics, do you believe the rubric is a valid measure of
the student's ability to analyze the forces influencing global dynamics? In other words, do
the levels of the rubric enable the scorer to distinguish between levels of achievement of this
skill?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: No. Many of the student answers that were given low scores reflected a lack of
knowledge about the different parts of the world. The Dutfield quote should be replaced.
ENG 2012, Section B: Yes
COM 3461: Yes
8. Using your knowledge of both "A Monumental Dilemma" and perspective consciousness,
do you believe there is enough content presented in the case to enable the student to answer
this question?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. The story is very good in presenting a wide variety of perspectives on tourism
to sacred sites in Cambodia.
ENG 2012, Section B: Yes. There's certainly plenty of information here and it's easy to
understand because it involves people, companies, governments, and places, but not acronyms.
I'm still a little confused about why ecotourism is at issue here as a perspective rather than just
plain old tourism, but I'll leave that to wiser heads than mine.
COM 3461: Yes. The story weaves in many perspectives at various points as the story unfolds.
9. Using your knowledge of perspective consciousness, do you believe this question measures
the student's ability to assemble a multi-perspective analysis of a problem?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. The students had enough information about the perspectives of various social
groups with regard to this tourism and its impact on the temple.
ENG 2012, Section B: Yes
COM 3461: Yes
113
10. Using your knowledge of perspective consciousness, do you believe the rubric is a valid
measure of the student's ability to assemble a multi-perspective analysis? In other words, do
the levels of the rubric enable the scorer to distinguish between levels of achievement of this
skill? ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. The rubric differentiated well between different levels of understanding
perspective consciousness.
ENG 2012, Section B: Yes
COM 3461: Yes. The question for this story mentions the idea of having a solution to the
dilemma.
11. Using your knowledge of both "A Monumental Dilemma" and global dynamics, do you
believe there is enough content presented in the case to enable the student to answer this
question?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. Students can see the problem from several sides. It does take imagination for
students to figure out alternatives to the current practices.
ENG 2012, Section B: Yes. No problem here.
COM 3461: Yes. The fact that the question says (historical, economic, political, social,
environmental) is a definate clue to my students waht you were getting at as these terms were part
of my lectures.
12. Using your knowledge of global dynamics, do you believe this question measures the
student's ability to analyze the forces influencing global dynamics?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. Students can clearly see the interactions of people from several countries in
Cambodian tourism. They can see how the world is changing over time.
ENG 2012, Section B: Yes
COM 3461: Yes
13. Using your knowledge of global dynamics, do you believe the rubric is a valid measure
of the student's ability to analyze the forces influencing global dynamics? In other words, do
the levels of the rubric enable the scorer to distinguish between levels of achievement of this
skill?
ENG 2012, Section A: Yes
BOT 1010: Yes
WHO 2001: Yes. The students have to bring a certain analytical and creative talent to this
question, but this rubric accurately differentiates between different levels of understanding.
ENG 2012, Section B:
COM 3461: Yes
114
APPENDIX I
Outside Expert Judge Opinions of Case Response Assessment
115
Expert Judge #1
Dr. Kenneth Tye, College of Educational Studies, Chapman University
-----Original Message----From: Tye, Kenneth [mailto:ktye@chapman.edu]
Sent: Wednesday, January 14, 2009 4:31 PM
To: Hilary Landorf
Subject: RE: global education case study assessment
Hillary,
I cannot open the case study, except in what looks like Vietnamese. I did view the
rubrics and think they are superb. Resend me the case study. If it is as good as the
rubrics, you should get a lot of mileage out of a separate publication, involving yourself
and your team members.
There may be similar work in the field, but I am not aware of it. If there is, this adds to
the field. If not, it is a breakthrough. I have had a number of inquiries of the years about
assessment, but nothing with potential such as this.
I do recommend that you contact Ann Baker at the National Peace Corps Assn. to see if
there is similar work. globaled@rpcv.org If there is, please let me know.
Ken
-----Original Message----From: Tye, Kenneth [mailto:ktye@chapman.edu]
Sent: Wednesday, January 14, 2009 5:17 PM
To: Hilary Landorf
Subject: RE: global education case study assessment
Hillary,
The case is "interesting." The degree to which it is an appropriate assessment tool in
conjunction with the rubrics depends in great measure, upon what is taught and how it is
taught. To put together a "package" the teaching issue needs to be addressed (I assume
this represents assessment of some curriculum or "curriculum infusion.") That would be
interesting to me. Also, I would be interested in knowing the results of a field test.
Another question is age appropriateness. This is pretty sophisticated stuff. Is this
directed at AP students, all 12th graders, or some other target population?
Finally, after field testing this package, do you have plans for developing a second case
with additional questions, using the same rubrics? A whole lot of other questions would
probably come to mind as other people read this material. I hope you are sending this
out to others, also.
Ken
116
Expert Judge #2
Dr. Ethan Lowenstein, Department of Teacher Education, Eastern Michigan
University
-----Original Message----From: Ethan Lowenstein [mailto:elowenste@emich.edu]
Sent: Tuesday, January 13, 2009 1:11 PM
To: Hilary Landorf
Subject: Re: attachment for global learning
Hi Hilary,
I like the case study. In principle I like the rubric categories for goal#1 and goal#2.
However, I would be wary of using them, without first testing the validity of the
instrument through a data-driven approach. This approach would involve asking 10-15
people who are not in the evaluation to respond to the case study, scoring the responses
using your rubric and then developing "anchor responses" that can be attached to the
rubric. For example, when someone scores in the acceptable category, what might a
"typical response" look like. This will increase the reliability of scoring--i.e. scorers can
have "anchors" to refer to while scoring responses. It will also increase validity by
increasing the plausibility that the questions and case study are measuring what you are
intending to measure and not something else. Another way to increase
construct/external validity is to have a short narrative that accompanies the rubric that
explains how the categories for scoring you've developed are grounded in the literature.
I hope that this feedback is helpful.
With warm regards,
Ethan
Ethan Lowenstein, Ph.D.
Associate Professor of Curriculum and Instruction
Department of Teacher Education
Eastern Michigan University
(734) 487-7120, ext. 2480
117
Expert Judge #3
Dr. William Gaudelli, Teachers College, Columbia University
-----Original Message----From: Gaudelli, William [mailto:gaudelli@exchange.tc.columbia.edu]
Sent: Tuesday, January 20, 2009 10:14 PM
To: Andrew Gomez
Subject: RE: global learning assessment
Greetings Hilary,
Thanks for sharing this work with me. I have a few responses to what I generally find to
be an interesting approach to assessment.
-The focus on global dynamics and systems seems appropriate, though not exclusive to
the situation presented by the Hoodia case. Can there be a way to sharpen this so that it
is solely focused on these dimensions? I certainly see human choice as part of this
scenario as well but not addressed in the rubric.
-I'm not sure that the student is asked to use knowledge in the case of this scenario as I
could imagine a student with little or no knowledge of pharmaceuticals responding
appropriately to this question...isn't this as much a matter of ethics?
-I can't comment on the appropriateness of the domains themselves though I think this
aspect of the assessment needs attention as it needs a fuller exploration of what
'perspective consciousness' looks/reads/sounds like in assessment terminology that is
somewhat measurable.
Thanks for passing this on and good luck!
Bill
_________________________________________
William Gaudelli
Associate Professor of Social Studies and Education
Project Leader for Teachers College to the Global Education Leadership Foundation
Teachers College, Columbia University
Zankel 420B
Box 80
525 W. 120th Street
New York, NY 10027-6696
(212) 678-3150
(212) 678-4118 FAX
gaudelli@tc.edu
118
APPENDIX J
Final Draft Rubric, “The Problem with Hoodia”
119
120
121
122
APPENDIX K
Final Draft Rubric, “A Monumental Dilemma”
123
124
125
APPENDIX L
Sample Faculty Request Email
126
From: Stephanie Doscher
Sent: Tuesday, December 28, 2010 3:52 PM
To: Peter J. Hargitai; James M. Sutton
Subject: Global Learning Assessment Study
Dear Professor Hargitai,
I am writing to you upon the recommendation of Dr. James Sutton.
FIU’s QEP, Global Learning for Global Citizenship, has three student learning outcomes.
These are:
Global Awareness: Students will be able to demonstrate knowledge of the
interrelatedness of local, global, international, and intercultural issues, trends, and
systems.
Global Perspective: Students will be able to conduct a multi-perspective analysis of
local, global, international, and intercultural problems.
Global Engagement: Students will be able to demonstrate willingness to engage in
local, global, international, and intercultural problem solving.
FIU is using two instruments as pre/post-assessments of the QEP’s global learning
SLOs. One of the instruments, the case response assessment, will be used to assess
the first two SLOs and was developed in-house. In the spring, our office will be
conducting a validity study of this assessment.
The purpose of the spring study is to determine the extent to which quantitative data
supports the validity and reliability of assessment scores. Field tests have been very
positive — this study will be a quasi-experimental comparison of scores from students
enrolled in global learning and non-global learning courses.
I am writing to request your assistance in the delivery of the case response pre/postassessment in your three sections of Approaches to Literature. Since the assessment is
a writing exercise, you may use the assessment artifact for students’ Gordon Rule
requirement. Our office will score the assessments according to the accompanying
rubrics — the only investment participating you would need to make is the time
necessary to deliver the assessment in class (two 45-minute sessions). We would like to
deliver the pre-assessment within the first two weeks of class. The post assessment can
be delivered within the final two weeks of class.
I have attached the pre/post instruments and the scoring rubric for your examination.
Thank you very much for your consideration. I’m happy to discuss this with you further,
either over the phone or via email, at your convenience.
Very best,
Stephanie Doscher
Associate Director
11200 S.W. 8th Street, University Park, GL 470
Tel: 305-348-4146
Fax: 305-348-1008
127
APPENDIX M
Case Response Assessment Pretest
128
Case Response Pre-Assessment
Multinational pharmaceutical firms commonly explore, extract, develop, and distribute
drugs made from traditional medicinal plants used by indigenous people. Please read
“The Problem with Hoodia,” a fictional account of a real-life healthcare issue that
involves multiple stakeholders.
After completing the reading, answer questions the demographic questions and
questions 1 and 2.
**ANSWERS MUST BE AT MINIMUM 150 WORDS IN LENGTH.**
"The Problem with Hoodia"
“Hello, I'm Roger. So, what takes you to Africa? Work or pleasure?”
Angela Bingham turned to her seatmate and tried to muster a genuine smile. Although
she was proud of her work, Angela thought it odd that a stranger would try to start a
conversation by asking such a personal question. Nevertheless, she was stuck sitting
next to this man for the remainder of the 11-hour flight to Cape Town, so she decided to
open up and try to be friendly. A little small talk might even make the time pass more
quickly...
“I'm Angela," she replied, shaking Roger's hand. "I'm going to Africa for work. My
company, Pharmedics, is involved in pharmaceutical drug development. The medicines
we work on are used to treat asthma, Parkinson’s disease, Alzheimer’s disease,
AIDS…you name it."
"Sounds interesting. So, are you going to Africa to find a cure for a disease?" Roger
asked.
"Well, sort of...my latest project involves an extract from a plant called Hoodia gordonii. It
grows in the wild all over southern Africa and has been used by the San, or the
Bushmen of the Kalahari, for thousands of years. The San are the first human
inhabitants of Africa. They take Hoodia to diminish hunger and thirst on long hunting and
gathering expeditions and during times of drought. Hoodia's extract, called P57, may
turn out to be an anti-obesity wonder drug.”
“That sounds like very good work. Obesity is a terrible health problem, an epidemic,
especially in the States. Are you a scientist?”
“No, I’m a manager. Actually, Pharmedics is a virtual company—there are very few of us
who are employed directly by the company itself. I work with outsourced field
researchers, clinicians, and lab scientists all over the world. It's a British company, but
I'm based in New York. I develop a communications strategy between the stakeholders
and I coordinate feasibility studies for research and production. Pharmedics only works
on initial isolation of extracts, though. We leave the actual drug development and
commercialization up to the big boys."
129
“The ‘big boys’?”
“Yeah, Pfizer, Unilever—big multinational pharmaceutical firms. They’ve got the money
and the power to create the drugs and push them through the American Food and Drug
Administration and such. But tell me, what do you do, um...I can’t believe I forgot your
name already…”
Angela’s seatmate smiled graciously. “Roger. Don’t worry about it—I’m an artist, a
sculptor. I’m bringing a commissioned work to Cape Town to be placed in front of the
headquarters of a shipbuilding company. I work with metal. The pieces of the sculpture
are all down in the baggage compartment. I’m going to South Africa to put them
together.”
“Well, well,” beamed Angela, “that’s basically what I’m going to Cape Town to do. I’m no
artist, though. I'm supposed to figure out how a whole bunch of puzzle pieces fit
together, even though I have no idea what the end product is supposed to look like.”
Temporarily saved by the impending arrival of the dinner cart from having to explain
further, Angela leaned back in her seat and decided to close her eyes for a moment. She
recalled the conversation she’d had the previous week with her company’s president,
David Campbell, when she was initially dispatched on this mission.
-----------------------------“Angela, I want you to know that you’ve done incredible work coordinating the clinical
trials of P57. It has enormous commercial potential for the development of weight loss
drugs and Pfizer is very interested in taking it to the next level. But Angela, we've hit a
major roadblock. We can't sell P57 to Pfizer until we work things out with the San. They
are claiming that they have rights to the extract because they originally discovered its
medicinal qualities. I’ve got their lawyer, reporters from the International Herald Tribune,
a bunch of human rights organizations, and the governments of Namibia, Botswana, and
South Africa breathing down my neck…it’s an unbelievable mess. I didn’t even know that
Bushmen existed anymore. I need you to go over to South Africa, meet with the different
groups, and make everybody happy.”
Angela’s heart pounded. She was used to bringing diverse people together from multiple
countries to work as a drug development team, but this sounded much more
complicated than what she usually did. “David, I’m not sure I understand what you want
me to do. Why do we have a problem with the San? They don’t have a license to P57,
we do.”
Taking off his glasses, David Campbell stood and began pacing the room. “We
purchased the license to develop an extract from the initial patent holder for Hoodia
gordonii plant, the Council for Scientific and Industrial Research (CSIR). Although it is a
government institution sponsored by Namibia, Botswana, and South Africa, the CSIR did
not consult with the San, who live in those countries, before applying for the patent.
Even if they had approached the San, they may not have cooperated because they don't
trust the government. The San’s nomadic way of life has been seriously endangered by
130
development in southern Africa. The San are poverty-stricken and they lack education
and access to information, so they have little power to negotiate or profit from developing
their knowledge of medicinal plants such as Hoodia gordonii…anyway, a South African
non-governmental organization called BioWatch found out about the CSIR agreement
with Pharmedics and leaked it to the San and to the press. That's how this whole
problem started.”
Angela was starting to catch on. “So do the San believe they are the true owners of
Hoodia? Do they want some sort of monetary compensation for their knowledge of
Hoodia?”
“I wish it were that simple. To tell you the truth, the San find the very idea that anyone
should pay them for their knowledge morally abhorrent. The San value knowledge as a
collective resource. What’s more, the whole patent process makes little sense to them.
They don’t see how life—even plant life—can be ‘owned.’”
Sitting back down at his desk, Campbell went on to explain how matters were made
even more complicated by the fact that the San are not a single community, but a group
of multiple far-flung nomadic communities that travel throughout South Africa, Namibia,
and Botswana. Although the San decided not to pursue their ‘no patents on life’ beliefs in
court, they did want to negotiate a benefits-sharing agreement, with Hoodia royalties
being used to alleviate poverty and sustain endangered aspects of San culture. The
distribution of such benefits was problematic. Even if an agreement could be reached
between the CSIR, Pharmedics, and the San, how could a system be created to fairly
compensate multiple nomadic San groups across three countries?
Angela was overwhelmed but determined. “David, I can’t believe what a puzzle you’ve
placed in front of me. There are so many groups involved...I’ll go to Cape Town, but I
can’t promise I’ll make everyone happy. I'll try to help everyone see how complicated this
is and work out some sort of compromise." David sighed. "That's what we need, Angela,
a compromise. Just remember, P57 could change a lot of lives for the better, but if we
don't put the pieces together no one will benefit."
-----------------------------“Well good morning, sleepyhead, just in time for breakfast! You passed out without even
taking a bite of dinner. I didn’t want to wake you—I hope that’s o.k. We’ve only got a few
more hours before landing.”
“Oh yes, of course. I didn't intend to fall asleep...Roger, can I ask you something? You
said you are going to Cape Town to put the pieces of your metal sculpture together. How
exactly are you going to do that?”
“Well, you choose your method depending on the types of metals you are working with.
If the metals are the same, you can weld them together. It takes a lot of heat and it’s
dangerous, but if you are careful the joining will last a long time. If the metals are
different, it’s very difficult to force them together with welding. You generally have to use
some sort of fastener like bolts or rivets. You pick the process to match the parts.”
131
Angela took a moment to consider this. "'Pick the process to match the parts.' Maybe I
should think of my job in South Africa more as a sculpture than as a puzzle. Thanks,
Roger. You’ve helped me a lot.”
Angela leaned back in her seat. She was grateful Roger had asked her what she did for
a living; moreover, she was glad she’d chosen to open up to him. She smiled to herself,
and this time it was genuine.
132
Case Response Assessment
DEMOGRAPHIC QUESTIONS
Complete all items for your responses to count.
1. Panther ID#:
2. Gender (circle one):
M
F
3. My status at the college/university in which I am enrolled (circle one):
Freshman
Sophomore
Junior
Senior
4. Select the ethnic identity that best describes you (circle one):
African-American
Hispanic/Latino
American Indian/Alaskan Native
Multiple ethnicities
Asian/Pacific Islander
Unknown/Other
European/White
5. How many languages do you speak fluently (circle one)?
One
Two
Three or more
6. What is the longest period of time you have spent abroad (circle one)?
None
Two weeks or less
More than two weeks
7. What is the primary reason you have travelled abroad (circle one)?
Academic
Service
Work
Residence
Tourism
8. Have you ever been a student in an International Baccalaureate (IB) or global
education magnet program (circle one)?
Yes
No
9. Have you ever taken a global learning course at FIU before (circle one)?
Yes
No
10. Have you ever taken this assessment before (circle one)?
Yes
No
134
Panther ID #________________________
QUESTION 1:
What is the problem in "The Problem with Hoodia"? Given what you know about
the world, what are the issues (environmental, economic, cultural, political, etc.)
influencing this problem?
**Answer must be at minimum 150 words in length.**
135
Panther ID #________________________
QUESTION 2:
What perspectives need to be taken account in order to find a solution to the
problem?
**Answer must be at minimum 150 words in length.**
136
APPENDIX N
Case Response Assessment Posttest
137
Case Response Post-Assessment
According to Dr. Martha Honey, Co-Director of the Center on Ecotourism & Sustainable
Development, ecotourism is "travel to fragile, pristine, and usually protected areas that
strives to be low impact and (usually) small scale. It helps educate the traveler; provides
funds for conservation; directly benefits the economic development and political
empowerment of local communities; and fosters respect for different cultures and for
human rights." Please read “A Monumental Dilemma,” a fictional account of a real-life
ecotourism issue that involves multiple stakeholders.
After completing the reading, answer questions 1 and 2.
**ANSWERS MUST BE AT MINIMUM 150 WORDS IN LENGTH.**
"A Monumental Dilemma"
It is 4:30 a.m. and as promised, my guide and driver, Kim San, is waiting for me at the
hotel entrance. We had met the previous day to work out a sightseeing schedule for the
week. He insisted that I begin my tour of the 7Angkor Archaeological Park by watching
the sun rise over Angkor Wat, the largest religious monument in the world.
I climb aboard Kim San’s motorbike, and we’re off. My heart races as we weave in and
out of the streets of Siem Reap, the boomtown launching point for millions of yearly
visitors to Angkor. In the darkness, the motorbike headlights reveal shadowy forms of
men and women bustling to set up shops and restaurants that will serve the waking
hordes of tourists.
It’s a seven-kilometer drive to the main ticket booth to Angkor Wat. Kim San stops in
front of a large, modern complex, built to move large crowds quickly through the
concession.
Climbing off the bike, I look around. “Kim San, you said this place would be packed, but
there’s hardly anyone here.”
Kim San smiles. “Many people wait to come until just before the sun rises. I have guided
journalists before. I know you want to have the best view, and that is why I brought you
here early. You will see, believe me. Here, you must take a flashlight or you will trip and
fall. You must purchase your ticket at the booth,” says Kim San. “I will bring water.
Follow me.”
Looming in the distance, I sense the presence of Angkor Wat, although it lies nearly 2
kilometers away. Designated in 1993 as a United Nations Educational, Scientific, and
Cultural Organization (UNESCO) World Heritage Site, Cambodia’s Angkor Wat temple
was also a finalist in the New Seven Wonders of the World competition in 2007. It is the
best-preserved structure in the complex of over 1000 temples known collectively as
Angkor, the Sanskrit word for city. Angkor flourished between the 9th and 15th centuries
A.D. as the seat of the Khmer empire, which ruled over parts of present-day Laos,
Thailand, Vietnam, Myanmar, and Malaysia. It was the largest preindustrial metropolis in
138
the world, with a population of nearly one million and an urban footprint roughly the size
of modern Los Angeles. Since its founding in the 12th century, the temple complex of
Angkor Wat has remained an active religious center, first dedicated to the Hindu god
Vishnu, then re-dedicated to Theravada Buddhist use in the 14th or 15th century. It is a
source of great national pride and has been depicted on every version of the Cambodian
flag since 1863.
Kim San leads me to a ticket window. At this hour, there are more employees and
guards lingering about than tourists. I pay my $60 US fee for a week’s entrance to the
park and am led to a side room to have my photo taken for the pass. While waiting I do
some quick mental calculations. In my background research, I read that there were
nearly three million yearly visitors to Angkor. That’s $180 million US—a huge revenue
source for a country with a Gross Domestic Product of only about $10 billion US.
“Tickets are expensive, aren’t they?” I comment to Kim San as we make our way back to
his bike for the remaining 2-kilometer ride. “Angkor Wat brings in a huge amount of
money to Cambodia."
“I guess so,” he responds. “Cambodians get to enter for free, which is good, but no one
really knows exactly where the money goes that is collected from foreigners. In 1999 the
government gave a 10-year lease to a private company called Sokimex to handle all of
the ticket sales in Angkor. A man named Sok Kong owns Sokimex, and he is a personal
friend and creditor to Prime Minister Hun Sen and his family. Sokimex is supposed to
give $10 million US per year to Aspara, the government agency that oversees and
manages the archaeological park. People think that most of that money actually ends up
in the hands of corrupt government officials, because hardly any of it is spent to
conserve the sites in the park.”
“Is Angkor falling into disrepair?”
“Yes,” says Kim San, “three million pairs of hands and feet brushing up against the
sandstone bricks of the temples does a lot of damage, not to mention looting and
vandalism, all of the waste produced, and the water used. Overuse of water destroyed
the original city of Angkor, and now overuse is undermining the temples’ sand
foundation—the ground is literally sinking.”
As we speed towards Angkor Wat, I realize I have a problem. The magazine dispatched
me on this assignment to cover Angkor as an ecotourism site—to describe how tourism
has helped revive Cambodia’s ailing economy and preserve the local culture and
environment. This information about ticket sales, temple destruction, and pollution
seems to go against the ecotourism focus of my story.
Kim San stops along the long moat we’ll have to cross to enter the main temple
complex. As we walk, Kim San continues his commentary. “Most Cambodians are happy
with the tourism. Even the anchovy paste sellers in Siem Reap are making money. We
are safe—the Khmer Rouge is gone—so most Cambodians feel that letting Sok Kong,
Hun Sen, and their cronies keep the money is a small price to pay for the improvement
of our safety, economic standing, and cultural recognition throughout the world.”
139
From my research, I know that Khmer Rouge is the name given to Cambodia’s ruling
party between 1975 and 1979. When the Khmer Rouge came to power in 1975, they
declared that year to be Year Zero. All Cambodian history and culture prior to Year Zero
was to be destroyed and replaced by the new revolutionary culture, starting from
scratch. Foreigners weren’t allowed in the country; essentially, Cambodia was cut off
from the rest of the world until 1992, when the United Nations began its peacekeeping
mission.
When we reach the top of the tower, Kim San instructs me to find a place to sit. There’s
nothing to do now but wait for the sun to rise and reveal the view. In the stillness, I slowly
become conscious of the sound of water buffalo moving through the waters of the moat
and muffled chants of nearby monks. Over 100,000 people live within the boundaries of
the archaeological park, making Angkor a living, breathing model of Cambodia’s cultural
heritage.
At last dawn breaks, the sun bathes the temple towers in a golden light, and thousands
of intricate sculptures, carvings, and stone reliefs emerge from the shadows. I’m startled
out of my reverie by a group of tourists huffing and puffing up the steps behind us and
fussing to their guide that they’re late and they’ve missed the sunrise.
“I’m sorry, Madame. I’m afraid the sun waits for no one, not even someone who forgot
her camera in the room.”
I laugh to myself at the clever retort. I’d been impressed to learn through my background
research that official guides like Kim San are certified by the National Tourism Agency of
Cambodia. They all speak exceptional English, hold university degrees, and are steeped
in the culture and history of the area. All this work earns them a daily rate of between
$10 and $20 US per day—a king’s ransom compared to the average Siem Reap salary
of approximately $40 US per month. Even off-duty policemen, paid approximately $30
US per month, hang out around the temples, ready to guide those who decide against
hiring someone in town.
The arrivals are increasing with the light. Busses are lining up on the other side of the
moat and the souvenir sellers are beginning their steady sales pitch.
“You were right, Kim San, it’s getting crowded around here. Shall we explore?”
A group of monks walk past, chanting and holding flowers, incense, and candles. The
cameras click away. Kim San explains, “They are celebrating Magha Puja, a day of
veneration for Buddha and his teachings. The ceremony traditionally takes place at
night. Nowadays the monks also perform the l ritual during the day to receive money
from tourists. This money is used to fund a school where the monks teach traditional arts
and crafts skills to the locals. The locals then make products to sell to the tourists. When
the monks have finished performing their ceremony they will accept tips to have their
pictures taken with the tourists."
I turn and notice three little girls, bracelets and bamboo flutes in hand, standing in the
middle of a group of shouting tourists.
140
“Canada! What’s the capitol of Canada?”
“Ottawa!” responds one girl eagerly. “Ottawa in Ontario. Canada have 10 provinces.
Ontario, Quebec, Nova Scotia, New Brunswick, Manitoba, British Columbia…” The child
goes on to rattle off the rest of the provinces, plus their capitals and relative populations.
The crowd loves it. Video cameras whirr away, recording the scene.
“These kids are going to be on YouTube next week, aren’t they?” I quip.
Kim San smiles. “They already are. They are most likely earning money to pay their
teachers. The Khmer Rouge is gone, but we still have a big enemy in Cambodia:
corruption. It is everywhere. Teachers charge children to enter the classroom, and even
white-haired old women must pay off the army and police for the right to beg in the
temples. We pay under the table for everything—birth certificates, travel visas, fair
rulings from judges, everything. Everyone needs the money and everyone pays.”
A little girl is tugging at my shirttail. “Handsome mister, where you from?”
“America,” I respond.
“America, very good country. Capitol Washington, D.C. You buy flutes for your children?
Two flutes 2000 riels.”
“I’ll buy your flutes if you answer some questions for me,” I bargain. “Tell me, do you go
to school?”
“No. My brothers go to school. I earn money so they go to school.”
“Why do your brothers go to school? What do they want to do when they grow up?”
“My brothers want to have a hotel. Make lots of money. They don’t want to work on farm.
Too hard work. No money. Now you buy flutes?”
“Yes, now I’ll buy your flutes.” Digging deep in my pockets for the 2000 riels, I glance at
Kim San, who, with his university degree, observes these interactions with detached
amusement. I look back at the determined face of this little salesgirl, who, at 8 or 9years-old, probably knows more geography than I do. I hand her the 2000 riels and turn
around to look at Angkor Wat. With the sun rising behind it, it glows like a beacon of
hope and casts a wide shadow below. At that moment, I know what the title of my article
will be—Angkor Wat: A Monumental Dilemma.
141
Case Response Assessment
DEMOGRAPHIC QUESTIONS
Complete all items for your responses to count.
1. Panther ID#:
2. Gender (circle one):
M
F
3. My status at the college/university in which I am enrolled (circle one):
Freshman
Sophomore
Junior
Senior
4. Select the ethnic identity that best describes you (circle one):
African-American
Hispanic/Latino
American Indian/Alaskan Native
Multiple ethnicities
Asian/Pacific Islander
Unknown/Other
European/White
5. How many languages do you speak fluently (circle one)?
One
Two
Three or more
6. What is the longest period of time you have spent abroad (circle one)?
None
Two weeks or less
More than two weeks
7. What is the primary reason you have travelled abroad (circle one)?
Academic
Service
Work
Residence
Tourism
8. Have you ever been a student in an International Baccalaureate (IB) or global
education magnet program (circle one)?
Yes
No
9. Have you ever taken a global learning course at FIU before (circle one)?
Yes
No
10. Have you ever taken this assessment before (circle one)?
Yes
No
142
Panther ID #________________________
QUESTION 1:
What is the problem in "A Monumental Dilemma”? Given what you know about the
world, what are the issues (environmental, economic, cultural, political, etc.)
influencing this problem?
**Answer must be at minimum 150 words in length.**
143
Panther ID #________________________
QUESTION 2:
What perspectives need to be taken account in order to find a solution to the
problem?
**Answer must be at minimum 150 words in length.**
144
145
APPENDIX O
Faculty Rater Instruction Letter
146
Dear Faculty Rater,
Thank you for agreeing to serve as a rater of the case response assessment for FIU’s
Quality Enhancement Plan (QEP), Global Learning for Global Citizenship. Your skilled
participation will advance the effectiveness of the QEP and is greatly appreciated.
You will be working with a team to score student responses in two four-hour sessions.
The first 60 minutes of the first session will consist of training. The training will apprise
you of your responsibilities as a scorer and will increase scoring agreement across team
members.
To prepare for the training, please review the information below and the documents
attached.
QEP Case Response Assessment -- Description
Activity
Students read a case narrative. After reading the narrative, students respond to two
open-ended questions concerning the case. Suggested response length = 150 words
minimum per question.
Outcomes Assessed
The QEP has three student learning outcomes (SLOs). Two of these are assessed with
the case response assessment*:
Global Awareness -- knowledge of the interrelatedness of local, global, international, and
intercultural issues, trends, and systems (Question 1)
Global Perspective -- ability to develop a multi-perspective analysis of local, global,
international, and intercultural problems (Question 2)
Scoring
Students will receive two final scores (0-4), one for each outcome/question. Scores are
determined on the basis of two holistic rubrics (one for each question). The final score is
an average of a minimum of two raters’ scores (if the two raters’ scores are discrepant
by more than 1, a third rater will read the response and the final score is an average of
the three raters’ scores).
*The third QEP SLO is Global Engagement -- willingness to engage in local, global,
international, and intercultural problem solving. Global Engagement is assessed via
another assessment, the Global Perspective Inventory (GPI) survey
(https://gpi.central.edu/).
Attachments
1. Case narratives and questions prompts for “The Problem with Hoodia” (read by
freshmen and transfers) and “A Monumental Dilemma” (read by graduating seniors).
2. Rubrics for both cases with explanation of scoring scale
If you have any questions prior to the first training and scoring session, please contact
Hilary Landorf, Director, Office of Global Learning Initiatives, landorfh@fiu.edu, or 305348-2410.
147