Program in Interdisciplinary Education Research (PIER)
Carnegie Mellon University
What is the Program in Interdisciplinary Educational Research?
The Program in Interdisciplinary Education Research (PIER) at Carnegie Mellon
University (CMU), initially funded by the Institute for Education Sciences in 2004, is
designed to train scientists whose rigorous research on learning conditions related to
curriculum, instruction, and assessment will improve academic outcomes for prekindergarten to postsecondary students. PIER is led by CMU faculty from several
academic departments (at present, this includes Psychology, Statistics, Philosophy,
Physics, Human-Computer Interaction, Robotics, Machine Learning, Economics, Public
Policy, Computer Science, Eberly Center for Teaching Excellence) and varies from year
to year. PIER fellows are recruited primarily from these participating departments, but
can come from any CMU doctoral program flexible enough to accommodate all PIER
requirements.
Under the most recent five-year grant from IES, starting in 2015, CMU will offer
approximately 18 doctoral students five-year fellowships (4 funded by the PIER grant
and 1 funded by the student’s department). All PIER fellowships will include the
following: $30,000 stipend, a small research and travel budget of up to $2,000 per year,
and $10,500 toward tuition, health insurance and other normal graduate fees (the
remainder of which are paid by the fellow’s home department). This level of support will
be continued for up to 5 years, as long as PIER trainees remain full-time students in
good standing in both their home department and in PIER . (Although financial support
from PIER is limited to U.S. Citizens or permanent residents, other qualified students
may also apply to the program. These students are considered “PIER Associates”, and
they participate fully in all aspects of PIER.)
PhD students in the PIER training program will participate in an interdisciplinary core
curriculum consisting of coursework in the learning sciences, instruction, assessment,
research methodology, and statistics. Fellows will also participate in field-based
experiences in which they conduct collaborative research with practitioners,
interdisciplinary research projects with other PIER fellows, and research
apprenticeships with PIER faculty. The PIER training program also includes an ongoing
interdisciplinary colloquium in the education sciences and a biweekly brown bag series
on education research.
The total projected costs of the training program are approximately $8 million. In
addition to the $3.7 million grant from the Institute of Education Sciences, CMU will be
contributing approximately $3.6 million to cost-share fellows' stipends/tuition during the
award period and approximately $765,000 in fellowship support to continue to train
fellows for up to two years after this grant concludes.
Why does this training grant exist?
There is a pressing need for high-quality educational research that can help the nation
to solve its complex and challenging educational problems. However, the existing
institutional structures and human capital flows in educational research fail to produce a
sufficient number of adequately trained researchers. Moreover, most of the new PhDs
1
graduating from strong disciplinary programs (e.g., in Computer Science, or Statistics,
or Psychology) with the skills necessary to conduct rigorous educational research are
unaware of the exciting challenges and opportunities in this area, and therefore
disinclined to apply their training to real-world educational problems. The need for more
researchers who view education as their main focus has been expressed from a variety
of perspectives:
One striking fact is that the complex world of education – unlike defense, health care, or
industrial production—does not rest on a strong research base. In no other field are personal
experience and ideology so frequently relied on to make policy choices, and in no other field is
the research base so inadequate and little used (National Research Council, 1999).
Our major mission in our field of education is…to make a science out of education…. We don’t
have…anywhere near the number of trained people to do the kind of large-scale research
efforts that we need in this country (Bruce Alberts, President, National Academy of Sciences,
11/18/02).
There are significant capacity issues within the education research community. Many schools
of education are not providing rigorous research training for doctoral students. While such
training is often provided elsewhere in universities, e.g., psychology and economics
departments, these training programs are seldom focused on topics in education, and
students are pointed towards other careers and research topics. As a result, there are simply
not enough well trained researchers in the pipeline to fulfill the need for high quality research,
statistics, and evaluation work of the type that the Institute is funding (Grover J. Whitehurst,
Director, Institute of Education Sciences, 3/13/03).
PIER addresses these problems by training a new breed of scientists who view
education as their primary research area. It provides a well-defined identity, vision, and
common core of training and experience for students aiming to become educational
researchers, while concurrently providing opportunities for faculty to engage in
collaborative teaching and research activities related to educational problems.
Who are the faculty who designed and run the PIER Training
Program?
Program Director. PIER’s Program Director, David Klahr, is the Walter van Dyke
Bingham Professor of Cognitive Development and Education Science in the Department
of Psychology, where he served as Head from 1983 to 1993. In addition to his PIER
administrative responsibilities, Klahr is the designer and teacher of the introductory
course in the PIER core course sequence. He has published over 125 journal articles,
and is the author, editor or co-editor of several volumes including Cognition &
Instruction (1976), and Cognition & Instruction, 25 years of Progress (2001) (with
Sharon Carver, the Associate Director of PIER). Klahr is a member of the National
Academy of Education.
Associate Director. Sharon Carver, PIER’s Associate Director, has a dual
appointment in Carnegie Mellon’s Psychology Department as the Director of the early
childhood laboratory school, the Children’s School, and as a teaching professor in
developmental psychology. Carver is the designer and teacher of the second PIER
core course, which focuses on development and evaluation of well-aligned goals,
instruction, and assessment based on learning sciences principles. She also serves as
the child development expert on Carnegie Mellon’s Institutional Review Board and
coordinator of all research conducted at the laboratory school.
2
Core Faculty. PIER’s core faculty members include the PIER Steering Committee as
well as the advisory committees of PIER fellows. In addition to the Director (Klahr) and
Associate Director (Carver), the Steering Committee consists of Vincent Aleven
(Assistant Professor, HCII), Dennis Epple (Professor, Economics), Anna Fisher
(Associate Professor, Psychology), Brian Junker (Professor, Statistics), Kenneth
Koedinger (Professor HCII / Psychology, Director of the Pittsburgh Science of Learning
Center), Marsha Lovett (Associate Director Eberly Center for Teaching Excellence /
Associate Research Professor, Psychology), Brian MacWhinney (Professor,
Psychology), Jack Mostow (Research Professor Language Technologies Institute, HCII,
Machine Learning, and Robotics), Amy Ogan (Assistant Professor, HCII), Robert
Siegler (Teresa Heinz Chair, Cognitive Psychology). In addition, the Steering
Committee includes a student member, who has full voting and participation rights
(except for confidential issues).
PIER Curriculum
Fit between individual departmental doctoral programs and PIER requirements. A
unique feature of our program is the way it integrates and coordinates a set of doctoral
training programs where individual faculty members and students had interest in
education sciences but whose doctoral programs – prior to PIER’s creation -- did not
have the infrastructure to support systematic training in education science via
coursework, funding, etc. Though faculty in the Departments of Psychology, HumanComputer Interaction, Philosophy, and Statistics had long been engaged in research
either directly funded by IES or by other agencies with similar goals of advancing
scientific research in education, there was insufficient critical mass for grounding
students in educational content and methodology. Because education science was not
the major thrust of any department’s overall research agenda, students considering
careers in education research were given only limited institutional, social, or intellectual
support to deviate from the traditional paths in their individual disciplines.
The vision that motivated the creation of PIER, and that has sustained it since its first
IES award in 2004, was the possibility and desirability of transforming the substantial
scientific and intellectual effort at Carnegie Mellon that was already relevant to
education science into an “engine” that prepared students for a research career in
education science, while still being grounded in the disciplinary strengths characteristic
of each department. We created a set of courses, procedures, administrative practices,
and social engineering to enrich and integrate the somewhat disparate elements into a
coherent program. By building a curriculum that augmented the highly selective
graduate recruitment process and rigorous training in each of the participating
departments, without reducing any of their already demanding graduate training
requirements, PIER has produced a coherent training program in education science that
extends graduate training by one year and yields a Certificate in Education Science. In
the following sections, we describe our integrated curriculum.
Curriculum Structure. The overall curriculum consists of a set of PIER-specific core
requirements listed in Table 1, augmented by a rigorous set of departmental Ph.D.
requirements that are specific to the departmental concentration chosen by individual
students
Curriculum Content Overview. Our program, described below, ensures that PIER
trainees have mastered both the basic knowledge and skills in their research domain
3
and the diverse methods for tackling the challenges of applying that knowledge to
evidence-based research, such that the findings will be relevant and useful for
educational practice and policy. Thereby, we produce scientists who can identify, within
the rich and challenging array of educational problems facing our country, a set of
questions that are not only fundamentally important, but also amenable to cutting edge
disciplinary research methods.
Each PIER student receives core disciplinary training via the rigorous graduate
curriculum in his or her home department. In addition, PIER adds requirements that are
roughly equivalent to one full extra year of graduate training, spread across the
graduate program. The Core Program Components shown in Table 1 are designed
both to offer a broad range of opportunities for students to explore topics and methods
in education science and to encourage depth of both content and methodology
development. This section includes detail regarding each of the program aspects
bolded in Table 1
Table 1: PIER Core Curriculum
Years 1-2: Engagement in education research as part of an augmented research apprenticeship, with an
interdisciplinary faculty committee, according to home department requirements. This includes:
Department coursework plus a 3.5-course PIER Core Sequence:
 Mini-course: Introduction to Education Sciences at CMU (Starts mid-semester Fall of 1st Year)
 Scientific Research in Education (Klahr: Spring of 1st Year)
• Educational Goals, Instruction, and Assessment (Carver: Fall of 2nd Year)
• Research Methods in the Learning Sciences (Coordinated by Koedinger, with modules by Lovett, Junker, and
several other faculty: Spring of 2nd Year)
Year 3: Field-Based Experience (FBE) in an Educational Context to develop understanding and appreciation
for the reality of classroom / school based systems.
Yrs 1-3: Additional Coursework / Individualized Training Experiences to meet PIER Core Competencies not
otherwise covered in the student’s prior experience or the Home Department’s Program (e.g., Experimental
Design, Statistics, Ed Technology, etc.)
Year 3: Integrated, Interdisciplinary Project (IIP) to experience immersion in a PIER research team for an
education research project culminating in a public presentation and possible conference presentation and/or
publication.
Yrs 4-5: Dissertation Research addressing practical questions in education.
Yrs 1-5: Integration into PIER Community.
 PIER Networking Events (e.g., welcome event, PIER professional networking discussions, etc.)
 Professional Development Seminar (Distributed through the AY):
 Speaker Series: Several colloquia each year, plus 1.5 days of Student-only Pro-Seminar with Speakers
 Bi-Weekly Brown Bag Series (the “EdBag”)
 Progress Evaluation via student’s self-assessment, PIER Steering Committee Evaluation Meeting, and
departmental graduate evaluation processes.
. PIER Curriculum Components. PIER requirements emphasize integrated theory,
research, and practice related to education via three core courses as well as a Mini
course, typically taken during the first and second years. The PIER-specific course
sequence detailed below provides the content and methodology foundation for
preparing “researchers who are able to prepare competitive proposals that address
relevant education topics and meet the methodological requirements specified” by IES.
The sequence (a) begins with a broad view of education research in the United States
(“Scientific Research in Education”, (b) then presents a cognitively guided approach to
educational design (“Educational Goals, Instruction, and Assessment”), followed by (c)
by an introduction to a range of process-tracing methodologies relevant to analysis of
4
learning and achievement (“Research Methods in the Learning Sciences”). Each course
has undergone revision based on student input and instructor assessments. The
courses also serve post-doctoral fellows, and upper-level undergraduates. Each course
is briefly described below.
“Mini-course: Introduction to Education Sciences at CMU” (Coordinated by PIER
Training Director David Klahr) The purpose of this mini-course is expose new PIER
Fellows to the rich array of activities, initiatives, people, and experiences at CMU that
are relevant to PIER’s training goals and requirements. The course will introduce varied
ways in which CMU researchers are using disciplinary theory and methodology to frame
education research questions and to provide solutions that are being implemented in
the real world of educational policy and practice.
“Scientific Research in Education” (Taught by PIER Training Director David Klahr)
Overview: This course is inspired by the NRC report of the same name (Shavelson &
Towne, 2002). Its primary goal is to introduce PIER students to the historical, political,
and scientific forces that led to the establishment of IES, and the set of issues and
challenges faced by educational researchers, practitioners, and policymakers.
“Educational Goals, Instruction, and Assessment” (Taught by PIER Associate
Director Sharon Carver).
Overview: This project-based course -- the second in the PIER series – is explicitly
designed to train future education researchers to identify underlying processes that are
predictive of student achievement in a domain (goals), develop interventions for
improving student learning (instruction), develop measurement tools for formative and
summative assessment, and plan controlled research studies to test the efficacy of
particular features of interventions.
“Research Methods in the Learning Sciences” (Co-Taught by PIER Steering
Committee Members and Colleagues)
Overview: This modular course on process-tracing methodologies, including cognitive
task analysis, video & protocol analysis, ethnography & design experiments, surveys &
interviews, psychometrics, and data mining is designed to supplement the graduate
experimental design and statistics courses required by students’ home department with
methods that are central to the education sciences, particularly those focusing on
understanding cognition & learning.
Field-Based Experience (FBE)
The Field-Based Experience (FBE) is one of the cornerstones of our training program
(the other one is the Integrative Interdisciplinary Project description follows).
The purpose of the Field-Based Experience is to provide students with an
understanding and appreciation, through first hand exposure, of the reality of classroom
and school based systems, and to experience the real-world management,
organizational, interpersonal, cultural, contextual and practical constraints that they will
face when they attempt to conduct research in field settings. In addition to providing
students with the necessary field experience, improving their abilities to understand
practitioners’ needs and to communicate directly with teachers and children, the FBE
serves as a launch pad for the field-inspired integrative interdisciplinary project (IIP) in
the third year and dissertation projects in the students' fourth and fifth year.
5
FBEs require approximately 10 hours per week on average for one semester (i.e.,
essentially equivalent to a one-semester course experience). PIER faculty help
students with site suggestions and access issues for FBEs, which may involve interning
with a teacher one day per week for a semester, designing and teaching a weekly
enrichment course in a local school, participating in the field component of an ongoing
research intervention, collaborating with a teacher to track the adoption and
implementation of a new curriculum, etc. Students prepare a 1-page proposal for
approval by the PIER Steering Committee prior to beginning the experience. Recent
FBE experiences have helped students encounter current, complex, concrete
educational problems in a variety of ways.
Extensive experience as a classroom teacher prior to graduate study in the PIER
program (e.g., having been employed as a real teacher in real school for at least
a year) may fulfill this requirement, if approved by the PIER Steering Committee.
For example, many of our fellows were public or private school teachers prior to
entering graduate school and several of those have teacher certifications.
Because the FBE is designed to complement and facilitate other aspects of the PIER
program, students must provide a written report at the end of the experience that
summarizes their specific FBE and describes their perspective on what they learned
from the experience and how it will improve their ability to do research in the field of
education science (approximately 2500 words). This reflection requirement applies both
to students who complete the FBE during the PIER program and those whose prior
experience exempts them from doing additional fieldwork.
Integrative, Interdisciplinary Project (IIP)
The purpose of the IIP is for PIER fellows from several different departments with
different methodological and content emphases to participate in an interdisciplinary
research team for an education research project culminating in an informal public
presentation (e.g., EdBag) and dissemination via more formal venues if possible (e.g.,
journal article or conference paper). They develop project management, organizational,
and research skills necessary for interdisciplinary education research and stretch by
learning new methodologies and theoretical frameworks. The project must be approved
by the PIER Steering Committee and should be completed prior to the 4th year of
graduate study.
The IIP research team, which ideally consists of at least two PIER students, must be
clearly interdisciplinary, though not necessarily interdepartmental. The faculty advisory
group for IIPs must include at least one PIER faculty member. The PIER Steering
Committee coordinates the match between students seeking project opportunities and
ongoing education research happening on campus and in the community. The project
has to be designed so that each team member has expertise to contribute and specific
areas in which to be “stretched”. Students submit a one-page proposal outline to get the
concept approved prior to completing a full proposal that situates the project in the field
of education science, explains the research question and methods, details the project
management plans, and identifies the relation of the project to PIER’s content and
methodology emphases. IIPs require approximately 10 hours per week spread over 2
semesters and a summer. Students provide the PIER community with a midway
progress report (e.g., via an EdBag) and must complete a written report of the project at
its culmination describing the project process and products, together with reflection on
the experience and the results.
6
EdBag & Colloquium Series
PIER EdBag. The biweekly EdBag series is coordinated by a PIER student. The 14
diverse sessions held during our most recent academic year included research
presentations, discussions of recent articles, presentations by educational practitioners
about current school reform efforts, reflections by learning sciences pioneers regarding
advances in the field, etc. EdBag participation ranges from approximately 15 to 30, with
an average attendance of roughly 18. EdBags have covered a broad range of
outcomes, conditions, levels, and research goals relative to IES’ priorities
PIER Colloquium Series. Our colloquium program is unique among Carnegie Mellon
colloquia in that it has an explicit goal of connecting PIER students with prominent
education researchers for exposure to a wide range of research topics, theoretical and
methodological approaches, and career trajectories relative to the learning sciences. A
senior PIER student coordinates the series, and each two-day visit is co-hosted by a
PIER faculty member and a PIER student. Unlike the “typical” faculty visit, in which
much of the visitor’s schedule is devoted to one-on-one meetings with faculty from the
host department, these visits are intensely student oriented, with almost all of the
visitor’s time spent interacting with individual or small groups of PIER students on a
variety of issues from the state of education research in a particular content area, to
personal history and career paths, to specific scientific research projects and
methodologies. In addition, visitors generally give one main colloquium talk and one
EdBag with audiences drawn from the broad Pittsburgh education research community.
For a complete list of topics and related abstracts for EdBags and Colloquia click here:
Additional Requirements for Fellows
In addition to the program elements described above, all PIER fellows must complete
an annual self-evaluation that is reviewed by the Steering Committee and a dissertation
addressing practical questions in education. Each of these requirements is described
briefly below.
Each June, PIER students complete an Annual Self-Evaluation. This evaluation
provides the Steering Committee with detailed information regarding each fellow’s
participation in PIER and progress toward the program goals. Students begin by
documenting their participation in the PIER community events, with emphasis on any
leadership roles they have played. Fellows list core course, field-based, and other
educational experiences and teaching that advance their knowledge and skills related to
learning theories / principles, research methods / statistics, technology research and
teaching tools, and educational systems. Research documentation includes their
primary program of research, their field-based experience, their interdisciplinary project,
plus any publications, presentations, and grants related to education research.
Professional, university, and community service is also included in the self-evaluation.
The more reflective part of the self-evaluation includes a description of the student’s
current conceptualization of his or her long-term goals, together with a frank
assessment of progress toward those goals. The student then specifies goals for the
summer, the next semester, and the relation of these plans to the achievement of the
future goals. Given this information for each student, the Steering Committee meets to
review student progress, using the student’s Self-Evaluation together with other data
from courses, presentations at CMU, IES, etc. to formulate a consensus about the
student’s standing in the PIER program and to generate constructive feedback to be
7
compiled in writing by the program directors for the student. This process is modeled
after that used in most of the students’ home doctoral programs but is focused on
assessing students’ progress toward PIER’s goals. The process has been especially
productive for helping students balance the sometimes-competing expectations of their
primary department and the PIER program, as well as for focusing faculty members’
mentoring efforts.
The fundamental criterion used by the Steering Committee in assessing students’
annual progress is the extent to which they are advancing toward PIER’s overarching
goal for its trainees: the acquisition of “knowledge and skills that will enable {students}
to prepare competitive grant proposals and/or carry out research projects that reflect the
Institute’s content and methodological foci”. Holding our evaluation meeting
immediately after the June IES research conference is helpful because Steering
Committee members often receive feedback from both IES personnel and other
education researchers about the students’ research programs.
Research requirements prior to the dissertation vary among PIER’s participating
departments, but all PIER students must conduct a dissertation addressing practical
questions in education. Each student is mentored by an interdisciplinary committee,
beginning with a primary advisor in the trainee’s home department and broadening as
the student’s research focus develops.
Research / Dissertation Committees We expect you to work with your advisors, in
consultation with the PIER Steering Committee, to ensure that your dissertation work
has clear educational implications. Your Dissertation Committee must include at least
one PIER Steering Committee member, plus another faculty member (typically, but not
necessarily, on CMU’s faculty) who is a major contributor to Education Sciences
research, and who adds breadth to the distribution of specific research foci among the
committee. Your advisor can count toward one of these roles if she or he is an
appropriate fit.
You may propose alternatives to this dissertation committee composition, but you must
make a case that the proposed alternative would contribute to the overall goal of
producing a strong dissertation with a solid interdisciplinary education research thrust.
Final approval of any such requested modifications will be determined by the PIER
Steering Committee.
In addition to these requirements for committee membership, PIER requires that when
you get around to scheduling the dates for your dissertation proposal and your
dissertation defense, you ensure that PIER’s Director and co-Director (currently David
Klahr and Sharon Carver) are included in the process.
Application and Admissions Process
PIER students enter via the normal graduate application process in the appropriate
department, indicating an interest in PIER on those application forms, and completing a
PIER application form in addition to the form for the home department. (See PIER web
site for application form, http://www.cmu.edu/pier/) Applications are reviewed in
parallel by both the departmental admissions committee and the PIER admissions
committees. Before acceptance into PIER, students must first be admitted into their
respective departments via the normal review process. Participating departments
include an option on their admissions forms for students to indicate that they are
interested in PIER.
8
All of the faculty advisors on the training grant will review the complete set of
applications in their respective departments in order to identify the top candidates,
based upon letters of reference, undergraduate record, research experience. GRE’s,
background in educational contexts (e.g., as teachers), commitment to educational
research, and evidence of potential productivity. Each department, during its normal
graduate admissions process may identify students whom they deem appropriate for
support from the PIER training grant and forward that slate to the PIER Program
Committee, who will make the final selection of PIER candidates.
Each of the participating graduate programs is highly selective: in Psychology,
approximately 100 students apply each year, and 9 or 10 are admitted; the Human
Computer Interaction Institute similarly receives about 100 applications and admits
about 10 students; Statistics receives about 175 applications and accepts about 20, and
Philosophy gets 60 applications and admits 2 PhD students; the Department of
Computer Science receives over 800 applications and accepts about 20.
9