New SERU Graduate
Student Survey
October 9, 2014
Gregg Thomson
(SERU PR)
Daniel White
Olena Horner
Ron Huesman
Tiffany Thayer
(University of Minnesota)
SERU Consortium
2014 SERU GSS Workgroup
2014 SERU GSS Workgroup Members
Chair Ron Huesman – Managing Director, SERU-AAU Consortium, UMN*
Melissa Anderson – UMN Graduate School*
Igor Chirikov – Higher School of Economics, Moscow
Ken Doxsee – Oregon
Louis Myers – Virginia
Gregg Thomson – SERU PR/CSHE*
Daniel White – UMN*
Staff Support
Olena Horner – UMN Research Assistant*
Shelva Hurley – CSHE SERU Research Associate*
Workgroup Consultants/Advisors:
Tom Dohm – UMN
Anne Maclachlan – CSHE
Maresi Nerad – University of Washington
* Denotes Planning Team responsible for developing a preliminary draft for the Workgroup
Graduate SERU Workgroup
Goals
− Primary:
a. Done: Developed the core of the survey
b. Done: Populations, survey design (e.g., census, across
academic levels),
c.
Identified central data elements for exchange
a.
Seed file (Done)
b.
Supplemental file (tbd)
d. Fall 2014 SERU pilot: launches late October, 2014 - 6 weeks
–
University of Minnesota: PhD and MA students
–
University of Virginia: PhD only
• Winter 2015 International pilot:
–
International: Higher School of Economics (Moscow)
–
Unicamp(Brazil) Winter
RATIONALE & NEED
Challenges Faced by
Graduate Education in the U.S.
• Ph.D. Attrition
− 6.6% leave in their first year
− 30.6% leave without a degree by year ten
• Ph.D. Time to Degree
− 56.6% complete their degree within ten years
− 7.7 years – the national median time to degree in 2008
• International Competition
−
−
−
−
33,000 - S&E doctoral degrees awarded in the U.S.
28,000 - S&E doctoral degrees awarded in China
15,000 - S&E doctoral degrees awarded in Russia
11,000 – S&E doctoral degrees awarded in Germany
Note: data on doctoral degrees are presented for 2007-2008
Source: Council of Graduate Schools (2008), National Science Foundation (2009, 2012)
Survey Rationale
A survey can provide vital information
about graduate/professional student
experiences while enrolled that may help
improve graduate education outcomes
(i.e., increased graduation rates, faster
time to degree, and help students find
successful careers).
The Need for the New Graduate
Student Survey
• Existing Surveys
− External exit surveys: Survey of Earned Doctorates &
Survey of Doctorate Recipients (NSF)
− Institutional exit surveys: encouraged to use Graduate
Education Exit Survey (AAUDE) core as a component
− Locally developed institutional student experience
surveys: (UC-Berkeley, the University of Texas at Austin,
the University of Michigan, Rutgers)
• Benefits of the New Survey
− a joint effort between AAU and SER-I
− the survey will focus on graduate student experiences
while enrolled in school
− the survey will provide data for cross-institutional and
international comparisons
THE PROCESS
An Opportunity
Graduate Students:
Desire for resources to help
with the analysis of student
developed survey of graduate
experience.
SERU Consortium:
Administration:
2012 an opportunity for a
survey of the enrolled
Graduate student experience
survey in the Research
University and desire for
multi-institutional data
Demand for high quality
information about the
graduate student experience
at UMN.
GradSERU
Consensus Building:
Identifying Stakeholders and Expertise
Graduate School
Council of Graduate
Students
Graduate and
Professional
Student Association
Office of
Institutional
Research
Office of
Measurement
Services
Academic Health
Center
Law School
Office of Public
Engagement
Boynton Health
Center
Center for Teaching
and Learning
Office of Student
Affairs
Graduate Review
and Improvement
Process
Carousel Brainstorming Session
• What opportunities would an enterprise-wide graduate and
professional student survey create for the University and individual
units?
• What challenges would an enterprise-wide graduate and
professional student survey create for the University and individual
units?
• What benefits would result from an enterprise-wide graduate and
professional student survey for the University and individual units?
• What concerns or fears do you have in regard to an enterprisewide graduate and professional student survey for the University
and individual units?
• What impact would an enterprise-wide graduate and professional
student survey have on the University and individual units?
Consensus Workshop
If we were to create an effective enterprise-wide
graduate and professional student survey, what
information would need to be collected about students
and their experiences?
Outcomes
Expectation
Fulfillment
Finances
Development
Opportunities
Support
Community
Challenges
and Barriers
Advising
Health and
Wellbeing
Why are you
here?
Getting to Started
1. What is out there?
•
•
•
•
•
•
•
Survey of Earned Doctorates (NSF).
Survey of Doctorate Recipients (NSF/NIH).
PhD Completion Project (CGS)
Pew Trusts Survey on Doctorate Educ.
Ivy League Graduate Student Survey.
Assessment of Research Doct. Prog. (NRC).
Graduate Education Survey (AAUDE).
2. What is missing?
Developing the Survey:
A Parallel Process
UMN Workgroup
collaboration on initial
draft.
SERU/CSHE
collaboration on initial
draft.
Open comment period
for stakeholders to raise
questions/issues.
Open comment period
for stakeholders to
raise questions/issues.
Dozen one-hour
interviews with
stakeholders.
Teleconference review
of the draft.
Revisions.
Revisions.
THE INSTRUMENT
Review of Current Research
Literature on Student Experience
Astin’s
model of student
development in
higher education
(1970)
Tinto’s
theory of graduate
persistence
(1993)
Graduate
SERU
Survey
Conceptual
Framework
Research literature on key
individual- and institutionrelated factors that impact
on student success
(degree completion & time
to degree) in graduate
school
Graduate SERU Survey
Conceptual Framework
Conceptual framework: provides a basis/blue-print for survey
development; mapping of final items to each domain will provide
an item rationale document and a useful guide for conducting
research with the survey in the future.
INPUT
Attributes
Student
Background/
Current Status
• Demographics
• Parents’
educational
attainment
• Current program
• Type of degree
• Stage in the
program
Previous
Higher Education
Experience
• Degree, field,
institution
• Time elapsed
since completing
a previous
degree
Financial
Resources
• Ability to pay
• Undergraduate/
graduate loans
PROCESS
Entry
Orientations
Admissions &
Entrance
• Goals
• Reasons for
selecting a
program
• Program
communications
Financial
Support
• Sources
• Degree of
support
External
Commitments
• Work
• Family
Note. Derived from Tinto (1993) and Astin (1970).
OUTCOMES
Student Experience
Socialization into
the Profession &
Professional
Development
• Professional values
and ethical issues
• Opportunities
offered by the
program/
college/school/
Graduate School
Other Institutional
Support
Program Climate
• Satisfaction with
the program
climate
Advising &
Mentoring
• Quality of advising
• Areas of support
from an advisor
• Areas of support
from a mentor
Dissertation/Thesis
Stage
• Quality of amenities
at the institution
• Dissertation topic
• Dissertation research
process
Financial Support
Research
Experience
• Satisfaction with
financial support
• Impact of
employment outside
of the university on
degree progress
• Anticipated
consequences of
debt burden
• Experience with
research-related
activities
• Sources of financial
support
• Interdisciplinary
research and its
challenges
Outcomes
Teaching
Experience
• Training
• Teaching
experience
• Impact on degree
progress
Graduate/
Professional
Degree
Completion
• Commitment to
complete a
degree in the
current program
Career Plans &
Changes
Proficiency
Levels
• Changes in career
plans during
grad/prof studies
• Careers toward
which a degree
program is
oriented
• Factors influencing
career choices
• Development of
academic,
research, and
professional skills
Obstacles to
Degree Progress
• Factors that hinder
degree progress
Overall
Satisfaction
• Satisfaction with
various aspects
of a program
• Fit between
students’ values,
expectations and
the program
• Choose the same
field, program,
university
Graduate SERU Survey
Structure
The survey covers 16 areas of student experience:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
A. Your Graduate/Professional Program
B. Previous Higher Education Experience
C. Admissions and Entrance
D. Socialization into the Profession and Professional Development
E. Financial Support
F. Other Institutional Support
G. Program Climate
H. Proficiency Levels
I. Advising and Mentoring
J. Dissertation/Thesis Stage (for doctoral students only)
K. Research Experience
L. Teaching Experience
M. Career Plans & Changes
N. Obstacles to Degree Progress
O. Overall Satisfaction
P. For International Students Only
Graduate SERU Survey
Structure
Core Survey
• Factual questions: 42
• Opinion questions: 37
• Demographics: 12
International Students only
• Factual questions: 5
• Opinion questions: 4
Examples of Survey Questions
E. FINANCIAL SUPPORT
To what extent do you agree or disagree with: “Having a job outside the
university while going to school
a.
b.
c.
d.
e.
f.
g.
helped me with career preparation
helped me secure a job
helped me advance my career
restricted my choice of classes
limited the number of classes I could take
slowed my degree progress
limited my access to campus facilities or services”?
Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable
When you complete your degree, how much do you think your debt burden will
affect:
a.
b.
c.
d.
the type of job you will seek
the part of the country/world in which you will live
your family planning
your life style?
Scale: Not at all, Little, Some, A great deal
Current Version
• Graduate SERU
– CORE within academic level (Ph.D. and not)
and across all post-baccalaureate students
• Items tailored to the academic level (similar items
with term-specific references to the level or
completely different items aimed at all students at
a specific academic level)
Future
• Pilot test of the survey
– Reviews (staff and students)
− Additional “think-aloud” interviews or
verbal probing
– Post collection: Item Response Theory
analysis of items; examples of competency
items
Revise and focus survey: Winter/spring of 2015
SERU consortium: Vision
• Graduate SERU and Undergraduate
SERU, (and their associated modules)
become part of a suite of surveys available
to members for administration.
• Costs? TBD information from pilot will be
useful in estimating actual costs
• SERU institutional reps, same?
Examples of Survey Questions
PROFICIENCY LEVELS (v1)
To what extent have your skills in the following areas developed during your
graduate/professional program?
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
Analytical and critical thinking skills
Understanding your specific field of study
Ability to speak effectively in your field
Ability to write effectively in your filed
Ability to design and conduct original research
Ability to write grant proposals
Ability to work collaboratively on a project
Ability to work internationally
Ability to collaborate across disciplines
Ability to critically analyze the research literature (or performance products) from your
field
k. Having transferable skills for various career opportunities
l. Adherence to high standards of ethics and professional responsibility
m. Ability to teach in your field
n. Ability to innovate, be entrepreneurial
o. Other (please specify): ___________________
Scale: Not at all, Not very much, To some extent, To a great extent, Not
applicable
Examples of Survey Questions
PROFICIENCY LEVELS (v2)
Please rate your level of proficiency in the following areas when you started your
graduate/professional program and now.
Currently
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
k.
l.
m.
n.
o.
When you started
Analytical and critical thinking skills
Understanding your specific field of study
Ability to speak effectively in your field
Ability to write effectively in your filed
Ability to design and conduct original research
Ability to write grant proposals
Ability to work collaboratively on a project
Ability to work internationally
Ability to collaborate across disciplines
Ability to critically analyze the research literature (or performance products) from your field
Having transferable skills for various career opportunities
Adherence to high standards of ethics and professional responsibility
Ability to teach in your field
Ability to innovate, be entrepreneurial
Other (please specify): ___________________
Scale: Poor, Fair, Good Excellent, Not applicable
Examples of Survey Questions
ADVISING AND MENTORING
To what extent do you agree or disagree with the following statements
concerning your current, primary advisor?
a. My advisor has expertise in the area I’m studying.
b. My advisor knows how to effectively help me.
c.
My advisor provides me with information I need to help me think about my career.
d. My advisor has time for me when I need help or advice.
e. My advisor helps me get financial support.
f.
My advisor assists me in writing for presentations/publications.
g. My advisor advises me about teaching.
h. My advisor teaches me the details of good research practice.
i.
My advisor respects me as an individual.
j.
My advisor considers my personal abilities, talents, and interests when advising me.
k.
My advisor is interested in having students help with his/her research.
l.
My advisor prompts me to seek out opportunities that I would not have otherwise
considered.
Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable
Examples of Survey Questions
OBSTACLES TO DEGREE PROGRESS
Rate the extent to which the following factors have been an obstacle to
your degree progress?
a. Difficult coursework and academic requirements
b. Diminished interest in the field of study
c. Family responsibilities or obligations
d. Inadequate advising
e. Inadequate financial support
f.
Need to work
g. Nonsupportive or unfriendly environment for students like me
h. Physical or emotional health problems
i.
Poor or uncertain employment prospects after degree completion
j.
Immigration issues
k. Other (please specify): ______________________________
Scale: Not an obstacle, A minor obstacle, A major obstacle, Not applicable
Questions
Contact
– Ron Huesman huesm003@umn.edu
– Daniel Jones-White djwhite@umn.edu
– Olena Horner glus0005@umn.edu
References
Astin, A. W. (1970). The methodology of research on college impact, part one. Sociology of Education, 43(3), 223-254.
Retrieved from http://www.jstor.org/stable/2112065
Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the
doctorate. Research in Higher Education, 27(1), 3-14.
Ampaw, F. D., & Jaeger, A. J. (2012). Completing the three stages of doctoral education: An event history analysis. Research in
Higher Education, 53, 640-660. doi: 10.1007/s11162-011-9250-3
Andrieu, S. C., & St. John, E. P. (1993). The influence of prices on graduate student persistence. Research in Higher Education,
34(4), 399-425.
Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral
study. Research in Higher Education, 31(4), 369-385.
Baird, L. L. (1993). Using research and theoretical models of graduate student progress. New Directions for Institutional
Research, 80, 3-12.
Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press.
Bowen, W. G., Lord, G., & Sosa, J. A. (1991). Measuring time to the doctorate: Reinterpretation of the evidence. Proceedings
of the National Academy of Sciences, 88(3), 713–717.
Council of Graduate Schools. (2008). Ph.D. Completion and attrition: Analysis of baseline program data from the Ph.D.
Completion Project. Washington, D.C.: Council of Graduate Schools.
de Valero, J. F. (2001). Departmental factors affecting time-to-degree and completion rates at one land-grant research
institution. The Journal of Higher Education, 72(3), 341-367.
Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and
completion rates? The Journal of Human Resources, 30(3), 581-609.
Ethington, C., & Pisani, A. (1993). The RA and TA experience: Impediments and benefits to graduate study. Research in Higher
Education, 34(3), 343-354.
Gardner, S. K. (2009). The development of doctoral students: Phases of challenge and support. ASHE Higher Education Report,
34(6), 1-127. doi: 10.1002/aehe.3406
Gillingham, L., Seneca, J. J., & Taussig, M. K. (1991). The determinants of progress to the doctoral degree. Research in Higher
Education, 32(4), 449-468.
Girves, J. E., & Wemmerus, V. (1988). Developing models of graduate student degree progress. The Journal of Higher
Education, 59(2), 163-189.
Golde, C. M. (2000). Should I stay or should I go? Student descriptions of the doctoral attrition process. The Review of Higher
Education, 23(2), 199-227.
Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments.
The Journal of Higher Education, 76(6), 669-700.
Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. U.S. Bureau of Labor Statistics. Retrieved
from http://www.ilr.cornell.edu/cheri/upload/cheri_wp146.pdf
Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in
graduate education. Economics of Education Review, 27(2), 111–124.
Haldaway, E., Deblois, C., & Winchester, I. (1995). Supervision of graduate students. The Canadian Journal of Higher Education,
XXV(3), 1-29.
Jiranek, V. (2010). Potential predictors of timely completion among dissertation research students at an Australian faculty of
sciences. International Journal of Doctoral Studies, 5, 1-13.
Nerad, M., & Miller, D. S. (1996). Increasing student retention in graduate and professional programs. New Directions for
Institutional Research, 92, 61-76.
Ott, M. D., Markewich, T. S., & Ochsner, N. L. (1984). Logit analysis of graduate student retention. Research in Higher
Education, 21(4), 439-460.
Park, C. (2005). War of attrition: patterns of non-completion amongst postgraduate research students. Higher Education
Review, 38(1), 48-53.
Pascarella, E .T., & Terenzini, P. T. (2005). How College Affects Students: A Third Decade of Research. San Francisco: Jossey-Bass.
Picciano, J., Rudd, E., Morrison, E., & Nerad, M. (2008). Does time-to-degree matter? CIRGE Spotlight on Doctoral Education
#3. CIRGE: University of Washington, Seattle, WA. Retrieved from www.cirge.washington.edu
Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of
doctoral degrees. Research in Higher Education, 39(3), 319-335.
Smith, R. L., Maroney, K., Nelson, K. W., Abel, A. L., & Abel, H. S. (Spring, 2006). Doctoral programs: Changing high rates of
attrition. Journal of Humanistic Counseling, Education and Development, 45, 17-31.
Stricker, L. J. (1994). Institutional factors in time to the doctorate. Research in Higher Education, 35(5), 569-587.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago and London: The University of
Chicago Press.
Wendler, C., Bridgeman, B., Cline, F., Millett, C., Rock, J., & McAllister, P. (2010). The Path forward: The future of graduate
education in the United States. Princeton, NJ: Educational Testing Service.
Wilson, S. B., Mason, T. W., & Ewing, M. J. M. (1997). Evaluating the impact of receiving university-based counseling services
on student retention. Journal of Counseling Psychology, 44(3), 316-320.