The Productive Postdoc:
Do Working Conditions Affect Outcomes?
Geoff Davis
Visiting Scholar and Survey Principal Investigator
Sigma Xi, The Scientific Research Society
gdavis@sigmaxi.org
Improving the Postdoctoral
Experience
• Many calls for changes to the postdoc
– National Academies, AAU, NPA, etc
• Big question: What, if anything, works?
What Works?
 Changes have costs (money, time)
 Do benefits justify investments?
 What should priorities be?
 What gives the biggest bang for the buck?
 These are empirical questions
Our “Experiment”
 Postdoc administration takes place largely at the
level of the PI
 Tremendous variability in conditions from lab to lab
 Recent, limited introduction of new practices
 Natural experiment
 Ask postdocs about their working conditions
 Ask about how well they are doing
 Find conditions associated with positive outcomes
Sigma Xi Postdoc Survey
 Ran a big web survey
 Contacted 22,400 postdocs at 47 institutions
 ~40% of all postdocs in US
 Overall response rate: 38%*
 (*See tech report for details)
Our Sponsor
The Alfred P. Sloan Foundation
Alfred P. Sloan
Michael Teitelbaum
Additional Support
Werthheim Fellowship, Harvard University
Partner Organizations
 National Postdoc
Association
 Science’s Next Wave
 NBER/Sloan Scientific
Workforce Group
Sketch of Our Analysis
• Create measures of inputs (working
conditions, demographics, etc) and
outcomes
• Build linear models to test hypothesis that
inputs have an impact, gauge magnitude of
impact (if any)
How Do We Determine Success?
• Ideal: track people down in 10 years, see what
they are doing / have done
• Problems:
– Very expensive
– Takes 10 years to learn anything
• Driving via the rear view mirror
• Instead, look at immediate proxies for longitudinal
data
Outcomes
• What makes for a “good” experience?
• No single “best” measure
– Different people want different things
• Create collection of outcome measures
– Look at impact of inputs on each
Subjective Outcome Measures
• Subjective success measure
– Overall satisfaction, preparation for independent
research, quality of training in research / teaching /
management
• Advisor relations measure
– How is your advisor doing? Is s/he a mentor? How
would s/he say you are doing?
• Generate numerical scores by summing Likert
scored answers
Objective Outcome Measures
• Absence of Conflict/Misconduct
– Has postdoc had a conflict with advisor? Has
s/he seen misconduct in the lab?
• Productivity
– Rate at which papers submitted to peer
reviewed journals
Outcome Measure Distributions
Outcome Measure Details
• Correlations all fairly low
– Subjective success and advisor relations ~0.45
– Other pairwise correlations all < 0.2
Our Explanatory Variables
• Model outcomes as function of explanatory
variables
– Field of research
– Institution
– Basic demographic variables
•
•
•
•
Sex
Citizenship
Minority/Majority Status
Type of degree (MD vs PhD)
– Total time as a postdoc
– “Working Conditions”
“Working Conditions”
• How do we measure working conditions?
• Inspiration comes from various calls for
changes
– Look at rate of implementation
Recommended Changes
• 5 broad classes of recommended changes
–
–
–
–
–
Pay people more
Fellowships rather than assistantships
Better benefits
More structured oversight
Transferable skills training
Measures of Working Conditions
• Salary measure
– log(annual salary), full-time people only
• Independent Funding measure
– Dummy variable, 1 if fellowship, 0 otherwise
• Benefits measure
– Count of different benefits received (health
insurance, retirement plan, etc)
Structured Oversight
• Structured Oversight measure
– Count of administrative measures in place
•
•
•
•
Individual development plans
Formal reviews
Policies (authorship / misconduct / IP / etc)
Letters of appointment
– High values = lots of structure, low = little
Training
• Transferable Skills Training measure
– Count of areas in which postdoc reports
receiving training
– Grant writing, project/lab management,
exposure to non-academic careers, negotiation,
conflict resolution, English language, etc
– High values = training in lots of areas
– Low values = no training in lots of areas
Working Conditions Distributions
Working Conditions Details
• Again, correlations all fairly low
– Structured oversight and skills training ~0.30
– Other pairwise correlations all < 0.15
What Has Biggest Impact?
• Who is most satisfied, most productive, etc?
• People with
–
–
–
–
–
Independent funding?
High salaries?
Lots of benefits?
Lots of structured oversight?
Lots of types of transferable training?
Simple Analysis
• Crude analysis: compare satisfaction,
productivity, etc for people in appointments
with
–
–
–
–
–
Fellowships / other funding
High / low salaries
High / low benefits
High / low structure
High / low training
Independent Funding
Fellowship
Other
% satisfied
74%
70%
Advisor grade
(0=F, 4=A)
3.0
3.1
% reporting
conflicts
14%
14%
Papers
1.1
submitted / year
1.2
Salary
Highest 25%
Lowest 25%
% satisfied
71%
68%
Advisor grade
(0=F, 4=A)
3.0
3.1
% reporting
conflicts
16%
13%
Papers
1.2
submitted / year
1.2
Benefits
Highest 25%
Lowest 25%
% satisfied
76%
62%
Advisor grade
(0=F, 4=A)
3.2
2.9
% reporting
conflicts
11%
18%
Papers
1.3
submitted / year
1.2
Structured Oversight
High structure
80%
3.4
% satisfied
Advisor grade
(0=F, 4=A)
% reporting
9%
conflicts
Papers
1.4
submitted / year
Low structure
60%
2.7
21%
1.0
Transferable Skills Training
High training
83%
3.4
% satisfied
Advisor grade
(0=F, 4=A)
% reporting
10%
conflicts
Papers
1.3
submitted / year
Low training
56%
2.7
17%
1.1
Regression Coefficients
total structure
total training
total benefits
log(salary)
funding
male
citizen
underrepresented
medical degree
months total
Subjective
Success
0.158 ***
0.455 ***
0.094 ***
0.024
0.178 ***
0.089 **
0.081 **
0.051
-0.178 ***
-0.004 ***
Advisor
Relations
0.159 ***
0.247 ***
-0.000
0.112 ***
0.048
0.015
0.035
0.013
-0.107 *
-0.003 ***
Absence of
Conflict
0.283 ***
0.120 ***
0.125 ***
-0.036
0.131
0.138
0.077
0.017
-0.452 ***
0.018 ***
Productivity
0.045 ***
0.050 ***
-0.033 *
0.031 .
0.015
0.081 **
-0.058 *
-0.019
-0.032
0.001
Take Home Message #1
• Structured oversight and
transferable skills training
make a big difference
Causality?
• We have correlation. Is there causation?
– Psych literature gives reasons to believe in causation
• Alternative explanations
1. Structure and training attract people who are
intrinsically more satisfied / productive / successful
2. Structure / training correlate with some other
unobserved factor
– Advisors are effective managers / have more resources
– Postdocs take more initiative / are better organized / etc
Causality?
•
2 classes of explanation
1. Structure/training attract intrinsically more productive
people
2. Structure/training directly cause productivity or are
indicators for some causal mechanism
(Some combination of 1 & 2 also possible)
•
Should be able to differentiate between 1 & 2 by
looking at people with multiple appointments
Intrinsic vs. Time-Localized
Causality?
• Add in terms that allow for change in slope of
papers(t) curve starting at beginning of most
recent postdoc
• Equivalent to adding interactions with ratio
(months in current postdoc / total months as
postdoc) to regression model
• Training appears to have a time-localized effect
• Other inputs ambiguous
Don’t Pay Postdocs?
• Not saying postdocs shouldn’t be paid!
– Hard to attract US students to science if you don’t pay them
• Maslow’s hierarchy of needs
– Must meet basic physical security needs first
– Living wage, basic benefits
• More nuanced interpretation of data: beyond a certain
threshold, structure and training matter more than
compensation
• Institutional “postdoc tax” to support service provision?
More Details
• Look at individual components of structure
and training measure
• What specific measures have the greatest
impact?
Impact
• One measure appears to have significant
impact all 4 outcomes:
– Research / career plans
• Written plans
• Plans that spell out what both postdoc and PI will do
• Advocated by FASEB, National Academies
Plans
• Compare those with such a plan to those
without:
– Much less likely (~40%) to be dissatisfied
– Much less likely (~30%) to have conflicts
• After controlling for field, institution,
demographics:
– Submitted ~14% more papers for publication
Why?
• Plans:
– Expectation setting device
• Postdocs without plans were much more likely to report PI had
not lived up to expectations
– Contract
• Research shows that people are more likely to live up to
explicit (esp. written) commitments
– Forces postdocs to take responsibility for their careers
early
• More time to take advantage of training opportunities
– Time management device
• Mechanism for focusing effort
Take Home Message #2
• Individual development
plans make a big
difference
Additional Measures
 Several other measures show concrete
benefits:





Teaching experience
Exposure to non-academic careers
Training in proposal writing
Training in project management
Training in ethics
Policy Implications
 For postdocs, more effective to invest
additional dollars in management than in
salaries
 Management at all levels:
 Infrastructure for institutional oversight /
training
 Management training for PIs
 Management training for postdocs
Further information
 More information at
http://postdoc.sigmaxi.org
 Workshop (with NPA) in January 2006
 Contacts
 Geoff Davis, PI, gdavis@sigmaxi.org
 Jenny Zilaro, Project Manager, jzilaro@sigmaxi.org
Extra Material
End Products
 Sigma Xi:
 Highlights in May/June issue of American Scientist
 Tech reports (2 out now, more to come)
 Scholarly paper this fall
 NPA: Analyses of various topics
 NBER SEWP
 Workshop in January 2006
Aside: Postdoc Definition
• Half a dozen different definitions
– AAMC, AAU, FASEB, NAS, NSF
• BUT if you read and compare them, they all say
the same thing
– Only substantive difference is that FASEB includes
narrow subset of clinical fellows
– (We excluded them from this analysis)
• Most people don’t fully satisfy definition anyway
Postdoc Definition
•
•
•
•
•
•
•
The appointee has a PhD or equivalent degree,
the degree was received recently,
the appointment is temporary,
the purpose of the appointment is training for a research
career,
the appointment involves substantially full-time research
or scholarship,
the appointee is expected to publish the results of his or
her research, and
the appointee works under the supervision of a senior
scholar or a department in a university or research
institution.
Survey Non-Response
 30-second summary of non-response
analysis:
 Non-citizens and African Americans appear to
be slightly under-represented
 No evidence of bias based on level of
satisfaction (respondents not overly
disgruntled)
Survey Non-Response
• Survey respondents atypical in one
important way
– Participating institutions all had PDO, PDA, or
administrator interested in postdoc affairs
• Participating institutions probably better
off than average
Salaries
• Median salary: $38,000
• Up from $28,000 in 1995
Inflation
• A 10% increase above inflation since 1995
– ($28,000 in 1995 = $34,700 in 2004)
• NIH budget doubled over the same period
(in inflation-adjusted dollars)
Experience
• Salaries increase at about 2.9% per year of
experience
Field
• Overall average = $39,300
• Average salary in most common fields ranges from
$37,500 to $40,000
• Higher:
–
–
–
–
Electrical engineering ($45,000)
Physics ($42,600)
Oncology ($41,400)
Materials science ($41,200)
• Lower:
– Ecology ($35,600)
Institution Type
• Govt labs pay 20% more than average
• Public universities pay 9% less than average
Taxes
• Tax loophole: some postdocs don’t have to pay FICA
(7.65% of income)
– 23% benefit
– New IRS rules affect this
• Tax penalty: some postdocs pay extra self-employment tax
(also 7.65% of income)
– 12% pay
– Independent contractor status carries hidden tax penalty!
• Potential $6,000 impact on salary
Part-time
• 3% report part-time status
• Average hours worked previous week: 45
Hours
• 51 hours/week median
• Postdoc hourly wage ~ $14.90
Hours
• 51 hours/week median
• Postdoc hourly wages = $14.90/hour
• Harvard janitors = $14.00/hour
Foreign Postdocs
• International Men and Women
of Mystery
Basic Demographics
 Citizenship:
 Citizens:
 Permanent residents:
 Temporary visa holders:
40%
6%
54%
 PhD:
 US PhD:
 Non-US:
53%
47%
Non-US PhDs
 Where PhD earned:
All
US citizens
(41%)
Permanent
residents (6%)
Temporary
(53%)
US
53%
97%
51%
21%
Elsewhere
47%
3%
49%
79%
 Almost 80% of postdocs on temporary
visas earned their PhDs outside the US
 Non-US PhDs invisible in NSF stats
Non-US PhDs
 Where non-US PhDs were earned:
 Country of citizenship
 Different country, same continent
 Different continent
86%
7%
7%
Temporary Visa Holders
Citizenship
Source of PhD
China
24%
China
18%
India
11%
India
10%
Germany
6%
Japan
8%
South Korea
6%
UK
8%
Japan
6%
Germany
8%
Canada
5%
France
6%
France
5%
Canada
5%
United Kingdom
4%
South Korea
4%
Spain
3%
Israel
3%
Italy
3%
Spain
3%
Top 10
73%
Top 10
73%
Non-US Postdocs and PhDs
 China and India dominate
 Market share of postdocs comparable to share
of doctorates (China = 23%, India = 10%)
 Next largest LDC is Argentina, #16 for both
citizenship and PhDs, with 1% of each
Temporary Visa Holders by Field
Electrical engineering
72%
Physics
Chemistry
Molecular biology
Biochemistry
67%
61%
58%
57%
Cell biology
Earth sciences
Ecology
57%
52%
36%
Psychology
21%
Broad Field
Temporary visas Non-US PhDs
Life/health
sciences
Physical
sciences /
engineering
Social sciences
52%
47%
63%
44%
23%
18%
Other Characteristics
US postdocs:
 49% men/51% women
International postdocs:
 65% men/35% women
 69% married
 33% have children
 Median age: 33
 69% married
 35% have children
 Median age: 33
Other Characteristics
 One notable difference for married postdocs
 US postdocs: 15% have non-working spouse
 Non-citizen postdocs: 44% have non-working spouse
 Some visas (e.g. H) don’t have provision for spouse to work
Domestic vs International: Papers
 International postdocs publish more
 Average peer-reviewed publications as a postdoc
 Citizens/PR
 Temporary
2.6
3.3 (27% more)
 Difference is smaller (.1 papers/year) after we
control for time as a postdoc, field, institution, sex,
but statistically significant
Domestic vs International: Hours
 Non-citizens work longer hours
 Average weekly hours worked
 Citizens/PR
 Temporary
50
52 (4% more)
 Difference is smaller (1.3 hours/week) after we
control for time as a postdoc, field, institution,
sex, but still statistically significant
Domestic vs International: Salary
 BUT non-citizens are paid substantially less
 Median annual salary
 Domestic
 International
$40,000
$37,000 (8% less)
 Domestic postdocs earn $2,200/year more than
international postdocs after controlling for field,
institution, sex, time as a postdoc, and funding
mechanism
Domestic vs International: Grants
 Citizens write more grant proposals (results
suggest mostly fellowship applications)
 Grant proposals written while a postdoc
 Citizens
 Non-citizens
1.6
1.1 (31% fewer)
 International postdocs write fewer grant
proposals even after controlling for field,
institution, sex
Domestic vs International: Satisfaction
 Non-citizens report slightly lower levels of
satisfaction with the postdoc experience
 Average satisfaction
(-2 = dissatisfied / 2 = satisfied)
 Citizens/PR
 Temporary
0.8
0.6
 Difference disappears when one controls for
salary, discipline, institution, sex, and time as a
postdoc
Security Problems
 To what extent have US national security regulations
affected your ability to do the following:
(% responding “Some” or “A lot”)





Conduct your research in the US:
Travel outside the US to conduct your research:
Visit your country of citizenship:
Re-enter the US after leaving the country:
Bring your immediate family members to the US:
30%
40%
55%
57%
36%
 Free-text comments express considerable frustration
More information
 More information at
http://postdoc.sigmaxi.org
 Contacts
 Geoff Davis, PI, gdavis@sigmaxi.org
 Jenny Zilaro, Project Manager,
jzilaro@sigmaxi.org
Survey Responders
 Difficult to obtain ground truth for assessing
results
 Plan: compare results of pilot survey to known
values for one institution with good records
 Reality: survey revealed that the institution in
question was missing lots of postdocs (~10% of
the local population)
Survey Responders
 Fortunately we found an alternative with better
records
 Differences in response rates consistent with
levels of variation in a random sample for
 Sex
 Citizenship
 Minority status
 No strong evidence of non-response bias
Further Non-response Analysis
 Survey literature: propensity to respond is a
continuous variable
 Early responders: high propensity
 Late responders: lower
 Non-responders: lowest
 Idea is that non-responders are more similar to late
responders than early responders
 Compare early and late responders. Differences
suggest potential non-response bias.
Non-response Bias?
 Who are missing 66% of postdocs?
 No significant difference between early and late responders
by
 Sex
 Overall satisfaction
 Significant but small difference by citizenship (p ~0.04)
 Early responders: ~49% citizens
 Late responders: ~45% citizens
 Non-citizen postdocs are probably slightly
underrepresented
Domestic vs International: Satisfaction
 Non-citizens report slightly lower levels of
satisfaction with the postdoc experience
 Average satisfaction
(-2 = dissatisfied / 2 = satisfied)
 Citizens/PR
 Temporary
0.8
0.6
 Difference disappears when one controls for
salary, discipline, institution, sex, and time as a
postdoc
Settlement Interests
 Level of interest (0=None, 2=High) in settling
in various regions (ignoring visa issues)*
US
Europe
Asia
US citizens
2.0
0.8
0.2
European
citizens
1.4
1.8
0.3
Asian citizens
1.6
1.2
1.3
Settlement Interests
 Level of interest (0=None, 2=High) in settling in
various regions (ignoring visa issues)*
US
Europe
Asia
US citizen, US PhD
1.97
0.75
0.20
US citizen, non-US PhD
1.67
1.50
0.25
European citizen, US PhD
1.64
1.43
0.21
European citizen, non-US PhD
1.35
1.86
0.28
Asian citizen, US PhD
1.73
1.04
1.33
Asian citizen, non-US PhD
1.58
1.20
1.26