The Struggles of New College Graduates
in their First Software Development Job
Andrew Begel,
Human Interactions in Programming, MS Research
Beth Simon*,
University of California, San Diego
* Visiting researcher at Microsoft
SIGCSE 2008
March 14, 2008
Study Questions
How do new developers learn their jobs?
What are the critical skills they need for learning
and productivity?
How well does university prepare CS graduates
for a first job in software engineering?
Goals: Improve CS educational pedagogy
Improve developer training at Microsoft
http://research.microsoft.com/hip/
Study Methodology
8 new Microsoft developers
4 BS, 1 MS, 3 PhD
 2 USA education, 2 Chinese, 1 Pakistan, 1
Australia, 1 Kuwait, 1 Mexico. All PhD education
took place in USA.
 Only one had work experience prior to Microsoft
(2 years).
 Hired between Oct 2006 and March 2007.

http://research.microsoft.com/hip/
But wait, this is Microsoft!
Surely, this study couldn’t be relevant
for my students.
http://research.microsoft.com/hip/
The Takeaway
New developers’ problems are mainly
due to poor communication, not
technical skills.
http://research.microsoft.com/hip/
Study Methodology
Observations 4-5 times over a two month period
(concentrated in early April and late May 2007)
Each observation was 2-3 hours. Total of 85 hours.
Participants recorded 3-5 minute videos to answer
reflective questions (which we provided).
Each recorded around 27 videos. Total of 216
videos.
$50 Amazon gift certificate paid for each week of
participation.
http://research.microsoft.com/hip/
Timestamp
Description
Task Type
Subtask
Type
Task
Type #2
Subtask
Type #2
11:45:43 AM
reruns copy script.
Debugging
Reproducing
11:46:18 AM
script done. checks over script
output to make sure it looks
right. Says that the script is
complaining that the files aren't
signed. Email with source
directory says that they are
signed. Weird. copied
successfully, but binaries aren't
signed.
Debugging
Reproducing
MetaCognition
Struggling
with Tools
11:47:26 AM
Shakes head. T is confused.
Team lead says they’re signed.
But empirical evidence says
they're not.
Debugging
Reproducing
MetaCognition
Confusion
11:48:11 AM
T says maybe he wants to sign
the binaries himself.
MetaCognition
Planning
11:48:36 AM
T mutters to himself “bad bad
very bad”
MetaCognition
Reflection
http://research.microsoft.com/hip/
Analysis Methodology
Each observation, record action at the moment
with timestamp.
Each day, code tasks and summarize
observations.
Each week, abstract activities into categories.
Later, resort activities into categories. Recode
actions into events a third party could have seen.
http://research.microsoft.com/hip/
What Do New Developers Do?
•Programming (reading >> writing, commenting,
proofreading)
•Working on bugs ((repro, reporting, triage) >> debugging)
•Testing (writing, running)
•Revision Control (check in, check out, revert)
•Documentation (reading >> writing, search)
•Specifications (reading >> writing)
•Tools (discovering, finding, installing, using, building)
•Communication (asking questions, persuasion, coordination,
email, meetings, meeting prep, finding people, managers,
teaching, learning, mentoring)
http://research.microsoft.com/hip/
The Good and the Bad
New devs are good at




Programming
Design
Peer mentoring
Using many strategies when
debugging
They interact naïvely with
other team members when



Asking questions
Triaging and resolving bugs
Attending meetings
http://research.microsoft.com/hip/
New devs are bad at
Engaging a large, legacy
codebase without using
written docs
 Reproducing bugs
 Using new tools
 Reflecting on their own
progress (knowing when
they are stuck)

Beliefs of New Developers
1. I must prove I am smart, productive and write
quality code from day 1.
Perfectionism 2. I must do everything by myself.
3. I must fix all bugs I personally find in the code.
4. Don’t ask questions.
1. It compiles, therefore it works. If it has bugs…
Delusion well, that’s what testers are for.
2. I know when I get stuck.
Self
Esteem
1. It can’t be the bug repro instructions, it must be
me.
2. My self-worth is directly related to how much I
coded today.
http://research.microsoft.com/hip/
Recommendations for CS Educators
Scaffold and support social interactions in software teams

Employ legitimate peripheral participation, communities of practice
Design higher-fidelity software engineering courses

Work with larger, legacy codebases, pre-existing teams, minimal
docs

Make bug fixes, implementation changes, or non-critical new
features

Deliver capricious management directives
Meta-cognitive skill development: Am I stuck now?
http://research.microsoft.com/hip/
Future Work on this Study
Describe differences between novice and expert
professional developers.
•
Recode observation logs with “functional coding”
schema.
Analyze video diaries: What do new developers
say when reflecting on their own learning?
http://research.microsoft.com/hip/
Future Research Plans
•Improve “onboarding” process at Microsoft
•
Mentoring, mentoring, mentoring
•
•
Modeling appropriate behavior
Teach managers how to teach
•Study new hires in
Other roles (requirements engineering, testing)
• Non-USA locations (e.g. China, India)
• Globally distributed development
• Mass onboarding (corporate acquisitions)
•
http://research.microsoft.com/hip/