Jeffrey Murray
Test Manager PowerPoint
Microsoft Silicon Valley
The parts of the Office Product Cycle
Measurements and quality
Who Microsoft hires
Stories
• Over 1800 employees, plus 400
vendors/contractors. Approx. 450
employees in San Francisco.
Write code
Graduated
from SUNY
Albany
Manage
people
Manage
products
About every 3 years a major release of
Office comes out with features aimed at
increasing productivity and ease of use
for our customers.
800 Software Design Engineers (Developers)
800 Software Test Engineers (Testers)
400 Program Managers (Feature designers)
200 Localizers, Lab Managers, etc.
200 Planners, Recruiters, Sales and Marketing
2400 Total
Features
• What are you
going to do and
how risky is it?
Resources
• Who are they and
how many?
Schedule
• How much time do
we have?
1. What is the vision for the
product?
2. Make lists of features you
want to do, listen to customers,
and see what is possible
3. Estimate and prioritize what is
you can do in the time you
have
4. Triage these until you can fit
the schedule
5. Then write page 1 specs and
go to step 3
New ribbon
Better graphics
Animation painter
On line editing
Projector setup
Save to video
New animation timeline
Single Document interface
SlideShow broadcast
Co-authoring
New transitions
Better animations
Sections
Native video support
Split video
Camera integration
Hardware support
ODF support
1
Developer
1 Tester
1 Program
Manager
• Feature team makes the decisions
• Must fit into allowed development time
• Must be fully resourced
• Responsible for getting it done
• Management will approve features via
•
•
•
•
Adds/Cuts
Product priorities and opportunities
Manage risk
8 questions
Feature
team
Feature
crew
Feature
crew
Feature
crew
Feature
Feature crew
crew
Feature
Feature
Feature
crew
crew
crew
Feature
Feature crew
Feature crew
crew
Feature
crew
Feature
Feature crew
Feature crew
crew Feature
crew
Feature crew
Feature
Feature cr
Feature
crew
crew
Feature crew
Feature
Feature
crew
crew
Feature crew
Feature
Feature
Feature crew
Feature
crew
Feature
crew
crew
crew
Feature
Feature
Feature
Feature
crew
crew
Feature crew crew
crew
Feature
Feature
Feature
crew
Feature
Feature
crewFeature
Feature crew
crew
crew Feature crew Feature
crew
Feature
crew
crew
Feature
Feature
crew
Feature
Feature
crewcrew
crew
Feature
Feature crew
Feature
crew
Feature crew
crew
Feature crew
Featurecrew
Feature
crew
Feature
crew
Feature crew
crew
Feature
crew
Feature cr
Feature
crew
8 months: Design and implement
4 Months Beta 1 about 10,000 users
Unit test and validate
Beta 2 about 1,000,000 users
8 questions
Plan
Test
Code
RTM
Beta
Planning Phase 4-6 months
12 months
4 Months
CreatePlanned
lists andtesting
1 page
phase,
specsvalidation, user scenarios,
Features ready to go
Development and test
international,
estimate and
stress,
risk security,
assessment
configuration,
Fix last remaining important bugs
Adds/cuts
accessibility, compatibility etc.
No coding
without
dev/test/pm
resource
Code
complete: no
changes
without a bug
entered
Plan
Test
Code
Feature demo
and 8
questions
answered
Triage teams
in place
Product must
be internally
dogfoodable
RTM
Beta
All metrics and
goals must be
met
• Good planning is the key to good quality
• “If you fail to plan you plan to fail”
• Features added or changed late are always more buggy and risky
• Proper design, test, automation support produces better code
• Bug rates are good way to track quality all else being equal
• Customer feedback through Watson
Typical Office Product Bug Trend
• Checks and balances
• Testing signs off on specs
• Dev signs of on test plans
• PM charged with overseeing progress of dev/test
• Constant and never ending improvement
• Test involved earlier
• Automation
• Technical innovations
• Auto code review
• Automation validation before release to testers
• Listening to customers and competition
Hits
Example Watson Curve
450,000
400,000
350,000
300,000
250,000
We don’t have user steps or data
We know what line of code
caused the crash and can often
guard against it
200,000
150,000
100,000
50,000
0
Bucket number
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49
Passion
Potential
Technical
Impact
Microsoft
• Companies are better at identifying talent within you than you
are at bluffing your way through an interview. Make sure you
are there for the right reasons and don’t hold back.
• Don’t plan your whole career all at once, you will miss out on
interesting opportunities
• High tech companies need fresh idea, and that is a great open
door for you
• You are a professional, act like it
• When you screw up (and you will) what you do next is critical
• Ask yourself each week, what do I like about my job?
• Interview the company beyond the job, a good part of your life
will be spent there.
•
•
•
•
•
How I got my Job at Microsoft
Copy protection
Steve’s laptop
OneNote
Office pranks
•
•
•
•
•
Elevator
Beach
Peanuts
Disco
balloons