NASA Goddard Space Flight Center
Learning Lessons
at NASA Goddard
January 22,
G O D D A R D
“It is difficult to say what is impossible, for
the dream of yesterday is the hope of today
and the reality of tomorrow.”
Dr. Robert H. Goddard
1882 - 1945
2008
S P A C E
F L I G H T
C E N T E R
Why Do We Need to Manage Knowledge?
http://mediastream.ndc.nasa.gov/Publ
ic/KM/case_studies/failure_to_listen/
default.html?iframe=true&width=100%
&height=100%
G O D D A R D
S P A C E
F L I G H T
C E N T E R
2
What Have We Done?
How Did We Do It?
What Have I Learned?
May 2003 to October 2011
Edward W. Rogers
20, 2011
October 19-
GSFC Chief Knowledge Officer
Office of the Director
Goddard Space Flight Center
“Your Knowledge Management Programs should be like a
good pair of shoes—they should fit your organization well
and they should take you someplace interesting.”
Ed Rogers, 2009
G O D D A R D
S P A C E
F L I G H T
C E N T E R
What Some Say…
“If I hadn’t had to prepare for that workshop, I wouldn’t really have figured out
what I learned from that mission.”
Project Scientist at
GSFC
“That one PaL probably saved my project. “
Project Manager at GSFC
“When one is buried deep in a project, one loses sight of everything else—other
opportunities, visions, culture, GSFC as a whole. The course has given me a
vast overview of GSFC that will allow me to be a more efficient employee.
There are so many elements behind Goddard’s history that I really didn’t
know. Knowing the history is even more helpful and exciting as a member of
the NASA team.”
RTMS Participants 2011
“Most effective part was the first two case studies. I really enjoyed
hearing about other contracts & what they have encountered & how
they’ve worked through them.”
Procurement Forum, 2011
G O D D A R D
S P A C E
F L I G H T
C E N T E R
4
What Have We Done in Numbers?
• Developed an Internal Case Study Collection
– Over 50 Case Studies Developed and Published: Quarterly Case
Catalog
– Person Case Interactions (one person reading and learning from a
case)
• NASA and External Use of Goddard Developed Case Studies
– 1000’s through APPEL NASA HQ Programs (total unknown)
– 3000 at PMChallenge NASA and Contractor Public Conferences
– Unknown Academic use outside of NASA through public
release
• Goddard Application of Lessons Through Cases
–
–
–
–
–
5000 at Road to Mission Success at Goddard (Ten series taught)
1000 at Various Goddard Open Knowledge Workshops
600 with Flight Project Managers
400 during Goddard Safety Weeks
150 during Procurement Training
• PaLs done with Projects, Teams and Org. Units
• Advanced Concepts and Formulation Office captured over 150 key
insights from PaLs done after proposal efforts
• WIKI’s within Engineering Directorate for bench level sharing
• 100’s of pages created and updated regularly
G O D D A R D
S P A C E
F L I G H T
C E N T E R
5
Costly Lessons About Learning
COLUMBIA
Accident Investigation Board
The Challenger Launch Decision
G O D D A R D
S P A C E
F L I G H T
C E N T E R
6
The Problem is a Failure to Function
as a Good Learning Organization
“Shuttle management declined to have the crew
inspect the Orbiter for damage, declined to request
on-orbit imaging, and ultimately discounted the
possibility of a burn-through.”
“The Board views the failure to do so as an illustration
of the lack of institutional memory in the Space
Shuttle Program that supports the Board’s claim…
that NASA is not functioning as a learning
organization.”
CAIB Report (2003) Section 6.1, Page 127
Reasons: Too busy, low value of typical KM activities, bureaucracy
G O D D A R D
S P A C E
F L I G H T
C E N T E R
7
Find the Leverage Point in the Equation
Organization
Interaction
Coefficient
Openness
& Sharing
Loss of
Human
Capital
Inflow of
Human
Capital
Opportunity
& Tools
(
Rate of
Knowledge
Discovery
Innovation &
Creativity
G O D D A R D
S P A C E
- )=
Customer
Need
Focus
Rate of
Knowledge
Loss
Potential
Knowledge
Utilization
Knowledge
Decay
Project
Execution
F L I G H T
C E N T E R
8
Translating KM for NASA
• Reliable Project Processes
1.
2.
3.
Team make up (members) should be independent of quality of
knowledge applied to the project
Organizational communication processes must reduce risk for the
project
Knowledge flow loops must be shorter than project life cycles to
facilitate timely learning
• Sustainable Competency Base
1.
2.
3.
Replenishing of social networks to support knowledge sharing as
knowledge expertise ages
Knowledge sharing legacy systems adapted to modern work
structures, protocols and practices
Effective mentoring and knowledge sharing to attract and retain
the future competency base for the Agency
G O D D A R D
S P A C E
F L I G H T
C E N T E R
9
Address KM as a Learning Challenge
•Lessons are learned when we reflect on our
experiences
– If we don’t stop and reflect: we generally don’t learn much
– Reflection helps us overcome near-miss-bias: see HBR, April 2011
– Helps us learn from what we did right
– The best time to reflect is right after the experience
•Rules are a reflection of what we think will work
– Often derived from failures
– Tested and flown is the standard
“Civilization advances by
extending the number of important
operations which we can perform
without thinking about them.”
Introduction to Mathematics
Alfred North Whitehead, 1911
G O D D A R D
S P A C E
F L I G H T
C E N T E R
10
Must Understand System Tensions
G O D D A R D
S P A C E
F L I G H T
C E N T E R
11
Must Understand Our Success Model
Responsible
Leadership
Open
Discussion
& Debate
Rigorous
Engineering
Requirements
Management
Risk
Management
G O D D A R D
S P A C E
F L I G H T
C E N T E R
12
Must Augment How We Do Learn
G O D D A R D
S P A C E
F L I G H T
C E N T E R
13
Simply, Start Smart, Stay Relevant
Early Learning Zone
Formal LL Zone
G O D D A R D
S P A C E
F L I G H T
C E N T E R
Draw a Picture and Talk It
G O D D A R D
S P A C E
F L I G H T
C E N T E R
15
Write a Learning Plan and Revise It
Building the Goddard Learning Organization:
A strategic plan for managing our collective knowledge and changing
our culture to help GSFC function more like a learning organization
A learning organization facilitates the
sharing of knowledge among people
as much as among systems.
The Challenge to Change
The Need for a Plan to Manage
Knowledge and Build a Learning
Organization at NASA has been
highlighted in a number of official
documents. This Plan for GSFC is
in direct response to those
challenges and builds on the draft
Agency KM strategic plan1.
Goddard desires to become the
learning organization NASA needs
to be in order to carry out the next
generation of space exploration.
“The Goddard Plan is designed to overcome the
previous Agency focus on IT as a KM driver with its
over-emphasis on capturing knowledge from
The Goddard
Planorganization
is designed to overcome
the previous
Agency
workers
for the
and
instead
focuses on
focus on IT as a KM driver with its over-emphasis on capturing
knowledge
from workers forsharing
the organization
and instead
facilitating
knowledge
among
workers.”
focuses on facilitating knowledge sharing among workers.
p5 of draft Goddard Learning Plan
Goddard must not sit by
expecting our successes
of the past to carry us
through the times ahead.
Future Goddard projects should
never accept risk or experience
failure because the organization did
not apply its own best knowledge.
Goals of Learning Plan
1. Manage Knowledge
Assets Efficiently
2. Facilitate Effective
Knowledge Use
3. Build a Learning
Organization Culture
1.
2.
3.
4.
5.
6.
Learning Practices
Pause and Learn
Sharing Workshops
Case Studies
Lessons Learned
Case-Based Training
Experience-Based
Design Rules
1
Strategic Plan for Knowledge Management, NASA Knowledge Management Team, April 2, 2002 (unsigned draft
document) available on the NASA KM website at: http://www.km.nasa.gov/home/index.html
G O D D A R D
S P A C E
F L I G H T
C E N T E R
16
Pause And Learn Sessions (PAL)
• Pause and Learn is a team based reflection activity for the
team/group to process their own learning.
• Five key parameters:
– Non-attribution environment
– Not a reporting activity
– Close to the action (work level)
– Participant discussion
– Objective facilitator
• Inserted into process
– After major reviews/gateways
– After incidents or mishaps
– When team seems at risk
• Participants have all expressed the benefit of the process to them
and their teams.
G O D D A R D
S P A C E
F L I G H T
C E N T E R
17
G O D D A R D
S P A C E
F L I G H T
C E N T E R
18
Case Studies
• We write good reports when things go wrong but,
we don’t write much when things go right and
we don’t write anything when we were just lucky.
• Case Studies are a channel for learning
– Collects ambiguous wisdom into a context
– Opportunity for protagonists to share their stories
– Proven methodology for hundreds of years
• We need to be better at examining why things went right and not just
assuming it was always because of our great planning and execution.
• Cases help overcome a reticence to discuss things that were less than
perfect but still worked which are a great learning opportunities.
“In fact, most people learn from others,
learn by doing and learn from stories. …
Most training is almost worthless.”
Interview with E-Learn, Larry Prusak, 2008
G O D D A R D
S P A C E
F L I G H T
C E N T E R
19
Case Study Products
Write Them
Teach Them
Publish Them
Wrap Them
G O D D A R D
S P A C E
F L I G H T
C E N T E R
20
The Case
Study
Case Study Best Practice
Participan
ts
Facilitato
r
Protagonis
t
Teaching
Note
Exercise
Handout
Video
Clips
G O D D A R D
S P A C E
F L I G H T
C E N T E R
21
Knowledge Sharing Workshops
•
•
•
•
•
•
Usually a project team (instrument, engineering, mgmt. science etc.)
– Share in a panel discussion format their personal experiences
– Keep it focused on “What I learned”
Sometimes use a case study or draft case study to spark discussion
Facilitated by a Moderator (usually the CKO)
Lasts about 2 hours
Sometimes needed after an incident/mishap etc. to get the real story
Help bring out personal lessons
“If I hadn’t had to prepare for that workshop, I wouldn’t
really have figured out what I learned from that mission.
We must be learning from all our missions. “
Project Scientist at GSFC
G O D D A R D
S P A C E
F L I G H T
C E N T E R
22
The Road to Mission Success
Workshop Series Objectives
1.Engage in dialogue with senior management on “Mission Success: The
Way Goddard Does Business”
2.Articulate a clear and consistent statement of the rules, processes, and
values that contribute to Goddard’s success with flight missions
3.Identify the array of
support mechanisms in
place at Goddard
4.Expand and empower
the cadre of existing and
potential leaders and
managers within Goddard
G O D D A R D
S P A C E
F L I G H T
C E N T E R
23
Goddard Design Rules
A search on the term “FPGA” will yield:
FPGA
GOLD Rule 2.10 – Electronic Design
Link to specific NASA LLIS data
Link to klabs.org page on FPGA design
Link to JPL Blue Book FPGA rules
G O D D A R D
S P A C E
F L I G H T
C E N T E R
Link to specific MIL-STD
Link to relevant case study
24
What Have I Learned?
“It’s not what you don’t know that will kill
you.
It’s what you think you know that just isn’t
so.”
G O D D A R D S P A C E
F L I G H T C E N T E R
Mark Twain
25
Learn from Decisions: Not Just Outcomes
Success
No
Lessons
Learned
Mishap
Investigation
with Technical
Lessons
Learned
G O D D A R D
Wrong
Decision
Right
Outcome
Right
Decision
Right
Outcome
Wrong
Decision
Wrong
Outcome
Right
Decision
Wrong
Outcome
No
Lessons
Learned
Wrong
Lessons
Learned
Failure
S P A C E
F L I G H T
C E N T E R
26
Good Organizational Depth Perception
If people have a good grasp of what
the organization is all about they will
be more likely to openly share and
communicate with each other.
I can see how what I know matters to others.
Leader led training !
G O D D A R D
S P A C E
F L I G H T
C E N T E R
27
Open Communications
If people are satisfied with the
communication systems and processes in
place they will be more likely to openly
share and communicate with each other.
If I speak something happens.
Facilitated discussions !
G O D D A R D
S P A C E
F L I G H T
C E N T E R
28
Reward Fairly (and Punish Fairly)
If people perceive the organizational
employment game to be fair and open
they will be more likely to openly share
and communicate with each other.
If I work hard, it matters.
Career plans and IDPs !
G O D D A R D
S P A C E
F L I G H T
C E N T E R
29
Most People Love to
Learn
Many Fewer Want to Be
Taught, Told or
Controlled
Inspire People with
Knowledge and You Will
Live Long and Prosper
January 22, 2008
G O D D A R D
S P A C E
F L I G H T
C E N T E R