Advances in Group Model Building
Reflections on recent work with Colin
Eden and Fran Ackermann of Strathclyde
University
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
1
Outline of Remarks
• Part I: Context of the study
• The TSA Aviation Security Simulator
• The Emerging TPI Approach
• Part II: What Happened Last Week?
• Logistics
• Scripts
• Part III: Discussion
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
2
Context: TSA Aviation Security Simulator
• Contract Between Transportation Security
Administration and Argonne National Labs
• Argonne, Sandia, and Los Alamos as part of TriLabs collaboration
• UAlbany as “special teams” subcontractor for
Group Model Building
• Eden and Ackermann invited to expand team
• Some material in this study is SSI
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
3
Context: The Emerging TPI Approach
• At UAlbany, our work grew out of DTG (with thanks
to John Rohrbaugh)
• Nearly a decade of cooperative discussions
• Joint Seminar at Albany, Humphrey Institute, and
Strathclyde University
• Recent paper in JPART
• Cooperative work with British Health Service in
Peebles, Scotland: January 2007
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
4
Ancient History:
Policy Resources in the Welfare Reform Sessions
•
•
•
•
Prevention
Child support enforcement
Case management & assessment
TANF services
• Employment services, child care, drug treatment, $
•
•
•
•
Diversion services
Self-sufficiency promotion
Safety net services
...all aggregated up from detailed resources...
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
5
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
6
An Example of a Resource Cluster:
Employment Services to Families on TANF
• Education & training slots
and referrals for jobs
• Substance abuse & mental
health treatment
• VESID
• Workfare and emergency
services
• Job readiness programs
• DOL & JTPA & private
Rockefeller College of Public Affairs and Policy
University at Albany
• Transportation
• Federal dollars for training
(JTPA)
• Moneys for grant diversion
• Transitional Medicaid
• Licensed day-care and other
child care
• Establish paternity & child
support
DF Andersen & GP Richardson
February 2007
7
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
8
Logistics: Plan for the First Half of Day One
•
•
•
•
•
•
•
Start at 8:00 AM
Initial Issue Identification
Stakeholder Issue Identification
Initial Policy Ranking
BREAK
Graphs Over Time
LUNCH
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
9
TSA Overview schedule
DAY 1
Public agenda
Time
Topic
7:50 Assemble
Team agenda
Time
Topic and roles
8:00 Coffee logistics
8:00 Introduction
8:30
Problem definition
and model boundary
Opening introduction: TSA
head of screening, Michael
8:05
Samsa from Argonne.
Introductions of participants
8:30
Issue surfacing and
structuring
9:20 Stakeholder views
What's in and out of analysis
9:45 to make it believable and
useful?
Policy priorities - preliminary
10:00 view from different
stakeholder perspectives
Break
10:15 Break
Dynamic perspectives
10:30 Graphs over time
Details
All the technogy works!
Orders for the break
DA facilitates & organizes.
Name tags, name tents
Pairs for DE set up and
arranged. FACE structure and
facilitate
FACE using DE/GE
FACE using DE/GE
FACE using DE/GE
Andrew, Steve organize
lunch with the group
DAGR works the group. FACE
takes notes in DE.
FACE preferencing on the
variables and review
11:45
elaboration around the
variables
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
10
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
11
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
12
Initial Issues identified
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
13
Issues from Stakeholder Perspectives 1
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
14
Initial Policy Priorities
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
15
Graphs over time drawn
by the participants
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
16
More graphs over time
drawn by the participants
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
17
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
18
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
19
Graph over time ONE (training)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
20
Logistics: Plan for the Second Half of Day One
•
•
•
•
•
•
Concept Model
Elicitation of Model Structure
Modeler Feedback
BREAK
Scenarios
END at 4:15 PM
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
21
Schedule for afternoon of the first day
12:00 Lunch
1:00 Concept model
Preliminary model
structuring
12:00 Lunch
1:00 Concept model
1:30
Model structuring I linked to
DE issues map
2:45 Modeler feedback
3:00
Steve extracted dynamic
variables; George desgined
the stock-and-flow backbones
for TSOs and Technology and
Skills.
GR & IM
DA brought the variable pack
to the board. GR began the
stock-and-flow backbone for
TSOs and facilitated the initial
group elaboration of that. DA
facilitates & structures; GR
supports; FACE annotates if
possible
GR presented; asked for
group acceptance; diagrams
reproduced for handouts, with
DE maps
3:00 Break
Scenario events
3:25 Scenario events and stories
FACE
4:00 End
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
22
Concept Model
Desired
personnel
Hiring rate
Part time
screeners
Part timers
quitting
Total
personnel
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
23
Concept Model
Desired
personnel
Hiring rate
Total
personnel
Part time
screeners
Part timers
quitting
Promoting
to full time
Desired full time
screeners
Fraction full time
Full time
screeners
Rockefeller College of Public Affairs and Policy
University at Albany
Full timers
quitting
Frac fulltimers
leaving p month
DF Andersen & GP Richardson
February 2007
24
Concept Model
Desired
personnel
Hiring rate
Total
personnel
Part time
screeners
Part timers
quitting
Dissatisfaction
leading to quits
Time to become
full time
Promoting
to full time
Desired full time
screeners
Fraction full time
Full time
screeners
Rockefeller College of Public Affairs and Policy
University at Albany
Full timers
quitting
Frac fulltimers
leaving p month
DF Andersen & GP Richardson
February 2007
25
Concept Model Behaviors
Part time screeners
Full time screeners
60,000
40,000
30,000
20,000
0
0
0
15
30
Time (p month)
45
Part time screeners : TSA1
Hiring rate
Part time screeners : TSA2
Part time screeners : TSA3
60
people
people
people
20,000
0
15
30
Time (p month)
Full time screeners : TSA1
Time
Full time screeners
: TSA2to
Full time screeners : TSA3
45
become full timepeople
people
people
60
10,000
30
0
0
0
Hiring rate : TSA1
Hiring rate : TSA2
Hiring rate : TSA3
15
30
Time (p month)
45
60
people/month
people/month
people/month
Rockefeller College of Public Affairs and Policy
University at Albany
0
15
60
30
Time (p month)
Time to become full time : TSA1
Time to become full time : TSA2
Time to become full time : TSA3
45
60
month
month
month
DF Andersen & GP Richardson
February 2007
26
Eliciting Model Structure—What we did
• “Seed” for elicitation was backbone stock and flow
structure from Concept Model as elaborated by group
• Used “variable” pack available from “key variable” list
made up in the morning
• Ability to link model structure to Group Explorer
explicitly through variable numbers
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
27
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
28
Eliciting Model Structure—What we should do
next time
• Use Stakeholder goals and sanctions exercise to generate
feedback kernels and “seeds”
• Use Decision Explorer to generate a list of key model
variables for inclusion
• Experiment with ways to more tightly link DE and
Vensim Maps
• Explore further “fusion” of methods
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
29
Modeler Feedback
• A Standard part of our Group Model Building
• Completed using ordinary overhead projector
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
30
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
31
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
32
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
33
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
34
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
35
Scenarios
• Group Explorer used to begin elicitation of scenarios
• Process returned to on second day
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
36
Scenario: TSA rapidly create processes to innovate (R=relative impact, G=relative probability across all scenario events)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
37
DAY 2
Public agenda
7:50 Assemble
8:00
Review day 1, intro
day 2
Model structuring
10:15 and
Team agenda
7:50
Get acceptance of refined
8:00 TSO structure map; revise
as necessary
Build customer sector on
8:35
back board
10:15 Break
10:30 Modeler feedback
System Goals
12:00 Lunch
1:00 Model structuring
and
10:45 Goal structure
12:00 Lunch
Airport, Customer,
HeadQuarters structure
2:30 Break
1:00
2:45 Modeler feedback
Policy priorities
Takeaways on issues,
3:00 goals, priorities, scenarios,
and policies
Next steps: TSA, Tri-Lab
3:30
Team
3:55 Wrap up
GR facilitates?
DA facilitates. FACE
annotates
GR on Customer Sector,
linkages to TSO sector
FACE
DAGR plans next modeling
piece
DA facilitates. FACE
annotates
GR on Goal structure
and/or final model
sector(s)
FACE
Michael Samsa, Andrew
Cox
DA, GR, FA, CE, …
4:00 End
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
38
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
39
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
40
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
41
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
42
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
43
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
44
Final Policy
Priorities College
(red=short
term, green=long
Rockefeller
of Public
Affairs and term)
Policy
University at Albany
DF Andersen & GP Richardson
February 2007
45
Discussion
• Stages in the Development of TPI
• Curiosity (both sides work with facilitated
group, computers, and word-and-arrow
diagrams)
• Cooperative Sharing
• Limited Assimilation
• Integration (where we are now)
• “Fusion” of Approaches: a goal?
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
46
What Fusion Might Mean
• Duality of Vensim and Decision Explorer Maps
• Seamless approach to client groups
• Ability to “zoom lenses” between micro and
macro views
• New support for model formulation and
documentation
• New products that enhance value to clients
• Eventually perhaps integrated software suites
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
47
Thank You for Your Attention
Questions and Comments
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
48
You really don’t want to go beyond this
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
49
Issues to be addressed to ensure model is believable and useable
(ranked from most important to least, {5}=must include)
25 ** Human Factors - ability to detect IEDs using technology{5,0}
221 ** staff efficiency {4,9}
123 ** 13 consistency in interpretation and application of sops {4,8}
141 ** increase in training {4,7}
11 ** Throughput {4,6}
138 ** reduce attrition of "good" screeners to retain security knowledge {4,6}
105 ** [criminals] intent on deceiving screening {4,5}
12 * improve person to person communication between TSO and passengers {4,4}
102 * [ aviation system]safety of aircraft {4,3}
107 * 1 & 2= Communicate, communicate, communicate!!! passengers {4,2}
15 * Maintain customer service {4,2}
51 * Lack of conduit for best practices and/or information sharing {4,0}
32 keeping costs reasonable {3,8}
118 [criminals] predictability {3,8}
133 11 willing to die for cause in completing the mission {3,6}
34 shift focus from finding things to identifying hostile intent in people {3,4}
50 distinguish between airport and aviations security {3,1}
64 foster the mindset of investigative scepticism {3,1}
86 [airports mgt] airlines satisfied {3,1}
120 1 no profiling {3,1}
95 [politicians] need for re-election {2,1}
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
50
Likely SD variables (involved in feedback)
derived from Group Explorer gatherings
16 Avoid repetitive activity boredom
17 cooperation between airport authority,FSD, local LEO
18 empowerment for STSOs and LTSOs
19 Ability to manage systematic integration of technology and people
22 Passenger awareness of process and expectations
24 checkpoint & baggage communication
32 keeping costs reasonable{3,8}
36 realistic scheduling based on pax loads
40 better federal cooperation at the airport
43 effective deployment of staff
44 reaction procedures to security threat
52 proper relationship with airport stakeholders
54 potential conflicts with local law enforcement
57 deployment of leos so to enable immediate response of unruly passengers
58 maintaining screener interest in job function
64 foster the mindset of investigative skepticism{3,1}
65 reduce line waits
67 [airports mgt] want all available lanes open all the time
69 [airlines] airlines would like to see screening efficiency increase such that there is no wait time at the
checkpoint
71 16 enough TSo's to eliminate fatigue
77 airlines would like the checkpoint experience to very pleasant and polite--No anxiety
80 14 screening procedures change too often
81 16 burn out
99 17: reducing hassle to passengers
104 [airlines] reducing passenger fear
113 [airlines] providing high levels of service to frequent/first class travelers
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
51
Most central themes (lots of links in and out for 3 levels) and summary links between them
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
52
Stakeholder Goals as a System (red=media; LEO=gray; maroon=airport; green=airlines; blue=passengers; teal=screeners; olive=politicians)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
53
Issues from Stakeholder Perspectives 2
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
54
Scenario: IED explosion (R=relative impact, G=relative probability across all scenario events)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
55
Scenario: chemical or biological attack on airport (R=relative impact, G=relative probability across all scenario events)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
56
Scenario: employee based attack (R=relative impact, G=relative probability across all scenario events)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
57
Scenario: remaining material (R=relative impact, G=relative probability across all scenario events)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
58
Graph over time TWO (staffing)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
59
Graph over time THREE (IED’s)
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
60
Stakeholder Responses to Security System Failure
Rockefeller College of Public Affairs and Policy
University at Albany
DF Andersen & GP Richardson
February 2007
61