Project Management Technique
Simulation Modeling Analysis
By:
Jackie Holohan
&
Penny Leahy
December 7, 2000
Simulation and Modeling Analysis
Fall 2000
Professor Ernesto Gutierrez-Miravete
1
Table of Contents
Topic
Executive Summary
1.0 Introduction
1.1 Purpose & Performance Metrics
1.2 Scope
1.3 Target Audience
1.4 Application of Simulation
1.5 Flow Diagram of PMT
1.6 Data Collection
2.0 Model Details
2.1 Locations
2.2 Entities
2.3 Path Networks
2.4 Input Factor & Processing
2.5 Model Assumptions
3.0 How to run the Model
4.0 Model Results
4.1 Simulation Options
4.2 Example Problem to Solve
4.3 Analysis of Results
4.4 Confidence Intervals
4.5 Conclusions from Example
5.0 Model Verification & Validation
5.1 Verification
5.2 Validation
6.0 Conclusion
References
Page Number
3
4
4
5
5
5
6
6
8
8
15
15
17
20
21
22
22
22
28
30
31
31
31
31
34
35
2
Executive Summary
Currently project managers in Information Technology are leading projects using Project
Management Technique, a checklist of project tasks to complete a project. Often,
projects are delivered late or over budget because hasty estimates are made with respect
to the schedule and cost estimates.
The goal of this project is to create a model of managing a project. The model can be
tailored based on project complexity and experience level of the team members. The
number of projects completed per time period to be studied can also be altered. The
output of the model is the time to complete the project based on the input specified
(complexity and experience level of the team). Cost can be calculated based on the time
estimate.
The model will enable both the 30-day project manager and 30-year project manager to
make an educated guess for the schedule and budget. Project goals will be met and
proper expectations will be set for upper management and customers.
3
1.0 Introduction
There are several Information Technology (IT) challenges existing in business today
varying in levels of complexity, duration and experience level of the resources. In order
to increase the productivity and decrease variability in IT project management, businesses
are designing methodologies to be used through the entire project. Project Managers need
to meet their schedule and budget targets; therefore, they should be given sufficient time
ad funds to do so. Generic procedures for project management are defined, including
standard tasks and checkpoint reviews. These checkpoints ensure the project is
proceeding correctly and will deliver a quality solution that customers want. Assuming
that all projects delivered will meet the success criteria while using PMT, two
fundamental questions IT professionals need to answer while managing a project are:
 How much will this project cost?
 How long will the project take using these resources?
1.1 Purpose
The purpose of this simulation is to develop a tool that can be used by IT professionals to
assess the cost and time requirements of a project. It is based on a five-step methodology
called Project Management Technique (PMT). Each step includes necessary project
tasks, as well as a checkpoint reviews. The simulation begins when the user specifies the
following:
 Number of people and size of the project
 Complexity of the project
 Experience level of the resources
Performance Metrics:
The simulation will help to estimate the following:
 The amount of time to complete the 5 steps of a project.
 The cost to complete the project can subsequently be calculated based on
the time applied.
The results will help IT professionals make educated decisions about how to proceed with
the project. Unreasonable guesses of project completion time and cost will be minimized.
Instead, project leaders can be set up for success, given reasonable goals and targets. This
simulation will help businesses create strategies, prioritize projects and plan resource
allocation. The number of people, which people and scope of the project, can be
optimized at the beginning. In addition, it will reduce idle time of resources (i.e. money,
employees, etc.), enable people to complete realistic project plans and set a project team
up for a successful delivery of a desired outcome.
4
1.2 Scope
The topics/tasks that are in scope are:
 Information Technology projects using PMT methodology
 Estimate time to complete all project tasks in the PMT methodology
 Model should estimate the time to complete each phase in the PMT, as well as time to
complete the entire project
 Cost can be determined based on the time to complete the project
The topics that are out of scope are:
 All projects outside Information Technology
 All projects in IT not using the PMT methodology
1.3 Requirements
The requirements of this project, given by project leaders at GE, are:
 Gather time data for each project task in PMT. Use the time in the simulation in the
processing of project task times.
 Consider projects of various size, complexity, number of people on the team and
experience levels on the team.
 Develop a model to simulate the amount of time to complete a project, as well as the
time to complete each project task and phase.
 Allow the user to deduce the cost to complete a project.
 Be sure all bugs are worked out through the verification process.
 Validate the model with real-world data from the IT department at GE Indusrial
Systems.
1.4 Target Audience



IT Project Managers. The simulation will provide accurate cost and time estimates
at the time of project conception.
IT Project Team Members. This simulation will be a visual guide to project
management and act as a training tool.
Upper Management in the IT Organization. Upper Managers will be able to
estimate the number of projects that can be done in a year. Planning for budgets and
resources on a large scale will be feasible.
1.5 Application of the Simulation





Prediction of cycle time and reduction of cost for IT project development work
Prioritization of tasks in PMT and improvement of project quality
Planning of work-flow for the entire project life cycle
Analysis of throughput (i.e. number of projects which can be completed per year)
Estimation of the optimal number of resources given project complexity, size and
experience level of the team
5
1.6 Basic Flow Diagram of PMT
The Project Management Technique is designed to help an individual, or a team, navigate
through a project. The following flow chart displays the 5 phases of PMT. Once the
required tasks are completed in each phase, the team meets with managers and
stakeholders for a project update in a checkpoint review. The individual modules of PMT
are discussed in the following sections.
Figure 1.6 Project Management Technique Process Flow
Project Management Technique (PMT) Process Flow
C
Control
C
6
Checkpoint
I
Create
Checkpoint
B
Investigate
Checkpoint
Baseline
Checkpoint
S
Checkpoint
Scope
1.6 Data Collection
18 managers, each experienced with PMT, were surveyed to find out the time statistics of
each project task in different project scenarios. In order to allow the user of the
Simulation model to study projects ranging in complexity and resource experience, an
input factor (described in detail in section 2.4) was employed to change the task times
depending on the project the user is studying.
The basis of this input factor is in Figure 1.7. It was gathered from a group of highly
experienced IT professionals who lead a project with low complexity. This type of
project was chosen as the basis because it has the least variability, being a project with
low complexity. In addition, this project would be run optimally considering the high
team experience.
18 project leaders, ranging from various levels of experience, were asked the following
for each of the project tasks:



Based on your team's experience level, how long did it take to complete each project
task most of the time?
Based on your team's experience level, what is the maximum amount of time it takes
to complete each project task?
Based on your team's experience level, what is the minimum amount of time it takes
to complete each project task?
Figure 2.4d shows the responses to the above questions as given by 18 IT managers.
These values were used to determine the adjustment factor (outlined in section 2.4).
The type of project teams considered were:
 Teams with little project management technique experience (0 - 1 years)
 Teams with moderate project management technique experience (2-5 years)
 Teams with strong project management technique experience (>5 years)
The feedback from each of the 18 project leaders was analyzed, validated and fitted in the
simulation model as a triangular distribution. The triangular distribution is used because
it provides a rough model in the absence of data. The 18 project managers estimated the
minimum, maximum and mode (most likely value) for each PMT phase. Although the
triangular distribution will create more variation than the true distribution, and the
extreme values will not be captured, there isn't enough data to estimate which type of
distribution each task takes. The extra variation is acceptable for the purposes of this
simulation.
7
Figure 1.7: Project Task Time Data (In Hours)
High Experienced Resources working with a Low Complexity Project
Phase
Task
Max
2
1
3
2
3
4
5
6
7
Evaluate Alternatives/Risk Assessment
Cost/Benefit Analysis
Work/Resource Management
Quality Plan /Customer Needs
Architecture/Requirements
Training (Project Team)
4
3
8
5
8
16
2
1
4.5
3
6
8
6
4.5
10
7
10
24
Baseline
8
9
6
10
Project Management
Process Maps (As Is)
Architecture/Requirements
Resources/Cost Benefit/Schedule Status
16
8
8
5
14
6
6
1
18
10
10
7
Investigate
8
9
11
12
6
13
14
Project Management
Process Maps (To Be)
Ranking of Defects/Designing Solution
Support Plan
Architecture/Requirements
Test Scenarios
Peer Reviews
16
8
24
4
8
24
2
14
6
20
3
6
20
1
18
10
32
5
10
28
3
Create
8
15
13
12
14
16
Project Management
Development of Solution
Test Scenarios
Support Plan
Peer Reviews
Prototype
16
24
24
4
2
12
14
20
20
3
1
8
18
36
28
5
3
16
Control
8
17
7
14
18
19
20
Project Management
Finalize Development/Testing
Training (User Groups)
Peer Reviews
Contingency Plans
Lessons Learned
Team Celebration
16
8
16
2
3
1
4
14
6
8
1
2
0
0
18
16
24
3
4
2
16
Scope
1 Mission/Problem Statement
Mode Min
8
2.0 Model Details
The following sections outline the theory behind the PMT simulation model. The task
processing times used in the model are based on the data gathered from 18 projects.
Section 2.4 describes how the user can adjust the model based on the complexity level of
the project and the experience of the team. The simulation model consists of 20 locations
and 1 entity, described below.
2.1 Locations
There are 20 locations represented in the simulation by a yellow box and a code number.
Locations are the project tasks to be completed at each project task phase. The locations
belong to the "common task" phase or specifically to one of five core modules including:





Scope
Baseline
Investigate
Create
Control
The Figure 2.1 displays the tasks associated with each PMT phase. The task code
numbers are included since they are used to represent the tasks in the simulation model
layout.
The "common task" section of the chart contains project tasks to be completed at more
than one of the phases. For example, task C.3 (Project Management) needs to be
completed at the Baseline, Investigate, Create and Control phases.
9
Figure 2.1 Locations and Codes
Module
Scope
Code
S.1
S.2
S.3
S.4
S.5
Baseline
B.1
Resources/Cost Benefit/Schedule Status
Investigate
I.1
Ranking of Defects/Designing Solution
Create
Cr.1
Cr.2
Control
Co.1 Finalize Development/Testing
Co.2 Contingency Plans/Lessons Learned
Co.3 Team Celebration
Common
Tasks
C.1
C.2
C.3
C.4
C.5
C.6
C.7
Task Name
Mission/Problem Statement
Evaluate Alternatives/Risk Assessment
Work/Resource Management
Cost/Benefit Analysis
Quality Plan /Customer Needs
Development of Solution
Prototype
Architecture/Requirements
Training (Project Team)
Project Management
Process Maps (To Be)
Support Plan
Test Scenarios
Peer Reviews
There is also a location called "Checkpoint Review", in which the team meets with
management to describe the progress of the project. This meeting is held when all of the
required tasks are completed for each of the five core modules.
10
2.1.1 Scope
The first section of Project Management Technique is the scope. During this step the
project team defines the mission statement, evaluates risks, alternatives and resource
needs, and prepares the team for the project with necessary training. The following chart
outline the tasks required for completion of the scope module. Please note, the mode,
minimum and maximum values are in hours.
Figure 2.1.1 Scope Requirements
Code
S.1
S.2
S.3
S.4
S.5
C.1
C.2
Task Name
Mission/Problem Statement
Task Requirements
Define Objective, Scope &
Approach Plan for the project.
Evaluate Alternatives/Risk Assessment Define High Level Risks
Risk Abatement Procedure
Evaluate Alternative Designs
or Software Packages
Work/Resource Management
Identify Project Approach
Progress Report. Determine
Detailed Work Tasks, Who is
Responsible and
Cost/Benefit Analysis
Cost Analysis and Benefit
classification
Quality Plan / Customer Needs
Identify Customer
Requirements
Quality Scorecard
Architecture/Requirements
Define & Document Project
Requirements
Define & Document
Architecture Requirements
Training (Project Team)
Project Required Training
Min Mode Max
1
2
3
2
4
6
4.5
8
10
1
3
4.5
3
5
7
6
8
10
8
16
24
TOTAL HOURS 25.5
TOTAL DAYS 3.188
46
5.75
65
8.1
Each task is assigned a code, used so that the task can be identified in the simulation
model layout. The minimum, maximum and mode service time, listed above in hours, is
based on a triangular distribution. This is because of the natural large variation in project
task times and the lack of sufficient data.
11
2.1.2 Baseline
The second module in the PMT process is called baseline, in which the team performs
background studies on the project. The "As Is" business and information technology
process is studied. There are 4 major tasks listed below that need to be completed before
the checkpoint review.
Figure 2.1.2 Baseline Requirements
Code
Task Name
C.3
Project Management
C.4
Process Maps (As Is)
C.1
Architecture/Requirements
B.1
Resources/Cost Benefit/Schedule
Status
Task Requirements
Min Mode Max
Team Meeting
14
Project Plan
Progress Report
Resource Management
Quality/Risk Abatement Plan
Cost/Benefit Analysis Updated
Identify Financial Structure
6
Benchmark Current Processes
Define Business Requirements
Process Map Summary
Review & Document Project
6
Requirements. Obtain sign-off
Review & Document
Architecture Requirements.
Obtain sign-off
Modify Schedule/Resources
1
Update Cost Benefit Analysis
TOTAL HOURS
27
TOTAL DAYS 3.375
16
18
8
10
8
10
5
7
37
4.63
45
5.6
12
2.1.3 Investigate
During the investigation phase of PMT, the team begins designing the solution and
developing the test scenarios. Alternatives of the "To Be" solution is designed; options
are investigated. There is also a step called Project Management that includes time for
team meetings and administrative tasks. The following tasks must be completed before
the checkpoint review meeting.
Figure 2.1.3 Investigate Requirements
Code
C.3
C.4
I.1
C.5
C.1
C.6
C.7
Task Name
Project Management
Task Requirements
Min Mode Max
Team Meeting
14
Project Plan
Progress Report
Resource Management
Quality/Risk Abatement Plan
Process Maps (To Be)
Develop Future Process Maps
6
Ranking of Defects/Designing Solution Rank the defects
20
Design a solution without
defects
Support Plan
Specify Document Needs
3
Document Prototypes
Architecture/Requirements
System Availability
6
Future Application Deployment
Test Scenarios
Develop Test Scenario Model
20
Develop a Test Strategy
Peer Reviews
Conduct structured walk1
through meetings to review the
code or design documents
TOTAL HOURS
70
TOTAL DAYS 8.75
16
18
8
24
10
32
4
5
8
10
24
28
2
3
86 106
10.8 13
13
2.1.4 Create
In the forth phase of PMT, the team begins building the solution and finalizing the test
strategy. They also take time to do peer reviews, to gather feedback from the team on the
project details. The plan to support the application post launch is also created. The
following 6 tasks in Figure 2.2.4 must be completed before the checkpoint review can
occur.
Figure 2.1.4 Create Requirements
Code
Task Name
C.3
Project Management
Cr.1
Development of Solution
C.6
Test Scenarios
C.5
C.7
Support Plan
Peer Reviews
Cr.2
Prototype
Task Requirements
Min Mode Max
Team Meeting
14
Project Plan
Progress Report
Resource Management
Quality/Risk Abatement Plan
Design Database
20
Identify Security Profiles
User Process Narrative
Conversion Data Mapping
Setup Configuration
Develop Test Strategy
20
Develop Detailed Test Plans
Identify who will Test
Design Production Support
3
Conduct structured walk-through
1
meetings to review the code or design
documents
Create a prototype of the design and
8
pilot it to key users groups.
TOTAL HOURS
66
TOTAL DAYS 8.25
16
18
24
36
24
28
4
2
5
3
12
16
82 106
10.3 13
14
2.1.5 Control
During the last phase of PMT, the team installs the solution and runs the final test
scenarios. They also take time to train users and write a contingency plan. Upon
completion of the tasks the team participates in a celebration then gives a final report
during the last checkpoint review.
Figure 2.1.5 Control Requirements
Code
Task Name
C.3
Project Management
Co.1
Finalize Development/Testing
C.2
Training (Project Team)
C.7
Peer Reviews
Co.2
Contingency Plans/Lessons Learned
Co.3
Team Celebration
Task Requirements
Min Mode Max
Team Meeting
14
Project Plan
Progress Report
Resource Management
Quality/Risk Abatement Plan
Run Test Scenarios
6
Installation Test
Systems Integration Test
Test Report
Training Manual
8
Develop On-line Help
Conduct structured walk1
through meetings to review the
code or design documents
Determine plans for the
2
unexpected. Document lessons
learned and share with other
teams.
Celebrate hard work with the
0
entire team
TOTAL HOURS
31
TOTAL DAYS 3.875
16
18
8
16
16
24
2
3
3
4
4
16
49
6.13
81
10
15
2.2 Entities
The system has one entity -- the person or group of people working on the project. The
entity enters the system at the mission statement task in the Scope module. This is the
first task to complete in the project management life cycle. The arrivals are set at a
frequency of 4 (number of projects per year). The frequency of the projects can be
changed in the model at the discretion of the user by going to the ProModel menu option:
Build  Arrivals  Type in frequency number (This is the number of projects to be
completed in the time allotted).
2.3 Path Networks and Processing
When the entity enters the system, it starts at the mission statement task in the scope
module. It proceeds through the system, going to each task and stopping at the
checkpoint review after completion of each module (Scope, Baseline, Investigate, Create
and Control). Figure 2.3 shows an entity flow diagram that maps the entity as it works its
way through the PMT requirements.
16
Fig. 2.3 Entity Flow Diagram
S.3
S.4
S.5
C.1, C.2
S.2
Scope
ENTER
1
S.1
1
C.3, C.4,
C.1
B.1
Baseline
2
2
C.3, C.4
I.1
Investigate
3
C.5, C.1,
C.6, C.7
3
C.3
C.6, C.5,
C.7
4
C.3
Cr.1
Create
4
Cr.2
Co.1
C.2, C.7
Co.2
EXIT
Control
Co.3
Common Tasks:
C.1 C.2 C.3 C.4
C.5
C.6 C.7
Checkpoint
Reviews:
17
2.4 Input Factor and Model Processing
Input Factor
There are several things to consider when planning for a project. Simplification is used to
model the complexity of project management. This model considers 2 major factor when
identifying the type of project to run through the ProModel simulation:
 Team experience level
 Number of team members and complexity of project
This "input factor" depends on the team experience, number of team members and
complexity of the project.
The average experience level of the team is displayed in figure 2.4.
Figure 2.4a Experience Level Rating
Rating
3
2
1
Experience
Level
Low
Medium
High
Years using
PMT
0-1
2-5
>5
For the purpose of this simulation model, the number of team members and complexity
are rated in the following chart.
Figure 2.4b Duration and Complexity Rating
Rating
1
2
3
Complexity
Low
medium
High
People
1-2
3-4
5-6
Name
Small Project
Medium Project
Large Project
Once the user finds the appropriate rating from the above tables, he or she can choose the
input factor from the following chart.
Figure 2.4c Input Factor Table
Team Experience Level Rating
Duration
Or
Complexity Rating
Small
Medium
Large
1
2
3
Low
3
1
4
7
Medium
High
2
1
2
5
8
3
6
9
18
Section 3.0 describes how the user changes the input factor from the default value of 1.
To change the task times based on this input factor, the model uses the "adjustment
value" (specified in Figure 2.4d) corresponding with the input factor chosen by the user.
The task times, (listed in section 2.1) are multiplied by the below adjustment value. See
Appendix B for more detailed calculations.
Figure 2.4d Adjustment Factor
Input
Factor
1
2
3
4
5
6
7
8
9
Adjustment
Value
1.3
1.2
1.1
3.2
2.8
2.4
5.7
4.8
3.9
Model Processing
Based on the input factor, the service times at each of the locations (project tasks) are
multiplied by the according adjustment factor. Several "If...then…else" statements are
executed in the processing model in ProModel to adjust the service times accordingly.
The service times are adjusted to account for the user's complexity and team experience
level of the project. To summarize:



Small projects that have low complexity and a low experience level (input factor 1;
adjustment factor 1.3) will take longer to complete than small projects with low
complexity and a high team experience level (input factor 3; adjustment factor 1.1).
A large complicated project with a low team experience level will take the longest
(input factor 7; adjustment factor of 5.7).
A medium sized project with high experience on the team (input factor 6; adjustment
factor 2.4) will take less time than a medium sized project with low experience on the
team (input factor 4; adjustment factor 3.2).
19
Detail calculations behind the adjustment factor:
Figure 2.4d Data Collected from 18 Projects
Proj Input
# Factor
Scope
Baseline
Investigate
Mode Min Max Mode Min Max Mode
Min
Create
Control
Adjust
Factor
Max Mode Min Max Mode Min Max
1
2
1
1
AVG
72
50
61
30
38
34
90
78
84
45 40
53 30
49 35
70
46
58
100
124
112
90
94
92
149
131
140
100
120
110
85 150
87 125
86 138
70
62
66
40 110
42 112
41 111 1.316
3
4
2
2
AVG
60
54
57
30
34
32
84
74
79
45 39
47 27
46 33
52
60
56
100
114
107
88
80
84
120
143
132
105
93
99
87 122
73 132
80 127
68
56
62
35 104
41 97
38 101 1.227
5
6
3
3
AVG
45
47
46
30
21
26
60
70
65
33 22
41 32
37 27
40
50
45
88
84
86
50
90
70
114
98
106
90
74
82
63 107
69 108
66 106
30
70
50
35
26
31
7
8
4
4
AVG
145
152
149
88
76
82
212
210
211
115 86 140
123 88 152
119 87 146
280
272
276
220
230
225
350
337
344
267 210 344
259 212 336
263 211 340
150 90 266
170 110 272
160 100 269 3.225
9
10
5
5
AVG
120
140
130
77
67
72
190
174
182
100 77 125
110 75 127
105 76 126
250
232
241
196
200
198
277
318
298
222 173 400
240 199 200
231 186 300
144
140
142
89 240
85 230
87 235
2.82
6
6
AVG
114
110
112
70
52
61
190
126
158
70 70 130
108 62 90
89 66 110
205
209
207
170
166
168
300
217
259
190 157 255
210 161 257
200 159 256
177 50 155
65 100 242
121 75 199
2.42
13
14
7
7
AVG
200 100
328 192
264 146
334
412
373
200 150 255
226 158 259
213 154 257
499
483
491
400
404
402
650
563
607
460 388 670
480 366 537
470 377 604
286 175 465
288 181 483
287 178 474 5.727
15
16
8
8
AVG
270 120 312
175 126 318
223 123 315
150 130 250
206 132 186
178 131 218
440
400
420
400
380
390
506
518
512
299
291
295
310 510
330 508
320 509
290
200
245
100 405
200 396
150 401 4.808
17
18
9
9
AVG
190 150 259
172 50 249
181 100 254
181 100 254
140
152
146
146
300
380
340
340
250
300
275
275
412
420
416
416
300
340
320
320
256
261
259
259
195
199
197
197
125
116
121
121
11
12
112
100
106
106
177
175
176
176
412
416
414
414
20
77
89
83
1.1
330
317
324 3.925
324
Calculations for Adjustment Factor:
For input factor 1 ---(using the averaged times):

Each time was divided ( i.e. time for mode of scope) by the corresponding time in the
row for input factor 3.

Those values were summed 15 numbers in the row for input factor 1.

That total was divided by 15 to get the average multiplier or adjustment factor.
Example (for input factor of 1):
61/48 + 34/26 + 84/65 +………..+ 111/83 = 19.74
19.74/15 = 1.3
Note: The Adjustment Factor is 1.3 for Input Factor 1.1.
2.5 Model Assumptions
To narrow the scope of the project, the following assumptions were made during model
creation:









Between each location there is a .25 hour (15 minute) wait to account for breaks or
meetings.
The team members will be working full time on the project. There will be no "sideprojects".
The workweek is 40 hours per week.
The model is based on a 50 week-year (excluding holidays and vacations), working 5
days per week.
Only one type of project (duration, complexity and experience level) can be studied
during a simulation run.
The inter-arrival frequency of the projects is currently set, but can be adjusted as
needed by the user.
The model output is in time, but this can be converted to cost in dollars at the user’s
discretion using simple calculations.
Due to the ProModel student version limitation on the number of locations (causing
the use of common tasks) the output could not be divided to reflect the time needed
for each module of the project.
Checkpoint meetings are most often 1 hour, have a 30-minute minimum and a 1.5hour maximum. This service time for the checkpoint meeting is fit as a triangular
distribution model {t(.5,1,1.5)}.
21
3.0 How to run the model
1. Open PMT.mod (Project Management Technique ProModel File).
2. Choose the Simulation  Options menu option.
3. The PMT.mod should be set to run 10 replications of 2000 hours (1 years time = 40
hours per week * 50 weeks per year excluding vacation time). Change these options
if desired.
4. Press "OK" on the "Simulation Options" screen.
5. Verify arrival rates by choosing the Build  Arrivals. The frequency is currently set
at 4 (i.e. 4 projects arriving to the team in 1 year, or 2000 hours). Change these
options if desired.
6. Choose the Simulation  Model Parameters menu option.
7. Press the " Change" button on the "Model Parameters" screen.
8. Change the Input Factor to the desired setting (1-9) in the highlighted cells below.
This Input Factor represents the type of project you are simulating, with respect to
team experience rating ad the duration/complexity of the project.
Duration
Or
Complexity
Rating
Small
Medium
Large
1
2
3
Team Experience Level
Rating
Low Medium High
3
2
1
1
2
3
4
5
6
7
8
9
9. Press the "Run" button on the "Model Parameters" screen. This will run the
Simulation according to the Input Factor specified. Once the simulation has
completed, ProModel will prompt the user if he or she would like to see the results.
10. Press "Yes" to load the results.
11. A "General Report Type" screen will appear. You may want to choose to see "All"
replications to view the time to complete each of the project tasks in the replication.
This will also display the mean time of the 10 replications and the standard deviation.
22
4.0 Model Results
The model for completing a project has been built using ProModel Student 4.2, based on
the assumptions in section 2.6. See Appendix D for a view of the Model text details.
4.1 Simulation Options
The standard model is set to run for 2000 hours, or 1 year's time (40 hours per week * 50
weeks per year -- excluding vacation time and holidays)
10 replications are run, to obtain average values for the time to complete each task.
4.2 Example Problem to Solve using PMT.mod
To demonstrate and validate the simulation model, the following 3 scenarios were chosen
for testing:



Small complexity project with a low experienced team (Input Factor = 1)
Medium complexity project with a medium experienced team (Input Factor = 5)
Large complexity project with a highly experienced team (Input Factor = 9)
These 3 scenarios were chosen because they are the most likely situations to happen in
the real world. For example, a manager is more likely to assign a highly experienced
person to a more complex project. Teams with low experience are unlikely to be
assigned complicated projects.
4.2.1 Background For Example
A new Information Technology enhancement has been requested by one of the key
customers. The project entails changing the quoting and ordering software package to
include one of the new products just released. The IT project manager has several options
in planning for this enhancement request. The maximum amount of time allowed to
complete this project is 6 months.
1. Scope down the request to be a series of 3 small projects (i.e. with iterative
deliverables) with low experience levels on the team. New employees cost $40,000
per person year. The team only would have 2 people on the team.
This project will only deliver the customer "satisfiers". The business needs to do this
enhancement to stay competitive in the market.
2. Deliver the request as a medium project using moderately experienced people on the
team. The team would have 3 people on the team. Moderately experienced people
cost $65,000 per person year.
23
This project will only deliver the customer "satisfiers". The business needs to do this
enhancement to stay competitive in the market.
3. Combine the enhancement with the large project currently in progress with highly
experienced personnel. These employees cost $90,000 per person year. This project
will have 5 people on the team.
This project will deliver customer "satisfiers" and customer "delighters". The
business will gain market share and sales if this project is completed.
Figure 4.2.1 displays the output for the 3 simulation runs.
Figure 4.2.1 Output from Examples
Number of Employees
Number of Projects
Average Cost per Employee / year
Average Cost per day
Input Factor
Phase
Scope
Task
1
3
1
$ 65,000.00
$ 250.00
5
5
1
$ 90,000.00
$ 346.15
9
1
2
3
$ 40,000.00
$ 153.85
Mean
Stdev
26.828
2.465
25.654
1.702
Work/Resource Management
30.580
2.668
Cost/Benefit Analysis
12.002
1.818
Quality Plan /Customer Needs
34.493
5.470
39.412
1.675
Architecture/Requirements
66.946
6.447
Training (Project Team)
Subtotal (hours)
235.916
9.679
Days
29.489
1.210
Total Cost for Scope $ 4,536.84 $ 186.13
1 Mission/Problem Statement
2 Evaluate Alternatives/Risk Assessment
4
3
5
6
7
Baseline
2
3
8 Project Management
9 Process Maps (As Is)
6 Architecture/Requirements
10 Resources/Cost Benefit/Schedule
67.544
46.486
39.412
16.746
1.984
5.527
1.675
3.358
Mean
5.048
11.091
21.507
7.302
14.065
22.064
43.165
124.241
15.530
$ 3,882.53
44.905
22.290
22.064
13.539
Stdev
Mean
1.275
7.0314
2.359
15.4478
3.483
29.9559
1.021
10.1706
2.194
19.5903
1.758
30.7318
6.704
60.1219
8.557
173.050
1.070
21.631
$ 267.40 $ 7,487.73
1.429
1.160
1.758
3.418
Stdev
1.77625668
3.28631313
4.85162492
1.42194392
3.05562677
2.44832728
9.33779075
11.918
1.490
$ 515.69
62.5457
31.0462
30.7318
18.8581
1.99038453
1.61596985
2.44832728
4.76042647
4.262
143.182
.533
17.898
133.17 $ 6,195.37
5.935
.741
$ 256.82
Status
Subtotal
170.188
Days
21.274
Total Cost for Baseline $ 3,272.85 $
Investigate 8 Project Management
9 Process Maps (To Be)
11 Ranking of Defects/Designing Solution
12 Support Plan
6 Architecture/Requirements
13 Test Scenarios
14 Peer Reviews
67.544
46.486
100.100
15.632
39.412
93.835
7.580
6.968
.871
134.01
1.984
5.527
4.953
0.369
1.675
2.769
0.311
102.797
12.850
$ 3,212.41 $
44.905
22.290
71.246
11.601
22.064
68.783
5.732
1.429
1.160
6.540
0.512
1.758
3.649
0.535
62.546
31.046
99.236
16.159
30.732
95.805
7.984
24
1.990
1.616
9.110
0.713
2.448
5.082
0.745
Create
Subtotal
370.590
Days
46.324
Total Cost for Investigate $ 7,126.72 $
8.350
1.044
160.57
246.620
30.828
$ 7,706.88
67.544
114.246
93.835
15.632
7.580
49.352
Subtotal
348.189
Days
43.524
Total Cost for Create $ 6,695.94 $
1.984
4.494
2.769
0.369
0.311
3.520
6.665
.833
128.18
44.905
76.567
68.783
11.601
5.732
34.214
241.802
30.225
$ 7,556.30
1.429
62.546
9.616
106.647
3.649
95.805
0.512
16.159
0.535
7.984
4.041
47.655
11.167
336.796
1.396
42.099
$ 348.97 $ 14,572.89
1.990
13.394
5.082
0.713
0.745
5.628
15.554
1.944
$ 673.01
67.544
38.345
66.946
14 Peer Reviews
7.580
18 Contingency Plans/Lessons Learned
11.832
20 Team Celebration
24.391
Subtotal
216.638
Days
27.080
Total Cost for Control $ 4,166.11 $
1.984
4.806
6.447
0.311
0.902
9.874
12.923
1.615
248.52
44.905
30.374
43.165
5.732
8.287
14.980
147.442
18.430
$ 4,607.57
1.429
62.546
7.804
42.306
6.704
60.122
0.535
7.984
0.842
11.543
9.139
20.865
13.871
205.366
1.734
25.671
$ 433.46 $ 8,886.04
1.990
10.870
9.338
0.745
1.173
12.729
19.320
2.415
$ 835.96
14.656
1.832
281.84 $
1.360
0.170
26.15
8 Project Management
15 Development of Solution
13 Test Scenarios
12 Support Plan
14 Peer Reviews
16 Prototype
Control
8 Project Management
17 Finalize Development/Testing
7 Training (User Groups)
Checkpoint Review Meetings (hours)
Days
Cost for Checkpoint Meetings $
Grand Total (Hours)
1356.176
91.506
Days
169.522
11.438
Total Cost $ 26,080.31 $ 1,759.72
Total Weeks
Total Months
Total Months / 1 project *
33.90
8.48
2.83
$
1.027
0.128
32.09 $
7.945
343.507
.993
42.938
248.27 $ 14,863.29 $
0.081
0.010
2.53 $
1.027
0.128
44.43 $
11.066
1.383
478.80
0.081
0.010
3.50
863.929
88.458
1202.927
123.176
107.991
11.057
150.366
15.397
$26,997.79 $ 2,764.31 $ 52,049.74 $ 5,329.71
21.60
5.40
5.40
30.07
7.52
7.52
* 3 project examples do compare with the real world. Small projects usually take under 3 months,
medium projects take from 3-6 months and large projects are > 6 months.
25
4.2.2 Run Simulation #1
1. Change the arrivals to have only 3 occurrences (Build  Arrivals. Change
occurrences to be 3 projects)
2. Change the simulation options to run for 6 months (8 hours per day * 5 days per week
* 4 weeks per month * 6 months = 960 hours). This can be done by going to the
Simulation menu, then choose Options. The run hours can be changed to 960. The
replications can stay at 10 to get an average time to complete the 3 projects.
3. Set the input factor to be 1 (Small project with low experience levels on the team).
Go to the Simulation  Model Parameters menu. Change the Input Factor to = 1.
4. Press "Run" on the Model Parameters screen.
Output
Phase
Scope
Baseline
Investigate
Create
Control
Total Cost
Total (Days)
Total (Weeks)
Total (Months)
95% Confidence
Interval for Cost
Cost
Mean
$ 4536.84
$ 3272.85
$ 7126.72
$ 6695.94
$ 4166.11
$ 26,080
Stdev
$ 427.78
$ 241.21
$ 338.22
$ 258.60
$ 467.77
$ 1,760
Time (Days)
Mean
29.49
21.27
46.32
43.52
27.08
Stdev
2.78
1.57
2.20
1.68
3.04
169.52
33.90
8.48
11.44
2.29
.57
95% Confidence
Interval for Time
26
4.2.3 Run Simulation #2
1. Change the arrivals to have only 1 occurrence (Build  Arrivals. Change
occurrences to be 1.)
2. Be sure the simulation options will run for 6 months (8 hours per week * 5 days per
week * 4 weeks per month * 6 months = 960 hours). This can be done by going to the
Simulation menu, then choose Options. The run hours can be changed to 960. The
replications can stay at 10 to get an average time to complete 1 project.
3. Set the input factor to be 5 (Medium project with medium experience levels on the
team). Go to the Simulation  Model Parameters menu. Change the Input Factor to
= 5.
4. Press "Run" on the Model Parameters screen.
Output:
Phase
Scope
Baseline
Investigate
Create
Control
Total Cost
Total (Days)
Total (Weeks)
Total (Months)
95% Confidence
Interval for Cost
Cost
Mean
$ 3882.53
$ 3212.41
$ 7706.88
$ 7556.30
$ 4607.57
$ 26,998
Stdev
$ 587.33
$ 242.66
$ 486.97
$ 618.17
$ 826.65
$ 2,764
Time (Days)
Mean
15.53
12.85
30.83
30.23
18.43
Stdev
2.35
.97
1.95
2.47
3.31
107.99
21.60
5.4
11.06
2.21
.55
95% Confidence
Interval for Time
27
4.2.4 Run Simulation #3
1. Change the arrivals to have only 1 occurrence (Build  Arrivals. Change
occurrences to be 1).
2. Be sure the Simulation Options will run for 6 months (8 hours per day * 5 days per
week * 4 weeks per month * 6 months = 960 hours). This can be done by going to the
Simulation menu, then choose Options. The run hours can be changed to 960. The
replications can stay at 10 to get an average time to complete 1 project.
3. Set the input factor to be 9 (Large project with great experience levels on the team).
Go to the Simulation  Model Parameters menu. Change the Input Factor to = 9.
4. Press "Run" on the Model Parameters screen.
Output:
This large project takes 7.52 +/ .77 months. This breaks the limit of 6 months. For an
extra 1.5 months and extra $26,000, this project can be completed. Cutting the project off
at 6 months will leave the in the "Final Development and Testing" phase. The project
will not complete development, testing, identifying lessons learned and best practices, or
the team celebration.
Phase
Scope
Baseline
Investigate
Create
Control
Total Cost
Total (Days)
Total (Weeks)
Total (Months)
95% Confidence
Interval for Cost
Cost
Mean
$ 7487.73
$ 6195.37
$14863.29
$14572.89
$ 8886.04
$ 52,050
Stdev
$ 1132.70
$ 467.96
$ 939.14
$1192.18
$ 1594.24
$ 5330
Time (Days)
Mean
21.63
17.90
42.94
42.10
25.67
Stdev
3.27
1.35
2.71
3.44
4.61
150.37
30.07
7.52
15.40
3.08
.77
95% Confidence
Interval for Time
28
4.3 Analysis of Results
Comparing Scenario 1 versus Scenario 2
The total cost of the 2 scenarios is quite similar.
Scenario
Scenario 1: 3 small projects in 6 months with a low
experienced team. Input Factor = 1.
Scenario 2: 1 medium project in 6 months with a
medium experienced team. Input Factor = 5.
Total Cost
$ 26,080.31 = Xbar
$ 1,759.72 = s
$ 26,997.79 = Xbar
$ 2,764.31 = s
Xbar is the sample mean. "s" is the sample standard deviation.
Using Minitab, a 2 sample t test was run to see if the cost of Scenario 1 is significantly
different that Scenario 2.
H0: The 2 scenarios have equal costs (c1 = c2)
H1: The 2 scenarios have unequal costs (c1 <> c2)
In Minitab, the menu item:
Stat  Basic Statistics  2 sample t
was run to test the hypothesis. The following results were obtained:
Two Sample T-Test and Confidence Interval
Two sample T for Design1cost vs Design2cost
Design1
Design2
N
5
5
Mean
5160
5433
StDev
1671
2155
SE Mean
747
964
95% CI for mu Design1c - mu Design2c: ( -3159, 2612)
T-Test mu Design1c = mu Design2c (vs not =): T = -0.22
P = 0.83
DF = 7
Since the p value is .83, we can not reject the null hypothesis at a 95% confidence level.
In other words, there is not enough evidence to conclude that the 2 scenario's costs are
significantly different. Either scenario can be completed to get the job done. There is no
significant difference between the 2.
Scenario 3
Scenario 3 costs $ 52,050 with a standard deviation of $ 5,330. This is nearly double the
cost for either Scenario 1 and/or 2. The benefits of spending more are:
 Scenario 1 or 2 cost less; however, the business will only deliver benefit to its
customers. These projects will only deliver the customer "satisfiers". The business
needs to do this enhancement to stay competitive in the market.
29

For Scenario 3, approximately 26,000 more will need to be spent; however, more
benefit will be delivered to the customers. This project will deliver customer
"satisfiers" and customer "delighters". The business will gain market share and sales
if this project is completed.
4.4 Confidence Intervals
In order to verify that the model is producing data that is representative to the population
the following confidence intervals were calculated. The mean of each simulation
example falls within the confidence interval therefore verifying that the outcome
represents the population.
Figure 4.4.6 Confidence Intervals
Simulation Example Number
1
Scope
Baseline
Investigate
Control
Create
Average Hours
Standard Deviation
Hours Confidence Interval
235.916
22.244
272.51
Average Hours
Standard Deviation
Hours Confidence Interval
170.188
12.543
190.82
Average Hours
Standard Deviation
Hours Confidence Interval
370.590
17.588
399.52
Average Hours
Standard Deviation
Hours Confidence Interval
348.189
13.447
370.31
Average Hours
Standard Deviation
Hours Confidence Interval
216.638
24.324
256.65
2
199.32
124.241
18.795
155.16
149.56
102.797
7.765
115.57
341.66
246.620
15.583
272.25
326.07
241.802
19.781
274.34
176.63
147.442
26.453
190.96
3
93.32
173.050
26.178
216.11
129.99
90.02
143.182
10.815
160.97
125.39
220.99
343.507
21.704
379.21
307.80
209.26
336.796
27.553
382.12
291.47
103.93
205.366
36.845
265.98
144.76
30
4.5 Conclusion of example above


There is no significant difference between Scenario 1 and Scenario 2.
Scenario 3 costs approximately twice as much ($26,000); however, the business will
gain market share and sales. The question is "How much market share and sales will
be gained?"
If the market share and sales gained exceed $26K, Scenario 3 is recommended if the
customer is willing to wait an extra 2 months for the project to be delivered.
If the customer is not willing to wait an extra 2 months for the extra benefits, either
Scenario 1 or Scenario 2 are recommended.


5.0 Model Verification and Validation
Verification is used to debug the model. Validation is used to relate the conceptual model
to the output of the simulation.
5.1 Model Verification
Model verification was used to remove program bugs from the model. In order to verify
that the conceptual system is representative of our model we asked an outsider to try to
run the model. As a result of this action we found the following problems with our
model:

The last task was not connected in the path networks and therefore was not producing
output.
The model’s output should have been based on hours, but it was giving us minutes.

After performing this exercise and running numerous replications of the model we have
concluded that our model is representative of the conceptual system of PMT.
5.2 Model Validation
In order to validate that our model is producing output that is representative of the actual
system data was collected from managers who have worked with projects that match the
profile for input factors 1, 5, and 9. The managers were asked the questions listed in
section 1.5. Figure 5.2a shows the data that was gathered.
Figure 5.2a Example Output
Proj Input
# Factor
1
2
3
1
5
9
Scope
Baseline
Investigate
Create
Control
Mode Min Max Mode Min Max Mode Min Max Mode Min Max Mode Min Max
70
33
85
50
30
60
115 95 135
110 85 140
70 40 110
125
70 185
103
75 130
246 200 300
241 185 300 148 90 235
174 100 260
145 100 180
345 280 400
335 260 415 206 130 350
31
To validate that the data collected in Figure 5.2a is the same as data that would have been
produced by the simulation model the following hypothesis test was employed:
Ho: 1 = 2
Ha: 1 <> 2
In this test 1 is the value as predicted by managers that is displayed in Figure 4.4. The
2 represents the mode value given by the output of the simulation runs from section 4.2
displayed below in the Figure 5.2b.
Figure 5.2b Output from Example in Section 4.2
1
Scope
Input Factor
2
3
Average Hours 235.916
Standard Deviation
22.244
124.241
18.795
173.050
26.178
Average Hours 170.188
12.543
102.797
7.765
143.182
10.815
Investigate
Average Hours 370.590
Standard Deviation
17.588
246.620
15.583
343.507
21.704
Control
Average Hours 348.189
Standard Deviation
13.447
241.802
19.781
336.796
27.553
Create
Average Hours 216.638
Standard Deviation
24.324
147.442
26.453
205.366
36.845
Baseline
Standard Deviation
The test statistic is based on an  level of 10% and n equal to 10 replications:
t(1-/2, n-1) = t(.95,9) =1.833
The tested value (t*) is based on the formula:
t* = (1 - 2)/ (s/√n)
32
The t* values for the 3 scenarios in Figure 5.2a are:
Figure 5.2c T* Test Values
t* values
Scope
Baseline
Input Factor
1
5
9
22.62491
-0.1265 -0.10962
30.047
Investigate 45.10412
-0.087
-0.5454
0.124 -0.21329
Control
54.96669 0.121632 0.199556
Create
18.32975 -0.06438 -0.05283
The hypothesis test criteria is if:
|t*| <= t(1-/2, n-1)
conclude that the null hypothesis (Ho) is true, else conclude that the alternate hypothesis
(Ha) is true.
For Input Factor equal to 5 and 9, projects with:


Medium experienced people and medium complexity (input factor = 5)
Highly experienced people and high complexity (input factor = 9)
The absolute values for t* given (in Figure 4.4b) by the model are less than the test
statistic t. Therefore we would accept the null hypothesis that the 2 means are the same.
For Input Factor equal to 1, projects with:

Low experienced people and low complexity (input factor = 1)
The t values given by the model are greater than the test statistic t. Therefore we would
reject the null hypothesis that the 2 means are the same.
This indicates that the outcome of our simulation is not producing data that is from the
same population as the data given by this manager. This could be due to inaccurate data
given by the team or manager, or an incorrect input factor rating. For example, maybe
the project should have been rated more complex or maybe the complexity increased
33
during the project. It is important to point out that because of the large range of IT
projects that can be performed, a study of this kind would have to be very specialized to
a specific project type and team.
Because we were able to accept the null hypothesis for the 2 other tests under all of the
modules, we feel that our model is validated. We feel that the data given by the manager
should have been given a different input value rating. The outcome of the hypothesis
tests verifies that the applications of this simulation are very specified and projects data
must be clearly defined to produce adequate output.
6.0 Conclusion
The purpose of this simulation model is to provide managers in IT businesses a tool to
estimate project duration and costs. In order to provide adequate output, the model must
be tailored to reflect actual data collected in certain office situations. Based on the data
gathered from a General Electric IT department, the output from the simulation model has
been validated. However, in some instances, as shown in Section 4.4, the output may
vary from actual data gathered from the population because of



Assignment of an incorrect input factor
Incorrect task times
Increased variation in the project
To avoid inaccurate project time output the simulation must be modified to reflect data
collected from the office that is being studied. For instance, this model would need to be
modified if Rensselaer wanted to use it to predict the time and cost of IT projects in their
school. In conclusion, this project has shown that simulation models can be used to
predict the time and cost of projects as long as the data used to create the model is based
on
data
collected
from
the
specific
population
being
studied.
34
References
Harrell, Charles, Ghosh, Diman and Bowden, Royce. Simulation using ProModel.
Boston: The McGraw Hill Companies, Inc., 2000.
Law, Averill and Kelton, W. David. Simulation Modeling and Analysis - Third Edition.
Boston: The McGraw Hill Companies, Inc., 2000.
McKenna, Christine. Project Management Methodology © 1999. General Electric
Company.
Neter, John, Kutner, Michael, Nachtsheim, Christopher, and Wasserman, William.
Applied Linear Regression Models. Chicago: The McGraw Hill Companies, Inc., 1996.
35