Liana Algarín, Ph.D. Candidate, The George Washington University
Thomas A. Mazzuchi, D.Sc., The George Washington University
Shahram Sarkani, Ph.D., P.E., The George Washington University
Presented at Systems Engineering in DC
Chantilly, VA
April 3-5, 2014

1.
Problem Statement
2.
Motivation
3.
Life Cycle of an Acquisition Program
4.
Human Systems Integration Requirements
5.
Case Study: WIN-T
6.
Training Burden
7.
Methodology
8.
Pilot Study
9.
Conclusion & Future Research

 Incorporating Human Systems Integration (HSI) requirements earlier in the acquisition life cycle may minimize cost. One way to predict the effect that HSI requirements will have on life cycle cost would be to identify HSI language within acquisition documents. Regression analysis using specific variables identified from the text of HSI language is a potential method for making a prediction about life cycle cost. The predictor variables will be generally categorized by time
(when HSI language appeared in documentation), training recommendation and component, and specific HSI issue.
 HSI has nine domains:
1.
Human Factors Engineering,
2.
Environment,
3.
Safety,
4.
Occupational Health,
5.
Habitability,
6.
Survivability,
7.
Manpower,
8.
Personnel, &
9.
Training
 Project Manager (PM) Warfighter Information
Network-Tactical (WIN-T) is a tactical communications network
 “WIN-T is like the ‘Verizon’ of the Army.” Dr. Bev
Knapp, Army G-1, MANPRINT
 WIN-T is an ubiquitous tactical communications network that the Army will continue to implement and improve during the coming years
 The Army is committed to making WIN-T work efficiently

4

 Primary Stakeholders
• US Army’s Manpower Personnel Integration (MANPRINT) program
• Human Research and Engineering Directorate (HRED), which is one of six directorates in the US Army Research Laboratory (ARL)
 MANPRINT program mission is to cut costs and prevent soldier loss during the system development life cycle
 MANPRINT’s goal is for Army’s Soldiers to experience less of a burden in both training and long-term Operations and Maintenance (O&M)
 My goal is to deliver new analytics that would assess the benefits (or costs) of implementing HSI early in the life cycle
 Stakeholders have been wanting to examine this idea, and they would like to have insight on how WIN-T can/could have been improved

 Major decision points occur during Milestones A, B, and C; some requirements are not included at Milestone A but are later introduced during Milestone C
 Schedule slippage and cost overruns are affected by long-term manpower planning
 In addition to poor planning, poor project management strategy can cause a program to fail to meet its objectives
 When Systems Engineering is applied early in the life cycle, the Technology Development
Phase and Knowledge Points are benefited
 Function allocation early in the life cycle helps a complex work system, such as WIN-T, to be more effective
 It is also beneficial to create a system value from stakeholders’ opinions and apply this system value to the life cycle

 Human Systems Integration (HSI) involves incorporating people with the systems that they use to complete various tasks
 When an acquisition program begins, there may not be a clear idea of what the end user will need in the long-term
 However, this is not a justification for entirely leaving out HSI requirements at
Milestone A
 Usability defines how well a human can use a system; usability issues can be anticipated early in the life cycle
 Including HSI up-front at the life cycle planning stage helps to reduce long-term costs by means of reducing the burden of training the workers
 Early incorporation of HSI requirements and HSI methods can benefit a system or program later in its life cycle
 HSI requirements can be successfully incorporated when the stakeholders are open to including HSI requirements and following HSI methods and when HSI happens early in the life cycle

 A case study with the Army’s tactical communications network, known as
Warfighter Information Network-Tactical (WIN-T), will illustrate how there have been cost overruns as a result of integrating HSI requirements late in the life cycle
 WIN-T will be used at locations where Soldiers must communicate with other units
 There is no common interface across all of WIN-T’s components
• WIN-T combines user interfaces from several individual systems
• Combining these interfaces has created some confusion for the Soldier end user
• Confusion has created a heavy training burden on Soldiers
 There are 4 WIN-T increments being completed/scheduled since 2004 onward
• Each increment of WIN-T is expected to show improvements for usability
• WIN-T Increment 2’s team is currently conducting usability analysis of WIN-T’s components

 As more time is spent training soldiers to interact with and use WIN-T, cost increases
 Some Military Occupational Specialties (MOS) have the appropriate experience for working with the types of interfaces that are incorporated with WIN-T
 However, Soldiers with different MOS may sometimes use WIN-T and find it difficult to use, particularly due to the several user interfaces

 Objective is to create new analytics to measure the extent to which early implementation of HSI requirements will affect an acquisition program
 Data collection will be taken from source documents provided by stakeholders at the MANPRINT and HRED offices
 Potential WIN-T Source Documents
• Assessments (including MANPRINT Assessments)
• Statements of Work
• Analysis of Alternatives
• Engineering Change Proposals
• Emails
• Test Reports
• Meeting Minutes
• Initial Capability Document
• Capabilities Production Document
• Capability Development Document
• etc.

 Using existing data from WIN-T Increments 1 & 2, we can make predictions for WIN-T Increments 3 & 4
 Regression analysis is a statistical method that can be used for making a general prediction based on previously made observations
 The planned analysis will be a regression analysis with one Dependent
Variable and several Predictor Variables

 Dependent Variable
• Will be defined using MANPRINT assessment scores
 Predictor Variables
• Time: Date in month/year (number of months into WIN-T’s acquisition life cycle)
• Time: If a document is released sometime after the events to which the document refers, such as a report about a test from a few months ago, then the date in month/year will indicate closest approximation to the appropriate time in WIN-T’s life cycle
• Training: Recommendations for training improvement (by each WIN-T component)
• HSI-Related Language: Previously selected HSI words/phrases
– Selected by Expert Elicitation – Asking stakeholders what words or phrases they think are relevant
– Selected by Cluster Analysis (CA) – Do CA on a primary word that is synonymous with HSI, such as a unique word related to any of the nine HSI domains

 New analytics will be created to demonstrate that applying HSI requirements earlier in the acquisition life cycle would be beneficial
 The method of looking for HSI language within documents and noting when certain language was used can be repeated with any program
 Milestones and assessments are commonly used within systems acquisition programs, particularly for military and weapons systems
 These new analytics could potentially be used to:
• increase accuracy of schedule predictions,
• minimize Life Cycle Costs for hours spent on training, and
• minimize component usability issues related to HSI

Ahram, Tareq Z., and Waldemar Karwowski. "A Framework for Human Total Ownership Cost Based on
Universal Human Performance Cost Components." Paper presented at the Proceedings of the
Human Factors and Ergonomics Society 56th Annual Meeting, HFES 2012, October 22, 2012 -
October 26, 2012, Boston, MA, United states, 2012.
Boehm, Barry, Ricardo Valerdi, and Eric Honour. "The Roi of Systems Engineering: Some Quantitative
Results for Software-Intensive Systems." Systems Engineering 11, no. 3 (2008): 221-34.
Brinkman, W. P., R. Haakma, and D. G. Bouwhuis. "Component-Specific Usability Testing." IEEE
Transactions on Systems Man and Cybernetics Part a-Systems and Humans 38, no. 5 (Sep 2008):
1143-55.
Browning, Tyson R., and Eric C. Honour. "Measuring the Life-Cycle Value of Enduring Systems." Systems
Engineering 11, no. 3 (2008): 187-202.
Challenger, R., C. W. Clegg, and C. Shepherd. "Function Allocation in Complex Systems: Reframing an Old
Problem." Ergonomics 56, no. 7 (Jul 2013): 1051-69.
Chatterjee, Samprit, and Ali S. Hadi. Regression Analysis by Example. Wiley Series in Probability and
Statistics. Fifth ed. Hoboken, NJ: John Wiley & Sons, Inc., 2012.
Chung, Lawrence, and Kendra Cooper. "Defining Goals in a Cots-Aware Requirements Engineering
Approach." Systems Engineering 7, no. 1 (2004): 61-83.
Colombi, John M., Michael E. Miller, Michael Schneider, Major Jason McGrogan, Colonel David S. Long, and John Plaga. "Predictive Mental Workload Modeling for Semiautonomous System Design:
Implications for Systems of Systems." Systems Engineering 15, no. 4 (2012): 448-60.
Dacus, Chad L. "Improving Acquisition Outcomes through Simple System Technology Readiness Metrics."
Defense ARJ 19, no. 4 (October 2012): 444–61.
Deitz, Daniel, Timothy J. Eveleigh, Thomas H. Holzer, and Shahryar Sarkani. "Improving Program Success through Systems Engineering Tools in Pre-Milestone B Acquisition Phase." Defense ARJ 20, no. 3
(October 2013): 283–308.
Eigbe, Arekhandia Patrick, Brian J. Sauser, and John Boardman. "Soft Systems Analysis of the Unification of Test and Evaluation and Program Management: A Study of a Federal Aviation
Administration's Strategy." Systems Engineering 13, no. 3 (2010): 298-310.
Hardman, Nicholas, and John Colombi. "An Empirical Methodology for Human Integration in the Se
Technical Processes." Systems Engineering 15, no. 2 (2012): 172-90.
Hart, C. G., Z. He, R. Sbragio, and N. Vlahopoulos. "An Advanced Cost Estimation Methodology for
Engineering Systems." Systems Engineering 15, no. 1 (2012): 28-40.

Horowitz, B. M., and J. H. Lambert. "Assembling Off-the-Shelf Components: 'Learn as You Go' Systems
Engineering." Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on
36, no. 2 (2006): 286-97.
Karwowski, W. "A Review of Human Factors Challenges of Complex Adaptive Systems: Discovering and
Understanding Chaos in Human Performance." Human Factors 54, no. 6 (Dec 2012): 983-95.
Knapp, Dr. Bev. "Army Manprint Perspective." Paper presented at the NDIA HS Workshop, Washington,
DC, 2012.
———. "Moving Manprint (Manpower Personnel Integration) Left (Mml)." 2013.
———. "Moving Manprint to the Left (Mml)." 41. Washington, DC, 2013.
Lattin, James M., J. Douglas Carroll, and Paul E. Green. Analyzing Multivariate Data. Belmont, CA:
Brooks/Cole, 2003.
Liu, Kevin K., Ricardo Valerdi, Donna H. Rhodes, Larry Kimm, and Alvis Headen. "The F119 Engine: A
Success Story of Human Systems Integration in Acquisition." Defense ARJ (April 2010).
Madni, Azad M. "Integrating Humans with Software and Systems: Technical Challenges and a Research
Agenda." Systems Engineering 13, no. 3 (2010): 232-45.
Marchant, Alan B. "Obstacles to the Flow of Requirements Verification." Systems Engineering 13, no. 1
(2010): 1-13.
Patanakul, Peerasit, and Aaron Shenhar. "Exploring the Concept of Value Creation in Program Planning and Systems Engineering Processes." Systems Engineering 13, no. 4 (2010): 340-52.
Pflanz, Mark, Chris Yunker, Friedrich N. Wehrli, and Douglas Edwards. "Applying Early Systems
Engineering: Injecting Knowledge into the Capability Development Process." Defense ARJ 19, no.
4 (October 2012): 422–43.
Pharmer, James A. . "The Challenges and Opportunities of Implementing Human Systems Integration into the Navy Acquisition Process." Defense ARJ (February 2007).
Sohoni, Milind G., Ellis L. Johnson, and T. Glenn Bailey. "Long-Range Reserve Crew Manpower Planning."
Manage. Sci. 50, no. 6 (2004): 724-39.
Tan, W. P., B. J. Sauser, J. E. Ramirez-Marquez, and R. B. Magnaye. "Multiobjective Optimization in
Multifunction Multicapability System Development Planning." IEEE Transactions on Systems
Man Cybernetics-Systems 43, no. 4 (Jul 2013): 785-800.
U. S. A. R. L. "Improved Performance Research Integration Tool." U. S. Army Research Laboratory, http://www.arl.army.mil/www/default.cfm?page=445 .
U. S. Army. "Manprint Program Mission." U. S. Army, http://www.manprint.army.mil/manprint/about.html
.
———. "Our Mission." U. S. Army, http://www.manprint.army.mil/directorate/about.html
.

 Liana Algarín is working toward a Ph.D. in Engineering Management and Systems Engineering at The George Washington University. She is also a Human Factors Engineer at Alion Science and Technology, where she has designed user interfaces, training manuals, and user guides to support the Army and NASA. She has also conducted a critical task and usability analyses of
Coast Guard cutters.
 Thomas A. Mazzuchi, D.Sc., is Professor of Engineering Management and Systems
Engineering at The George Washington University. His research interests include reliability, life testing design and inference, maintenance inspection policy analysis, and expert judgment in risk analysis. He served as Research Mathematician at Royal Dutch Shell, and has conducted research for the U.S. Air Force, Army, and Postal Service; for NASA, and for the
Port Authority of New Orleans, among others.
 Shahram Sarkani, Ph.D., P.E., is Professor of Engineering Management and Systems
Engineering, and Director of EMSE Off-Campus Programs, at The George Washington
University. He designs and administers graduate programs that enroll over 1,000 students across the U.S. and abroad. In over 150 technical publications and in sponsored research with
NASA, NIST, NSF, AID, and Departments of Interior, Navy, and Transportation, his research has application to risk analysis, system safety, and reliability.