HSI Lessons Learned Status Report FORCEnet HSI Working Group George Edw. Seymour, Ph.D. * LCDR Sally Van Horn * * Naval Network Warfare Command * Space and Naval Warfare Systems Center, San Diego ** San Diego, California: April 7, 2004 1 Fn Human Systems Integration Lessons Learned Update Agenda The value of HSI….leader comments….at the deck plates Recent HSI timeline HSI focus deficit in OIF LL HSI Lessons Learned info design HSI LL status report Preliminary findings and recommendations Interview process findings Value and next steps 2 The Value of HSI “In short, only by embracing a human-centered and HSIfocused philosophy that transcends virtually every element of the Navy will we be able to win the "war for people" and also ensure our future warships and other platforms can go in harm's way and emerge victorious.” — CNO (2002) Source: http://www.chinfo.navy.mil/navpalib/policy/vision/vis02/vpp02-ch3a.html 3 HSI at the Deckplates “It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, who's face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.” 4 Traditional HSI Human Systems Integration Human Factors Engineering Physical & Mental Capabilities & Limitations Anthropometrics & Biomedical Criteria Man-Machine Interface Mission, Function & Human Rqmts Analyses Skill, Knowledge & Abilities Manpower Wartime Rqmts Deployment Considerations Force Structure Personnel Training Safety And Health Personnel Classification & Selection Training Concepts & Strategy System Safety / Health Hazards Plan Task Analysis Methods Human Error Analyses Demographics Operating Strength Accession & Attrition Rates Manning Concepts Retention Rates Rate / Rating Career Progression Officer and Enlisted Workload Training Flow Media / Equipment Simulation OP Tempo Training System Evaluation Training Development Plan System Reliability Analysis Habitability Personnel Survivability Quality of Life Anti-Fratricide Quality of Work Personnel Protection Environment Limits and Controls Perform Effects of Ensembles Personnel Services Damage Control Lessons Learned Environmental Considerations Protective Equipment Performance Assessments 5 Tactical Technology Comment “Over the past decade, the exponential growth of e-mail and the Internet in command and control of joint, combined, and Fleet operations has made network reliability and redundancy perhaps our most "mission critical" concern for the warfighter.” Admiral Natter, Commander, Fleet Forces Command, 2002 6 Information Overload Comment “The Global Hawk was awesome and the amount of data that it collects is incredible. The thing that we've learned from our lessons learned is that the Global Hawk will collect all of this great data, dump it down to the people and we got so much data that we were overwhelmed. So we had the Global Hawk go off into a holding pattern and we told it, 'Don't send us any more stuff.'” Brig. Gen. Gregory H. Power, Vice Commander, Headquarters 8th Air Force, February 2004 7 Information Deficit Comment “Tactical intelligence was impeded by our speed of movement," Conway said. "We just didn't have enough time to get the reconnaissance out and mature the picture as to what was in front of the division and the regiment on a day-today basis. I don't think we can say we had too much" intelligence, surveillance and reconnaissance.” Marine Lt. Gen. James Conway, Commander, 1st Marine Expeditionary Force, February 2004 8 Recent HSI Timeline 10/01: Task Force EXCEL: 13 Learning Centers 7/02: CNO Stands Up NETWARCOM 4/03: SPAWAR Stands Up HSI Department 5/03: FORCEnet Conference: NAB Little Creek 5/03: Paul Wolfowitz USD (AT&L) signs the 5000.1 Defense Acquisition Directive 6/03: CNO addresses HSI Symposium 7/03: NETWARCOM FORCEnet OAG 8/03: First HSI-focused OIF L.L. Initiative 9/03: Human Performance Center Established 9 HSI Focus Deficit in OIF LL Example: Dienhart, Roos, & Howard (2003) collected USMC Lessons Learned at two locations in Iraq. Despite their focus of communication technology, a USMC Major who also holds a Ph.D. and works in the human factors area examined the list of 115 items. His analysis concluded that 18 items or 15.7 percent were definitely HSI related, although not directly stated as such. Another 60 of them definitely were not HSI items, and the remaining 37 could have some component of HSI, but more information was required to make that determination. NETWARCOM initiated an HSI-focused lessons learned research initiative 10 OIF Tech Lessons Learned: Federal Computer Week, March 29, 2004 “What didn't work: Bandwidth allocation. DOD officials admit they need a better way to get bandwidth to the troops who need it. Data analysis on the fly. Battlefield commanders said they often received raw data and images that they were expected to interpret. Military and civilian defense leaders have called for reducing reaction time by providing more machine-to-machine information and data analysis. Soldier intercom radios. Soldiers have no confidence in the intercom radios, according to Army officials. Soldiers bought handsets and longer antennas to increase the radios' range. Ruggedized laptop computers. According to Army officials, the systems had a 30 percent failure rate in the desert. Apparently, they weren't rugged enough. Communications interoperability. Communication among U.S. military services was relatively reliable, but talking to allied or coalition troops proved to be more difficult.” Directly relates to FORCEnet and HSI 11 HSI LL Research Initiative OIF Platforms: Two of each kind (CV to SS) Inclusive vs. narrow HSI focus “Deckplate” focus (O-5 – E-3) versus unit commanders Tactical & Non-tactical network experience Scientifically based: Sampling error goal 5 % Two formats (pilot tested twice) Interview Self-administered form Minimize impact on both commands & platform personnel Resource limited (i.e., researchers and funding) Kick-off message: COMNAVNETWARCOM/081748ZAUG2003// 12 HSI LL Initiative: 16 Command Visits As of 8 March 2004, visited 9 commands on east coast / 7 on the west coast Carrier Air Wing Three Carrier Group Two USS Harry S. Truman (CVN-75) USS Kearsarge (LHD-3) USS Donald Cook (DDG-75) USS Montpelier (SSN 765) USS Iwo Jima (LHD-7) USS Mitscher (DDG-57) USS Boise (SSN-764) USS Shiloh (CG-67) VAW 116 USS Tarawa (LHA-1) VFA-137 USS Nimitz (CVN-68) USS Milius (DDG-69) USS Pearl Harbor (LSD-52) Conducted 60 OIF HSI-focused interviews which provided 484 comments Side 1 of survey form Opportunity to choose from 19 categories for discussion Collected nearly 1,400 HSI-focused forms (as of 1 April 2004) Side 1 – rate, paygrade, OIF and current tasks, SW/HW used Side 2 – nine areas to provide comments, plus six-point satisfaction score Note: Form data from commands listed in blue have been entered into analysis database 13 HSI Categories Survey form Habitability Health Hazards Human Factors Engineering Information & Knowledge Management Internet Manpower, Personnel & Training Medical Factors Personnel Survivability Safety Factors Interview – All categories from survey form plus: Education Battlespace Awareness Command & Control Decision Speed & Superiority Collaboration Data and Information Distribution (included Internet) Knowledge and Wisdom ISR Joint, Coalition & NGO Operations Focused Logistics Miscellaneous 14 OIF HSI-focused Interviews Table 1: Interviewee Rank Enlisted N Officer N E-4 2 W-2 1 E-5 6 W-3 2 E-6 10 O-1 2 E-7 10 O-2 7 E-8 2 O-3 9 O-4 7 O-5 2 All 60 interviews were conducted with OIF-experienced personnel, having either tactical or non-tactical (or both) network experience. Interviews lasted about 40 minutes each, and provided a total of 484 anonymous comments in 19 categories 15 Frequency HSI Form Respondents 400 40 350 35 300 30 250 25 200 20 150 15 100 10 50 5 0 0 E-2 E-3 E-4 E-5 E-6 E-7 Enlisted Pay Grade E-8 E-9 W-2 W-3 O-1 O-2 O-3 Officer Pay Grade O-4 O-5 16 HSI Interview Responses Education & Training Collaboration HFE Data and Info Distro Focused Logistics C2 Battlespace Aw areness Category Personnel Manpow er Based on 60 Interviews from 15 Commands Know ledge & Wisdom Habitability Medical Factors Joint, Coalition & NGO Ops ISR Personnel Survivability Decision Speed & Superiority Health Hazards System Safety Misc Not HSI 0 20 40 60 Number of Comments 80 100 120 17 6 450 5.5 400 5 350 4.5 300 4 250 3.5 200 3 150 2.5 100 2 Number of Comments Received (Enlisted) Number of Comments Received (Officers) Average Rating (Scale 1 – 6) Habitability Internet Manpower, Personnel & Training Human Factors Engineering Health Hazards Information & Knowledge Mgmt Medical 0 1.5 Safety 50 1 HSI Category 18 Average Rating 500 Personnel Survivability Number of Comments HSI Form Responses Preliminary Interview Findings OIF HSI LL Outcome Themes: Training: The number one issue in the Fleet Collaboration: The modern (21st Century) issue is second Other important themes Information and Knowledge Management Personal and Personnel Readiness Technology Usability Issues Habitability and Morale 19 HSI Capability Issue 1 - Chat Findings: Chat has emerged as one of the primary means of operational communication and tasking Chat was used by all platforms tactically and non-tactically Evidence: 56 interview comments from 15 different units regarding chat usage (all units interviewed) 19 form comments from 9 different units regarding monitoring chat and maintaining connectivity for chat “We lived and died via chat and e-mail.” 20 Capability Issue 1 (Continued) Preliminary Analysis: Chat has become an integral element of both tactical and nontactical warfighting operations Why? Facilitates coordination and collaboration Facilitates timely information exchange Wide availability User friendly Can be used as person to person or to broadcast messages 21 Capability Issue 1 (Continued) Preliminary analysis (cont.) Current shortfalls of chat becoming increasingly visible No standard for chat software applications Multiple applications results in potential interoperability problems and increased costs due to multiple licenses and duplication of training effort Limited TTP, protocols and policy (varied between chat rooms and theaters) Notable limited central management of chat rooms Requires constant connectivity to participate in chat rooms No user-specific feedback that all recipients have received and understand the communication Unreliable “logging” capability due to lack of official timestamp and backup capability 22 Capability Issue 1 (Continued) Recommendations: Investigate establishing chat as an official method to disseminate tasking (CFFC / NETWARCOM / NWDC) Conduct chat requirements analysis (timestamp, security, number of users, joint/allied interoperability, etc.) (NETWARCOM / SSC-SD) Research available chat software applications and establish/field the standard (JFCOM / SPAWAR) Develop, and test via Trident Warrior and other appropriate experimentation venues, chat TTP, protocols and policy for usage of chat (including investigating the most efficient means of monitoring chat on each platform) (C2F / C3F) Develop contingencies for when chat is unavailable (C2F / C3F) 23 HSI Capability Issue 2 – Info Mgmt Findings: Double-edged sword: There is too much information from too many different sources AND required information is not easily found Problem will only get worse as the Navy moves forward digitally No consistent way to verify accuracy or reliability of data once found Evidence: 56 interview comments from 15 different units (all units interviewed) 81 form comments from 13 different units (all units in database) “Defending America, in today's rapidly changing world, is contingent upon the ability to process data and information quickly. Now, more than ever, knowledge superiority and integrated IT capabilities are essential to not only winning the war against terrorism, but defending our Nation as well.” “There’s too much information out there – we need information management.” 24 Capability Issue 2 (Continued) Preliminary analysis: Personnel are being flooded with too much information Difficult to compile/reconcile data from multiple sources Difficult to determine which sources are authoritative Unsure of where to find needed data Personnel don’t have ready access to the data needed to perform their jobs because Access (bandwidth, internet, time) Unsure of appropriate source Moreover, “114.5 million or 39 percent of Americans used a search engine during January 2004.” * * 114.5 million unique users represent “76 percent of the active online U.S. population.” 25 Capability Issue 2 (Continued) Recommendations: Develop / field intelligent agents for SIPRNET (“Ask Jeeves” type of capability) (SPAWAR) Develop and implement information management plans during Trident Warrior and other appropriate experimentation venues (NETWARCOM / SPAWAR / C2F / C3F) Identify Navy POC for developing IM / KM DOTMLPF (DoN CIO) Buy / develop information management processes and tools (SYSCOMs) Ensure HSI is integral in individual C4I system design/updates and across systems (NAVSEA) Install more computers / LAN drops throughout the ship to improve access (OPNAV / SPAWAR) 26 HSI Capability Issue 3 - Skills Findings: Skills acquisition and performance support tools have become increasingly critical given optimum manning and increased optempo Junior personnel arrive at their command with inadequate skills to perform required tasks Senior personnel are seldom adequately trained to address technology upgrades Evidence: 68 interview comments from 15 different units (all units interviewed) 242 form comments from 13 different units (all units in database) “Proficiency level on ships does not match requirements.” 27 Capability Issue 3 (Continued) Preliminary analysis: Personnel arrive at their command with inadequate skills to perform required tasks Results in under-utilization of untrained personnel Results in over-tasking of the trained ship’s crew, who have to provide the needed training when they can Tasks take longer when personnel don’t receive adequate training Sailors require more access to computers to take advantage of computer based training “With each advance in technology, the human has become the increasingly weak link in systems," says Harold P. Van Cott, author of the foreword. To be competitive in today's economy, corporations spend billions of dollars each year to train their employees. Yet only a tiny portion of this expense is invested in research and development to make training more effective.” (2000) 28 Capability Issue 3 (Continued) Recommendations: Sailors should arrive at commands with more hands-on expertise and troubleshooting skills (NETC) Each rate should have a standardized skills continuum tailored to platform (i.e., ship class) (Task Force Warrior) Develop performance support tools (Human Performance Center) Officers and enlisted need basic skills on computer applications and web sites they routinely use (MS Office, Turboprep, NAVFIT, NKO, etc.) (NETC) Ensure adequate connectivity for training systems at sea (NETWARCOM) Install more computers / LAN drops throughout the ship to support education and training (OPNAV / SPAWAR) Incorporate innovative / web-enabled training events during Trident Warrior events (NETWARCOM / SPAWAR / C2F / C3F) 29 Identified Technology Issue: NALCOMIS “We use both NALCOMIS and IETMS but they do not work together. They are two stovepipes by two different contractors, and we have to retype the information (it is double work). They are used for downloading discrepancies from jets, and then creating the work order. They should be integrated to work together.” “I use three computer systems and they do not work together (NMCI, NALCOMIS and IETMS). The whole thing frustrates me. None of them talk to each other. I can’t download what I need, and NALCOMIS is a DOS (command line) system. It has not changed in 10 years or so.” 30 Interview Process Findings OIF HSI LL Process Themes: Categories (e.g. Personnel & Education) sometimes convey different meanings to interviewees Current focused-interview process is insufficient Interview format should focus on specific tools and work processes Should require interviewees sit at their workstations Ideal time for interviews is during transit from theater to CONUS Need more efficient way of collecting / analyzing data Recommendations: Phase 2 of HSI LL should include small warfighter teams at pre-selected workstations Data collection process needs to be more automated 31 Link Between HSI LL Initiative and TW04 TW04 Initiative FORCEnet Systems Issue examples identified from HSI Research Initiative Addressed in TW04 Sea Warrior Integrated Learning Env, 5-Vector Model, Threaded Discussion, Distance Spt Centralized web-enabled foundational and system specific training for several C2 and network systems Web-Enabled Warrior Chat, Email, NMCP, NITES, TMIP Information Mgmt / Knowledge Mgmt IM Plan Standardized policy for sharing / finding information (info rules of the road) Networks ADNS, EHF TIP, INMARSAT, VIASAT Means of providing better situational awareness of the network environment (BW status, constraints) ISR-Fires SACC-A, NFCS, TBMCS, GCCS-M Improved situational awareness and C2 processes (information exchange and collaboration) Standardized chat policy 32 Connection between FORCEnet and HSI LL Initiative Preliminary (Incomplete) Alignment Analysis Expeditionary, multi-tiered sensor and weapon information 6.4% Dynamic, multi-path and survivable networks 14.5% Human -centric Integration FORCEnet Information Effects 36.3% Distributed, collaborative command & control 31.8% 0.6% Adaptive / automated decision aids 10.3% 33 Value and Next Steps Value of HSI Lessons Learned Effort Seldom, if ever, has the HSI perspective been given this empirical visibility and this depth of analysis HSI Lessons Learned effort needs to be continued during these times of rapid technology and system changes Recommended Next Steps Phase 1 Platform Visits Completed Enter the remaining survey form data. Requires support! Analysis and report generation Derive requirements from findings and forward as appropriate Design and Initiate Phase 2 Develop plan and methodology Expand to USMC Develop website Conduct onsite briefs / train crew prior to data collection 34 HSI Lessons Learned Summary Although the Navy moves smartly into the 21st Century, there are substantial HSI issues in the Fleet. Heretofore, there has been no mechanism for addressing the wide scope of traditional and newer IT HSI topics collectively. Important Fleet HSI LL and HSI process methodology LL have been identified in this preliminary report. The final report is expected before the end of FY04. 35 Resources Carl, C. (2003). "Bloggers and Their Blogs: A Depiction of the Users and Usage of Weblogs on the World Wide Web." M.A. thesis in Communication, Culture and Technology at Georgetown University: <link> Clark, V. (2003). CNO’s Talk at HSI Symposium, June: <link> <Covenant leadership> Dienhart, P., Roos, S. & Howard, M. (2003). Field Report Marine Corps Systems Command Liaison Team Central Iraq 20 April to 25 April. FORCEnet HSI webpage (2003) (I.E. only): <link> FORCEnet: Turning Information into Power: <link> Greenspan, R. (2003). "Blogging By The Numbers," CyberAtlas <link> Human Systems Integration: <link> Lyman, P. & Varian, H. (2003). How Much Information? Regents of the University of California, October 27: <link> Naval Network Warfare Command: <link> NETWARCOM Stands Up: <link> President Roosevelt: <link> Sea Power 21: <link> SC-OAG Seeks Deckplate's Input to Shape Future Navy: <link> POC: George Edw. Seymour (DSN or 619) 553-8008 36 Contact Information Points of Contacts: George Edw. Seymour, Ph.D., Space and Naval Warfare Systems Center, San Diego, CA DSN or (619) 553-8008 E-mail: seymour@spawar.navy.mil http://www.2-SIR.com/TwinFalls/ LCDR Sally Van Horn, NETWARCOM FORCEnet Architecture Team, Norfolk, VA (757) 417-6752 x6 DSN: 537-6752 x6, E-mail: sally.vanhorn@navy.mil 37 Addenda: Backup Slides Identified Technology Issues: SPINS TAMPS SAMS Capability Issues Quotes: CHAT Information / Knowledge Management Training / Skills Management 38 HSI Technology Issue: SPINS “Had one guy on the ship creating SPINS. He was very talented. It was on 8.5 x 11" paper. Should be pictures in the cockpit. These are mission critical and go along with the ATOs. They are Joint, but standardization is a big issue. All military aircraft use them. There needs to be a better way - automated.” Recommend: This issue be investigated. 39 Identified Technology Issue: TAMPS “TAMPS machine constantly down---the rep on board was working it constantly. It is a mission computer, a single point of failure. We have to put a brick in JRT. MIDS uses it and HARM uses it.” “WEPS 1.1 is terrible.During OIF used ATACS and it was just as bad. Now we are expecting WASP (why did it take so long) and use TAMPS (but it is UNIX based; that is not good). We have a $60 million jet that is useless without the proper tools.” 40 Identified Technology Issue: Medical SAMS “We are using legacy systems in Medical SAMS (SNAP Automated Medical System), but it was designed for smaller ships (about 500 people), and then expanded for CVs. But it is archaic. Tried to make it fit Windows, but did not work. It’s terrible. Need user-friendly system that communicates with database (e.g., personnel) because people have to maintain. At NPGS they had a web-based system where all sailors logged on to check their medical readiness. Tracking immunizations and health protection topics are important.” 41 Capability Issue 1 – Chat Quotes Representative Quotes from the Fleet concerning chat’s effectiveness: “Tomahawk lived and died by chat…” “Often got advance notification of launch involvement via chat.” “Decision making information came via chat and e-mail.” Representative Quotes from the Fleet regarding their concerns: “We need chat and NetMeeting on the same system. As of today, half the ships say they do not have the bandwidth, tools or the space that the CVs have.” “Daily Intentions Message – received via official message traffic, but by the time the message was received, it was outdated, since it had been revised continually via chat.” “Used chat continuously. Two problems with chat – no timestamp associated with entries (so it was difficult to get a sense of the timing involved); if unit submerges or if connectivity is lost, there is no record of what transpired on the chat when you re-establish contact.” “Do we need a chat phone talker? Some ships did. Had 1 guy who did only type and read. Another only on GCCS and back-up chat. The Launch Area Coordinator needs to think….Sound power phones a good analogy because Fireman Tim was not paid to think but to relay information.” 42 Capability Issue 1 – Chat Quotes Chat and e-mail are tips of the digital iceberg, and the Navy must explore other IP technologies: “America Online’s AOL Instant Messenging Service (AIM) "estimates 1.6 billion AOL and AIM IMs are sent every day" (Edwards, 2003). Similarly, the Kaiser Family Foundation found that 74 percent of online teens use instant messaging (Henry J. Kaiser Foundation, 2002).” “84 percent of Internet users say they have contacted or joined online groups (Pew Internet and Family Life, 2001).” “Blogcount (http://dijest.com/bc), a Web site dedicated to finding how many blogs and bloggers are out there, suggest there are 2.4–2.9 million active weblogs ("active" means they have been updated in the last eight weeks) as of June 2003 (Greenspan, 2003).” The Navy should take the initiative to create a Naval Semantic Web. 43 Capability Issue 1 – Chat Quotes “So here's my prediction: Blogs will soon become a staple in the information diet of every serious businessperson, not because it's cool to read them, but because those who don't read them will fail. In short, blogs offer an accelerated and efficient approach to acquiring and understanding the kind of information all of us need to make business decisions... Until recently, blogs have proven to be an incredibly lousy source of information for most businesspeople. Finding and keeping up with the relevant ones is far too time-consuming. But I've recently started using a newsreader, and after spending hours setting the damn thing up, my business life has changed forever.” — John Battelle, 2004 Visiting Professor, UC Berkeley and one of the cofounders of Wired magazine. 44 Capability Issue 2 – Info Mgmt Quotes Representative Quotes from the Fleet: “How do we determine the wheat from the chaff? Need a better way of searching. Google is good. When we search on SIPRNet, we get garbage.” “Search Engine on SIPR doesn’t work.” “Lessons learned are distributed on ships. Using message traffic. Need a database or a website.” “Too many sources – message traffic, GCCS-M, GBS, SIPRNET, CNN, e-mail – need to get things from a single source.” 45 Capability Issue 3 - Skills Quotes Representative Quotes from the Fleet: “Need to train sailors better before coming to the Fleet (they are booksmart, but have difficulty troubleshooting equipment).” “Proficiency level on ships does not match requirements.” “Too much work is being dumped on the Fleet, especially in the area of providing training.” “Training for many of new systems that are being fielded isn’t adequate.” “Command only uses experienced personnel at sea.” “Lack of manpower leading up to cruise. Two check-ins needed training, rapidly. Training did not always result in necessary skills acquisition.” “Gained four new personnel just prior to cruise, useless, until end of cruise. They created more work.” “Need to standardize training and have set people give it.” “Need single source for OJT.” 46 Tactical Training Comment “I feel very strongly that we have lost our bearings when it comes to transformation because most of the talk is not backed up by solid intellectual analysis.” Konetzni also blasted technology development and experimentation across the Navy, singling out four programs for criticism: the Joint Fires Network, Navy-Marine Corps Intranet, mine warfare efforts and the Battle Force Tactical Training (BFTT) system.” Vice Adm. Albert Konetzni, deputy commander and chief of staff Atlantic Fleet, May, 2003 47