This slide contains information in Note View. Switch to note view and print all of these out. Be sure to set your printer to black and white. You may need to change fonts if we have used some that you do not have (or if your printer or computer does funny things to our slides) George Mason University 1 Human Factors & Applied Cognitive Program George Mason University 2 Human Factors & Applied Cognitive Program George Mason University 3 Human Factors & Applied Cognitive Program Cognitive Analysis of Dynamic Performance: Cognitive process analysis and modeling Workshop Presented at the HFES/IEA-2000 Conference in San Diego, CA Wayne D. Gray, Ph.D. Deborah A. Boehm-Davis, Ph.D. George Mason University 4 Human Factors & Applied Cognitive Program Outline of Workshop Intro to the GOMS family of models Keystroke Level GOMS NGOMSL GOMS CPM-GOMS REVISED Tutorial Materials may be downloaded from: http://hfac.gmu.edu/~graypubs George Mason University 5 Human Factors & Applied Cognitive Program Definition of GOMS Characterization of a task in terms of The user's Goals The Operators available to accomplish those goals Methods (frequently used sequences of operators and sub- goals) to accomplish those goals, and If there is more than one method to accomplish a goal, the Selection rules used to choose between methods. George Mason University 6 Human Factors & Applied Cognitive Program What GOMS Is GOMS is a task analysis technique Very similar to Hierarchical Task Analysis (Indeed, GOMS is a hierarchical task analysis technique) Hard part of task analysis is goal-subgoal decomposition Hard part of GOMS is goal-subgoal decomposition Different members of the GOMS family provide you with different sets of operators With established parameters But set is not complete (extensionable) George Mason University 7 Human Factors & Applied Cognitive Program Where does GOMS go? (When do you need to do a CTA versus a TA?) Task analysis versus cognitive task analysis? What distinguishes GOMS cognitive task analysis from other task analysis techniques? E.g., Cognitive Task Analysis™ George Mason University 8 Human Factors & Applied Cognitive Program TASK ANALYSIS George Mason University 9 Human Factors & Applied Cognitive Program Time Scale for GOMS Levels of analysis (based on Newell’s Time Scale of Human Action) Syst em Analysis Act iviti es/ Processes Task Task analysis Subtasks min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters Scale (sec) Time Unit s 105 days 104 hours 103 10 min 102 World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 10 Human Factors & Applied Cognitive Program Level 1: Task Analysis: Task Subtasks Subtasks Duration Complete Argus Prime Experiment TASK SUBTASKs Complete Session #1 SUBTASKs Scenario #1 SUBTASKs Hook Target ID#1 Complete Session #2 Scenario #2 Hook Target ID#2 5 hr Complete Session #3 Scenario #3 • • • • Hook Target ID#3 • • • • Hook Target ID#n Scenario #n 1-2 hrs 12-30 min 30 sec to 3 min George Mason University 11 Human Factors & Applied Cognitive Program Levels of analysis (based on Newell’s Time Scale of Human Action) Scale (sec) Time Unit s Syst em Analysis Act iviti es/ Processes 105 days 104 hours Task Task analysis Subtasks 103 10 min 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 12 Human Factors & Applied Cognitive Program Level 2: Subtask Unit Tasks Target acquisition yes Classified? no no Target classification Feedbk check? yes Feedback check George Mason University 13 Human Factors & Applied Cognitive Program Levels of analysis (based on Newell’s Time Scale of Human Action) Scale (sec) Time Unit s Syst em Analysis Act iviti es/ Processes 105 days 104 hours Task Task analysis Subtasks 103 10 min 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 14 Human Factors & Applied Cognitive Program Level 3: Cognitive Task Analysis: Unit Tasks Activities Target acquisition yes Classified? Classified? no Target classification no Method for goal: Determine if target is classifed Step 1 Select target by mouse click Step 2 Note target ID# Determine if radio button in info Step 3 window is black Verify that info window id# is same Step 4 as target id# Feedbk check? yes Feedback check George Mason University 15 Human Factors & Applied Cognitive Program Levels of analysis (based on Newell’s Time Scale of Human Action) Scale (sec) Time Unit s Syst em Analysis Act iviti es/ Processes 105 days 104 hours Task Task analysis Subtasks 103 10 min 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 16 Human Factors & Applied Cognitive Program Level 4: Embodied Cognition: Activity Microstrategy Method for goal: Determine if target is classifed Step 1 Select target by mouse click Step 2 Note target ID# Determine if radio button in info Step 3 window is black Verify that info window id# is same Step 4 as target id# 17 http://hfac.gmu.edu/~mm George Mason University Human Factors & Applied Cognitive Program Levels of analysis (based on Newell’s Time Scale of Human Action) Scale (sec) Time Unit s Syst em Analysis Act iviti es/ Processes 105 days 104 hours Task Task analysis Subtasks 103 10 min 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 18 Human Factors & Applied Cognitive Program Level 5: Microstrategies Elements (Where GOMS doesn’t go!) Production Rules Declarative memory elements Internal (created on RHS of production rules) External (created by shifts of attention in the external environment) George Mason University 19 Human Factors & Applied Cognitive Program Levels of analysis (based on Newell’s Time Scale of Human Action) Scale (sec) Time Unit s Syst em Analysis Act iviti es/ Processes 105 days 104 hours Task Task analysis Subtasks 103 10 min 102 min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 20 Human Factors & Applied Cognitive Program Level 6: Elements Parameters (GOMS doesn’t go here either!) w -- amount of attentional capacity d -- decay rate s -- fluctuations in the strength of declarative memory elements rt -- retrieval threshold George Mason University 21 Human Factors & Applied Cognitive Program KR not KE GOMS (as with most task analysis methods) focuses on Knowledge representation, not on Knowledge elicitation There are many sources of knowledge elicitation techniques George Mason University 22 Human Factors & Applied Cognitive Program Task Analysis versus Functionality A task analysis does not guarantee functionality, see Kieras, D. E. (in press). Task analysis and the design of functionality, CRC Handbook of Computer Science and Engineering.: CRC Press, Inc. for a cogent discussion of this issue George Mason University 23 Human Factors & Applied Cognitive Program Definition of GOMS Characterization of a task in terms of The user's Goals The Operators available to accomplish those goals Methods (frequently used sequences of operators and sub- goals) to accomplish those goals, and If there is more than one method to accomplish a goal, the Selection rules used to choose between methods. George Mason University 24 Human Factors & Applied Cognitive Program Example of G-O-M-S To carry out a GOMS analysis of the following task involving a digital clock: Set the clock Top level goal: SET CLOCK George Mason University 25 Human Factors & Applied Cognitive Program Example of G-O-M-S: Goals Goals and subgoals Set Clock Set Hour Set Min George Mason University 26 Human Factors & Applied Cognitive Program Example of G-O-M-S: Operators Operators are the most elementary steps in which you choose to analyze the task. Reach <type> button Hold <type> button Release <type> button ClickOn <type> button Decide: if <x> then <y> Verify George Mason University 27 Human Factors & Applied Cognitive Program Example of G-O-M-S: Methods Top-level user goals SET-CLOCK Method for goal: SET-CLOCK Step 1. Hold TIME button Step 2. Accomplish goal: SET-HOUR Step 3. Accomplish goal: SET-MIN Step 4. Release TIME button Step 5. Return with goal accomplished Method for goal: SET-<digit> Step 1. ClickOn <digit> button Step 2. Decide: If target <digit> = current <digit>, then return with goal accomplished Step 3. Goto 1 George Mason University 28 Human Factors & Applied Cognitive Program Example of G-O-M-S: Selection rules No selection rules in this example as this clock has only ONE method for accomplishing each goal, but . . . Selection rule for goal: SET-HOUR If target HOUR ≤ 4 hours from current HOUR, then Accomplish Goal: ClickOn HOUR If target HOUR > 4 hours from current HOUR, then Accomplish Goal : Click&Hold HOUR George Mason University 29 Human Factors & Applied Cognitive Program Applications of GOMS Case 1. Design of mouse-driven text editor Case 2. Directory assistance workstation Case 3. Space operations database system (for orbital objects) Case 4. Bank deposit reconciliation system. Case 5. CAD system for mechanical design. Case 6. Television control system. Case 7. Nuclear power plant operator's associate. Case 8. Intelligent tutoring system. Case 9. Industrial scheduling system. Case 10. CAD system for ergonomic design. Case 11. Telephone operator workstation. List compiled by John & Kieras (1997a). George Mason University 30 Human Factors & Applied Cognitive Program Where does GOMS go? Usability writ large Organization Issues [anthropology, social Conceptual Issues, Mental Models [cognitive science] Procedural Issues (methods) [GOMS] Perceptual Issues [tradtional HF, cognitive engineering, Tullis, Wickens] George Mason University 31 Human Factors & Applied Cognitive Program Development process without analytic modeling George Mason University 32 Human Factors & Applied Cognitive Program Development process with analytic modeling GOMS as Analytic Modeling GOMS analysis produces a model of behavior Given a task, the model predicts the methods, or sequences of operators, that a person will perform to accomplish that task Can look at the GOMS model in different ways to qualitatively and quantitatively assess different types of performance George Mason University 34 Human Factors & Applied Cognitive Program Scope of GOMS: What it can do Predict the sequence of operators an expert will perform Predict performance time of expert users - even in real- world situations Predict learning time in relatively simple domains Predict savings due to previous learning Help design on-line help and manuals George Mason University 35 Human Factors & Applied Cognitive Program Scope of GOMS: What it can do, con’t GOMS has been applied to both: User-driven interaction “Situated” or event-driven interaction George Mason University 36 Human Factors & Applied Cognitive Program Scope of GOMS: What it might be able to do Research has made progress on Predicting the number of some types of errors see discussion in: Gray, W. D. (2000). The nature and processing of errors in interactive behavior. Cognitive Science, 24(2), 205-248. Predicting the effects of display layout on performance time George Mason University 37 Human Factors & Applied Cognitive Program Scope of GOMS: What it can't do Predict problem-solving behavior Predict how GOMS structure grows from user experience Predict behavior of casual users, individual differences... Predict the effects of fatigue, user preference, organizational impact... George Mason University 38 Human Factors & Applied Cognitive Program General Factors to Consider in GOMS Models When deciding what type of GOMS model you need, you must consider... what control structure what level of analysis whether to approximate behavior with serial or parallel processes ...different uses of GOMS models lead to different values of these factors These factors will be a recurring theme George Mason University 39 Human Factors & Applied Cognitive Program GOMS Family of Analysis Methods Keystroke-Level Model CMN-GOMS NGOMSL CMN-GOMS for Highly Interactive Tasks CPM-GOMS = “worked example” provided in this HFES workshop George Mason University 40 Human Factors & Applied Cognitive Program Break time George Mason University 41 Human Factors & Applied Cognitive Program Keystroke-Level Model: Intro The simplest of all GOMS models: OM only!!! No explicit goals or selection rules Operators and Methods (in a limited sense) only “Useful where it is possible to specify the user’s interaction sequence in detail” (CMN83, p. 259). Control structure: Flat Serial or Parallel: Serial Level of Analysis: Keystroke-level operators George Mason University 42 Human Factors & Applied Cognitive Program Keystroke-Level Model: Example TAO example area code 1 2 3 num 7 0 3 exchange 4 5 6 9 9 3 7 1 line 8 9 10 3 5 7 pin 11 12 13 14 1 2 3 4 George Mason University 43 Human Factors & Applied Cognitive Program Keystroke-Level Model: Overview Step 1: Lay out assumptions Step 2: Write out the basic action sequence (list the keystrokelevel physical operators involved in doing the task) Step 3: Select the operators and durations that will be used Step 4: List the times next to the physical operators for the task Step 4a: If necessary, include system response time operators for when the user must wait for the system to respond Step 5: Next add the mental operators and their times Step 6: Sum the times of the operators George Mason University 44 Human Factors & Applied Cognitive Program Keystroke-level Model: Operators K: Keystroke T(n): Type a sequence of n characters on a keyboard P: Point with mouse to a target on a display B: Press or release mouse button BB: Click mouse button H: Home hands to keyboard or mouse M: Mental act of routine thinking W(t): Waiting time for system to respond George Mason University 45 Human Factors & Applied Cognitive Program Card, Moran, and Newell on “Mentals” “M operations represent acts of mental preparation for applying physical operations. Their occurrence does not follow directly from the physical encoding, but from the specific knowledge and skill of the user” p. 267 “The rules for placing M’s embody psychological assumptions about the user and are necessarily heuristic, especially given the simplicity of the model” p. 267. George Mason University 46 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit. •Pointing to a cell on a spreadsheet is pointing to an argument -- no M •Pointing to a word in a manuscript is pointing to an argument -- no M •Pointing to a icon on a toolbar is pointing to a command -- M •Pointing to the label of a drop-down menu is pointing to a command -- M George Mason University 47 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Rules 1-4 are heuristics (rules of thumb) for deleting mentals “A single psychological principle lies behind all the deletion heuristics . . . physical operations in methods are chunked into submethods” p. 268 George Mason University 48 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of directmanipulation operations belonging to a cognitive unit. RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB). •That is, the “M” drops out because the “P” and “BB” belong together in a chunk -- mental unit. •The button press “BB” is fully anticipated as the cursor is being moved to the target. George Mason University 49 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit. RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB). RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first. •Works with command names -- but what is a command name in a GUI interface? •Physical actions: P(File)+ B + P(Save) + B •RULE 0: MP + MB + MP + MB •RULE 1: MPB + MPB •Does rule 2 apply to eliminate the middle mental? MPBPB ? George Mason University 50 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit. RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB). RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it. •Applies to clicking OKAY in dialog buttons after you select a command; e.g., in Powerpoint, you have selected text, gone to the FORMAT:FONT palette, clicked on bold, and now point and click on OKAY -- pointing to and clicking on OKAY is PBB, not MPBB George Mason University 51 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit. RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB). RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it. RULE 4: If a K terminates a constant string (e.g., a command name), then delete the M in front of it; but if the K terminates a variable string (e.g., an argument string), then keep the M in front of it. George Mason University 52 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators The four heuristics do NOT capture the notion of method chunks precisely -- these are only approximations Ambiguities: Is something “fully anticipated” or is something else a “cognitive unit”? Much of this ambiguity stems from variations in expertise of the users we are modeling George Mason University 53 Human Factors & Applied Cognitive Program Heuristics for inserting mental operators Basic psychological principle: physical operations in methods are chunked into submethods. RULE 0: Insert M’s in front of all K’s or B’s that are not part of argument strings proper (e.g., text or numbers). Place M’s in front of all P’s that select commands (not arguments) or that begin a sequence of direct-manipulation operations belonging to a cognitive unit. RULE 1: If an operator following an M is fully anticipated1 in an operator just previous to M, then delete the M (e.g., PMK --> PK or PMBB --> PBB). RULE 2: If a string of MK’s or MB’s belongs to a cognitive unit (e.g., the name of a command), then delete all M’s but the first RULE 3: If a K is a redundant terminator (e.g., the terminator of a command immediately following the terminator of its argument), then delete the M in front of it. RULE 4: If a K terminates a constant string (e.g., a command name), then delete the M in front of it; but if the K terminates a variable string (e.g., an argument string), then keep the M in front of it. George Mason University 54 Human Factors & Applied Cognitive Program KLM--mentals: example 1. Example: SET COLUMN WIDTH 5<cr> List the keystroke level physical operators involved in doing the task RULE 0 M+KKKK+M+KKKKKKK+M+KKKKKK+K+M+K or M+4K(set_)+M+7K(column_)+M+6K(width_)+1K(5)+M+1K(<cr>) RULE 1 M+17K(set_column_width_)+1K(5)+M+1K(<cr>) RULE 3 no change in this example RULE 2 KKKKKKKKKKKKKKKKKKK (19 K’s) No change in this example Rule 4 No change in this example George Mason University 55 Human Factors & Applied Cognitive Program KLM--mentals: example 2 Example: spellcheck “catelog” List the keystroke level physical operators involved in doing the task RULE 0 n/a (“catelog” + spellcheck do not form a cognitive unit) RULE 3 n/a RULE 2 P+M+BBBB+M+P+BB RULE 1 P+BBBB+P+BB (where BB is a mousedown + mouseup, and BBBB is a doubleclick) n/a RULE 4 n/a George Mason University 56 Human Factors & Applied Cognitive Program KLM--mentals: example 3 Example: save a file on a Mac using menus List the keystroke level physical operators involved in doing the task P+B+P+B RULE 0 M+P+M+B+M+P+M+B RULE 1 M+P+B+M+P+B RULE 2 n/a or M+P+B+P+B ??? Issue: Is this FILE-->SAVE menu selection a single cognitive unit or two? George Mason University 57 Human Factors & Applied Cognitive Program Keystroke-Level Model: m1 current Step 1: Lay out your assumptions There are several fields on the display, first thing that any error recovery method must do is to identify the field to be changed. In this case the field is the calling-card field (CCN). For purposes of this exercise, we assume the error is made in the second number of the exchange. TAO’s hands are on the keyboard Step 2: Write out the basic action sequence (the physical operators) ƒkey(ccn) + digit(14) + enterKey George Mason University 58 Human Factors & Applied Cognitive Program Keystroke-Level Model: m1 current Step 3: select the operators and durations that will be used We will use the ones from Kieras (1993). operator mental keystroke-<key> mouseDown or Up click (mouseD & Up) homing pointing w/mouse doubleClick abbrev M K B BB H P BBBB duration 1200msec 280msec 100msec 200msec 400mec 1100msec 400msec George Mason University 59 Human Factors & Applied Cognitive Program Keystroke-Level Model: m1 current Step 4: List the times next to the physical operators for the task. Method 1 of TAO task press reset function keyfKEY(ccn) type digits digit outpulse new number toenter dbase NUM op type 1 K 14 K 1 K total time time 0.28 3.92 0.28 4.48 George Mason University 60 Human Factors & Applied Cognitive Program Keystroke-Level Model: m1 current Step 5: Next add the mental operators and their times Method 1 of TAO task M before cmd press reset function key type digits M terminates argument string outpulse new number to dbase fCCN digit enter NUM op type 1 M 1 K 14 K 1 M 1 K total time time 1.20 0.28 3.92 1.20 0.28 6.88 Step 6: Sum the times of the operators Predicted time for current method is 6.88 sec (note: this time is the same regardless of “where” the error is made) George Mason University 61 Human Factors & Applied Cognitive Program Keystroke-Level Model: m2 bs/delete Step 1: Lay out your assumptions s/a model 1 except; delete key backs up and deletes each digit Step 2: Write out the basic action sequence (the physical operators) ƒkey(ccn) + delKey(10) + digit(10) + enterKey Step 3: Same operators as for model 1. George Mason University 62 Human Factors & Applied Cognitive Program Keystroke-Level Model: m2 bs/delete Step 4: List the times next to the physical operators for the task. Method 2: bs/delete press reset function key bs/delete to digit digits to retype outpulse new num to dbase total time fCCN delKey digit enter NUM op type 1 K 10 K 10 K 1 K time 0.28 2.80 2.80 0.28 6.16 George Mason University 63 Human Factors & Applied Cognitive Program Keystroke-Level Model: m2 bs/delete Step 5: Next add the mental operators and their times Method 2: bs/delete M before cmd press reset function key bs/delete to digit digits to retype verify done outpulse new num to dbase total time fCCN delKey digit enter NUM op type 1 M 1 K 10 K 10 K 1 M 1 K time 1.20 0.28 2.80 2.80 1.20 0.28 8.56 Step 6: Sum the times of the operators George Mason University 64 Human Factors & Applied Cognitive Program Keystroke-Level Model:m3 bkup/delete Step 1: Lay out your assumptions s/a model 1 except; backup key backs up without deleting. Delete key backs up and deletes Step 2: Write out the basic action sequence (the physical operators) ƒkey(ccn) + bkupKey(9) + delKey(1) + digit(1) + enterKey Step 3: Same operators as for model 1. George Mason University 65 Human Factors & Applied Cognitive Program Keystroke-Level Model: m3 bkup/delete Step 4: List the times next to the physical operators for the task. Method 3: bkup-delete press reset function key backup to digit delete digit digits to retype outpulse new num to dbase total time fCCN bkup del digit enter NUM op type 1 K 9 K 1 K 1 K 1 K time 0.28 2.52 0.28 0.28 0.28 3.64 George Mason University 66 Human Factors & Applied Cognitive Program Keystroke-Level Model: m3 bkup/delete Step 5: Next add the mental operators and their times (Your turn!!! Our answer are on the next page, no peeking!!!) Method 3: bkup-delete NUM op type time press reset function key fCCN 1 K 0.28 backup to digit bkup 9 K 2.52 del 1 K 0.28 digits to retype digit 1 K 0.28 outpulse new num to dbase enter 1 K 0.28 delete digit total time George Mason University 67 Human Factors & Applied Cognitive Program Keystroke-Level Model: m3 bkup/delete Step 5: KLM w/mentals. Method 3: bkup-delete set up workstation to retype number press reset function key backup to digit delete digit digits to retype verify done outpulse new num to dbase total time NUM op type fCCN bkup del digit enter 1 1 9 1 1 1 1 M K K K K M K time 1.20 0.28 2.52 0.28 0.28 1.20 0.28 6.04 George Mason University 68 Human Factors & Applied Cognitive Program Keystroke-Level Model: m4 zap-gp Step 1: Lay out your assumptions s/a model 1 except; Four separate function keys, zaps (deletes) either area code, exchange, line, or pin number. Retyping need only retype the zapped numbers. Step 2: Write out the basic action sequence (the physical operators) ƒkey(ccn) + zapExch(1) + digit(3) + enterKey Step 3: Same operators as for model 1. George Mason University 69 Human Factors & Applied Cognitive Program Keystroke-Level Model: m4 zap-gp Step 4: List the times next to the physical operators for the task. Your turn!! (Our answer are on the next page, no peeking!!!) Method 4: zap-gp NUM op type time total time George Mason University 70 Human Factors & Applied Cognitive Program Keystroke-Level Model: m4 zap-gp Step 5: Next add the mental operators and their times (Your turn!!! Our answer are on the next page, no peeking!!!) Method 4: zap-gp press reset function key NUM op type time fCCN 1 K 0.28 zap-group zap 1 K 0.28 digits to retype digit 3 K 0.84 outpulse new num to dbase total time enter 1 K 0.28 George Mason University 71 Human Factors & Applied Cognitive Program Keystroke-Level Model: m4 zap-gp Step 5: KLM model w/mentals Method 4: zap-gp set up workstation to retype number press reset function key zap-group digits to retype verify done outpulse new num to dbase total time NUM op type fCCN zap digit enter 1 1 1 3 1 1 M K K K M K time 1.20 0.28 0.28 0.84 1.20 0.28 4.08 George Mason University 72 Human Factors & Applied Cognitive Program Keystroke-Level Model: Summary Method 1: Current Method 2: bs/delete Method 3: bkup-delete Method 4: zap-gp predicted time 6.88 8.56 6.04 4.08 Of the three new methods, only one seems likely to be fast enough to justify expense of redesign George Mason University 73 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface The task: To issue a query on telephone numbers to one out of several databases. The users will be working in a window system, and the information that is to be looked up in the databases can be assumed to be visible on the screen. Fi l e Edi t Dat abases Fig 1 123234345345765- 4567 5678 6789 7899 4321 The screen as it looks before the menu choices. The telephone numbers to be looked up are in the window at the bottom of the screen. Thanks to Bonnie John for allowing us to use this material!!! George Mason University 74 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (2) To use this interface, the user first pulls down a menu from the menubar at the top of the screen. This menu contains the names of the databases and the user positions the mouse cursor over the name of the relevant database in this list. File Edit Databases Databases DB1 DB2 DB3 DB4 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 2 George Mason University 75 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (3) A hierarchical submenu appears with legal queries for the chosen database. This submenu contains an average of four alternatives, and the user moves the mouse until it highlights the option "query on telephone number." The user then releases the mouse button. File Edit Databases Databases DB1 DB2 DB3 DB4 query on name query on address query on telephone query on business 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 3 George Mason University 76 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (4) This causes a standard-sized dialog box to appear in the middle of the screen as shown in Figure 4. The large field at the top is initially empty. This field will eventually list the telephone numbers to be submitted to the database as a query. The dialog box does not overlap the window with the list of telephone numbers. File Edit Databases Inquiry Box Add Delete Update Inquiry on telephone number OK Cancel Help 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 4 George Mason University 77 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (5) "The user clicks in the input field (the one-line field below the prompt "Inquiry on telephone number") and types the telephone number. File Edit Databases File Edit Databases Inquiry Box Inquiry Box Add Add Delete Delete Update Inquiry on telephone number 123-4567 OK Update Inquiry on telephone number 123-4567 Cancel Help 123-4567 234-5678 345-6789 345-7899 765-4321 OK Cancel Help 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 5 Fig 6 "The user clicks on the "Add" button to add the number to the query. George Mason University 78 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (6) The state of the dialog box after the user's click on "Add." File Edit Databases File Edit Databases Inquiry Box Inquiry Box Add 123-4567 Delete Add 123-4567 Delete Update Inquiry on telephone number 123-4567 OK Update Inquiry on telephone number 123-4567 Cancel Help 123-4567 234-5678 345-6789 345-7899 765-4321 OK Cancel Help 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 7 Fig 8 "If the query is for a single telephone number, the user then clicks on the "OK" button to submit the query. George Mason University 79 Human Factors & Applied Cognitive Program Keystroke-Level Model: Dialog box interface (7) Model 1: Dialog Box, one query action/operator sequence Select dbase & query type point to "Databases" menu clickOn menu point to "DB3" point to "query on telephone" clickOn Enter telephone number point to "input field" clickOn input field move hands to keyboard type in telephone number move hand to mouse point to "Add" button clickOn "Add" button point to "OK" button clickOn "OK" button #ops KLM op time 1 1 1 1 1 P BB P P BB 1.10 0.20 1.10 1.10 0.20 1 1 1 8 1 1 1 1 1 P BB H K H P BB P BB 1.10 0.20 0.40 2.24 0.40 1.10 0.20 1.10 0.20 total time 10.64 George Mason University 80 Human Factors & Applied Cognitive Program Keystroke-Level Model: Model 1: Dialog Box, one query action/operator sequence Select dbase & query type begin direct-manipulation sequence point to "Databases" menu clickOn menu point to "DB3" point to "query on telephone" clickOn "query on telephone" Enter telephone number begin direct-manipulation sequence point to "input field" clickOn input field move hands to keyboard get number type in telephone number begin terminate entry move hand to mouse point to "Add" button clickOn "Add" button Exit palette begin close palette point to "OK" button clickOn "OK" button Dialog box interface (8) #ops KLM op time 1 1 1 1 1 1 M P BB P P BB 1.20 1.10 0.20 1.10 1.10 0.20 1 1 1 1 1 8 1 1 1 1 M P BB H M K M H P BB 1.20 1.10 0.20 0.40 1.20 Must either reread it or recall it. 2.24 1.20 0.40 1.10 0.20 1 M 1 P 1 BB total time 1.20 1.10 0.20 16.64 George Mason University 81 Human Factors & Applied Cognitive Program Keystroke-Level Model: Pop-up menu (1) As previously mentioned, it can be assumed that the telephone number(s) in question is/are already visible on the screen. (Figure 9) File Edit Databases File Edit Databases 123-4567 234-5678 345-6789 345-7899 765-4321 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 9 DB1 DB2 DB3 DB4 Fig 10 To query a number, the user moves the mouse cursor to the telephone number on the screen and clicks on the mouse button. This causes a pop-up menu to appear over the number with one element for each database for which queries can be performed with telephone numbers as keys. (Figure 10) George Mason University 82 Human Factors & Applied Cognitive Program Keystroke-Level Model: Pop-up menu (2) The user moves the mouse until the desired database is highlighted in the popup menu. (Figure 11) File Edit Databases 123-4567 234-5678 345-6789 345-7899 765-4321 DB1 DB2 DB3 DB4 File Edit Databases 123-4567 234-5678 345-6789 345-7899 765-4321 Fig 11 Fig 12 The user then clicks the mouse button (the system knows which number to search on because the user pointed to it when calling up the menu). (Figure 12) George Mason University 83 Human Factors & Applied Cognitive Program Keystroke-Level Model: Pop-up menu (3) •Your turn!! Action/operator sequence only. No mentals. (Our answer are on the next page, no peeking!!!) Model 3: Pop-Up Menu, one query action/operator sequence Enter telephone number #ops KLM op time total time George Mason University 84 Human Factors & Applied Cognitive Program Keystroke-Level Model: Pop-up menu (3) Now try adding the mentals. (Our answer are on the next page, no peeking!!!) Model 3: Pop-Up Menu, one query action/operator sequence Enter telephone number #ops KLM op time point to first telephone number 1 P 1.10 clickOn number 1 BB 0.20 point to "DB3" 1 P 1.10 clickOn 1 BB 0.20 total time 2.60 George Mason University 85 Human Factors & Applied Cognitive Program Keystroke-Level Model: Pop-up menu (4) KLM w/mentals Model 3: Pop-Up Menu, one query action/operator sequence Enter telephone number begin direct-manipulation sequence point to first telephone number clickOn number point to "DB3" clickOn #ops KLM op 1 1 1 1 1 M P BB P BB total time time 1.20 1.10 0.20 1.10 0.20 3.80 George Mason University 86 Human Factors & Applied Cognitive Program KLM vs “expert judgment” Cold heuristic estimates 12 Warm heuristic estimates 10 Hot heuristic estimates 15 original GOMS (L&C College) 19 CMU students 19 PSYC645 Sp96 12 Dialog 1 query mean sd 20.30 26.10 12.90 10.70 13.80 6.10 16.60 3.70 14.70 3.30 15.76 2.97 Box Pop-Up 2 queries 1 query mean sd mean sd 29.40 30.40 8.00 7.40 21.40 21.60 4.70 2.70 20.00 9.50 6.10 3.50 22.60 5.40 5.80 0.80 22.60 4.80 4.60 1.00 24.66 3.28 5.02 1.78 User testing 15.40 25.50 N 20 3.00 Difference from UT [UT - <condition> = ??] Cold heuristic estimates -4.90 Warm heuristic estimates 2.50 Hot heuristic estimates 1.60 original GOMS (L&C College) -1.20 CMU students 0.70 PSYC645 Sp96 -0.36 -3.90 4.10 5.50 2.90 2.90 0.84 4.70 4.30 -3.70 -0.40 -1.80 -1.50 -0.30 -0.72 0.60 Menu CV 2 queries sd as % of mean sd means 14.00 15.10 110% 9.10 5.30 84% 9.40 5.50 50% 11.20 1.90 21% 8.70 1.80 22% 9.30 3.09 20% 6.50 0.90 18% -7.50 -2.60 -2.90 -4.70 -2.20 -2.80 George Mason University 87 Human Factors & Applied Cognitive Program Break time (Lunch Time!) George Mason University 88 Human Factors & Applied Cognitive Program NGOMSL Natural Language GOMS based on structured natural language notation and a procedure for constructing them models are in program form Control Structure: Hierarchical goal stack Serial or parallel: Serial Level of Analysis: As necessary for your design question George Mason University 89 Human Factors & Applied Cognitive Program NGOMSL - why? More powerful than KLM. Much more useful for analyzing large systems More built-in cognitive theory Provides predictions of operator sequence, execution time, and time to learn the methods George Mason University 90 Human Factors & Applied Cognitive Program NGOMSL - Overall Approach Step 1: Perform goal/subgoal decomposition Step 2: Develop a method to accomplish each goal List the actions/steps the user has to do goal (at as general and highlevel as possible for the current level of analysis) Identify similar methods/collapse where appropriate Step 3: Add flow of control (decides) Step 4: Add verifies Step 5: Add perceptuals, etc. Step 6: Add mentals for retrieves, forgets, recalls Step 7: Add times for each step Step 8: Calculate total time George Mason University 91 Human Factors & Applied Cognitive Program NGOMSL - Example Car clock Presetting radio stations simple perverse George Mason University 92 Human Factors & Applied Cognitive Program NGOMSL--Car Radio example (1) Provides predictions of methods and operators used to complete a task. If you provide estimates of operator-duration, you can get predictions of error-free expert performance time. SEEK AM – ON TUNE FM EJ + 2:41 OFF BASS 1 2 3 4 5 6 TREBLE – + – + BALANCE L R FADE L R PUSH CLOCK George Mason University 93 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (2) Goal/subgoal hierarchy set clock set mode turn knob depress knob (and keep it depressed) set hour set minute press tune “-” button press tune “+” button release knob George Mason University 94 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (3) Assumptions User's hands are NOT on the buttons at the beginning. Time to move hand & arm to time change level is estimated by 2ft "reach" from Barnes (1963): 410 msec. D = device time, time for clock to move forward one number, = 500 msec (we estimate this on the next slide) I am assuming a 12-hr clock. Seems good assumption for a car clock. Radio is off at beginning and end. Begin knowing the time you want to set the clock to. George Mason University 95 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (4) D = device time, time for clock to move forward one number, = 500 msec 290 Visual Perception perceive info (X) 50 Cognitive Operators 50 verify info (X) initiate release (X) 100 Right Hand release (X) Minimum time required to perceive clockTime=targetTime and to release the toggle lever George Mason University 96 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (5) Method for goal: SET-CLOCK Accomplish goal: set Mode to Step 1 set Clock Decide: If curHr°targetHr Then: Accomplish goal: Change Hour Step 2 display Decide: If curMin°targetMin Then: Accomplish goal: Change Step 3 Minute display stmt time oper 0.1 op time tot time 0.10 0.1 0.10 0.1 0.10 0.1 0.10 Step 4 Release "vol/on/off" knob 0.1 B Step 5 Return with goal accomplished 0.1 0.10 assumptions 0.20 0.10 George Mason University 97 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (6) Method for goal: set Mode to set Clock Step 1 Recall "vol/on/off" is modeSwitch Step 5 Step 6 Decide: If location of modeSwitch is not known then Locate modeSwitch Reach & Grasp "volume/on/off" knob Turn knob Home thumb to "volume/on/off" knob Press and hold knob Step 7 Return with goal accomplished Step 2 Step 3 Step 4 stmt time oper 0.1 op time tot time 0.10 0.1 M 0.50 0.60 0.1 M 1.20 1.30 0.1 R 0.1 T 0.48 0.34 0.58 0.44 0.1 H 0.1 B 0.40 0.10 0.50 0.20 0.1 assumptions retrieval from LTM or equivalent (location and perception of labels?) needed and takes - 500 msec For this example we assume that location is NOT known. If known then tot. time for this step would be the stmt time, 0.10 sec source: Barnes, 1963, reach = 410 msec, grasp = 070 msec source: Barnes, 1963 0.10 George Mason University 98 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (7) stmt time Method for goal: Change <time> display Decide: If location of setTime switch is not known then Locate Step 1 setTime switch Decide: If finger is NOT on setTime switch then: Reach to Step 2 setTime switch Step 3 Step 4 Retrieve-from-LTM that <time> = "+ or -" Step 5 Determine current setting Press and hold setTime switch to <time> Step 6 Step 7 Step 8 Hold until <curTime> = <targetTime>, (500 x #digits) Verify display <time> Return with goal accomplished oper op time 0.1 tot time 0.10 assumptions 0.1 M 1.20 1.30 assume that loc is not known 0.1 R 0.41 0.51 0.50 retrieval from LTM or equivalent (location and perception of labels?) 0.60 needed and takes - 500 msec 0.1 M 0.1 P 0.32 0.1 B 0.10 W 0.1 M 0.1 1.20 based upon cpm-goms analysis it should take 420 msec for eye to move to clock 0.42 and for user to perceive the hour. cpm-goms calc. is -250 msec, go with 0.20 KLM System response time is 500 msec/digit, this is enough time for Ss to perceive and initiate keyUp response. 1.30 0.10 George Mason University 99 Human Factors & Applied Cognitive Program NGOMSL-- Car Radio example (8) time for SetClock goal time for setMode goal time for setHour from 1 to 10 time to setMin from 56 to 50 total 0.70 3.82 9.03 31.53 45.08 George Mason University 100 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting a station (1) George Mason University 101 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting a station (2) Goal hierarchy for optimal design PreSET Network Show Set Mode to Radio Locate Station Set PreSET etc Select AM/FM seek & identify press & hold preset button until sound returns Note differences between this and previous slide. George Mason University 102 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting a station (3) MODEL 1: SETTING A PRESET TO A NETWORK SHOW OPTIMAL DESIGN Method for goal: preset network show Step1: Decide: IF radio is not playing, THEN Accomplish goal: set-mode-to-radio Step2: Decide: IF show N.E. intended show, THEN Accomplish goal: locate-station Step3: Accomplish goal: set-preset Step4: Return with goal accomplished Method for goal: locate-station Step1: Decide: IF band N.E. desired band, THEN Accomplish goal: select AM/FM Step2: Accomplish goal: seek&identify Step3: Return with goal accomplished Method for goal: select AM/FM Step1: Reach to button for desired band Step2: Press button for desired band Step3: Verify band correct Step4: Return with goal accomplished George Mason University 103 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting a station (4) Method for goal: seek&identify Step1: Reach to right side of "Seek" button Step2: Press "Seek" button Step3: Decide: IF show N.E. intended show, THEN Goto 2 Step4: Verify show correct Step5: Return with goal accomplished Method for goal: set-preset Step1: Reach to desired "Preset" button Step2: Press and hold "Preset" button Step3: Wait until sound returns Step4: Remove finger from "Preset" button Step5: Return with goal accomplished George Mason University 104 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting perverse (1) Goal hierarchy for perverse design PreSET Network Show Locate Station Set Mode to Radio Set PreSET etc Select AM/FM Enter seek mode Delete old PreSET Initiate seek seek & identify Exit seek MODE Enter delete MODE Enter PreSet Button Assign new PreSET Exit delete MODE Enter PreSet MODE Enter PreSet Button Exit PreSet MODE George Mason University 105 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting perverse (2) Radio for Perverse Preset Design SEEK MODE AM FM EJ SEEK – ON TUNE ERASE PRESET + OFF 2:41 MEMORIZE PRESET BASS 1 2 3 4 5 6 TREBLE – + – + BALANCE L R FADE L R PUSH CLOCK George Mason University 106 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting perverse (3) MODEL 2: SETT ING A PRESET T O A NET WORK SHOW PERVERSE DESIGN Method for goal: preset network show (s/a optimal) Step1: Decide: IF radio is not playing, THEN Accomplish goal: set-mode-to-radio Step2: Decide: IF show N.E. intended show, THEN Accomplish goal: locate station Step3: Accomplish goal: set-preset Step4: Return with goal accomplished Method for goal: locate-station (s/a optimal) Step1: Decide: IF band N.E. desired band, THEN Accomplish goal: select AM/FM Step2: Accomplish goal: initiate-seek Step3: Return with goal accomplished Method for goal: select AM/FM (s/a optimal) Step1: Put finger on button for desired band Step2: Press button for desired band Step3: Verify band correct Step4: Return with goal accomplished George Mason University 107 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting perverse (4) Method for goal: initiate-seek Step1: Accomplish goal setMode-SEEK Step2: Accomplish goal: seek&identify Step3: Accomplish goal: exit seek mode Step4: Return with goal accomplished Method for goal: setMode-<mode> Step1: Reach for <mode> button Step2: Press <mode> button Step3: Verify <mode> correct Step4: Return with goal accomplished Method for goal: seek&identify (s/a optimal) Step1: Reach to right side of "Seek" button Step2: Press "Seek" button Step3: Decide: IF show N.E. intended show, THEN Goto 2 Step4: Verify show correct Step5: Return with goal accomplished Method for goal: set-preset Step1: Accomplish goal: delete-previous-preset Step2: Accomplish goal: assign-new-preset Step3: Return with goal accomplished George Mason University 108 Human Factors & Applied Cognitive Program NGOMSL-- PreSetting perverse (5) Method for goal: delete-previous-preset Step1: Accomplish goal: setMode-deletePreSet-on (setMode-<mode>) Step2: Accomplish goal: enter-preset-button Step3: Accomplish-goal: setMode-deletePreSet-off (setMode-<mode>) Step4: Return with goal accomplished Method for goal: assign-new-preset Step1: Accomplish goal: setMode-preset-on (setMode<mode>) Step2: Accomplish goal: enter-preset-button Step3: Accomplish-goal: setMode-preset-off (setMode<mode>) Step4: Return with goal accomplished Method for goal: enter-preset-button Step1: Reach to desired "Preset" button Step2: Press "Preset" button Step3: Return with goal accomplished George Mason University 109 Human Factors & Applied Cognitive Program NGOMSL--comparison of optimal and perverse designs Optimal number of different methods Perverse 6 11 6 18 number of different steps 21 38 number of steps required 21 66 number of methods used George Mason University 110 Human Factors & Applied Cognitive Program Break time George Mason University 111 Human Factors & Applied Cognitive Program CPM-GOMS: The Embodied Cognition level Analyzing activities into microstrategies Syst em Analysis Act iviti es/ Processes Task Task analysis Subtasks min Subtask Unit Task Analysis Unit Tasks 101 10 sec Unit task Cognitive task analys is Activities 100 1 sec Activity Embodied cognition Microstrategies 10-1 100 ms Microstrategy Computational models of embodied cognition Elements 10-2 10 ms Elements Architectural Parameters 10-3 1 ms Parameters Scale (sec) Time Unit s 105 days 104 hours 103 10 min 102 World ( th eory) Bound ed Rationality Cognitive Band Biological Band George Mason University 112 Human Factors & Applied Cognitive Program CPM-GOMS Extension to GOMS Critical Path Method (or Cognitive Perceptual Motor) -GOMS Control structure: Relaxed hierarchy, can be interrupted and continued based on new information in the world Serial or Parallel: Parallel Level of Analysis: Primarily elementary perceptual, cognitive and motor operators George Mason University 113 Human Factors & Applied Cognitive Program CPM-GOMS - why? Need for analysis suitable for parallel activities Human cognition is embodied cognition In some cases we need to understand the interleaving and interdependencies between elementary cognitive, perceptual, and motor operations George Mason University 114 Human Factors & Applied Cognitive Program Brief Introduction to PERT Charts Talk to real estate agents 20 20 7 Show site to management Visit potential locations 4/21/89 5/18/89 3/24/89 4/20/89 3/15/89 3/23/89 0 0 Start project Propose location 3/15/89 3/15/89 4/21/89 5/4/89 20 3/15/89 4/11/89 Make presentation to Committee Investigate zoning laws 4/20/89 4/20/89 Get approval from Board of Directors 1 10 5/19/89 5/19/89 15 Prepare offer 4/21/89 5/11/89 Aadapted from "MacProjectII: Getting Started" (Claris Corp., 1989) George Mason University 115 Human Factors & Applied Cognitive Program CPM-GOMS Schedule Charts: A Notation for Representing Parallelism. System Operations System Event Call Arrival Tone Visual Perceptual Operations Aural System Event display COIN PRE display 0+ Perceive Tone Perceive display Cognitive attend attend initiate eye verify Operations aural display movement tone R-hand verify 0+ attend display Customer Pause verify COIN PRE Initiate greeting Perceive Customer Speech attend initiate aural function customer key press R-home to function key CPM-GOMS Model L-hand M otor Operations Verbal EyeM ovement Perceive display New England P ublic Telephone may I help y ou? eye movement George Mason University 116 Human Factors & Applied Cognitive Program But are composites of simpler, basiclevel activities Activities Microstrategies Unit-tasks We will illustrate these in the context of moving and clicking a mouse George Mason University 117 Human Factors & Applied Cognitive Program George Mason University 118 Human Factors & Applied Cognitive Program 0 SLOW-M/C ne w -curs or-loca tion 10 0 10 0 pe rce ivecursor @ta rge t pe rceive -ta rge t Predicted time: 845 msec START in iti tatem ovecurs or 50 50 50 atte ndta rge t 50 veri fyta rge t po s in iti ate POG atte ndcurs or@ ta rge t 50 verifycursor@ ta rget 50 845 initiatem ous eDn END 10 0 54 5 m ovecursor m ous eDn 30 POG 0 FAST-M/C ne w-curs or-lo cati on 10 0 pe rceive ta rge t Predicted time: 695 msec 50 START in iti tatem ovecurs or 50 50 atte ndta rge t in iti ate POG 695 initiatem ous eDn END 10 0 54 5 m ovecursor 119 veri fyta rge t po s 50 m ous eDn George Mason University 30 POG Human Factors & Applied Cognitive Program George Mason University 120 Human Factors & Applied Cognitive Program The Button Study ?? ?? ?? ?? ?? ?? ?? home ?? George Mason University 121 Human Factors & Applied Cognitive Program 0 0 displa y-change ne w-curso r-locatio n 10 0 10 0 10 0 pe rc eivetarge t-btn pe rc eive-dis play pe rc eive-cursorlocation Predicted time 970 sec 50 START mo use Dn 50 attend di spla ychang e 15 0 verifydispla ychange 50 initiate movecurs or 50 50 atte ndvisua l 50 initiate -POG HOME verifycurs orloc 50 970 initiate -mouse Dn END mouse Dn mo ve cursor TARGET 30 POG 122 atte ndcurs or @ btn 50 90 30 1 mouse Up verify targe t -loc 50 George Mason University Human Factors & Applied Cognitive Program 0 0 di spla y-chan ge ne w-curso r-locatio n 10 0 10 0 pe rc eivetarge t-btn pe rc eive-cursorlocation Predicted time: 820 msec 50 START mo use Dn 15 0 initiate movecurs or 50 atte ndvisua l 50 50 verify targe t -loc initiate -POG verifycurs or -loc 50 820 initiate -mouse Dn END mouse Dn mo ve cursor TARGET HOME 30 123 atte ndcurs or @ btn 50 90 30 1 mouse Up 50 POG George Mason University Human Factors & Applied Cognitive Program Experimental Data Predicted Found Absolute Difference HOME-toTARGET 970 msec 994 msec (19.7) TARGET-toHOME 820 msec 840 msec (19.7) 2.41% 2.38% Difference 150 msec 154 msec Data and examples are from: Gray, W. D., & Boehm-Davis, D. A. (in press). Milliseconds Matter: An introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experiment Psychology: Applied. Advanced copy available from: http://hfac.gmu.edu/~mm George Mason University 124 Human Factors & Applied Cognitive Program Telephone Operator Workstation: The Real-World Problem A telephone company wanted to replace old telephone operator workstations with a new workstation. For this company, each second saved per call translates into a savings of $3 million/year in operating costs. Will the new workstation be faster than the current workstation, and if so, how much will it save each year? George Mason University 125 Human Factors & Applied Cognitive Program Telephone Operator Workstation: The Task Assist a customer in completing calls Record the correct billing information NOT directory assistance George Mason University 126 Human Factors & Applied Cognitive Program Telephone Operator Workstation: An Example Call Workstation: TAO: Customer: TAO: Workstation: TAO: TAO: “Beep” “New England Telephone may I help you?” “Operator, bill this to 412-555-1212-1234.” Hits a series of function & numeric keys Displays the authorization for the calling card “Thank you.” Hits a key to release the workstation so another call can come in George Mason University 127 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Goals Overall Goal Handle the call Subgoals explicitly trained Determine and enter who pays for the call Determine and enter the appropriate billing rate Determine when the call is complete and release the workstation George Mason University 128 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Operators Criteria for selecting a level of analysis Captures observed behavior Captures distinctions between workstations Different levels of analysis Unit Task Level Functional Level Cognitive task analysis Level Embodied cognition Level George Mason University 129 Human Factors & Applied Cognitive Program Telephone Operator Workstation Operators: The cognitive task analysis level Operators reflect the activities the TAO must perform • listen-to-beep • read-screen • greet-customer • listen-to-customer • enter-command • enter-credit-card-number • thank-customer George Mason University 130 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Cognitive Task Analysis goal: handle-calls . goal: handle-call . . goal: initiate-call . . . goal: receive-information . . . . listen-to-beep . . . . read-screen . . . goal: request-information . . . . greet-customer . . . . . . goal: enter-who-pays . goal: receive-information . . listen-to-customer . . . . . . . . . . . . . goal: enter-information . . enter-command . . enter-calling-card-number goal: enter-billing-rate . goal: receive-information . . read-screen Workstation: beep Workstation: displays information TAO: "New England Telephone may I help you?" Customer: “Operator, bill this to 412-555-1212-1234.” TAO: hit F1 key TAO: hit 14 numeric keys and so on George Mason University 131 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Cognitive Task Analysis Problems with Assumption of Sequential Operators System RT LISTEN-TO-BEEP READ-SCREEN GREET-CUSTOMER LISTEN-TO-CUSTOMER ENTER-COMMAND ENTER-CREDITCARD-NUMBER THANK-CUSTOMER 0 Time of call (seconds) 13 George Mason University 132 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Embodied Cognition Level Operators at the level of elementary operations of Cognition Perception Motor acts Uses the Critical Path Method George Mason University 133 Human Factors & Applied Cognitive Program Telephone Operator Workstation: CPM-GOMS Level 400 System RT other systems 330 system-rt(1) system-rt(2) 0 workstation display time 30 begin-call 30 display-info(2) display-info(1) 100 290 Visual Perceptual Operators 100 perceiveBEEP Aural 50 Cognitive Operators attend-auralBEEP 50 attendinfo(1) perceivebinaryinfo(2) perceivecomplexinfo(1) 50 50 initiate-eyemovement(1) 150 perceivesilence(a) verifyBEEP 50 verifyinfo(1) 50 attendinfo(2) 50 verifyinfo(2) 50 initiategreeting L-Hand Movements Motor Operators R-Hand Movements LISTEN-FOR-BEEP activity-level goal READ-SCREEN(2) activity-level goal 1570 Verbal Responses 30 "New England Telephone may I help you?" eye-movement (1) Eye Movements READ-SCREEN(1) activity-level goal GREET-CUSTOMER activity-level goal George Mason University 134 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Making Quantitative Predictions Use the CPM-GOMS model of the benchmark call on the current workstation as a baseline Modify the model to reflect design decisions substitute the layout, response times and procedures of the proposed workstation propose new features or designs obtain quantitative predictions of the effects on performance time George Mason University 135 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Quantitative Predictions Difference between the proposed workstation and the current workstation for 15 benchmark calls 2 Seconds 1.5 1 0.5 0 cc01 cc02 cc03 cc04 cc05 cc06 cc07 cc08 cc09 cc10 cc11 cc12 cc13 cc16 cc18 -0.5 Call Category George Mason University 136 Human Factors & Applied Cognitive Program Telephone Operator Workstation: Qualitative Explanations Examine the critical path of the different models to explain the quantitative predictions. E.g., why this call is longer on the proposed than on the current workstation. Current Works tation operators removed from slack time Proposed Workstation Beginning of call George Mason University 137 Human Factors & Applied Cognitive Program Qualitative Explanations Models as explanation. Current Works tation Proposed Workstation operators added to critical path End of call 138 George Mason University Human Factors & Applied Cognitive Program Uses of CPM-GOMS in Design Project Ernestine emphasized Quantitative comparisons between alternative systems Qualitative explanations for differences between alternative systems George Mason University 139 Human Factors & Applied Cognitive Program Uses of CPM-GOMS in Design CPM-GOMS can also be used to Direct design effort Bracketing, Profiling, and Diagnosis George Mason University 140 Human Factors & Applied Cognitive Program Directing Design Effort Designing telephone operator workstations: Should we redesign keyboards? screens? system response times? George Mason University 141 Human Factors & Applied Cognitive Program Directing Design Effort Keyboards & Screens How much of the time is each activity on the critical path? Call Cat. cc01 cc02 cc03 cco4 cc05 cc06 cc07 cc08 ... Average Sys RT Talking Keying Reading Ring/Coin 25% 40% 1% 3% 31% 3% 93% 0% 4% 0% 20% 71% 2% 6% 0% 25% 31% 6% 4% 35% 19% 44% 3% 2% 22% 12% 79% 2% 6% 0% 30% 57% 8% 3% 0% 26% 41% 23% 10% 0% ... ... ... ... ... 16% 64% 6% 5% 8% George Mason University 142 Human Factors & Applied Cognitive Program Directing Design Effort Bracketing Profiling Diagnosis George Mason University 143 Human Factors & Applied Cognitive Program Bracketing: Slow & Fastest Reasonable Description FASTEST REASONABLE Microstrategy SLOW BEST FIT KEYPRESSMOVE MOVE-CLICK SLOW-KP/M P1 FAST-KP/M P2 MED-KP/M SLOW-M/C FAST-KP/M move from zoom-point to EXP2 button CLICK-MOV E SLOW-C/M FAST-C/M mous e down on EXP2 bu tton MOVE-CLICK SLOW-M/C P1 SLOW-C/M P2 MED-C/M SLOW-M/C move from EXP2 bu tton to way -point CLICK-MOV E SLOW-C/M mous e down on way -point MOVE-CLICK SLOW-M/C CLICKKEYPRESS SLOW-C/KP P1 SLOW-C/KP P2 MED-C/KP FAST-C/KP KEYPRESSMOVE MOVE-CLICK SLOW-KP/M P1 SLOW-KP/M P2 MED-KP/M SLOW-M/C FAST-KP/M Part 1: K3 to mDn@zoo mPt from keyp ress to move cursor mous e down on zoo m-point SLOW-M/C SLOW-M/C Part 2: mDn@zoo mPt to mDn@EXP2 btn SLOW-M/C Part 3: mDn @ EXP2 btn to mDn @ trgt P1 MED-C/M P2 SLOW-C/M SLOW-M/C MED-C/M SLOW-M/C Part 4: mDn @ trgt to K2 keyp ress to select NA V-his displa y Part 5: K2 to mDn @ NavDesg B tn move from center to NAVDE SG button mous e down on NAV DESG button SLOW-M/C SLOW-M/C George Mason University 145 Human Factors & Applied Cognitive Program Bracketing SLOW FASTEST-REASONABLE data 1250 1000 slow-2 750 500 250 0 1 2 3 4 5 part George Mason University 146 Human Factors & Applied Cognitive Program Profiling: BEST FIT models Description FASTEST REASONAB LE Microstrategy SLOW BEST FIT KEYPRESSMOVE MOVE-CLICK SLOW-KP/M P1 FAST-KP/M P2 MED-KP/M SLOW-M/C FAST-KP/M move from zoom-point to EXP2 button CLICK-MOV E SLOW-C/M FAST-C/M mous e down on EXP2 bu tton MOVE-CLICK SLOW-M/C P1 SLOW-C/M P2 MED-C/M SLOW-M/C move from EXP2 bu tton to way -point CLICK-MOV E SLOW-C/M mous e down on way -point MOVE-CLICK SLOW-M/C CLICKKEYPRESS SLOW-C/KP P1 SLOW-C/KP P2 MED-C/KP FAST-C/KP KEYPRESSMOVE MOVE-CLICK SLOW-KP/M P1 SLOW-KP/M P2 MED-KP/M SLOW-M/C FAST-KP/M Part 1: K3 to mDn@zoo mPt from keyp ress to move cursor mous e down on zoo m-point SLOW-M/C SLOW-M/C Part 2: mDn@zoo mPt to mDn@EXP2 btn SLOW-M/C Part 3: mDn @ EXP2 btn to mDn @ trgt P1 MED-C/M P2 SLOW-C/M SLOW-M/C MED-C/M SLOW-M/C Part 4: mDn @ trgt to K2 keyp ress to select NA V-his displa y Part 5: K2 to mDn @ NavD esg Btn move from center to NAVDE SG button mous e down on NAV DESG button SLOW-M/C SLOW-M/C George Mason University 147 Human Factors & Applied Cognitive Program Profiling SLOW BEST-FITTING data FASTEST-REASONABLE 1250 P1 1000 msec 750 500 250 0 1 2 3 4 5 part George Mason University 148 Human Factors & Applied Cognitive Program Diagnosis: example CLICK-MOVE Used in part 2 and part 3 Use of slower microstrategies may reflect a confusion among the three unit tasks Not perceptually distinct Participants might have had to rely more on memory than is usual for interactive behavior with current interactive technology George Mason University 149 Human Factors & Applied Cognitive Program Summary of CPM-GOMS Predicts expert performance time and provides qualitative explanations for tasks involving parallel activities Once the models are built, schedule charts allow designers to rapidly play what-if games with design ideas and parameters Not intended to predict the occurrence of learning time, or errors - other forms of GOMS models are more helpful there Subject to the limitations of GOMS models in general (no casual use, fatigue, etc.) George Mason University 150 Human Factors & Applied Cognitive Program Summary of Workshop Intro to the GOMS family of models Keystroke Level GOMS NGOMSL GOMS CPM-GOMS http://hfac.gmu.edu/~graypubs http://hfac.gmu.edu/~mm George Mason University 151 Human Factors & Applied Cognitive Program George Mason University 152 Human Factors & Applied Cognitive Program George Mason University 153 Human Factors & Applied Cognitive Program George Mason University 154 Human Factors & Applied Cognitive Program