Lay down on the couch
please
It’s time for test therapy
Kai Petersen
Human factors
Looking at research there exist
10 times more research on
technology and processes
compared to human factors
Human factors
Question: What comes to mind
when you hear the word
“Human Factors”?
State a few keywords!
Looking at research there exist
10 times more research on
technology and processes
compared to human factors
Human factors – some keywords from cognitive
science/behavioral economics, psychology
Work life balance
Need theory
Self control
Positive psychology
Self esteem
Self discipline
Personality
Organizational commitment
Conformity
Social value orientation
Group dynamics
Decision making
Organizational culture
Loyalty
Alienation
Risk taking
Communication
Stress
Stereotypes
Emotion
Group polarization
Group composition
Leadership
Organizational learning
Organizational climate
Per Lenberg, Robert Feldt, Lars Göran Wallgren:
Behavioral software engineering: A definition and systematic literature review. Journal of Systems and Software
107: 15-37 (2015)
What is there to understand?
affect
Individual
Methods, tools,
techniques, work,
practices, products
Group
Human factors
Organization
SE-metrics
Pratigeces used
Productivity
Efficiency
Quality
etc.
affect
Psychological measurement
Myers-Briggs type indicator (MBTI)
Nasa task load index (Nasa TLX)
Individualized Trust Scale (ITS)
Generalized Immediacy Scale (GI)
Human factors – some keywords from cognitive
science/behavioral economics, psychology
Work life balance
Need theory
Self control
Positive psychology
Self esteem
Self discipline
Personality
Social value orientation
Group dynamics
Decision making
Organizational culture
Organizational commitment
Conformity
Stereotypes
Emotion
Group polarization
Loyalty
Alienation
Risk taking
Communication
Stress
Group composition
Leadership
Organizational learning
Organizational climate
Per Lenberg, Robert Feldt, Lars Göran Wallgren:
Behavioral software engineering: A definition and systematic literature review. Journal of Systems and Software
107: 15-37 (2015)
Personality
Persons characteristic patterns of behavior, thoughts and feelings
Evaluate the link between personality
traits and work preferences
Personality
• Method
– Questionnaire of 279 M.Sc. Level students and 47 SE professionals (earlier
study)
– Utilized statistical analysis (cluster analysis)
• Observed variables “human factors”
– Five factor model (personality)
– Emotional intelligence (ability to use, understand and manage emotions)
– Self-compassion
• Observed variables “Work preferences”
–
–
–
–
Plan vs. ad-hoc
Workload (one thing at a time vs. multiple)
Work alone vs. in a team
Responsibility with respect to the development process (all, parts, no
experience)
– Project duration
– Technical vs. softer parts
– Prioritizing tasks yourself vs. your boss
Personality indicator used
Each model has pros and cons, in
particular there are debates (e.g.
the five factor model does not
capture the complete personality
sphere), misses e.g.
manipulativeness, honesty, etc.
Openness to experience
1. I have a rich vocabulary
2. I have a vivid imagination
3. I have excellent ideas
4. I am quick to understand things
5. I use difficult words
6. I am full of ideas
7. I am not interested in abstractions
8. I do not have a good imagination
9. I have a difficulty understanding
abstract ideas
Personality - Results
Two clusters (red and blue) were identified, with
the red cluster having more intense personalities
Personality – Results
Personality – Results (2/2)
• Students with more intense personalities
– Prefer to work in a team
– Prefer to be responsible for the overall development
process
– Prefer to work with softer parts of the development
project
• Relationship between personalities and work
preferences are visible
Key findings:
Study validates assumptions (i.e. also gives confidence in instruments)
Illustrates that personalities are important to consider (and is here only focused on
a very limited scope)
Interesting from a testing perspective: What personalities make the best testers?
What testing approach (exploratory, scripted, execution, design) does one prefer?
Emotion
State of mind raised by external stimuli and directed towards
the stimulus by which they are raised (joy, fear, surprise)
Goal
Emotion
• Method
– Subjects: 4 software developers, 4 students
– Work on own software projects (familiar environments)
– Data collection – survey instrument on a screen, selfassessment manikan (SAM)
– Interviews and annotation of events during interview
– Observed variables “human factors”
• Valence (pleasure) – attractiveness of an event, object, or situation
• Arousal – intensiveness of the emotional activation (vigor, energy
vs. fatigue and tiredness)
• Dominance – change in the sensation of control of a situation
Emotion
• Measurement
unhappy
happy
calm
excited
controlled
in-control
Results
Emotion
Emotions - results
• Hypotheses tests showed that there is
significant evidence that:
– The real-time valence affective state is positively
correlated with their self-assessed productivity.
– The real-time dominance affective state is
positively correlated to their self-assessed
productivity.
– Impact of arousal could not be determined
Stress/time-pressure
• Goal
– Understand the effect of moderate time pressure
on task performance (requirements review and
test case development) and perceived workload
Stress/time-pressure
• Method
– Controlled experiment with 54 subjects inspecting
two systems
– Treatment: time pressure vs. non time pressure
• Time pressure generated through incentives (= the
faster you perform, benefits – credits – are given)
– Observed “Human factors”
• Perceived workload (NASA Task-load Index)
– Observed development performance measures
• Effectiveness (no. of defects detected during
requirements review while developing test cases, test
case score based on correctly identified input and
output variables, and created equivalence classes)
• Efficiency (mean number of defects per hour)
Stress/time-pressure
Stress/time-pressure
Results
Effectiveness: could not reject null-hypotheses that there is no difference (left
two figures), also no negative effects could be observed.
Efficiency: significant difference
Stress/time-pressure
Results
Workload: No statistical significant evidence that time pressure affects
motivation, frustration or perceived performance (note: moderate time
pressure, limited amount of time, well specified task)
What we know overall so far…
• Top studied concepts are:
– personality
• personality diversity strong predictor of success
• personality diversity positively influences communication quality
• extraversion correlates positively with the quality of the developed software
– communication
• self-efficacy has a positive effect on the intention to share knowledge
– job satisfaction
• level of job satisfaction and cohesion dropped the greater the level of conflict
• group composition and climate have an effect on software development team
performance
Road ahead
• To truly understand we need to apply
psychological measurements to understand
impact
• Impact not just means that there is an effect,
but also how large the effect is
• Need to study group behavior (e.g. a nice
example is group polarization – not studied at
all but highly relevant)
Example group polarization
Group polarization
– If a group shares opinions (i.e. is tilted in a certain
direction), and discuss/decide in a group, the
decision ends up on the extreme spectrum
– For example, studied in law
– Interesting for decision making in SE
• example: Estimation