David Greathead
David.Greathead@ncl.ac.uk
TA Weinberg
2005 Dircshop
Outline of TA Weinberg
• Three main aspects of TA Weinberg
– Inspired by Gerry Weinberg’s ‘Psychology of
Programming’
– Continue work on Code Review
– Examine personality effects on the Valladolid
Online Programming Competition
– Examine personality effects on code
comprehension
Code review research
•
•
•
•
Follow on from previous work
Replicated with different piece of code
Similar (but not identical results)
One criticism was homogeneous set of
participants
• Currently attempting to replicate the study
in a Brazilian university
The Valladolid competition
•
•
•
•
•
An online programming competition
Several thousand competitors
Several hundred programming problems
Choose a particular problem
Examine solutions, and type and number
of failed submissions
• Have participants fill in online background
questionnaire and MBTI style personality
questionnaire
Details of code comprehension
experiment
• Initial plan was to examine design
– Very difficult to do this in the timescale
• How do you objectively decide what makes a design ‘good’?
• Decided on code comprehension due to
experience of code review research (difficulty
some students had with the task)
• Present participants with a piece of java code
– Lift simulator program
The lift simulator
• Number of preset options
– Lift moves up 1 floor and collects passenger
– 3 lifts, 2 passengers
– Random rider insertion
• Number of user definable options
–
–
–
–
–
Number of lifts
Number of floors
Number of passengers
Maximum delay before decision
Random seed
• SCREENSHOT FROM LIFT SIM?
• VIDEO FROM LIFT SIM?
Details of code comprehension
experiment
• Pose specific questions, such as:
– Where do the lifts “rest”
– If no lifts are moving and there are two or
more call requests, does more than one lift
move?
– A person presses the button to go up. Does
the upwards moving lift always stop at their
floor?
• Offer multiple choice answers for ease of
assessment
Details of code comprehension
experiment
• Answers to questions cannot be assumed
as the lift simulator has its own quirks
– A person will not get into a lift unless it is
‘their’ lift
– People will wait for the lift doors of ‘their’ lift to
let someone out, close and open again before
getting in, if the lift is changing direction
Expanding the program
• Have participants consider methods for upgrading
the program
– Give them requirements for a new addition to the
program (e.g. add an express lift, which doesn’t stop on
floors 2-6).
– Offer alternative designs.
– Participants have to choose best design.
• For example:
– Q. Which design is best, A, B, C, or D?
» 1. B because the others do not meet requirements 2, 3 or 4.
» 2. B because the others do not meet requirements 5 or 6.
» 3. C because the others do not meet requirements 1, 5 or 9.
» 4. None of the designs meet all of the requirements.
Other factors to measure
• MBTI type
– Are any particular types better at the task?
• Cognitive style
– Specifically field dependence / independence
• Impact of commented assertions
– Two experimental groups, one with and one without assertions
commented in
– Assumed that participants with commented code will find the
task easier, but the question is, how much easier is it?
– Is any combination of MBTI and cognitive style better able to
cope with the absence of the assertion comments?
• Level of experience
– How much does experience influence ability to cope without
comments?
Commented
Code
Uncommented
Code
Which factor is a better indicator of performance?
How much easier is commented code to comprehend?
MBTI
Which type (if any) is best?
Cognitive Style
(Field dependence / independence)
Which style (if any) is best?
GROUP A
40 Level 2 Students
GROUP B
40 Level 2 Students
GROUP C
40 Level 3 Students
GROUP D
40 Level 3 Students
GROUP E
20 Professionals
GROUP F
20 Professionals
Problems
• Even looking only at the simulator part of the
code (not including GUI, etc.), there is a lot of
code to look at
– Use some form of abstraction
– Have participants decide what would happen in a
given situation by examining interaction of modules
• Not everyone likes Java…
– Use some form of pseudocode
Suggestions?
(other than ‘give up’)
David.greathead@ncl.ac.uk