FLOW-Workshop, Hannover, 13./14.09.2007
Information flow in the microprocess
Lutz Prechelt
Freie Universität Berlin, Institut für Informatik
http://www.inf.fu-berlin.de/inst/ag-se/
• Microprocess?
• The five orders of ignorance
• Study 1: Explicit documentation
•
•
Lutz Prechelt, prechelt@inf.fu-berlin.de
of design pattern use
Study 2: Design documentation
in tour format
Study 3: Comparing
communication across 9 teams
1 / 32
Microprocess: What and why?
• It is known that what developers and teams really do
("actual process") is much more complex than what is
described by process models
• The microprocess is a close-up, fine-grained view
of the development process
• The basis for capturing and understanding the actual process
• Microprocess activities take on the order of 1 second to a few
minutes
• Qualitative analysis of the microprocess also helps understand
individual technologies and techniques
Lutz Prechelt, prechelt@inf.fu-berlin.de
2 / 32
Results of information flow:
The five orders of ignorance
• Phillip G. Armour: "The Five
Orders of Ignorance",
CACM 43(10):17-20,
October 2000
• 0th Order of Ignorance (0OI):
Knowledge
• I know
• 1st Order of Ignorance (1OI):
Lack of Knowledge
• I don't know, but I know
that/what I don't know
• 2nd Order of Ignorance (2OI):
Lack of Awareness
• I don't know and
I am not aware of this fact
• But I do have a process for
locating my knowledge gaps
Lutz Prechelt, prechelt@inf.fu-berlin.de
• 3rd Order of Ignorance (3OI):
Lack of process
• I don't know and
I am not aware of this fact
• And I do not have a process for
locating my knowledge gaps:
• I cannot determine the nature
of my ignorance
• 4th Order of Ignorance (4OI):
Meta-Ignorance
• I am not aware that 2OI or 3OI
exist
• 1OI vs. 2OI is a crucial
difference w.r.t. information
flow
3 / 32
What next?
• I will present three previous studies of my group that
contribute bits about information flow
• Two of them concern documentation formats
• One looks at actual team-level communication frequencies
• No coherent whole
• Interpretation is left as an exercise to the workshop
Lutz Prechelt, prechelt@inf.fu-berlin.de
4 / 32
1) Explicit documentation of
design pattern use in source code
A controlled experiment (actually two, with some differences):
• Two groups of student subjects each perform a number of
understanding and modification tasks on two small programs
• 6-11 classes, 362-565 LOC (including comments)
• Both programs use design patterns
• Program "And/Or tree": Composite pattern, Visitor pattern
• Program "Phonebook": Observer method, Template Method pattern
• One group receives a well-commented version of each program
• 133-197 LOC are comments only
• The other receives the same programs but with redundant
pattern comment lines (PCL) added
• An additional 10-22 lines of comments
Lutz Prechelt, prechelt@inf.fu-berlin.de
5 / 32
Pattern Comment Lines (PCL)
Lutz Prechelt, prechelt@inf.fu-berlin.de
6 / 32
Results (And/Or tree)
• Dependent variables:
• correctness of solutions according to review (measured in points)
• time required (in minutes)
Lutz Prechelt, prechelt@inf.fu-berlin.de
7 / 32
Interpretation
• Each of the four results
(2 programs, experiments UKA/Java and WUSTL/C++)
supports one of the following two hypotheses:
• H1: Pattern-relevant maintenance tasks will be completed faster
if PCL are added
• 0 to 40 percent speedup found
• H2: Fewer errors will be committed in pattern-relevant
maintenance tasks if PCL are added
• Twice as many completely correct solutions for UKA And/Or tree
• (None of the results supports both hypotheses at once)
Lutz Prechelt, prechelt@inf.fu-berlin.de
8 / 32
Results and Interpretation
• PCL allowed
more of the
slower
participants to
arrive at a
correct
solution
Lutz Prechelt, prechelt@inf.fu-berlin.de
9 / 32
What is the cause of these results:
PCL format or just "more comments"?
• a
Lutz Prechelt, prechelt@inf.fu-berlin.de
10 / 32
What is the cause of these results:
PCL format or just "more comments"?
Lutz Prechelt, prechelt@inf.fu-berlin.de
11 / 32
Conclusion
• An attempted information flow may happen more quickly or
more reliably when the information takes a suitable form
• For instance one that uses known concepts
(such as specific design patterns)
• Explicit design pattern documentation avoids 2OI
• i.e. avoids a lack of the awareness that understanding the
presence of design pattern instances may be helpful
Lutz Prechelt, prechelt@inf.fu-berlin.de
12 / 32
Literature
• Lutz Prechelt, Barbara Unger, Michael Philippsen,
Walter F. Tichy:
Two Controlled Experiments Assessing the Usefulness of
Design Pattern Documentation in Program Maintenance.
IEEE Transactions on Software Engineering, 28(6):595-606,
June 2002.
Lutz Prechelt, prechelt@inf.fu-berlin.de
13 / 32
2) Design documentation
in tour format
A controlled experiment:
• Two groups of student subjects perform a rather difficult
understanding and change-planning task
on a substantial program
• JHotDraw: 448 classes, 27 KLOC (plus 34 KLOC of comments)
• Both groups receive the same program and equivalent design
documentation
• But the design documentation comes in different formats
• Group P: Relevant design documentation as a plain text file
• Group J: Equivalent documentation embedded piecewise in the
source code in in the form of tours
• With tool support (JTourBus) for navigating along the tour stops
• Idea: Easy access to further detail from the source code
Lutz Prechelt, prechelt@inf.fu-berlin.de
14 / 32
Tour stop source code format
and navigation tool (JTourBus)
• JavaDoc-style annotations
• interpreted by an Eclipse plugin
Lutz Prechelt, prechelt@inf.fu-berlin.de
15 / 32
Result
• Wrong answers and givers-up were much quicker with
JTourBus (J) than with plaintext (P)
Lutz Prechelt, prechelt@inf.fu-berlin.de
16 / 32
Conclusion
• Providing details (complexity) closely along with design
documentation makes overstrained developers recognize their
problems faster
• Such details-embedding converts large-scale 2OI (lack of
awareness) into large-scale 1OI (lack of knowledge)
• i.e. makes developers recognize how much they don't know
Lutz Prechelt, prechelt@inf.fu-berlin.de
17 / 32
Literature
• Christopher Oezbek, Lutz Prechelt:
"JTourBus: Simplifying Program Understanding by
Documentation that Provides Tours through the Source Code",
23rd IEEE Int'l. Conf. on Software Maintenance,
October 2007, to appear
Lutz Prechelt, prechelt@inf.fu-berlin.de
18 / 32
3) Comparing communication
across 9 teams
• 9 teams of 3 each build the same system in 30 hours
• simple web-based community portal
• 20-page specification, 150 req's
• functionality prescribed precisely
• complete freedom for GUI
Lutz Prechelt, prechelt@inf.fu-berlin.de
19 / 32
What: "People by Temperament"
Registration for community portal
Lutz Prechelt, prechelt@inf.fu-berlin.de
20 / 32
What (2):
Trivial Temperament Test (TTT)
• After registration, members can take the TTT personality test
• to determine their MBTI personality type
Lutz Prechelt, prechelt@inf.fu-berlin.de
21 / 32
What (3):
Search for members
• Search for members by complex criteria
Lutz Prechelt, prechelt@inf.fu-berlin.de
22 / 32
What (4):
Member list (e.g. for search results)
Lutz Prechelt, prechelt@inf.fu-berlin.de
23 / 32
What (5):
Member status page
Lutz Prechelt, prechelt@inf.fu-berlin.de
24 / 32
What (6):
Further requirements
• The above screenshots showed a solution for the
108 Web GUI requirements
in addition, there were
• 19 requirements regarding a SOAP webservice interface
• 19 non-functional requirements
• browser compatibility, performance, etc.
• 5 rules describing the form of solution delivery
• Each requirement was marked with priority
MUST, SHOULD, or MAY
Lutz Prechelt, prechelt@inf.fu-berlin.de
25 / 32
Purpose
• Teams used different web development platforms
• Java, Perl, PHP
(3 teams each)
• Plat_Forms is looking for consistent platform differences
• in either process or product
• Product characteristics we attempted to analyze were
Completeness, ease-of-use, robustness/error-handling, security,
correctness, scalability, size, structure, maintainability
• For process analysis we took notes and photographs:
• One activity descriptor for each team member every 15 minutes
• Descriptors: thinking, reads specification, uses IDE, email,
browser prototype, browser other, computer other, talk to team,
talk to other, pause for food, pause other, pause&absent
• Collaborative activities were marked with '2' or '3'
Lutz Prechelt, prechelt@inf.fu-berlin.de
26 / 32
Example process behavior
Talk team 2
Talk team 2
Read Specification
9:29
18:29
Develop 2
Develop
Develop 2
Lutz Prechelt, prechelt@inf.fu-berlin.de
27 / 32
Observed communication behavior
• T: Talk
• d: Develop
• b: Browser use
9:00
Lutz Prechelt, prechelt@inf.fu-berlin.de
(2 or more people together)
(2 or 3 people together)
(2 or 3 people together)
1:00
9:00
15:00
28 / 32
Observations on the data
• Some teams hardly co-develop (3, 5), others do it a lot (4, 8)
• Some teams communicate much less (3, 7) than others (8)
• Some teams have phases of intense communication (4, 6, 8),
but these lie at different times
9:00
Lutz Prechelt, prechelt@inf.fu-berlin.de
Inter
preta
1:00
tion?
9:00
29 / 15:00
32
Conclusion?
Maybe:
• Oberving the frequency of information flows without
observing their content is hardly helpful for understanding
Lutz Prechelt, prechelt@inf.fu-berlin.de
30 / 32
Literature
• Lutz Prechelt:
"Plat_Forms 2007: The Web Development Platform
Comparison – Evaluation and Results",
Freie Universität Berlin, Institut für Informatik,
Technical Report B-07-10
http://www.plat-forms.org
Lutz Prechelt, prechelt@inf.fu-berlin.de
31 / 32
Thank you!
Lutz Prechelt, prechelt@inf.fu-berlin.de
32 / 32
Results 1:
Completeness of solutions
GUI requirements
Note:
- Team Java 4 was hampered by a
huge VMware setup problem for
almost a full day
- Team Java 9 used a framework
still in alpha development (RAP)
Lutz Prechelt, prechelt@inf.fu-berlin.de
33 / 32
Results 2:
Size of solution
source lines-of-code
Note:
Further manually written
source code resides in
modified reused files.
Lutz Prechelt, prechelt@inf.fu-berlin.de
34 / 32
Results 3:
Robustness and Security
1 test per
column
red:
security risk
reddish:
broken
yellow:
acceptable
green:
OK
Lutz Prechelt, prechelt@inf.fu-berlin.de
35 / 32
Results: Other
• Many other aspects were compared
•
•
•
•
•
Ease-of-use
Correctness/reliability
Modifiability, solution structure
Team behavior during the development process
Teams' self-reported subjective experience
• Some of them exhibit further platform differences
• in particular often smaller variance among the PHP teams
Lutz Prechelt, prechelt@inf.fu-berlin.de
36 / 32
Who: Teams
• Team 3 Java:
abaXX Technology (abaxx.de)
• Team 4 Java:
Accenture Technology
Solutions (accenture.de)
• Team 9 Java:
Innoopract Informationssysteme (innoopract.de)
• Team 1 Perl:
Etat de Genève/Optaros
(ge.ch, optaros.com).
• Team 2 Perl:
plusW (plusw.de)
• Team 5 Perl:
Revolution Systems
(revsys.com)
• Team 6 PHP:
OXID eSales
(oxid-esales.com)
• Team 7 PHP:
Globalpark (globalpark.de)
• Team 8 PHP:
Zend Technologies
(zend.com)
Lutz Prechelt, prechelt@inf.fu-berlin.de
37 / 32