Bell Laboratories
James O. Coplien
Bell Laboratories
Naperville, Illinois, USA cope@research.bell-labs.com
JaCC Software Developers
16 September 1999
11:00 - 12:30
1

Bell Laboratories
James O. Coplien
Bell Laboratories
Naperville, Illinois, USA cope@research.bell-labs.com
IIT Colloquium Series
Tuesday, February 8, 2000
M-50 Auditorium
2

 What are organizational patterns?
 Methodology
 Our model of organizational change
 Using patterns for change management
 The patterns themselves can be found at http://www.bell-labs.com/cgi-user/OrgPatterns/OrgPatterns
Bell Laboratories 3







A solution to a problem in a context
Architectural patterns ideas first published by
Alexander in 1977
Look at issues of structure, not just parts
Build on proven practice, not “promising theories”
Adopted by the software community starting in 1990; started gaining critical mass by 1993
Today, widely used as a design tool, especially in the
OO community
Bell Laboratories 4




Solutions to organizational problems in a context
First appeared in the Alexander + software context at
PLoP in 1994 (Coplien, Whitenack); received with some skepticism
Now, a growing body of knowledge


Or, a construct from anthropology, Kroeber:
» Universal patterns: transcend cultures
» Systemic patterns: have a common root in an ancient culture
» Total culture patterns: give a culture its identity
Patterns define culture
Bell Laboratories 5






Because process-based approaches have serious limitations that hark back to the era of the industrial revolution
Because experience with project management, technology transfer, etc. show software to be a primarily social activity
Because of the tie to cultures, and the study of cultures is about social activity
Because we should care about human comfort in addition to product quality
Because they work!
Bell Laboratories 6

 Goal: To know how to design highly productive software development organizations
 Premise: Organization drives process, which drives productivity
 Strategy: Correlate properties of organizations with order-of-magnitude productivity
 Technique: Patterns
Bell Laboratories 7






The idea of an organization as an anthropomorphic entity
» Some are intelligent: good reactions, learn from mistakes
» Some are stupid: react in insane ways, don’t learn
Organizations can learn!
Organizations can’t be taught
Therefore, learning is experiential
» Experience is a hard teacher
Can be helped with empirical foundations and introspection
Bell Laboratories 8


“Pattern Mining”
 Finding recurring structures and practices in healthy organization
 To a degree, seeking the absence of these structures in dysfunctional organizations
 A soft science
 Akin to anthropology and its many branches
Bell Laboratories 9

 Based on social network analysis with informal extensions
 Organizational role-play to gather social network data
 Dyadic and triadic sociometric data
 Uses CRC cards
 Light facilitation
 Half-day exercise
Bell Laboratories 10

Subsystem coord.
Validate MR lists
Build group products
Administer ENVY
Resolve logical deps.
Subsystem coord.
Change committee
Designers
System test
Bell Laboratories 11

 Identify project roles
 Study subjects play roles
 Development scenarios drive role-play
 Capture interaction & coupling on CRC cards
 Social Network Analysis Tools
» Organization Structure Visualization
» Organizational Metrics
 Capture Trends as Generative Patterns
Bell Laboratories 12

 Sociometric analyses
» Pasteur analysis tool
» Analytical studies of data
 Catalog sociograms, sociomatrices, sociometric data
 The pattern must recur
 The pattern must solve a problem
 Document in pattern form
Bell Laboratories 13







Fourth generation product in one generation
Remarkable press reviews
The most effective organization we have found
» Productivity
» Quality
» Interval
» Market
Used to calibrate our “spectrum”
Analysis done at my request
I am not a QPW user
Bell Laboratories 14

u e n c y
F r e q
6
5
4
3
2
1
0
10
9
8
7
2
QPW
3 4 5 6 7 8
Average Collaborators Per Role
9
Bell Laboratories 17





A forum to review patterns
Borrowed from the creative literature community
Designed to protect the dignity of the author
Several basic steps:
» Moderator introduces author
»
Author reads the work
» Author becomes a fly on the wall
»
Someone summarizes the pattern
» The strengths are emphasized
»
Suggestions for improvement
» Author asks for clarification
» Thank the author
Bell Laboratories 18

 Let dysfunctional organizations become exposed to the patterns to see what is possible
 Use organizational role-play as an introspection tool
 Collect and process sociometric data from the role play
 Present the data to the organization for a second round of introspection —a “mirror”
 Learning and growth build on the insights
Bell Laboratories 19

 Same technique as used for pattern mining
 Largely self-running
 Role-based
» Useful in most development organizations
 Actor-based
» Useful for diagnosing specialized team problems
» “Ethnological” studies
Bell Laboratories 20




Involve everybody
Don’t try solving other peoples’ problems
» Other organizations must solve their own problems
» Interface issues can be dealt with in another forum
» Interfaces generally involve politics
Don’t look for “the problem”
» Problems are systemic
» Solutions are generative
» Let the patterns cause the “aha”s
Bell Laboratories 21




Happens bottom-up, perhaps with the top-down support of a sponsor or patron
Patterns are closely tied to value systems —which must be elicited bottom-up
Organizational change means building a new culture: reverse anthropology
» New language
» New normative behaviors
» New values
» New symbols
» New stories (patterns)
» New rituals
Bell Laboratories 22

 No silver bullet
 Well-known productivity enablers
 A little luck
 The right balance of techniques to make a working social (and economic and technical and...) system
Bell Laboratories 23


The “Buffalo Mountain” pattern
 Decentralized, yet a graceful distribution
 Almost fully connected
 Anti-schmisogenetic
» No splinter groups
» No time-serial sub-processes
» No “subroutines”
Bell Laboratories 24

Bell Laboratories
100
90
80
70
60
50
40
30
20
10
0
0
QPW
60 20 40
Number of Roles
25

 Easy to socialize the architecture
 Self-directing
» Correlates to Gerry Weinberg’s SEI/CMM parallel
» The true hallmark of a mature organization
 Divide and conquer
 Essentially large organizations need hierarchy
Bell Laboratories 26

Problem: Preserving the architectural vision through to implementation
Context: An organization of developers that needs strategic technical direction.
Forces: Totalitarian control is viewed by most development teams as a draconian measure. The right information must flow through the right roles. The architect is one of the few people, or may be the only person, with the big picture. Big picture knowledge is important even during implementation, especially when using iterative development. But the architect usually isn’t around during implementation.
Solution: Beyond advising and communicating with developers, architects should also participate in implementation.
Resulting Context: A development organization that perceives buy-in from the guiding architects, and that can directly avail itself of architectural expertise. The architect learns from implementation constraints and may modify the architecture accordingly.
Rationale: Vitruvius notes: “...[A]rchitects who have aimed at acquiring manual skill without scholarship have never been able to reach a position of authority to correspond to their pains, while those who relied only upon theories and scholarship were obviously hunting the shadow, not the substance. But those who have a thorough knowledge of both, like men armed at all points, have the sooner attained their object and carried authority with them.”
Larry Constantine emphasizes the value of “street creds” in an architect.
Bell Laboratories 27

 Like QPW adjacency graph
 Sociometric analysis: low graph centrality
 Can be remedied by load leveling, staffing, splitting roles, combining roles
Bell Laboratories 30

Bell Laboratories
QPW
45
40
35
30
25
20
15
10
5
0
0 2 4 6
Communication Intensity Ratio
8
31

 Personal excellence and integrity
 Highly specialized
 Balance of teamwork (communication) and individuation (specialization, personal signoffs)
 How does one assemble such a group?
1. Magic: A secret that only Philippe knows
2. Outside interests? (Gabriel)
3. Self-selecting Team (another pattern)
Bell Laboratories 32

 QA is tightly coupled in the social network
 Large beta program –customer engagement
 Developers reserve early verification to themselves
 QA first-in, last-out
Bell Laboratories 33





A sense of contribution
» Reinforced in project signoffs
» Reinforced in reward system
Financial incentives
» No first-hand insights into QPW
» Controversial, but claims demonstrable results
Two cultures with extravagant rewards:
1. The U. S. West Coast
2. Financial trading
Overcomes fear of change
Bell Laboratories 34

 The ability to introspect well
 The ability to self-manage
 Organizational structure over task sequencing
 Effective meetings, and lots of them
Bell Laboratories 35

 Organizational patterns are:
» Patterns that capture the “architecture” of organizations
» In anthropology, the fabric of social structure
 Role-playing data gathering and introspection
 Organizational change is generative
 Organizations can use patterns as exemplars for improvement
 A rich body of literature exists and is growing
Bell Laboratories 36