Mythical Man Month Notes David Darus 1/7/2002 Chapter 1 “The Tar Pit” Pg. 4 In software development, no one thing seems to cause the difficulty, but the accumulation of simultaneous and interacting factors brings slower and slower motion. Pg. 5 Developing a professional system is 9 times the cost and time of an amateur development of the same system. Pg. 7-9 Joys of programming: Sheer joy of making something. Making something useful. Fascination of putting together a puzzle. Learning. Working is such a creative and abstract medium. Woes of programming: One must perform perfectly. Other’s set a programmers objectives, resources, information. Disjoint is responsibility and authority. Dependence on other’s work which is often mal-designed, poorly implemented, incompletely delivered and poorly documented. Debugging is drudge work. Testing drags on and on, finding the last bug is harder than the first. Product seems to be obsolete upon completion. However, something that is available is better than something planned. 1/7/2002 Chapter 2 “The Mythical Man-Month” Pg. 14 Most project go awry for lack of calendar time. Estimating time is difficult and tools are poorly developed and reflect the assumption that all will go well. There is always time to do something over and over again but not enough time to do it right. Effort and progress are confused as if men and months are interchangeable. When schedules slip the natural and worse response is to add manpower. Pg. 15 Programmers are optimists. They design systems in “thought-space” a very flexible medium. However implementation reveals that our ideas are faulty but we blame the physical medium because of pride; we have bugs; hence our optimism is unjustified. Pg. 16 Men and month are only interchangeable when tasks can be partitioned among workers and no communication among them is needed; woefully untrue of system programming. Pg. 18 Communication made up of two parts: training and intercommunications. Training (technology, goals, overall strategy and work plan) can not be partitioned; more effort does no good. Intercommunications is worse, effort increases as n(n-1)/2. The more workers the more sets of pairwise interactions. Pg. 19 Testing and debugging usually the most underestimated because of optimism. This usually occurs at the end of the schedule; it is bad news, late and without warning. This also occurs when project is fully staffed thus the daily expense is at its maximum. Pg. 21 Desired completion time cannot govern actual completion time. This must be understood and accepted, trying to violate this will result in nothing. Can’t cook an omelette is the desired time; if tried you get it raw. Pg. 23-26 Don’t throw manpower at the problem of schedule slippage. Increase the schedule time. As a schedule slips so does quality and features. The number of month is dependent on the sequential constraints, the maximum number of men depends upon the number of independent subtasks. 1/8/2002 Chapter 3 “The Surgical Team” The problem: small teams are best but sometime there is so much work that even a small team will take to long to accomplish the project. What to do? Mill’s Proposal: Several small teams each working on a separate segment of the project. The teams are organized as a surgical unit – one surgeon with a support staff. The surgeon is the chief programmer – designs, codes, documents, tests. Needs great talent and lots of experience. The co-pilot is the backup chief programmer – works with the chief to help in thinking, etc... ready to take charge but not responsible for anything. The administrator handles all the personnel matters based on the decision of the chief programmer. The editor polishes the documentation that must be written by the chief programmer so that accuracy is achieved. The system support person The tester The coders – small, very specific implementation programming tasks Note this team structure is NOT a partnership; the chief programmer is it all with helpers who obey. This means the project is of one mind and intercommunications and design decision and compromises are minimized. On large projects the chief programmers of the surgical teams must communicate. An overall systems architect must arbitrate between all the teams. 1/9/2002 Chapter 4 “Aristocracy, Democracy, and System Design” Pg. 42 Software conceptual disunity is usually due to the separation of design into many tasks done by many people; not be a serial succession of master designers. Conceptual integrity is the most important consideration in system design. It is better to have fewer anomalous features and improvements, but one set of design ideas than to have many good but independent and uncoordinated ideas. Pg. 43 This raises some issues which are addressed next. Achieving conceptual integrity: Functionality is at war with simplicity, a delicate balance must be struck. Conceptual integrity dictates that design must proceed from one mind, or from a very small number of agreeing resonant minds. Yet, many hands are needed to do all the work. This dilemma can be resolved with two techniques. 1. A careful division of labor between architecture and implementation. 2. The surgical team. A large system is going to be more successful being mostly a aristocracy of architects than a democracy of architects and implementers. Implementers can contribute ideas but that must be consistent with the conceptual integrity. Architects should be mindful of known techniques of implementation that can be used to implement their designs. Discipline and limits can be very helpful in the creative process; it provides needed form. This chapter discusses in detail the arguments of architects vs. implementers and common objections. I think we will need a small architecture team that will design a new billing system. Some of the people on this team will continue as architects once implementation starts in earnest but the other will become implementers. 1/9/2002 Chapter 5 “The Second-System Effect” To deal with the problem of the architect doing to much “blue-sky” design the architects and builders must be in thoroughgoing, careful, and sympathetic communications. Deal with people’s pride and emotion carefully when crossing boundaries. Be careful not to under design the first system because this often leads to over designing the second system. Second system design often falls prey to refining functionality that is contrary to the basic assumptions of the system. In other words, much work is done to make something really good but because the basic way of doing things has changed the effort is wasted. For example, fine tuning a batch debugging system when interactive computing was breaking onto the scene. Solutions: assign costs and benefits to each function so that improvements can be properly evaluated. Have self-discipline in system design, be aware of the special temptations. 1/14/2002 Chapter 6 “Passing the Word” How can a group of 10 architects maintain the conceptual integrity of a system which 1000 people are building? The written specification is a necessary but not a sufficient tool. The specification should describe the users needs but not implementation details. However, the architect should always be prepared to show an implementation for any feature described but not dictate the implementation. The specification must be precise, full and accurately detailed even at the cost of being boring. A formal definition and a prose definition may be used to describe a specification, but one must be the standard and the other the derivative. Formal meetings are held with the architects and representatives of the implementers. Additions and changes to the project are proposed by anyone, usually in writing beforehand. Discussion ensues with a bent towards getting multiple solutions and the deciding on the best. The chief architect arbitrates are deadlocks by making the decision. This information is disseminated to all. To deal with issues that the formal meetings don’t address satisfactorily, problematic decisions, disgruntlements a type of supreme court session is held. Everyone is heard and everyone better understands the intricate constraints and interrelationships among decisions. Multiple implementations being built at the same time serves as a mechanism to avoid the easy way out of just changing the specification when the specification and an implementation doesn’t agree. A diary of events, questions and answers resolving specification and implementation issues should be kept and disseminated so that minor issues are clarified. Implementers shouldn’t remain puzzled about details and just guess at an answer an proceed, they should ask the architects and the discussion should be reflected in the diary. The test group should be an independent, devil’s advocate group that keeps the developers true to the specification. This group must be involved early in the design. Care must be taken to mitigate this group and the developers because of the adversarial relationship. 2/4/2002 Chapter 7 “Why Did the Tower of Babel Fail?” For a project not to fail there needs to be lots of communication. Problems will arise due to subtle changes in assumptions among the different teams about the project specification. Regular, good communication needs to occur to ferret out these changes and resolve issues. Regular technical briefing meetings are invaluable – many minor misunderstanding get revealed and resolved by this means. Another major form is the formal project workbook. See the article for details about the what, why and mechanics of the workbook. The job done least well by project managers is to utilize the technical genius who is not strong on management talent. See chapter 18 item 7.14 & 7.15. 2/11/2002 Chapter 8 “Calling the Shot” Small project time estimates do not linearly scale to provide estimates for larger projects; it is more of an exponential scale. Some studies yielded a function of: effort = (constant) * (# of instructions) 1.5 Another study revealed that estimates were off by one-half (projects taking twice as long as estimated) because only 50% of time thought to be for programming was actually for programming. The other 50% was accounted for by machine downtime, higher-priority short unrelated jobs, meetings, paperwork, company business, sickness, personal time. 2/11/2002 Chapter 9 “Ten Pounds in a Five-Pound Sack” Consider overall impact of particular strategies. What may be the “best” thing to do in one place may cause a larger problem overall; thus suboptimization must take a back seat to overall system optimization. A total-system, user-oriented software development attitude should be fostered. Remember data representation is the essence of programming. “Show me your flowcharts and conceal your data, and I shall continue to be mystified. Show me your data, and I won’t usually need your flowchart; they’ll be obvious.” 2/19/2002 Chapter 10 The Documentary Hypothesis There is a set of documents that are useful in many diverse settings (hardware product, software, university department, etc…). These documents describe the: what, when, how much, where and who of a project. They are titled in a generic sense as Objectives, Specifications, Schedule, Budget, Space allocation and Organization chart. Documentation is needed to make clear, exact policies because writing them down forces hundreds of mini-decisions to be made. Furthermore, documentation acts as a vehicle to communicate decisions to others. “This small set of critical documents encapsulates much of the manager’s work. If their comprehensive and critical nature is recognized in the beginning, the manger can approach them as friendly tools rather than annoying busywork. He will set his direction much more crisply and quickly by doing so.” 3/4/2002 Chapter 11 Plan to Throw One Away Build a pilot system and plan to throw it away. Then redesign and rebuilt and deliver that to the customer, never the throwaway system. You will always have a throwaway system; it’s not an option. Plan for and embrace change instead of fighting it; it is inevitable. The organization needs to be able to handle change. An organization that is critical of decision will cause decision-makers not to make decisions; an atmosphere of encouragement and creativity must be fostered so that people are willing to take risks. Just because you are a senior/valuable person doesn’t mean you should not write programs, you should. Program maintenance is an entropy-increasing process, and even its most skillful execution only delays the subsidence of the system into unfixable obsolescence. Therefore, plan to rebuild at a future date. 3/26/2002 Chapter 12 Sharp Tools Refers mostly to old technology, these things have been adopted, enhanced and surpassed by today’s development environments. 3/26/2002 Chapter 13 The Whole and the Parts Eliminate bugs by doing good specification writing, have non-programmers test the specification, use stepwise refinement with top-down design and structured programming techniques. When doing system tests, use debugged components, provide lots of testing scaffolding, control changes, add components one at a time and schedule updates. Updates should be either large and very far apart or very small and frequent. 3/26/2002 Chapter 14 Hatching a Catastrophe Schedules slip one day at a time. However, we must have schedules. Setting schedules is a matter of experience. Furthermore note that 90% of the code is done half the time, 99% of the debugging is done most of the time. Meaning that the last little bit of these tasks take the most time because the hardest part is usually left until the end. Have milestones that are unambiguously defined, not temptation is soften bad news by deceiving. There will be problems and delays, plan for them – be pessimistic and conservative. Get concerned about small slips in time, fix the problems. Some of the ideas in this chapter seem out-dated. Problems should be reported to the boss but the boss should be wise in when to take action and when to just be educated on the status. A boss who acts appropriately will get honest appraisals of problems and status. 3/26/2002 Chapter 15 The Other Face Instill the practice of good documentation by showing good documentation not just exhorting that it be done. There is documentation for users and documentation for programmers. Flow-charting is obsolete, it has been since high-level languages were introduced. 4/2/2002 Chapter 16 No Silver Bullet – Essence and Accident Now is the time to start concentrating on improving the effort spent of the essence, not the accidents of software engineering. The essence is the fashioning of the complex conceptual structures that compose the abstract software entity. The accidents are representing the abstract entities in a programming language and the mapping to machine language within speed and space constraints. Much work has been done to improve productivity concerning the accidents of software engineering. To improve productivity about the essential parts of the software task do: Buy instead of build. Use rapid prototyping as part of a planned iteration in establishing software requirements. Incrementally and iteratively develop software; grow it. Develop great designers. No silver bullets for software. Hardware advances are the anomaly in technology in the history of civilization. The essence of a software entity is a construct of interlocking concepts: data, relationship among data items, algorithms and invocation of functions. The errors are in correctly specifying systems that solve a problem domain. This will always be hard so therefore software will always be hard. The inherent properties of the irreducible essence of modern software are: complexity, conformity, changeability, and invisibility. Complexity Software doesn’t usually have a repetition of the same element in a larger size, but an increase in the number of different elements. This causes complexity to increase for than linearly. Complexity causes communication problems and makes understanding the whole system impossible. This leads to all the typical software problems of over-schedule and over-budget, etc… Conformity Software must conform to all the different systems it must interface with. This is complex because different people did the other systems; there is no central designer. This adds to complexity because making software conform often seems arbitrary and no amount of redesign will help the situation. Changeability Software is more easily changed and gets changed more often in the field. One of the problems is not everyone understands the cost of the change as they understand the cost of changing a building already built. Invisibility Software has no geometrical shape and is thus unvisualizable. The graphs used to describe software are not hierarchical, nor even planar; the software is too complex. This impedes the process of design in a developers mind and much more in communicating ideas to others. Accidental difficulties solved with high-level programming languages, time-sharing systems with faster turnaround time, IDE’s. Ada, OO, AI, Expert Systems, Automatic programming, Graphical programming, Program verification all contribute a bit to the accidental problems of software construction but are no silver bullet for attacking the essence of software construction. Promising attacks on the conceptual essence are build vs. buy (buy it if you can), requirements refinement and rapid prototyping (do prototyping to help determine system requirements), and incremental development (grow not build software). Foremost, great designer should be sought, compensated and retained. 4/4/2002 Chapter 17 No Silver Bullet Refired Complexity is the most serious difficulty, but oftentimes complexity is software arises from trying to model the complex realities of the malfunctions of an organization. The malfunctioning business processes of an organization should be fixed before modeling it in software. The details must be specified exactly and not statistically. Complexity also stems from software implementation, data structures, algorithms, and connectivity. Thus, software should be built upon other work; grown. Focus on quality of specifications, design and implementation and productivity will follow, but don’t do quality in the extreme or productivity will suffer greatly. Apply quality to avoid major disasters. 4/4/2002 Chapter 18 Propositions of the Mythical Man-Month: True or False? A summary of the chapters, stating the true assertions with updated comments. 4/9/2002 Chapter 19 The Mythical Man-Month after 20 Years Conceptual integrity is central to product quality; must have one system architect or if a large system, one master architect and one architect per subsystem. The subsystem boundaries must be at those places where interfaces between subsystems are minimal and easiest to define rigorously. Separate the task of the architect from that of implementation, plenty of work on both sides. The “throw the first one away” proverb is wrong. It is wrong because it implicitly assumes the waterfall method. The iterative and incremental model should be used. However, it should be known that the first iteration will not be fielded, so in this narrow sense the “throw the first one away” does still apply. Projects should be organized with the architect as the top decision maker. Project management issues that are essentially administrative should be delegated to the architects “right-hand man”. A clear separation of the architect, implementers and problem domain experts should be made. Person of the right education, knowledge and skills should be placed in each of these positions. A person in a position gives answers to questions that are properly within that domain of responsibility. They give suggestions to other domains. The lead person for a software development project should be well versed in software engineering and constructions, not just someone who has managed other projects (typically not software projects).