Chapter 1.A: Introduction BER2043 Software Engineering Student Learning Outcome • Describe the flow of Software engineering • Define Software Engineering Principles • Discuss the Software cycle time. • Learn about types of project management struture • Learn about basics of computer based system engineering Introduction: Software is Complex • Complex complicated • Complex = composed of many simple parts related to one another • Complicated = not well understood, or explained Complexity Example: Scheduling Fence Construction Tasks Setting posts [ 3 time units ] Cutting wood [ 2 time units ] Nailing [ 2 time units for unpainted; 3 time units otherwise ] Painting [ 5 time units for uncut wood; 4 time units otherwise ] Setting posts Nailing, Painting Cutting Nailing …shortest possible completion time = ? [ “simple” problem, but hard to solve without a pen and paper ] 4 The Frog in Boiling Water • Small problems tolerate complacency—lack of immediate penalty leads to inaction • Negative feedback accumulates subtly and by the time it becomes painful, the problem is too big to address • Frog in gradually heated water analogy: The problem with little things is that none of them is big enough to scare you into action, but they keep creeping up and by the time you get alarmed the problem is too difficult to handle Consequently, “design smells” accumulate, “technical debt” grows, and the result is “software rot” https://en.wikipedia.org/wiki/Design_smell https://en.wikipedia.org/wiki/Technical_debt https://en.wikipedia.org/wiki/Software_rot 5 The Role of Software Engg. (1) A bridge from customer needs to programming implementation Customer Programmer First law of software engineering Software engineer is willing to learn the problem domain (problem cannot be solved without understanding it first) 6 The Role of Software Engg. (2) Customer: Requires a computer system to achieve some business goals by user interaction or interaction w ith the problem domain in a specified manner System-to-be (includes hardware) User Software-to-be Problem Domain Software Engineer’s task: To understand how the system-to-be needs to interact w ith the user or the problem domain so that customer’s requirement is met and design the software-to-be May be the same person Programmer’s task: To implement the software-to-be designed by the software engineer 7 Example: ATM Machine Understanding the money-machine problem: 7 4 1 0 2 5 3 8 6 9 ATM machine Communication link Bank’s remote datacenter Bank customer 8 Problem-solving Strategy Divide-and-conquer: • Identify logical parts of the system that each solves a part of the problem • Easiest done with the help of a domain expert who already knows the steps in the process (“how it is currently done”) • Result: A Model of the Problem Domain (or “domain model”) 9 How ATM Machine Might Work Domain Model How may I help you? Transaction record Cash Bookkeeper Speakerphone Safe keeper Safe Phone Window clerk Datacenter liaison Dispenser Customer Bank’s remote datacenter 10 : How ATM Machine Works Cartoon Strip A Enter your PIN C B Verify account XYZ D Verify this account Typing in PIN number … E How may I help you? Withdraw $60 Account valid. Balance: $100 XYZ valid. Balance: $100 F G Release $60 Dispense $60 Record $60 less H Dispensing! Please take your cash 11 Software Engineering Blueprints Specifying software problems and solutions is like cartoon strip writing Unfortunately, most of us are not artists, so we will use something less exciting: UML symbols However … 12 Second Law of Software Engineering • Software should be written for people first ( Computers run software, but hardware quickly becomes outdated ) Useful + good software lives long To nurture software, people must be able to understand it 13 Software Development Methods Method = work strategy The Feynman Problem-Solving Algorithm: (i) Write down the problem (ii) think very hard, and (iii) write down the answer. Waterfall Unidirectional, finish this step before moving to the next Iterative + Incremental Develop increment of functionality, repeat in a feedback loop Agile Continuous user feedback essential; feedback loops on several levels of granularity 14 Waterfall Method Requirements Design Implementation Waterfall method Testing Deployment & Maintenance Each activity confined to its “phase”. Unidirectional, no way back; finish this phase before moving to the next 15 UML – Language of Symbols UML = Unified Modeling Language «interface» BaseInterface ClassName # attribute_1 : int # attribute_2 : boolean # attribute_3 : String Three common compartments: Actor 1. Classifier name 2. Attributes 3. Operations + operation() + operation_1() : void + operation_2() : String + operation_3(arg1 : int) Class1Implement Class2Implement + operation() + operation() Stereotype «» provides additional info/ annotation/ explanation Inheritance relationship: BaseInterface is implemented by two classes Software Class Comment Software Interface Implementation instance1 : Class1 instance5 : Class2 doSomething() instance8 : Class3 doSomethingElse() Interaction Diagram doSomethingYetElse() Online information: http://www.uml.org 16 Understanding the Problem Domain • System to be developed • Actors Agents external to the system that interact with it • Concepts/ Objects Agents working inside the system to make it function • Use Cases Scenarios for using the system 17 ATM: Gallery of Players 1 7 4 0 Bank customer Actors 8 5 2 9 6 3 System (ATM machine) Bank’s remote datacenter (Easy to identify because they are visible!) 18 Gallery of Workers + Tools Window clerk Datacenter liaison Bookkeeper Safe keeper Dispenser Speakerphone Telephone Transaction record Safe Cash Concepts (Hard to identify because they are invisible/imaginary!) 19 Use Case: Withdraw Cash A B Verify account XYZ Enter your PIN C How may I help you? 1 4 2 7 85 6 3 0 9 Typing in PIN number … 1 4 2 7 85 6 3 0 9 XYZ valid. Balance: $100 D Withdraw $60 E Please take your cash XYZ withdrew $60 1 4 2 7 85 6 3 0 9 Collecting cash … Acknowledged 20 How ATM Machine Might Works (1) Domain Model How may I help you? Transaction record Cash Bookkeeper Speakerphone Safe keeper Safe Phone Window clerk Datacenter liaison Dispenser Customer Bank’s remote datacenter 21 How ATM Machine Works (2) Domain Model (2) Solution modification Alternative solution How may I help you? Transaction record Bookkeeper Speakerphone Draftsman Window clerk Dispenser Customer How ATM Machine Works (3) Domain Model (3) Alternative solution Solution modification How may I help you? Transaction record Bookkeeper Speakerphone Courier Window clerk Dispenser Remote bank Customer Which solution is the best or even feasible? Software Engineering Principles Principles form the basis of methods, techniques, methodologies and tools Seven important principles that may be used in all phases of software development Apply to the software product as well as the development process 24 Key principles • Rigor and formality • Separation of concerns • Modularity • Abstraction • Anticipation of change • Generality • Incrementally 25 Rigor and formality • Software engineering is a creative design activity, BUT It must be practiced systematically • Rigor is the quality or state of exactness, cautious and strictness. It is a necessary complement to creativity that increases our confidence in our developments • Formality is rigor at the highest degree. Often verified with mathematical or logical rules. 26 Examples: Rigor and formality • Product: • – Formal-Mathematical analysis of program correctness • – Rigorous-Systematic test data derivation • • Process: • – Rigorous- detailed documentation of each development step in waterfall model • – Formal- automated transformational process to derive program from formal specifications 27 Separation of concerns • To dominate complexity, separate the issues to concentrate on one at a time • – "Divide & conquer" • Supports parallelization of efforts and separation of responsibilities 28 Examples: • Process: Go through phases one after the other as in waterfall Model • –Do separation of concerns by separating activities with respect to time • – Product: Keep different types of product requirements separate • –Functionality discussed separately from the performance constraints 29 Modularity • A complex system may be divided into simpler pieces called modules • • A system that is composed of modules is called modular • • Supports application of separation of concerns • – when dealing with a module we can ignore details of other modules 30 Modularity Modularity: Cohesion and coupling • Each module should be highly cohesive – module understandable as a meaningful unit – Components of a module are closely related to one another • Modules should exhibit low coupling – modules have low interactions with others – understandable separately 32 An Example 33 Abstraction • Identify the important aspects of a phenomenon and ignore its details – Special case of separation of concerns – The type of abstraction to apply depends on purpose • Example: the user interface of a watch (its buttons) abstracts from the watch's internals for the purpose of setting time; other abstractions needed to support repair 34 Anticipation of change • Ability to support software evolution requires anticipating • potential future changes • –It is the basis for software evolvability 35 Generality • While solving a problem, try to discover if it is an instance of a more general problem whose solution can be reused in other cases • Sometimes a general problem is easier to solve than a special case –Carefully balance generality against performance and cost 36 Incrementality • Process proceeds in a stepwise fashion (increments) • Examples (process) – deliver subsets of a system early to get early feedback from expected users, then add new features incrementally – deal first with functionality, then turn to performance 37 Software Development Life Cycle (SDLC) “You’ve got to be very careful if you don’t know where you’re going, because you might not get there.” Yogi Berra Waterfall Model • Requirements – defines needed information, function, behavior, performance and interfaces. • Design – data structures, software architecture, interface representations, algorithmic details. • Implementation – source code, database, user documentation. Waterfall Strengths • Easy to understand, easy to use • Provides structure to inexperienced staff • Milestones are well understood • Sets requirements stability • Good for management control (plan, staff, track) • Works well when quality is more important than cost or schedule Waterfall Deficiencies • All requirements must be known upfront • Deliverables created for each phase are considered frozen – inhibits flexibility • Can give a false impression of progress • Does not reflect problem-solving nature of software development – iterations of phases • Integration is one big bang at the end • Little opportunity for customer to preview the system (until it may be too late) When to use the Waterfall Model • Requirements are very well known • Product definition is stable • Technology is understood • New version of an existing product • Porting an existing product to a new platform. V-Shaped SDLC Model • A variant of the Waterfall that emphasizes the verification and validation of the product. • Testing of the product is planned in parallel with a corresponding phase of development V-Shaped Steps • Project and Requirements Planning – • provide for enhancement and corrections allocate resources • Product Requirements and Specification Production, operation and maintenance – • System and acceptance testing – check the entire software system in its environment Analysis – complete specification of the software system • Architecture or High-Level Design – • Integration and Testing – check that modules interconnect correctly defines how software functions fulfill the design • Detailed Design – develop algorithms for each architectural component • Unit testing – check that each module acts as expected • Coding – transform algorithms into software V-Shaped Strengths • Emphasize planning for verification and validation of the product in early stages of product development • Each deliverable must be testable • Project management can track progress by milestones • Easy to use V-Shaped Weaknesses • Does not easily handle concurrent events • Does not handle iterations or phases • Does not easily handle dynamic changes in requirements • Does not contain risk analysis activities When to use the V-Shaped Model • Excellent choice for systems requiring high reliability – hospital patient control applications • All requirements are known up-front • When it can be modified to handle changing requirements beyond analysis phase • Solution and technology are known Structured Evolutionary Prototyping Model Structured Evolutionary Prototyping Model • Developers build a prototype during the requirements phase • Prototype is evaluated by end users • Users give corrective feedback • Developers further refine the prototype • When the user is satisfied, the prototype code is brought up to the standards needed for a final product. Structured Evolutionary Prototyping Steps • A preliminary project plan is developed • An partial high-level paper model is created • The model is source for a partial requirements specification • A prototype is built with basic and critical attributes • The designer builds the database user interface algorithmic functions • The designer demonstrates the prototype, the user evaluates for problems and suggests improvements. • This loop continues until the user is satisfied Structured Evolutionary Prototyping Strengths • Customers can “see” the system requirements as they are being gathered • Developers learn from customers • A more accurate end product • Unexpected requirements accommodated • Allows for flexible design and development • Steady, visible signs of progress produced • Interaction with the prototype stimulates awareness of additional needed functionality Structured Evolutionary Prototyping Weaknesses • Tendency to abandon structured program development for “code-and-fix” development • Bad reputation for “quick-and-dirty” methods • Overall maintainability may be overlooked • The customer may want the prototype delivered. • Process may continue forever (scope creep) When to use Structured Evolutionary Prototyping • Requirements are unstable or have to be clarified • As the requirements clarification stage of a waterfall model • Develop user interfaces • Short-lived demonstrations • New, original development • With the analysis and design portions of object-oriented development. Project Management Example of an Organizational Chart Project Roles • Management roles • Organization and execution of the project within constraints. Examples: project manager, team leader. • Development roles • Specification, design and construction of subsystems. Examples: Analyst, system architect, implementor. • Cross functional roles • Coordination of more than one team. Example: API Engineer, configuration manager, tester • Consultant roles • Support in areas where the project participants lack expertise. Examples: End user, client, application domain specialist ( problem domain), technical consultant (solution domain). • Promoter roles • Promote change through an organization. Power Promoter • •Also called executive champion • •Pushes the change through the existing organizational hierarchy. • not necessarily at the top of the organization, but must have protection from top level management, otherwise project opponents might be able to prevent the success of the project. • •Tasks: • constantly identify difficulties, resolve issues, and communicate with the project members, especially with the developers. • •Example at project level:Project Manager. • •Example at corporate level:Chief Executive Officer (CEO). Knowledge Promoter • •Also called the technologist, • •Promotes change arising in the application domain or the solution domain. Usually associated with the power promoter. • •Tasks:Acquire information iteratively, understand the benefits and limitations of new technologies, and argue its adoption with the other developers. • •Example at project level:System architect. • Reports to project manager • Does not have any direct subordinate in the reporting hierarchy • Has final say over all technical decisions in the system. • •Example at corporate level:Chief Technical Officer (CTO). Process Promoter • •The process promoter has intimate knowledge of the projects processes and procedures. • •The process promoter is in constant interaction with the power promoter to get consensus on the overall goals. • •Tasks:Bridge between the power and knowledge promoters, who often do not speak or understand the same language. • •Example at project level:Development lead. Responsible for the administrative aspects of a project, including planning, milestones definition, budgeting and communication infrastructure. • •Example at corporate level:Chief Information Officer (CIO) Process Management Structure Hierachical (Chief Programmer team) Egoless Programming Team (Weinberg) Project Based Project Organization Comparison of Management Structures • •Hierarchical structures • “Reports”, “Decides” and “Communicates-With” all mapped on the same association • Do not work well with iterative and incremental software development process • Manager is not necessarily always right • Projects with high degree of certainty, stability, uniformity and repetition. • Requires little communication • Role definitions are clear • •When? • The more people on the project, the more need for a formal structure • Customer might insist that the test team be independent from the design team • Project manager insists on a previously successful structure Comparison of Management Structures • •Project-based structures • “Reports”, “Decides” and “Communicates-With” are different associations • Cut down on bureaucracy reduces development time • Decisions are expected to be made at each level • Hard to manage • •Project with degree of uncertainty • Open communication needed among members • Roles are defined on project basis • •When? • Requirements change during development • New technology develops during project Task Assignment •One-to-One Ideal but often not worth to be called a project •Many-to-Few Each project member assumes several roles ("hats") Danger of overcommittment Need for load balancing •Many-to-"Too-Many" Some people don't have significant roles Bystanders Loosing touch with project Project Roles: Coach • •Listen to gripes from individual teams • •Review weekly team reports • •Attend weekly project meetings • •Schedule and prepare meetings with client • •Insist that guidelines are followed • •Assign presentations (in-class project meetings, client review, client acceptance test) • •Resolve conflicts if they cannot be resolved otherwise Project Roles: Group Leader • •Responsible for intra-group communication (Meeting • Run the weekly project meeting • Post agenda before meeting • Define and keep track of action items (who, what, when) • Measure progress (Enforce milestones) • Deliver work packages for the tasks to the project management • Present problems and status of team to project manager • •The group leader has to be rotated among members of the Management: Primary Facilitator) team. Group Leader: Creating an Agenda Project Roles: Liaison • • Responsible for inter-group communication • Make available public definitions of subsystem developed by the team to the architecture teams (ensure consistency, etc) • Coordinate tasks spanning more than one group with other teams • • Responsible for team negotiations • • Examples: API Engineer, Configuration manager Project Roles: Planner • •Plans and tracks the activities of an individual team and has the following responsibilities. • •Define project plan for team: • PERT chart, resource table and GANTT chart showing work packages • Enter project plan into project management tool • •Make project plan available to management • •Report team status to project manager. No explicit planner in PAID. Responsibilities assumed by coaches Project Chart Project Roles: Document Editor • •Collect, proofread and distribute team documentation • •Submit team documentation to architecture team • •Collect agendas • •Take minutes at meetings Web Master • •Maintain team home page • •Keep track of meeting history • •Keep track of design rationale • •Publish Meeting Information on Team Homepage • Must contain Agenda, minutes, action items and issues • Possibilities: • One HTML document per meeting, with anchors (maintained by one role) • Separate HTML documents for Agenda, Minutes, etc(maintained by several roles) Process Model Shows relationships among Functions, activities, tasks Milestones Baselines Reviews Work breakdown structure Project deliverables Sign-offs Visualization of process model + Project Management Aids MS Project (Microsoft) MAC Project (Claris) EasyTrak(Planning Control International) Creating Work Packages • •Work Breakdown Structure (WBS) • Break up project into activities (phases, steps) and tasks. • The work breakdown structure does not show the interdependence of the tasks • •The identification of the work breakdown structure is an instance of object identification and associating these objects • • Influences cost and schedule Project Management Tools •Visualization Aids for Project Presentation • Graphs (Schedule), Trees (WBS) • Tables (Resources) • •Task Timeline • Gantt Charts: Shows project activities and tasks in parallel. Enables the project manager to understand which tasks can be performed concurrently. • •Schedule Chart • Cornerstone in many project management tools • Graphically shows dependencies of tasks and milestones • PERT: Program Evaluation and Review Technique • A PERT chart assumes normal distribution of tasks durations • Useful for Critical Path Analysis • CPM: Critical Path Method (PERT Chart) Project: Building a House •Activity 1: Landscaping the lot Task 1.1: Clearing and grubbing Task 1.2: Seeding the Turf Task 1.3: Planting shrubs and trees •Activity 2: Building the House Activity 2.1 : Site preparation Activity 2.2: Building the exterior Activity 2.3: Finishing the interior •Activity 2.1 : Site preparation Task 2.1.1: Surveying Task 2.1.2: Obtaining permits Task 2.1.3: Excavating Task 2.1.4: Obtaining materials •Activity 2.2: Building the exterior Task 2.2.1: Foundation Task 2.2.2: Outside Walls Task 2.2.3: Exterior plumbing Task 2.2.4: Exterior electrical work Task 2.2.5: Exterior siding Task 2.2.6: Exterior painting Task 2.2.7: Doors and Fixtures Task 2.2.8: Roof •Activity 2.3 : Finishing the Interior Task 2.3.1: Interior plumbing Task 2.3.2: Interior electrical work Task 2.3.3: Wallboard Task 2.3.4: Interior painting Task 2.3.5: Floor covering Task 2.3.6: Doors and fixtures Slack Time and Critical Path • •Slack Time • Available Time -Estimated (“Real”) Time for a task or activity • Or: Latest Start Time -Earliest Start Time • •Critical Path • The path in a project plan for which the slack time at each task is zero. • Determined by pre-requisite tasks, task duration and longest path in the schedule. • The critical path has no margin for error when performing the tasks (activities) along its route. Risk Management • Risk:Members in key roles drop the course. • Contingency: Roles are assigned to somebody else. Functionality of the system is renegotiated with the client. • •Risk:The project is falling behind schedule. • Contingency: Extra project meetings are scheduled. • •Risk:One subsystem does not provide the functionality needed by • Contingency: ? • •Risk:Ibuttonruns only under JDK 1.2 • Contingency: ? another subsystem. How to be a good project planner? • • Establish a project plan • Start with the plan based on your experience with the last project(s) • • Keep track of activities and their duration Determine difference between planned and actual performance • • • • Make sure to do a post-mortem • Lessons learned • Ask developers for feedback • Write a document about what could have been improved Project Management Heuristic • •Make sure to be able to revise or dump a project plan • Complex system development is a nonlinear activity • •If project goals are unclear and complex use team-based • Avoid GANTT charts and PERT charts for projects with changing requirements • Don’t look too far into the future • •Avoid micro management of details • •Don’t be surprise if current project management • project management. In this case work: tools don’t They were designed for projects with clear goals and fixed organizational structures Computer Based System Engineering What is a System? •A purposeful collection of inter-related components working together towards some common objective. •A system may include software, mechanical, electrical and electronic hardware and be operated by people. •System components are dependent on other system components •The properties and behaviour of system components are inextricably inter-mingled System Engineering Software and System Engineering •The proportion of software in systems is increasing. Software-driven general purpose electronics is replacing special-purpose systems •Problems of systems engineering are similar to problems of software engineering •Software is (unfortunately) seen as a problem in systems engineering. Many large system projects have been delayed because of software problems System Engineer Task • •Design and creation of a complex system and efficient system • •Preparation of catalogs listing requirements • •Modeling, simulation, optimization and final evaluation of • the system design • •Documentation, functional descriptions and manual creation • •Management and configuration of controls Emergent Properties • •Properties of the system as a whole rather than properties that can be derived from the properties of components of a system • •Emergent properties are a consequence of the relationships between system components • •They can therefore only be assessed and measured once the components have been integrated into a system The overall weight of the system can be computed from individual component properties. The reliability of the system depends on the reliability of system components and the relationships between the components. The usability of a system complex property which is not simply dependent on the system hardware and software but also depends on the system operators and the environment where it is used Types of Emergent Properties • •Functional properties • These appear when all the parts of a system work together to achieve some objective. • For example, a bicycle has the functional property of being a transportation device once it has been assembled from its components. • •Non-functional emergent properties • Examples are reliability, performance, safety, and security. • These relate to the behaviour of the system in its operational environment. • Critical for computer-based systems as failure to achieve some minimal defined level in these properties may make the system unusable. System Reliability Engineering • •Because of component inter-dependencies, faults can be propagated through the system • •System failures often occur because of unforeseen inter-relationships between components • •It is probably impossible to anticipate all possible component relationships • •Software reliability measures may give a false picture of the system reliability Hardware reliability What is the probability of a hardware component failing and how long does it take to repair that component? Signal Software reliability How likely is it that a software component will produce an incorrect output. Software failure is usually distinct from hardware failure in that software does not wear out. Alarm Operator reliability How likely is it that the operator of a system will make an error? Environment Human and Organizational Factors • •Process changes • Does the system require changes to the work processes in the environment? • •Job changes • Does the system de-skill the users in an environment or cause them to change the way they work? • •Organisational changes • Does the system change the political power structure in an organisation? System Architecture • •An architectural model presents an abstract view of the sub-systems making up a system • •May include major information flows between sub-systems • •Usually presented as a block diagram • •May identify different types of functional component in the model Functional System Components Sensor components Collect information from the system’s environment e.g. radars in an air traffic control system Communication components Allow system components to communicate with each other e.g. network linking distributed computers Actuator components Cause some change in the system’s environment e.g. valves in a process control system which increase or decrease material flow in a pipe Co-ordination components Co-ordinate the interactions of other system components e.g. scheduler in a real-time system Computation components Carry out some computations on an input to produce an output e.g. a floating point processor in a computer system Interface components Facilitate the interactions of other system components e.g. operator interface •All components are now usually software controlled