1 INTRODUCTION TO THE COSMIC METHOD OF MEASURING SOFTWARE Bruce Reynolds, November 2015 Outline 2 Cosmic Definition Functional Size Measurement Functional User Requirements Non-Functional Requirements COSMIC Approach In-Class Exercise COSMIC—a Real World example Conclusion COSMIC Definition 3 COSMIC stands for the Common Software Measurement International Consortium. COSMIC is a voluntary organization that has defined an open, ISO standard functional size measurement (FSM1) method, based on fundamental software engineering principles. The International Standards Organization (ISO) FSM standards are: COSMIC (ISO, ISO/IEC 19761) IFPUG (ISO, ISO/IEC 20926) Mark-II (ISO, ISO/IEC 20968) NESMA (ISO, ISO/IEC 24570) FiSMA (ISO, ISO/IEC 29881) What is Functional Size Measurement (FSM2)? 4 An FSM method defines a process to measure a size of the Functional User Requirements (FUR) of software in units of ‘function points’. Functional sizes are independent of the technology used to develop the software, and they can be used as a basis to: 1. 2. 3. 4. 5. Control requirements (AKA scope creep) Measure productivity (= size/effort) of completed projects Compare productivity across projects (e.g., waterfall vs. agile) Measure defect density for operational systems Estimate effort for new projects What are Functional User Requirements (FUR3, 6)? 5 A functional requirement specifies what the software must do in terms of the services it provides to its users. The users of the software being measured, as defined in the FUR, may be: Humans (typical for business applications) Hardware devices (e.g., sensors, actuators typical for real-time software) Other pieces of software (all software domains) The quality of the FUR determines the quality of the functional size measurements (and whether the developers can deliver what the user wants!) 6 COSMIC has approximate sizing approaches for use early in a project before requirements are known in detail. COSMIC concepts map with UML concepts; size measurement of UML sequence diagrams has been automated. Measurement of COSMIC sizes of designs in Simulink has been automated. COSMIC matches perfectly with Agile for sizing User Stories, Iterations, etc., at all levels of aggregation. Software size is the biggest driver of project effort 7 SLOC16: Sizing method options: Functional size: Other sizing methods: Can’t estimate until software is designed Technology-dependent, no standards Accounts for all requirements International standard methods: What about Non-Functional Requirements? UCP, OOP, SP No reliable standards (So no publicly-available benchmarks) Serious project effort estimation needs a measure of software size and data on past performance 8 Development Productivity = Software Size Effort How big is it? Estimated development effort for “New-Project” Est. Soft. Size for “New-Project” = X Past Productivity Adjustments for “New-Project” Once upon a time (i.e. last century), software size measurement was easy 9 Whole business applications Measure Functional Requirements Measure NonFunctional Requirements (Example NFR) Albrecht/IFPU G Function Point Analysis Waterfall (or SLOC) Real-time software systems SLOC Value Adjustment Factor (Mainly on-line vs batch processing) (Mainly timing constraints) Nowadays, the measurement challenge is greater 10 Business Systems, Real-time systems, Infrastructure software Waterfall & Agile (Function Pts. Story Pts) Measure Functional and NonFunctional Requirements UML (UC Pts) SOA Big Data OO (Obj Pts) Systems of systems BYO Device (SNAP) (Example NFR) Whole, distributed, Multi-layer (SLOC) Embedded Model-based development Internet of things Usability, Portability, 24/7 Availability, Security, Privacy, Maintainability, Dependability, Safety, Reusability, Architecture, etc. Why use COSMIC when I can use the IFPUG method? 11 The IFPUG method The COSMIC method Is recognized as the first FSM method (1979) Is primarily designed to measure business applications Is difficult to apply to the following systems: Is a second generation method based on fundamental software engineering principles Is applicable to business, realtime and infrastructure software Fits perfectly with agile, component-based development methods Is ‘Open’ Real-Time systems Distributed systems Mobile applications Does not fit well with modern development methods (Agile, UML, etc.) All documentation is freely available Is easy to understand (100 versus IFPUG’s 500 pages of basic rules, examples, etc.) The advantage of open source, a real world example, Microsoft Encarta17 versus Wikipedia Percentage* of website visits from 2001 to 2010 100% 90% 80% The same can be said for COSMIC versus other FSM methods! 70% 60% 50% 40% 30% 20% 10% 12 0% 2001 2002 2003 2004 2005 Encarta (annual subscription) 2006 2007 2008 Wikipedia (open source) 2009 2010 The COSMIC approach to NFR The measurement of NFR is integrated into the COSMIC approach, versus having a separate method [e.g., IFPUG’s General Systems Characteristics (GSC19), and the Software Non-functional Assessment Process (SNAP20)]. Requirements for a Software System Project 13 Software Functional User Requirements (FUR) System and Software Non-Functional Requirements (NFR) Project Requirements and Constraints (PRC) Quality Requirements System Environment Requirements Technical Requirements NFR typically evolve, wholly or partly, as a project progresses into FUR that COSMIC can measure24 Size by analogy or expert judgement Outline Funct-ional Requts. Outline NFR Project Reqts. & Constraints 14 A r c h i t e c t u r e Approx. COSMIC size measurement Approx Funct-ional Reqts. Precise COSMIC size measurement FUR Implemented software system or software product “True” NFR Requirements Analysis Definition & Design Build, Test & Implement The three phases of the COSMIC functional size measurement process4 Input from measurement sponsor Software Context Model FUR Phase 1 Measurement Strategy5 FUR Generic Software Model12 Definition of each piece of software to be measured and of the required measurement Phase 2 Mapping9 Phase FUR in the form of the Generic Software Model Phase 3 Measurement Phase 15 Functional size of the software in units of CFP A software architecture may exhibit different layers13 depending on the view of the architecture Three possible Software Layer Structures25 View of application ‘A’ as a whole Application ‘A’ components in a 3layer architecture Layers for SOA components of Business Rules Application Layer User Interface Component Application Layer Application ‘A’ Business Rules Component Orchestration Layer Utility Layer 16 Data Services Component Phase 2—Mapping9, Part 1 The four main principles of the COSMIC Generic Software Model12 Software functionality consists of functional processes The task of each functional process is to respond to an event that has happened in the world of the software’s functional users. Functional processes consist of sub-processes Data movement sub-processes (Entries, Exits, Reads and Writes) move data describing a single object of interest. Each data movement accounts for the associated data manipulation. Each data movement is measured as 1 CFP (COSMIC Function Point). 17 Phase 9 2—Mapping , Part 2 18 The relationship between events, functional users and functional processes7 Boundary Triggering Event causes Functional User to generate a data group that is moved into a FP by the FP’s Triggering Entry Functional Process Phase 2—Mapping, Part 3 19 The four types of data movements8 Boundary Functional Users Hardware devices Other software Humans Entries Exits Software being measured Reads Writes Persistent Storage Phase 3—Measurement Phase11, Part 1 The COSMIC measurement principle10 The functional size of a piece of software is equal to the number of its data movements. A functional size is measured in units of ‘COSMIC Function Points’, abbreviated as ‘CFP’. 1 CFP is defined by convention as the size of a single data Movement (Entry, Exit, Read or Write). A way of recording the results of an analysis of measuring functional processes using the Generic Software Model matrix12. 20 E E, W R, X E, W R X X X E X Totals for Personnel System: X 2 1 2 1 6 1 3 1 2 7 1 2 Total Writes Reads Exits Entries Employee Totals Number of Data Movements Employee Current salary End of month 'clock tick' Error/Confirmation Message E, R, W R, X E, W R Employee Salary History Create Employee Read Employee data Update Employee data End of month report Employee ID Personnel System Functional Processes Employee Base Data Data Group Names 2 2 2 5 4 6 6 5 5 22 Calculating the software size with COSMIC14 21 The size of a functional process is the count of its Entries, Exits, Reads and Writes. The size of the FUR of a piece of software is the sum of the sizes of its functional processes. The size of a change to a piece of software is the sum of the changed (added, modified and deleted) data movements. The minimum size of a functional process is 2 CFP (1 Entry + 1 Write or 1 Exit). There is no upper limit to the size of a functional process. Once I have calculated the data movements, how can I estimate the software development effort? The following estimating tools accept COSMIC sizes as input KnowledgePlan (Software Productivity Research) PRICE TruePlanning (Price Systems) ProjectIT (Telmaco, UK) SEER (Galorath) SLIM (QSM) In addition, the above tools will provide the estimator with all aspects of estimating the cost of the software project, not just the software development. You can develop a COSMIC benchmarking database consisting of successfully completed projects which has: 22 Output in CFP (A) Number of work hours (B) A practical example follows… A ÷ B = Benchmark Productivity IN-CLASS EXERCISE A practical example of calculating software size and effort with COSMIC 23 IN-CLASS EXERCISE Calculating the software size with COSMIC First, take the Functional User Requirements, then determine the application boundary. Application Boundary Wylie College Users (Students, Professors, Course Registrar) Course Catalog System 24 E X E X C-Registration System Student schedule Mail system X X Billing system Legend: E – Entry X – Exit R – Read W – Write Calculating the software size with COSMIC IN-CLASS EXERCISE Second, build a sub-functional process matrix (below), then identify the data movements. CFP Registrar enters student data Student data E 1 The system validates the data and checks if a student of the same name already exists Student data R 1 The system creates a new student Student data W 1 Display error message Messages X 1 Triggering event Sub-process Description Add a student Registrar selects “add student” Legend: 25 Data Group Data movement Type Process descriptions E – Entry X – Exit R – Read W – Write In our in-class exercise example, we have 1 Entry, 1 Read, 1 Write, and 1 Exit, for a total of 4 data movements, or COSMIC Function Points. ƩCFP 4 IN-CLASS EXERCISE Calculating the software size with COSMIC Third, document all of the data movements by creating a consolidation table, such as the Generic Software Model Matrix below. Add a Student Totals for System: 26 Translating the COSMIC function points into effort in hours and dollars follows… Total Writes Reads X Exits E, R, W Number of Data Movements Entries Messages C-Registration System Functional Processes Student data Data Group Names 1 1 1 1 1 1 1 1 COSMIC function points 4 0 0 0 4 IN-CLASS EXERCISE Calculating the effort in hours and dollars Fourth, use an estimating tool, such as SEER (Galorath), or Price True Planning, then input the COSMIC function points into this tool in order to calculate the hours and dollars. Description Input 4 COSMIC Function Points PRICE TruePlanning estimating tool, Version 14.2: Size Units: COSMIC Language: Java Development Process: Agile All other variables: set to the 27 default values The PRICE True Planning model then calculates the effort in hours and dollars. Project Initiation and Planning for Development Project Management and Control for Development Quality Assurance Management for Development Configuration Management for Development Documentation for Development Requirements Definition and Analysis System Design Software Integration and Test Hardware Software Integration and Test Operational Test and Evaluation Software Requirements Analysis Software Design Code and Unit Test Software Qualification Test Total Hours Dollars 0.7 $ 109 13.3 $ 2,489 2.6 $ 319 2.6 $ 292 11.3 9.6 5.7 68.2 $ $ $ $ 1,632 1,548 845 8,506 11.1 $ 1,367 9.9 15.5 78.5 91.0 29.0 349.1 $ $ $ $ $ $ 1,494 2,511 11,983 11,446 3,777 48,318 COSMIC—a Real World example 28 Problem or challenge: Renault was unable to predict its software costs early for the Electronic Control Unit (ECU) software in cars Solution: Renault selected the COSMIC FSM standard for measuring the size of real-time embedded software and for estimating project costs. Apply the COSMIC method to develop an automated measurement tool using the Simulink21 model and the MATLAB22 programming language. 29 Outcome: ECU software costs are now automatically calculated with dramatically reduced measurement times. By using the AUTOSAR*23 architecture, ECU software reuse is now possible for different hardware targets. This enables software enhancements to be used instead of new software development. Based on the results from the automated tool, productivity models are prepared and then leveraged to negotiate with suppliers to obtain the best value in supplier costs. COSMIC is also used to predict the memory size needed for embedded software. Once the memory size is predicted, this enables Renault to anticipate the needed margin associated with the start of production milestone. 30 *(AUTomotive Open System ARchitecture) In conclusion, Renault18 uses CFP sizing to control the development and enhancement of Electronic Control Units (ECU) Tracks progress of ECU specification teams… who create designs in Matlab Simulink… which are automatically measured in CFP 31 Motivation for automation: speed and accuracy of measurement … achieving remarkable cost estimation accuracy from the designs Cost vs. size (CFP) Memory size vs. software size (CFP) 32 Conclusion COSMIC is easier to understand and use versus other FSM methods. COSMIC is an open source method. COSMIC can be used to estimate: Business applications Real-Time applications Distributed systems COTS tools or systems Mobile applications Many tools are available that accept COSMIC sizes as input. Cosmic is Based on fundamental software engineering principles, which makes the method future-proof. Used around the world, and the Measurement Manual has been translated into 12 languages besides English. Most importantly: All of the COSMIC documentation is available for free on the internet at: www.cosmic-sizing.org 33 Questions? 34 References 35 References 36 [1] http://en.wikipedia.org/wiki/Function_point [2] http://en.wikipedia.org/wiki/Software_sizing [3] S. Robertson, J. Robertson, Mastering the Requirements Process, Getting Requirements Right, Addison-Wesley, Boston, MA, 2013, p. 223. [4] The COSMIC Functional Size Measurement Method Version 4.0, Introduction to the COSMIC method of measuring software, Version 1.0, May 2014, p. 15, Figure 4.1. [5] The COSMIC Functional Size Measurement Method Version 4.0.1, Measurement Manual, April 2015, p. 20, Figure 2.0. [6] The COSMIC Functional Size Measurement Method Version 4.0, Introduction to the COSMIC method of measuring software, Version 1.0, May 2014, p. 16. [7] The COSMIC Functional Size Measurement Method Version 4.0, Introduction to the COSMIC method of measuring software, Version 1.0, May 2014, p. 22, Figure 6.1. [8] The COSMIC Functional Size Measurement Method Version 4.0, Introduction to the COSMIC method of measuring software, Version 1.0, May 2014, p. 16. References [9] The COSMIC Functional Size Measurement Method Version 4.0.1, Measurement Manual, April 2015, p. 37, Figure 3.0. [10] The COSMIC Functional Size Measurement Method Version 4.0, Introduction to the COSMIC method of measuring software, Version 1.0, May 2014, p. 31. [11] The COSMIC Functional Size Measurement Method Version 4.0.1, Measurement Manual, April 2015, p. 64, Figure 4.0. [12] The COSMIC Functional Size Measurement Method Version 4.0.1, Measurement Manual, April 2015, Appendix A, p. 74. [13] The COSMIC Functional Size Measurement Method Version 4.0.1, Measurement Manual, April 2015, p. 25, Figure 2.2. [14] Software Functional Size with ISO 19761:2003 COSMIC-FFP Measurement Method, C-Registration System, Updated: February 23, 2008. [15] ICEAA Cost Estimating Body of Knowledge (CEBoK), Unit IV – Module 12, page 17. [16] ICEAA Cost Estimating Body of Knowledge (CEBoK), Unit IV – Module 12, page 19. 37 References [17] N. Cohen, Microsoft Encarta Dies After Long Battle with Wikipedia, The New York Times, March 30, 2009. [18] Alexandre Oriou et al, Manage the automotive embedded software development cost & productivity with the automation of a Functional Size Measurement Method (COSMIC), IWSM 2014, Rotterdam, www.ieeexplore.org. [19] International Function Point Users Group (IFPUG), Function Point Counting Practices Manual, Part 5, Appendix C. [20] International Function Point Users Group (IFPUG), Software Non-functional Assessment Process (SNAP), Assessment Practices Manual, Release 2.1. [21] https://en.wikipedia.org/wiki/Simulink [22] https://en.wikipedia.org/wiki/MATLAB [23] https://en.wikipedia.org/wiki/AUTOSAR [24] The COSMIC Functional Size Measurement Method Version 4.0.1, Guideline on NonFunctional & Project Requirements, VERSION 1.0, November 2015. [25] The COSMIC Functional Size Measurement Method Version 4.0.1 Measurement Manual, page 26, Figure 2.4. 38 www.cosmic-sizing.org