Sensitivity Analysis for Search-Based Software Project Management Francisco de Jose King’s College London MSc Computing And Internet Systems 2007/2008 School of Physical Sciences and Engineering Computer Science Deparment Supervised by Mark Harman August 2008 “Live as if your were to die tomorrow. Learn as if you were to live forever” Gandhi Abstract This paper introduces a new perspective in the field of Software Engineering in pursuance of a feasible alternative to the classical techniques of Software Project Management through the use of Genetic Algorithms (GAs) in Sensitivity Analysis (SA). A beneficial solution is important from the point of view of the manager as a result of the increasing complexity of the software projects. The use of GAs in SA can provide new means to improve the initial schedule of a project and thereby tackle the classical Project Scheduling Problem (PSP). The proposed implementation will develop an answer to the managers in their necessity to identify the most sensitive tasks as well as new ways to optimize their project in terms of duration. This paper describes the application of GAs in a process of resource allocation. Moreover, it analyses the impact of breaking dependencies within the definition of a project. The alternative detailed in this paper indicates the suitable direction of future work to achieve a proper results for an implementation of SA through the use of GAs to all the parameters of a project. In so doing that, the biggest negative impact due to the smallest alteration in one of the parameters can provide the most sensitive factors of the entire project. Key words: search-based, software engineering, genetic algorithms, sensitivity analysis, project management. Acknowledgements First I wish to convey my sincere gratitude to my supervisor, Prof. Mark Harman who offered invaluable assistance, support and guidance. Without whose knowledge and counsel this study would not have been successful. I appreciate his direction, technical support and supervision at all levels of the research project. I must also acknowledge special thanks to my colleges from the University of Malaga Juan Jose Durillo and Gabriel Luque for their suggestions and advise which have been inestimable help in the development of this project. The author would like to recognise to Simon Poulding from the University of York for proving valuable advice and assistance in the statistical analysis of this dissertation, for which I am really grateful. Most of all I would like to express my deepest love to my family, my parents Francisco and Maria Del Carmen and my sister Irene. Their endless support, motivation and encouragement through this Master and my life has contributed notably to this project, I owe them my eternal gratitude. ii Contents Abstract i Acknowledgements ii List of Figures iv List of Tables v Abbreviations vi Symbols vii 1 Introduction 1.1 Simple Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Roadmap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 4 2 Problem Statement 2.1 Problem Definition . . . . . . . . . . . . . . . . . 2.2 Task Precedence Graph . . . . . . . . . . . . . . 2.2.1 Task Precedence Graph (TPG) . . . . . . 2.2.2 Critical Path Method (CPM) . . . . . . . 2.2.3 Project Evaluation and Review Technique . . . . . 5 5 6 6 7 8 . . . . . . . . . . . . . . . . . . . . (PERT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Literature Survey 3.1 Search-based Software Engineering . . . . . 3.2 Search-Based Software Project Management 3.2.1 Genetic Algorithms . . . . . . . . . . 3.2.2 Sensitivity Analysis . . . . . . . . . 3.3 Validation of the work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9 11 12 17 20 4 The Model Introduced by this Thesis 4.1 Specific Details of the Model . . . . . 4.1.1 Scenario . . . . . . . . . . . . . 4.1.2 Evaluation (Fitness Function) . 4.1.3 Recombination (Operator) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 24 24 24 25 iii . . . . . . . . . . . . Contents iv 5 Resource Allocation 5.1 Non-Classical GA . . . . . . . . . . . . . . . . 5.1.1 Features . . . . . . . . . . . . . . . . . 5.1.2 First Version . . . . . . . . . . . . . . 5.1.2.1 Individuals and Populations 5.1.2.2 Evaluations . . . . . . . . . . 5.1.2.3 Method of Selection . . . . . 5.1.2.4 Recombination . . . . . . . . 5.1.2.5 Mutation . . . . . . . . . . . 5.1.2.6 Results . . . . . . . . . . . . 5.1.3 Second Version . . . . . . . . . . . . . 5.1.3.1 Method of Selection . . . . . 5.1.3.2 Results . . . . . . . . . . . . 5.2 Classical GA . . . . . . . . . . . . . . . . . . 5.2.1 First Version . . . . . . . . . . . . . . 5.2.1.1 Features . . . . . . . . . . . 5.2.1.2 Individuals and Populations 5.2.1.3 Evaluation . . . . . . . . . . 5.2.1.4 Method of Selection . . . . . 5.2.1.5 Recombination . . . . . . . . 5.2.1.6 Mutation . . . . . . . . . . . 5.2.1.7 Results . . . . . . . . . . . . 5.2.2 Second Version . . . . . . . . . . . . . 5.2.2.1 Features . . . . . . . . . . . 5.2.2.2 Individuals and Populations 5.2.2.3 Evaluation . . . . . . . . . . 5.2.2.4 Method of Selection . . . . . 5.2.2.5 Recombination . . . . . . . . 5.2.2.6 Mutation . . . . . . . . . . . 5.2.2.7 Results . . . . . . . . . . . . 5.2.3 Third Version . . . . . . . . . . . . . . 5.2.4 Features . . . . . . . . . . . . . . . . . 5.2.5 Evaluation . . . . . . . . . . . . . . . 5.2.5.1 New Parent population . . . 5.2.5.2 Results . . . . . . . . . . . . 5.3 Conclusion . . . . . . . . . . . . . . . . . . . 6 SensitivityAnalysis 6.1 Methodology to adapt data . . . 6.2 Project defintions . . . . . . . . . 6.2.1 Project 1: CutOver . . . . 6.2.2 Project 2: Database . . . 6.2.3 Project 3: QuotesToOrder 6.2.4 Project 4: SmartPrice . . 6.3 Sensitivity Analysis Methodology 6.4 Sensitivity Analysis Results . . . 6.4.1 Results Project 1 . . . . . . . . . . . . . . iv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 26 27 28 28 28 29 29 29 29 32 32 32 36 36 36 36 37 37 37 37 37 40 40 41 41 41 41 42 42 44 44 45 45 45 47 . . . . . . . . . 48 49 50 51 51 51 51 52 52 53 Contents 6.5 6.6 6.7 v 6.4.2 Results Project 2 . . . . . . . . . . . . . 6.4.3 Results Project 3 . . . . . . . . . . . . . 6.4.4 Results Project 4 . . . . . . . . . . . . . Statistical Analysis . . . . . . . . . . . . . . . . 6.5.1 Statistical Techniques and Methodology 6.5.2 Statistical Tools . . . . . . . . . . . . . 6.5.3 Statistical Results . . . . . . . . . . . . 6.5.3.1 Statistical Results Project 1 . 6.5.3.2 Statistical Results Project 2 . 6.5.3.3 Statistical Results Project 3 . 6.5.3.4 Statistical Results Project 4 . Threats to Validity . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 60 64 68 68 69 69 70 71 71 72 73 74 7 Conclusions 76 8 Future Work 78 A Appendix 80 B Appendix 97 Bibliography 118 v List of Figures 1.1 1.2 1.3 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 3 2.1 2.2 TPG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 7 3.1 GA’s methodology. Evolutionary Testing. . . . . . . . . . . . . . . . . . . 13 4.1 Two-point crossover operator method . . . . . . . . . . . . . . . . . . . . 25 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 TPG Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sony Results Non-Classical GA V1 . . . . . . . . . . . . . . . . . . . . . Comparison Non-classical GA V1 and V2 with 1000 evaluations . . . . . Comparison Non-classical GA V1 and V2 with 10000 evaluations . . . . Comparison Non-classical GA V1 and V2 with 100000 evaluations . . . Average comparison Non-classical GA V1 and V2 . . . . . . . . . . . . . Comparison Non-classical GA V1 and V2 and Classical GA V1 . . . . . Comparison Average Non-classical GA V1 and V2 and Classical GA V1 Comparison Average Non-classical GA V1 and V2 and Classical GA V1 and V2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.10 Comparison Average Non-classical GA V1 and V2 and Classical GA V1, V2, and V3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 6.14 6.15 . . . . . . . . . . . . . . . Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec Projec 1. 1. 1. 1. 2. 2. 2. 2. 3. 3. 3. 3. 4. 4. 4. Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Normalisation Test Test Test Test Test Test Test Test Test Test Test Test Test Test Test 1. 2. 3. 4. 1. 2. 3. 4. 1. 2. 3. 4. 1. 2. 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 32 34 34 35 35 38 40 . 42 . 47 54 54 55 55 57 57 58 58 61 61 62 62 65 65 66 List of Figures vii 6.16 Projec 4. Normalisation Test 4. . . . . . . . . . . . . . . . . . . . . . . . . 66 vii List of Tables 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17 5.18 TPG representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Duration representation . . . . . . . . . . . . . . . . . . . . . . . . . . . Resource representation . . . . . . . . . . . . . . . . . . . . . . . . . . . Resource allocation representation . . . . . . . . . . . . . . . . . . . . . Sony project resource allocation . . . . . . . . . . . . . . . . . . . . . . . Sony project results for Non-classical GA V1 . . . . . . . . . . . . . . . Sony Average Results Non-Classical GA V1 . . . . . . . . . . . . . . . . Sony project results for Non-classical GA V2 . . . . . . . . . . . . . . . Average comparison Non-classical GA V1 and V2 . . . . . . . . . . . . . Sony project resource allocation . . . . . . . . . . . . . . . . . . . . . . . Sony project results for Classical GA V1 . . . . . . . . . . . . . . . . . . Average Non-classical GA V1 and V2 and Classical GA V1 . . . . . . . Resource allocation representation Classical GA V2 . . . . . . . . . . . . Sony project results for Classical GA V2 . . . . . . . . . . . . . . . . . . Average Non-classical GA V1 and V2 and Classical GA V1 and V2 . . . TPG Representation Second Version . . . . . . . . . . . . . . . . . . . . Sony project results for Classical GA V3 . . . . . . . . . . . . . . . . . . Average Non-classical GA V1 and V2 and Classical GA V1, V2, and V3 . . . . . . . . . . . . . . . . . . 27 27 28 28 30 31 31 33 34 38 39 39 40 43 43 44 46 46 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12 6.13 Original project definition . . . . . . . . . . . . . . . . Transformation project definition table . . . . . . . . . Adapted project definition . . . . . . . . . . . . . . . . Final project definition . . . . . . . . . . . . . . . . . . Projects definition . . . . . . . . . . . . . . . . . . . . Top 10 dependencies Project 2 . . . . . . . . . . . . . Top 10 dependencies Project 3 . . . . . . . . . . . . . Top 10 dependencies Project 4 . . . . . . . . . . . . . Top 10 dependencies Project 1 . . . . . . . . . . . . . P-value Rank Sum test Top 10 dependencies Project 1 P-value Rank Sum test Top 10 dependencies Project 2 P-value Rank Sum test Top 10 dependencies Project 3 P-value Rank Sum test Top 10 dependencies Project 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 50 50 50 51 59 63 67 70 70 71 72 72 A.1 A.2 A.3 A.4 A.5 Top Top Top Top Top . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 82 83 84 85 10 10 10 10 10 Results Results Results Results Results Project Project Project Project Project 1 1 1 1 2 Test Test Test Test Test 1 2 3 4 1 . . . . . . . . . . . . . . . viii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . List of Tables A.6 Top A.7 Top A.8 Top A.9 Top A.10 Top A.11 Top A.12 Top A.13 Top A.14 Top A.15 Top A.16 Top 10 10 10 10 10 10 10 10 10 10 10 ix Results Results Results Results Results Results Results Results Results Results Results Project Project Project Project Project Project Project Project Project Project Project 2 2 2 3 3 3 3 4 4 4 4 Test Test Test Test Test Test Test Test Test Test Test 2 3 4 1 2 3 4 1 2 3 4 . . . . . . . . . . . . . . . . . . . . . . ix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 87 88 89 90 91 92 93 94 95 96 Abbreviations AEC Architecture, Engineering and Construction BRSA Broad Range Sensitivity Analysis CPD Critical Path Diagram CPM Critical Path Method DOE Design of Experiments FAST Fourier Amplitude Sensitivity Test GA Genetic Algorithm PERT Project Evaluation Review Technique PSP Project Scheduling Problem SA Sensitivity Analysis SD System Dynamic TPG Task Precedence Graph x Symbols RAN D M AX number xi 32767 interger To my family xii Chapter 1 Introduction The main goal of this project is the development of an implementation able to provide an improvement over the original schedule in software project management by the use of search-based software engineering techniques. The first phase aims to the deployment of a solution for the process of resource allocation in the definition of the project by the application of genetic algorithms (GAs). At the same time, this work supplements this first goal by a second aspect of improving planning in software project management by breaking dependencies and performing sensitivity analysis (SA). Software project management is a significant up to date area with increasing complexity and demand from almost every sector in the industry and not only in the field of New Technologies. Nowadays, projects in every engineer or industrial sector such as architecture, energy, aeronautic, aerospace, and much more involve the use of specific software and therefore the Project Scheduling Problem (PSP) is more present than ever. As a result, the demand for new tools and techniques by companies and their managers to improve the cost and duration, variables which define the PSP, of software projects is a visible concern. Hence, the idea highlighted of contributing with new means to advance, enhance, and improve the solutions in software project management is significant enough to justify this paper. The remainder of the paper is organised as follows. The next section starts setting out the problem covered in this research in general terms. Thereafter, it is mentioned the previous related work and the main background material, analyzing the ideas proposed in that material and the main differences regarding the focus of this research. Then this paper introduces a section with the main foundations which support it and present the base of the model developed. The next section explains the work that was performed during this research, specifying the main techniques used during its approach as well as the methodology adopted. In this sense, this section concretely states the implications 1 Introduction in software project management of this paper. Likewise, the first subsection describes an accurate specification of the technical details. Straight afterwards, the following section details the two different algorithms developed and the results obtained in the application of those GAs to diverse scenarios constructed from data of real projects. Next, a complete section explains the process of SA performed as well as the process of breaking dependencies in addition to the results obtained. The following section fully details the conclusions obtained after analysing the results. The last section of this paper details which could be the next steps and the future work in the development of this research. 1.1 Simple Example The idea behind this paper is exemplified in Figure 1.1, Figure 1.2, and Figure 1.3. The first figure represents the task precedence graph of a simple project definition. In this project definition there are five tasks with their duration as well as the dependencies between those tasks. For this simple example it is assumed that the resource allocation for those tasks has been made producing the duration indicated in the number inside the octagons, which represent the tasks. In addition, it is also assumed that every task can be at any time due to the resource allocation. This part would correspond to the first phase of this research. Taking into consideration all the premises detailed in the previous paragraph and according to the information revealed in Figure 1.1, the optimal completion time for this project definition would be 26 units of time. Figure 1.1: TPG + Duration. Example 1. The second phase of this research rests in the process of breaking dependencies and performing sensitivity analysis to evaluate whether it is possible to optimise the overall completion time for the project definition. If this idea is applied to this particular 2 Introduction example, it is feasible to reduce considerable the schedule. As it can be observed in Figure 1.2, if the dependence between the task 2 and the task 3 is removed the scenario represented in Figure 1.3 would allow new completion times. In this case, the new optimal completion time would be 16 units of time. Hence, it would be possible to decrease the original schedule by ten units of time which is a significant improvement. Figure 1.2: TPG + Duration. Example 1. Breaking dependencies. Figure 1.3: TPG + Duration. Example 1. Resulting TPG after breaking dependencies. Furthermore, it is necessary to indicate that by removing dependencies the process of resource allocation it would be performed again and therefore, the number of possible combinations is considerable yet not always desirable. The process explained and illustrated in the previous figures is a simplification of the model developed. The purpose of this example is to provide a general idea of the main goal aimed in this research. 3 Introduction 1.2 Roadmap The project first developed an algorithm able to cover the considered parameters of the PSP in order to provide an accurate benchmark for the resource allocation in the project definition. This algorithm worked on different combinations, understanding by combinations the different possibilities of the assignment between the resources available and the tasks defined. This part corresponds to the first phase the research, and as it shows the section 5 of this paper various alternatives were considered and several scenarios were tested. The second phase of the project carried out the process of breaking dependencies and re-running the algorithm developed in the first part in order to evaluate whether it is possible to produce improvements by reducing the benchmark or original optimal completion time. In order to produce trustworthy results, the model developed was applied to different scenarios based on real data projects. Thus, the input data set which fed the model produced valuable output since the possible spectrum of scenarios is infinite whereas the real framework might be limited. The results obtained are evaluated performing sensitivity analysis over the solutions provided by the model. This procedure measures the relative impact of every modification introduced in the original scenario. The data analysed is the effect produced on the original completion time by removing dependencies and altering the resources available. This method tries to identify whether the dependencies which produce improvement are always the same and how they behave in the different scenarios. In addition, based on the results collected in the sensitivity analysis statistics analysis was performed to add reliability to the data produced by the model developed. 4 Chapter 2 Problem Statement 2.1 Problem Definition In the discipline of software project management the main objective is to successfully achieve the project goals within the classical constraints of time and budget utilizing the available resources. In order to be able to accomplish this aim in the best way, companies and managers desire to optimize the allocation of the resources in the tasks, which define the project, to meet the objectives. Based on this distribution, tasks have a specific start and completion time, and the whole set defines the schedule of the project. This plan is classically illustrated using Gantt charts, which is the most common technique for representing the phases of a project. Nevertheless, this assignment is not straightforward since the group of tasks might have dependencies between them. Therefore, it might be necessary to first finish one or more tasks to be able to start a next one. The Task Precedence Graph is the main technique used to represent the tasks of the project and their dependencies. This method of representing for a project should not be mistaken with the other two main techniques, the Project Evaluation and Review Technique (PERT) and the Critical Path Method (CPM), used to analyse and represent the schedule of the set of activities which composes the definition of the project. This difference is explained in detail in the section 2.2 of this paper. The scope of this research is clearly defined within the context of developing new techniques in software project management to improve the optimal solutions of the PSP. In pursuance of this aim, this research focuses on resource allocation and breaking dependencies between tasks to find a new schedule which improves the completion time of the original optimal one. In the interest of this objective, the main technique fully detailed 5 Problem Statement in the section 3.2.1 of this paper is GAs with the use of SA. In consequence, the idea highlighted of optimizing completion times, probably the most complicated concern in software project management as PSP, reasonably validates this paper. 2.2 Task Precedence Graph This section establishes the differences between the the Task Precedence Graph (TPG), which is a method of representation, and the Project Evaluation and Review Technique (PERT) and the Critical Path method (CPM), which are techniques or methods for project management. The main reason in so doing that is that the similarity in terms of concepts and patterns used to represent in schemas these procedures could lead to confusion. Usually, specifying the tasks and identifying dependencies between them is the first step in definition of a project, and the TPG is the best approach to depict this early stage. After this point, it is common in the management of a plan to decide the time that it is necessary to spend in every task in order to complete it. Here, it lays the key point which distinguishes the TPG from the PERT and CPM. 2.2.1 Task Precedence Graph (TPG) The TPG is just a method of illustrating the definition of a project by representing two different features: 1. Tasks 2. Dependencies Figure 2.1: Example of Task Precedence Graph. This methodology only represents the two features mentioned and it does not take into consideration measures about the duration of each task and therefore, cannot establish 6 Problem Statement the known Critical Path Diagram (CPD) neither the minimum time required to complete the whole project. Hence, the exercise of determining the time necessary to finalised a task is a complete different activity. This fact allows establishing a separate list of durations for the tasks calculated in terms of ”unit of time / person” or other kind of measure. As a result the completion time of every task can vary depending on the resources assigned to it. 2.2.2 Critical Path Method (CPM) The CPM is a mathematical method to schedule the definition of a project which is able to calculate the completion time. Although the management of a project involves the consideration of many factors which can be added enhancing the CPM, the basic representation entails the three main features: 1. Tasks 2. Dependencies 3. Duration Figure 2.2: Example of Critical Path Method In this sense, it is understood that the CPM shows a representation where the duration of the task is fixed due to different reasons such as that the resources which perform the tasks have been already assigned or the estimation in terms of duration do not depends on the allocation of resources. The CPM is able to demarcate which tasks are critical and which not regarding the finalisation of the project, and as a result, construct the Critical Path Diagram in addition to the best completion time using the information of the duration of the tasks and their dependencies. Furthermore, this methodology determines the earliest and latest time in which every task can start and finish. 7 Problem Statement New developed versions and tools of the CPM allow introducing the concepts of resources producing variation in the schedule of the project, this fact can lead to misunderstanding between CPM and TPG. 2.2.3 Project Evaluation and Review Technique (PERT) The PERT is second technique used for analysing and scheduling the definition of a project. This method has common similarities in its approach to the CPM, but it is also a technique used in project management for planning the completion time and not just for representing tasks and dependencies as the TPG. The features considered and the means used to displays the information is almost analogue to the CPM; nevertheless, this technique has some differences with respect to. PERT is considered a probabilistic methodology due to its technique of estimating the duration of each task, whereas CPM is considered a deterministic technique. 8 Chapter 3 Literature Survey This chapter introduces the theoretical foundation for the implementations developed, which tries to provide an improvement in the process of planning software project management in a first stage, and a further evolution of breaking dependencies through the use of genetic algorithms (GAs) in sensitivity analysis (SA). Therefore, the aim of this section is to settle the theoretical base of the main techniques and methodologies used in the development of this paper. In so doing that, definitions are given for the key concepts necessaries in the different approaches followed throughout the implementations. In addition, the aim of this chapter is a comprehensive identification of the most relevant literature whose information has direct impact on supporting the background area over this paper is built on. In so doing that, we meet one basic requirement for deploying a remarkable thesis. 3.1 Search-based Software Engineering Search-based software engineering is a field inside software engineering whose approach is based on the utilization and application of metaheuristic search techniques such genetic algorithms, hill climbing algorithm, tabu search, and simulated annealing. These techniques aim to produce exact or approximate solutions to optimization and search problems within software engineering. Its application has been successfully accomplished in different areas of software engineering which were reviewed in the literature survey section. Search-based software engineering and its metaheuristic techniques are mainly represented and based on mathematical data and representation. This fact, as a result, 9 Literature Survey requires the reformulation of the software engineering problem as a search problem by means of three main stages [1]: • Problem’s representation • Fitness Function • Operators However, it has to be taken into consideration that particular search-based techniques may need specific necessities due to their intrinsic theoretical definition. The representation of the problem allows the application of the features of the searchbased technique. Furthermore, it usually entails the description of the framework by expressing the parameters, which are involved in the definition of the software engineering problem and the search-based technique applied, in a numerical or mathematical system. In addition, this manner allows an exhaustive statistical analysis of the results. Fitness function is a characterization in terms of the problem’s representation of an objective function or formula able to measure, quantify, and evaluates the solution domain provided by the search-based technique. Operators define the method of reproduction to create candidate solutions. A considerable variety of operators can be applied depending of the search-based technique chosen, and the way in which it is applied is subject to the features of that technique. The scheme divided in the three key stages proposed by Harman and Jones in [1] is the main base of the model developed in this thesis. Therefore, the main contribution of this paper is the supply of the basic structure necessary in the application of search-based techniques to the problem stated in the section 2.1 of this paper. However, the model developed differs from this work offering its own implementation of a GA as well as the application of SA to tackle the problem. The recent development in search-based software engineering and its different techniques as well as its successful application in different areas has been exhibited in a significant variety of papers. It is important highlighting certain articles such as [2] which states the benefits of the techniques of this particular area of software engineering and provides an overview of the requirements for their application. Moreover, it denotes a set of eight software engineering application domains and their results. The relevance of these techniques is demonstrated by their application in a wide variety of areas with encouraging results [3][4][5][6][7]. 10 Literature Survey [2] stated that search-based optimization techniques have been applied widely to a large range of software activities. The list mentioned covers different areas of the life-cycle such as service-oriented software engineering, project planning and cost estimation, compiler optimization, requirements engineering, automated maintenance, and quality assessment. Furthermore, [2] offered a broad spectrum of optimization and search techniques for this purpose such as genetic algorithms and genetic programming. In addition to the allusion to GAs the same paper also remarked one key point that is directly related to interest of this research. It cited the crucial importance of the fitness function as the main difference between obtaining good or poor solutions no matter the search technique. Harman in [2] analysed different optimization techniques by dividing them into two main groups: classical techniques such as linear programming; and metaheuristic search such as Hill Climbing, Simulated Annealing, and GAs. Moreover, Harman [2] showed a special concern about the role of sensitivity analysis in search-based software engineering. It turned the focus of the approach of the software engineering problems from global optimum to the impact of the input values in different questions such as the shape of the landscape and the location of the search space peaks. This thesis is influenced by the work done by Harman in [2] because it also states motivation for metaheuristics techniques, particularly GA. However, this work developed an implementation of that particular technique to cope with the PSP. In addition, it performs an SA which is enhanced by the use of statistical tests. Therefore, this thesis does not raise SA as a future work, but it produces it as an essential part of the model. 3.2 Search-Based Software Project Management Currently, software project management is a major concern software engineering. This fact is proof by the considerable amount of techniques that are being applied in this field. One of the areas that is making a special effort is search-based software engineering. Within this area, diverse authors use techniques such as hill climbing, GAs, and simulated annealing. The main technique used in this thesis is GAs. As a result, despite successful application of other techniques in search-based software engineering problems such as hill climbing, the literature survey of this thesis is mainly based on GAs. There is considerable amount of information related to the field of software project management and more recently in the application of new techniques such as GAs [3][4][5][6][7][8][9][10]. The use of SA in project management is relatively new in terms of application. Hence, the literature survey mentions not only papers with the basic notions of SA, but also 11 Literature Survey different works where this technique was applied with interesting results in areas not related to software project management. The main reason is to show the capacity of this technique particularly measuring input factors of a model. This contributed significantly to the idea of establishing which dependencies produce a bigger impact and therefore reduce the completion time in a greater proportion when they are removed. 3.2.1 Genetic Algorithms Genetic algorithms are a stochastic search-based technique and a particular class of evolutionary algorithms which performs an iterative optimization method based on the processes of natural genetics to resolve or approximate a solution. The foundation of GAs is to extrapolate the Charles Darwin’s evolution theory to an algorithm which is able to evolve possible candidate solutions in a process of optimization in the pursuance of the solution of a problem. It is generally agreed that genetic algorithms have application in a significant variety of areas such engineering, computer science, mathematics, physics and so on. In the development of the GAs it can be distinguished three main points: • Definition of the process. Evolutionary testing is going to be used as a technique. Which procedure is shown in Figure 3.1. • Definition of the Fitness Function. The Fitness Function is the objective function used in the process of evaluation of the candidate solution or individuals. • Definition of the genetic operator. The election of the operator will define the way in which a population of chromosomes or individuals is recombined to generate a new individual. The methodology of GAs consists in a sequence of actions explained following and illustrated in Figure 3.1. In the first one, a population of individuals also known as chromosomes is generated randomly. Each of those individuals represents a candidate solution and therefore the initial search space of the algorithm. The next step consists in the evaluation of every possible candidate solution through the use of the fitness function and verifying whether the problem has been resolved or not. The fitness function has been previously defined for the particular problem and it is able to assess the genetic representation of the individual. The third action in the algorithm specifies a sequence of steps included within a loop looking for an evolution in the candidate solutions or individuals by recombining and mutating them. The sequence continuously performs a selection whose main purpose is to choose the individuals to generate a new population. 12 Literature Survey Figure 3.1: GA’s methodology. Evolutionary Testing. Individuals with better fitness function evaluation are likely to be elected, although not in an exclusively way, since this fact would incur in a problem of non-diversity within the population. There is a considerable diversity of methods of selection. A case in point, used in the implementation of this paper, is the tournament selection [11]. The next step is the process of recombination of the individuals previously selected by the methodology defined by the genetic operator agreed. In the same way that with the method of selection, there is a considerable variety of genetic operators, a good and relevant example deployed in the implementation of this paper in the two point crossover operator [12]. The intention of this action is to combine different candidate solutions in order to generate the new individuals of the regenerated population. After that, the sequence performs the step of mutation whose main purpose is to try to introduce genetic diversity by randomly mutating the individuals. In so doing that, the algorithm attempts to keep the process of evolution. The last two steps of the loop consist in evaluating the new candidate solutions generated by the algorithm and reinserting them into the new population. In the use of GAs this research is significantly inspired by the work done by Massimiliano Di Penta, Mark Harman, and Giuliano Antoniol [3][5] about software project management in a search-based approach. In addition, [4] reinforce the application of GAs in Search-based technique for optimizing the project resource allocation. Likewise, the work of this research is encouraged by the concerned shown by Shin Yoo in his work [10] 13 Literature Survey about data sensitivity in the application of a Novel Semi-Exhaustive Search Algorithm. In [3][5], based on previous work [4], is declared the application of GAs, which obtains significant better results than random search, to implement the search-based approach. This methodology provides empirical evidences of the variance in the completion time and the allocation of the resources by the overhead communication produced in the Brook’s law. Moreover, this approach is able to evaluate the impact of Brook’s law in the completion time of the project providing a prudent technique to improve software project management scheduling. The work done by Antoniol et al in [3][4][5] contributed to this thesis in the idea of using search-based approach to face the resource allocation in the project management and the scheduling. However, those papers are orientated in complete different direction. [3] was particularly interested in attempting to evaluate the effect of communication overhead on software maintenance project staffing. It analysed the impact of different factors such as the dependencies between tasks on project completion time, which is also related to this thesis, and on staff distribution. The base of this work is to disclose the influence or effect of Brook’s law on the two goals previously mentioned. The empirical study presented an evaluation of the relation between the tasks and the effort (persons/month). Whereas the main focus of this paper is to apply search-based techniques to decrease the completion time by removing dependencies and performing SA. The paper [4] primary compares different search-bases techniques, using queuing simulation in the fitness function evaluation. This similarity can be also appreciated in [3] , where the use of GAs and a queuing system is part of the model. The use of GAs to analyse the resource allocation is common in the approach of this thesis and [4][3], yet its use is different. The GA implemented in this thesis does not use this queuing system and it was developed to cover the parameters considered for the duration aim of the PSP in order to allow the process of altering the TPG and study its impact over the project completion time. In the case of [5], it contributed to this thesis in its demonstration of the capacity of the GAs to tackle resource allocation. This is essential part of the model generated in this work. However, the main effort of [5] is concentrated on a search-based approach over undesirable but frequent situations in project management, such as rework or abandonment of the tasks, uncertain estimations, and errors. The idea behind this thesis also implies resource allocation, yet it focuses on offering alternatives to the project manager by approaching the PSP in a new way, breaking dependencies. Hence, the robustness of the initial scheduling offered and the possible setbacks or issues arisen during the project are not considered. 14 Literature Survey The second main source of previous related work in GAs, which is considered as significant valuable background, is the material of Enrique Alba and Francisco Chicano [8][9] in the application of GAs to software project management. In this work, the application of GAs to different scenarios to resolve the PSP provides significant results in terms of the sensitivity of the classical variables: number and duration of tasks, skills, available resources, dedication, and cost. These results are obtained through the use of an instance generator developed specifically by this purpose, which automatically creates different project scenarios based on the customize file configured. As it is always done in the exercise of GAs, a fitness function calculates the achievement of a successful result. This work states interesting details of the fitness function, the different considerations within the algorithm developed, and the behaviour the of the instance generator which are fully detailed in [8]. Yet, the most remarkable attainments are the results obtained and described in the section 6 about the experimental study. This study could be considered a sensitivity analysis of the different parameters, accurately represented in various tables and an absolutely complete set of figures. Moreover, the study concludes the variable importance the parameters. A case in point is the number of tasks, which considerably increases the difficulty to obtain a solution in the application of the GAs. This work done in [8] exhibited the ability of the genetic algorithms to face the Project Scheduling problem. Nevertheless, this work also evidences certain level of incapacity to offer solutions when the number of possible combinations is increased by the rise of the input values which feed the model. The algorithm developed follow the classical convention for the GAs as can be appreciated in Algorithm 1. Algorithm 1 GA Pseudocode. Adopted from [8] Initalize Evalute while do Select Recombine Mutate Evaluate Replace end while The model produced by Alba and Chicano in [8] has one primary strength and one main weakness. The major positive point of this work is its competence to deal with the parameters of the two goals in conflict that define the PSP the cost and the duration of the project. This work takes into consideration tasks, duration and dependences between those tasks, resources, skills of those resources, effort between tasks and resources, salaries, and dedication of the resources to the tasks. However, the method of representation for the dedication within the algorithm could generate non-feasible solutions. 15 Literature Survey This important fact is the main disadvantage of the model. By non-feasible solution it is understood those where one resource has overwork and therefore the dedication assigned to multiple resources is impossible to be reproduce in the real work environment. In addition, the model could generate non-feasible solutions in two more situations. First, if one of the tasks has no resource assigned. The last one, when the resource assigned does not have the necessary skils to perform the tasks. As a result, the authors of the paper introduced a fitness function able to discern the viability of the solutions by adding a variable of penalty. Forming the fitness function 3.1, adapted from [8]. The term (q) measures the quality of the solutions generated and the second term (p) calculates the penalty. ( f (x) = 1/q if f easiblesolution 1/(q + p) otherwise (3.1) Alba and Chicano in [9] demonstrated the capacity of the search-based technique to approach the Project Scheduling Problem. This paper exhibited the ability of the metaheuristics techniques and the genetic algorithms in particular to manage optimally the time and the cost in the project management. Alba and Chicano perform an incredible realistic work by considering the most classical parameters in the PSP. This fact and the different scenarios processed allow them to exercise a precise analysis of the influence of the attributes in the solutions of the problem. Both authors mentioned the difference between the application of GAs in software engineering and other fields such as bioinformatics, mathematics, telecommunications, and so on. This metaheuristics technique is not so intensively used in software project management. Nevertheless, their results and conclusion proved the capacity and accuracy of the genetic algorithms to help the managers addressing the project management. The work done by Alba and Chicano in [8][9] significantly inspired the model developed in this thesis contributing with the idea of using GAs to face the resource allocation in the PSP. Nevertheless, the focus between those papers and this work is completely different. In [8][9] Alba and Chicano studied the behaviour of the model developed in order to produce an optimal resource allocation in diverse scenarios facing both goals of the PSP, cost and duration. For this purpose, they took into consideration parameters which affect both aspects such as skills of the resources, for instance. Whereas the main aim of this thesis is to provide new means and methods to improve the overall completion time of a project by removing dependencies and performing sensitivity analysis in this action. All the efforts done in this work is only focus on the improving only one of the goals of the PSP, duration. Therefore, despite both works attempt to help managers 16 Literature Survey in software project management by the use of GAs the ultimate targets are completely different. 3.2.2 Sensitivity Analysis This section briefly and concisely defines sensitivity analysis and the main concepts surrounding its study. In addition, it analyses the main previous related work in the field of sensitivity analysis and in the area of software project management in particular. Sensitivity analysis (SA) is the study of the impact in the output model value due to variation in the input model source parameters. Therefore, sensitivity analysis is mainly used in the study of the sensitivity of a model or a system in changing the input value parameters which define that model or system [13]. This kind of analysis is generally known as parameter sensitivity. Despite other authors also mention structure sensitivity, that study is not matter of interest to the main purpose of this paper. Consequently, sensitivity is the statistical measure of the effect in the outcome in comparison with the modification in the parameters that determine it. Ergo, the greater in the variation of the result per unit the greater the sensitivity is. A key point in this research is a comprehensive, complete and thorough related background documentation of SA, which is necessary as a base. The general introduction done by Lucia Breierova and Mark Choudhari for the MIT [13], as well as the definition of Global Sensitivity Analysis proposed by Andrea Saltelli [14] provide a good support for the understanding of the topic, giving premises and hints to carry out this methodology. As it is mentioned in the introduction of this paper project management and software development are present in all the engineering areas, and the application of SA has already been experimented in fields such as planning process in architecture, engineering and construction (AEC) [15] and detecting and elimination errors in software development [16]. The research done by Breierova and Choudhari in [13] and the one done by Saltelly in [14] guided this thesis towards the concept of parameters sensitivity. Within sensitivity analysis, it is the action of analysing, evaluation and measuring the impact of variations in the input of the model to examine the behaviour responds of it. These two sources mentioned, helped to understand the force of this theory providing a consistent base, precise definitions and relevant examples. However, this thesis uses all this established knowledge in the specific application of software project management, evaluating the impact of removing in turns all the dependencies that compose the TPG in the PSP. 17 Literature Survey [15] does not have a significant similarity with the work developed in this thesis. However, the work done by Lhr and Bletzinger was beneficial to exemplify the capacity and the application of sensitivity analysis in different areas. Furthermore, the idea of evaluating the impact of diverse factors on the development of the planning process and therefore, on the goal of time for optimal planning join the focus of this thesis in a certain level. Since, it also performed sensitivity analysis to factors which have direct impact in the duration of the PSP for software project management. The work produced by Wagner in [16] coincided with the area of this thesis. It proposed a model able to evaluate the quality costs in software development by deploying an analytical model and performing sensitivity analysis. This work is also partially based on the concept of global sensitivity analysis arisen by Saltelly in [14]. Therefore, Wagner contributed notably to the idea of applying SA to software project management. Nevertheless, this thesis differs from this work since the scenario and the factors which feed the model and the model indeed are completely different. In general terms, the research of Wagner focused on costs whereas this thesis tackled the other main goal of the PSP, the duration. In addition, in [16] faced defect-detection techniques and its quality cost associated. Thus, it did not approach the PSP directly. Sensitivity analysis helps to identify the critical parameters which have more repercussion or influence in the output of the model or system. The current relevance of performing SA in the development of model or system is stated by Saltelli in [17] where it is supported by the allegation of the adequate advance of its theoretical methods. A good example of its application and its positive results collecting information of the impact of the input parameters is [18]. Furthermore, SA can alleviate the problem of uncertainty in input parameters [19]. The papers [17] [18] [19] assisted significantly in the generation of this thesis, although there are substantial differences. The work of Saltelli in [17] did not contribute to the model developed in this thesis to a particular specific part. Yet, it was considerably helpful to understand the concept behind the sensitivity analysis. Furthermore, it contributed to comprehend the importance of the factors which determine a model in order to reveal information in the context of model-based analysis. In the case of [18], despite the work was a clear sensitivity analysis within the context of project management, its focus lay in analysing uncertainty and risk to justify or not investment in projects. By contrast, in this thesis, sensitivity analysis is performed to offer the manager different options to approach the PSP. In conclusion, both models the one developed by Jovanovi in [18] and the one developed in this thesis work over parameters which usually are involved in project management. Nevertheless, the final aim of this analysis is completely different. 18 Literature Survey Johnson and Brockman in [19] demonstrated the capacity of sensitivity analysis to identify and reveal the mechanisms that have the greatest impact on design time in design process. Thus, the idea behind this paper is not related at all with the PSP in project management. However, it added the concept of using sensitivity analysis to measure improvements in completion time when this is an essential factor of the model or problem that wants to be faced. In addition, it exists the application of SA in project management with different approaches such as Hybrid Models over Design of Experiments [20], MonteCarlo method [21], and Fourier Amplitude Sensitivity Test (FAST) [16][22]. The main contribution of these papers was its extensive use of SA to different aspects of software project management or software engineering. However, there are relevant differences with the main focus of this thesis. All these paper used specialised techniques of SA to their particular issues. Whereas, the study done this thesis, produced a detailed evaluation of the behaviour of the model in the different tests performed in common SA. Furthermore, the scenario of application for this thesis, which is the classical PSP, was tackle in none the papers mentioned. Particularly, the action of removing dependencies and measuring its impact in the completion time of the project entailed a complete new are of survey. Consequently, despite there is a common field of research between theses works, their kernel of experiment is entire different. First, Hybrid Model over Design of Experiments (DOE) [20], which is based on previous models: System Dynamic (SD) Models, to obtain the dynamic behaviour of the project elements and their relations; State Based Models, which reproduces process activities and represents the dynamic processes all over the state transitions initiated by events; and Discrete Models, to reproduce the development process. This hybrid model consists in associating the discrete model and a process of continuous simulation able to reproduce tasks which are affected by the constant changes of the parameters, and using the state based model to express the changes of the states. Thus, Hybrid Models are able to analyse the impact of changes in a dynamic project environment. Furthermore, the most interesting affirmed aspect of this paper [20] is the possibility to show non-linearities not discovered by the common sensitivity analysis using DOE in combination with Broad Range Sensitivity Analysis (BRSA). The second main approach in the use of SA is Montecarlo method and its software covered in the research of Young Hoon Kwak and Lisa Ingall [21]. This paper applies software based in MonteCarlo method to plan the project by the analysis, identification and assessment of the possible problems and their circumstances within the context of the development. This methodology has not been totally accepted in project management for a real use although it has been used in several areas which have connections to 19 Literature Survey modelling systems in biological research, engineering, geophysics, meteorology, computer applications, public health studies, and finance. The last approach is mentioned by Stefan Wagner [16] in his research of defect-detection techniques in SA. The pillars of this work are the use of Fourier Amplitude Sensitivity Test (FAST) based on the performance of Fourier functions and Simlab software. According to Wagner this method is able to provide a quantification and qualification of the influence of the parameters. Again, although the Simlab and its capacity due to its features seem to be very interesting, the FAST method may be not as appropriate as BRSA for the purpose of this research. 3.3 Validation of the work There is a considerable amount of material where a visible concerned about project management is exposed. Furthermore, the use of techniques such as GAs and SA is widely accepted and used in tackling a significant variety of issues as it has been described in among this section of the paper. Nonetheless, there are significant differences from existing work related and the main objective of this research. Next, it is made a brief, yet clear and explicit comparison with the closest previous related work that undoubtedly distinguishes this research. [3][4][5] attempt to improve the completion time based on the application of searchbased over different formulations of Brook’s law. Which means, increasing the number of employees, whereas the main focus of this paper is breaking dependencies between tasks. The second main related source is [8][9], where the kernel is the analysis of the differences of applying GAs taking into consideration the mentioned parameters of a project in the pursuance of the optimal solution for the plan. The focus of this paper, on the other hand, is to improve the optimal solution breaking the dependencies of the TPG. Thus, it is possible to state that although this research is thoroughly based on different works already done in this field such as software project management with GAs [8] and search-based approach on software maintenance project [3][4][5], as well as, planning process in architecture, engineering and construction (AEC) [15] and Detecting and Elimination Errors in Software Development [21]. Indeed, it does not comprise an extension or validation of earlier work. The framework of this research is defined within the scope of relevant researchers in this field, yet it deploys an area in the pursuance of a new solution with the application of GAs and sensitivity analysis. 20 Chapter 4 The Model Introduced by this Thesis The main aim of this chapter is to describe the technical specification that is going to be adopted for the purpose of this work. This means the methodology as well as base over which is going to be built on. Hence, taking into consideration all the information described in the introduction of this paper as well as the detailed previous related work, the context of this research is defined within the classical PSP and the use of GAs in the scope of SA. In this empirical study the principal goal is to obtain a better completion time based on the optimal schedule for a particular project. In order to fulfil this purpose, the key point proposed is the action of breaking dependencies of the TPG and evaluating the completion time with the parameters of the PSP through the use of GAs. The ambition of this research is to offer the manager a range of alternatives with the maximum improvement over the initial optimal schedule with the minimum modification. The individual in this role will be then in a position to estimate whether the choice proposed by the run of the model can be carry out or not. In the furtherance of a successful culmination of this work, the first step necessary to perform is the definition of the scenario, premises, variables, and constraints that are going to be considered. The proposed scenario will be one able to enclose the model of the PSP described in the introduction of this paper. As a result, the scenario will cover a task precedence graph that will represent the dependencies between the tasks that compound the project. In addition to this representation, the scenario will be defined by: number and size of resources which perform the tasks; duration of every task; and assignment or allocation between resources and tasks. Note that the information related to the cost is not going to be considered, since the main objective of this research is to improve 21 The Model Introduced by this Thesis the completion time. Therefore, variables of the cost of every resource are beyond the scope of the work proposed. Nevertheless, if a positive result is obtained in a fully implementation of the problem presented, the opportunity of enhancing it considering the parameters related to the cost would allow to have a complete improvement to the PSP. Since, cost and duration of the project are the two principal factors which define the solution to the PSP. The constraints of this work are mainly imposed by the parameters considered and developed in the implementation, as a consequence of the impossibility of adapting the model to all particular features of every single project. However, it is generally agreed that the set of parameters already listed are powerful enough to represent the main aspects of a project and at the same time provide a complete solution. The next step in the approach of this research is the definition to the methodology that is going to be used. In the methodology adopted for the implementation of this research the first main technique used is GAs. The election of this search technique is due to the fact that it outperformed other techniques such as random search in finding or approximating a solution in an optimization or search problem. This fact is corroborated for example in [3] where it is stated a considerable improvement regarding the use of Hill Climbing. The second and most relevant method within this research is the use of sensitivity analysis, mainly focus on parameter sensitivity. The essential idea is to perform a series of tests using the developed GA and evaluate the impact on the project completion time response. In this sense, the principal aim is to break the dependencies between tasks in a minimum way to obtain the biggest impact and therefore the greatest improvement in the completion time of the schedule’s project. Nevertheless, this basic approach can be enhanced with the application of more sophisticated techniques of sensitivity analysis. These techniques were mentioned in the section 3 of this paper as Broad Range Sensitivity Analysis (BSRA) combined with Hybrid Models over Design of Experiments (DOE), MonteCarlo method and Fourier Amplitude Sensitivity Test (FAST). Yet, these techniques have an intrinsic complexity and their effectiveness is more difficult to measure. According to the documentation researched and mentioned in the section 3, probably BSRA is the one which best link with the direction of this research. The last step in the methodology is the evaluation of the model, its implementation, and the results collected. The aim was clearly defined as the greatest reduction in the completion time, which can be easily measure using a unit of time such as days. However, the establishment of the minimum variation in the initial schedule regarding the action of breaking dependencies is more difficult to measure. Likewise, it is significantly complicated to endow a benchmark in the measurement. The first logical consideration 22 The Model Introduced by this Thesis is that if fewer dependencies are broken the impact is minor, yet this way does not face the fact that the importance or weight of the dependencies within the project can be dissimilar. Ergo, the ideal scenario would be taken into consideration both aspects in a unique formula. Consequently, the process of the model developed within this project has five different phases clearly differenced: 1. Definition of the initial scenario. In this phase of the model, the parameters already detailed which define a project within a PSP are taken into consideration to expose the start point. 2. Best schedule algorithm. The second phase consists in the execution of the GA to provide the best resource allocation for the initial scenario considering the variables entertained in the previous phase. 3. Establishment of the optimal resource allocation. This best optimal solution produced in the preceding phase is settled as benchmark for comparing the results obtained in the next stage of the model. 4. Process of breaking dependencies. The aim of this phase is producing results using the same GA developed for the second phase of this model but feed it with a different project definition. The new project definition will be the same TPG but omitting in turn every of the dependencies which conform it. 5. Sensitivity Analysis. In this last stage of the process in the model, sensitivity analysis is performed to obtain a list with the most potential sensitive dependencies as well as the possible explanation of the impact produced. In spite of the fact that time and the scope of this research does not allow to extent the sensitivity analysis further. It would be extremely interesting to consider the possibility of a section to identify the most critical parameters in combination with tasks using the same concept of GAs and sensitivity analysis. Developing an algorithm able to measure the maximum variance in the optimal schedule produced as a result of a minimum variance in one of the parameters which define the tasks and the project. 23 The Model Introduced by this Thesis 4.1 4.1.1 Specific Details of the Model Scenario In the pursuance of this goal all the algorithms and implementations tried so far are based on scenarios which are defined by the following features: • TPG • Duration (unit: days) • Resources (teams: people) The TPG was defined in the section 2.2.1 of this paper and is considered as the basic definition of the project. This basic definition presents exclusively the list of tasks which conforms the project as well as the dependencies between those tasks. Therefore, there is absolutely no mention about the duration of those tasks neither minimum completion time for the project nor critical path diagram. The next step in the definition of the scenario is the establishment of the duration of each tasks measured in ”unit of time / unit of resource” which is specified as ”day / person”. This action and the precedent one of defining the TPG demarcated this framework for a specific project. Lastly, it is necessary to delimit the amount of resources available to perform the project. The definition of this parameter is subject to the way in which the duration of the tasks where set. As a result, it is possible to characterize a project depending on the resources, expressed in terms of teams composed by different number of people, and its assignment to the tasks. This fact leads to different specific durations for each task. Throughout the diverse implementations developed different scenarios with different compositions have been tested and are fully detailed and explained in the sections 5.1 and 5.2 of this paper. 4.1.2 Evaluation (Fitness Function) The fitness function of this model evaluates the completion time of the project definition. This consists on calculating the duration of the project based on the TPG and the duration of each tasks plus the resource allocation generated by the GA. Thus, the fitness function obtains the start time for each of the tasks, by assigning the latest end time of all the tasks which depends on and the latest end time of all the possible 24 The Model Introduced by this Thesis previous assignments of the same resource to other tasks in a slot time which could overlap. Finally, it obtains the end time by diving the duration of the task -measured in ”day / person”- by the composition of the resources -measured in number of people- and adding to the start time. Hence, the latest time of all the tasks will be the completion time of the project. It has to be taken into consideration that this fitness function in addition to the structures of representation of the model, later describe in the section 5 of this paper, as well as the methodology of the GAs and evolutionary testing explained in the section 3.2.1 do not always produce the real unique and absolute solution. This fitness function always evaluates the individuals in the same order from 1 to the number of tasks. Therefore, two tasks which are not dependant between themselves are always performed in the same order. A better possible solution could be achieved if the order is swapped, since they can have different duration releasing a resource in a different time as a result. The number of possible alternatives to this problem is considerable, it could be established a queue to store the order in which the tasks are performed, for example. However, they main focus of this research is the sensitivity analysis process by breaking dependencies. Thus, the method of evaluation developed satisfied the necessities of this research. Moreover, it rational and legitimate to use and consider this fitness function, because the solutions provided by the model implemented using this methodology of evaluation, ensures computing the solutions in the same way when the sensitivity analysis is performed. As a result, the complete process of removing dependencies and measuring their impact in the overall completion time of the project has not been compromised. 4.1.3 Recombination (Operator) The operator for the recombination process of this implementation is the two-point crossover. The method implemented is common to all the different versions the algorithm developed. An example of this mode of recombination is displayed in Figure 4.1. Figure 4.1: Two-point crossover operator method. The methodology consists in choosing randomly two points of the individuals that are going to be recombined and swap the composition of those candidate solutions in both sides of each point. 25 Chapter 5 Resource Allocation The main aim of this chapter is to explain the process performed in order to obtain the most appropriate GA to provide the resource allocation between teams and tasks. In so doing that, it was possible to establish a benchmark to compare the results in the procedure of sensitivity analysis and breaking dependencies. The number of possible combinations in this process of resource allocation is considerable, overall when the number of tasks and dependencies exceed 40 in each case. As a result, the following section details the effort done in developing different versions of GAs with different techniques to obtain the best approximation. By the best approximation it was considered that version of the GA that after 30 runs is able to produce the shortest completion time for the project definition in average of all these executions. This average was used as the benchmark to compare with the results obtained in the process of breaking dependencies. All the features of the GA, the method of representation as well as the project definitions which compose the complete scenario tested are thoroughly described in the following sections of the paper. 5.1 Non-Classical GA This section explains the features and the results obtained using a non-classical genetic algorithm developed. The main characteristic which lead to this non-classical denomination is that instead of having a new population generated in every iteration of the loop of the GA, there is only one in which two new individuals are added. Thus, the process of selection is always performed over one only population which increases in size in every iteration of the loop. 26 Resource Allocation with GAs 5.1.1 Features This subsection describes the method of representation for the main structures of the implementation used, which are common to the different scenarios, as well as other relevant features of non-classical GA’s implementation. The first part cited in the section 4.1.1 of this paper was the task precedence graph. For the representation of the TPG a simple binary matrix is used, where the pair between the column and the row represents the dependency between the tasks named by those indexes. An example is illustrated in Figure 5.1 and Table 5.1. Figure 5.1: TPG Example. T1 T2 T3 T4 T1 0 1 1 0 T2 0 0 0 1 T3 0 0 0 1 T4 0 0 0 0 Table 5.1: TPG Representation. For the second part which defines a scenario the structure used is an array that contains the duration of every task as it is shown in Table 5.2. T1 T2 T3 T4 10 7 15 12 Table 5.2: Duration representation. The last part that is necessary to represent in a scenario according the section 4.1.1 of this paper is the resources which are going to perform the tasks. The structure of representation implemented is an array that indicates the size of the resource in terms of number of members which conform the team. An example can be appreciates in Table 5.3. In addition to the structures necessaries for the representation of the scenario it is also needed one more to express the resource allocation. For this version of the non-classical genetic algorithm the structure used is a binary matrix where rows describe tasks and columns resources. Therefore, a 1 in the row 2 and column 3 indicates that the task 2 in 27 Resource Allocation with GAs R1 R2 R3 3 4 5 Table 5.3: Resource representation. assigned to be performed by the resource 3. An illustrative example is shown in Table 5.4. T1 T2 T3 T4 R1 0 1 1 0 R2 1 0 0 1 R3 0 0 0 0 Table 5.4: Resource allocation representation. In the development of this implementation has been considered that only one resource can be assigned to one task at the same time. Thus, the population of the genetic algorithm will be composed by individuals or candidate solutions with this architecture of representation. The last relevant structure to complete the configuration necessary to develop the complete implementation is the representation of the population. This population is an array of individuals and therefore, and array of binary matrixes. 5.1.2 5.1.2.1 First Version Individuals and Populations In this first version of the non-classical genetic algorithm the implementation generates one only population which is initialised with 100 individuals generated randomly. As a result, there is no parent and children population. In the iterations of the evolutionary testing process of the algorithm, two individuals of the population are chosen to generate two new ones. This action increases in each step the size of the population without discarding individuals generated previously. This leads to immortal individuals and it is the key point in which this algorithm differs from the classical implementations of the GAs. 5.1.2.2 Evaluations Taking into consideration the previous section 5.1.2.1 the number of evaluations corresponds to the number of individuals generated inside the population. Therefore, the 28 Resource Allocation with GAs algorithm works all the time over only one population until it reaches the number of evaluations previously indicated in a variable. The method of evaluation is the application of the fitness function specified in the section 4.1.2 to the different individuals. 5.1.2.3 Method of Selection The method of selection implemented in this first version of the non-classical algorithm is a non-orthodox one. This method always takes the two individuals with the best FF in order to generate the next two individuals. 5.1.2.4 Recombination The method of recombination used is the two point crossover operator detailed in the section 4.1.3 of this paper. Although the representation of the individuals of the population is a matrix as it is explained in the section 5.1.1, in terms of programming is treated as an array linking one after other all the different rows that compounds the matrix. 5.1.2.5 Mutation The process of mutation developed in this version of the algorithm does not follow the classical standards of the GAs. First, random number of mutation between zero and the number of tasks is generated. After that, for every of this mutation a position of the binary matrix, which represents the candidate solution for the resource allocation or individual of the population, is generated randomly. The mutation of the position consists in producing a random number between 0 and 1. If the position mutates from 0 to 1 a check modify the previous resource assigned to perform that task to 0. In case the mutation is from 1 to 0, another resource is randomly assigned to that task. 5.1.2.6 Results The results of this section shows the solutions obtained in the application of this version of the non-classical genetic algorithm. In a first approach the algorithm was applied to an example project with only 7 tasks and 3 resources with different composition with satisfactory results. The optimal resource allocation obeying the dependencies could be calculated by hand. 29 Resource Allocation with GAs In order to test the strength and capacity of the algorithm a much more complicated project definition was used. For this purpose a real data project that it will be denominated Sony was used, although the information related to the duration of tasks was incomplete. As a result, the information not provided was completed manually. The definition of the project is: • Number of tasks: 139 • Number of dependencies: 1257 • Resources: Table 5.5 Resource Team1 Team2 Team3 Members 3 4 5 Table 5.5: Resource composition of the Sony project. The Table 5.6 shows the results of the application of the algorithm with a different number of evaluations. The first column named as Run indicates the number of the execution. After that, the first group displays information of 1000 evaluations in the column FF Evaluation Series 1. The middle group represents 10000 evaluations in the column FF Evaluation Series 2, and the last group 100000 evaluations in the column FF Evaluation Series 3. As it can be observed from Table 5.6 there is not a significant decrease in the completion time of the project, even if the number of evaluations is increased from 10000 to 100000 times. This fact is clearly depicted in Figure 5.2 which is the graphic representation of Table 5.6. Series 1 corresponds to the results obtained after 1000 evaluations, series 2 after 10000, and series 3 after 100000. Although there is a slight decrease from series 1 to series 2, there is no appreciable difference between series 2 and series 3. This data is also corroborated by Table 5.7 which shows the average of the 30 executions of the algorithm. At this stage of the implementation, it remains unclear the main reason for this lack of success in achieving a positive result despite the considerable increase in the number of iterations and therefore evaluations of the algorithm. The intrinsic special features of this non-classical genetic algorithm may be partly responsible for these results. Nevertheless, it also can be attributed to the non-orthodox method of selection or the process of mutation of the individuals. 30 Resource Allocation with GAs Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 FF Evaluation Series 1 (1000 evaluations) 586.500 574.083 584.217 569.333 572.500 580.567 569.867 574.167 576.717 571.250 576.267 577.400 572.367 581.017 575.050 583.100 571.550 578.800 568.917 579.133 578.917 581.967 566.417 567.800 570.176 575.167 586.467 567.000 575.667 579.467 FF Evaluation Series 2 (10000 evaluations) 574.867 563.183 563.367 561.200 562.467 571.867 561.267 567.083 562.017 563.667 563.333 562.433 562.517 571.783 563.517 566.400 563.133 563.067 560.967 561.567 562.617 563.000 561.833 563.533 562.767 564.467 562.767 574.333 571.483 570.717 FF Evaluation Series 3 (100000 evaluations) 560.600 562.933 561.567 561.167 562.583 565.467 561.333 561.117 562.083 560.083 561.717 561.750 561.250 561.667 553.000 565.667 561.967 562.267 562.067 563.917 561.667 561.500 562.267 568.317 562.467 561.967 561.317 561.433 560.867 533.950 Table 5.6: Sony project results for Non-classical GA V1. Series 1 (1000 evaluations) 575.728 Series 2 (10000 evaluations) 564.907 Series 3 (100000 evaluations) 560.998 Table 5.7: Sony Average Results Non-Classical GA V1. 31 Resource Allocation with GAs Figure 5.2: Sony Results Non-Classical GA V1. Note: there is no significance in the order of elements in the x axis. 5.1.3 Second Version The following subsection details certain modifications done over the first version of the non-classical genetic algorithm due to the unsatisfactory results obtained in the first version. Apart from minor improvements and changes, the main difference regarding the previous version is the change in the method of selection of the individuals. 5.1.3.1 Method of Selection The new method of selection implemented is the binary tournament. This method selects two random individuals of the complete population and chooses the best in terms of fitness function. This operation is repeated in order to obtain the second individual necessary for the process of recombination. The reason for the election of this method of selection is its wide acceptance in the development of GAs. 5.1.3.2 Results The main analysis of the results in this stage of the development is trying to improve the non-classical GA. Therefore, this new version of the algorithm is applied to the same scenario the Sony project. See section 5.1.2.6 for the definition of the project. Thus, we are aiming for an improvement in the completion time obtained with the different resource allocations or candidate solutions. For this purpose, the same process of test is done with the following data collected in Table 5.8. 32 Resource Allocation with GAs Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 FF Evaluation Series 1 (1000 evaluations) 656.833 624.267 626.767 639.333 654.833 600.500 652.700 655.833 619.167 624.900 651.417 647.500 650.150 667.600 658.000 658.833 645.167 647.833 663.300 654.483 621.583 643.250 630.550 631.567 653.583 647.250 615.000 661.000 629.833 680.677 FF Evaluation Series 2 (10000 evaluations) 611.750 612.400 607.317 583.033 611.867 615.650 625.250 606.367 625.267 602.700 619.367 609.333 613.900 614.250 607.083 608.433 604.583 615.200 616.467 617.467 609.250 601.483 602.467 605.967 621.167 608.667 605.033 608.333 608.917 612.367 FF Evaluation Series 3 (100000 evaluations) 609.500 599.583 584.067 586.500 593.967 591.033 594.967 585.733 589.733 590.767 600.533 589.333 596.300 596.717 596.367 600.750 589.500 598.900 603.167 596.700 598.667 597.433 599.867 592.900 592.083 590.167 588.317 599.900 598.167 595.000 Table 5.8: Sony project results for Non-classical GA V2. This table gather the same kind of information as Table 5.6. The first column enumerates the execution of the algorithm, and the three following groups correspond to the application to 1000, 10000, and 100000 evaluations in this order. The first point observed in this second version of the algorithm is a similar pattern of behaviour. As the number of evaluation increases the solutions achieved improve regarding the fitness function. However, this new modification has resulted in no apparent improvement over the results of the previous version. On the contrary, the results are becoming worse. If it is compared the results of this new version with the previous one, it is obtained the following sequence of graphs: Figure 5.3, Figure 5.4, and Figure 5.5. 33 Resource Allocation with GAs Figure 5.3: Comparison Non-classical GA V1 and V2 with 1000 evaluations. Note: there is no significance in the order of elements in the x axis. Figure 5.4: Comparison Non-classical GA V1 and V2 with 10000 evaluations. Note: there is no significance in the order of elements in the x axis. 1st Version 2nd Version Series 1 (1000 evaluations) 575.728 643.790 Series 2 (10000 evaluations) 564.907 610.378 Series 3 (100000 evaluations) 560.998 594.887 Table 5.9: Average comparison Non-classical GA V1 and V2. 34 Resource Allocation with GAs Figure 5.5: Comparison Non-classical GA V1 and V2 with 100000 evaluations. Note: there is no significance in the order of elements in the x axis. Figure 5.6: Average comparison Non-classical GA V1 and V2. This fact can also be observed if it is compared the average Table 5.9 and Figure 5.6. In both Table 5.9 and Figure 5.6, series 1 correspond to the results over 1000 evaluations, series 2 over 10000 evaluations, and series 3 over 100000 evaluations.Although the results obtained without doubt show no improvement in this second version of the genetic algorithm, the Figure 5.6 displays a trend of convergence to similar results as the number of evaluations increases. The difference is greater in Series 1 than in Series 2, and the smallest difference correspond to Series 3. 35 Resource Allocation with GAs At this stage of the implementation and according to the adverse results produced not only with the first version of the Non-classical GA but also with the second version, it was decided to develop a classical implementation of a GA based on the same premises to compared the results in the application to the same scenario. 5.2 Classical GA This section details the features and results of the new algorithm developed following the classical premises of the genetic algorithms. Therefore, the principal difference regarding the algorithm of the section 5.1 of this paper is the fact that this new one creates different populations in each iteration. Hence, the individuals of the children population are based on the individuals of the parent population. The implementation of this classical genetic algorithm is based on the code of the nonclassical genetic algorithm with the necessary modifications. Furthermore, minor modifications were done to keep the performance of this new algorithm due to the new features. 5.2.1 5.2.1.1 First Version Features The main features in terms of structures and method of representation for the project definition and candidate solutions or individuals are exactly the same as the ones of the non-classical genetic algorithm detailed in the section 5.1.1 of this paper. 5.2.1.2 Individuals and Populations The methodology used for the individuals and populations in this new GA is totally different from the previous one and works on the premises of the GAs. For this purpose, the algorithm generates a new population of individuals in every iteration. Thus, this new process discards certain candidate solutions, but not all of them, and creates new ones. It keeps the best fitness function top individuals of the parent population to the children population. After that, it applies the process of selection, recombination and mutation to generate the new candidate solutions of the children population with the rest of the non-top individuals. 36 Resource Allocation with GAs 5.2.1.3 Evaluation The process of evaluation is the same as the one used in the previous algorithm already defined. Nevertheless, in this classical GA the number of evaluation performed in each iteration is equal to the size of the population minus the top individuals already evaluated in the previous iteration. 5.2.1.4 Method of Selection The method of selection of this new algorithm is again binary tournament. However, this algorithm chooses the candidate solutions within the parent population of that iteration. As a result, the diversity of the individuals to intervene in the process could be considerably smaller depending on the configuration. Moreover, a number of top individuals are always kept from one iteration to another. By top individuals it is understood those which have better fitness function. As a result, those top individuals are directly copied from the parent population to the children population. Therefore, the following steps of recombination, mutation and evaluation are only performed over the really new individuals. By new it is understood those which are truly generated in every iteration. 5.2.1.5 Recombination The process of recombination is based on the two point crossover operator explained in the section 5.1.2.4 of this paper. 5.2.1.6 Mutation The method elected to mutate the candidate solutions is exactly the same as the one used in the previous algorithm and detailed in the section 5.1.2.5 of this paper. 5.2.1.7 Results In this section the results obtained in the application of this first version of the classical genetic algorithm are analysed. For this aim, the solutions achieved are evaluated and compared to the ones produced by the two versions of the non-classical genetic algorithm. Table 5.11 shows the results in the application of this GA to the Sony project defined in the section 5.1.2.6 of this paper. 37 Resource Allocation with GAs • Number of TOP individuals: 10 • Number of tasks: 139 • Number of dependencies: 1257 • Resources: Table 5.10 Resource Team1 Team2 Team3 Members 3 4 5 Table 5.10: Resource composition of the Sony project. The first column Run is the number of the execution. The next one FF Evaluation symbolises the best completion time or fitness function obtained in that particular execution of the algorithm. In order to be able to compared both algorithm the number of iterations and the size of the population produce the same number of evaluations, 100000, as the Series 3 or last group of results of the previous algorithm. Figure 5.7: Comparison Non-classical GA V1 and V2 and Classical GA V1. 100000 evaluations. Note: there is no significance in the order of elements in the x axis. The comparison of 100000 evaluation between the first version of the Non-classical GA, the second version of the Non-classical GA, and the first version of the Classical GA has resulted in extremely surprising results. The Classical GA produces considerably worse results than the different version of the Non-classical GA. 38 Resource Allocation with GAs Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 FF Evaluation (100000 evaluations) 635.167 632.667 615.050 612.883 634.033 633.234 618.917 615.800 618.583 611.917 608.650 647.400 639.583 628.833 632.400 593.167 633.833 626.917 618.833 610.633 630.167 624.500 632.833 620.833 624.833 631.833 627.317 618.317 625.967 627.733 Table 5.11: Sony project results for Classical GA V1. 100000 evaluations. This fact is also corroborated by the average results illustrated in Table 5.12 and Figure 5.8. In addition, the average shows a worsening greater than the results generated by the second version of the non-classical genetic algorithm. Non-Classical GA V1 (100000 evaluations) 575.728 Non-Classical V2 (100000 evaluations) 594.887 Classical GA V1 (100000 evaluations) 624.428 Table 5.12: Average Non-classical GA V1 and V2 and Classical GA V1. 100000 evaluations. 39 Resource Allocation with GAs Figure 5.8: Comparison Average Non-classical GA V1 and V2 and Classical GA V1. 100000 evaluations. 5.2.2 Second Version In this second version of the Classical GA important changes were made in the pursuance of increasing the performance of the algorithm. The main reason for this action, was trying to aim to the next step of sensitivity analysis in the model of this paper, section 4. Furthermore, this new version of the algorithm also contains modifications in a new attempt to improve the accuracy of this classical GA. 5.2.2.1 Features This subsection describes the modifications in the method of representation for the main structures of the implementation regarding previous versions of the algorithm. In addition, it explains the changes that were made to improve the performance and which are directly related to the programming code. The new structure to represent the resource allocation was turned from a binary matrix to an array of integers. In this array each position represents a task and the integer represents the resource assigned to perform that task. An illustrative example it displayed in Table 5.13. T1 1 T2 3 T3 2 T4 1 Table 5.13: Resource allocation representation Classical GA V2. 40 Resource Allocation with GAs In so doing this, it was no longer necessary to process a matrix which involves a considerable number of positions. This fact was aggravated by the different loops implemented in the distinct parts of the algorithm that produced an exponential increase of the operations performed. This modification resulted in necessary changes in the stages which define the algorithm of the evolutionary testing that was developed. 5.2.2.2 Individuals and Populations The only significant change regarding the individuals and the populations is the data that they contain and the way they are fed. Now the population is formed by as many individuals as it is chosen and those individuals are formed by an array of integers of size equal to the number of tasks. Therefore, it was necessary to change the method to feed the initial population. In this new version every position or tasks of each individual was assigned randomly a number between 0 and the number of resources available. 5.2.2.3 Evaluation The method of evaluation and the fitness function were modified to be able to process the new structure that represents individuals. However, the methodology used to evaluate those individuals is exactly the same as the one explained in the section 4.1.2 of this paper. 5.2.2.4 Method of Selection The method of selection as well as the process of evaluation was modified only to be able to cope with this new method of representation of the individuals. Again, the binary tournament is used to select the individuals which are going to be recombined in the next step. 5.2.2.5 Recombination The process of recombination by two point cross-over operator was only modified in the strict necessary parts to allow the new structure of the individuals. 41 Resource Allocation with GAs 5.2.2.6 Mutation In the process of mutation important changes were made apart for those essential for the new method of representation of the resource allocation. A new methodology of mutating the individuals was implemented since the previous one was not completely appropriate for a GA. This new approach established a probability of mutation based on the number of tasks or size of the individual of each population. This probability is equal to 1 divided by the number of tasks. The process of mutation consist in a loop which traverse the all the positions of the candidate solution and generated a random number which is divided by RAND MAX. If the number generated is smaller than the probability of mutation, the position is mutated to a new resource. This action is done by generating randomly a new resource between 1 and the total number of them. 5.2.2.7 Results Figure 5.9: Comparison Average Non-classical GA V1 and V2 and Classical GA V1 and V2. 100000 evaluations. In order to be able to compare the results of this new version of the algorithm, it was used the same project as the previous ones. The Table 5.14 shows the results in the application of this GA to the Sony project defined in the section 5.1.2.6 of this paper. The number of top individuals selected for this test was 10 the same as the one done in the previous version of the algorithm in the section 5.2.1.7. It can be observed in table 5.14 that this new version of the algorithm improves considerably the results obtained. In general terms this second version the Classical GA is able 42 Resource Allocation with GAs Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 FF Evaluation (100000 evaluations) 563,467 573,233 564,667 553,467 563,917 566,667 564,283 562,500 575,500 562,867 566,917 566,767 567,700 569,050 576,667 566,550 572,250 567,667 560,983 565,667 575,767 564,083 565,333 562,333 570,500 565,000 565,417 563,750 571,000 563,633 Table 5.14: Sony project results for Classical GA V2. 100000 evaluations. Non-Classical GA V1 (100000 evaluations) 575.728 Non-Classical V2 (100000 evaluations) 594.887 Classical GA V1 (100000 evaluations) 624.428 Classical GA V2 (100000 evaluations) 566.587 Table 5.15: Average Non-classical GA V1 and V2 and Classical GA V1 and V2. 100000 evaluations. 43 Resource Allocation with GAs to produce better solutions considering the average of 30 runs. This fact is illustrated in Table 5.15 and Figure 5.9. The most reasonable explanation for this improvement is the changes made in the process of mutation and the rest of the changes already mentioned in the pursuance of a more accurate GA. 5.2.3 Third Version The third version of the Classical GA contains two relevant modifications. The first one is related to the method of representation of the TPG which has direct impact in the performance of the evaluation process. And the other one entails changes in the generation of the parent population of each iteration, in the pursuance of better results in the evaluation of the fitness function. Both modifications are detailed in the following sections. 5.2.4 Features The idea of changing the method of representation for the TPG lies in the study done over the performance of the code. In that analysis it was found that the majority of the time spent running the genetic algorithm was consume by the process of evaluation of the fitness function and particularly in the section of checking the dependencies. The previous structure and methodology consisted in a binary matrix of size number of tasks square, where the index of the matrix represented the number of the tasks which are dependents, as it was illustrated in Table 5.1 and Figure 5.1. In the fitness function evaluation the complete matrix was traversed to check all the possible dependencies. Nevertheless, based on the same structure a new method of storing the data was introduced in this version. Instead of a binary matrix, the first position of every column now contains the number of dependencies of the task associated to the index of that column, and in the following rows the index of the task involved in the dependency. Based on the same example Figure 5.1, the new representation is displayed in Table 5.16. T1 0 0 0 0 T2 1 1 0 0 T3 1 1 0 0 T4 2 2 3 0 Table 5.16: TPG Representation Second Version. 44 Resource Allocation with GAs 5.2.5 Evaluation As a result of the modification in the method of representing the TPG the new version of the GA increased its performance considerably. The main reason for this fact is that now every time that was necessary to check the dependencies of the project definition the number of position consulted is exactly the number of dependencies plus the number of tasks, which represents the positions where it is stored the number of dependencies for every task. Whereas, in the previous version it was necessary to evaluated the whole matrix in order to obtain the same information. 5.2.5.1 New Parent population In this new version of the GA the parent population generated in every iteration of the algorithm is composed in the following way. First using the method of selection of binary tournament and the two-point crossover operator individuals of the old parent population are selected to create the children population. After that, those individuals are mutated and in this point there are two complete populations: old parent population and children population. The top 10 individuals, those which have better fitness function evaluation are copied directly to the new parent population. Then binary tournament is again performed between the non-top ten individuals of the old parent population and the non-top ten individuals of the children population until the new parent population is completed. 5.2.5.2 Results The scenario used in this version to analyse the results is again the Sony project. This allowed to compared the results and decide which of all the algorithms and its different versions is better obtaining a shortest completion time. Table 5.17 shows very similar results regarding the completion times obtained in the second version of the Classical GA and showed in Table 5.14. In order to have a clear idea it can be observed in Table 5 that the average results in this version is better than in the rest tested so far. This new version of the Classical GA as all the table and figures of this sections illustrated achieved the best results. 45 Resource Allocation with GAs Run 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 FF Evaluation (100000 evaluations) 563.467 563.333 565.083 558.817 564.417 567.067 553.117 552.750 562.867 566.583 562.417 563.267 565.733 560.217 563.417 563.333 572.250 563.167 557.000 567.667 563.267 562.933 563.533 565.267 560.750 562.667 563.000 563.000 563.267 558.467 Table 5.17: Sony project results for Classical GA V3. 100000 evaluations. Non-Classical GA V1 (100000 evaluations) 575.728 Non-Classical V2 (100000 evaluations) 594.887 Classical GA V1 (100000 evaluations) 624.428 Classical GA V2 (100000 evaluations) 566.587 Classical GA V3 (100000 evaluations) 562.737 Table 5.18: Average Non-classical GA V1 and V2 and Classical GA V1, V2, and V3. 100000 Evaluations 46 Resource Allocation with GAs Figure 5.10: Comparison Average Non-classical GA V1 and V2 and Classical GA V1, V2, and V3. 100000 evaluations. 5.3 Conclusion The results collected in the development of this chapter can be essentially divided in to two main groups. First, the data obtained in the application of the Non-classical GA and second, the information gathered in the application of the Classical GA. The initial results of the first version of the Non-classical GA were better than the ones obtained in the second version according to all the tables and figures of the section 5 of this paper. Therefore, it did not make sense to keep developing this non-classical algorithm. In spite of the fact that the data collected in the first version of the Classical GA was notably discouraging, the changes made in the third version allowed to generate the best solutions of all the algorithms in all the different versions. This certainty is clearly reflected in the Figure 5.10. In conclusion, despite the last version of the classical GA shows encouraging results it has to be taken into consideration that in none of the different GAs and scenarios it was possible to obtain a unique best solution to the different problems or projects posed. Hence, it was decided to use the possibility of establishing an average for the benchmark of the optimal resource allocation. This corresponds to the third stage of the model produced in the section 4 of this paper. 47 Chapter 6 SensitivityAnalysis This chapter of the paper explains and details the study of sensitivity analysis performed in the process of breaking dependencies. Therefore, it describes the method carried out to break the dependencies, which are defined by the TPG, in the definition of a project. For this purpose, four projects based on real data were used as a source of information. Nevertheless, the data provided in those project in terms of duration and dependencies was a adapted by a process described in the section 6.1 of this paper. The process of sensitivity analysis performed basically consisted in removing one by one all the possible dependencies and run the GA developed in the section 5.2.3 of this paper. After that, the results of both executions are compared in order to try to establish a list of the most sensitive dependencies as well as other information which could be relevant in the management of the scheduling of the project. A case in point is how affect the variation in the availability of resources at the same time that dependencies are broken. This could lead to corroborate a list of most sensitive dependencies in a project definition without being affected by the other parameters considered. In addition, it might give access to information of the intrinsic behaviour of the project such specific impact of one particular group of dependencies over the completion time of the entire project. The implementation developed basically traverse the TPG suppressing each dependency and then run thirty times the third version of the GA detailed in the section 5.2.3 of this paper. The average of these thirty executions is taken into consideration as the completion time for the project definition without that particular dependency. The results obtained are analysed by comparing them with the average of thirty executions of the same GA using the original TPG with all the dependencies. 48 Sensitivity Analysis 6.1 Methodology to adapt data In order to adapt the original data provided in MS Project format, to be clearly represented in the structures of the implementation developed, it was necessary to generate a methodology of transformation. This process of transforming the source data can be divided in two main parts. The first part consisted in removing the ids which represented a set of tasks and not a task. The second part was focused on reducing the number of dependencies by removing those which are redundant. In the first stage of this approach of removing dependencies all the ids which identify a group of tasks are eliminated. This operation produces a new list of ids for all the tasks which really have duration, but at the same time involves the redefinition of the dependencies. A complete process example is explained in the tables 6.1, 6.2, 6.3, and 6.4. Where Table 6.1, shows the original data. As it can be appreciated there is a column named Task (Id) which contains the id of the tasks that compounds the group denominated Group (Id). As a result, the group need to be removed and the id of the task are restated from 1, naming only real tasks that are going to be assigned to resources. At the same time, if following tasks are dependant in this group id, which is composed by various task ids, the new dependency is represented by the id of the tasks that form that group. This process is reflected in Table 6.2. The final definition for a project will be the composed by the columns which store the new id and new dependencies as illustrated in Table 6.3. It has to be taken into consideration that this process was performed trough different levels since a group can be constituted by subgroups and those by subsubgroups until tasks are reached. Group (Id) 1 4 - Task (Id) 2 3 5 Dependencies 2 1 Table 6.1: Original project definition. The second stage of this process of adapting the data resides in regrouping and removing non-necessary dependencies. This procedure has to be performed in combination with the first stage in certain situations when project definition is adapted. By non-necessary dependencies it is considered those which are already defined indirectly by other. An example is displayed in the table 6.3, where the task 3 due to this process of removing group depends now on task 1 and task 2. However the dependency between task 3 and 49 Sensitivity Analysis Old Id 1 2 3 4 5 New Id 1 2 3 Composition (Old Id) 2,3 5 - Old dependencies 2 1 New dependencies 1 1,2 Table 6.2: Transformation project definition table. Id 1 2 3 Dependencies 1 1,2 Table 6.3: Adapted project definition. task 1 is redundant since task 3 depends on task 2 which in turn depends on task 1. Thus, the final project definition would have only two dependencies as shown in Table 6.4. Id 1 2 3 Dependencies 1 2 Table 6.4: Final project definition. It is considerably important to highlight that these process of adapting the data and the following process of removing indiscriminately dependencies could lead to work in non-logical scenarios or even non-possible solutions. For example in the previous example illustrated which final project definition was Table 6.4, if the process of removing dependencies eliminates the dependency between task 3 and task 2 it also entails loosing the dependency between the task 3 and the task 1. 6.2 Project defintions This section of the paper describes the specification of the four projects that are going to be used in the process of sensitivity analysis. The source of this data is four real projects and therefore, the information used in these implementations gain special importance, since utilising real data add considerable value to the results collected and analysed. A brief explanation of the origin of the projects as well as a concise description of them is given in order to understand the context of the subjects used in this process of testing the impact of removing dependencies for the sensitivity analysis. Moreover, for each 50 Sensitivity Analysis of these projects it is going to be specified the number of tasks and the number of dependencies before and after the process of adaptation. Project Project Project Project 1 2 3 4 Before adaptation Tasks Dependencies 148 81 140 103 95 66 129 94 After adaptation Tasks Dependencies 107 83 106 103 78 168 92 93 Table 6.5: Projects definition. 6.2.1 Project 1: CutOver There was not possible to provide a description of the project. 6.2.2 Project 2: Database The project purpose was database upgrade procedure. It consisted in a migration from the Oracle version 9g to 10g. In addition, the upgrade affected 70% of the internal applications, since a considerable number of them relied on Oracle Forms. There were different layers of the organisation involved including DBAs, BSAs, developers and users. Furthermore, the project also included the training of the stuff for the new features of the system. 6.2.3 Project 3: QuotesToOrder The aim of this project was an enhancement of the internal structure of the company. The main parts affected on many different levels were the website, most internal applications and their integration. In addition, as part of the project there were tasks involving the training of the future users of the new parts of the system. 6.2.4 Project 4: SmartPrice This project consisted in a supply chain enhancement of medium size affecting mostly the website as well as a few internal applications. 51 Sensitivity Analysis 6.3 Sensitivity Analysis Methodology The procedure used in this process is known as sampling-based. This approach lies on executing the model repeatedly with all the possible combinations. In this case, the main aim is removing in turns all the dependencies which define the TPG of the project. The model developed considers three different parameters: tasks, dependencies, and resources. The first one, duration of the tasks is fixed and we study the impact of modifying the second one, dependencies. Therefore, there is a third input factor, resources, which feeds the model that can be altered to analyse whether by varying it the behaviour of the model describe a certain pattern. In this sense, with this practice the observation of the results collected can add more reliability of the sensitivity of particular dependencies. In so doing that, it was evaluated if the list of most relevant tasks were always the same with a different resource allocation. For all the different experiments it was decided to run the GA in a loop of 1000 iterations generating a population of 100 individuals in each of the iterations. In addition, the number of top individuals kept for not loosing best fitness function evaluations were 10% of the size of the population. The procedure keep a simple but concise methodology detailed following: 1. Calculate the average of running 30 times the original TPG. 2. Establish this average as benchmark to compare and therefore, it was considered the 100 % completion time. 3. In turns, remove one by one all the dependencies which define a project and calculate the average of running 30 times the algorithm with the TPG without that particular dependency. 4. Compare the results regarding the benchmark previously established. 6.4 Sensitivity Analysis Results This section displays the results obtained in the process of removing dependencies in the four real projects detailed in the section 6.2 of this paper. The main purpose of this action is to perform an analysis to assess the impact of the most sensitive task in terms of reducing the overall completion time of the project. 52 Sensitivity Analysis For each of the projects previously detailed a series of four different tests were performed. The difference between the tests lies on the composition and number of resources available to be assigned to tasks. From now and on in this paper the set of experiments run will be referred as Test 1, Test 2, Test 3, and Test 4. The composition and number of those tests are as follows: • Test 1: 3 teams and 12 people – Team 1: 3 people – Team 2: 4 people – Team 3: 5 people • Test 2: 10 teams and 10 people (1 person per team) • Test 3: 6 teams and 6 people (1 person per team) • Test 4: 3 teams and 3 people (1person per team) In each of this test one graph is shown. The information represented corresponds to the normalisation of the average completion time of running 30 times the Classical GA in its third version, section 5.2.3. The graph illustrates the normalisation of the data collected. For this purpose, the benchmark or original TPG of the first graph is establishes as 100% of the completion time. This graph represents the difference of the completion time of removing in turns all the dependencies regarding the benchmark. The results obtained are ordered in increasing order of completion time. 6.4.1 Results Project 1 The configuarion of the original TPG for Project 1 is: • Number of tasks: 107 • Number of dependencies: 83 The results obtained for Project 1 and the set of test are displayed in the following figures: • Test 1 (3 teams with 3, 4 and 5 people): Figure 6.1 53 Sensitivity Analysis • Test 2 (10 teams with 1 person each): Figure 6.2 • Test 3 (6 teams with 1 person each): Figure 6.3 • Test 4 (3 teams with 1 person each): Figure 6.4 Figure 6.1: Project 1. Normalisation Test 1 (3 teams with 3, 4 and 5 people). Figure 6.2: Project 1. Normalisation Test 2 (10 teams with 1 person each). As it can be appreciated it in the graphs of the four different tests, although the model was able to decrease the completion time by removing certain dependencies it also achieved an increase. In spite of the fact that theoretically this solution is impossible, it is not unreasonable within the context of our development. Logically if one dependency 54 Sensitivity Analysis Figure 6.3: Project 1. Normalisation Test 3 (6 teams with 1 person each). Figure 6.4: Project 1. Normalisation Test 4 (3 teams with 1 person each). is removed the overall completion time should not be greater than the one obtained after processing the original TPG. Nevertheless, the genetic algorithm developed and used for this implementation is not always able to obtain a unique optimal solution for the resource allocation. As a result, this GA produces near optimal solutions in a process of approximation. Therefore, it is possible that removing one particular dependency there is not improvement at all in the completion time for the project analysed. This increase can be attributed to the fact that removing one particular dependency modifies the landscape that the GA is working on. Furthermore, withdrawing certain dependencies could result in the algorithm converging to a local optima, even getting stuck to it. Since 55 Sensitivity Analysis with the GA developed it is not possible to ensure a global optima solution in all the scenarios. Thus, that changing the landscape can lead to more difficulty at searching and finding better solutions rather than the stochastic variance of the GA. It is remarkable to mention that for Test 1 illustrated in Figure 6.1 nearly half of the total dependencies produced an increase in the completion time. Similar behaviour was obtained for Test 3 depicted in Figure 6.3. The results got worse running the experiments for Test 4, Figure 6.4. This situation was aggravated when the results for the Test 2 were collected. As it can be observed in Figure 6.2 approximately three quarters of the dependencies removed incurred in an increment. Hence, from this project no clear assumption can be made regarding the impact of playing with the resources available, which is the third parameter that defines the scenario for the model. As a last observation in the results analysed for the four different tests in this project, it can be stated that the decrease in the completion time in the best situation is always greater than the increase in the worst case. In all the tests performed the greater decrease varies from slightly more than 4% to nearly 5%. Whereas, in all the tests completed the greater increase never exceed the 3%, Figure 6.1, Figure 6.2, Figure 6.3, and Figure 6.4. It was decided to perform a series of statistical analysis in order to clarify whether these results can be considered reliable due to the facts described at the beginning of this section and taking into consideration the similarity in the results collected. All the techniques used, a briefly description of them and the conclusion deduced from the results are fully detailed and explained in the section 6.5 of this paper. 6.4.2 Results Project 2 The configuarion of the original TPG for Project 2 is: • Number of tasks: 92 • Number of dependencies: 93 The results obtained for Project 2 and the set of test are displayed in the following figures: • Test 1 (3 teams with 3, 4 and 5 people): Figure 6.5 • Test 2 (10 teams with 1 person each): Figure 6.6 • Test 3 (6 teams with 1 person each): Figure 6.7 56 Sensitivity Analysis Figure 6.5: Project 2. Normalisation Test 1 (3 teams with 3, 4 and 5 people). Figure 6.6: Project 2. Normalisation Test 2 (10 teams with 1 person each). • Test 4 (3 teams with 1 person each): Figure 6.8 The results obtained in this new project revealed interesting information. Performing Test 1 and Test 4 we can appreciated as well as in Project 1 that there is an increase as well as an decrease. However, for Test 2 and Test 3 only there is only reduction in the completion time when the dependencies are removed. The availability of resources may be partly responsible for this behaviour. Since both tests allows the model to dispose of a large number of teams 6 and 10 respectively. As a result, it might be possible that there is always one team available to perform a task. This phenomenon can be 57 Sensitivity Analysis Figure 6.7: Project 2. Normalisation Test 3 (6 teams with 1 person each). Figure 6.8: Project 2. Normalisation Test 4 (3 teams with 1 person each). corroborated by the fact that in both cases Test 2 and Test 3 the completion time is exactly the same 201 days. Thus, the availability of teams could have an essential effect on the completion time. The intrinsic nature of the project definition and the TPG can be an important cause in the ability of the GA to obtain a global optima solution. It seems that for this particular project the best solution processing the original TPG is 201 with this particular availability of resources. As a result, during the process of removing dependencies it is always obtained this value or a smaller one. 58 Sensitivity Analysis In addition, there is one a new conduct in the completion times for the results of this project in Test 1 and Test 4 which are illustrated by Figure 6.5, and Figure 6.8. In spite of the fact that there is a similar percentage of increase in the completion time, regarding the results of Project 1, when certain dependencies are omitted from the TPG, the decrease in notably superior. As shows Figure 6.5 the decrease can achieve nearly 20% of reduction, whereas the increase never reaches 5%. In the case of the Test 4 the reduction of the completion time exceed 20% and the increase again never reaches 5%, 6.8. The situation previously described probably indicates that those dependencies which produce a minor increase do not have real impact of the completion time of the project, whilst the ones which produce that significant improvement could be considered extremely sensitive. To ensure that it was compared if all the top ten dependencies where are able to reduce the completion time in the greatest proportion were the same for all the tests. In order to establish that comparison those top ten dependencies were ordered in increasing level of completion time and represented in Table 6.6. The cells contain the index of the tasks which are dependant. The table displays a considerable similarity with slight differences between the tests. For example the dependency between the task 96 and the task 97 appears as Top 1 dependency in the tests 2, 3, and 4 and as Top 3 in the test 1. In the case of Test 2 and Test 3 the top 10 most sensitive dependencies are exactly the same and its importance measured by the impact that they produce is also the same. Every dependency obtains the same order in the top 10 in both tests as it is shown in the table. Therefore, the behaviour that those tests produced in the model seems to have the same pattern. They have the same global optima solution in 201 days, they never produced increase by removing dependencies, they achieve similar reduction in the completion time, and they have exactly the same top 10 most sensitive dependencies. In the rest of the cases there is a significant number of coincides in the most sensitive tasks as well as the importance of their impacts with a little variation in the results for Test 1. Furthermore, it is plausible to believe that the differences are affected by the composition of the teams or configuration of the third parameter of the model. Nevertheless, this could emphasize the consideration that even modifying the resource allocation the importance of the certain set of dependencies remains. Test Test Test Test 1 2 3 4 Top 1 90-91 97-98 97-98 97-98 Top 2 98-99 98-99 98-99 89-90 Top 3 97-98 91-97 91-97 90-91 Top 4 99-100 90-91 90-91 91-97 Top 5 102-103 89-90 89-90 98-99 Top 6 89-90 88-90 88-90 99-100 Top 7 100-101 99-100 99-100 88-89 Top 8 101-102 87-88 87-88 87-88 Top 9 91-97 86-87 86-87 101-102 Top 10 85-86 85-86 85-86 85-86 Table 6.6: Top 10 dependencies Project 2. The content represents the indexes between the two tasks which compose the dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). 59 Sensitivity Analysis This fact could lead to the idea of having accomplished certain level of success, positive and encouraging results measuring the sensitivity of the dependencies. It is not unreasonable to think that valid and demonstrable solutions would be achievable if it is possible to developed GA with enough power and capacity to cover the complete landscape of the scenarios of a project definition. Nonetheless, there is also a possible explanation about the enormous decrease in the completion. As it can be detected in Figure 6.5, Figure 6.6, Figure 6.7, 6.8 the reduction reach almost 20%, 50%, 40%, and 20% respectively. It might be possible that breaking these particular dependencies other important are also lost due to the process of adapting the data and the issue described in the section 6.1 of this paper. Nevertheless, this fact does not necessarily involve a lack of validity of the results obtained. In the case that a set dependencies was removed and lost during the methodology it could entail that a dependency which is referred to a group of tasks is sensitive. Hence, it would require to study more deeply the impact of that group of tasks. Consequently, it still indicates that a dependency or set of dependencies associated to a task or a particular group of tasks might reduce the overall completion time of the project if they are removed. This statement is the main focus of this research, since every project has a complete different nature. Then the project manager would be in a position to decide whether that dependency or set of dependencies can be broken in the real world. Therefore, the supervision of the project manager or the person responsible of the management of the project is absolutely necessary to apply the data and results collected in the real development of the project. Nevertheless, it has to be taken into consideration that the rest of the factors which classically define the PSP, detailed in the section 2 of this paper and that they have been discarded, could modify the results. Yet, as it has been mentioned previously and according to the results obtained overall in Test 2 and Test 3 despite varying the third parameter of resources the top 10 most sensitive tasks remains exactly the same. These promising results encourage thinking about positive future if a deeper research is performed. As a result, it would be extremely interesting to develop a model able to cope with all these parameters such as skills, cost, effort, and so on. 6.4.3 Results Project 3 The configuarion of the original TPG for Project 3 is: • Number of tasks: 78 • Number of dependencies: 168 60 Sensitivity Analysis The results obtained for Project 3 and the set of test are displayed in the following figures: • Test 1 (3 teams with 3, 4 and 5 people): Figure 6.9 • Test 2 (10 teams with 1 person each): Figure 6.10 • Test 3 (6 teams with 1 person each): Figure 6.11 • Test 4 (3 teams with 1 person each): Figure 6.12 Figure 6.9: Project 3. Normalisation Test 1 (3 teams with 3, 4 and 5 people). Figure 6.10: Project 3. Normalisation Test 2 (10 teams with 1 person each). 61 Sensitivity Analysis Figure 6.11: Project 3. Normalisation Test 3 (6 teams with 1 person each). Figure 6.12: Project 3. Normalisation Test 4 (3 teams with 1 person each). The results collected and represented in the Figure 6.9, and Figure 6.12 show a pattern of behaviour which could be situated between the results of Project 1 and Project 2. There were increases in all the tests performed to a greater or lesser extent. Yet, as it happened in previous results the increments in the completion time are always considerably smaller than the reductions produced by breaking certain dependencies. In this particular project the increases never reach 6% of the original completion. Although, this number entails an increase of almost 1% regarding the two previous projects as the previous test it might be produced by the nature of the GA. In spite of the fact that 62 Sensitivity Analysis this statement cannot be undoubtedly validated, it might be possible to considere that this effect is produced as a result of the limited capacity of the GA already mentioned in the section 6.4.1. As it is mentioned in that section, there is likely to be a problem with respect to the average produced after thirty runs of the algorithm. The availability of resources one more time seems to be partly responsible in those increments. When the availability of the resources is the greatest, running Test 2 Figure 6.10 there is null increase for every dependency broken. And the greater number of increments corresponds to the tests with smaller availability of resources Figure 6.9 and Figure 6.12. It was constructed again the table with the top 10 more sensitive dependencies in order to verify if the sensitivity of the dependencies were affected by the composition of the teams. This fact could decreases the validity of the results obtained in Project 2. Despite the results obtained and illustrated in Table 6.7 showed more variance in terms of which dependencies are more sensitive in every test, it still remains relative similar between the tests. For instance, the dependency between the task 5 and the task 6 remains as top 1 dependency in Test 2, Test 3, and Test 4. Another example is the dependency between the task 6 and the task 8 that appears in all the test performed. Test Test Test Test 1 2 3 4 Top 1 8-9 5-6 5-6 5-6 Top 2 66-67 3-5 6-7 6-8 Top 3 15-56 2-3 73-74 8-65 Top 4 74-75 1-2 3-5 8-9 Top 5 24-54 6-8 59-60 6-7 Top 6 5-6 8-9 8-9 74-75 Top 7 9-12 65-71 6-8 75-76 Top 8 6-8 9-65 48-66 7-10 Top 9 75-76 Top 10 17-56 34-35 10-12 36-37 24-28 Table 6.7: Top 10 dependencies Project 3. The content represents the indexes between the two tasks which compose the dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). The second test only displays 8 dependencies because there was not found other dependencies able to reduce the completion time. Therefore, the variability in the third factor that defines the model could have a greater importance over the sensitivity of the dependencies than it was mentioned in the results remarked in Project 2, section 6.4.2. Yet, this fact does not entail that it is not possible to present a list of sensitive dependencies that by removing could produce a considerable reduction in the overall completion time of the project. Moreover, it would be possible to feed the model with the desired configuration for a particular project to evaluate the dependencies and their impact. In this project the decrease and increase indicated in Figure 6.9 and Figure 6.12 for Test 1 and Test 4 seem to show almost identical results regarding the original overall completion time of the project establish as 100% benchmark. Hence, it can be understood that 63 Sensitivity Analysis under specific circumstances the resources are always busy and removing one particular dependency make no difference at all. Yet, this phenomenon does not involve that the impact of that particular dependency could not lead to a significant reduction in the completion time with other resource configuration. In other words, the completion time of a project is limited by the resources available but the importance in terms of sensitivity of one or more dependencies could be used in the adequate scenarios. The reduction in the completion time achieved in Test 2 and Test 3 depicted in Figure 6.10 and Figure 6.11 for the first dependency broken obtained approximately 31% and 20%. It can be appreciated that the same dependency produced this effect if this information is combined with the one provided in Table 6.7. In the case of the Test 4 illustrated in Figure 6.12 the percentage of decrease for this particular dependency exceed 4%. Nevertheless, the same graph displays a considerable number of increments in the completion time. In conclusion, even though the results analysed do not show as strong evidences as the ones evaluated in the section 6.4.2 of this paper for Project 2, it is still possible to recognise certain similarities in the patterns of behaviour. Moreover, when the data do not corroborate thoroughly the impact of the most sensitive dependencies there still is enough reasonable proof to justify the conduct of the response of the model. Thus, it rational and sensible to believe that the implementation developed is reviling interesting results in the direction of the focus of this research. 6.4.4 Results Project 4 The configuarion of the original TPG for Project 4 is: • Number of tasks: 92 • Number of dependencies: 93 The results obtained for Project 4 and the set of test are displayed in the following figures: • Test 1 (3 teams with 3, 4 and 5 people): Figure 6.13 • Test 2 (10 teams with 1 person each): Figure 6.14 • Test 3 (6 teams with 1 person each): Figure 6.15 • Test 4 (3 teams with 1 person each): Figure 6.16 64 Sensitivity Analysis Figure 6.13: Project 4. Normalisation Test 1 (3 teams with 3, 4 and 5 people). Figure 6.14: Project 4. Normalisation Test 2 (10 teams with 1 person each). The results obtained in all the tests performed over the Project 4 reinforce the ability of the model to suggest and probably identify the most sensitive dependencies. In addition, in this case the solutions which produce an increase over the original completion time are considerably few. The reductions in the completion time for all the tests always exceed 10% and in the case of the Test 2 and Test 3 the 20%. This statement is illustrated in Figure 6.13, Figure 6.14, Figure 6.15, and Figure 6.16. Therefore, the model was able to decrease significantly the establish benchmark in different percentage with all the different configurations of 65 Sensitivity Analysis Figure 6.15: Project 4. Normalisation Test 3 (6 teams with 1 person each). Figure 6.16: Project 4. Normalisation Test 4 (3 teams with 1 person each). the resources. This phenomenon is only produced in this scenario and would represent the ideal behaviour that this research is trying to identify. As a result, it is reasonable to think that a deeper analysis of the parameters that compose the model could lead to reveal not only the increments produce in certain dependencies, but also the validity of model and its focus. It is interesting to emphasize that if the results are observed in a particular order by looking at the results displayed in performing Test 2, Test3, Test4, and Test 1 seems to appear a unique pattern of behaviour. As the number of teams decreases the reduction 66 Sensitivity Analysis times tend to level off to the benchmark. Furthermore, there seems to be specific set of dependencies which produce exactly the same reduction in the completion time, Figure 6.14 and Figure 6.15. This situation was not encountered in the previous projects and might be caused by the critical position of certain tasks and its surrounding dependencies. Hence, removing one of those dependencies could have a direct effect on the sensitivity of that task over the completion time of the project. The top 10 dependencies of the tests were studied in order to clarify partly the results previously mentioned. For this purpose the top 10 dependencies were illustrated in Table 6.8. The conclusion achieved is that despite the position varies the most sensitive dependencies for all the tests are almost exactly the same. Thus, it is sensible to believe that the configuration of the resources does not have direct impact over the sensitivity of the dependencies, yet it can affect the reduction in the completion time of the project removing dependencies. This assumption is absolutely logical since the lack of resources as unconditional factor of the PSP can affect the completion time of the project in the original TPG. Test Test Test Test 1 2 3 4 Top 1 52-53 20-21 49-50 52-53 Top 2 51-52 21-46 50-51 51-52 Top 3 21-46 46-48 51-52 50-51 Top 4 50-51 48-49 52-53 21-46 Top 5 20-21 49-50 20-21 48-49 Top 6 48-49 50-51 48-49 20-21 Top 7 46-48 51-52 46-48 49-50 Top 8 49-50 52-53 21-46 46-48 Top 9 55-56 10-20 10-20 10-20 Top 10 9-10 9-10 9-10 55-56 Table 6.8: Top 10 dependencies Project 4. The content represents the indexes between the two tasks which compose the dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). Nevertheless, it is also true in this project it was obtained the greatest increments of all the tests. As it can be appreciated in Figure 6.13 the majority of the dependencies removes do not produce an increase but the last three ones resulted in an increment of 6%, 8% and 14%. According to the rest of the outputs and the limitations of the model implemented and previously detailed this phenomenon could be attributed to reasonable explanations. To summarize, in spite of the fact of there were found significant increments, they can be attributed as in previous cases to the circumscription of the GA and the model developed. On the other hand, the positive reductions accomplished in all the tests as well as the high level of parallelism between the top 10 dependencies in them demonstrated relatively positive attainment in the direction of this research. 67 Sensitivity Analysis 6.5 Statistical Analysis The results obtained in part of the test in the sensitivity analysis as well as the limitations of the implementation could express certain reservations about the validity of the model developed. The responsibility for the increments obtained in the previous section were partly attributed to the possibility not having real difference between the distributions of the thirty executions and the same could be truth for the reductions in the completion. As a result, it was decided to use statistics to add more reliability to the results collected. For this purpose this section evaluates the distribution of the thirty runs of the top 10 dependencies of each test regarding the distribution of the thirty runs of the benchmark. In so doing that, it would be adding certain level of validity by comparing whether the distribution of both cases is genuinely different or not. The idea behind this procedure is to clarify those situations where the reduction of the completion time is not enough clear because there are also increments in similar proportion but where the tables of the most sensitive dependencies have shown similarities. Hence, the statistical analysis tries to validate the specific situations where the decrease in the completion time could have been attributed to random results produce by the model in that particular set of execution for the genetic algorithm. This fact is also known as the stochastic variance of an algorithm. Nevertheless, the repetition of particular dependencies in the top ten list produced by the different tests for a concrete project might reveal relative level of sensitivity in those dependencies. The techniques used in these tests as well as the tools employed are detailed in the following sections. 6.5.1 Statistical Techniques and Methodology The rank sum test is the main technique used in this paper to compared the results of the top 10 dependencies with the ones obtained using the original TPG. The rank sum test is a non-parametric technique for assessing whether two sets of observations have its origin in the same distribution. This test compares the two samples in order to test the hypothesis that the sets of observations are not different. Every set corresponds to the group of thirty executions of the algorithm. In this case, the difference between the sets is the effect of removing a dependency, while the variance is the due to the stochastic nature of the GA. 68 Sensitivity Analysis The methodology to interpret the results of the rank sum test will be based on the level of certainty of the output parameters produce. Rank sum test outputs a p-value that indicates how likely the null hypothesis is to be true. The null hypothesis is that the set of observations are from the same probability distribution. Therefore, a smaller p-value indicates a larger degree of confidence that the hypothesis is incorrect. It is generally believed that a threshold of 5% can be chosen to indicate a statistically significant difference between sets of data, so that a p-value less than 5%. If the p-value is less than 5% for the rank sum test, then it can be assumed that removing the dependency makes a statistically significant difference to the completion time found by the GA. In other words, it is possible to state that the data before and after removing one dependency are positively different with a 95% level of confidence. If the p-value is still under 10% it could still claimed a moderately significant result. 6.5.2 Statistical Tools R is the statistical software package that was used to perform all the statistical test of this paper. This software for statistical computing and graphics is free and widely accepted by the scientific community as a powerful tool to perform a broad range of classical statistical test, linear and non-linear modelling, time-series analysis, and so on. The rank sum test within R package is executed under the command wilcox.text{stats}. 6.5.3 Statistical Results The statistical results of this research are analysed and divided in different sections according to the different projects evaluated in the sensitivity analysis. In addition, in every project the results are processed considering the information collected through the four test performed. The statistical results are represented using a table similar to the one used to indicated the top ten most sensitive dependencies. Nevertheless, this time the table contains the p-value returned by R after performing the rank sum test. In so doing that, it is possible to add enough level of confidence in the result if there is a particular dependency which appears several times in different tests but which percentage of reduction is relatively small. The data necessary to perform the statistical analysis for each project is completely detailed in tables annexed in the section Appendix B of this paper. Furthermore, these tables provide important information to understand the output of the rank sum test. 69 Sensitivity Analysis Additional presumptions regarding the behaviour of the model can be made from the information revealed in these tables. 6.5.3.1 Statistical Results Project 1 This project obtained the worst and more complicated results in the sensitivity analysis of breaking dependencies according to the section 6.4.1.By worst and complicated results it is understood the ones which produced the smallest reduction in the completion time as well as increase in more cases than the rest of the projects If the top ten dependencies represented in Table 6.9 are studied it is possible to appreciate that despite it shows less number of coincidences between the test regarding the tables of the other projects, it still has repetitions. For instance, the dependency between the task 50 and the task 51 appears in all the tests. In addition, the dependency between the tasks 49 and 50 occurred in three of the four tests. There are also other dependencies that exist in at least two of the test such the pairs 101-107, 66-100, and 102-103. Test Test Test Test Top 1 66-100 23-24 101-107 21-22 1 2 3 4 Top 2 101-102 64-88 50-51 32-33 Top 3 29-97 49-50 2-4 43-44 Top 4 59-49 27-28 53-106 47-49 Top 5 61-62 53-101 49-50 63-64 Top 6 62-49 102-103 29-86 71-73 Top 7 53-101 79-80 66-100 101-107 Top 8 101-105 26-27 102-103 49-50 Top 9 50-51 14-13 48-49 50-51 Top 10 62-54 50-51 65-66 96-97 Table 6.9: Top 10 dependencies Project 1. The content represents the indexes between the two tasks which compose the dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). It can be observed that when the completion time is reduced by more than 2% according to Figure 6.1, Figure 6.2, Figure 6.3, and Figure 6.4 the top ten dependencies have different distributions, even if the dependencies are coincident in more than one test or not. Hence, it could be argued than in the cases when the impact of the dependency broken is 2% or more of decrease in the completion the solution offered by the model is reliable. Test Test Test Test 1 2 3 4 Top 1 1.14E-07 2.362e-05 3.875e-06 0.03329 Top 2 0.00003 0.00763 0.00011 0.02589 Top 3 0.00904 0.02220 0.00208 0.07240 Top 4 0.02260 0.15920 0.00910 0.03882 Top 5 0.00858 0.15790 0.02033 0.02565 Top 6 0.04664 0.51610 0.04225 0.01425 Top 7 0.03690 0.43200 0.04382 0.00317 Top 8 0.03565 0.55210 0.12520 0.10750 Top 9 0.05250 0.55590 0.10490 0.18730 Top 10 0.02550 0.67320 0.09356 0.23160 Table 6.10: P-Value Rank sum test Top 10 dependencies Project 1. P-value returned comparing the distribution of running 30 times the GA with the original TPG and with the TPG without the Top X dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). 70 Sensitivity Analysis Nevertheless, it still remains unclear the absolute validity of this information since the dependencies do not always produce the same impact and differs between the tests. This fact corroborated the data produced in Figure 6.1, Figure 6.2, Figure 6.3, and Figure 6.4. To conclude, despite there seems to be certain threads of supposition there are not enough evidences in the solutions provided by the model for this project to establish a pattern of behaviour or statement over the ability to find the most sensitive tasks with an absolute level of certainty. 6.5.3.2 Statistical Results Project 2 The statistical results for Project 2 and displayed in Table 6.11 show absolute confidence in all the top ten dependencies in all the tests. Thus, it is possible to state that the distribution of the thirty runs in the process of breaking dependencies performed during the sensitivity analysis is absolutely different from the execution of the model with the original TPG. As a result, it can be claimed that the differences in the average completion times, at least for these top ten dependencies, is not a matter of casualty. Test Test Test Test 1 2 3 4 Top 1 3.33E-11 9.77E-13 1.15E-12 2.94E-11 Top 2 3.01E-11 1.59E-13 1.16E-12 2.90E-11 Top 3 3.33E-11 1.69E-14 1.10E-12 2.95E-11 Top 4 3.69E-11 1.69E-14 9.07E-13 2.95E-11 Top 5 6.06E-11 1.69E-14 1.17E-12 2.93E-11 Top 6 6.06E-11 1.69E-14 8.82E-13 2.91E-11 Top 7 1.61E-10 1.69E-14 1.06E-12 2.95E-11 Top 8 9.91E-11 1.69E-14 9.38E-13 2.93E-11 Top 9 9.90E-11 1.69E-14 8.73E-14 2.93E-11 Top 10 9.75E-10 1.69E-14 1.06E-12 2.95E-11 Table 6.11: P-Value Rank sum test Top 10 dependencies Project 2. P-value returned comparing the distribution of running 30 times the GA with the original TPG and with the TPG without the Top X dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). The information collected through the rank sum test is not suppressing since the reductions of the completion time in this project proportion are greater than in the rest of them. This information can be observed in the figures of the section 6.4.2 of this paper. In summary, the results of the sensitivity analysis of the section 6.4.2 in addition to the statistical ones acquire in this section positively shows integrity of the model and the focus of this research in the process of measuring the sensitivity of the dependencies beyond doubt for this particular project scenario. 6.5.3.3 Statistical Results Project 3 Except the top 8, top 9, and top 10 dependencies of the Test 1 the rest show absolutely no doubt about the difference between the distribution. Therefore, the statistical results 71 Sensitivity Analysis for Project 3 add more reliability to idea of a successful beginning in the pursuance of identifying the most sensitive dependencies trough the use of GAs. Test Test Test Test 1 2 3 4 Top 1 0.001893 2.01E-13 1.52E-11 5.29E-06 Top 2 1.83E-03 1.69E-14 1.33E-10 1.84E-05 Top 3 1.19E-02 1.69E-14 3.33E-08 4.37E-03 Top 4 9.03E-03 1.69E-14 3.84E-04 3.04E-03 Top 5 8.25E-03 1.47E-09 2.00E-07 1.09E-02 Top 6 6.88E-02 1.83E-08 2.85E-05 3.89E-02 Top 7 1.35E-02 2.85E-04 2.65E-05 2.87E-02 Top 8 2.17E-01 4.18E-02 6.80E-04 2.34E-02 Top 9 1.07E-01 Top 10 1.31E-01 9.67E-04 2.20E-02 2.71E-03 6.14E-02 Table 6.12: P-Value Rank sum test Top 10 dependencies Project 3. P-value returned comparing the distribution of running 30 times the GA with the original TPG and with the TPG without the Top X dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). The output of the rank sum test for this project illustrated in Table 6.12 describe positive achievement even when the reduction of the completion time is smaller than 2%. This statement can be observed by the combination of the data of Table 6.12 and Figure 6.9. In this case, for Test 1 only the top 1 dependency was able to accomplish a decrease greater than 2%, yet from the top 2 to the top 7 all the dependencies have a different distribution with a level of confidence of 95% regarding the original TPG. To summarise, in spite of the fact that the results of the first project were not very concise and the limitations of the model represented by certain increments in this project as well as in Project 2, the results in both of them in the sensitivity analysis and the statistical corroboration encourage believing a breakthrough. 6.5.3.4 Statistical Results Project 4 The results for the statistical tests performed for this project and displayed in Table 6.13 show the expected output taking into consideration the reduction completion time achieved by the model. The statistics evidenced absolute level of confidence for all the dependencies in all the tests as it occurred in Project 2. Test Test Test Test 1 2 3 4 Top 1 2.99E-11 1.69E-14 1.69E-14 7.30E-13 Top 2 2.99E-11 1.69E-14 1.69E-14 9.30E-13 Top 3 3.00E-11 1.69E-14 1.69E-14 9.43E-13 Top 4 2.99E-11 1.69E-14 1.69E-14 9.49E-13 Top 5 2.99E-11 1.69E-14 2.71E-14 8.38E-13 Top 6 2.99E-11 1.69E-14 2.71E-14 8.62E-13 Top 7 3.66E-11 1.69E-14 4.15E-14 8.54E-13 Top 8 2.99E-11 1.69E-14 1.19E-13 8.76E-13 Top 9 2.99E-11 1.69E-14 1.69E-14 9.24E-13 Top 10 2.97E-11 1.69E-14 1.69E-14 9.88E-13 Table 6.13: P-Value Rank sum test Top 10 dependencies Project 4. P-value returned comparing the distribution of running 30 times the GA with the original TPG and with the TPG without the Top X dependency. Test 1 resource composition: 3 teams (3,4,5 people). Test 2 resource composition: 10 teams (1 person). Test 3 resource composition: 6 teams (1 person). Test 4 resource composition: 3 teams (1 person). The statement previously mentioned in addition to the results and the statistics of Project 2 and Project 3 demonstrate that the model developed is able to find in the majority of the cases the sensitive tasks. 72 Sensitivity Analysis To conclude, the model revealed almost perfect behaviour processing the scenario of this project. There were extremely few cases that produced increase in the completion time, and the cases that produce reduction exhibited a considerable level of similarity as well as complete confidence in the statistical outputs. 6.6 Threats to Validity The use of search-based software engineering and particularly the development of GAs to perform sensitivity analysis involves risk points. Following, this section details the main key points which could lead to threat the validity of this research: • The model developed is limited finding solutions when the scenario proposed entails a considerable number of tasks and dependencies. In those cases, it is only possible to approximate solutions, yet it provides significant candidate individuals. In addition, the model only covers three parameters whereas the classical PSP is defined taking into consideration a greater number of them. Therefore, the results collected could be affected by the introduction of those parameters. Furthermore, the fitness function evaluated is not perfect as it is detailed in the section 4.1.2 of this paper. Hence, the implementation of an ideal fitness function could be very complicated. • The process of adapting the data could mask possible dependencies as it is explained in the section 6.1 of this paper. Yet, the process has been designed logically to allow identifying dependencies which could lead to the most sensitive tasks or group of tasks. • The lack of certainty in the explanation over the results collected for Project 1 in the sensitivity analysis as well as in the statistical test adds certain level of reticence to the model developed. However, it is possible to believe a reasonable explanation through the limitation of the model in addition to the composition of the resources for the test and the duration of the tasks. • The use of genetic algorithms to tackle the classical PSP does not guarantee finding a solution for the resource allocation. Particularly if the scenario raised tries to cover all the possible parameters involved the project management. The complexity in that situation increases the difficulty not only to generate solutions or candidate individuals due to the different factor and constraints that should be taken into consideration, but also the computational cost as well as the high probability of being only able to approximate solutions. The fact of the algorithm 73 Sensitivity Analysis converging to a local optima is likely to happen, yet it still can reveal hints and could lead to identify the most sensitive dependencies. The computational cost it is an engineering problem that might be resolved by the use of different techniques, or programming languages, or similar approach or might not be. In spite of the fact that the approach introduced in this paper and the techniques used as well as the model proposed present relative level of weakness, the literature survey and the results collected encourage to believe that those barriers can be overcome. 6.7 Discussion This section emphasis the most important points that should be discussed regarding the results collected in the application of the model developed to the different scenarios raised. The results obtained showed that the idea behind this paper could be in the right direction towards a complete process of improving the PSP for the managers. In spite of the fact that the solutions produced by the model in the different test for the first project reveal certain level of failure, the information analysed for the rest of the projects and the corroboration of the statistical results probably mean relative degree of success. It is also necessary to analyse in a general way what the results obtained in the section 6.4 and the section 6.5 tell us about the projects. The results collected in Project 1 showed lack of certainty identifying the most sensitive dependencies. In spite of the fact, that there were coincidences between the Top 10 dependencies for the different test illustrated in Table 6.9, the truth is that they hardly fit. Furthermore, the statistical analysis cannot corroborate part of these cases. In conclusion, it is very unlikely to ensure for this project which are the most sensitive dependencies. However, the outcomes obtained in the application of the model developed for Project 2 and Project 4 resulted in exactly the main goal of this project. Not only the process of removing dependencies almost never reached increments and on the contrary was able to generate considerable reductions in the completion time. Also, the analysis of the top ten dependencies for both projects had a high level of similarity and an absolute verification statistically. Therefore, the model in these projects exhibited undeniable capacity. In conclusion, it would be possible to state that in these particular projects it is possible to indentify certain sensitive dependencies. The results gathered for Project 3 are not as positive as the ones for Project 2 and Project 5, but they still presented relative level of success. Analysing the top ten dependencies, 74 Sensitivity Analysis at least 3 of them appeared in all the tests. Nevertheless, it is also truth that in one of the cases cannot be corroborated by the statistical test performed. In general, it could be claimed that for this particular project it is possible to identify few dependencies that are sensitive in almost every scenario. At the same time and as well as in Project 2 and Project 4, there are dependencies which can be identified that could generate a massive reduction in the completion time. In addition, despite the results collected for Project 1 it is possible to establish certain level of coincidence in the results gathered for Project 2, Project 3, and Project 4. Thus, the similarities in these three projects allow identifying patterns of behaviour in the different test performed. Since, the top ten dependencies in the different test showed considerable degree of coincidence. However, it still remains unclear why there are various differences between the tests. This fact can be partly attributed to the diverse configuration of the resources available, although it cannot be completely assure to this factor. The next point to take into consideration is if the manager can use and trust the information provided by this model. Regarding the first point, it is undoubtedly that the information about the most sensitive tasks could be used by the project manager. Although, this does not necessarily means that can be applied. Since, the model developed does not understand the complexity of every project and the importance of each dependency. Therefore, the list of most sensitive tasks can be also the list of the most critical or crucial tasks, and as a result they cannot be removed. The second aspect about trusting the solutions produced by the model is more delicated to deal with. There is not manner to provide 100% confidence to the model developed. Nevertheless, the fact that the model has been applied to real project in addition to the statistical analysis performed as well as the strong foundations on which is based on allow to consider the results reliable, feasible, and solid. Moreover, it is also truth that there is an emergent necessity to improve and enhance the model. In so doing that, it would be possible to cover the rest of the parameters and aspects of the PSP not considered. Furthermore, this fact would add more reliability to the results provided endowing managers with more accurate information about sensitivity of tasks and dependencies as well as other factors of the project. 75 Chapter 7 Conclusions This section of the paper summarizes the main conclusions obtained in the application of the model developed on the scenario proposed. The model developed was fully detailed in the section 4, were five steps were described. In addition, the implementation used was the third version of the classical genetic algorithm, the process of breaking dependencies and the following sensitivity analysis, and a last part the statistical analysis of the results. The scenario used was composed by the four real projects detailed in the section 6.2 of this paper and adapted by the methodology described in the section 6.1. The importance of the data processed increases the relevance of the results collected since despite the application of search-based techniques to software project management could cover all the possible scope imaginable, the panoramic of the projects of the real world might be limited to a specific ambit. The results obtained in the sensitivity analysis for the process of breaking dependencies reveal the possibility to identify with a considerable level of accuracy the list of most sensitive dependencies in the definition of a project. This statement was clearly observed in at least three of the four project analysed, which correspond to Project 2, Project 3, and Project 4. Moreover, the use of statistical analysis clarified the difference between the distributions of the set of observations of running thirty times the GA without the most sensitive dependencies. In so doing that, it was corroborated the validity of the output which could have been attributed to the stochastic variance of the GA. Nonetheless, it still remains unclear the main reason for the despair results obtained in Project 1. Although, the statistic results for this project exhibited certain level of 76 Conclusions confidence, the fact that there is not a clear list of most sensitive tasks trough all the tests and the minor decrease produced in the completion time inspire hesitation. In addition, in spite of the fact that the limitation of the model developed, particularly the GA, and the resource availability have been name as partly responsible for the increments obtained breaking dependencies, this fact seems absolutely illogical within the project management. In conclusion, the idea behind this paper as well as the relative encouraging results accomplished in the application of the model developed to real data can settle a plausible and moderate as well as legitimate based for the use of search-based technique and sensitivity analysis for software project management. Nevertheless, it is also truth that deeper development and analysis are necessary to establish an absolute level of confidence in the use of this approach. The application and utilisation of techniques such genetic algorithms and sensitivity analysis involves therefore more exhaustive progress and evaluation. The main contributions of this paper are: • Genetic algorithm able to provide reasonable and competent resource allocation for different project scenarios. • Success identifying the most sensitive dependencies in the TPG of the project definition. • Introduction of the statistical techniques to corroborate the results collected and evaluated. • The model implemented could guide project managers through important information regarding the dependencies that could decrease the overall completion time of the project considerably. • This paper settles a new base for a use of search-based techniques in the software project management and it entails a breakthrough in the process of approaching the PSP. • Beyond the insights on the research question this paper deploys a new area survey. It opens a new perspective in the means and method to improve the PSP. The means used to reduce the completion time was breaking dependencies. The techniques used were genetic algorithms and sensitivity analysis. 77 Chapter 8 Future Work This section of the paper elaborates the next steps in the development of this research and introduces the first notions of what could be the future work to enlarge the dimension of it. In order to achieve as far as possible in the aims established at the beginning of this paper, and according to the last results obtained in the third version of the classical genetic algorithm, it seems reasonable to keep working in this flank of developing a reliable GA for the second part of the model proposed in the section 5 of this paper, which main focus is breaking dependencies and sensitivity analysis. Consequently, there are different alternatives in this sense. Probably the most relevant one might be to develop a new fitness function able to overcome the limitations of the current one. In addition, it is also reasonable the use of new operators to recombine the candidate solutions to generate the new parent population. There is an extensive list of options in this question. Therefore, it makes sense to improve the accuracy of the genetic algorithm or until it was able to approximate solutions much more precisely. Moreover, it would be interesting to enhance the scope of this research one step further in a natural evolution from the point of view of providing new methods to improve project management. A desirable progression is the possibility of being able to cover the application of SA not only to the dependencies between tasks, but also to every parameter inside the PSP. In the typical project management the range of most common parameters are: resources, in terms of teams, sub teams, and employees; skills, in terms of abilities and capacities of the employees and requirements of the tasks; cost, in terms of the salary of every employee according to the role fulfilled inside the project; dedication, in terms of hours per day/month of an employee to the project and/or task; duration, in terms of month/person necessary to conclude a task; and the already mentioned 78 Future Work dependencies. The considered parameters can be extended, yet these are the most suitable ones. In the application of SA to this list of factors the main purpose is to find the parameter or parameters that produce the biggest negative impact in the original schedule with the smallest variation in their initial estimation. In such a way that, the manager is able to identify the most critical elements in the project and to ensure that they are accurately performed according to the initial schedule. Moreover, this fact may reveal to the manager non-considered dependencies inside the project. For instance, the application of SA could show that an alteration of a minor percentage in the primitive estimation of a parameter associated to a particular task can delay the tasks which depend on the first one. In turn, these delays could be reflected in the appearance of new dependencies, and a result of all this facts a significant global delay would affect the original schedule of the project. 79 Appendix A Appendix 80 15.0000 15.9000 15.2000 14.8667 15.2500 15.6667 15.3333 15.9000 15.5333 15.8333 15.5833 16.5833 15.0000 16.1500 15.0500 15.5000 15.4000 15.0000 15.8500 15.0000 15.3167 15.5833 15.4000 15.7333 15.7833 16.0667 16.0000 15.2000 14.9500 15.6667 15.51000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 81 14.83166 TOP1 66-100 14.4500 14.6833 14.6667 14.6667 14.6500 14.7833 14.8500 14.9000 14.8500 14.4833 15.4833 14.4833 14.5833 15.4333 15.0500 14.8167 15.6000 15.2500 14.5500 15.3333 14.8333 14.9000 15.1833 14.3333 15.1667 14.9500 14.7000 14.6667 14.6667 13.9833 15.14389 TOP3 29-97 14.4000 14.4500 15.2500 16.2167 14.2667 14.9667 15.4833 14.8500 15.3333 14.6167 15.3500 15.6333 14.7500 15.2667 15.2667 15.7500 14.5333 15.5833 15.1167 14.6000 15.5333 15.6167 14.8500 15.3333 14.8833 14.4667 14.6000 16.0833 15.9167 15.3500 15.19333 TOP4 59-49 14.3500 14.5333 14.5333 15.0000 15.1667 15.0333 15.6667 15.4000 15.6667 14.7333 15.2500 15.0000 14.6000 14.9000 14.9500 14.3667 15.0000 16.1333 14.5000 15.3500 15.6667 15.5000 15.1500 15.8000 15.9333 15.4000 15.0833 15.6000 16.1333 15.4000 15.19555 TOP5 61-62 15.1000 15.1333 15.5500 15.2167 15.3333 15.5167 14.6667 15.2833 15.0000 14.7500 15.1500 15.0000 15.2000 15.0667 15.4000 15.0000 15.4500 15.3833 15.8500 15.3333 15.0000 14.6833 15.4000 15.6500 15.8667 15.5000 15.0833 14.6000 14.5000 15.2000 15.21555 TOP6 62-49 15.7333 15.1000 16.2000 14.8833 14.9667 15.4500 14.9333 15.8500 15.5000 14.9500 15.0667 16.0000 14.3000 15.6500 15.7000 15.5500 14.5333 15.2000 15.8333 14.3500 15.3833 15.5000 14.9333 15.5000 14.7167 15.4167 13.9833 15.5833 15.3000 14.4000 Table A.1: Top 10 Results Project 1 Test 1. 14.92388 TOP2 101-102 15.1833 14.1333 15.0500 15.0000 15.1333 15.0000 14.7500 15.7500 14.6833 15.1833 14.2000 14.5833 14.4167 14.7500 14.3500 14.2000 15.8000 16.1167 14.7333 15.9167 15.0000 14.4833 15.0000 14.9000 14.7667 14.5000 15.5333 14.6667 15.3333 14.6000 15.24000 TOP7 53-101 15.0833 15.8667 14.7000 14.7833 15.6000 15.8500 16.0667 14.6667 14.6000 15.5500 15.3667 15.0000 16.2333 15.4000 15.5000 14.7833 16.0000 14.7167 15.7500 15.2000 15.0000 14.7000 15.2500 14.7167 15.0000 15.2333 15.3500 14.7000 14.6000 15.9333 15.26000 TOP8 101-105 15.3167 15.1167 15.4000 16.3000 16.2667 14.6500 14.8500 14.9333 15.4000 15.2500 15.5167 14.5667 14.4667 15.2000 14.6333 15.5500 14.6500 15.7833 14.9333 15.9833 15.5833 14.6667 15.8167 15.4833 15.4000 14.9167 15.5000 15.3333 15.1333 15.2000 15.26223 TOP9 50-51 15.3000 15.7667 15.2000 15.1167 16.5000 14.8167 15.7500 14.7000 15.5000 15.3167 15.6667 14.8500 14.7500 13.8000 15.3333 15.0000 15.2000 15.1000 15.9833 15.8833 15.4667 15.0000 15.1833 15.0000 15.6667 15.7500 14.6667 14.6833 15.8000 15.1167 15.26278 TOP10 62-54 15.3833 15.8667 15.2000 16.0000 14.5833 15.0500 15.1000 14.9333 16.6667 15.3667 15.8667 15.4000 15.5167 15.0167 15.0000 15.4667 15.0333 15.0667 15.3000 15.0833 15.2500 14.6167 15.3500 14.9833 15.6000 15.2000 14.8333 15.1333 15.2167 14.8000 Appendix 30 31 31 33 30 31 30 31 30 28 30 30 29 31 31 31 30 30 29 28 29 30 30 30 29 29 31 28 31 33 30.13333 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 82 28.90000 TOP1 23-24 30 30 29 29 29 29 29 29 29 29 30 28 30 28 29 29 28 29 29 29 28 30 29 28 29 29 29 28 29 27 29.36667 TOP3 49-50 30 29 32 31 32 30 28 29 30 29 30 29 28 29 31 29 27 30 27 30 27 29 31 30 29 29 28 30 29 29 29.63333 TOP4 27-28 28 28 28 29 30 30 29 29 29 31 29 30 30 30 31 28 31 31 30 28 31 31 28 30 30 32 30 31 29 28 29.63333 TOP5 53-101 28 31 28 28 28 30 31 28 29 32 30 29 32 32 29 32 28 28 31 29 29 28 30 32 31 28 29 32 29 28 29.80000 TOP6 102-103 31 28 30 31 31 30 31 28 31 30 29 29 31 31 31 28 30 30 30 27 31 30 28 30 30 29 30 31 30 28 Table A.2: Top 10 Results Project 1 Test 2. 29.16667 TOP2 64-88 29 31 28 28 28 31 28 29 30 29 29 30 30 29 28 27 31 31 29 27 28 30 28 30 28 31 28 29 31 30 29.93333 TOP7 79-80 28 30 29 30 30 31 32 30 31 28 29 30 29 29 31 29 31 30 32 32 29 29 28 29 30 31 33 29 29 30 29.96667 TOP8 26-27 31 30 31 28 28 29 33 30 31 29 29 30 31 31 29 32 31 30 31 31 29 33 29 30 29 30 29 29 28 28 30.03333 TOP9 14-13 33 30 30 29 30 29 29 29 31 30 30 30 32 29 31 31 29 31 28 33 32 30 29 28 29 30 31 29 29 30 30.03333 TOP10 50-51 30 32 28 30 30 31 32 30 30 30 29 29 31 30 28 28 33 29 31 30 32 31 29 29 30 30 31 29 30 29 Appendix 38 43 41 41 40 39 40 40 38 43 40 40 40 41 40 41 38 41 40 41 38 39 41 38 39 39 40 40 40 40 39.96667 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 83 38.06667 TOP1 101-107 36 37 35 38 40 38 36 39 36 39 37 39 37 38 38 37 38 37 38 38 38 38 38 39 40 39 39 39 40 41 38.93333 TOP3 2-4 40 41 38 38 38 38 39 40 39 39 40 40 39 35 39 40 40 39 39 39 38 39 38 40 39 39 38 38 39 40 39.06667 TOP4 53-106 39 38 37 40 41 39 37 38 39 40 38 40 38 40 40 41 40 41 39 39 40 38 39 39 40 39 39 39 38 37 39.20000 TOP5 49-50 41 38 41 39 39 40 39 39 38 39 39 39 39 39 39 40 39 39 41 39 41 38 37 40 39 37 38 41 39 40 39.20000 TOP6 29-86 39 38 40 40 38 40 39 36 39 41 42 37 38 39 37 42 40 39 39 40 39 40 38 40 41 40 41 38 38 38 Table A.3: Top 10 Results Project 1 Test 3. 38.50000 TOP2 50-51 37 38 37 42 40 38 38 37 39 37 37 38 37 38 39 37 39 39 41 41 37 39 40 39 39 40 38 38 38 38 39.30000 TOP7 66-100 39 39 41 37 39 39 40 40 37 38 40 41 41 40 38 40 40 39 40 39 38 40 39 39 40 40 40 38 39 39 39.30000 TOP8 102-103 41 36 39 40 40 36 40 39 38 39 39 40 39 40 41 37 39 39 40 38 39 41 40 37 42 41 40 38 41 40 39.40000 TOP9 48-49 37 38 40 40 39 42 38 39 40 40 39 40 41 39 37 40 41 41 40 40 39 38 38 42 40 38 39 39 38 40 39.40000 TOP10 65-66 38 40 39 41 39 38 40 37 40 41 40 40 40 38 39 39 38 39 40 38 39 39 41 41 40 41 40 39 39 39 Appendix 60 59 58 63 61 66 64 62 61 63 64 63 64 63 63 63 62 59 60 66 64 62 63 62 60 63 61 64 63 61 62.23333 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 84 61.26667 TOP1 21-22 58 60 62 61 62 63 62 59 61 59 62 62 63 61 65 60 61 62 59 62 60 63 60 65 60 58 62 61 63 62 61.36667 TOP3 43-44 62 59 59 62 58 60 62 64 61 63 63 65 61 63 64 60 60 62 66 60 60 61 57 61 61 61 62 61 60 63 61.06667 TOP4 47-49 57 64 63 62 61 61 63 61 60 59 60 62 58 64 64 63 60 63 59 64 59 62 60 63 58 61 59 61 59 62 61.03333 TOP5 63-64 61 61 57 60 59 58 58 61 60 64 61 59 60 59 60 60 60 63 61 63 62 63 61 61 61 64 62 64 63 65 61.13333 TOP6 71-73 60 63 61 63 60 61 62 62 60 64 64 61 59 61 59 63 59 60 62 61 62 60 62 62 60 60 59 61 61 62 Table A.4: Top 10 Results Project 1 Test 4. 61.23333 TOP2 32-33 61 61 62 61 59 63 62 60 62 62 62 64 60 63 59 60 59 59 63 61 59 61 62 61 61 62 61 64 63 60 60.83333 TOP7 101-107 61 59 59 61 61 61 60 60 59 60 63 61 61 60 65 61 62 63 61 62 62 64 59 58 60 62 61 60 59 60 61.50000 TOP8 49-50 61 58 60 62 63 62 62 62 66 61 63 62 60 62 58 64 59 60 62 58 64 61 62 62 59 64 62 67 60 59 61.50000 TOP9 50-51 61 57 61 60 61 65 59 60 62 62 63 59 58 60 64 63 61 63 63 64 58 64 60 63 64 62 62 62 62 62 61.70000 TOP10 96-97 64 61 58 59 60 60 59 57 62 59 64 60 62 67 62 62 62 64 62 60 62 61 63 64 61 65 60 67 62 62 Appendix 47.0000 45.9333 51.8333 53.0333 51.4000 52.3833 51.2167 51.5833 51.8167 48.1000 48.9167 49.4333 49.4667 52.3167 48.6667 50.7333 51.2500 54.7500 53.3000 51.3000 47.5833 50.5500 48.4500 50.4167 49.1333 51.8167 50.9667 50.4333 50.3500 52.2833 50.54722 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 85 42.30999 TOP1 90-91 41.1667 41.8833 42.3333 43.0333 42.3833 42.0000 44.6167 42.1000 41.3000 40.9833 40.9667 41.3500 41.6000 41.2000 42.2333 41.2500 40.9500 43.4167 38.1833 44.5833 41.7833 40.7167 44.3333 46.7667 43.1833 44.5833 42.8333 43.0000 40.8833 43.6833 43.18167 TOP3 97-98 41.9667 41.7000 44.5500 42.4667 43.8000 40.2167 42.3500 44.5167 41.7667 44.7000 43.7333 44.6333 43.0167 43.3167 43.3500 43.4333 42.3500 42.6667 41.9667 43.2000 43.6000 44.7333 44.1500 46.5167 41.6000 43.1833 44.2000 42.5500 42.8833 42.3333 43.70112 TOP4 99-100 42.9667 42.4667 42.5333 44.9667 45.1333 42.8000 44.4833 45.0000 41.2500 43.6667 41.8667 45.8667 42.8667 45.8333 42.5333 44.6667 44.4667 44.3667 41.6167 46.3500 40.3000 42.3000 43.7167 42.7000 42.8333 46.6500 43.5333 45.5167 43.8833 43.9000 44.18890 TOP5 102-103 47.4333 42.4333 42.5000 44.8667 46.9167 44.1000 43.4667 46.6000 43.0000 43.1333 42.0667 43.3667 45.6000 43.9167 42.8333 45.6333 42.4500 44.2167 44.5833 41.1667 45.1667 46.0167 42.1167 46.5667 43.2667 43.8667 45.7667 45.9833 45.4500 41.1833 44.47610 TOP6 89-90 42.5333 44.1833 46.2833 40.3333 43.5000 45.1000 44.9000 45.3667 42.0667 42.3000 47.1333 44.7833 42.8833 44.4333 46.0833 44.0833 44.6667 45.8167 45.7833 45.8333 41.8667 45.2833 44.4667 44.7833 41.5167 44.5667 44.1500 45.5833 46.6667 47.3333 Table A.5: Top 10 Results Project 2 Test 1. 42.87666 TOP2 98-99 43.0333 41.8833 42.4833 43.3000 41.2000 42.4167 42.4667 43.4000 44.6833 41.2833 41.6500 44.8000 43.7000 44.0000 41.5000 45.8667 43.7500 42.7000 42.7833 42.7500 41.8167 43.0333 43.6000 42.6167 42.0167 41.5000 43.4000 42.3500 42.2333 44.0833 44.53167 TOP7 100-101 46.8667 42.0333 44.0667 46.4000 43.3333 42.1000 47.0667 44.9500 46.0667 41.1833 45.7500 42.2167 45.4000 45.4333 41.9000 40.5167 43.6500 44.2000 46.4167 43.5500 47.5333 47.3167 42.1167 43.0833 47.4667 42.1667 47.8167 43.6000 47.5167 44.2333 44.63222 TOP8 101-102 45.4167 48.3333 42.6833 44.8833 41.1500 43.6000 46.8833 44.7667 45.7833 42.8333 45.2000 42.7833 45.9667 44.9667 46.2333 45.4833 42.9167 41.3333 42.3000 45.3667 45.6333 45.5167 45.4333 47.3667 45.9500 42.0167 45.9000 47.1667 42.9333 42.1667 44.80389 TOP9 91-97 41.6000 42.6000 46.4000 43.7667 41.8833 45.2167 46.4333 42.6333 44.3833 43.4167 41.3000 46.3333 44.5333 43.9167 44.2167 45.1167 46.5667 46.4333 46.9000 45.4833 46.9833 45.3833 45.2333 44.1333 47.0167 44.8667 42.5500 47.2667 44.6500 46.9000 44.96334 TOP10 85-86 42.9500 46.2500 47.3333 43.8500 48.3667 48.1500 41.6667 46.0667 44.2000 43.2333 45.4500 45.2167 48.7000 43.8500 42.1000 46.2000 40.9833 42.4000 48.6500 42.0833 47.5333 42.3833 48.8667 47.2667 40.9167 46.9000 44.4000 43.1167 46.4167 43.4000 Appendix 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201.00000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 86 104.20000 TOP1 97-98 107 109 101 101 101 101 102 107 101 102 101 102 102 104 108 103 101 111 101 105 104 101 107 107 101 107 107 107 108 107 120.00000 TOP3 91-97 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 125.00000 TOP4 90-91 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 125 127.00000 TOP5 89-90 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 127 129.00000 TOP6 88-90 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 129 Table A.6: Top 10 Results Project 2 Test 2. 111.43333 TOP2 98-99 111 111 111 112 111 111 111 114 111 111 111 111 111 111 111 111 111 114 111 111 111 111 111 114 111 111 112 113 111 111 131.00000 TOP7 99-100 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 131 134.00000 TOP8 87-88 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 134 139.00000 TOP9 86-87 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 144.00000 TOP10 85-86 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 144 Appendix 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201 201.00000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 87 121.13333 TOP1 97-98 127 126 124 120 114 128 120 126 118 118 122 120 126 124 125 111 108 124 117 118 126 114 114 123 124 124 118 130 124 121 125.03333 TOP3 91-97 120 120 126 126 131 131 133 125 125 127 127 124 123 124 125 121 127 123 120 125 125 126 125 124 125 124 125 121 124 129 127.53333 TOP4 90-91 125 126 125 125 136 125 125 125 127 130 126 127 125 125 125 127 126 139 130 136 129 133 126 125 126 125 130 126 125 126 132.33333 TOP5 89-90 133 133 129 131 129 132 141 127 128 129 127 129 141 139 138 131 139 137 133 135 127 127 136 135 128 132 137 127 132 128 131.86667 TOP6 88-90 129 131 138 138 137 129 129 132 129 130 129 135 138 129 129 129 135 133 131 134 135 137 129 130 130 129 129 129 129 135 Table A.7: Top 10 Results Project 2 Test 3. 121.16667 TOP2 98-99 124 123 120 121 118 122 122 117 118 122 118 127 118 118 119 126 128 117 117 124 122 119 124 123 121 125 122 115 117 128 133.10000 TOP7 99-100 133 131 132 131 135 132 133 133 134 132 139 131 134 137 131 131 132 138 131 133 133 132 132 131 134 132 134 133 135 134 136.56667 TOP8 87-88 134 134 134 139 140 135 143 134 134 139 140 137 137 144 135 142 138 134 135 134 136 134 136 134 135 134 135 143 134 134 139.50000 TOP9 86-87 139 139 139 139 139 139 140 145 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 139 145 141 139 139 139 139 147.16667 TOP10 85-86 144 144 145 144 144 146 149 145 146 150 144 149 144 154 146 152 145 144 144 149 154 145 150 145 147 151 150 144 150 151 Appendix 219 220 221 228 217 215 215 234 218 219 222 213 219 221 210 220 213 212 233 221 216 215 229 216 222 216 206 211 220 219 218.66667 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 88 173.30000 TOP1 97-98 172 158 179 172 170 184 170 171 173 168 188 183 178 170 169 160 168 170 169 179 164 185 176 171 184 182 177 166 176 167 174.50000 TOP3 90-91 166 171 161 174 173 169 175 190 174 170 168 168 182 160 170 177 188 193 184 184 185 172 175 175 178 172 176 171 161 173 176.33333 TOP4 91-97 181 164 186 176 175 171 165 166 184 185 180 172 167 172 178 166 161 190 174 182 173 173 179 184 172 183 179 189 183 180 177.13333 TOP5 98-99 179 175 180 181 175 178 177 183 178 170 166 170 169 178 176 186 172 186 172 161 187 177 185 178 184 176 182 173 177 183 180.26667 TOP6 99-100 175 182 183 171 188 182 181 180 180 184 178 173 187 169 191 177 184 180 175 183 187 182 180 174 170 190 187 178 180 177 Table A.8: Top 10 Results Project 2 Test 4. 174.36667 TOP2 89-90 166 176 172 180 172 172 175 179 163 181 164 164 171 163 176 185 173 176 177 172 184 176 163 179 183 184 168 172 187 178 180.83333 TOP7 88-89 189 168 194 189 183 176 186 184 185 178 171 165 182 177 187 167 194 160 175 199 192 169 183 190 181 186 185 181 167 182 183.00000 TOP8 87-88 168 178 199 180 184 185 173 188 167 181 172 179 184 179 200 187 195 202 186 181 179 186 170 166 193 189 186 183 184 186 184.76667 TOP9 101-102 192 184 180 183 182 200 184 193 182 181 175 184 173 201 191 176 181 181 188 187 183 185 182 179 187 183 190 172 202 182 185.16667 TOP10 85-86 192 196 179 193 196 166 194 193 179 197 183 176 183 169 185 185 194 188 177 172 178 176 180 181 178 205 189 188 188 195 Appendix 9.45000 9.16667 9.40000 9.78333 9.48333 9.40000 9.66667 9.55000 9.65000 10.10000 9.50000 9.30000 9.53333 9.80000 9.45000 9.90000 9.55000 9.56667 9.55000 9.60000 9.46667 9.45000 9.68333 9.56667 9.43333 9.85000 9.31667 9.23333 9.61667 9.96667 9.56611 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 89 9.36556 TOP1 8-9 9.80000 9.36667 9.30000 9.63333 9.46667 9.58333 9.71667 9.38333 9.63333 9.23333 9.36667 9.28333 9.31667 9.30000 9.38333 9.20000 8.91667 9.26667 8.81667 9.16667 9.23333 9.51667 9.63333 9.53333 9.41667 9.55000 9.26667 9.20000 9.53333 8.95000 9.39167 TOP3 15-56 9.90000 9.26667 9.35000 8.95000 9.55000 9.55000 10.10000 9.15000 9.76667 9.36667 9.40000 9.20000 9.76667 9.75000 9.25000 9.60000 9.50000 9.20000 9.11667 8.86667 9.50000 9.10000 9.45000 9.05000 8.98333 9.86667 9.20000 9.35000 9.10000 9.55000 9.40389 TOP4 74-75 9.28333 9.13333 9.38333 9.26667 9.38333 9.53333 9.86667 9.40000 9.55000 9.30000 9.35000 9.70000 9.38333 9.21667 9.26667 8.98333 9.10000 8.96667 9.23333 9.30000 9.65000 9.65000 9.75000 9.46667 9.11667 9.20000 9.93333 9.95000 9.10000 9.70000 9.40889 TOP5 24-54 9.76667 9.21667 9.48333 9.40000 9.28333 9.35000 9.31667 9.46667 9.40000 9.55000 9.31667 9.61667 9.10000 9.23333 9.26667 9.20000 9.70000 9.53333 9.31667 9.68333 9.45000 9.35000 9.40000 9.70000 9.65000 9.55000 9.26667 9.53333 9.26667 8.90000 9.42333 TOP6 5-6 8.91667 9.48333 9.55000 9.13333 9.11667 9.11667 9.41667 9.11666 9.53333 9.45000 9.90000 9.73333 9.71667 9.56667 10.01670 9.03333 9.38333 9.05000 9.76667 9.35000 9.60000 9.53333 9.90000 9.21667 9.21667 8.78333 9.71666 9.65000 9.33333 9.40000 Table A.9: Top 10 Results Project 3 Test 1. 9.38889 TOP2 66-67 9.90000 9.25000 9.43333 8.85000 9.40000 9.45000 9.11667 9.20000 9.45000 9.75000 9.53333 9.11667 9.30000 9.51667 9.40000 9.01667 9.43333 9.30000 9.88333 9.45000 9.31667 9.76667 9.30000 9.25000 9.63333 9.11666 9.53333 9.40000 9.40000 9.20000 9.42833 TOP7 9-12 9.30000 8.96667 9.48333 9.76667 9.20000 9.65000 9.61667 9.25000 9.15000 9.43333 9.15000 9.35000 9.20000 9.25000 9.51667 9.58333 9.40000 9.05000 9.45000 9.25000 9.26667 10.23330 9.15000 9.10000 9.23333 9.48333 9.45000 10.05000 9.90000 9.96667 9.43833 TOP8 6-8 9.20000 9.51667 8.60000 9.61667 10.21670 8.96667 9.68333 9.48333 8.71667 9.83333 9.90000 9.11667 9.36667 9.46667 10.00000 10.08330 9.38333 8.91667 9.70000 9.31666 9.68333 9.90000 9.43333 9.15000 8.85000 9.33333 9.08333 9.18333 9.66667 9.78333 9.44500 TOP9 75-76 9.45000 8.75000 9.45000 9.56667 9.51667 9.28333 9.65000 9.51667 9.23333 9.35000 9.63333 9.86667 9.28333 9.65000 9.30000 9.36667 9.85000 9.86667 9.53333 9.15000 9.36667 9.26667 9.45000 9.46667 9.08333 9.81667 8.73333 9.85000 9.45000 9.60000 9.45111 TOP10 17-56 9.31667 8.65000 9.11667 9.95000 9.50000 8.70000 9.65000 9.18333 9.25000 9.06667 9.40000 9.23333 9.38333 9.15000 9.51667 9.75000 9.66667 9.53333 9.70000 9.26667 9.76667 9.50000 10.13330 9.60000 10.31670 9.65000 9.15000 9.38333 9.45000 9.60000 Appendix 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29.00000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 90 26.00000 TOP2 3-5 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 26 27.00000 TOP3 2-3 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 27 28.00000 TOP4 1-2 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28 28.23333 TOP5 6-8 28 28 29 29 28 28 28 29 28 28 28 28 28 29 29 28 29 28 28 28 28 29 28 28 28 28 28 28 28 28 Table A.10: Top 10 Results Project 3 Test 2. 20.13333 TOP1 5-6 18 20 20 20 21 20 20 20 21 20 20 20 20 20 20 20 20 20 20 20 20 21 21 20 20 20 21 20 21 20 28.30000 TOP6 8-9 28 29 28 28 28 29 28 28 28 29 29 28 28 29 29 28 28 29 28 28 28 29 28 28 29 28 28 28 28 28 28.63333 TOP7 65-71 29 29 29 28 29 29 29 29 29 29 28 28 28 28 28 29 28 29 29 29 29 29 29 29 28 28 29 29 28 28 28.86667 TOP8 9-65 29 29 29 29 29 29 28 29 29 29 29 28 29 29 28 29 29 29 29 29 29 29 29 29 28 29 29 29 29 29 Appendix 30 30 31 30 31 32 33 30 30 31 29 29 30 33 31 30 33 30 30 30 29 30 33 30 30 30 330 33 29 33 30.76667 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 91 24.90000 TOP1 5-6 26 26 25 25 25 25 22 25 25 25 24 26 25 26 23 26 24 27 24 26 25 25 25 24 23 25 25 24 26 25 29.16667 TOP3 73-74 29 29 30 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 30 30 29 29 29 31 29 29 29.20000 TOP4 3-5 28 30 30 27 30 29 28 30 32 28 30 31 27 29 30 31 29 28 30 28 30 27 30 30 30 26 30 30 28 30 29.23333 TOP5 59-60 29 29 29 29 29 30 29 29 29 29 29 29 29 29 29 29 30 29 31 30 29 29 29 29 29 29 29 29 31 29 29.43333 TOP6 8-9 29 29 29 29 30 29 30 29 30 29 29 29 30 28 29 30 30 29 30 29 29 30 29 31 30 29 30 29 30 30 Table A.11: Top 10 Results Project 3 Test 3. 29.00000 TOP2 6-7 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29 29.46667 TOP7 6-8 29 30 29 29 30 29 29 30 30 30 30 30 30 29 30 29 29 30 29 30 29 30 30 29 30 29 29 29 29 29 29.70000 TOP8 48-66 30 29 29 29 31 29 29 29 31 33 30 30 29 29 30 30 30 29 32 29 31 29 30 30 29 29 29 29 29 29 29.73333 TOP9 34-35 29 30 32 29 29 30 30 30 31 29 29 29 30 29 29 30 29 29 30 32 29 32 30 30 30 29 29 29 29 30 29.73333 TOP10 36-37 30 30 30 30 30 29 31 30 29 31 31 29 31 29 29 29 29 30 31 29 30 30 30 29 29 30 29 30 29 29 Appendix 41 40 41 40 42 41 41 39 41 42 41 40 42 39 40 41 41 39 41 41 42 41 42 38 37 40 41 41 40 40 40.50000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 92 38.73333 TOP1 5-6 41 37 37 36 40 38 38 37 40 40 40 39 39 40 41 36 39 39 38 37 40 39 39 37 40 40 40 38 39 38 39.46667 TOP3 8-65 38 41 41 39 41 39 41 38 39 42 38 38 41 38 40 41 38 38 39 40 38 38 41 39 39 41 37 40 41 40 39.53333 TOP4 8-9 38 39 40 41 40 42 39 39 39 41 39 38 37 41 39 39 39 40 38 38 41 40 40 42 40 38 39 42 39 39 39.80000 TOP5 6-7 38 39 40 37 40 41 40 39 39 42 39 39 40 41 40 40 42 41 40 40 39 39 41 39 39 41 41 39 40 39 39.86667 TOP6 74-75 41 38 41 39 43 41 38 41 39 39 41 40 41 41 37 38 39 40 41 39 39 38 41 40 41 39 40 40 40 41 Table A.12: Top 10 Results Project 3 Test 4. 39.10000 TOP2 6-8 40 38 38 38 39 40 39 40 37 39 41 39 38 39 39 40 39 37 41 41 39 39 39 40 40 39 38 40 38 39 39.86667 TOP7 75-76 42 41 40 41 40 39 39 41 40 39 38 39 38 39 42 43 40 38 40 39 39 39 38 40 41 40 42 39 41 39 39.90000 TOP8 7-10 39 40 39 41 40 40 38 38 40 40 42 38 39 41 39 41 42 40 40 39 41 39 39 42 41 40 40 40 40 39 39.90000 TOP9 10-12 38 40 40 39 40 40 38 41 38 44 39 42 39 40 37 40 39 41 40 40 44 42 40 40 41 39 40 39 40 37 39.93333 TOP10 24-28 40 40 39 37 40 40 39 41 40 39 41 43 43 41 38 41 39 41 39 37 41 39 41 42 40 38 39 40 39 41 Appendix 16.2667 16.4167 16.2167 15.6667 16.0500 16.2667 16.1500 16.3000 16.5833 16.0500 16.3833 15.9333 16.2333 16.5500 16.3667 16.3833 16.1500 15.7000 16.3000 16.5167 16.3333 15.8000 16.0167 16.2833 16.3667 16.0667 15.8833 16.2833 16.5500 16.0667 16.20445 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 93 14.29278 TOP1 52-53 13.9833 14.4333 14.3167 14.5333 14.0000 14.2833 14.5333 14.5500 14.7333 14.4333 14.1500 14.4167 14.1167 14.0667 14.0333 14.6167 14.5167 14.1667 14.0000 14.6167 13.8833 14.0833 14.1500 14.2667 14.1333 14.3167 14.2667 14.5833 14.5667 14.0333 14.40944 TOP3 21-46 14.1333 14.2333 14.6667 13.7833 14.0500 14.9500 14.3667 14.2500 14.3500 14.5833 14.5667 14.7333 14.6000 14.1333 14.2000 14.2833 14.2500 14.7667 13.7000 14.6833 14.5167 14.4667 14.7333 14.5333 14.8500 14.6333 14.3333 14.5500 14.4500 13.9333 14.48166 TOP4 50-51 14.9667 14.4333 14.3667 14.6833 14.3833 14.8000 14.2167 14.7333 14.6000 14.5000 14.0333 14.3167 14.3167 14.2500 14.0333 14.7333 14.7000 14.7500 14.7167 14.0833 14.1167 14.2000 14.7167 14.9833 14.1667 14.4333 14.4333 14.3500 14.5333 14.9000 14.54722 TOP5 20-21 14.6833 14.9333 14.5833 14.6167 14.8000 14.7167 14.0833 14.2833 14.2500 14.5000 14.8333 14.3667 14.5000 14.4000 14.5000 14.5500 14.4167 14.8667 14.3000 13.9833 14.2667 14.5333 14.3667 15.1333 14.6000 14.1000 14.4333 14.8000 15.2333 14.7833 14.60001 TOP6 48-49 14.3167 14.6167 14.6667 15.2667 14.2333 14.5500 14.2500 14.9833 15.1833 14.2833 14.1833 14.4500 14.2667 14.6833 14.5167 14.3500 15.0000 14.6167 14.5500 14.7167 14.2500 14.6500 14.7667 15.0167 14.0333 14.5333 15.2167 14.5167 14.7167 14.6167 Table A.13: Top 10 Results Project 4 Test 1. 14.38111 TOP2 51-52 14.8500 14.2833 14.1833 14.4167 14.0333 14.4000 14.6000 14.2500 15.0667 14.3667 14.1833 14.3667 14.8667 14.4167 14.4333 14.3833 14.9167 13.9333 14.3333 14.2500 14.2500 13.9333 13.7333 14.6667 14.2667 14.6333 14.2333 14.5167 14.7500 13.9167 14.69778 TOP7 46-48 15.7333 14.5167 14.8833 14.6667 15.3667 14.5500 14.8500 14.8333 14.9667 14.4167 14.1833 14.6000 14.5667 14.2000 14.2333 14.1000 14.2667 14.0000 14.7000 14.5500 14.6667 15.3500 15.5167 15.2000 14.9500 14.7333 14.5833 14.1000 15.0000 14.6500 14.70833 TOP8 49-50 14.4000 14.7500 14.8833 14.2167 14.7000 14.5167 14.8000 15.4167 14.5333 14.3833 14.0833 15.2167 14.7000 15.2167 14.6667 14.5500 14.1000 14.6500 15.1833 15.3000 14.8833 14.4333 14.8833 15.0000 14.0833 14.6833 14.7667 14.1333 14.6167 15.5000 15.03056 TOP9 55-56 15.0333 15.3667 14.6500 15.4333 14.6667 15.3667 14.9000 14.8833 15.3667 15.1500 14.7833 15.3000 15.2667 15.6000 14.9833 15.3333 14.9333 14.5333 14.7667 15.0000 14.8500 15.0000 14.9667 14.7667 15.6167 14.6500 15.2333 14.9167 15.0500 14.5500 15.04390 TOP10 9-10 15.1667 15.3500 14.9500 14.8833 14.9500 14.8333 15.3500 15.2000 14.5000 15.3500 15.3167 14.8667 15.3667 15.2000 15.1000 14.9500 15.6333 15.1167 14.6500 15.1167 14.5833 14.9500 14.7167 14.9167 15.2333 15.2500 14.9167 14.7667 15.0167 15.1167 Appendix 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 94 54 54 TOP2 21-46 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP3 46-48 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP4 48-49 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP5 49-50 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP6 50-51 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 Table A.14: Top 10 Results Project 4 Test 2. TOP1 20-21 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP7 51-52 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 TOP8 52-53 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 56 TOP9 10-20 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 57 TOP10 9-10 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 Appendix 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72.00000 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 95 54.00000 TOP1 49-50 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.00000 TOP3 51-52 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.00000 TOP4 52-53 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.03333 TOP5 20-21 54 54 54 54 54 54 55 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.03333 TOP6 48-49 54 54 54 54 54 54 54 54 55 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 Table A.15: Top 10 Results Project 4 Test 3. 54.00000 TOP2 50-51 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.06667 TOP7 46-48 54 54 54 54 55 54 54 54 55 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54 54.20000 TOP8 21-46 55 54 54 54 54 54 54 54 54 54 54 56 54 54 54 54 55 54 54 55 54 54 54 55 54 54 54 54 54 54 56.00000 TOP9 10-20 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 57.00000 TOP10 9-10 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 57 Appendix 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 72 73 72 72 72 72 72 72 72 72 72 72.03333 Dependeny broken Execution 1 Execution 2 Execution 3 Execution 4 Execution 5 Execution 6 Execution 7 Execution 8 Execution 9 Execution 10 Execution 11 Execution 12 Execution 13 Execution 14 Execution 15 Execution 16 Execution 17 Execution 18 Execution 19 Execution 20 Execution 21 Execution 22 Execution 23 Execution 24 Execution 25 Execution 26 Execution 27 Execution 28 Execution 29 Execution 30 Average Original 96 61.16667 TOP1 52-53 60 62 62 61 61 64 62 62 61 63 61 60 61 61 61 60 61 60 61 63 61 63 61 59 61 61 60 61 60 61 61.60000 TOP3 50-51 60 62 63 62 63 61 60 61 61 60 62 61 64 63 60 61 63 61 61 61 63 62 63 62 61 61 62 62 62 60 61.83333 TOP4 21-46 60 62 64 64 62 62 61 62 60 61 64 59 60 62 63 62 61 61 61 62 63 62 61 61 60 62 65 62 62 64 61.86667 TOP5 48-49 62 60 63 60 64 62 62 62 63 61 63 62 63 61 64 62 62 62 61 61 61 63 61 62 62 60 62 62 62 61 62.03333 TOP6 20-21 62 62 61 62 64 62 62 63 62 65 60 64 61 60 61 62 62 60 62 63 60 62 64 62 62 63 60 63 62 63 Table A.16: Top 10 Results Project 4 Test 4. 61.20000 TOP2 51-52 61 62 60 63 62 61 62 62 62 61 61 62 62 61 62 62 62 61 64 58 61 61 61 60 60 60 60 59 63 60 62.40000 TOP7 49-50 63 64 64 63 63 62 63 63 61 62 65 63 63 61 61 63 63 63 62 62 62 63 63 60 61 63 64 59 62 61 62.76667 TOP8 46-48 64 65 63 64 62 62 64 62 63 62 61 65 63 62 60 62 64 63 61 62 62 64 64 62 62 62 64 64 62 63 64.16667 TOP9 10-20 63 65 64 64 64 63 63 64 64 64 66 64 67 65 63 64 65 64 65 64 63 65 65 65 65 62 64 62 67 62 64.23333 TOP10 55-56 64 64 63 66 64 65 63 63 66 63 66 66 65 62 65 64 62 64 62 65 64 66 64 64 65 65 65 64 63 65 Appendix Appendix B Appendix MAIN PROGRAM # include < cstdlib > # include < iostream > # include < stdio .h > # include < math .h > # include < fstream > # include < exception > # include < ctime > # include " Functions . h " # include " Teams . h " # include " Sony . h " # include " QuotesOrder . h " # include " Example . h " # include " SmartPrice . h " # include " CutOver . h " # include " DataBase . h " # include " SoftChoice . h " # include " GA . h " using namespace std ; using namespace std ; int main ( int argc , char * argv []) { // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / / / / / / / / / / / V A R I A B L E S & CONST / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // C O N S T A N T S int num_ite rations (1000) ; // Number of p o p u l a t i o n s g e n e r a t e d int n um _i n di vi du a ls (100) ; // Size of the p o p u l a t i o n 97 Appendix B, Code int top (10) ; // Number of best i n d i v i d u a l s to keep in the binary t o u r n a m e n t // V A R A I B L E S // Timing clock_t start , finish ; double time ; // Num of e x e c u t i o n s of the GA int exe (0) ; // R e s o u r c e s int num_teams (0) ; // Number of teams // C o m p o s i t i o n Teams int teams [10]; // Array to store the c o m p o s i t i o n in member of every team // Tasks int num_tasks (0) ; // Number of task int Project (0) ; // V a r i a b l e to know which Project has been choosen cout << " 0. - Example Project (7 Tasks ; 10 Dependencies ) " << endl ; cout << " 1. - Sony Project (139 Tasks ; 1257 Dependencies ) " << endl ; cout << " 2. - QuotesOrder Project (78 Tasks ; 168 Dependencies ) " << endl ; cout << " 3. - SmartPrice Project (92 Tasks ; 93 Dependencies ) " << endl ; cout << " 4. - CutOver Project (107 Tasks ; 83 Dependencies ) " << endl ; cout << " 5. - DataBase Project (106 Tasks ; 103 Dependencies ) " << endl ; // cout << " 6. - S o f t C h o i c e Project (328 Tasks ; ??? D e p e n d e n c i e s ) " << endl ; do { cout << " Choose Project ( default 0) : " ; cin >> Project ; } while (( Project < 0) || ( Project > 5) ) ; switch ( Project ) { case 0: num_tasks = 7; break ; case 1: num_tasks = 139; break ; case 2: num_tasks = 78; break ; case 3: num_tasks = 92; break ; case 4: num_tasks = 107; break ; 98 Appendix B, Code case 5: num_tasks = 106; break ; // case 6: // n u m _ t a s k s = 328; // break ; } float dur_tasks [ num_tasks ]; // Array to store the d u r a t i o n of each task in person / month do { cout << endl << " Enter number of iterations between 100 and 100 ,000: " ; cin >> num_iter ations ; } while (( num_ite rations < 100) || ( n um_itera tions > 100000) ) ; do { cout << endl << " Enter size of the populations between 10 and 10 ,000: " ; cin >> n um _i nd i vi du al s ; } while (( n um _i n di vi du a ls < 10) || ( nu m _i nd iv i du al s > 10000) ) ; do { cout << endl << " Enter number of TOP individuals between 2 and population ’s size , must be even : " ; cin >> top ; } while (( top < 2) || ( top > n um _i nd i vi du al s ) || (( top %2) !=0) ) ; do { cout << endl << " Enter number of Teams between 2 and 10: " ; cin >> num_teams ; } while (( num_teams < 2) || ( num_teams >10) ) ; do { cout << endl << " Enter of executions of the GA between 1 and 30: " ; cin >> exe ; } while (( exe < 1) || ( exe >30) ) ; // TPG int mtpg [ num_tasks * num_tasks ]; int * tpg = mtpg ; // A s s i g m e n t of the Task P r e c e d e n c e Graph for ( int i = 0; i < num_tasks ; i ++) for ( int j = 0; j < num_tasks ; j ++) tpg [ i * num_tasks + j ]=0; // Fitness F u n c t i o n s float dpff [ nu m_ i nd iv id u al s ]; // Fitness F u n c t i o n of the i n d i v i d u a l s of the parent p o p u l a t i o n 99 Appendix B, Code for ( int i = 0; i < n u m_ in di v id ua ls ; i ++) // I n i t i a l i s e f i t t n e s s f u n c t i o n of the i n d i v u a l s of the current p o p u l a t i o n dpff [ i ] = 0; float * pff = dpff ; float dcff [ nu m_ i nd iv id u al s ]; // Fitness F u n c t i o n of the i n d i v i d u a l s of the children population for ( int i = 0; i < n u m_ in di v id ua ls ; i ++) // I n i t i a l i s e f i t t n e s s f u n c t i o n of the i n d i v u a l s of the current p o p u l a t i o n dcff [ i ] = 0; float * cff = dcff ; float dnff [ nu m_ i nd iv id u al s ]; // Fitness F u n c t i o n of the i n d i v i d u a l s of the new parent p o p u l a t i o n for ( int i = 0; i < n u m_ in di v id ua ls ; i ++) // I n i t i a l i s e f i t t n e s s f u n c t i o n of the i n d i v u a l s of the current p o p u l a t i o n dnff [ i ] = 0; float * nff = dnff ; float iteFF [ n um_iter ations ]; // Best Fitness F u n c t i o n of each of the populations for ( int i = 0; i < num_i teration s ; i ++) // I n i t i a l i s e f i t t n e s s f u n c t i o n of the p o p u l a t i o n s iteFF [ i ] = 0; float BFF (0) ; // Best Fitness F u n c t i o n int Bite (0) ; // I t e r a t i o n of the best fitness funtion float averageFF (0) ; // Average of the best ff of all the e x e c t u i o n s of the GA // P o p u l a t i o n int mppopulation [ nu m _i nd i vi du al s * num_tasks ]; int * ppopulation = mppopulation ; // C h i l d r e n p o p u l a t i o n int mcpopulation [ nu m _i nd i vi du al s * num_tasks ]; int * cpopulation = mcpopulation ; // New Parent p o p u l a t i o n int mnpopulation [ nu m _i nd i vi du al s * num_tasks ]; int * npopulation = mnpopulation ; // P r o b a b i l i t y of m u t a t i o n float n ( num_tasks ) ; float mut_pro (1/ n ) ; // V a r i a b l e s for o r d e r i n g by q u i c k s o r t int inf (0) ; int sup ( num_individuals -1) ; // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / 100 Appendix B, Code // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / CODE / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // system (" PAUSE ") ; // Files ofstream file ; file . open ( " results . txt " ) ; // CODE file << " Number of Iterations : " << num_i teratio ns << endl ; file << " Size of each population : " << n um _ in di vi d ua ls << endl ; file << " Number of TOP individuals : " << top << endl ; // Teams I n i t i l i s a t i o n T e a m s ( num_teams , teams , file ) ; // D u r a t i o n of each task in the future can be s u s t i t u t e d switch ( Project ) { case 0: file << " EXAMPLE " << endl ; I n i t i a l i s a t i o n E x a m p l e ( dur_tasks , num_tasks , tpg , file ) ; break ; case 1: file << " SONY PROJECT " << endl ; I n i t i a l i s a t i o n S o n y ( dur_tasks , num_tasks , tpg , file ) ; break ; case 2: file << " QUOTES ORDER PROJECT " << endl ; I n i t i a l i s a t i o n Q u o t e s ( dur_tasks , num_tasks , tpg , file ) ; break ; case 3: file << " SMARTPRICE PROJECT " << endl ; I n i t i a l i s a t i o n S m a r t P r i c e ( dur_tasks , num_tasks , tpg , file ) ; break ; case 4: file << " CUTOVER PROJECT " << endl ; I n i t i a l i s a t i o n C u t O v e r ( dur_tasks , num_tasks , tpg , file ) ; break ; case 5: file << " DATABASE PROJECT " << endl ; I n i t i a l i s a t i o n D a t a B a s e ( dur_tasks , num_tasks , tpg , file ) ; break ; // case 6: // file << " S O F T C H O I C E PROJECT " << endl ; // I n i t i a l i s a t i o n S o f t C h o i c e ( dur_tasks , num_tasks , tpg , file ) ; // break ; } // - - - - - - - - - - - - - - - V A R I A B L E S ( for b r e a k i n g d e p e n d e n c i e s ) 101 Appendix B, Code int num_dep (0) ; // Number of d e p e n d e n c i e s of that tasks int dep_rem (0) ; // D e p e n d e n c y removed // - - - - - - - - - - - - - - - PROCESS for ( int k = 0; k < num_tasks ; k ++) { // Process of b r e a k i n g d e p e n d e n c i e s for every task num_dep = tpg [ k * num_tasks +0]; // Number of d e p e n d e n c i e s of the task k if ( num_dep > 0) { // There are d e p e n d e n c i e s file << endl << " Task " << k << " has " << num_dep << " dependencies " << endl ; tpg [ k * num_tasks +0] = num_dep - 1; // D e c r e a s e the number of d e p e n d e n c i e s by 1. The one which is going to be removed if ( num_dep == 1) { // if there is only 1 dependency , there is no need to r e c o n s t r u c t new length "0" , then the program do not enter in this TPG section dep_rem = tpg [ k * num_tasks +1]; file << " Remove dependency number 1 from task " << dep_rem << " to task " << k << endl ; averageFF = 0; GA ( ppopulation , cpopulation , npopulation , pff , cff , nff , iteFF , BFF , Bite , averageFF , exe , tpg , dur_tasks , num_tasks , num_teams , teams , num_individuals , num_iterations , top , inf , sup , mut_pro , start , finish , time , file ) ; file << " Remove dependency number 1 from task " << dep_rem << " to task " << k << " with average FF : " << averageFF << endl ; } else { for ( int h = 1; h <= num_dep ; h ++) { // Loop to process every d e p e n d e n c y dep_rem = tpg [ k * num_tasks + h ]; file << " Remove dependency number " << h << " from task " << dep_rem << " to task " << k << endl ; // TPG is r e c o n s t r u c t e d r e m o v i n g the d e p e n d e c y for ( int l = h ; l < num_dep ; l ++) // all d e p e n d e n c i e s are moved to one p o s i t i o n b a c k w a r d s to remove the d e p e n d e n c y broken tpg [ k * num_tasks + l ] = tpg [ k * num_tasks + l +1]; tpg [ k * num_tasks + num_dep ] = 0; // Now there are one d e p e n d e n c y less so the last one is removed averageFF = 0; 102 Appendix B, Code GA ( ppopulation , cpopulation , npopulation , pff , cff , nff , iteFF , BFF , Bite , averageFF , exe , tpg , dur_tasks , num_tasks , num_teams , teams , num_individuals , num_iterations , top , inf , sup , mut_pro , start , finish , time , file ) ; file << " Remove dependency number " << h << " from task " << dep_rem << " to task " << k << " with average FF : " << averageFF << endl ; // TPG is r e c o n s t r u c t e d to the o r i g i n a l one for ( int l = num_dep ; l > h ; l - -) // all d e p e n d e n c i e s are moved to one p o s i t i o n a f t e r w a r d s to re - i n t r o d u c e the d e p e n d e n c y broken tpg [ k * num_tasks + l ] = tpg [ k * num_tasks +l -1]; tpg [ k * num_tasks + h ] = dep_rem ; // D e p e n d e n c y removed in r e s t o r e d } } tpg [ k * num_tasks +0] = num_dep + 1; } } // Close the file with the results file . close () ; // system (" PAUSE ") ; return EXIT_SUCCESS ; } FUNCTIONS # include < iostream > using namespace std ; // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / F U N T I O N S / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / / // *** Reset the c o m p l e t i o n times of each taks ***// void R e s e t _ c o m p l e t i o n _ t i m e s ( float start_time [] , float end_time [] , int num_tasks ) { for ( int i = 0; i < num_tasks ; i ++) { start_time [ i ] = 0; end_time [ i ] = 0; } } // *** Print i n d i v i d u a l ***// void P ri nt In d iv id u al ( int * population , int individual , int num_tasks , ofstream & file ) { file << " Individual " << individual << " : " << endl ; for ( int i = 0 ; i < num_tasks ; i ++) file << " Task " << i << " : " << population [ individual * num_tasks + i ] << " " ; 103 Appendix B, Code file << endl ; } // *** Finish Tasks ***// void Finish_task ( int * population , int individual , int task , int * tpg , float dur_tasks [] , int num_tasks , float start_time [] , float end_time [] , int num_teams , int teams [] , ofstream & file ) { // V A R I A B L E float latest_time (0) ; // Lastest time of the p r e c e d e n t s tasks float dur_task (0) ; // D u r a t i o n of the tasks a c c o r d i n g to the team a s s i g n e d float sta rt_time_ tmp (0) ; // T e m p o r a r y start time of the task float end_time_tmp (0) ; // T e m p o r a r r y end time of the task int team ; int num_dep (0) ; // Number of d e p e n d e n c i e s of the tasks that it ’s being checked int pre_task (0) ; // P r e c e d e n t task that need to be f i n i s h e d // CODE // team = p o p u l a t i o n . get ( individual , i ) ; team = population [ individual * num_tasks + task ]; num_dep = tpg [ task * num_tasks +0]; // Number of d e p e n d e n c i e s of task i if ( num_dep > 0 ) for ( int j = 1; j <= num_dep ; j ++) { // Check p r e c e d e n t tasks have been finished pre_task = tpg [ task * num_tasks + j ]; if ( end_time [ pre_task ] == 0) Finish_task ( population , individual , pre_task , tpg , dur_tasks , num_tasks , start_time , end_time , num_teams , teams , file ) ; // C o m p l e t e task if ( latest_time < end_time [ pre_task ] ) latest_time = end_time [ pre_task ]; } // C o m p l e t i o n time dur_task = 0; dur_task = ( dur_tasks [ task ] / teams [ team ] ) ; 104 Appendix B, Code start _time_t mp = 0; start _time_t mp = latest_time ; end_time_tmp = 0; end_time_tmp = start _time_tm p + dur_task ; // Verify Overlap // Check all the tasks p e r f o r m e d by the team which is p e r f o r m i n g this tasks and see // whether there is enough gap to do the tasks , o t h e r w i s e perform at the end of the lastest task // T r a v e r s e all the tasks f i n i s h e d in order to d e t e r m i n e whether there is overlap or not for ( int i = 0; i < num_tasks ; i ++) { if ( ( population [ individual * num_tasks + i ] == team ) && ( i != task ) && ( end_time [ i ] != 0) ) { if ( ( ( end_time_tmp < end_time [ i ]) && ( end_time_tmp >= start_time [ i ] ) ) || ( ( start_ti me_tmp >= start_time [ i ]) && ( sta rt_time_ tmp < end_time [i] ) ) ){ star t_time_t mp = end_time [ i ]; end_time_tmp = start_ time_tm p + dur_task ; } } } // C o m p l e t i o n times end_time [ task ] = end_time_tmp ; start_time [ task ] = st art_time _tmp ; } // *** Fitness F u n c t i o n ***// float Fi tn es s Fu nc t io n ( int * population , int individual , int * tpg , float dur_tasks [] , int num_tasks , float start_time [] , float end_time [] , int num_teams , int teams [] , ofstream & file ) { // V A R I A B L E float latest_time (0) ; // Lastest time of the p r e c e d e n t s tasks float dur_task ; // D u r a t i o n of the tasks a c c o r d i n g to the team a s s i g n e d float sta rt_time_ tmp ; // T e m p o r a r y start time of the task float end_time_tmp ; // T e m p o r a r r y end time of the task 105 Appendix B, Code int team ; int num_dep (0) ; // Number of d e p e n d e n c i e s of the tasks that it ’s being checked int pre_task (0) ; // P r e c e d e n t task that need to be f i n i s h e d // CODE // C a l c u l a t e the c o m p l e t i o n time of each tasks for ( int i = 0; i < num_tasks ; i ++) { latest_time = 0; start _time_t mp = 0; end_time_tmp = 0; team = population [ individual * num_tasks + i ]; // Team which perform tasks i if ( end_time [ i ] == 0 ) { // Check the tasks has not been c o m p l e t e d yet num_dep = tpg [ i * num_tasks +0]; // Number of d e p e n d e n c i e s of task i if ( num_dep > 0 ) for ( int j = 1; j <= num_dep ; j ++) { // Check p r e c e d e n t tasks have been finished pre_task = tpg [ i * num_tasks + j ]; if ( end_time [ pre_task ] == 0) // P r e c e d e n t tasks has not been f i n i s h e d Finish_task ( population , individual , pre_task , tpg , dur_tasks , num_tasks , start_time , end_time , num_teams , teams , file ) ; // C o m p l e t e task if ( latest_time < end_time [ pre_task ] ) latest_time = end_time [ pre_task ]; } // C o m p l e t i o n times dur_task = 0; dur_task = ( dur_tasks [ i ] / teams [ team ] ) ; star t_time_t mp = 0; star t_time_t mp = latest_time ; end_time_tmp = 0; end_time_tmp = start _time_tm p + dur_task ; // Verify Overlap // Check all the tasks p e r f o r m e d by the team which is p e r f o r m i n g this tasks and see // whether there is enough gap to do the tasks , o t h e r w i s e perform at the end of the lastest task 106 Appendix B, Code // T r a v e r s e all the tasks f i n i s h e d in order to d e t e r m i n e whether there is overlap or not for ( int k = 0; k < num_tasks ; k ++) { if ( ( population [ individual * num_tasks + k ] == team ) && ( i != k ) && ( end_time [ k ] != 0) ) { if ( ( ( end_time_tmp < end_time [ k ]) && ( end_time_tmp >= start_time [ k ] ) ) || ( ( s tart_tim e_tmp >= start_time [ k ]) && ( star t_time_ tmp < end_time [ k ] ) ) ) { star t_time_t mp = end_time [ k ]; end_time_tmp = start _time_tm p + dur_task ; } } } // Final c o m p l e t i o n times end_time [ i ] = end_time_tmp ; start_time [ i ] = start_t ime_tmp ; } } // Return the c o m p l e t i o n time float ff (0) ; for ( int i = 0; i < num_tasks ; i ++) if ( end_time [ i ] > ff ) ff = end_time [ i ]; return ( ff ) ; } // *** I n i t i a l i z e the p o p u l a t i o n s ***// void F i l l _ i n i t i a l _ i n d i v i d u a l s ( int * population , float * ff , int * tpg , float dur_tasks [] , int num_tasks , int num_teams , int teams [] , int num_individuals , ofstream & file ) { // V a r i a b l e s bool one ( false ) ; // To i n d i c a t e whether the tasks has already been a s s i g n e d to one team int rd ; // To store the random number to create the initial p o p u l a t i o n s float s_time [ num_tasks ]; // Start c o m p l e t i o n time for each task float e_time [ num_tasks ]; // End c o m p l e t i o n time for each task 107 Appendix B, Code for ( int i = 0; i < num_tasks ; i ++) { s_time [ i ] = 0; e_time [ i ] = 0; } // Code srand (( unsigned ) time (0) ) ; // to ensure real random // fill r a m d o m l y the p o p u l a t i o n s of the a s s i g m e n t between tasks and teams // Obtain the f i t t n e s s f u n c t i o n or c o m p l e t i o n time of each p o p u l a t i o n for ( int i = 0 ; i < nu m _i nd iv i du al s ; i ++) { for ( int j = 0 ; j < num_tasks ; j ++) population [ i * num_tasks + j ] = rand () % num_teams ; ff [ i ] = F it ne s sF un ct i on ( population ,i , tpg , dur_tasks , num_tasks , s_time , e_time , num_teams , teams , file ) ; R e s e t _ c o m p l e t i o n _ t i m e s ( s_time , e_time , num_tasks ) ; } } // *** C r o s s o v e r 1 ***// void Crossover1 ( int * population , int * cpopulation , int newindividual , int individual1 , int individual2 , int num_tasks , int point1 , int point2 , ofstream & file ) { for ( int i = 0; i < point1 ; i ++) cpopulation [ newindividual * num_tasks + i ] = population [ individual1 * num_tasks + i ]; for ( int i = point1 ; i < point2 ; i ++) cpopulation [ newindividual * num_tasks + i ] = population [ individual2 * num_tasks + i ]; for ( int i = point2 ; i < num_tasks ; i ++) cpopulation [ newindividual * num_tasks + i ] = population [ individual1 * num_tasks + i ]; } // *** C r o s s o v e r 2 ***// void Crossover2 ( int * population , int * cpopulation , int newindividual , int individual1 , int individual2 , int num_tasks , int point1 , int point2 , ofstream & file ) { for ( int i = 0; i < point1 ; i ++) cpopulation [ newindividual * num_tasks + i ] = population [ individual2 * num_tasks + i ]; for ( int i = point1 ; i < point2 ; i ++) 108 Appendix B, Code cpopulation [ newindividual * num_tasks + i ] = population [ individual1 * num_tasks + i ]; for ( int i = point2 ; i < num_tasks ; i ++) cpopulation [ newindividual * num_tasks + i ] = population [ individual2 * num_tasks + i ]; } // *** Copy I n d i d u a l ***// void CopyInd ividual ( int * population1 , int * population2 , int num_tasks , int individual , ofstream & file ) { // V A R I A L B E S int value ; // CODE for ( int i = 0; i < num_tasks ; i ++) population2 [ individual * num_tasks + i ] = population1 [ individual * num_tasks + i ]; } // *** Copy I n d i d u a l 2 ***// void C op yI nd i vi du a l2 ( int * population1 , int * population2 , int num_tasks , int individual , int pos , ofstream & file ) { // V A R I A L B E S int value ; // CODE for ( int i = 0; i < num_tasks ; i ++) population2 [ pos * num_tasks + i ] = population1 [ individual * num_tasks + i ]; } // *** Swap ***// void Swap ( int * population , float * ff , int num_tasks , int a , int b , ofstream & file ) { // V a r i a b l e s float tmp_ff ; int tmp [ num_tasks ]; // Code if ( a != b ) { tmp_ff = ff [ a ]; ff [ a ] = ff [ b ]; ff [ b ] = tmp_ff ; for ( int i = 0; i < num_tasks ; i ++) tmp [ i ] = population [ a * num_tasks + i ]; 109 Appendix B, Code for ( int i = 0; i < num_tasks ; i ++) { population [ a * num_tasks + i ] = population [ b * num_tasks + i ]; population [ b * num_tasks + i ] = tmp [ i ]; } } } // *** Pivot ***/ /* I m p l e m e n t a c i n no c l s i c a de la f u n c i n Pivote . En lugar de r e c o r r e r el vector s i m u l t n e a m e n t e desde ambos e x t r e m o s hasta el cruce de n d i c e s , se recorre desde el c o m i e n z o hasta el final */ int Partition ( int * population , float * ff , int num_tasks , int inf , int sup , int pivotIndex , ofstream & file ) { // V a r i a b l e s float pivotValue (0) ; int index (0) ; // Code pivotValue = ff [ pivotIndex ]; // Mueve el pivote a la l t i m a p o s i c i n del vector Swap ( population , ff , num_tasks , pivotIndex , sup , file ) ; index = inf ; /* Recorre el vector m o v i e n d o los e l e m e n t o s menores o iguales que el pivote al c o m i e n z o del mismo */ for ( int i = inf ; i < sup ; i ++) { if ( ff [ i ] <= pivotValue ) { Swap ( population , ff , num_tasks ,i , index , file ) ; index ++; } } // Mueve el pivote a la p o s i c i n que le c o r r e s p o n d e Swap ( population , ff , num_tasks , index , sup , file ) ; return index ; } // *** Q u i c k S o r t for o r d e r i n g a p o p u l a t i o n ***// void O r d e r P o p u l a t i o n Q u i c k S o r t ( int * population , float * ff , int num_tasks , int inf , int sup , ofstream & file ) { // V a r i a b l e s int pivot (0) ; // pivot int pivotNew (0) ; // pivot new 110 Appendix B, Code // Code if ( inf < sup ) { /* S e l e c c i o n a un e l e m e n t o del vector y coloca los menores que l a su i z q u i e r d a y los mayores a su derecha */ pivot = sup ; pivotNew = Partition ( population , ff , num_tasks , inf , sup , pivot , file ) ; /* Repite el proceso para cada una de las p a r t i c i o n e s g e n e r a d a s en el paso a n t e r i o r */ O r d e r P o p u l a t i o n Q u i c k S o r t ( population , ff , num_tasks , inf , pivotNew -1 , file ) ; O r d e r P o p u l a t i o n Q u i c k S o r t ( population , ff , num_tasks , pivotNew +1 , sup , file ) ; } } // *** R e c o m b i n a t i o n of the GA ***// void Recombination ( int * population , int * cpopulation , float * ff , int num_individuals , int num_teams , int num_tasks , ofstream & file ) { // V a r i a b l e s int inf (0) ; int sup ( num_individuals -1) ; int point1 (0) , point2 (0) ; // Two points to use as part of the o p e r a t o r int tmp_ind1 (0) , tmp_ind2 (0) ; // T e m p o r a r y i n d i v i d u a l s for the binary t o u r n a m e n t int ind1 (0) , ind2 (0) ; // I n d i v i d u a l 1 and I n d i v i d u a l 2 to r e c o m b i n e int tmp ; // CODE // G e n e r a t e c h i l d r e n p o p u l a t i o n based of the non - top i n d i v i d u a l s for ( int i = 0; i < n um _ in di vi d ua ls ; i ++) { // C a l c u l a t e two random point to R e c o m b i n a t e the p o p u l a t i o n point1 = rand () % num_tasks ; do point2 = rand () % num_tasks ; while ( point2 == point1 ) ; // Order both points if ( point1 > point2 ) { tmp = point1 ; point1 = point2 ; point2 = tmp ; } // C a l c u l a t e First i n d i v i d u a l to r e c o m b i n e by binary t o u r n a m e n t tmp_ind1 = rand () % nu m_ i nd iv id u al s ; do tmp_ind2 = rand () % nu m_ i nd iv id u al s ; 111 Appendix B, Code while ( tmp_ind1 == tmp_ind2 ) ; // Choose the one with the best FF if ( ff [ tmp_ind1 ] < ff [ tmp_ind2 ]) ind1 = tmp_ind1 ; else ind1 = tmp_ind2 ; // C a l c u l a t e Second i n d i v i d u a l to r e c o m b i n e by binary t o u r n a m e n t do tmp_ind1 = rand () % nu m_ i nd iv id u al s ; while ( tmp_ind1 == ind1 ) ; do tmp_ind2 = rand () % nu m_ i nd iv id u al s ; while (( tmp_ind1 == tmp_ind2 ) || ( tmp_ind2 == ind1 ) ) ; // Choose the one with the best FF if ( ff [ tmp_ind1 ] < ff [ tmp_ind2 ]) ind2 = tmp_ind1 ; else ind2 = tmp_ind2 ; Crossover1 ( population , cpopulation ,i , ind1 , ind2 , num_tasks , point1 , point2 , file ) ; Crossover2 ( population , cpopulation ,++ i , ind1 , ind2 , num_tasks , point1 , point2 , file ) ; } } // *** M u t a t i o n and e v a l u a t i o n of the GA ***// void Mutation ( int * population , int num_individuals , float dur_tasks [] , int num_tasks , int num_teams , int teams [] , float mut_pro , ofstream & file ) { // V A R I A B L E float rd ; float pro_mut (0) ; // P r o b a b l i t y of m u t a t i o n for a tasks int old_team (0) , new_team (0) ; // CODE for ( int i = 0; i < n um _ in di v id ua ls ; i ++) for ( int j = 0; j < num_tasks ; j ++) { rd = rand () ; pro_mut = rd / RAND_MAX ; if ( pro_mut < mut_pro ) { // if the p r o b a b i l i t y c a l c u l a t e d for that tasks is minor that the total p r o b a b i l i t y then mutated old_team = population [ i * num_tasks + j ]; do new_team = rand () % num_teams ; while ( old_team = new_team ) ; 112 Appendix B, Code population [ i * num_tasks + j ] = new_team ; } } } // *** E v a l u a t i o n ***// void Evaluation ( int * population , int * tpg , float * ff , int num_tasks , float dur_tasks [] , int num_teams , int teams [] , int num_individuals , ofstream & file ){ // V a r i a b l e s float start_time [ num_tasks ]; // Start c o m p l e t i o n time for each task float end_time [ num_tasks ]; // End c o m p l e t i o n time for each task for ( int i = 0; i < num_tasks ; i ++) { start_time [ i ] = 0; end_time [ i ] = 0; } // Code for ( int i = 0; i < n um _ in di v id ua ls ; i ++) { R e s e t _ c o m p l e t i o n _ t i m e s ( start_time , end_time , num_tasks ) ; ff [ i ] = F it ne s sF un ct i on ( population ,i , tpg , dur_tasks , num_tasks , start_time , end_time , num_teams , teams , file ) ; } } // *** New Parent P o p u l a t i o n ***// void G e n e r a t e N e w P o p u l a t i o n ( int * ppopulation , int * cpopulation , int * npopulation , float * pff , float * cff , float * nff , int num_individuals , int top , int num_tasks , int num_teams , ofstream & file ) { // V A R I A B L E S int index1 (0) , index2 (0) ; int inf (0) ; int sup ( num_individuals -1) ; int index_c hildren (0) ; int index_parent (0) ; // CODE // O r d e r i n g the parent p o p u l a t i o n using quick sort O r d e r P o p u l a t i o n Q u i c k S o r t ( cpopulation , cff , num_tasks , inf , sup , file ) ; 113 Appendix B, Code // TOP I N D I V I D U A L S // Copy Best Top I n d i v i d u a l s from the parent and c h i l d r e n p o p u l a t i o n to New Parent p o p u l a t i o n for ( int i = 0; i < top ; i ++) { if ( pff [ index_parent ] < cff [ index_ children ] ) { C op yI nd i vi du al 2 ( ppopulation , npopulation , num_tasks , index_parent ,i , file ) ; nff [ i ] = pff [ index_parent ]; index_parent ++; } else { C op yI nd i vi du al 2 ( cpopulation , npopulation , num_tasks , index_children ,i , file ) ; nff [ i ] = cff [ index_ childre n ]; index _childr en ++; } } // NON - TOP I N D I V I D U A L S // G e n e r a t e c h i l d r e n p o p u l a t i o n based of the non - top i n d i v i d u a l s for ( int i = top ; i < nu m_ i nd iv i du al s ; i ++) { // C a l c u l a t e First i n d i v i d u a l to add by binary t o u r n a m e n t do index1 = rand () % nu m_ in d iv id ua l s *2; while (( index1 < top ) || (( index1 >= n um _i n di vi du a ls ) && ( index1 < n um _i nd i vi du al s + top ) ) ) ; // C a l c u l a t e Second i n d i v i d u a l to add by binary t o u r n a m e n t do index2 = rand () % nu m_ in d iv id u al s *2; while (( index1 == index2 ) || ( index2 < top ) || (( index2 >= n um _i nd i vi du al s ) && ( index2 < n u m_ in di v id ua ls + top ) ) ) ; // Choose the one with the best FF // I n d i v i d u a l s from " index >= top " to " index < n u m _ i n d i v i d u a l s " c o r r e s p o n d to the non - top parent p o p u l a t i o n // I n d i v i d u a l s from " index > n u m _ i n d i v i d u a l s + top " to " index < n u m _ i n d i c i d u a l s * 2" c o r r e s p o n d to the non - top c h i l d r e n p o p u l a t i o n if ( ( index1 >= top ) && ( index1 < n u m_ in di v id ua ls ) ) // index 1 in parent population if ( ( index2 >= top ) && ( index2 < n u m_ in di v id ua ls ) ) // index 2 in parent population if ( pff [ index1 ] < pff [ index2 ]) { C op yI n di vi du a l2 ( ppopulation , npopulation , num_tasks , index1 ,i , file ) ; nff [ i ] = pff [ index1 ]; } else { C op yI nd i vi du al 2 ( ppopulation , npopulation , num_tasks , index2 ,i , file ) ; nff [ i ] = pff [ index2 ]; 114 Appendix B, Code } else // index 2 in c h i l d r e n population if ( pff [ index1 ] < cff [ index2 ]) { C op yI n di vi du a l2 ( ppopulation , npopulation , num_tasks , index1 ,i , file ) ; nff [ i ] = pff [ index1 ]; } else { C op yI nd i vi du al 2 ( cpopulation , npopulation , num_tasks , index2 ,i , file ) ; nff [ i ] = cff [ index2 ]; } else // index 1 in c h i l d r e n population if ( ( index2 >= top ) && ( index2 < n u m_ in di v id ua ls ) ) // index 2 in parent population if ( cff [ index1 ] < pff [ index2 ]) { C op yI n di vi du a l2 ( cpopulation , npopulation , num_tasks , index1 ,i , file ) ; nff [ i ] = cff [ index1 ]; } else { C op yI nd i vi du al 2 ( ppopulation , npopulation , num_tasks , index2 ,i , file ) ; nff [ i ] = pff [ index2 ]; } else // index 2 in c h i l d r e n population if ( cff [ index1 ] < cff [ index2 ]) { C op yI n di vi du a l2 ( cpopulation , npopulation , num_tasks , index1 ,i , file ) ; nff [ i ] = cff [ index1 ]; } else { C op yI nd i vi du al 2 ( cpopulation , npopulation , num_tasks , index2 ,i , file ) ; nff [ i ] = cff [ index2 ]; } } } // *** Reset ***// void Reset ( int * population , int * cpopulation , int * npopulation , float * pff , float * cff , float * nff , float iteFF [] , int num_tasks , int num_teams , int num_individuals , int num_iterations , ofstream & file ) { for ( int i = 0 ; i < nu m _i nd iv i du al s ; i ++) { pff [ i ] = 0; cff [ i ] = 0; nff [ i ] = 0; 115 Appendix B, Code for ( int j = 0 ; j < num_tasks ; j ++) { population [ i * num_tasks + j ] = 0; cpopulation [ i * num_tasks + j ] = 0; npopulation [ i * num_tasks + j ] = 0; } } for ( int i = 0; i < num_i teration s ; i ++) iteFF [ i ] = 0; } GENETIC ALGORITHM void GA ( int * ppopulation , int * cpopulation , int * npopulation , float * pff , float * cff , float * nff , float * iteFF , float BFF , int Bite , float & averageFF , int exe , int * tpg , float dur_tasks [] , int num_tasks , int num_teams , int teams [] , int num_individuals , int num_iterations , int top , int inf , int sup , float mut_pro , float start , float finish , float time , ofstream & file ) { for ( int i = 0; i < exe ; i ++) { start = clock () ; // Fill initial i n d i v i d u a l s with data F i l l _ i n i t i a l _ i n d i v i d u a l s ( ppopulation , pff , tpg , dur_tasks , num_tasks , num_teams , teams , num_individuals , file ) ; // O r d e r i n g the parent p o p u l a t i o n using quick sort O r d e r P o p u l a t i o n Q u i c k S o r t ( ppopulation , pff , num_tasks , inf , sup , file ) ; // Genetic A l g o r i t h m for ( int iterations = 1; iterations < num_it erations ; iterations ++) { Recombination ( ppopulation , cpopulation , pff , num_individuals , num_teams , num_tasks , file ) ; // R e c o m b i n a t i o n ( O p e r a t o r ) Mutation ( cpopulation , num_individuals , dur_tasks , num_tasks , num_teams , teams , mut_pro , file ) ; // M u t a t i o n ( Random change ) Evaluation ( cpopulation , tpg , cff , num_tasks , dur_tasks , num_teams , teams , num_individuals , file ) ; G e n e r a t e N e w P o p u l a t i o n ( ppopulation , cpopulation , npopulation , pff , cff , nff , num_individuals , top , num_tasks , num_teams , file ) ; if (( BFF == 0) || ( nff [0] < BFF ) ) { BFF = nff [0]; Bite = iterations ; } // New parent become parent ppopulation = npopulation ; pff = nff ; 116 Appendix B, Code } // Finish time finish = clock () ; time = ( double ( finish ) - double ( start ) ) / CLOCK S_PER_SE C ; averageFF = averageFF + BFF ; file << endl ; file << " " << BFF ; // << " " << Bite << " " << time ; BFF = 0; Bite = 0; time = 0; Reset ( ppopulation , cpopulation , npopulation , pff , cff , nff , iteFF , num_tasks , num_teams , num_individuals , num_iterations , file ) ; } averageFF = averageFF / exe ; file << endl ; } 117 Bibliography [1] M. Harman and B. F. Jones. Search-based Software Engineering. Information and Software Technology, 43(14):833–839, 2001. [2] M. Harman. The Current State and Future of Search Based Software Engineering. In FOSE 2007: 2007 Future of Software Engineering, pages 342–357, Washington, DC, USA, 2007. IEEE Computer Society. [3] G. Antoniol, M. Di Penta, M. Harman, and F. Qureshi. The Effect of Communication Overhead on Software Maintenance Project Staffing: a Search-Based Approach. In ICSM 2007: Proceedings of the 23rd IEEE International Conference on Software Maintenance, pages 315–324, Paris, France, October 2007. IEEE Computer Society. [4] G. Antoniol, M. Di Penta, and M. Harman. Search-Based Techniques Applied to Optimization of Project Planning for a Massive Maintenance Project. In ICSM 2005: Proceedings of the 21st IEEE International Conference on Software Maintenance, pages 240–249, Washington, DC, USA, 2005. IEEE Computer Society. [5] G. Antoniol, M. Di Penta, and M. Harman. A Robust Search-Based Approach to Project Management in the Presence of Abandonment, Rework, Error and Uncertainty. In METRICS 2004: Proceedings of the Software Metrics, 10th International Symposium, pages 172–183, Washington, DC, USA, 2004. IEEE Computer Society. [6] P. McMinn. Search-Based Software Test Data Generation: a Survey. Software Testing, Verification and Reliability, 14(2):105–156, 2004. [7] J.A. Clark and J.L. Jacob. Protocols Are Programs Too: the Meta-heuristic Search for Security Protocols. Information and Software Technology, 43:891–904(14), 15 December 2001. [8] E. Alba and J. F. Chicano. Software Project Management with GAs. Information Sciences, 177(11):2380–2401, 2007. [9] E. Alba and J. F. Chicano. Management of Software Projects with GAs. In MIC 2005: Proceedings of the 6th Metaheuristics International Conference, pages 13–18, Viena, Austria, August 2005. 118 Bibliography Appendix B, Code [10] S. Yoo. The Use of a Novel Semi-Exhaustive Search Algorithm for the Analysis of Data Sensitivity in a Feature Subset Selection Problem. Master’s thesis, King’s College London, London, UK, September 2006. [11] B. L. Miller and D. E. Goldberg. Genetic Algorithms, Tournament Selection, and the Effects of Noise. Complex Systems, 9:193–212, 1995. [12] T. Tian. The Nature of Crossover Operator in Genetic Algorithms. In RSCTC 2000: Revised Papers from the Second International Conference on Rough Sets and Current Trends in Computing, pages 619–623, London, UK, 2001. Springer-Verlag. [13] L. Breierova and M. Choudhari. An Introduction to Sensitivity Analysis. MIT System Dynamics in Education Project, Massachusetts, USA, 2001. URL http: //sysdyn.clexchange.org. [14] Kenneth M. Hanson and Franois M. Hemez, editors. Global Sensitivity Analysis: An Introduction, Joint Research Centre of Ispra, Italy, 2005. Los Alamos National Laboratory. URL http://library.lanl.gov. [15] A. Lhr and K. Bletzinger. Application of Sensitivity Analysis to a Planning Process in Architecture, Engineering and Construction (AEC). In WCSMO 2005: Proceedings of the 6th World Congress on Structural and Multidisciplinary Optimization, Rio de Janerio, Brazil, 2005. [16] S. Wagner. A Model and Sensitivity Analysis of the Quality Economics of Defectdetection Techniques. In ISSTA 2006: Proceedings of the 2006 International Symposium on Software Testing and Analysis, pages 73–84, New York, NY, USA, 2006. ACM. [17] A. Saltelli. Sensitivity Analysis for Importance Assessment. Risk Analysis, 22(3): 579–590, 2002. [18] P. Jovanovic. Application of Sensitivity Analysis in Investment Project Evaluation under Uncertainty and Risk. International Journal of Project Management, 17: 217–222, 1999. [19] E. W. Johnson and J. B. Brockman. Sensitivity Analysis of Iterative Design Processes. In ICCAD 1996: Proceedings of the 1996 IEEE/ACM International Conference on Computer-aided Design, pages 142–145, Washington, DC, USA, 1996. IEEE Computer Society. [20] W. H. Wakeland, R. Martin, and D. Raffo. Using Design of Experiments, Sensitivity Analysis, and Hybrid Simulation to Evaluate Changes to a Software Development 119 Bibliography Appendix B, Code Process: a Case Study. In John Wiley and Ltd Sons, editors, ProSim 2003: Proceedings of the 4th International Workshop on Software Process Simulation and Modeling, pages 107–119, Portland, Oregon, USA, November 2004. Wiley InterScience. [21] Y. Hoon Kwak and L. Ingall. Exploring Monte Carlo Simulation Applications for Project Management. Risk Management, 9(1):44–57, 2007. [22] S. Wagner. Global Sensitivity Analysis of Predictor Models in Software Engineering. In PROMISE 2007: Proceedings of the Third International Workshop on Predictor Models in Software Engineering, page 3, Washington, DC, USA, 2007. IEEE Computer Society. 120