Available online at www.sciencedirect.com
ScienceDirect
Available online at www.sciencedirect.com
Procedia Computer Science 00 (2018) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 124 (2017) 405–414
4th Information Systems International Conference 2017, ISICO 2017, 6-8 November 2017, Bali,
Indonesia
The Use of Analytic Hierarchy Process for Software Development
Method Selection: A Perspective of e-Government in Indonesia
Melisa Helingo, Betty Purwandari*, Riri Satria, Iis Solichah
Faculty of Computer Science, Universitas Indonesia, Depok 16424, Indonesia
Abstract
Information Technology (IT) has played an important role at the Ministry of Foreign Affairs of the Republic of Indonesia
(Kemlu). It provides e-government services to cut long bureaucracy and achieve priorities in foreign policies. As the organization
grows, there are needs to develop new software in this e-government context. In general, it requires a software development
standard procedure to meet software quality assurance. In Kemlu the standard procedure in software development becomes more
critical due to frequent changes among IT team members. It is the nature of this organization to send their staffs to foreign
countries for several years. Then they are back to home country for another couple of years, and get ready for their next
assignments abroad. A fundamental step in a software development standard procedure is determining the most suitable software
development method. However, there was lack of investigation for this selection. To address this problem, a study using Analytic
Hierarchy Process (AHP) is conducted. The factor variables are personnel, requirements, applications, organizations, businesses,
operations, and technology. The alternatives of software development methods are Waterfall, Incremental, Prototyping, Extreme
Programming, Scrum, and Rational Unified Process. The results show that Prototyping is the fittest software development
method for Kemlu.
© 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 4th Information Systems International Conference 2017.
Keywords: Software Development Process; Software Development Method; Analytic Hierarchy Process; e-Government
1. Introduction
Kemlu has been developing and using e-government services as an implementation of good IT governance and
bureaucratic reformation in Indonesia. Its Communication Center (Puskom) has developers and managers of
* Corresponding author. Tel.: +62-21-786-3419; fax: +62-21-786-3415.
E-mail address: bettyp@cs.ui.ac.id
1877-0509 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 4th Information Systems International Conference 2017.
1877-0509 © 2018 The Authors. Published by Elsevier B.V.
Peer-review under responsibility of the scientific committee of the 4th Information Systems International Conference 2017
10.1016/j.procs.2017.12.171
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
406
2
Information and Communication Technology (ICT) in Kemlu. It was stated in the 2015 Puskom’s Annual Report
that software development standard procedures are needed in Kemlu. Hence, the activities of Puskom in 2016
would focus on preparing the procedure. The Head of Puskom highlighted that these procedures could help the
development team to understand their work despite the team changes.
Fig. 1. Mind map of software development standard procedure [1].
Fig. 1 provides a framework of software development standard procedure. It shows detailed activities or processes
based on a specific method for completing a task [1]. Meanwhile, the chosen software development method would
determine the software development activities and its sequence [2]. It would also affect the process sequence in the
software development standard procedure. Hence, the research asked, “What is the most suitable software
development method for the Ministry of Foreign Affairs (Kemlu)?”
The scope of this research is the software development process to provide the e-government services in Kemlu,
which includes software for public services, administration and management, legislation administration,
development administration, finance, and human resource. The purpose of this study is to determine the most
suitable software development method for e-Government in Kemlu. The results of this study would be an input in
preparing the software development standard procedure in Kemlu.
2. Literature review
2.1. Software development method
Software development methods address the categories and ways of developing software by modeling the software
development process. The software development method term has several synonyms, such as process models [2, 3],
software development methodologies [4], and software development strategies [5]. This paper uses the term:
software development method.
Table 1. Software development methods in the government organizations of some countries.
Software development methods
USA
UAE
Malaysia
Korea
Indonesia
Waterfall








Incremental
Prototyping
XP
Scrum
RUP





The case study on software development methods used in government institutions also conducted in several
countries, as summarized in Table 1. This study discusses all methods mentioned in the first column of Table 1.
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
407
3
2.2. The suitable software development method for an organization
The suitable software development method for an organization is selected by analyzing the circumstances, needs,
and problems within the organization [6]. The analysis is conducted by evaluating some criteria in software
development [7]. However, the compliance of the sub criteria to determine software development methods for a
government organization has not been defined. Therefore, this study refers to the criteria defined in the literature as
summarized in Table 2 to determine software development method.
Table 2. Criteria to determine software development methods [7].
Software development methods
Example of sub criteria
Personnel
Change of personnel, team size, team diversity, team distribution, team members’ skill,
team productivity, team members’ commitment, and conflict between team members.
Requirement
Continues requirement changes, unclear requirements, requirements standard; the
necessity to meet the requirement.
Application
Risk level and application performance, complexity of products and activities,
architectural hardware, application type and size, an application with the existing and
future systems, configuration management and documentation, stage of development, the
number of application versions that have been used, product quality.
Organization
Maturity, size, and stability of the organization, standards and culture of the organization,
The use of the latest programming practices, facilities and technical support, the
management commitment to the project.
Operational
User involvement with the developer team, the number of users, user rotation,
prerequisites (for example: policies, regulations, common practices in the organization,
and ease of use).
Business
Financial considerations, marketing activities, dependence to the other projects, user
satisfaction, and the time needed for the project to complete, the level of the potential loss.
Technology
Experience in using the application development tools, new technology has not been used
in the previous projects, the introduction of new technologies.
Based on the case studies summarized in Table 1 and the literature, a criteria list to determine software
development method in this study is summarized in the first column of Table 3.
2.3. Previous research
The stages to choose the most suitable software development method for an organization are summarized from
the previous research, as follow:
•
•
•
•
Determine the criteria [8]
Define the characteristics of each software development method alternative and organization needs [9]
Use a decision-making method [10, 11]
Take the respondents which consist of the Chief Information Officer (CIO) [12], the Division of Information
Technology [8], the Project Manager, team leader of software developers, and software designers [11].
2.4. Analytic hierarchy process (AHP)
AHP is a method of decision making using objective calculations based on evaluation of several criteria [13]. The
AHP defines some stages of analysis [14]: the hierarchical structures formulation, prioritization, priority weight
calculation of each criteria or alternative, and consistency checking. The hierarchical structure is defined by
considering the scope, objectives, criteria, relevant actors, and alternatives [13]. Priority is a value that determines
the level of importance of an alternative or criteria [14]. The AHP defines pairwise comparison to determine
priorities using a matrix to compare variable of the same level in pairs. Comparisons were implemented using the
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
408
4
Saaty preference scales [13].
Priority weighting of each criteria or alternative is calculated using the Eigen value principle [10]. The calculation
could be done with the Expert Choice AHP template Tool. Consistency checking is performed to determine the
likelihood of conflicting inputs. The inconsistency value should not be more than 10% [13].
Table 3. Summary of criteria and sub criteria evaluation
Sub criteria
Number of respondent approval
Sub criteria status in this study
Team distribution (SK01)
1 out of 5
eliminated
Number of team members (SK02)
5 out of 5
used
Team skills (SK03)
5 out of 5
used
Team member turnover (SK04)
5 out of 5
used
Clarity of requirements (SK05)
5 out of 5
used
Requirement changes (SK06)
5 out of 5
used
Documentation (SK07)
5 out of 5
used
Software usage time (SK08)
1 out of 5
eliminated
Application type (SK09)
4 out of 5
used
Organization culture (SK10)
4 out of 5
used
Organization standard (SK11)
0 out of 5
eliminated
User involvement (SK12)
4 out of 5
used
External Policies (SK13)
4 out of 5
used
Cost (SK14)
5 out of 5
used
Time to Deliver (SK15)
5 out of 5
used
4 out of 5
used
Criteria 1: Personnel
Criteria 2: Requirement
Criteria 3: Application
Criteria 4: Organization
Criteria 5: Operational
Criteria 6: Business
Criteria 7: Technology
Technology availability (SK16)
3. Research methods
This study discusses and evaluates the software development process in Kemlu. The respondents from this
research are:
•
•
•
•
•
Head of the Communication Center (R1)
Former Head of the Information System Development and Management Division (R2),
Head of the Network and Information System Application Subdivision (R3),
Head of the News Documentation Subdivision (R4), and
Two staff of the Information and Communication System Development and Management Division (R5 and R6).
The respondents were chosen because their positions could determine the policy direction of software
development in Kemlu, as well as understand the process. The respondent selection process uses the approach
exercised in the literature [8, 11, 12]. The respondents fill out a questionnaire, while the resulted data would be
processed further with the AHP to determine the suitable software development method for Kemlu.
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
409
5
Fig. 2. Flow of research.
The results of the study were then analyzed and commented by the following stakeholders:
• Head of Civil Section, Citizens Protection in Foreign Countries Sub directorate, Citizens Protection and Legal
Entities Directorate (Stakeholder 2),
• Head of Aviation, Shipping, and Legalization Sub-Directorate, Consular Directorate (Stakeholder 3), and
• Staff of Information Systems Development and Management and Information Systems Division (Stakeholder 4)
The research flow is shown in Fig. 2, which consists of twelve stages. Stage 5 to 11 is the data processing using
AHP. The AHP pairwise comparison would be carried out by using Expert Choice 11.
4. Result
Variables in the theoretical framework, i.e. criteria, sub criteria, and alternatives as given in Table 1 and Table 3
are evaluated by all respondents through the interview sessions. The variables are maintained in the further analysis
if the majority of respondents, i.e. 3 of 5 respondents, provide the same respond. Based on the answers of each
respondent there is no need to change the criteria. However, we needed to eliminate some sub criteria, i.e. team
distribution, software usage time, and organizational standard, since the number of respondents who approve the sub
criteria does not reach the majority.
Based on the answers of each respondent, all of the proposed alternatives are approved as summarized in Table 4.
The variables are further arranged to form the AHP hierarchy structure as shown in Fig. 3.
Table 4. Summary of evaluation of the alternatives.
Alternatives
Number of respondent approval
(out of 5 respondents)
Status of the Alternative in This Study
Waterfall
4 out of 5
used
Incremental
5 out of 5
used
Prototyping
5 out of 5
used
XP
5 out of 5
used
Scrum
5 out of 5
used
RUP
5 out of 5
used
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
410
6
Fig. 3. Hierarchy structure for the AHP.
The hierarchy shows 6 software development method alternatives to be selected based on the criteria and sub
criteria to find the most suitable alternative. The questionnaire is prepared based on the hierarchy. The questionnaire
was distributed to the respondents (R2, R3, R4, R5, and R6). The results of questionnaires that have been processed
using the AHP method are shown in Fig. 4, Fig. 5, Fig. 6, and Table 5.
Table 5. Priority of software development methods in each sub criterion.
Sub criteria
Waterfall
Incremental
Prototyping
XP
Scrum
RUP
Cost
0,011
0,021
0,026
0,016
0,01
0,011
Clarity of requirements
0,01
0,012
0,021
0,013
0,008
0,014
Organization culture
0,006
0,012
0,02
0,009
0,009
0,008
Technology availability
0,005
0,005
0,012
0,005
0,004
0,005
External Policies
0,004
0,005
0,01
0,003
0,003
0,003
User involvement
0,004
0,008
0,008
0,004
0,004
0,007
Application type
0,002
0,004
0,007
0,003
0,003
0,004
Requirement changes
0,002
0,006
0,008
0,004
0,009
0,006
Time to Deliver
0,002
0,004
0,007
0,003
0,002
0,002
Documentation
0,004
0,004
0,003
0,003
0,004
0,006
Team Skill
0,001
0,002
0,002
0,001
0,001
0,001
Number of team members
0,004
0,003
0,006
0,004
0,002
0,002
Team member turnover
0
0,001
0,001
0,001
0
0
Table 5 shows that Prototyping the highest weight in several sub criteria: Cost, Clarity of requirements,
Organization culture, External regulation, Application type, Time of delivery, Availability of technology, Team
skill, as well as the number of team members. Scrum gets the highest weight in the Change of requirements sub
criteria. Incremental gets the highest weight for the User involvement and Team member turnover sub
criteria. Meanwhile, RUP gets the highest weight in the Documentation sub criteria.
Fig. 4 shows that the highest-weighted criterion is Business with a value of 24.5%. The consistency ratio of the
pairwise comparison of these criteria is 6% (less than 10%), which means the comparison is consistent and in
accordance with the Saaty rules so that the results are acceptable [13].
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
411
7
Fig. 5 shows that the cost sub criterion in the Business criteria is the highest priority with a weight of 9%,
whereas the sub criterion for the change of team members on the personnel criteria becomes the lowest priority with
a weight of 0.4%.
Fig. 6 shows that the consistency ratio in this whole study is 4% (less than 10%) so that the results are
acceptable. The alternative with the highest weight is Prototyping with a weight of 28%. Therefore, this study
concludes that Prototyping method is the most suitable software development method in Kemlu.
Fig. 4. Priority of the criteria.
Fig. 5. Priority of the sub criteria.
Fig. 6. Priority of the alternatives.
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
412
8
5. Discussion
Kemlu has some special problems in the software implementation compared to the other organization. Some
problems are explained in the last column of Table 6. The absence of a software development standard in Kemlu
leads to the unclear decision about which method to use whether the agile software development method or the
traditional one. On the other hands, some organizations have narrowed down their software development method
options to agile methodology [15]. Moreover, Kemlu has not yet implemented that every IT staff should handle a
special skill, for example analyst, designer, quality assurer, programmer, etc. The informal communication within
the organization also increases the complexity of the multi-criteria decision making to obtain the fittest software
development method for Kemlu.
The justification of the research results is collected and analyzed from the Performance Report of
Communication Center (Puskom) of Kemlu in 2015, interviews with respondents, and from the meeting notes about
the planned activities of 2017. The analysis is given in Table 6 about some correlation between the characteristics of
Prototyping with the software development circumstances in Kemlu.
In the sub criteria evaluation results in Table 4, the respondents state that team distribution is not necessary
because team members are in the same location, the time span of software use is not necessary to consider because
the length of time a software to be used does not matter as long as there is documentation and standard. The
organization could not be used as a basis for consideration because Kemlu does not currently have a standard on
software development process.
Fig. 5 shows that the cost sub criterion is the highest priority. On the other hands, the Team member turnover sub
criterion is the lowest priority. In the literature, cost factor is one of the main factors to determine the software
development method in government organizations. This is in accordance with the opinion of the respondents that
cost should be ensured in any software development. Therefore, cost is a sub criterion with the highest priority on
software development in Kemlu.
According to the respondents, the team member turnover factor is considered as important issue in Kemlu.
However, if a software development standard procedure is applied then the software development would still run
well despite the change of team members. Therefore, team member turnover is a sub criterion with the lowest
priority on software development in Kemlu.
Table 6. The Suitability of prototyping with kemlu circumstances on each sub criterion.
Subcriteria
Prototyping characteristic
Kemlu needs in terms of software development
RUP
Cost
Reduce the cost of repairing software16
Use the budget optimally (interview result and
document of Performance Report of Puskom 2015)
Clarity of requirements
Clarify the requirement17
Analyze the user's request (interview)
Organization culture
Use verbal communication to gather requirements17
Use informal communication in software
development requests (interview)
Technology
availability
Need technology which could help to build the
prototype
Could provide software development tools, for
example bootstrap template to help the process of
building the prototype (interview)
External Rules
Involve the collaboration of various organizations18
Involve other organizations that have different
regulations (interview)
Application type
Could model different versions of the software18
Improve the previous version of the software
(interview)
Time to Deliver
Start software development immediately after the
general description of user needs is captured so as to
provide ease of completion time prediction16
Produce the software in the agreed time
(interview).
Team Skill, Number of
Team Member
A team member should have the ability to act as
analyst, designer, quality
assurer, programmer, technical writer, UX designer3
The team in Kemlu are currently not in accordance
with these skills, but for 2017 Kemlu plans to carry
out the distribution of roles within the team
(interviews and activity plan meeting notes 2017)
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
413
9
Prototyping has advantages and disadvantages [3,16,17]. Prototyping could reduce the cost of software
development, especially at the stage of testing and management software. Prototype implementation and refinement
repeatedly based on the user feedback help to detect errors and deficiencies earlier in the software development
process so that cost for redesign is reduced. Prototyping also reduces misunderstanding between developers and
users. The users get an overview of the software, so they could give feedback or additional requirements to the
software. The developers implement the software as soon as the user's needs are captured. Prototyping improves the
quality of software in terms of functional adequacy, ease of use, possible modifications, and the ease of further
development, suitability and reliability. Prototyping could also help to clarify the requirements, so that the time of
completion could be better predicted to improve the project scheduling.
Prototyping also has some drawbacks. The development process could lead to scope creep, because each of the
user’s requests is fulfilled, users keep asking more features and more changing. To overcome this, it is necessary to
apply project management and the definition of the software scope clearly18.
Another downside of Prototyping is that the user might assume that the prototype which has been used to get
feedback is the working software that is ready to use. It could be difficult to explain to the user why the software
should be developed further. For this case, the developers should explain from the beginning that the prototype is
not ready-to-use software17. This should be understood and approved by all parties involved in software
development. The prototype is built to define more clear requirements, so it needs further development to provide
the ready-to-use software.
Some stakeholders (Stakeholder 2, Stakeholder 3, and Stakeholder 4) are asked to comment about the result of
this study via some interviews. The stakeholders say that the developer team should clarify about some software
change requests coming after the testing stage17, for example some additional features are requested when the
software is disseminated to wider users who were not involved since the initial process. The late change request
could be facilitated in next version of the software development18.
6. Conclusion
This study concludes that Prototyping is the most suitable software development method in Kemlu. The criteria
and sub criteria used to determine the method are available in Fig. 3. The obstacles faced by Kemlu when
using Prototyping is the difficulty of determining the completion time of the software development because changes
are occurred continuously as the requirements are keep changing. Prototyping needs to go together with a
determined software scope and a good project management. Another obstacle that should be anticipated is the
software change requests at the end of the development process. The solution is that such request could be
accommodated in the next version of the software development.
Acknowledgements
This research was supported by a PITTA grant at the University of Indonesia (398/UN2.R3.1/HKP.05.00/2017)
References
[1] D. Galin, Software quality assurance from theory to implementation, Harlow: Pearson, 2004.
[2] I. Sommerville, Software engineering 9th edition, Boston: Pearson, 2011.
[3] J. Arnowitz, M. Arent and N. Berger, Effective prorotyping for software makers, San Francisco: Morgan Kaufmann, 2007.
[4] A. Dennis, B. H. Wixom and R. M. Roth, System analysis and design fifth edition, John Wiley & Sons, Inc., 2012.
[5] P. Wallace, Introduction to information system, New Jersey: Pearson Education, Inc., 2015.
[6] N. Nikitina and M. Kajko-Mattsson, "Guiding the adoption of software development methods," in ICSSP 2014 Proceedings of the 2014
International Conference of Software and System Process, Nanjing, 2014.
[7] P. Clarke and R. V. O'Connor, "The situational factors that affect the software development process: towards a comprehensive reference
framework," Journal of Information and Software Technology, Vol. 54, Issue 5, Elsevier, pp. 433-447, 2012.
[8] H. Hajjdiab and A. S. Taleb, "Agile adoption experience: a case study in the UAE," 2nd International Conference on Software Engineering
and Service Science (ICSESS), IEEE, 2011.
414
10
Melisa Helingo et al. / Procedia Computer Science 124 (2017) 405–414
Melisa Helingo et al. / Procedia Computer Science 00 (2018) 000–000
[9] P. Khan and S. M. Beg, "Extended decision support matrix for selection of sdlc-models on traditional and agile software development
project," Third International Conference on Advanced Computing and Communication Technologies, IEEE, 2013.
[10] M. A. Khan, A. Parveen and M. Sadiq, "A method for the selection of sdlc models using analytic hierarchy process," International
Conference on Issues and Challenges in Intelligent Computing Techniques, IEEE, pp. 534-540, 2014.
[11] M. Hicdurmaz, "Fuzzy multi criteria decision making approach to software life cycle model selection," 38th Euromicro Conference on
Software Engineering and Advanced Applications, IEEE, pp. 384-391, 2012.
[12] C. Fulgham, J. Johnson, M. Crandall, L. Jackson and N. Burrows, "The fbi gets agile," IT Professional, Vol. 13, Issue 5, IEEE, pp. 57-59,
2011.
[13] T. L. Saaty and L. G. Vargas, Models, methods, concepts, and application of the analytic hierarchy process, New York: Springer, 2012.
[14] A. Ishizaka and P. Nemery, Multi-criteria decision analysis methods and software, Chichester: John Wiley and Sons, 2013.
[15] B. Sayed, Z. Shamsi and M. Sadiq, "A Method for the Selection of Agile Methods Using AHP," in International Conference on Frontiers in
Intelligent Computing: Theory and Applications, Singapore, 2017.
[16] W. Bischofberger and G. Pomberger, Prototyping-oriented software development concepts and tools, Heidelberg: Springer-Verlag, 1992.
[17] R. S. Pressman and B. R. Maxim, Software engineering: a practiotioner's approach 8th edition, New York: McGraw-Hill, 2014.
[18] X. Zhang, Z. Fu, W. Cai, D. Tian and J. Zhang, "Evolutionary prototyping model in developing fidss: an intelligent decision support system
for fish disease/health management," Expert System with Application, Vol. 32, Issue 2, Part 2, Elsevier, pp. 3091-3919, 2009.