Enhancing Agile Development Approach for Cloud Services Ecosystem Security and Dependability S. Hassan Adelyar adelyar@tlu.ee Institute of Informatics, Tallinn University Abstract Agile digital services development processes are used by industry to produce functionally correct digital services as quickly as possible. However, agility in the digital services development process does not embrace secure- and dependable practices, and to some extent, security and dependability has not been given the attention it needs when developing digital services with agile methods. Security and dependability are important and critical properties of digital services. But achieving security and dependability in digital services is a challenging task. This is because of the complexity and continuous change of digital services. The diversity in the form of vulnerabilities and threats add the security and dependability challenges in digital services ecosystem. The aim of this article is to enhance the Agile Development Approach for Cloud Services Ecosystem with a special focus on the quality goals of security and dependability. We employ the design-science method as an adopted research method. Keywords Software Security, Agile Development Methods, Cloud Services, Vulnerabilities, Threats, Attack, Risk. 1 Chapter 1: Introduction Software systems are common components of our daily life and the success of these systems depends greatly on its security and dependability. Therefore, security and dependability are important because so many critical functions have come to be completely dependent on software system. This makes software a very highvalue target for attackers, whose motives may be malicious, criminal, or adversarial. What makes it so easy for attackers to target software is the virtually guaranteed presence of vulnerabilities in the software systems, which can be exploited to violate one or more of the software’s security properties, or to force the software into an insecure state (Karen Goertzel, 2008). Software security has three aspects which are the preservation of the confidentiality, integrity, and availability of the software. Confidentiality refers to the prevention of unauthorized discovery and leak of information. Integrity means the prevention of unauthorized modification of information. Availability is the prevention of unauthorized destruction or denial of access or services (Algirdas Avizienis, 2004). Dependability is the ability of software to deliver trusted services to its users. Dependability includes the security aspects (confidentiality, integrity, availability) plus reliability, maintainability and safety. Reliability is the ability of software to deliver services as specified. Maintainability is software feature that allows the software to emerge new requirements and can be adapted to new changes. Safety is the ability of software to operate without disastrous failure (Algirdas Avizienis, 2004). 2 Today many software industries use agile development methods for developing software. Microsoft, one of the world wide popular software-company, also uses agile software development methods to build their applications (Microsoft, 2009). The reason for the broad usage of agile development methods by software industries refer to the flexibility of agile methods for developing today software systems. Agile development methods allow software developer to incorporate the new requirements into the software in a flexible and quick manner. Major features of agile development include the iterative and incremental development, reflective process improvement, customer participation, and a high level of communication. All agile methods are characterized by having a single goal, which is to produce functionally correct digital services as quickly as possible. However, these methods do not embrace secure service-development practices, and to some extent, security and dependability has not been given the attention it needs when developing digital services with agile methods (Bejan, 2011). Cloud computing is a model for enabling on-demand network access to a shared pool of computing resources such as networks, servers, storage, applications and services. These services can be rapidly provisioned and released with minimal management effort or cloud provider interaction (Bryan, 2013). Cloud ecosystem is a term used to describe the complex system of interdependent components that work together to enable cloud services (Bryan, 2013). Historically, security has not been given the attention it needs when developing software with agile methods. Therefore, security is the main concern with agile methods. Since agile process do impose limitations on the software development process. Some of the limitations of agile development methods are: 3 It is no longer possible to create a complete picture of a product as all requirements are not yet known. This lack of a complete overview prevents some common security engineering process from being performed in agile project. Using agile methods the system is developed by producing individual components of functionality, then integrating these components together. However, software security is a whole-system property, and even if individual components are secure, the combination of those components will not necessarily result in a measurably secure software system (Karen Goertzel, 2008). Therefore, there is a perception today that agile methods do not create secure code, and, on further analysis, the perception is reality. Due to the broad usage of agile methods this perception needs to change. But the only way the perception and reality can change is by actively taking steps to integrate security and dependability requirements into agile development methods (Microsoft, 2009). The main motivation behind this article is to stress that, security should be integrated with software development lifecycle. Our aim is to enhance agile development methods for building-secure and dependable cloud-services ecosystem. Therefore, the main components of our topic include agile-method and cloud-computing-ecosystem. This paper is organized into five chapters. The next chapter includes the literature review of software security. We will have a deep review of our topic in the literature. Chapter 3 includes the analysis and exploration of current approaches and our possible contribution. During the research we will expand the literature 4 review and analyze the finding from the literature review in order to understand the existing status and gap in agile development methods. Chapter 4 discusses the research approach and methodology. Initially our research methodology depends on the design-science paradigm which is a problem-solving process and it is suitable for problems which are characterized by unstable requirements and constraints. Chapter 5 is the summary and conclusion of the article. Finally the related terminology comes at the end of this article. Chapter 2: Literature Review The success of a software system depends on how well it fits the needs of its users and its environment. Software requirements include these needs, and requirements engineering is the process by which the requirements are determined. Successful requirements engineering involves understanding the needs of users, customers, stakeholders, and understanding the contexts in which the software will be used. More importantly requirements engineering involves understanding the constraints on functionality of the software systems. As computing systems become more improved and mobile, and as they automate and manage more critical processes and data, they increasingly become the targets of security attacks. Moreover, attacks originate from both outside the system and inside the system, are intentional, and are constantly changing. For these reasons security poses challenges to requirements engineering that exceed those posed by other nonfunctional requirements (Betty, 2007). The potential for damage by malicious developers is even greater for software based on modern application processing models which is based on service-oriented architectures. In this model of software, users include software processes that 5 operate independent of human intervention. This means malicious logic embedded by malicious developers in such independent software entities has an even greater likelihood of going undetected once that software is operational. The only hope of preventing such insertions during software’s development process is to include a thorough set of reviews of the design and implemented software that include looking specifically for unexpected logic, exploitable faults, and other weaknesses (Karen Goertzel, 2008). Security needs arise when stakeholders determine that some resources belonging to a software system, tangible such as money or intangible like confidential information, is valuable to the organization. Such resources are called assets, and stakeholders wish to protect these assets from any damage or attacks. Security requirements engineering is thus focused on the protection of these valuable assets from the requirements perspective (Chivers, 2005). But Security is uniquely complex and challenging among other non-functional requirements. This difficulty arises because of the following three main reasons (McGraw, 2003): All systems that involve software are complex and complex systems introduce multiple risks. Today software system contains a huge number of code lines which make difficult to analyze the logic and working manner of the program. For example the Windows XP operating system had 40 million lines of code (Greg Hoglund, 2004). The complexity is however not only due to size, but also the structure of the software. In such a large system vulnerabilities remain invisible to unsuspecting users until it is too late. In addition to this the system complexity is continuously rising which makes it difficult to plan for security, as it is an environment that is constantly 6 changing. Programmers often have to face not only the complexity of their own business domain, such as banking, but they also have to deal with concerns such as security. Even techniques of object-oriented software engineering and component based software improved the problem of complexity, but the security concerns have proven difficult to modularize due to their pervasive nature (Greg Hoglund, 2004). Unlike hardware, software is easily extendable and more functionality can be added to the software. However, extensible systems are particularly susceptible to hidden risks and malicious functionality problems. The risk of intentional and unintentional vulnerabilities behavior increases drastically. Programmers can modify systems software that is initially installed on the machine. Users may incorrectly install a program that introduces unacceptable risk. New plug-ins will also introduce weaknesses to the software system. Due to the growing of the web and Internet both the numbers of attacks and the ease with which an attack can be made have increased. Hackers and other intrusion can gain unauthorized access to software systems. More and more computers, ranging from home PCs to systems that control critical infrastructures, are being connected to the Internet. Furthermore, people, businesses, and governments are increasingly dependent on network-enabled communication. The software which is accessible from the Internet made the attacker’s job easier. Attackers can study the success of hundreds attacks on software accessible from the literature, and find the security vulnerabilities of software systems, and as a result they are able to very quickly perform attacks that exploit those specific vulnerabilities (Karen Goertzel, 2008). 7 In addition to the above mentioned reasons, software systems are developed by different people and are affected by different environments such as hardware, software, and stakeholders. The continuous changing of software systems, diversity in the form of vulnerabilities and threats, and since someone is deliberately trying to break the system make software security more complex and difficult. Therefore, securing software systems need continuous efforts and related work. Software Security and Information Society During the last two decades, we have moved from merely using software, to relying on it, and ultimately becoming dependent on software systems, for our day to day lives. We depend on software for many jobs and we also trust the software systems for its accuracy and correctness. In our life when we trust something but it behave unlikely then the risk will be greater and in some cases unrecoverable. In many cases software is at the root of all common computer security problems. If your software misbehaves, a number of diverse sorts of problems can crop up such as reliability, availability, safety, and security. For example malicious hackers don’t create security vulnerabilities but they simply exploit the existing weaknesses in software. Security vulnerabilities are the result of bad software design and implementation. Therefore, addressing security in software development is very important, particularly because of the effects of software security on the society. Today, software security problems are frequent, widespread, and serious and the number and variety of attacks by persons and malicious software from outside organizations, particularly via the Internet, are increasing rapidly (Redwine, 2004). 8 Security is a system characteristic as well as a set of mechanisms that span the system both logically and physically. Therefore, security needs to be considered a property of the system to be implemented. Software flaws and defects during the specification, design and implementation of software can cause the software to behave incorrectly and unpredictably, even when it is used purely as its designers intended. Software is also used in safety critical areas such as medicine, transportation, nuclear power generation, and national defense. Such areas are safety critical, and extremely sensitive to errors. The smallest flaw could have upsetting consequences that can lead to significant damage, including the loss of life. Thus, how much trust can we put in our software, and how much damage can erroneous software cause. We note that software may not be the only cause of all accidents but the causes are a combination of both software and human errors. Difficulties with requirements are the key root cause of the safety-related software errors, which have persisted until integration and system testing (Lutz, 1993). The specification of the software system may be incomplete in that it does not describe the required behavior of the system in some critical situations. Here are two examples for the role of safetycritical software. In September 1993, during a thunder-storm a plane landed at Warsaw airport. After landing the brakes on the computer-controlled braking system did not work because the braking system had not recognized that the plane had landed and assumed that the aircraft was still flying. A safety feature on the aircraft had stopped the deployment of the reverse push system, which slows down the aircraft, because this can be dangerous if the plane is in the air. The plane ran off the end of 9 the runway, hit an earth bank, and caught fire. The inquiry into the accident showed that the braking system software had operated according to its specification. There were no errors in the program. However, the software specification was incomplete and had not taken into account an infrequent situation (Sommerville, 2011). The second example is the event of shutdown of Atlanta International Airport on April 19, 2006. Atlanta International Airport is one of the world’s busiest airports. The alertness of the security screeners is tested by the random appearance of artificial bombs or other suspicious hard-to-detect devices on the X-ray machine displays. The test is followed by a brief delay, then a message indicating that it was a test. On April 19, 2006, an employee of the Transportation Security Administration identified the image of a suspicious device, but did not realize it was part of the routine testing for security screeners because the software failed to indicate such a test was underway. As a result, the airport authorities evacuated the security area for two hours while searching for the suspicious device, causing more than 120 flight delays, and forcing many travelers to wait outside the airport (Vidroha, 2009). Chapter 3: Current Approaches and Strategies Computer systems security and dependability also rely on countermeasures at the operating system, network technologies, database, and web server levels. But relying only on this type of security has two important shortcomings. First the security of the application depends completely on the robustness of the wall of protections that surround it. Secondly, the defense itself has exploitable development faults and other weaknesses as the application software they are protecting (Karen Goertzel, 2008). 10 Therefore, software specification, design and implementation as the major steps of software development are essential for successful and secure software system. Error in these phases will continue to other phases of the software. Therefore, software security needs to be considered as a property of the system and it needs to be dealt with since the inception of the software life cycle (Karen Goertzel, 2008). Threats to software may be present throughout its life cycle, during its development, deployment, and operation. For software in development and deployment, most threats will be insider threats which come from the software’s developers, testers, configuration managers, and installers or administrators. The threats they pose may be unintentional, intentional but non-malicious, or intentional and malicious. Unintentional threats can occur during the development, deployment and operation of the software. For example the developer may ignore some specification or the programmer may ignore the developer consideration during the coding. Intentional threats can be malicious or not malicious. Intentional but not malicious threats can be from the developer, programmer or operators. For example the programmer may ignore functionality during coding because of timeline. Intentional and malicious threats can also be from the developer, programmer or operators. For example programmer may intentionally include exploitable flaws and backdoor in the code. Because of the software complexity and importance of the software development lifecycle, there is substantial work on how to improve software security, in the form of solutions and strategies during the requirements engineering process to avoid vulnerabilities and threats, and protect the software and its data from attacks. However, no matter how faithfully a security-enhanced life cycle is adhered to, as 11 long as software continues to grow in size and complexity, some number of exploitable faults and other weaknesses are sure to exist. The nature of the threat is changing more rapidly than the software’s artifacts can be adapted to work against those threats. It is clear and without doubt that to be 100% effective, defenders must anticipate all possible vulnerabilities, while attackers need find only one (Karen Goertzel, 2008). It is only recently, that the idea of considering security in the development life cycle of a system has become popular, as traditional requirements engineering is not enough (Firesmith, 2007) and the requirements engineering can provide great support for ensuring that security is built into a software system (Lewis, 2002). Therefore, there is substantial work on how to improve software security during software development lifecycle that avoid vulnerabilities, protect software systems, and defend against or recover from attacks. Although, vulnerabilities and threats are not a new topic on software field, they still appear in the software system, and it need a continuous efforts and methods how to deal with these threats and vulnerabilities (Betty, 2007). Security controls in the software’s life cycle should not be limited to the design, implementation, and test phases. It is important to continue performing code reviews, security tests, and strict configuration control during the post-release support phase, to ensure that updates and patches do not add security weaknesses or malicious logic to existing software products (Karen Goertzel, 2008). The key elements of a secure software development life cycle process include the following areas: 12 Adequate requirements elicitation, derivation, and specification of the software. This process includes complete and consistent requirements for constraints on the software’s functionality and behavior. The architecture and design are carefully reviewed to ensure that they reflect correct developer assumptions about all possible changes that might arise in the software’s environment. Secure coding and integration of software components. Coding follows secure coding practices and adheres to secure coding standards. Static security analysis of code is performed iteratively throughout the coding process, to ensure that security issues are found and eliminated before code is released for unit testing and integration. Appropriate security-oriented reviews and tests are performed throughout the software development life cycle. Tests plans include scenarios for abnormal conditions among anticipated conditions under which the software may operate. Secure software configuration management and version/change control of the development artifacts as a countermeasure against subversion of those artifacts by malicious developers, testers, or others. Maintenance, vulnerability management, and patch issuance and distribution conform to secure sustainment principles and practices. Software customers are encouraged to apply patches and keep software updated, to minimize unnecessary exposure of vulnerabilities. Our Possible Contribution Agile software development methods have become increasingly popular. These methods take a less formal approach to software development which emphasize on 13 small teams, iterative development and quick customer feedback. This is also important to include security engineering activities when applying agile methods. The aim of this research is to enhance the agile development method for building secure and dependable cloud-services ecosystem. The main motivation behind this research is to enhance agile development approach for cloud services ecosystem security and dependability. Therefore, the main components of our topic include agile-method and cloud-computing-ecosystem. In this research, we intend to evaluate security and dependability requirements dynamically in agile development method, identify vulnerabilities and avoid security- and dependability errors caused by service-developer mistakes in carrying out agile methods. We believe that agile development methods for cloud services ecosystem security and dependability can be enhanced by evaluating the security and dependability requirements. Identifying vulnerabilities and avoiding developer mistakes in agile development methods help to enhance the agile development method for building secure and dependable cloud-services ecosystem. Research Questions For achieving the aim our research we will address the gap with the following three main questions: How to evaluate security and dependability requirements dynamically in agile development method for building secure and dependable cloud services ecosystem? How to identify vulnerabilities in agile development method for building secure and dependable cloud-services ecosystem? 14 How to avoid security- and dependability errors caused by service-developer mistakes in carrying out agile methods while building cloud-services ecosystem? In order to simplify the questions we decompose each of the above questions. For doing this for each “How” question we have 3 “What” questions. This will help us to precisely find the answer for each question systematically. The following are the sub-questions for each of the above main questions: Q1: How to evaluate security and dependability requirements dynamically in agile development method for building secure and dependable cloud services ecosystem? o Q1.1: What is the effective way to test quality requirements in agile method for building secure and dependable cloud services ecosystem? o Q1.2: What is the effective way to define the acceptance test cases that verify the effectiveness of the specified preventive measures? o Q1.3: What is the impact of code testing on agile method for building secure and dependable cloud services ecosystem? The expected result from this part of our research will be a dynamically tested and evaluated agile development method for building secure and dependable cloud services ecosystem. Q2: How to identify vulnerabilities in agile development method for building secure and dependable cloud-services ecosystem? o Q2.1: What vulnerabilities during functional requirements change of agile method impede building secure and dependable cloud services ecosystem? o Q2.2: What vulnerabilities in the late lifecycle of agile method impede building secure and dependable cloud services ecosystem? 15 o Q2.3: What is the effective way to identify misuse stories from use cases in agile method for building secure and dependable cloudservices ecosystem? After completing this part of our research, we expect to identify vulnerabilities in agile development approach in order to build secure and dependable cloud-services ecosystem. Q3: How to avoid security- and dependability errors caused by servicedeveloper mistakes in carrying out agile methods while building cloudservices ecosystem? o Q3.1: What vulnerabilities are the result of service-developer mistake in agile development method for building secure and dependable cloud services ecosystem? o Q3.2: What is the effective way to avoid service-developer mistakes in agile methods for building secure and dependable cloud services ecosystem? o Q3.3: What is the effective way to add new services using agile method for building secure and dependable cloud services ecosystem? By completing this part of our research, we contribute to the enhancement of agile development approach through avoiding service-developer mistakes. Avoiding service-developer mistakes has an important role for building secure- and dependable cloud services ecosystem. Chapter 4: Methodology Our research methodology depends on the deep study and investigation for evaluating security and dependability requirements dynamically in agile development method, identifying vulnerabilities and avoiding security- and dependability errors caused by service-developer mistakes in carrying out agile 16 methods. The first phase of our research is to conduct a comprehensive review of literature for the security and dependability of agile development methods. For the evaluation of the result we are using the design science framework and guidelines. Through evaluation we aim to improve the quality of the research and provide feedback information. The security and dependability attributes of software will be evaluated in terms of functionality, completeness, consistency, accuracy and reliability. To achieve rigor in our research, the existing methodologies and foundation will be used. For the quality of the result we will use descriptive, analytical and experimental methods. Further evaluation of the result will be performed via observational method / case study, in an organizational context, using qualitative methods. During evaluation of the quality and effectiveness of the result we will use computational and mathematical methods. We will also gather the related facts and analyze the facts by defining a set of appropriate metrics. Work Plan Activities Courses for 32 Credits Individual Research (Introduction & bridge chapter) Courses for 2014 Jan Apr – Mar Dec 2015 Jan – Apr Mar Dec X X X 17 2016 Jan Apr Mar Dec 2017 Jan Apr Mar Dec 2018 Jan 22 Credits Individual Research ( X Main part 1) Courses for 4 Credits Individual Research (Main part 2) Individual Research (Main part 3) Composition of the Thesis, Preliminary defence The Defence of the Thesis Note: The activities may have time overlapping. X X X X X Chapter 5: Conclusion Today many software industries use agile development methods for developing software. The reason for the broad usage of agile development methods refer to the flexibility of agile methods for developing software. Agile development methods allow software developer to incorporate the new requirements into the software in a flexible and quick manner. Major features of agile development include the iterative and incremental development, reflective process improvement and customer participation. However, agile methods do not embrace secure service-development practices. Historically, security has not been given the attention it needs when developing software with agile methods. Therefore, security is the main concern with agile methods. 18 Therefore, there is a perception today that agile methods do not create secure code, and, on further analysis, the perception is reality. Due to the broad usage of agile methods and the importance of software security and dependability this perception needs to change. The only way the perception and reality can change is by actively taking steps to integrate security and dependability requirements into agile development methods (Microsoft, 2009). Our aim is to enhance the agile development method for building secure and dependable cloud-services ecosystem. We believe that agile development methods for cloud services ecosystem security and dependability can be enhanced by evaluating the security and dependability requirements. Identifying vulnerabilities and avoiding developer mistakes in agile development methods help to enhance the agile development method for building secure and dependable cloud-services ecosystem. For conducting our research, we will apply the design-science method which is a problem-solving paradigm. In this paradigm, we define security- and dependability discipline into design science boundaries via a conceptual framework and following the guidelines of design science research (Alan R. Hevner, 2004). Computational methods will be used to evaluate the quality and effectiveness of our research. We will define a set of metrics and gather facts and then we will analyze the facts. Further evaluation of our research will be done in organizational context through case study. Bibliography 19 1. Alan R. Hevner & Sudha Ram, Design Science in Information System Research. College of Business Administration, University of South Florida Tampa, U.S.A, 2004. 2. Algirdas Avizienis, Fellow, IEEE, Jean-Claude Laprie, Brian Randell, & Carl Landwehr, Senior Member, IEEE. Basic Concepts and Taxonomy of Dependable and Secure Computing, 2004. 3. Bejan Baca (2011). Agile Development with Security Engineering Activities. ACM, USA. 4. Bryan Sullivan (2013). Practices for secure development of cloud applications. Cloud Security Alliance, Microsoft, USA. 5. Betty H.C. Cheng & Joanne M. Atlee, Research Directions in Requirements Engineering. Future of software engineering, IEEE, 2007. 6. Chivers, H. & Fletcher, M.: Applying Security Design Analysis to a service based system. Software Practice and Experience, vol. 35 no. 9. 2005. 7. Firesmith, D.: Engineering Safety and Security Related Requirements for Software Intensive Systems, ICSE Companion, 2007. 8. Greg Hoglund, Gary McGraw, Exploiting Software : How to Break Code, Addison-Wesley Professional (February 17, 2004) 9. Karen Mercedes Goertzel, Security in the Software Lifecycle, Department of Homeland Security, National Cyber Security Division, 2006. 10.Ian Sommerville, Software Engineering Ninth Edition, Addison-Wesley, USA, 2011. 11.Lewis, R., Design for Security Up Front. http:// articles. Techrepublic.com/ 5100-10878-1059545.html, 2002. 12. Lutz, R. R., Analyzing Software Requirements Errors in Safety-Critical Embedded Systems. RE'93, San Diego, Calif: IEEE, 1993. 20 13. McGraw, G., Software Security: Thought Leadership in Information Security. Cigital Software Security Workshop, 2003. 14. Software Test & Evaluation Panel (STEP), Requirements Definition Implementation Team: Operational Requirements for Automated Capabilities, Draft Pamphlet (Draft PAM). 1991. 15. Security Development Lifecycle for Agile Development. Microsoft Corporation, 2009. Appendices Appendix A: Terms and Definitions Accountability: Recording and tracking of all the security related actions with the attribution of responsibility is called accountability (Karen Goertzel, 2008). Attack: Attempt to gain unauthorized access to the system’s services or to compromise the system’s dependability (Karen Goertzel, 2008). Availability: The degree to which the services of a system are operational and accessible when needed by their authorized users (Karen Goertzel, 2008). Confidentiality: The prevention of unauthorized discovery and leak of information (Algirdas Avizienis, 2004). Countermeasure: An action, device, procedure, technique, or other measure that reduces the vulnerability of a system (Karen Goertzel, 2008). Dependability: The ability to deliver service that can justifiably be trusted. It is an integrating concept that encompasses availability, reliability, safety, integrity, confidentiality and maintainability (Algirdas Avizienis, 2004). Error: The deviation of a service failure is called error. In other word, a failure occurs when the error causes the delivered service to deviate from correct service (Algirdas Avizienis, 2004). 21 Failure: A failure is an event that occurs when the delivered service deviates from correct service (Algirdas Avizienis, 2004). Fault: The cause of the error has been called a fault (Algirdas Avizienis, 2004). Integrity: The quality of software that reflects its logical correctness, reliability, completeness, and consistency (Karen Goertzel, 2008). Maintainability: Software feature that allows the software to emerge new requirements and can be adapted to new changes (Ian Sommerville, 2011). Non-repudiation: The availability and integrity of the identity of the sender or receiver of a message (Algirdas Avizienis, 2004). Reliability: The ability of a system to deliver services as specified (Ian Sommerville, 2011). Requirements Engineering: The process by which the functional and nonfunctional requirements of the software are determined (Ian Sommerville, 2011). Safety: The ability of a system to operate without catastrophic failure (Ian Sommerville, 2011). Security: Security is a composite of the attributes of confidentiality, integrity, and availability (Algirdas Avizienis, 2004). Threat: Any entity, circumstance, or event with the potential to harm the software system or component through its unauthorized access (Karen Goertzel, 2008). Vulnerability: An internal fault that enables an external fault to harm the system (Algirdas Avizienis, 2004). 22