SoftwareX 19 (2022) 101126 Contents lists available at ScienceDirect SoftwareX journal homepage: www.elsevier.com/locate/softx Original software publication Ansible execution control in Python and Golang for cloud orchestration ∗ David Badalyan , Oleg Borisenko Ivannikov Institute for System Programming of the RAS, Moscow, Russia article info Article history: Received 12 February 2022 Received in revised form 31 May 2022 Accepted 9 June 2022 Keywords: Ansible Cloud orchestration Function hooking Golang Python a b s t r a c t The Ansible configuration manager is currently one of the most popular systems for software deployment. However, Ansible is difficult to debug when working with large scenarios and is difficult to embed into other systems. We propose the cotea (Python) and gocotea (Golang) tools that allow users to control Ansible execution programmatically, by iterating over the tasks and collecting progress information. We additionally propose gopython — a solution for embedding arbitrary Python code into a Golang application. The approach used was generalized up to the abstract architecture of the software tool, which allows for the control of an arbitrary Python program execution. © 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Code metadata Current code version Permanent link to code/repository used for this code version Permanent link to Reproducible Capsule Legal Code License Code versioning system used Software code languages, tools, and services used Compilation requirements, operating environments & dependencies If available Link to developer documentation/manual 0.2 https://github.com/ElsevierSoftwareX/SOFTX-D-22-00040 Apache-2.0 License git Python, Ansible Python 3+ and Ansible have to be installed. Tested on Ubuntu 20.04 with Ansible 2.9.4 and 2.12.2 README.md and docs/ on the main page: https://github.com/ispras/cotea dbadalyan@ispras.ru Support email for questions 1. Introduction The deployment stage is a crucial part of the systems development life cycle in modern times. Additionally, the deployment process is a concern for cloud engineers and users. Infrastructureas-Code (IaC) is a practice to specify and automate the environment in which a software system will be deployed [1,2]. Systems that implement this approach allow one to describe a scenario containing specific deployment strategies and provide mechanisms to determine the strategy to be executed. The strategies can contain various steps, ranging from downloading the necessary packages to running specific commands on computational nodes. One of the most popular systems of this kind is Ansible [3]. Ansible was used by the National Aeronautics and Space Administration (NASA) to reduce its multi-day patching process to 45 minutes [4].One use case for Ansible is embedding in other ∗ Corresponding author. E-mail addresses: dbadalyan@ispras.ru (David Badalyan), borisenko@ispras.ru (Oleg Borisenko). systems [5]. Ansible is essentially a black box when used this way, as there is only a command-line interface. The exception is embedding Ansible into Python systems, using ansible-runner [6]. However, ansible-runner does not allow one to control Ansible execution. It only provides the ability to get information about the events that occur during the execution of Ansible without the ability to stop Ansible at specific points. Moreover, Ansible execution control is significant when embedding Ansible into other systems to automate the determination of the current state of Ansible execution, as well as make dynamic decisions at specific points in script execution. When running Ansible automatically inside other systems, the following information and features may be extremely helpful: • execution results of specific plays and tasks (parts of Ansible scripts); • error messages; • value of Ansible variables and facts; • the ability to make dynamic decisions at certain execution points. https://doi.org/10.1016/j.softx.2022.101126 2352-7110/© 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). David Badalyan and Oleg Borisenko SoftwareX 19 (2022) 101126 Working with Ansible (and with other IaC tools) is also associated with other problems. There is currently a high demand for instruments to assist developers in quickly developing and maintaining infrastructure code [7,8]. There are many papers in which authors attempt to create such instruments [9–12]. The debugging process is a crucial part of developing and maintaining infrastructure code, a relevant issue for modern IaC developers [8]. Existing Ansible debugging tools assume interactive user-Ansible communication [13,14], which is challenging to use with large scenarios. A tool that permits embedding Ansible in other software systems, monitoring its execution, and receiving relevant information could also be used to debug Ansible more flexibly, done programmatically rather than interactively, making it easier to debug large scripts. We encountered difficulties embedding and debugging Ansible while working on our cloud orchestration tool, Michman [15]. Michman is an orchestration system implemented in Golang for deploying and managing clusters with a set of on-demand services in cloud environments, including those in biomedicine. The deployment phase in Michman is based on Ansible. Thus, it became necessary to run Ansible from Golang code, which caused the aforementioned difficulties. Additionally, Michman users can define the desired set of services, which does not allow readymade scripts; this makes these difficulties even more complex by forcing the combination of different Ansible scenarios into one execution. In this work, we propose the cotea [16] and gocotea [17] software tools, developed in Python and Golang, respectively. These tools allow users to launch Ansible programmatically, iterating over Ansible scenario components in sourcing information about the current launch. gocotea is the result of porting cotea into Golang. For porting purposes, a separate tool called gopython [18] was developed, which encapsulates the interaction with CPython API [19]. gopython allows embedding arbitrary Python code into Golang. This solved problem is a special case of a more general issue of controlling the execution of an arbitrary program. The method used for developing cotea can be used to solve this problem in the case of an arbitrary Python program. Python Decorators. Our implementation can be classified as inline hooking [23] that was made famous by Hunt and Brubacher [24]. It modifies the application’s source code and can be used on any platform where Python is executed. Further, our implementation of Python function hooking techniques will be described. 2.1. Decorator classes Using decorators allows embedding a specific program code before and after calling an arbitrary function. At the same time, the input function arguments can be accessed. Due to the openness of Ansible, any methods of Ansible internal objects can be selected as targets for decorators. Further, it is possible to get access to the arguments of these methods, i.e., internal Ansible objects. However, access is only available while the decorator code is executing. In turn, these objects also need to be accessible outside the decorator. To solve this problem, we constructed objects called decorator classes. The decorator classes in our article are not the same as class decorators [25]. A decorator class that can store a reference to the function’s arguments after its execution is shown in Listing 1. 1 class decorator_class : 2 def _ _ i n i t _ _ ( s e l f , t a r g e t _ f u n c ) : 3 s e l f . a r g s _ t o _ s t o r e = None 4 s e l f . func = t a r g e t _ f u n c 5 def _ _ c a l l _ _ ( s e l f , args ) : 6 # pre actions 7 s e l f . a r g s _ t o _ s t o r e = args 8 # function c a l l i s here 9 r e s u l t = s e l f . func ( args ) 10 # post actions . . . 11 return r e s u l t Listing 1: Decorator class As seen from the listing above, the function replacing the source one is the __call__ method. In this method, one can see the call of the source function, the set of actions that preceded the call, and the set of actions that can be performed after the call. Thus, when calling the target function, the decorator class object ‘‘will be called’’ (the __call__ method is the method to be called when calling an object). To replace the original function with the decorator class object, one has to overwrite the function pointer. Eventually, instead of calling the target_func(args) function, decorator_obj(args) will be called, which will lead to a call to the __call__ method. After calling the target function, precisely the args_to_store field of the dec_obj object will contain a reference to the needed argument. The described method can be classified as static technique [26] because we can add instrumented code before the target program has been launched. 2. Software description Cotea and gocotea are Ansible programmatic execution control solutions in Python and Golang, respectively. gocotea is a port of cotea to Golang that was created using gopython, which was separately developed for this purpose. gopython is a solution for embedding arbitrary Python code into a Golang application. Cotea is based on function hooking techniques [20]. This mechanism only intercepts execution at specific points in the program path and gives opportunities to analyze or alter function input and output. There are alternatives choices, namely the self-use of internal Ansible classes and the usage of Python interpreter interfaces. The first involves the novel implementation of core Ansible logic, creating a different tool; this entails many complexities associated with testing and maintenance. If we choose the second method, we will be dependent on a particular version of the Python interpreter, and its implementation. This will eventually lead to complications with maintenance and force one to go down a level, which would further complicate interaction with the Ansible source code. The implementation of function hooking techniques depends on a particular programming language. We needed to hook Python code because Ansible was implemented in Python. Suitable solutions were not found. The primary goal of existing solutions [21, 22] is to audit program execution, and they do not provide interfaces to control program execution. In this work, we present our implementation of a function hooking pattern in Python based on 2.2. cotea architecture In cotea, many decorator classes are set up to the particular Ansible classes methods. cotea runs Ansible in a separate thread and falls asleep during Ansible execution. When Ansible executes one of the decorated methods, the Ansible thread transfers control to the cotea main thread. After control is transferred, cotea methods have access to the corresponding decorator’s object fields. These fields contain all the information about the current Ansible launch. Such a mode of interaction allows one to control Ansible execution programmatically by receiving information about the current launch. 2 David Badalyan and Oleg Borisenko SoftwareX 19 (2022) 101126 2.3. Interfaces gopython provides high-level entities to work with existing Python code, which would correlate with Python language entities, providing an intuitive way for them to interact. Before considering each of the entities provided by gopython, we should note that, in CPython, almost any Python entity is represented in the form of PyObject structure. The provided entities are the following: Cotea contains controlling interfaces and the interfaces that provide information obtained from internal Ansible objects. The controlling interfaces were aimed at the control of execution. With their help, the user should be able to ‘‘program’’ specific behavior, which provides opportunities for dynamic decision-making during execution. They are based on the concepts of task and play (play contains a set of tasks), the constituent parts of any Ansible scenario. The below-listed interfaces allow one to get information about whether there is an unexecuted play or task and lead to the resumption of the Ansible thread, within which the next task is being executed. The interfaces: 1. PythonModule — encapsulates the CPython API calls to be made to create PyObject, which is a module. Using the PythonModule, any object in this module can be imported. 2. PythonClass — the GetClass method of the PythonModule structure returns an instance of the PythonClass structure which encapsulates interaction with PyObject, followed by a Python class. 3. PythonObject — the CreateObject method of the PythonClass structure returns an instance of the PythonObject structure which encapsulates interaction with PyObject, followed by an object of some Python class. Methods of this structure are the following: CallMethod(name, args), HassAttr(name), GetAttr(name), GetType(), IsStandardType(), ToStandardGoType(), CreateFromGoSlice(goSlice), GetGoSliceFromPyList(). 4. PythonMethodArguments — this structure is responsible for passing arguments to the CreateObject and CallMethod methods of PythonClass and PythonObject structures, respectively. • has_next_play() — returns true or false, depending on whether there is still unfulfilled play; • has_next_task() — returns true or false, depending on whether there is still an unexecuted task in the current Play; • run_next_task() — resumes the Ansible thread, leading to the execution of the next task, and returns control, also returning the same as get_last_task_result() (see below); • finish_ansible() — finishes Ansible execution; • schedule_last_task_again() — queues the last running task for re-execution. The presented interface allows executing any Ansible scenarios programmatically, iterating over the main ‘‘bricks’’ of any Ansible scenario, i.e., tasks. The primary components of the methods that provide information about the current execution of Ansible are listed below (see the documentation for a complete list): 2.5. Integration of cotea into Golang Due to gopython, the task of porting a solution developed in Python is greatly simplified. For porting, it is sufficient to create Golang structures corresponding to the main classes of the cotea. These structures will contain the PythonObject structure of the gopython tool, behind which the required Python objects will be hidden. All controlling cotea methods were ported without any additions, as they do not take any arguments and return values of the bool type. Several wrappers have been developed for porting methods that return Python objects that hide interaction with internal Ansible objects. The list of the wrappers is as follows: ErrorFlag(), GetErrorMsg(), GetNextTaskName(), GetCurrentPlayName(). Eventually, a software tool called gocotea was developed. • get_next_task()/get_next_task_name() — returns the instance • • • • • of ansible.task/the name for the next task (there are similar methods for the last executed (previous) task); get_cur_play_name() — returns the name of current play; get_error_msg() — returns Ansible failure message; get_variable(name) — returns Ansible variable with specified name; add_var_as_extra_var(new_var_name, value) — creates Ansible variable with specified name and value as an extra-var; get_last_task_result() — returns a list with TaskResult objects (see documentation for details) where each element contains the results of the last task on each of the hosts of the current group. 3. Usage examples and evaluation 3.1. Cotea and gocotea usage examples The above-listed interfaces were included in one class. This class has fields corresponding to the decorator objects, and each decorator object references an object that implements transferring control between threads. Processing of Ansible task results is often necessary during Ansible playbook development. For example, the arguments within an Ansible task can result from a previous task. Often, one has to extract the necessary arguments from the task results manually; this gives rise to many problems with string processing and type conversion that need to be solved directly in the Ansible runtime. Despite being able to request the result of a task in a convenient format (e.g., JSON), it will still be available from Ansible as a string (‘‘stdout’’ field value), which will require further transformations; this can be resolved using Ansible modules, but there are cases when a required module does not exist, whether at all or for a specific Ansible version. Cotea(gocotea) simplifies this issue by allowing the use of Python (Golang) libraries. Below, there is an example of an internal task where we used cotea to solve such issues. The task was about deploying secret storage, Vault [30], on the host. Ansible 2.9.4 was used (one of the most stable and popular versions). There are no modules for Vault and Podman in Ansible 2.9.4. Tasks that we had to solve in the runtime: 2.4. gopython architecture One of the goals of this work was the ability to embed Ansible into Golang. Porting of the cotea to Golang can be done using the CPython API (using CPythonAPI is carried out with the cgo [27]). While using the CPython API within the task at hand, sections of the program code that were often duplicated were identified. Eventually, the amount of code needed, for example, for class importing, was quite large, which led to development difficulties. The need to translate data from Golang to C also added additional complexity. While searching for existing solutions [28,29], it was found that they do not abstract the user from the CPython API, providing only wrappers over it that permit one not to work with C code directly. Thus, we decided to create an intermediate layer on Golang, called gopython, that encapsulates all the nuances of interaction with the C language and the CPython API. 3 David Badalyan and Oleg Borisenko 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 SoftwareX 19 (2022) 101126 # cotea . runner object i n i t i a l i z a t i o n # r = runner ( . . . ) while r . has_next_play ( ) : while r . has_next_task ( ) : t a s k _ r e s u l t s = r . run_next_task ( ) i f r . get_prev_task_name ( ) == podman_ps_task : task_result = task_results [0] cmd_stdout = t a s k _ r e s u l t . r e s u l t [ " stdout " ] # extracting container s t a t e cmd_stdout_json = json . loads ( cmd_stdout ) s t a t e _ f i e l d = cmd_stdout_json [ 0 ] [ " State " ] r . add_var_as_extra_var ( "VAULT_CONTAINER_STATE" , s t a t e _ f i e l d ) i f r . get_prev_task_name ( ) == v a u l t _ k e y s _ t a s k : task_result = task_results [0] cmd_stdout = t a s k _ r e s u l t . r e s u l t [ " stdout " ] # extracting keys unseal_key = get_unseal_key ( cmd_stdout ) root_token = get_root_token ( cmd_stdout ) r . add_var_as_extra_var ( "VAULT_UNSEAL_KEY" , unseal_key ) r . add_var_as_extra_var ( "VAULT_ROOT_TOKEN" , root_token ) i f r . get_prev_task_name ( ) == v a u l t _ k e y s _ t a s k : task_result = task_results [0] cmd_stdout = t a s k _ r e s u l t . r e s u l t [ " stdout " ] # extracting s e c r e t s c r e a t e d _ s e c r e t s = g e t _ s e c r e t s ( cmd_stdout ) michman_created = F a l s e i f "michman / " in c r e a t e d _ s e c r e t s : michman_created = True r . add_var_as_extra_var ( "MICHMAN_SECRETS_CREATED" , michman_created ) r . finish_ansible () i f r . was_error ( ) : print ( " ansible −playbook launch − ERROR: " ) print ( r . get_error_msg ( ) ) else : print ( " ansible −playbook launch − OK" ) Listing 2: cotea usage example • Parse ‘‘podman ps ...’’ command output to check if the Vault are stored in Ansible variables that will be accessible for the next tasks. container is already running • Extract Vault keys from the ‘‘vault operator init ...’’ command output • Parse ‘‘vault secrets list ...’’ command output to check if the specific Vault secret already exists 1 2 3 4 5 There is a solution to the issues described earlier using cotea in Listing 2. run_next_task returns a list of TaskResult objects for the last task run on every host in the current host group. The task that has just been executed is determined by get_prev_task_name. Podman command output processing takes two lines and is located in the main cotea loop for clarity. Processing of the remaining commands is moved to separate functions. The processing results from s k l e a r n . linear_model import Lasso reg = Lasso ( alpha = 0 . 1 ) reg . f i t ( [ [ 0 , 0 ] , [ 1 , 1 ] ] , [ 0 , 1 ] ) reg . p r e d i c t ( [ [ 1 , 1 ] ] ) Listing 3: gopython usage example: Python code 3.2. gopython usage example Gocotea has the same interfaces as cotea; this was done using gopython, which was separately developed for this purpose. As an 4 David Badalyan and Oleg Borisenko 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 SoftwareX 19 (2022) 101126 / / import gp " github . com/ i s p r a s / gopython / src / gopython" / / from sklearn . linear_model import Lasso var sklearnModule gp . PythonModule sklearnModule . SetModuleName ( " sklearn . linear_model " ) sklearnModule . MakeImport ( ) Lasso , _ := sklearnModule . GetClass ( " Lasso " ) / / reg = linear_model . Lasso ( 0 . 1 ) L a s s o I n i t A r g s , _ := gp . GetPythonMethodArgsWithSpecificLen ( 1 ) L a s s o I n i t A r g s . SetNextArgument ( 0 . 1 ) reg , _ := Lasso . CreateObject ( L a s s o I n i t A r g s ) / / reg . f i t ( [ [ 0 , 0 ] , [ 1 , 1 ] ] , [ 0 , 1 ] ) arg1 , _ := gp . CreatePythonListFromGoSlice ( [ ] [ ] i n t { { 0 , 0 } , { 1 , 1 } } ) arg2 , _ := gp . CreatePythonListFromGoSlice ( [ ] i n t { 0 , 1 } ) f i t A r g s , _ := gp . GetPythonMethodArgsWithSpecificLen ( 2 ) f i t A r g s . SetNextArgument ( arg1 ) f i t A r g s . SetNextArgument ( arg2 ) reg . CallMethod ( " f i t " , f i t A r g s ) / / reg . predict ( [ [ 1 , 1 ] ] ) predictArg1 , _ := gp . CreatePythonListFromGoSlice ( [ ] [ ] i n t { { 1 , 1 } } ) predictArgs , _ := gp . GetPythonMethodArgsWithSpecificLen ( 1 ) p r e d i c t A r g s . SetNextArgument ( predictArg1 ) resNumpyArray , _ := reg . CallMethod ( " predict " , p r e d i c t A r g s ) / / extracting numpy. ndarray into go s l i c e emptyArgs , _ := gp . GetPythonMethodArgsWithSpecificLen ( 0 ) r e s L i s t , _ := resNumpyArray [ 0 ] . CallMethod ( " t o l i s t " , emptyArgs ) r e s L i s t G o I n t e r f a c e , _ := r e s L i s t [ 0 ] . GetGoSliceFromPyList ( ) r e s L i s t G o S l i c e := r e s L i s t G o I n t e r f a c e . ( [ ] float64 ) Listing 4: gopython usage example: Golang code that embeds Python code from listing 3 Table 1 Overhead comparison at 100, 200 and 300 tasks. example of gopython usage, consider embedding the sklearn [31] library into Golang code. sklearn is a popular library in machine learning community. Here are simple usage of sklearn in the listing 3. Using gopython, one can create the same ‘‘reg’’ object during Golang program execution. The inputs can be created in Golang and passed to embedded ‘‘reg’’ object. The result can be returned to Golang and used further. In listing 4, the code that embeds Python code from listing 3 is presented. As arguments for the Python methods, both instances of PythonObject structure and Golang variables of supported types can be used (lines 12, 17, 18, 21, 22, 27, 30). The supported types can be found at the code repo [18]. Python’s return values can also be obtained in Golang. The result of CallMethod is an array of PythonObject structures. The length of this array is equal to the number of objects returned. If the result belongs to one of the supported python types, it can be converted to the corresponding Golang variable. In the above example, the result was converted from numpy.ndarray to golang slice (lines 35–36). 100 tasks 200 tasks 300 tasks ansible-playbook (21.28 ± 0.17)s (41.34 ± 0.35)s (60.97 ± 0.72)s cotea (21.48 ± 0.27)s (41.09 ± 0.42)s (60.65 ± 0.75)s gocotea (24.17 ± 0.47)s (47.13 ± 0.23)s (69.35 ± 0.46)s Table 2 Overhead comparison at 400 and 500 tasks. 400 tasks 500 tasks ansible-playbook (81.76 ± 0.92)s (103.04 ± 0.67)s cotea (79.48 ± 0.7)s (100.23 ± 0.27)s gocotea (92.76 ± 0.44)s (117.05 ± 0.69)s • cotea and gocotea compared to pure ansible-playbook; • gocotea compared to cotea (appearing due to the use of gopython). 3.3. Evaluation Tables 1 and 2 show the average of 5 corresponding launches and their standard deviation. To mitigate network instabilities, the playbooks did not contain network operations and were run against localhost. We compared playbook execution time using ansible-playbook cli, cotea, and gocotea. The goals of the comparison are the evaluation of the overheads of: 5 David Badalyan and Oleg Borisenko SoftwareX 19 (2022) 101126 Fig. 1. Architecture of the COTE (COntrol Thread Execution) class. 5. Conclusion As can be seen, cotea had no overhead in these launches. The average overhead of gocotea was (15.01 ± 1.59)%. As a result of the work done, three software solutions were developed: 4. Impact 1. cotea — Ansible programmatic execution control solution in Python. Source code is available on Github. 2. gopython — a solution for embedding arbitrary Python code into a Golang application. Source code is available on Github. 3. gocotea — a solution for program control of Ansible execution in Golang. It is a port of cotea, implemented with gopython. Source code is available on Github. As mentioned in Section 1, using Ansible comes with several caveats, such as embedding Ansible into other systems, debugging large scenarios, and controlling Ansible execution. Cotea and gocotea solve these issues by providing Python API (Golang API) that allows the following: • control of Ansible execution by iterating over Ansible plays and tasks; • embedding of Ansible into Python (Golang) systems; • debugging of Ansible execution. The developed cotea software solutions, and its analog in the Golang language, gocotea, programmatically control the progress of Ansible execution, iterated over tasks and plays — constituent parts of any Ansible scenario. They also provide information about the execution progress obtained from the fields of internal Ansible objects. The latter can be accessed using decorator objects. gocotea is embedded into Michman, a system for deploying and managing clusters (using Ansible), with a set of on-demand services in cloud environments, including the tasks of biomedicine. The method with which cotea was developed can be used to program control an arbitrary Python application. Cotea has previously been used in clouni [32,33] and a project that converts dockerfiles to Ansible scenarios [34]. Gocotea is embedded into Michman. 4.1. Approach generalization The developed technique can be used to control an arbitrary Python program. For every function or class method of the target program, a decorator class can be created. It will stop the target program’s execution as well as save function arguments and results. Saving function input and output allows one to monitor the target program execution. From the point of view of program execution control, the functions on which the decorator classes were installed will be breakpoints. Here, the continue() method comes into play. This method allows the step-by-step execution of the target program. Steps here mean function calls. By choosing a set of functions whose call sequence corresponds to the occurrence of certain events, it is possible to create higher-level interfaces based on the continue() method. Such interfaces will allow iteration defined by events for the user of the target program (in this work, this event may be Ansible task execution, for example). These interfaces can be in one single class that will also contain corresponding decorator classes objects as its field. This class can run the target program in a separate thread and provide interfaces for dynamic control of the execution and execution monitoring. We propose naming this class COTE (COntrol Thread Execution). An exemplary architecture of the COTE class is shown in Fig. 1. In addition to the methods of the public part, considered in 3.3, as well as the decorator classes, there is a class called the switch control kit that provides switching between threads. The switch control kit can be implemented with any Python thread sync tool. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgment This article was published with financial support from the Ministry of Science and Higher Education of the Russian Feder075-15-2022-294 of 15.04.2022. ation, agreement References [1] Humble J, Farley D. Continuous delivery: reliable software releases through build, test, and deployment automation. Pearson Education; 2010. [2] Artac M, Borovssak T, Di Nitto E, Guerriero M, Tamburri DA. DevOps: Introducing infrastructure-as-code. In: 2017 IEEE/ACM 39th international conference on software engineering companion. IEEE; 2017, p. 497–8. [3] Ansible official website. 2022, https://www.ansible.com/ (Accessed: 24 May 2022). 6 David Badalyan and Oleg Borisenko SoftwareX 19 (2022) 101126 [18] Badalyan D. Gopython (version 0.2.1) [computer software]. 2022, Available at https://github.com/ispras/gopython. [19] CPython official documentation. 2022, https://docs.python.org/3/c-api/ index.html (Accessed: 24 May 2022). [20] Lopez J, Babun L, Aksu H, Uluagac AS. A survey on function and system call hooking approaches. J Hardware Syst Secur 2017;1(2):114–36. [21] PEP 578 – Python runtime audit hooks, Python official website. 2022, https://www.python.org/dev/peps/pep-0578/ (Accessed: 24 May 2022). [22] Satori-NGorg. Hooker (version 0.4.9) [computer software]. 2019, Available at https://pypi.org/project/hooker/. [23] Kim S, Park J, Lee K, You I, Yim K. A brief survey on rootkit techniques in Malicious codes. J Internet Serv Inf Secur 2012;2(3/4):134–47. [24] Brubacher D. Detours: Binary interception of win32 functions. In: Windows NT 3rd symposium (windows NT 3rd symposium). 1999. [25] Carboni D. Pysense: Python decorators for wireless sensor macroprogramming. In: ICSOFT, Vol. 1. 2010, p. 165–9. [26] Kang P. Function call interception techniques. Softw - Pract Exp 2018;48(3):385–401. [27] Cgo official website. 2022, https://golang.org/cmd/cgo/ (Accessed: 24 May 2022). [28] apenella. Go-ansible (version 1.1.1) [computer software]. 2019, Available at https://github.com/apenella/go-ansible. [29] Czarkowski P. Gosible (version 0.1) [computer software]. 2017, Available at https://github.com/paulczar/gosible. [30] Vault official website. 2022, https://www.vaultproject.io/ (Accessed: 24 May 2022). [31] Buitinck L, Louppe G, Blondel M, Pedregosa F, Mueller A, Grisel O, et al. API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD workshop: languages for data mining and machine learning. 2013, p. 108–22. [32] Shvetcova V, Borisenko O, Polischuk M. Using ansible as part of TOSCA orchestrator. In: 2020 ivannikov ispras open conference. IEEE; 2020, p. 109–14. [33] Shvetcova V. Clouni (version 1.0) [computer software]. 2020, Available at https://github.com/ispras/clouni. [34] Popov M. Tool that converts dockerfiles to ansible scenarios (version 1.0) [computer software]. 2022, Available at https://github.com/SegFaulti4/ dockerfile-ansible-convert. [4] NASA: Increasing cloud efficiency with ansible and ansible tower. Technical report, Ansible; 2022, p. 1, https://www.ansible.com/hubfs/pdf/AnsibleCase-Study-NASA.pdf (Accessed: 24 May 2022). [5] TripleO official documentation. 2022, https://docs.openstack.org/tripleodocs/latest/ (Accessed: 24 May 2022). [6] Ansible runner official documentation. 2022, https://ansible-runner. readthedocs.io/ (Accessed: 24 May 2022). [7] Rahman A, Mahdavi-Hezaveh R, Williams L. A systematic mapping study of infrastructure-as-code research. Inf Softw Technol 2019;108:65–77. [8] Guerriero M, Garriga M, Tamburri DA, Palomba F. Adoption, support, and challenges of infrastructure-as-code: Insights from industry. In: 2019 IEEE international conference on software maintenance and evolution. IEEE; 2019, p. 580–9. [9] Weiss A, Guha A, Brun Y. Tortoise: Interactive system configuration repair. In: 2017 32nd IEEE/ACM international conference on automated software engineering. IEEE; 2017, p. 625–36. [10] Dalla Palma S, Di Nucci D, Tamburri DA. AnsibleMetrics: A Python library for measuring infrastructure-as-code blueprints in Ansible. SoftwareX 2020;12:100633. [11] Hummer W, Rosenberg F, Oliveira F, Eilam T. Testing idempotence for infrastructure as code. In: ACM/IFIP/USENIX international conference on distributed systems platforms and open distributed processing. Springer; 2013, p. 368–88. [12] Hanappi O, Hummer W, Dustdar S. Asserting reliable convergence for configuration management scripts. In: Proceedings of the 2016 ACM SIGPLAN international conference on object-oriented programming, systems, languages, and applications. 2016, p. 328–43. [13] Hochstein L, Moser R. Ansible up & running. 2nd ed.. O’Reilly Media, Inc.; 2017, p. 305–17, Chapter: Debugging Ansible Playbooks. [14] Playbook debugger, ansible official website. 2022, https://docs.ansible.com/ ansible/2.9/user_guide/playbooks_debugger.html (Accessed: 24 May 2022). [15] Aksenova E, Lazarev N, Badalyan D, Borisenko O, Pastukhov R. Michman: an orchestrator to deploy distributed services in cloud environments. In: 2020 ivannikov ispras open conference. IEEE; 2020, p. 57–63. [16] Badalyan D. Cotea (version 0.2) [computer software]. 2022, Available at https://github.com/ispras/cotea. [17] Badalyan D. Gocotea (version 0.2.2) [computer software]. 2022, Available at https://github.com/ispras/gocotea. 7