FACULTY OF ECONOMICS
AND BUSINESS ADMINISTRATION
GHENT UNIVERSITY
FACULTY OF ECONOMICS
AND BUSINESS ADMINISTRATION
ACADEMIC YEAR 2013 – 2014
The Development of an Optimal Visualisation
for Business Architecture (CHOOSE)
Dissertation presented to obtain the degree of
Master of Science in Applied Economics: Business Engineering
Sarah Boone
under supervision of
Prof. dr. Geert Poels & Maxime Bernaert FACULTY OF ECONOMICS
AND BUSINESS ADMINISTRATION
GHENT UNIVERSITY
FACULTY OF ECONOMICS
AND BUSINESS ADMINISTRATION
ACADEMIC YEAR 2013 – 2014
The Development of an Optimal Visualisation
for Business Architecture (CHOOSE)
Dissertation presented to obtain the degree of
Master of Science in Applied Economics: Business Engineering
Sarah Boone
under supervision of
Prof. dr. Geert Poels & Maxime Bernaert PERMISSION Ondergetekende verklaart dat de inhoud van deze masterproef mag geraadpleegd en/of gereproduceerd worden, mits bronvermelding. The undersigned states that the content of this master thesis may be consulted and/or reproduced, if cited. Sarah Boone i Dutch summary – Samenvatting Enterprise architecture (EA) streeft twee vormen van alignering na. Enerzijds is het belangrijk dat de bedrijfsvoering en de informatietechnologie op elkaar afgestemd zijn. Anderzijds moet het strategische luik van het bedrijf gekoppeld worden aan het operationele luik (Maes, 2007). Doordat kleine en middelgrote ondernemingen (KMO’s) op heel wat vlakken verschillen van grote ondernemingen, is bij hen een andere aanpak vereist om op een succesvolle manier EA te kunnen implementeren. Om aan de specifieke criteria van KMO’s tegemoet te komen hebben Bernaert et al. de EA techniek CHOOSE ontwikkeld (Bernaert, Poels, Snoeck, & De Backer, 2013b). In voorgaand onderzoek is veel aandacht besteed aan het verfijnen van de methode en het metamodel van CHOOSE (Bernaert, Poels, et al., 2013b) en zijn verschillende applicaties ontworpen die de techniek moeten ondersteunen (Bernaert, Maes, & Poels, 2013; Dumeez, Bernaert, & Poels, 2013; Ingelbeen, Bernaert, & Poels, 2013; Zutterman, 2013). Tot nu toe is echter geen onderzoek uitgevoerd naar de visuele notatie van CHOOSE, terwijl de manier waarop diagrammen voorgesteld worden een grote impact heeft op de cognitieve effectiviteit ervan (Larkin & Simon, 1987; Moody, 2009a). Het doel van deze masterproef bestaat er bijgevolg in een visualisatie te ontwerpen voor CHOOSE die werknemers van KMO’s in staat stelt de diagrammen accuraat, snel en gemakkelijk te interpreteren. In de masterproef wordt eerst de nodige achtergrondinformatie meegegeven. Vervolgens bestaat het onderzoek uit drie delen. Eerst wordt de huidige visualisatie van CHOOSE geëvalueerd aan de hand van negen design principes die opgesteld werden door Daniel Moody (Moody, 2009a). Nadien worden op basis van deze analyse drie alternatieve visualisaties ontworpen. Hierbij zijn op een graduele manier verbeteringen toegepast. Ten slotte wordt een gecontrolleerd experiment uitgevoerd om na te gaan welke optie aan te raden is om te implementeren. Tijdens dit experiment worden twee aspecten onderzocht. Enerzijds wordt nagegaan hoe accuraat, snel en gemakkelijk de verschillende notaties te interpreteren zijn. Anderzijds wordt bij de respondenten gepeild in hoeverre ze een notatie bruikbaar vinden. De resultaten van het onderzoek maken het mogelijk een advies te formuleren over welke visuele notatie het beste geïmplementeerd wordt in de context van CHOOSE. Daarnaast is dit onderzoek ook een evaluatie van Moody’s Physics of Notations en reikt het voorzichtig een methodologie aan om visuele notaties te evalueren en te verbeteren. ii Acknowledgement This document is the outcome of an intellectual journey that entailed a lot of commitment and perseverance. Yet, this result would not have been achieved without the contribution and assistance of many. Therefore, I would like to express my sincere thanks to everyone who had an input in this master thesis. First of all, I would like to thank prof. dr. Geert Poels and Maxime Bernaert for introducing the topic to me and supporting me with advice during the journey. Besides, I would like to thank them, together with Ben Roelens and Steven Mertens, for reviewing the content of this master thesis. In addition, I would like to thank prof. dr. Frederik Gailly for giving me the opportunity to conduct my experiment during his courses. Furthermore, I would like to thank everyone who provided me with useful insights during the process of improving the visual notation of CHOOSE. Finally, I would like to thank my family and friends for their interest in my master thesis. Special thanks go out to my parents and to Sylvia for their unconditional support. Sarah Boone th
20 of May, 2014 iii Table of Contents Abstract ......................................................................................................................................... 1 1. Introduction ....................................................................................................................... 2 2. Background ........................................................................................................................ 4 2.1. CHOOSE for EA in SMEs ............................................................................................... 4 2.2. Moody’s Physics of Notations ...................................................................................... 7 2.2.1. Semiotic Clarity ...................................................................................................... 8 2.2.2. Perceptual Discriminability ................................................................................... 8 2.2.3. Semantic Transparency ......................................................................................... 8 2.2.4. Visual Expressiveness ............................................................................................ 8 2.2.5. Complexity Management ...................................................................................... 9 2.2.6. Dual Coding ........................................................................................................... 9 2.2.7. Cognitive Integration ............................................................................................. 9 2.2.8. Graphic Economy .................................................................................................. 9 2.2.9. Cognitive Fit ......................................................................................................... 10 2.3. 3. Related Work ............................................................................................................. 10 Analysis of the CHOOSE Visualisation .............................................................................. 11 3.1. Semiotic Clarity .......................................................................................................... 11 3.2. Perceptual Discriminability ........................................................................................ 11 3.3. Semantic Transparency .............................................................................................. 11 3.4. Visual Expressiveness ................................................................................................. 12 3.5. Complexity Management .......................................................................................... 12 4. Alternative Visualisations Development .......................................................................... 13 4.1. Alternative 1 .............................................................................................................. 13 4.2. Alternative 2 .............................................................................................................. 15 4.3. Alternative 3 .............................................................................................................. 16 iv 5. Evaluation ........................................................................................................................ 18 5.1. Test Design ................................................................................................................. 18 5.2. Experiment Results .................................................................................................... 19 6. Discussion ........................................................................................................................ 22 7. Conclusion and Future Research ..................................................................................... 23 Bibliography ................................................................................................................................ 24 Appendices ..................................................................................................................................... I Appendix A Constructs of CHOOSE ....................................................................................... I Appendix B Symbols Applied in the Different Visual Notations .......................................... III Appendix C Outline Third Visualisation Alternative ............................................................ IV Appendix D Survey Questions ........................................................................................... XIII Appendix E Statistical analysis ........................................................................................... XX Appendix F Files for Implementing the New Notation in CHOOSE .................................. XXII v List of Abbreviations EA Enterprise Architecture SME Small and Medium‐sized Enterprises CEO Chief Executive Officer CHOOSE keep Control, by means of a Holistic Overview, based on Objectives and kept Simple, of your Enterprise KPI Key Performance Indicator CD Cognitive Dimensions of Notations SEQUAL Semiotic Quality UML Unified Modeling Language BPMN Business Process Model and Notation UCM Use Case Map RACI Responsible, Accountable, Consulted, Informed PEOU Perceived Ease of Use PU Perceived Usefulness IU Intention to Use TAM Technology Acceptance Model CE Cognitive Effectiveness A Accuracy T Time ME Mental Effort SD Standard Deviation ANOVA Analysis of Variance vi List of Figures Figure 1. CHOOSE metamodel (Bernaert, Poels, et al., 2013b, p.19) ........................................... 5 Figure 2. Model created with the current visual notation of CHOOSE ......................................... 6 Figure 3. Principles for designing cognitively effective visual notations ...................................... 7 Figure 4. Visual variables defined by Bertin (1983) .................................................................... 12 Figure 5. Model created with the first alternative visual notation ............................................. 14 Figure 6. Model created with the second alternative visual notation ........................................ 15 Figure 7. Single viewpoint ........................................................................................................... 16 Figure 8. Pairwise relationships .................................................................................................. 16 Figure 9. Entire diagram with cursor on the goal ‘Lower variability in production’ ................... 17 Figure 10. Boxplots of the variable PEOU ................................................................................... 21 Figure B.1. Legend: symbols applied in the different visual notations ........................................ III Figure C.1. Overview of the four viewpoints................................................................................ IV Figure C.2. Goal viewpoint ........................................................................................................... IV Figure C.3. Actor viewpoint ........................................................................................................... V Figure C.4. Operation viewpoint ................................................................................................... V Figure C.5. Object viewpoint ........................................................................................................ VI Figure C.6. Pairwise relationships between goals and actors ...................................................... VI Figure C.7. Pairwise relationships between goals and operations ............................................. VII Figure C.8. Pairwise relationships between goals and objects ................................................... VII Figure C.9. Pairwise relationships between objects and actors ................................................. VIII Figure C.10. Pairwise relationships between objects and operations ....................................... VIII Figure C.11. Pairwise relationships between actors and operations ........................................... IX Figure C.12. Entire diagram: all viewpoints included .................................................................... X Figure C.13. Entire diagram with cursor on the goal ‘Lower variability in production’ ............... XI Figure C.14. Entire diagram with cursor on the project ‘Start up implementation lean principle’ ...................................................................................................................................... XII vii List of Tables Table 1. Improvements relative to the current notation ............................................................ 13 Table 2. Descriptive statistics per group ..................................................................................... 20 Table 3. Test results of the pairwise comparisons ...................................................................... 20 Table A.1. Constructs of CHOOSE: definitions (adjusted from Zutterman, 2013, p.85‐86) ........... I Table D.1. Post Experiment Questionnaire ................................................................................ XIX Table E.1. Tests of Normality ...................................................................................................... XX Table E.2. Cognitive Effectiveness: test of homogeneity of variances ....................................... XXI Table E.3. Test results of the pairwise comparisons, p values included .................................... XXI viii Abstract Enterprise architecture (EA) serves as a means to improve business‐IT and strategy‐operations alignment in an organisation (Maes, 2007). While it is a fairly mature domain in large enterprises, the need for EA in small and medium‐sized enterprises (SMEs) has only been recently addressed (Bernaert, 2011; Jacobs, Kotzé, van der Merwe, & Gerber, 2011). As SMEs have different characteristics and cope with specific problems, a different approach is essential to enable a successful adoption of EA. In order to meet these particular requirements of SMEs, the EA approach CHOOSE has been developed (Bernaert, Poels, et al., 2013b). In previous research, emphasis has been put on refining the method and metamodel of CHOOSE (Bernaert, Poels, et al., 2013b) and on the development of supporting software tools (Bernaert, Maes, et al., 2013; Dumeez et al., 2013; Ingelbeen et al., 2013; Zutterman, 2013). However, the visual notation of CHOOSE has not been investigated yet, while the form of representation has a great impact on the cognitive effectiveness of a diagram (Larkin & Simon, 1987; Moody, 2009a). This master thesis assesses the current visualisation of CHOOSE, describes alternatives and conducts an experimental comparison. 1 1. Introduction Enterprise Architecture (EA) serves as a means to improve business‐IT and strategy‐operations alignment (Maes, 2007) and is a key instrument in controlling the complexity of an organisation (Bernaert, Poels, Snoeck, & De Backer, 2013c; Lankhorst, 2013). Yet, one might be more familiar with the concept of ‘architecture’ in the context of construction. In case someone wants to design a house, he or she often appeals to an architect. Together, they discuss the different wishes regarding the configuration of the house and a master plan is established. This can be done efficiently, because the client and the architect share a common frame of reference. They both interpret a ‘room’, a ‘door’ and a ‘window’ in the same way. When starting, running or growing a business, a similar frame of reference is required (Lankhorst, 2013). EA serves as an aid for this by creating a holistic overview of the organisation through describing and controlling the structure, processes, applications and technology in an integrated way (Lankhorst, 2013). Although EA is a fairly mature domain in large enterprises, the adoption in small and medium‐
sized enterprises (SMEs) is lagging behind due to the complexity involved in using the current EA approaches (Bhagwat & Sharma, 2007). SMEs often lack the expertise required to implement these approaches and do not have the financial resources to hire consultants (Dehbokry & Chew, 2014). In order to tackle this issue, Bernaert et al. have developed a new approach called CHOOSE, which is adapted to the needs of the target group (section 2.1) (Bernaert, Poels, et al., 2013c). In previous research, the method and metamodel of CHOOSE have been refined (Bernaert, Poels, et al., 2013b) and tool support has been developed (Bernaert, Maes, et al., 2013; Dumeez et al., 2013; Ingelbeen et al., 2013; Zutterman, 2013). These investigations have already put a lot of emphasis on the comprehensibility of the approach for inexperienced enterprise modellers. However, up to now the visual notation of CHOOSE has not been evaluated, while the form of representation has an important impact on the cognitive effectiveness of a diagram (Larkin & Simon, 1987; Moody, 2009a). This impact is especially crucial in the case of novice users (Moody, 2009a), which makes it very worthwhile to investigate the visual notation of CHOOSE. 2 The research in this master thesis therefore focuses on how CHOOSE should be visualised in order to allow the users to interpret the diagrams in a cognitively effective way. Besides this, the effect of the form of representation on the perceived ease of use, perceived usefulness and the intention to use is investigated as well. The result should enable effective and time efficient communication about the EA within SMEs. Section 2 provides the theoretical background needed to conduct this research. First, the EA approach CHOOSE is briefly explained (Bernaert, Poels, et al., 2013b). Next, Moody’s Physics of Notations (Moody, 2009a), a theory for visual notation design, is discussed. Last, related work is shortly summarized. The actual research comprises three major parts: first, the current visualisation is assessed based on the principles of the Physics of Notations (section 3) (Moody, 2009a). Second, alternative representations are developed (section 4). And third, an experiment is conducted to verify which visualisation has the best outcomes in terms of cognitive effectiveness on the one hand and perceived ease of use, perceived usefulness and intention to use on the other hand (sections 5 and 6). 3 2. Background In a lot of SMEs, there is a lack of overview of the company. In many cases, the CEO is the only one who has a clear overview of the business, because there is no mechanism that enables the transfer of that knowledge towards the employees. As a result, it is difficult for the CEO to discuss strategic matters with other people in the company (Bernaert, Poels, et al., 2013c; Kamsties, Hörmann, & Schlich, 1998). To overcome this problem, it would be very useful to implement EA. However, existing EA approaches are often complicated and require expertise, time and hence money (Glissmann, 2011; Moser, Junginger, Brückmann, & Schöne, 2009). Therefore, Bernaert et al. have developed CHOOSE (Bernaert, Poels, et al., 2013c). 2.1.
CHOOSE for EA in SMEs CHOOSE is an acronym for ‘keep Control, by means of a Holistic Overview, based on Objectives and kept Simple, of your Enterprise’, which refers to the essential requirements for implementing EA in an enterprise (Bernaert & Poels, 2011). Especially the term ‘Simple’ deserves some additional attention in the context of SMEs, because the word reflects six specific criteria an EA approach must satisfy in order to enable successful adoption in SMEs (Bernaert, Poels, et al., 2013b): 1. The approach should enable SMEs to deal with strategic issues in a time efficient way. 2. A person with limited IT skills should be able to apply the approach. 3. It should be possible to apply the approach with little assistance of external experts. 4. The approach should enable making descriptions of the processes in the company. 5. The CEO must be involved in the approach. 6. The expected revenues of the approach must exceed the expected costs and risks. The metamodel of CHOOSE incorporates these criteria, which means it enables SMEs to create simple, yet comprehensive models (Bernaert, Poels, et al., 2013b). These models represent an overview of the business architecture layer, integrating elements of the information systems and technology layers (Bernaert, Maes, et al., 2013; Bernaert, Poels, et al., 2013b). They consist of four viewpoints: goals (why), actors (who), operations (how) and objects (what) (Figure 1). 4 Conflict
Division
Supervision
- supervisor
Refinement
Id
OR-Ref
Wish
Goal
1
Name
Description
AND-Ref 1..
Actor
Name
Description
Assignment
- supervisee
Human Actor
Optional, Or
Monitoring
Role
Performs
Software Actor
Operationalization
Device
Control
Concern
Performance (RACI)
[Type]
Link
[Role]
[Multiplicity]
Association
Name
Description
2
Optional, Or
Object
Name
Description
Input
Operation
Specialization
Entity
Optional, Or
Process
Project
- subOperation
- superOperation
Optional, Or
Aggregation
Name
Description
Output
Includes
Actor
Figure 1. CHOOSE metamodel (Bernaert, Poels, et al., 2013b, p.19) An example of a model that has been created with CHOOSE is demonstrated in Figure 2. It can be analysed making use of the definitions (Appendix A) and the legend (Appendix B). The CEO wishes to increase the customer base of the company and identifies two possible routes to achieve this. First, the selling price could be decreased. However, this conflicts with the accountant’s goal to sustain the company’s profit margins. Second, the product quality could be improved. The decision is made to give the production manager the assignment to achieve this goal. He can accomplish this by lowering the variability in production and by procuring high quality raw materials. In order to lower the production variability, the company executes a project in which they start up the implementation of the lean principle. The CEO is held accountable for this, while the production manager is responsible for the execution and the team leader is consulted. This project is part of the supply chain management. A process that is also part of the supply chain management is inventory management. This process is automatically executed by software for automatic inventory replenishment and by an RFID scanner. The production manager is accountable for the process and the team leader is informed. In this process, actions should be taken to ensure the procurement of high quality raw materials. Executing these operations involves several objects. In order to start up the implementation of the lean principle, process variables are needed as input. This is a more general term for data concerning quality, efficiency and throughput, which are all incorporated in an input database. The output of the project consists of control charts of KPIs, which have an association with the output database. The process of inventory management obviously involves the inventory, both as input and output. This inventory is monitored by the team 5 leader and controlled by the production manager. This means the production manager can alter the inventory, while the team leader can observe the changes. Last, the diagram reveals that Steven, the CEO, supervises Katrien and Christophe. Katrien is the accountant of the company and hence works in the finance department. Christophe is the production manager and supervises Didier, who performs the role of team leader. Obviously, Supply chain
management
RFID scanner
Inventory
management
Output
database
Throughput
data
Efficiency data
Input database
Quality data
Process
variables
Inventory
Lower
variability in
production
Improve
product quality
Procure high
quality raw
materials
Decrease
price
Increase
customer base
Control charts
of KPIs
Start up
implementation
lean principle
Team leader
Production
manager
Accountant
Sustain profit
margins
CEO
Katrien
Finance
department
Steven
Software for
automatic
inventory repl.
Christophe
Didier
Production
department
Christophe and Didier work in the production department. Figure 2. Model created with the current visual notation of CHOOSE 6 Besides demonstrating the CHOOSE approach, the example also shows the current visual notation of CHOOSE. As will become clear in section 3, there is still a lot of room for improvement with respect to this visual notation. 2.2.
Moody’s Physics of Notations Numerous papers cover the evaluation of a notation on the semantic level (e.g. (Opdahl & Henderson‐Sellers, 2002; Recker, Indulska, Rosemann, & Green, 2005)). However, as stated in the introduction, the visual syntax of a notation has a great impact on the cognitive effectiveness of it as well (Larkin & Simon, 1987; Moody, 2009a). A couple of theories for evaluating the visual syntax of notations have been developed, such as the Cognitive Dimensions of Notations (CDs) framework (Green et al., 2006), the semiotic quality (SEQUAL) framework (Krogstie, Sindre, & Havard, 2006) and the Physics of Notations (Moody, 2009a). Genon et al. argue that the first two frameworks lack theoretical and empirical foundations concerning the visual aspects of notations (Genon, Heymans, & Amyot, 2011). Besides, in Moody’s evaluation of the CDs framework, several additional shortcomings of that framework can be found (Moody, 2009b). Therefore, the Physics of Notations is used as a basis for this research. Moody states that a clear design goal needs to be identified before a visual notation can be developed (Moody, 2009a). Common design goals are e.g. simplicity and expressiveness. However, according to Moody, these goals are vague and subjective. A more objective and scientific goal is cognitive effectiveness, which is defined as the speed, ease and accuracy with which a representation can be processed (Larkin & Simon, 1987). To enable designers to create cognitively effective visual notations, Moody has defined nine principles (Figure 3) (Moody, 2009a). These are explained in the next sections, together with their relevance for this thesis. Perceptual
Discriminability
Graphic
Economy
Cognitive
Fit
Semiotic
Clarity
Cognitive
Integration
Semantic
Transparency
Visual
Expressiveness
Manageable
Complexity
Dual
Coding
Figure 3. Principles for designing cognitively effective visual notations, image from the Physics of Notations (Moody, 2009a, p.761) 7 2.2.1. Semiotic Clarity According to the Theory of Symbols developed by Goodman, each semantic construct should be represented by exactly one graphical symbol, and vice versa (Goodman, 1976). This principle can be violated through four kinds of anomalies: ‐
Symbol redundancy: a semantic construct is represented by multiple symbols ‐
Symbol overload: one symbol represents more than one semantic construct ‐
Symbol excess: a symbol is created that does not represent any semantic construct ‐
Symbol deficit: there is no symbol provided for a certain semantic construct This principle is incorporated in this master thesis with the intention to obtain an unambiguous notation that inherently avoids misconceptions. 2.2.2. Perceptual Discriminability The principle of perceptual discriminability states that it should be possible to easily and accurately distinguish between different symbols. This is strongly influenced by the visual distance between symbols. Visual distance can be defined as the number of visual variables on which symbols differ, combined with the magnitude of the differences. In general, a greater visual distance between symbols leads to a faster and more accurate recognition. Shape is a detrimental factor in distinguishing between symbols. Therefore, it should be utilised as the primary visual variable. Perceptual discriminability is very important in the case of CHOOSE, because this notation is used by novices and the requirements for discriminability are higher in case novices use a notation than when experts do. 2.2.3. Semantic Transparency The representation of a construct should suggest its meaning. One way to design semantic transparent symbols is by using icons, which lead to a faster recognition and recall of the constructs. Besides, they especially enhance the comprehensibility of the notation for novice users, which makes it very worthwhile to incorporate this principle in this research. 2.2.4. Visual Expressiveness Visual expressiveness is determined by the number of visual variables used in a notation and the extent to which they are used. While perceptual discriminability is a measure for the pairwise discrepancy between graphical symbols, visual expressiveness measures the diversity 8 of the visual vocabulary as a whole. Colour is a strong mechanism for enhancing the visual expressiveness of a notation, as contrast in colour is seen faster than differences in other variables. However, colour should only be used in a redundant way. Otherwise, differences disappear when a diagram is printed in grayscale. 2.2.5. Complexity Management Diagrammatic complexity is measured by the number of elements in a diagram. This means that when there are a lot of elements displayed, it becomes difficult to interpret the diagram. This type of complexity can be reduced in two ways. First, the diagram can be split into smaller sub diagrams, which is called modularisation. Furthermore, diagrams can be hierarchically structured to limit the levels of detail. This principle is very important in the case of CHOOSE, because novices have more difficulties dealing with complexity than experts (Sweller, 1994). 2.2.6. Dual Coding According to Moody, text can be used as a supplement for graphics. However, it is still important that symbols are distinguishable based on the graphics rather than the text. Labels can be used to distinguish between symbol instances, not between symbol types (Moody, 2009a). Therefore, this principle is somewhat less addressed. 2.2.7. Cognitive Integration The notation should enable integrating information from different diagrams. Although this principle should not be neglected, it is not incorporated in this research. As CHOOSE targets novices in enterprise modelling, one notation to model everything is preferred. Besides, when SMEs grow and more detail needs to be added to the EA models, it might be useful to map the CHOOSE models on the ArchiMate standard (The Open Group, 2012). Bernaert et al. have already conducted a research on this (Bernaert, Poels, Snoeck, & De Backer, 2013a), which makes it less relevant to include this in this master thesis. 2.2.8. Graphic Economy According to this principle, the number of symbol types in a notation should be limited. Adopting this principle can be done in three ways. First, the number of semantic constructs can be reduced. However, the number of constructs in CHOOSE is already limited to the bare minimum. Second, symbol deficit can be introduced, but this harms the semiotic clarity of the 9 notation (section 2.2.1). Third, visual expressiveness can be used. Manipulating multiple visual variables reduces the need to lower the amount of symbols. In this research, this third action is applied in order to pursue graphic economy. Therefore, the principle by itself will not be individually investigated. 2.2.9. Cognitive Fit Cognitively effective notations for novices might not be cognitively effective for experts, and vice versa. This principle therefore states that different audiences need different notations. CHOOSE targets SMEs, which is a very diverse audience in terms of expertise. However, this principle is not included in this research, because in general it can be said that most users of the target group are novices in enterprise modelling. 2.3.
Related Work Several visual notations such as UML (Moody & Hillegersberg, 2009), i* (Moody, Heymans, & Matulevičius, 2010), BPMN (Genon, Heymans, et al., 2011) and UCM (Genon, Amyot, & Heymans, 2011) have been evaluated based on the principles of the Physics of Notations. These studies constitute a useful basis for this master thesis, because they demonstrate a methodology to identify shortcomings in a notation. This methodology is also applied for evaluating the CHOOSE visualisation (section 3). However, all four articles have two limitations in common: the suggested improvements have not been thoroughly elaborated and the findings have not been empirically evaluated. Gopalakrishnan et al. compare two notation alternatives for process modelling by conducting a controlled experiment (Gopalakrishnan, Krogstie, & Sindre, 2010). Although similar goals as in this research are pursued, they do not use the concept of cognitive effectiveness. Furthermore, Huang et al. have conducted an experiment to compare different graph visualisations, based on a cognitive load perspective (Huang, Eades, & Hong, 2009). Their research does not focus on visual notations, but several aspects of the test design provide useful insights for the experiment described in this master thesis. 10 3. Analysis of the CHOOSE Visualisation In this section, the current visual notation of CHOOSE is evaluated based on five principles from Moody’s theory. The analysis focuses on semiotic clarity, perceptual discriminability, semantic transparency, visual expressiveness and complexity management. As mentioned in the previous sections dual coding, cognitive integration, graphic economy and cognitive fit are not covered in this analysis. 3.1.
Semiotic Clarity In the current visual notation, there is no symbol redundancy, excess or deficit. The only anomaly that occurs is symbol overload. This can be a considerable problem, because it can lead to ambiguity and misinterpretation (Moody, 2009a). For CHOOSE, the relationships association, concern and control are represented by the same symbol, which is also the case for the relationships input and output (see Appendix B). For these latter two, the problem is not tremendous, because they represent the same content in the opposite direction. For association, concern and control, it is important to resolve this anomaly because the meaning of these relationships cannot be linked. 3.2.
Perceptual Discriminability Shape is a very important factor in distinguishing between different symbols. However, all ten entities (goal, actor, human actor, role, software actor, device, operation, process, project and object) are represented by a rounded rectangle. Besides, many relationships share an equal symbol as well. For the total of 32 semantic constructs, only 12 different shapes are used. This is an important shortcoming in the notation that will have to be eliminated when designing alternatives. 3.3.
Semantic Transparency A symbol is semantically transparent when its appearance suggests its meaning. When examining the current visual notation, it is clear that there is a lot of room for improvement regarding this principle. Only four symbols show a certain presence of semantic transparency, which are the symbols of human actor, goal, conflict and device. This means 28 symbols do not suggest the meaning of their construct at all. 11 3.4.
Visual Expressiveness In total, there are eight visual variables that can be modified (Figure 4) (Bertin, 1983). In the current visual notation, the variables shape, colour, brightness and horizontal and vertical position are used. This means five out of eight variables are already used, which is better than most visual notations (Moody et al., 2010). However, some of them are more adequately used than others. Constructs belonging to the same viewpoint are for example represented by one colour and they are grouped into the same corner. These variables are properly utilised. Brightness on the other hand is categorised as a used variable, because informed and monitor are represented by the same dotted line, only in a slightly different grey. One could doubt whether the variable is utilised in the right context, because informed and monitor do not have any meaning in common. Therefore, the notation of CHOOSE can still be enhanced for this principle. PLANAR
VARIABLES
RETINAL VARIABLES
Horizontal
Position
Shape
Size
Colour
Large
Red
Green
Blue
Medium
Small
Vertical
Position
Brightness
Orientation
45
Low
Medium
High
Texture
o
90 o
0o
Figure 4. Visual variables defined by Bertin (1983), image from the Physics of Notations (Moody, 2009a, p.760) 3.5.
Complexity Management Currently, all information is modelled in one diagram. This means no mechanisms are provided for managing complexity. However, diagrams can quickly become too complex for novices (Moody, 2009a). Hence, integrating this principle would benefit the cognitive effectiveness of the notation. As the metamodel of CHOOSE clearly distinguishes between four viewpoints (goals, actors, operations and objects), it might be useful to apply the mechanism of modularisation and as such split the diagram into smaller sub diagrams. 12 4. Alternative Visualisations Development The evaluation of the current visual notation served as a basis for the development of three alternatives. During the establishment of the first alternative, special attention was paid to the principles of semiotic clarity, perceptual discriminability, semantic transparency and visual expressiveness. When, as a little exploratory research, the resulting diagram was presented to four CEOs of SMEs, the major remark was the lack of uniformity in style. In their opinion, this rendered the visualisation less appealing. Although this aspect is not incorporated in the Physics of Notations, the interview revealed that it should not be neglected. Besides, the research of Sonderegger and Sauer showed that aesthetics have a positive influence on the users’ performance and the perceived usability (Sonderegger & Sauer, 2010). It is therefore worthwhile to incorporate this in the visualisation. Hence, a second alternative was developed with the intention to achieve this uniformity in style. After this, the principle of complexity management was integrated, which resulted in a third visualisation alternative. An overview of the improvements can be found in Table 1. Table 1. Improvements relative to the current notation (check mark: shortcoming is managed) Shortcomings current notation Alternative 1 Alternative 2 Alternative 3 1. Symbol overload 2. Shape similarity 3. Lack of semantic transparency 4. Inconsistent use of visual variables 5. No mechanism for complexity mgmt. 6. No attention for aesthetics 4.1.
Alternative 1 In this alternative (Figure 5), some essential problems of the original notation are handled. First of all, it is made sure that every semantic construct corresponds with exactly one graphical symbol, and vice versa. Only the relationships input and output are still represented by the same symbol, for reasons stated in section 3.1. Second, different constructs within one viewpoint are represented by symbols that have the same shape, while the shapes differ between the viewpoints. The contrast between the viewpoints is further enlarged by using clearly distinguishable colours. Third, icons are used in order to improve the semantic 13 14 &
Improve
product
quality
Input
database
Figure 5. Model created with the first alternative visual notation OBJECTS
Throughput
data
Efficiency
data
Quality
data
Lower
variability in
production
GOALS
Procure high
quality raw
materials
OR
Increase
customer
base
Output
database
Process
variables
Decrease
price
Control
charts
of KPIs
Inventory
Sustain
profit
margins
R
Team leader
Didier
Production manager
Christophe
Production
department
C
CEO
Steven
R
R
RFID
scanner
OPERATIONS
Inventory
management
Supply chain
I management
Start up
implementation
lean principle
A
A
Software for
automatic
inventory
replenishment
Accountant
Katrien
Finance
department
ACTORS
transparency of the symbols. Operations are represented by a gear, the relationship monitor by an eye, control by a steering wheel, etc. Last, visual variables are used in a consistent way. The variable brightness is only used when it can have a meaningful contribution. In the case of the symbols of RACI, relationships that involve a higher responsibility are represented by a darker colour. GOALS
15 Figure 6. Model created with the second alternative visual notation OBJECTS
THROUGHPUT
DATA
EFFICIENCY
DATA
QUALITY
DATA
INPUT
DATABASE
LOWER
VARIABILITEIT
INVARIABILITY
PRODUCTIE
IN PRODUCTION
&
IMPROVE
PRODUCT
QUALITY
DECREASE
PRICE
OUTPUT
DATABASE
PROCESS
VARIABLES
PROCURE
HIGH QUALITY
RAW MATERIALS
OR
INCREASE
CUSTOMER
BASE
CONTROL
CHARTS
OF KPIs
INVENTORY
SUSTAIN
PROFIT
MARGINS
R
I
A
C
TEAM LEADER
DIDIER
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
SUPPLY CHAIN
MANAGEMENT
R
RFID SCANNER
ACTORS
OPERATIONS
INVENTORY
MANAGEMENT
A
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
ACCOUNTANT
PRODUCTION
MANAGER
KATRIEN
FINANCE
DEPARTMENT
CHRISTOPHE
PRODUCTION
DEPARTMENT
CEO
STEVEN
4.2.
Alternative 2 The first alternative is used as a starting point for the development of the second one (Figure 6). This notation does not add any improvements in terms of Moody’s principles. However, as explained above, it is developed in order to obtain uniformity in style. 4.3.
Alternative 3 The previous alternatives display all information in one diagram. However, even for small examples as the one used in this master thesis, relationships between the viewpoints turn the diagram into a complicated maze of information. Therefore, incorporating mechanisms to enable complexity management might improve the comprehensibility of the notation. Several functionalities are hence applied in the previous alternative. First of all, it is made possible to interpret a single viewpoint at a time (e.g. Figure 7). Second, the relationships between the viewpoints can be analysed in a diagram that only displays the elements of two viewpoints and their interconnections (e.g. Figure 8). INCREASE
CUSTOMER
BASE
OR
IMPROVE
PRODUCT
QUALITY
SUSTAIN
PROFIT
MARGINS
DECREASE
PRICE
&
LOWER
VARIABILITY
IN PRODUCTION
PROCURE
HIGH QUALITY
RAW MATERIALS
Figure 7. Single viewpoint CHRISTOPHE
PRODUCTION
MANAGER
OBJECTS
ACTORS
INVENTORY
DIDIER
TEAM LEADER
Figure 8. Pairwise relationships These two measures drastically reduce the number of graphical elements displayed, which must lead to an easier and faster understanding of the content. However, if these two representations would be the only ways to access the content, the overview might get lost. This should be avoided because attaining a holistic overview is one of the major advantages of 16 implementing CHOOSE in an organisation. It should therefore still be possible to access the entire diagram. Hence, a third functionality is added. When the entire diagram is displayed, and the user places the cursor on an element in the diagram, that specific element is highlighted together with all adjacent elements (e.g. Figure 9). The combination of these three additional functionalities should lead to better results during the controlled experiment. The entire outline of this notation, applied on the example used in OPERATIONS
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
C
CONTROL
CHARTS
OF KPIs
GOALS
OBJECTS
THROUGHPUT
DATA
EFFICIENCY
DATA
QUALITY
DATA
INPUT
DATAB
A
ASE
DATABASE
LOWER
VARIABILITEIT
INVARIABILITY
PRODUCTIE
IN PRODUCTION
OUTPUT
DATABASE
DATAB
A
ASE
PROCESS
VARIABLES
PROCURE
HIGH QUALITY
RAW MATERIALS
&
IMPROVE
PRODUCT
QUALITY
OR
INCREASE
CUSTOMER
BASE
DECREASE
PRICE
SUSTAIN
PROFIT
MARGINS
INVENTORY
R
TEAM LEADER
DIDIER
PRODUCTION
PRODUCTION
N
DEPARTMEN
T
DEPARTMENT
I
SUPPLY
L CHAIN
ANAGEMENT
A
A
INVENTORY
MANAGEMENT
R
RFID SCA
SCANNER
C NNER
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
ACCOUNTANT
KATRIEN
K
KA
ATRIEN
CEO
STEVEN
FINANCE
DEPARTMENT
ACTORS
this master thesis, can be found in Appendix C. Figure 9. Entire diagram with cursor on the goal ‘Lower variability in production’ 17 5. Evaluation 5.1.
Test Design In order to determine which representation of CHOOSE is the most comprehensive one, a controlled experiment is conducted. For this research, this approach is considered to be more appropriate than carrying out case studies because it would be impossible to compare different notations based on a real‐life example of an SME without the results being influenced by learning effects. The setting of a controlled experiment allows us to eliminate these effects. However, it is very hard to find sufficient respondents for an experiment of this magnitude within the target group of CHOOSE (i.e. SMEs). Therefore, the experiment is conducted appealing to business engineering students without enterprise modelling experience, as they have many similar characteristics. Once this is known, the decision needs to be made whether a within‐subjects or a between‐
subjects design is used. A major advantage of a within‐subjects design is the need for fewer subjects (Brown & Melamed, 1990). However, this design would dramatically increase the duration of the survey, which could lead to a fatigue bias in the results. Therefore, a between‐
subjects design is applied. This means the students are divided into four groups, and each group receives the same survey but with another visual notation. The goal of the survey is to examine whether the newly established visualisations result in a better cognitive effectiveness on the one hand and in improved perceived ease of use (PEOU), perceived usefulness (PU) and intention to use (IU) on the other hand. These last variables originate from the Technology Acceptance Model (TAM), which states that improvements in these variables increase the chance of adoption (Davis, 1989). TAM is used in accordance to the research of Gopalakrishnan et al. (Gopalakrishnan et al., 2010). As shortcomings are gradually managed within the developed visualisations, it is expected that each alternative outperforms the previous one. The overall hypotheses can be described as follows (i = 1 to 3): Ha,0: visualisation i and visualisation i‐1 have the same cognitive effectiveness Ha,1: visualisation i outperforms visualisation i‐1 in terms of cognitive effectiveness Hb,0: visualisation i and visualisation i‐1 have the same PEOU, PU and IU Hb,1: visualisation i outperforms visualisation i‐1 in terms of PEOU, PU and IU 18 Cognitive effectiveness (CE) is a variable composed out of three other variables: accuracy, time and mental effort. Accuracy (A) is expressed as the percentage of correct answers in the survey. Time (T) is expressed as the average time used to answer a question, while the subjects are asked to report the mental effort (ME) needed to answer a content question on a 9‐point Likert scale (Paas, 1992). Since these variables are expressed in different units of measurement, the variables are standardised before they are combined into the formula of cognitive effectiveness. Analogous to (Tuovinen & Paas, 2004), CE is then calculated as follows: Z A
Z T
Z ME
√3
The survey consists of three major parts, which can be found in Appendix D. This appendix is in Dutch, as the survey is conducted in Dutch and translating the content could introduce a bias. In the first part, some general questions are asked to verify the students’ prior knowledge regarding enterprise architecture, conceptual modelling and IT in general (Appendix D.1). As a between‐subjects design is used, these questions are important to avoid an accidental group selection bias (Gopalakrishnan et al., 2010). The second part comprises 12 questions to examine the understanding of the diagram(s), which are all accompanied by a question that inquires for the mental effort needed to answer the content question (Appendix D.2). The question groups (content + mental effort) are randomised in order to avoid obtaining overall better results for the last questions. The third and last part consists of 14 questions based on (Gopalakrishnan et al., 2010) that gauge the PEOU, PU and IU (Appendix D.3). Some of these questions are negated in order to avoid one‐sided answers and a control question is included to verify whether the respondents conscientiously fill in the survey. The answers are measured on a 5‐point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’. 5.2.
Experiment Results In total, 120 useful observations can be analysed. Six results are omitted, because there are clear indications that those students have not conscientiously filled in the survey. The four sample sizes are slightly different, ranging from 29 responses to 32. Descriptive statistics for each variable can be found in Table 2. 19 Table 2. Descriptive statistics per group Current notation (N=29) Alternative 1 (N=30) Alternative 2 (N=29) Alternative 3 (N=32) Variable Mean SD Mean SD Mean SD Mean SD CE ‐1.0353 0.9941 ‐0.4365 1.0696 ‐0.3956 1.0331 0.7080 0.8134 A 0.8276 0.1230 0.8810 0.1344 0.8916 0.1303 0.9665 0.0544 8.6862 31.6336 6.3017 33.4239 5.7900 27.7378 4.6965 T 35.6616 ME 3.3736 1.0176 3.4333 1.0941 3.1695 1.0323 2.7891 0.9499 PEOU 3.2690 0.3752 3.1533 0.4862 3.2138 0.6255 3.4375 0.4172 PU 3.6621 0.5017 3.6533 0.7482 3.5448 0.6277 4.0250 0.3619 IU 3.0776 0.7621 3.0750 0.7689 3.3707 0.5733 3.5625 0.5198 The variable CE satisfies all criteria to be analysed by means of an ANOVA, while the other variables violate at least one of the assumptions (see Appendix E.1). Therefore, these variables are examined with the Kruskal‐Wallis test and the Mann‐Whitney U test. These tests assume that the distributions of the different groups have equal shapes. It should be mentioned however that this assumption is not entirely satisfied for the variables ME and PEOU. For the variable ME, the distribution shape of alternative 2 is the only one that deviates from the shapes of the other visualisations. Hence, the analysis of this variable still generates clear conclusions for the comparison of the current visualisation, the first alternative and the third alternative. Only for the comparisons in which alternative 2 is incorporated, careful interpretation is needed. For the variable PEOU, alternative 1 and 3 have comparable distribution shapes. Yet, the shapes of the current notation and the second alternative are different. Therefore, this variable should be cautiously analysed. All analyses have been conducted with a significance level of 5%. Table 3. Test results of the pairwise comparisons Variable Test statistic 0 – 1 CE MD A U 311* T U 275** ME U PEOU U 0.5988 406.5 323.5 0 – 2 0.6397 0 – 3 ***
1 – 3 ***
1.1036*** 0.0409 263** 128*** 429 296** 279** 337 182*** 333 284** 214*** 310* 352 312.5** 370.5 361.5 356.5 **
PU U 402 369 271 IU U 400.5 286.5 267.5** 20 320 1.1445 2 – 3 1.7433 Note: MD = mean difference (Tukey HSD); U = Mann‐Whitney U *
P<0.05; **P<0.01; ***P<0.001. 1 – 2 379 **
382 *
310.5 373 365 260** 286.5** 273** 360 The results in Table 3 demonstrate that the third alternative has a significantly higher cognitive effectiveness than the other visual notations, while the differences between the other notations are not significant. These results can be explained by analysing the component variables of cognitive effectiveness. All three alternatives have better scores for accuracy than the current notation, but alternative 3 outperforms alternative 1 and 2. Next to this, the average time needed to answer a question is tremendously lower for alternative 3 than for the other alternatives. And last, only for the third alternative, the mental effort required to interpret the notation is significantly lower than for the current notation. For the variable PEOU, the only significant result that can be observed is the difference between alternative 1 and 3. The boxplots in Figure 10 reveal that alternatives 2 and 3 have a higher median than the current notation and the first alternative, yet the differences are not significant. Possibly, the true significance level has shifted due to the unequally shaped distributions (Skovlund & Fenstad, 2001). For the variables PU and IU, the results are more straightforward. Concerning PU, alternative 3 has significantly better results than the other notations. Finally, the IU is significantly better for alternative 2 and 3 than for the current notation and the first alternative. The exact p values of the test results can be found in Appendix E.2. 4,50
PEOU_Avg
4,00
3,50
3,00
2,50
2,00
0
1
2
Visualisation
Figure 10. Boxplots of the variable PEOU 21 3
6. Discussion The experiment results demonstrate that the last visual notation is clearly the best alternative. First of all, this notation is cognitively more effective than the others. Besides, the respondents of this notation have indicated a high perceived usefulness and intention to use. It is therefore advised to implement this notation. In order to facilitate this implementation, a disc that contains the appropriate files is enclosed (Appendix F). Several statements can be made in the context of this experiment: 1. When alternative 1 is compared to the current notation, the conclusion can be made that incorporating semiotic clarity, perceptual discriminability, semantic transparency and visual expressiveness improves the accuracy and speed of the answers. However, the change in cognitive effectiveness is not significant due to the variable mental effort, which is not improved. 2. When, on top of these principles, complexity management is applied, an impressive difference can be observed. Adding this principle results in a significant increase in the cognitive effectiveness of the notation. This can be concluded when alternative 3 is compared to the other visualisations. 3. Enhancing the aesthetics of the notation does not improve the cognitive effectiveness of it, nor one of its component variables (alternative 2 vs. 1). 4. However, ameliorating the aesthetics does lead to a higher intention to use. The results for this variable are significantly better for alternative 2 and 3 compared to the current notation and the first alternative. 5. Integrating all five considered principles leads to a higher perceived usefulness of the notation. As the PU is not improved when only the first four principles are incorporated, the idea rises that the principle of complexity management causes the increase in PU. Overall, it can be said that both Moody’s principles and aesthetics have a positive influence on the notation, and this in a complementary way. Moody’s principles improve the comprehensibility of the notation and lead to an increase in perceived usefulness. Aesthetics on the other hand augment the intention to use the notation. 22 7. Conclusion and Future Research This research has investigated the visual notation of CHOOSE, which is an EA approach developed by Bernaert et al. with the aim to facilitate the implementation of EA in the context of SMEs (Bernaert, Poels, et al., 2013b). The current visualisation has been evaluated and alternative visualisations have been established, after which the different visualisations have been compared in an experiment. Based on this experiment, an advice has been made to implement one of the notations in the CHOOSE approach. The result of the investigation facilitates a cognitively effective interpretation of CHOOSE diagrams on the one hand, and improves the perceived usefulness and the intention to use the notation on the other hand. The only variable for which no clear conclusion can be made is the perceived ease of use of the notation. This could be further analysed in future work. In practice, implementing the suggested visualisation in the CHOOSE approach should lead to an effective and time efficient way of dealing with EA, and hence improve its adoption rate in SMEs. However, as the experiment is conducted appealing to students, this aspect is ought to be further analysed by means of executing case studies or experiments in SMEs. Although the students subjected to the experiment have several characteristics in common with employees of SMEs – they have for example a keen interest in business topics and are novices in enterprise modelling – it is difficult to extrapolate the results of this investigation to the target group of SMEs. Besides these practical implications, this master thesis also provides a validation for the Physics of Notations. The research reveals that applying its principles significantly improves the comprehensibility of the notation. On top of this, it becomes clear that aesthetics should not be neglected, as this increases the intention to use the notation. At last, this master thesis suggests a methodology to evaluate and improve visual notations. Although this research is conducted in the context of CHOOSE, the positive outcome of this case might motivate researchers to consider following the same path. 23 Bibliography Bernaert, M. (2011). De zoektocht naar know‐how, know‐why, know‐what en know‐who: architectuur voor kleinere bedrijven in vier dimensies. INFORMATIE (AMSTERDAM), 53(9), 34‐41. Bernaert, M., Maes, J., & Poels, G. (2013). An Android Tablet Tool for Enterprise Architecture Modeling in Small and Medium‐Sized Enterprises. Practice of Enterprise Modeling (Vol. 165, pp. 145‐160): Springer Berlin Heidelberg. Bernaert, M., & Poels, G. (2011). The quest for know‐how, know‐why, know‐what and know‐
who: using KAOS for enterprise modelling. Paper presented at the 6th International Workshop on BUSinness/IT ALignment and Interoperability (BUSITAL 2011), London, UK. Bernaert, M., Poels, G., Snoeck, M., & De Backer, M. (2013a). Bridging EA for SMEs to EA for Large Enterprises: Mapping CHOOSE on the ArchiMate Standard. Department of Management Information Systems and Operations Management. Ghent University, K.U. Leuven, University of Antwerp. Bernaert, M., Poels, G., Snoeck, M., & De Backer, M. (2013b). CHOOSE: Towards a Metamodel for Enterprise Architecture in Small and Medium‐Sized Enterprises. Department of Management Information Systems and Operations Management. Ghent University, K.U. Leuven, University of Antwerp. Bernaert, M., Poels, G., Snoeck, M., & De Backer, M. (2013c). Enterprise architecture for small and medium‐sized enterprises: a starting point for bringing EA to SMEs, based on adoption models. Information Systems and Small and Medium‐sized Enterprises: State of art of IS research in SMEs: Springer Berlin Heidelberg. Bertin, J. (1983). Semiology of Graphics: Diagrams, Networks, Maps. Madison, Wisconsin, USA: University of Wisconsin Press. Bhagwat, R., & Sharma, M. K. (2007). Information system architecture: a framework for a cluster of small‐ and medium‐sized enterprises (SMEs). Production Planning & Control, 18(4), 283‐296. Brown, S. R., & Melamed, L. E. (1990). Experimental design and analysis: Sage. Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319‐340. Dehbokry, S. G., & Chew, E. K. (2014). The Strategic Requirements for an Enterprise Business Architecture Framework by SMEs. Lecture Notes on Information Theory, 2(1). 24 Dumeez, J., Bernaert, M., & Poels, G. (2013). Development of Software Tool Support for Enterprise Architecture in Small and Medium‐Sized Enterprises. Advanced Information Systems Engineering Workshops, Lecture Notes in Business Information Processing (Vol. 148, pp. 87‐98): Springer Berlin Heidelberg. Genon, N., Amyot, D., & Heymans, P. (2011). Analysing the Cognitive Effectiveness of the UCM Visual Notation. System Analysis and Modeling (Vol. 6598, pp. 221‐240): Springer Berlin Heidelberg. Genon, N., Heymans, P., & Amyot, D. (2011). Analysing the Cognitive Effectiveness of the BPMN 2.0 Visual Notation. Software Language Engineering (Vol. 6563, pp. 377‐
396): Springer Berlin Heidelberg. Glissmann, S. M. (2011). An Approach to Building Effective Enterprise Architectures. Paper presented at the 44th Hawaii International Conference on System Sciences (HICSS). Goodman, N. (1976). Languages of art: An approach to a theory of symbols: Hackett Publishing. Gopalakrishnan, S., Krogstie, J., & Sindre, G. (2010). Adapting UML Activity Diagrams for Mobile Work Process Modelling: Experimental Comparison of Two Notation Alternatives. Practice of Enterprise Modeling (Vol. 68, pp. 145‐161): Springer. Green, T. R. G., Blandford, A. E., Church, L., Roast, C. R., & Clarke, S. (2006). Cognitive dimensions: Achievements, new directions, and open questions. Journal of Visual Languages & Computing, 17(4), 328‐365. Huang, W., Eades, P., & Hong, S.‐H. (2009). Measuring effectiveness of graph visualizations: A cognitive load perspective. Information Visualization, 8(3), 139‐152. Ingelbeen, D., Bernaert, M., & Poels, G. (2013). Enterprise Architecture Software Tool Support for Small and Medium‐Sized Enterprises: EASE. Paper presented at the 19th Americas Conference on Information Systems. Jacobs, D., Kotzé, P., van der Merwe, A., & Gerber, A. (2011). Enterprise Architecture for Small and Medium Enterprise Growth. Paper presented at the First Enterprise Engineering Working Conference, Antwerp. Kamsties, E., Hörmann, K., & Schlich, M. (1998). Requirements engineering in small and medium enterprises. Requirements Engineering, 3(2), 84‐90. Krogstie, J., Sindre, G., & Havard, J. (2006). Process models representing knowledge for action: a revised quality framework. European Journal of Information Systems, 15(1), 91‐102. Lankhorst, M. (2013). Enterprise architecture at work (3 ed.). New York: Springer. 25 Larkin, J. H., & Simon, H. A. (1987). Why a Diagram is (Sometimes) Worth Ten Thousand Words. Cognitive Science, 11(1), 65‐100. Maes, R. (2007). An integrative perspective on information management. Information Management: Setting the Scene, 11‐26. Moody, D. (2009a). The "Physics" of Notations: Towards a Scientific Basis for Constructing Visual Notations in Software Engineering. IEEE Transactions on Software Engineering, 35(5), 756‐778. Moody, D. (2009b, 20‐24 Sept.). Theory development in visual language research: Beyond the cognitive dimensions of notations. Paper presented at the Visual Languages and Human‐Centric Computing (VL/HCC). Moody, D., Heymans, P., & Matulevičius, R. (2010). Visual syntax does matter: improving the cognitive effectiveness of the i* visual notation. Requirements Engineering, 15(2), 141‐
175. Moody, D., & Hillegersberg, J. (2009). Evaluating the Visual Syntax of UML: An Analysis of the Cognitive Effectiveness of the UML Family of Diagrams. Software Language Engineering (Vol. 5452, pp. 16‐34): Springer. Moser, C., Junginger, S., Brückmann, M., & Schöne, K.‐M. (2009). Some Process Patterns for Enterprise Architecture Management. Paper presented at the Software Engineering ‐ Workshopband. Opdahl, A. L., & Henderson‐Sellers, B. (2002). Ontological evaluation of the UML using the Bunge‐Wand‐Weber model. Software & systems modeling(1), 43‐67. Paas, F. G. (1992). Training strategies for attaining transfer of problem‐solving skill in statistics: A cognitive‐load approach. Journal of educational psychology, 84(4), 429. Recker, J. C., Indulska, M., Rosemann, M., & Green, P. (2005). Do process modelling techniques get better? A comparative ontological analysis of BPMN. Skovlund, E., & Fenstad, G. U. (2001). Should we always choose a nonparametric test when comparing two apparently nonnormal distributions? Journal of Clinical Epidemiology, 54(1), 86‐92. Sonderegger, A., & Sauer, J. (2010). The influence of design aesthetics in usability testing: Effects on user performance and perceived usability. Applied Ergonomics, 41(3), 403‐
410. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295‐312. 26 The Open Group. (2012). ArchiMate® 2.0 Specification. Tuovinen, J., & Paas, F. (2004). Exploring Multidimensional Approaches to the Efficiency of Instructional Conditions. Instructional Science, 32(1‐2), 133‐152. Zutterman, S. (2013). Development of a Tool for Business Architecture Modeling in Eclipse. (Master of Science in Applied Economic Sciences: Business Engineering), Ghent University, Ghent. 27 Appendices Appendix A
Constructs of CHOOSE Table A.1. Constructs of CHOOSE: definitions (adjusted from Zutterman, 2013, p.85‐86) Construct Definition Goal viewpoint Goal An end state an actor wishes to or is assigned to achieve, which is pursued by executing the appropriate operations. AND relationship A goal can be achieved by realising all sub goals. OR relationship A goal can be achieved by realising different sub goals, yet not all of them must be realised. Conflict A goal can be in conflict with another goal. Actor viewpoint Actor A not specified organisational entity. Human actor A human being, able to perform a role in the organisation and to be involved in operations. Role A function, held by a human actor. Software actor A software system, or part of a software system, that encapsulates its behaviour and data to execute operations. Device A hardware resource, or physical equipment, able to execute operations. Division Indicates that an actor groups a number of other actors. Supervision A supervisee reports to a supervisor. Performs Links roles with human actors that fulfil them. Operations viewpoint Operation Internal behaviour that needs objects as input and produces objects as output with the intent to operationalise goals. Process A continuous type of operation. Project A temporary type of operation. Includes Indicates that an operation groups a number of other operations. Object viewpoint Object A passive element that is relevant from a business, information or technology point of view. It may or may not be physical. Aggregation Indicates that an object groups a number of other objects. Specialisation Indicates that an object is a more specific form of another object. Association Models a relationship between objects that is not covered by another, more specific, relationship. I Table A.1. (Continued) Constructs of CHOOSE: definitions (adjusted from Zutterman, 2013, p.85‐86) Construct Definition Relationships between viewpoints Goals – Actors Wish Captures the fact that an actor would like a goal to be satisfied. Assignment An actor is assigned to a goal if he/she/it is required to restrict his/her/its behaviour so as to satisfy the goal. Goals – Operations Operationalisation Goals are achieved by executing operations. Goals – Objects Concern Connects goals to the objects involved in attaining the goal. Actors – Operations Responsible An actor is responsible for the execution of the operation until the work is finished and approved by an actor who is accountable for the work. Accountable The actor who is responsible for the end result of an operation. This actor approves the work of the actor who is responsible for the execution. Consulted The actor whose opinion is sought while performing the work, and with whom there is two‐way communication. Informed The actor who is kept up‐to‐date about the progress or result of an operation, and with whom there is just one‐way communication. Actors – Objects Control An actor controls an object when he/she/it can create or alter the state of that object. Monitor An actor monitors an object when he/she/it can observe the state of an object, but cannot alter it. Operations ‐ Objects Input Output Connects an operation with the object needed to execute that operation. Connects an operation with the result yielded in the form of an object. II Appendix B
Construct
Current
Symbols Applied in the Different Visual Notations Alt. 1
Alt. 2 & 3 Construct
Goal viewpoint
Goal
Goal
Goal
&
OR
Alt. 2 & 3
GOAL
Actor
ACTOR
Human actor
Human
actor
HUMAN
ACTOR
Role
Role
ROLE
Software
SOFTWARE
ACTOR
Device
DEVICE
Actor
&
Human actor
OR relation
Alt. 1
Actor viewpoint
Actor
AND relation
Current
OR
Conflict
Role
Operations viewpoint
Operation
Operation
Operation
OPERATION
Process
Process
Process
PROCESS
Project
Project
Project
PROJECT
Software actor
Device
Software actor
Device
Division
Supervision
Includes
Performs
Objects viewpoint
Object
Object
Object
Human actor
HUMAN ACTOR
Role
R OLE
Relationships between goals and actors
OBJECT
Wish
Assignment
Aggregation
Relationships between actors and objects
Control
Specialisation
Monitor
Association
Relationships between goals and objects
Relationships between actors and operations
Responsible
R
R
Accountable
A
A
Consulted
C
C
Informed
I
I
Concern
Relationships between objects and operations
Input
Output
Relationships between goals and operations
Operationalisation
Figure B.1. Legend: symbols applied in the different visual notations III Appendix C
C.1
Outline Third Visualisation Alternative Overview GOALS
ACTORS
OBJECTS
OPERATIONS
Figure C.1. Overview of the four viewpoints C.2
Goal Viewpoint INCREASE
CUSTOMER
BASE
OR
IMPROVE
PRODUCT
QUALITY
DECREASE
PRICE
SUSTAIN
PROFIT
MARGINS
&
LOWER
VARIABILITY
IN PRODUCTION
PROCURE
HIGH QUALITY
RAW MATERIALS
Figure C.2. Goal viewpoint IV C.3
Actor Viewpoint STEVEN
CEO
PRODUCTION
DEPARTMENT
FINANCE
DEPARTMENT
CHRISTOPHE
KATRIEN
PRODUCTION
ACCOUNTANT
MANAGER
DIDIER
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
TEAM LEADER
RFID SCANNER
Figure C.3. Actor viewpoint C.4
Operation Viewpoint SUPPLY CHAIN
MANAGEMENT
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
INVENTORY
MANAGEMENT
Figure C.4. Operation viewpoint V C.5
Object Viewpoint INPUT
DATABASE
QUALITY
DATA
PROCESS
VARIABLES
INVENTORY
OUTPUT
DATABASE
CONTROL
CHARTS
OF KPIs
EFFICIENCY
DATA
THROUGHPUT
DATA
Figure C.5. Object viewpoint C.6
Pairwise Relationships between Goals and Actors STEVEN
INCREASE
CUSTOMER
BASE
GOALS
CEO
KATRIEN
SUSTAIN
PROFIT
MARGINS
ACCOUNTANT
ACTORS
CHRISTOPHE
IMPROVE
PRODUCT
QUALITY
PRODUCTION
MANAGER
Figure C.6. Pairwise relationships between goals and actors VI C.7
Pairwise Relationships between Goals and Operations LOWER
VARIABILITY
IN PRODUCTION
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
GOALS
OPERATIONS
PROCURE
HIGH QUALITY
RAW MATERIALS
INVENTORY
MANAGEMENT
Figure C.7. Pairwise relationships between goals and operations C.8
Pairwise Relationships between Goals and Objects LOWER
VARIABILITY
IN PRODUCTION
PROCESS
VARIABLES
GOALS
OBJECTS
PROCURE
HIGH QUALITY
RAW MATERIALS
INVENTORY
Figure C.8. Pairwise relationships between goals and objects VII C.9
Pairwise Relationships between Objects and Actors CHRISTOPHE
PRODUCTION
MANAGER
OBJECTS
ACTORS
INVENTORY
DIDIER
TEAM LEADER
Figure C.9. Pairwise relationships between objects and actors C.10
Pairwise Relationships between Objects and Operations INVENTORY
INVENTORY
MANAGEMENT
OBJECTS
PROCESS
VARIABLES
OPERATIONS
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
CONTROL
CHARTS
OF KPIs
Figure C.10. Pairwise relationships between objects and operations VIII C.11
Pairwise Relationships between Actors and Operations STEVEN
CEO
A
CHRISTOPHE
PRODUCTION
R
MANAGER
A
ACTORS
KATRIEN
DIDIER
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
OPERATIONS
C
I
TEAM LEADER
INVENTORY
MANAGEMENT
R
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
R
RFID SCANNER
Figure C.11. Pairwise relationships between actors and operations IX GOALS
Figure C.12. Entire diagram: all viewpoints included X OBJECTS
THROUGHPUT
DATA
EFFICIENCY
DATA
QUALITY
DATA
INPUT
DATABASE
LOWER
VARIABILITEIT
INVARIABILITY
PRODUCTIE
IN PRODUCTION
&
IMPROVE
PRODUCT
QUALITY
DECREASE
PRICE
OUTPUT
DATABASE
PROCESS
VARIABLES
PROCURE
HIGH QUALITY
RAW MATERIALS
OR
INCREASE
CUSTOMER
BASE
CONTROL
CHARTS
OF KPIs
INVENTORY
SUSTAIN
PROFIT
MARGINS
R
C
TEAM LEADER
DIDIER
I
A
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
SUPPLY CHAIN
MANAGEMENT
R
RFID SCANNER
OPERATIONS
INVENTORY
MANAGEMENT
A
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
ACCOUNTANT
PRODUCTION
MANAGER
KATRIEN
FINANCE
DEPARTMENT
CHRISTOPHE
PRODUCTION
DEPARTMENT
CEO
STEVEN
ACTORS
C.12
Entire Diagram &
IMPROVE
PRODUCT
QUALITY
XI Figure C.13. Entire diagram with cursor on the goal ‘Lower variability in production’ THROUGHPUT
DATA
EFFICIENCY
DATA
QUALITY
DATA
INPUT
DATAB
A
ASE
DATABASE
LOWER
VARIABILITEIT
INVARIABILITY
PRODUCTIE
IN PRODUCTION
OBJECTS
GOALS
DECREASE
PRICE
OUTPUT
DATABASE
DATAB
A
ASE
PROCESS
VARIABLES
PROCURE
HIGH QUALITY
RAW MATERIALS
OR
INCREASE
CUSTOMER
BASE
CONTROL
CHARTS
OF KPIs
INVENTORY
SUSTAIN
PROFIT
MARGINS
R
C
TEAM LEADER
DIDIER
SUPPLY
L CHAIN
ANAGEMENT
I
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
PRODUCTION
PRODUCTION
N
DEPARTMEN
T
DEPARTMENT
A
CEO
STEVEN
R
RFID SCA
SCANNER
C NNER
OPERATIONS
INVENTORY
MANAGEMENT
A
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
ACCOUNTANT
KATRIEN
K
KA
ATRIEN
FINANCE
DEPARTMENT
ACTORS
C.13
Entire Diagram with Cursor on ‘Lower Variability in Production’ GOALS
XII Figure C.14. Entire diagram with cursor on the project ‘Start up implementation lean principle’ OBJECTS
THROUGHPUT
DATA
EFFICIENCY
DATA
QUALITY
DATA
INPUT
DATABASE
DATAB
A
ASE
LOWER
VARIABILITEIT
VARIABILITY
IN PRODUCTION
&
IMPROVE
PRODUCT
QUALITY
DECREASE
PRICE
OUTPUT
DATABASE
DATAB
A
ASE
PROCESS
VARIABLES
PROCURE
HIGH QUALITY
RAW MATERIALS
OR
INCREASE
CUSTOMER
BASE
CONTROL
CHARTS
OF KPIs
INVENTORY
SUSTAIN
PROFIT
MARGINS
R
C
TEAM LEADER
DIDIER
MANAGER
PRODUCTION
CHRISTOPHE
SUPPLY CHAIN
MANAGEMENT
I
START UP
IMPLEMENTATION
LEAN
PRINCIPLE
PRODUCTION
PRODUCTION
N
DEPARTMEN
T
DEPARTMENT
A
CEO
STEVEN
R
RFID SCA
SCANNER
C NNER
OPERATIONS
INVENTORY
MANAGEMENT
A
SOFTWARE FOR
AUTOMATIC
INVENTORY
REPLENISHMENT
ACCOUNTANT
KATRIEN
K
KA
ATRIEN
FINANCE
DEPARTMENT
ACTORS
C.14
Entire Diagram with Cursor on ‘Start up Implementation Lean Principle’ Appendix D
D.1
Survey Questions Pre Experiment Questionnaire Algemene vragen 1. Welke opleiding volg je momenteel? (Indien je een GIT‐traject volgt, duid de meest gevorderde opleiding aan) o
o
o
o
o
o
2a. Wordt gesteld indien het antwoord op vraag 1 ‘Bachelor’ was Wat is het hoogste jaar waar je op dit moment voor bent ingeschreven? o
o
o
1e bachelor 2e bachelor 3e bachelor 2b. Wordt gesteld indien het antwoord op vraag 1 ‘Master’ was Wat is het hoogste jaar waar je op dit moment voor bent ingeschreven? o
o
Bachelor Master Master na master Schakelprogramma Voorbereidingsprogramma Doctoraatsopleiding 1e master 2e master 3a. Wordt gesteld indien het antwoord op vraag 1 ‘Bachelor, Master, Master na master, Schakelprogramma of Voorbereidingsprogramma’ was Hoe heet je studierichting? (Open vraag) 3b. Wordt gesteld indien het antwoord op vraag 1 ‘Doctoraatsopleiding’ was In welk domein situeert het onderwerp van je doctoraat zich? (Open vraag) 4. Welke vakken die betrekking hebben op informatica heb je reeds gevolgd, of volg je dit semester? (Meedere antwoorden mogelijk) o
o
o
o
o
Inleiding tot informatica (vb. informatica I voor HIR) Programmeren (vb. informatica II voor HIR) Beleidsinformatica ICT‐management Andere: _______________ XIII 5. Wat is je geslacht? o
o
Man Vrouw 6. Wat is je leeftijd? o
o
o
o
< 18 jaar 18 – 25 jaar 26 – 30 jaar > 30 jaar Voorkennis van EA en CHOOSE 7. Ken je de term ‘enterprise architecture’? o
o
Ja Nee 8. Wordt gesteld indien het antwoord op vraag 7 ‘Ja’ was Heb je al eens van de enterprise architecture techniek CHOOSE gehoord? o
o
Ja Nee 9. Wordt gesteld indien het antwoord op vraag 7 en 8 ‘Ja’ was Heb je reeds modellen gezien die gemaakt zijn met CHOOSE? o
o
Ja Nee Korte uitleg over CHOOSE 10. CHOOSE is een manier om verschillende aspecten van een onderneming weer te geven en te analyseren. Er bestaan 4 delen: Doelen  Wat wil de onderneming bereiken? GOALS
OBJECTS
ACTORS
OPERATIONS
Actoren  Iets of iemand die een actie kan uitvoeren. Operaties  Hoe doelen worden bereikt. Objecten  Passieve elementen die worden gebruikt. Binnen en tussen deze 4 delen bestaan er verschillende relaties. XIV Model en legende 11a. Wordt gesteld in de enquêtes van de huidige notatie, het 1e en het 2e alternatief Neem het bundeltje dat voor je ligt en draai de eerste pagina om. Nu zie je de legende en een diagram dat gemaakt is met CHOOSE. Bekijk beiden grondig. Nadien zullen een aantal inhoudelijke vragen over het diagram gesteld worden. Klik op volgende als je klaar bent om naar de vragen over te gaan. Na 2 minuten wordt automatisch naar de volgende pagina overgegaan. 11b. Wordt gesteld in de enquête van het 3e alternatief Neem het bundeltje dat voor je ligt en draai de eerste pagina om. Nu zie je de legende en een diagram dat gemaakt is met CHOOSE. Bekijk beiden grondig. Nadien zullen een aantal inhoudelijke vragen gesteld worden waarvoor je de diagrammen op de volgende pagina's nodig zal hebben. Klik op volgende als je klaar bent om naar de vragen over te gaan. Na 2 minuten wordt automatisch naar de volgende pagina overgegaan. 12a. Wordt gesteld in de enquêtes van de huidige notatie, het 1e en het 2e alternatief Hierna volgen een aantal inhoudelijke vragen over het diagram. Na elke inhoudelijke vraag volgt een vraag die polst hoe moeilijk het was om de inhoudelijke vraag op te lossen. Klik op ‘Volgende’ om naar de vragen te gaan. 12b. Wordt gesteld in de enquête van het 3e alternatief Hierna volgen een aantal inhoudelijke vragen over de diagrammen. Vóór elke vraag zal je worden gevraagd een bepaald diagram erbij te nemen. Klik dan pas op ‘Volgende’ als het diagram voor je ligt. Maak het eerste blad van de bundel los om de legende naast de diagrammen te kunnen leggen. Na elke inhoudelijke vraag volgt een vraag die polst hoe moeilijk het was om de inhoudelijke vraag op te lossen. Klik op ‘Volgende’ om naar de vragen te gaan. XV D.2
Experiment: Content Questions Gebruikte diagrammen in de enquête: Huidige notatie: Figure 2 p.6 Eerste alternatief: Figure 5 p.14 Tweede alternatief: Figure 6 p.15 Derde alternatief: Nederlandstalige versies van de diagrammen uit Appendix C, vermeld bij elke vraag. Vragen worden één voor één op het scherm getoond en worden per nummer in willekeurige volgorde gesteld (Xa en Xb blijven bij elkaar, met X het nummer van de vraag). 1a. Welke relatie bestaat er tussen de boekhouder en het doel om de marges op peil te houden? (Figure C.6 voor alternatief 3) De boekhouder heeft de opdracht om het doel 'marges op peil houden' te bereiken (assignment) De boekhouder heeft de wens om de marges op peil te houden (wish) De boekhouder wordt geraadpleegd om na te gaan hoe het doel 'marges op peil houden' kan worden bereikt (consulted) Geen van bovenstaande mogelijkheden o
o
o
o
1b. Hoeveel moeite heb je moeten doen om de vorige vraag op te lossen? Extreem weinig moeite 1 2 3 4
○ ○ ○ ○
Neutraal 5
6
7
8 9 ○
○
○
○ ○ 2a. Het doel ‘kwalitatieve grondstoffen aankopen’ heeft betrekking op: (Figure C.8 voor alternatief 3) o
o
o
o
Voorraad Procesvariabelen Controlekaarten van KPIs Input databank 2b. Idem 1b 3a. Hoe wordt het doel ‘variabiliteit in productie verlagen’ bereikt? (Figure C.13 voor alternatief 3) o
o
o
o
Extreem
veel moeite Door de kwaliteit van het product te verhogen Door de voorraad te manipuleren Door ook kwalitatieve grondstoffen aan te kopen Door het lean principe te implementeren 3b. Idem 1b XVI 4a. Welk verband bestaat er tussen de ploegbaas en de voorraad? (Figure C.9 voor alternatief 3) o
o
o
o
De ploegbaas kan de voorraad observeren, maar niet aanpassen (monitor) De ploegbaas heeft controle over de voorraad en kan dus ook aanpassingen doen (control) De ploegbaas is verantwoordelijk voor de voorraad (responsible) Er is geen verband tussen de ploegbaas en de voorraad 4b. Idem 1b 5a. Duid de correcte stellingen aan (meerdere antwoorden mogelijk): (Figure C.11 voor alternatief 3) o
o
o
o
o
o
De ploegbaas is verantwoordelijk voor de uitvoering van de lean implementatie (responsible) De ploegbaas wordt geraadpleegd bij de lean implementatie (consulted) De productiemanager is verantwoordelijk voor de uitvoering van de lean implementatie (responsible) De CEO wordt geïnformeerd over de lean implementatie (informed) De productiemanager is eindverantwoordelijke voor het resultaat van de lean implementatie (accountable) De CEO is eindverantwoordelijke voor het resultaat van de lean implementatie (accountable) 5b. Idem 1b 6a. Wat is de output van de activiteit ‘lean implementeren’? (Figure C.14 voor alternatief 3) o
o
o
o
Procesvariabelen Controlekaarten van KPIs Supply chain management Geen van bovenstaande mogelijkheden 6b. Idem 1b 7a. Het klantenbestand kan vergroot worden door: (Figure C.2 voor alternatief 3) o
o
o
én de kwaliteit van het product te verhogen én de prijs te verlagen de kwaliteit van het product te verhogen of de prijs te verlagen dit is niet uit het diagram af te leiden 7b. Idem 1b XVII 8a. De kwaliteit van het product kan verhoogd worden door: (Figure C.2 voor alternatief 3) o
o
o
én de variabiliteit in de productie te verlagen én kwalitatieve grondstoffen aan te kopen ofwel de variabiliteit in de productie te verlagen ofwel kwalitatieve grondstoffen aan te kopen dit is niet uit het diagram af te leiden 8b. Idem 1b 9a. Welke rol vervult Christophe? (Figure C.3 voor alternatief 3) (Open vraag) 9b. Idem 1b 10a. Van welk departement maakt Katrien deel uit? (Figure C.3 voor alternatief 3) (Open vraag) 10b. Idem 1b 11a. Wat is het verband tussen ‘procesvariabelen’ en ‘kwaliteitsgegevens’? (Figure C.5 voor alternatief 3) o
o
o
o
Kwaliteitsgegevens maken deel uit van procesvariabelen (aggregatie) Kwaliteitsgegevens zijn een specifiekere vorm van procesvariabelen (specialisatie) Kwaliteitsgegevens is 1 van de 3 verplichte onderdelen van procesvariabelen Geen van bovenstaande mogelijkheden 11b. Idem 1b 12a. Wat is het verschil tussen ‘voorraadbeheer’ en ‘lean implementeren’? (Figure C.4 voor alternatief 3) o
o
o
o
Voorraadbeheer is een niet‐nader‐gespecifieerde operatie en lean implementeren is een proces Lean implementeren is een proces en voorraadbeheer is een project Voorraadbeheer is een proces en lean implementeren een project Geen van bovenstaande mogelijkheden 12b. Idem 1b XVIII D.3
Post Experiment Questionnaire Table D.1. Post Experiment Questionnaire Id. Gerelateerde TAM factor Vraag 1 Ik vond deze notatie gemakkelijk te interpreteren. PEOU 2 Het zou gemakkelijk zijn om deze notatie in gebruik te nemen. PU 3 Deze notatie is gemakkelijk te onthouden. PU 4 De notatie gaf mij inzicht in de doelen, actoren, operaties en objecten van de onderneming. PEOU 5 Mocht ik in de toekomst de architectuur van een KMO willen uiteenzetten, dan zou ik deze notatie niet gebruiken. IU 6 Ik vond het moeilijk om deze notatie te gebruiken en te herkennen. PEOU 7 Duid hier ‘niet akkoord’ aan. / Als ik in een KMO zou werken waar een overleg plaatsvindt ivm de 8 techniek om de architectuur uiteen te zetten, dan zou ik deze notatie aanraden. IU 9 Ik kan de notatie snel begrijpen. PU 10 Ik zou de inhoud kunnen interpreteren zonder op voorhand een legende te zien. 11 Ik vond de verschillende elementen in het diagram niet verwarrend. PEOU PEOU 12 In het geval ik een bedrijfsarchitectuur zou moeten analyseren, dan zou ik deze notatie gebruiken. IU 13 Deze notatie laat mij toe de inhoud sneller te begrijpen dan mocht het in een doorlopende tekst weergegeven zijn. PU 14 Deze notatie gaf structuur aan de weergegeven informatie. PU Mocht ik als freelance consultant voor een klant werken die hulp nodig 15 heeft bij het uiteenzetten van de bedrijfsarchitectuur, dan zou ik deze notatie gebruiken tijdens overlegmomenten met die klant. IU XIX Appendix E
E.1.
Statistical analysis Assumptions ANOVA Test In order to analyse data by means of an ANOVA test, three assumptions must be satisfied: ‐
‐
‐
The variable is normally distributed The samples (groups) have the same variance (homogeneity of variance) The observations are independent of each other. The last assumption is satisfied, since a between‐subjects design is used for the experiment. The other two assumptions need to be examined. Tests of Normality Hypotheses: H0: the variable is normally distributed H1: the variable is not normally distributed Test: Table E.1. Tests of Normality Kolmogorov-Smirnova
Statistic
df
Shapiro-Wilk
Sig.
Statistic
*
df
Sig.
Cogn_Eff
.066
120
.200
.985
120
.191
Accuracy_percentage
.245
120
.000
.815
120
.000
Time_Avg
.088
120
.024
.967
120
.004
Effort_avg
.078
120
.069
.969
120
.007
PEOU_Avg
.108
120
.001
.964
120
.003
PU_Avg
.189
120
.000
.934
120
.000
IU_Avg
.137
120
*. This is a lower bound of the true significance.
.000
.943
120
.000
a. Lilliefors Significance Correction
Conclusion: As soon as the p value of one of the tests is lower than 0.05, it is assumed that the variable is not normally distributed. This means only the variable Cognitive Effectiveness is normally distributed. All variables, apart from Cognitive Effectiveness, will hence be analysed by means of a Kruskal‐
Wallis test and Mann‐Whitney U tests. XX Homogeneity of Variances In order to apply an ANOVA test on the variable Cognitive Effectiveness, homogeneity of variances is required. Therefore, a Levene’s test is conducted. Hypotheses: H0: For the variable Cognitive Effectiveness, the four groups have equal variances H1: Not all groups have the same variance Test: Table E.2. Cognitive Effectiveness: test of homogeneity of variances Levene Statistic
df1
df2
1.148
3
Sig.
116
.333
Conclusion: The p value is larger than 0.05, which means H0 is accepted. Equal variances are assumed for the ANOVA test. E.2.
Test Results Table E.3. Test results of the pairwise comparisons, p values included 0 – 1 Test statistic Stat. p 0 – 2 Stat. 0 – 3 p CE MD A U 311 0.028 263 0.006
T U 275 0.007 ME U PEOU Stat. 0.5988 0.093 0.6397 0.067 1.7433
1 – 2 p Stat. 1 – 3 p Stat. 2 – 3 p Stat. p 0.000 0.0409
0.999 1.1445 0.000 1.1036
0.000
128
0.000
429
0.465
296 0.002 279
0.002
337 0.099
182
0.000
333
0.062
284 0.003 214
0.000
406.5 0.335 370.5 0.221
310
0.013
352
0.105
312.5 0.009 379
0.111
U 323.5 0.065 361.5 0.392
356.5
0.086
320
0.087
310.5 0.008 382
0.224
PU U 402 0.391 369 0.441
271
0.004
373
0.305
365 0.039 260
0.004
IU U 400.5 0.382 286.5 0.060
267.5
0.003
286.5
0.027
273 0.001 360
0.134
Note: MD = mean difference (Tukey HSD); U = Mann‐Whitney U XXI Appendix F
Files for Implementing the New Notation in CHOOSE XXII