The factors affecting performance of software engineers. Bret van Westing – s3785297 – 25/06/2023 – Research Methods INTRODUCTION The aim of the proposed research is to understand how software engineers can be motivated, or otherwise enabled, to perform better within their discipline using resources originating from software development companies. Extensive research has been done to understand how to motivate employees. However, software engineers are “often stereotyped as being socially awkward or having difficulty communicating” (Morris et al., 2015). This distinction between employees and software engineers leads to 76% of papers including software engineers as their own distinct occupational group (Beecham et al., 2008) within their respective research. Yet, they are underrepresented within the field of managerial motivational practices, as often motivational characteristics are discussed in place of motivational practices. The goal of the proposed research is to answer the following research question: How can small-medium software development companies affect the performance of software engineers? Research has shown that software engineers whom are diagnosed with ADHD or autism, are often diagnosed in their adulthood. A large part of software engineers with ADHD or autism do not disclose this information to HR or management (Morris et al., 2015)., leading to the failure of typical motivational practices without a clear explanation. A comparison of motivational characteristics of typical employees (Lindner, 1998) and the motivational characteristics of software engineers (Beecham et al., 2008) show that there is some similarity, but large inconsistencies regarding prioritization of motivational characteristics. This research will try to help software development companies understand their software engineers and give them practices to increase performance of software engineers through enabling or motivation. Motivational practices tailored to the characteristics of software engineers can help software development companies help increase overall performance (Mohamud et al., 2007). Software development companies would want to increase performance of software engineers to increase overall quality of the (software) product. The increase in performance can be shown as a decrease of Time-To- Market for hotfixes, updates or new products, overall better maintainability of code, better documentation, better team cohesion and better individual or team-based problem solving. Software engineers also reside as an asset to the company, as internal knowledge plays a large role in solving problems effectively and increasing maintainability of the program (Rugaber, 2006). An increase in performance through motivation can increase the chances of each respective software engineers desire to stay with the current employer. As software engineers take part in a competitive market and are often approached by different possible employers, motivational practices tailored to software engineers could give software development companies the benefit of the doubt and simultaneously benefit of increased performance from software engineers with good domain knowledge. LITERATURE REVIEW Within the literature review, key concepts will be defined and explained to give background and a solid basis for the development of hypotheses. Topics mentioned within the literature review are: motivation, motivating software engineers, performance & neurodiversity. MOTIVATION Motivation is referred to as ‘the reasons underlying behaviour’ (Guay et al., 2010) or ‘the attribute that moves us to do or not do something’ (Broussard & Garrison, 2004). It is the reason why people, or employees, take actions with an amount of intensity and commitment. To explain the ‘why’, researchers such as Deci et al. (1999) often make a distinction between intrinsic and extrinsic motivation. Intrinsic motivation refers to the personal interest a person has in a subject or activity. Often combined with enjoyment of these activities, it can energize and stimulate a person to perform better within their respective field. Deci et al. (1999) puts it as ‘intrinsic motivation energizes and sustains activities through the spontaneous satisfactions inherent in effective volitional action.’ Deci et al. emphasises that the activities taken are often voluntary, however intrinsic motivation does not only take effect in volitional action but are often combined with interests of the person and the activities required by their employer. Extrinsic motivation refers to the motivation a person gets from external rewards (Deci et al., 1999), this can be the promise of monetary rewards, welfare gains or the promise of the removal of existing negative consequences (Deci et al., 1999). Taken these forms of motivation into account, Lai (2011) highlights that ‘motivation involves a constellation of believes, perceptions, values, interest, and actions that are all closely related’, meaning that typical motivational factors don’t always apply or have similar impact on the motivation of a person. Approaches to motivation should focus on cognitive behaviour of a person or non-cognitive aspects such as beliefs, attitudes and perceptions. MOTIVATING SOFTWARE ENGINEERS Whitaker (1997) explains the importance of motivation and welfare of software engineers within a company. De art of programming is being appreciated more and more each year, and it is difficult for companies to employ knowledgeable software engineers. The chances of losing a software engineer can be reduced by taking an interest in his welfare and overall motivation. Hall et al. (2007) explains that software engineers are often in favour of an ‘open communication culture’, where communication is always appreciated and not talked on negatively, even when the subject of the communication are personal failures, opinions or other controversial topics. Whitaker (1997) agrees with Hall et al., explaining that communication and transparency are important activities which can motivate employees intrinsically. When an open communication culture cannot be maintained within an organisation, a common consequence is that software engineers have to revise and rework more code. This is due to a lack of proper documentation, less cohesion between the team members and a lower quality of received feedback. Apart from communication, according to Hall et al. verbal appreciation and challenging work can also be very effective motivators for software engineers. To entice the extrinsic motivation of software engineers, Whitaker (1997) states that rewarding the individual is the most prominent factor for increasing motivation. Whitaker writes that handing out rewards will have positive effects on the performance of the software engineer, and the time-to-market of the project. In order to get the maximum effects, Whitaker states the rewards should be handed out not long after certain achievements. The timely delivery of a good product, maintaining an effective way of working or all-round good behaviour are types of achievements that can be rewarded. However, it is vital that the reward match the achievements. Handing out larger rewards than the achievement will leave skewed expectations of rewards, which can backfire on overall motivation. Hall et al. (2007) agrees with Whitaker, stating that rewarding is a very effective way of motivating software engineers and the statements made by Whitaker complement the results of the experiment executed by Hall et al. Whitaker adds that rewards should be team-based, as individuals that are left out will be demotivated, and will perform worse. Even though the rewards should be team based, according to Whitaker (1997) it is vital that the distributing of rewards should be done in private. Whitaker claims from personal experience that publicly announced rewards has a negative effect on the long term. Whitaker says that software engineers will then focus more on the receival of a possible reward, than the creation of a quality product. The achievable milestones which are worthy of a rewards would then be rushed, in turn lowering the overall quality of the product. Beecham et al. (2008) complement the findings of Whitaker (1997) and Hall et al. (2007), also finding that rewards and incentive are amongst the top motivators for software engineers. Beecham et al. (2008) however states that there are more motivators that can have more impact on motivation. They list ‘Identifying with the task’ as an important motivator, which would link to intrinsic motivation. They also mention that a sense of belonging, opportunity for a career and variety of work are also motivating needs that software engineers have. To conclude, motivation can be divided in intrinsic and extrinsic motivation. Where intrinsic motivation focusses on the internal needs, and extrinsic motivation refers to external factors such as rewards or punishments. Multiple studies find that software engineers are more motivated in open communication cultures. Along with transparency and team cohesion, these motivational factors entice the intrinsic motivation of software engineers. However, Whitaker (1997), Hall et al. (2007) and Beecham et al. (2008) argue that providing rewards or job opportunities are more effective in motivating software engineers. This is due to software engineers being more achievement and growth oriented than the average employee, as stated by Beecham et al. (2008) and Morris et al. (2015). Software engineers are often introverted, and thus have different needs for to become motivated. Software developing companies should execute different motivational practices for software engineers to ensure optimal effect on their motivation. PERFORMANCE OF SOFTWARE ENGINEERS In order to understand why certain companies have better performing software engineers, it is important to look at the characteristics that enable high performance teams to exceed expectations and produce extraordinary results. Dutra et al. (2015) state that high performance teams often produce these results based on the stimulation and support from the organization. Organizational commitment and managerial involvement also impact performance, but motivation seems to have a large positive effect. Dutra et al. describe various influences on performance, where they mention that influences have ties to motivation. Influences such as communication, trust, cohesion, learning ability and work satisfaction increase parallel to motivation. These influences act based on the individual characteristics, yet they can be stimulated to grow faster with higher motivation. Dutra et al. (2015) also show negative influences on performance, they briefly describe that the team size and the focus on project turnover can hinder progress. However, they also mention motivation can hinder performance. Dutra et al. warn that motivation can negatively impact performance within virtual teams, stating that the geographic dispersion of the team members makes it a significant challenge for the organization to maintain motivation and develop effective teams. NEURODIVERSITY Morris et al. (2015) surveyed 850 software engineers to understand the challenges neurodiverse employees face, as they claim employees with Autism Spectrum Disorder suffer from discrimination within the workplace. Their survey showed that 7% of respondents are neurodiverse, dealing either with dyslexia, ADHD or a form of autism. Within the survey, 42% of neurodiverse software engineers said they had been self-diagnosed, while 66% has received a formal diagnosis by a medical professional. The average age of diagnosis was 23, an age where junior software engineers can already be employed. For the most part, neurodiverse software engineers hide their diagnosis. About 20% claim to have revealed it to co-workers, 3% to management, 17% claimed to never have told anyone and none of the respondents have revealed it to HR. Neurodiverse software engineers showed to be less comfortable with social interactions, such as faceto-face interactions, asking for help or following verbal directions. They also don’t deal well with changes, or other tasks which are out of routine. These small differences between neurodiverse and neurotypical software engineers can lead to practices for motivation not having the desired effect. About 94% of the respondents that claimed to be neurodiverse, have not received or asked for any accommodations which could help them with their condition. Neurodiverse software engineers proclaim to be better than their neurotypical counterparts in multiple aspects of their work, lacking the proper adjustments to cope with neurodiverse-induced lack of social need could prove limiting to their overall performance. HYPOTHESES The proposed research aims to answer the afore mentioned research question by testing the following hypotheses. The hypotheses are created using the above literature review. Hypothesis 1: Managerial involvement in projects has an inverted U shape effect on individual motivation in software engineers. Dutra et al. (2015) describes managerial involvement and motivation as influences that have a large effect on performance. However, they evaluated the effects of these influences separately, ignoring the possible they can have on each other. Managerial involvement has shown to have a positive relation with job satisfaction and motivation (Orpen, 1997), it can however be argued that the motivation of software engineers have a different relation with managerial involvement. The lack of need for social interaction or involvement and the need for individuality could in turn mean that too much managerial involvement can impact motivation negatively. Hypothesis 2: Supporting organizational commitment reduces in the effect on team performance over time. Again, Dutra et al. (2015) describe organizational commitment and motivation as influences on team performance, however they fail to show the impact of these influences combined. Altindis’ research (2011) shows that normative commitment (conforming to social norms) and affective commitment (attachment, identification and involvement) have a large influence on intrinsic motivation, where normative commitment has the largest influence on extrinsic motivation. It can be argued that due to the lack of need for social interaction and involvement, software engineers do not conform to previous research. Organizational commitment could support software engineers until a certain level, resembling an exponential plateau. Hypotheses 3: Neurodiverse adjusted work environments have a positive effect on individual performance of software engineers. As Morris et al. (2015) discussed, neurodiverse software engineers often have trouble adjusting to social interaction or involvement, preferring textual communication above face-to-face communication. As neurodiverse software engineers struggle to find common ground with neurotypical software engineers, adjustments and methods created to support neurodiverse software engineers within social interaction and overall involvement could prove to increase performance of neurodiverse software engineers. Each of the aforementioned hypotheses have their own respective null hypothesis. These null hypotheses are mentioned below, respective of their order: • Managerial involvement in projects does not have an effect on individual motivation in software engineers. • Supporting organizational commitment does not reduce in the effect on team performance over time. • Neurodiverse adjusted work environments does not have a positive effect on individual performance of software engineers. METHODS AND DATA SUBJECT FOR STUDY Within the research, the subjects that are being studied must conform to the following characteristics: The subjects for study must have the title of software engineer, who is employed in a software development company which classifies itself as a Small-Medium Business (SMB). The mentioned subjects will be surveyed on their performance, motivation and how they are enabled to perform. Enablement can be measured by their work environment or possible policies in place which enable them to perform better (or worse). Organizational commitment and managerial involvement will also be surveyed, to understand the support an individual or team gets from upper management. This can be in terms of money, time, interactions or other resources/support. POPULATION The research will focus on the following population: All software engineers within western first-world countries that are employed within a SMB software development company. The population will be sampled from the LinkedIn register, where users with the title ‘Software engineer’ or synonyms will be targeted for the survey. They will have to be currently employed within a SMB software development company. Eligible users will be randomly selected and requested to participate. A sampling frame error could originate from users that use the title wrongfully, outdated profiles or missing neurodiverse software engineers due to them being to reluctant of social interaction/social media to participate on LinkedIn. MEASUREMENT By assigning independent and dependent variables to the hypotheses, it will be clear what exactly will be measured in order to test the hypotheses. Hypothesis 1: Managerial involvement in projects has an inverted U shape effect on individual motivation in software engineers. • Independent: Managerial involvement. o Measurement: Measure amount of daily interactions by management or other involved leaders [continuous]. o Motivation: In order to understand the amount of involvement there is from management, daily interactions such as meetings, face-2-face interactions, texts/emails, needs to be measured to understand how involved the manager is in the project. • Dependent: Individual motivation. o Measurement: Measure motivation through survey [1, 2, …, 7]. o Motivation: Motivation can best be measured by the individual himself. • Confounding: Complexity of project. o Measurement: Timespan [Days, Months, Years], Number of expected features/functions [0, 10, 20, …], team size [<5, 5, 10, …, >100], benefactors [continuous]. o Motivation: Complexity of the project can create more involvement of management, and have an impact on overall motivation. The project complexity need to be measured in order to find out if it has a causal relation between motivation and managerial involvement. Hypothesis 2: Supporting organizational commitment reduces in the effect on team performance over time. • Independent: Supporting organizational commitment. o Measurement: Amount [1, 2,…, 7] of resources [Money, equipment, knowledge, manpower, other] provided by the organization over time [Before, during, after] o Motivation: Organizational commitment can be measured by the resources allocated for a project. This needs to be measured to understand how supporting an organization is. • Dependent: Team performance over time. o Measurement: Survey individual opinion about team performance [1, 2,…, 7] over time [Before, during, after]. o Motivation: Team performance can be interpreted different by each member due to their priorities. Individual opinion about team performance can measured to understand if the organizational commitment can be continuous regardless of individual opinion about performance. Hypotheses 3: Neurodiverse adjusted work environments have a positive effect on individual performance of software engineers. • Independent: Neurodiverse adjusted work environments. o Measurement: Amount of adjustments for neurodiverse software engineers [ordinal]. o Motivation: The adjustments communicate the amount of effort put into creating neurodiverse adjusted environments, through physical items or policies (etc.). • Dependent: Individual performance. o Measurement: Individual opinion about individual performance [1, 2, 3, 4, 5]. o Motivation: The individual perceives the effects of policies or adjusted work spaces the best, one might find a solution to be more effective than another. VALIDITY & RELIABILITY To ensure all measurements and variables are reliable, each will be tested and retested. All variables are subject to face validity or concept validity, ensuring logical use of the measurement to test what needs to be tested. DATA COLLECTION METHODS Within the research, data will be collected using a survey (see Appendix A). As stated before, users within the LinkedIn register with the title ‘software engineer’ which are employed with a small-medium business will be contacted. This will be done via either an email or personal message. Eligible users will be found using a web scraper, identifying users. After a possible respondents list has been constructed, a list of randomized users will be selected for contact. Once the first survey has been filled in, the respondents will be removed from the list. Once again, a set of randomly selected users will be approached to fill in the survey. The subject of ‘Neurodiversity’ exists entirely of questions which are not required, to ensure that the respondent can still send in their response without having to answer difficult questions. This ensures that hypotheses 1 and 2 can still be answered. In order to motivate respondents to answer, gift cards will be randomly given out to respondents. These gift cards can be selected by the respondent, to ensure the prize will be effective regardless of interest. For example, a software engineer who is into gaming, will be less interested in winning a gift card for perfume. However, being able to choose a gift card for video games improves chances of the software engineers responding to the questionnaire. MOTIVATION In order to understand the effects on such a large scale, a survey is the least time-intensive option. The gathered quantitative data can more easily be analysed, which would then apply to a much larger population. However, it can be possible that respondents have limited experience and can therefore not answer the questions properly. Memory problems can also occur for respondents, implying that the survey should take this into account. The survey ask the respondents to choose a project of interest, before asking them the questions regarding that project. This way, there is a higher chance the respondent can remember events, as it will be his/her own first choice. Opinions/statements of social interactions might also be at risk of interpretation fault, as it can be that neurodiverse software rate social interactions heavier than others, implying they could overestimate the amount of social interactions. For the designed survey, please see Appendix A. REFERENCES Morris, M. R., Begel, A., & Wiedermann, B. (2015). Understanding the Challenges Faced by Neurodiverse Software Engineering Employees. https://doi.org/10.1145/2700648.2809841 Beecham, S., Baddoo, N., Hall, T., Robinson, H. P. C., & Sharp, H. (2008). Motivation in Software Engineering: A systematic literature review. Information & Software Technology, 50(9–10), 860–878. https://doi.org/10.1016/j.infsof.2007.09.004 Lindner, J. (1998). Understanding Employee Motivation. Copyright (C) 2023 Extension Journal, Inc. ISSN 1077-5315. https://archives.joe.org/joe/1998june/rb3.php Rugaber, S. (2000). The use of domain knowledge in program understanding. Annals of Software Engineering, 9(1/4), 143–192. https://doi.org/10.1023/a:1018976708691 Mohamud, S. A., Ibrahim, A. A., & Hussein, J. M. (2017). THE EFFECT OF MOTIVATION ON EMPLOYEE PERFORMANCE: CASE STUDY IN HORMUUD COMPANY IN MOGADISHU SOMALIA. ResearchGate. https://www.researchgate.net/publication/321974170_THE_EFFECT_OF_MOTIVATION_ON _EMPLOYEE_PERFORMANCE_CASE_STUDY_IN_HORMUUD_COMPANY_IN_MOG ADISHU_SOMALIA Guay, F., Chanal, J., Ratelle, C. F., Marsh, H. W., Larose, S., & Boivin, M. (2010). Intrinsic, identified, and controlled types of motivation for school subjects in young elementary school children. British Journal of Educational Psychology, 80(4), 712 Broussard, S. C., & Garrison, M. E. B. (2004). The relationship between classroom motivation and academic achievement in elementary school-aged children. Family and Consumer Sciences Research Journal, 33(2), 106. Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125(6), 627–668 Lai, E. R. (2011). Motivation: A literature review. Person Research’s Report, 6, 40-41. Whitaker, K. (1997). Motivating and keeping software developers. Computer, 126-128. Hall, T., Jagielska, D., & Baddoo, N. (2007). Motivating developer performance to improve project outcomes in a high maturity organization. Software Qual J, 15, 365-381 Dutra, A., Prikladnicki, R., & Franca, C. (2015). What Do We Know about High Performance Teams in Software Engineering? Results from a Systematic Literature Review. https://doi.org/10.1109/seaa.2015.24 Orpen, C. (1997). The Interactive Effects of Communication Quality and Job Involvement on Managerial Job Satisfaction and Work Motivation. Journal of Psychology, 131(5), 519. https://login.ezproxy.leidenuniv.nl/login??url=https://www.proquest.com/scholarlyjournals/interactive-effects-communication-quality-job/docview/1290670795/se-2 Altındiş, S. (2011). Job motivation and organizational commitment among the health professionals: A questionnaire survey. African Journal of Business Management, 5(21), 8601–8609. https://doi.org/10.5897/ajbm11.1086 APPENDIX A: SURVEY Dear respondent, Thank you for your interest in or study. The purpose of the study is to understand the effect of the work environment on your motivation and performance. The results will help small businesses to understand software engineers and enable them to support software engineers better during projects. The survey will take 5-10 minutes, whereafter you will be able to win a gift card of your choice. The survey will segmented in 5 parts, each segment will be displayed on their own page. For all segments, we ask you to keep a software engineering project in mind where you worked within a team. This can be a large, small or medium project, but one that you are able to recall. The survey will ask about your motivation, performance and overall situation. In order to be eligible for the gift card, all segments should be completed. We understand that data is very important. No personal information will be gathered without your approval, and all results will be anonymised and deleted after the survey. The data will be secured according to the GDPR. Mandatory questions are denoted with an *. Project In order to understand the complexity of your chosen project, please answer the following questions. How many people worked within the team?* [ ] amount of team members How long did the project go on? * [ ] O Weeks O Months O Years How many stakeholders/benefactors were involved? * [ ] stakeholders/benefactors How large, in terms of features, was the project? * [ ] features If this question does not apply, please describe. ______________________________________________________________ Motivation Research has shown that motivation can be big driver for performance in projects. However, not all organizations take measures to properly motivate employees to make sure they can perform their best. Please answer the following questions with your chosen project in mind, and how you’ve felt during the project. How do you identify with the following statements: During the project…. … I had sufficient support to resolve any issues I had. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree … I enjoyed working on my assigned work. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree … I motivated myself to get through the project. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree [ ] Neutral [ ] Somewhat agree [ ] Agree … others motivated me to work hard. * [ ] Disagree [ ] Somewhat disagree … I had the resources to have fun in my work. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree Were there any factors that enabled you to be motivated? * ________________________________________________________________________________ Were there any factors that inhibited you to be motivated? * ________________________________________________________________________________ How motivated were you (on average) during the project? * Not motivated O O O O O O O 1 2 3 4 5 6 7 Very motivated Performance Please answer the following questions with your and your team’s performance during the project. How do you identify with the following statements: During the project… … I had performed well with my team. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree [ ] Neutral [ ] Somewhat agree [ ] Agree [ ] Neutral [ ] Somewhat agree [ ] Agree [ ] Neutral [ ] Somewhat agree [ ] Agree … I performed better than my team. * [ ] Disagree [ ] Somewhat disagree … I held my team back. * [ ] Disagree [ ] Somewhat disagree … My team held me back. * [ ] Disagree [ ] Somewhat disagree … my performance enabled others to perform well. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree … the performance of others enabled me to perform well. * [ ] Disagree [ ] Somewhat disagree [ ] Neutral [ ] Somewhat agree [ ] Agree How do you rate the confidence you had in the team’s performance before the project? * O Not confident 1 O O O O O O 2 3 4 5 6 7 Very confident How do you rate your teams performance during the project? * Very Bad O O O O O O O 1 2 3 4 5 6 7 Very good How do you rate your teams overall performance after the project? * Very Bad O O O O O O O 1 2 3 4 5 6 7 Very good Leadership The following questions are about the involvement of management, or any leader figure, within the project. How many weekly average meetings did you or your team have with management or a leadership figure? This includes standups. * [ ] meetings How many daily interactions, such as emails, texts, etc. did you have with management or a leadership figure? * [ ] textual interactions How many daily social interactions, such as informal talks, watercooler conversations, did you have with management or a leadership type figure? * [ ] social interactions What percent of interactions with management or a leadership type figure do you think were about the project? * [ ]% Did you think the interactions about the projects were useful for team performance or motivation? * Not useful O O O O O O O 1 2 3 4 5 6 7 Very useful For next projects, would you prefer more or less interactions? * [ ] A lot less [ ] A bit less [ ] Similar [ ] A bit more [ ] A lot more Why? * __________________________________________________________________________________ Organisation This segment asks questions about the received support from the organization for your project. Please rate if the amount of received resources were sufficient: Amount of funds: * O Not sufficient 1 O O O O 2 3 4 5 Very sufficient O O O O 2 3 4 5 Very sufficient O O O O 2 3 4 5 Very sufficient O O O O 2 3 4 5 Very sufficient Equipment: * O Not sufficient 1 Manpower: * O Not sufficient 1 Knowledge: * O Not sufficient 1 Of the above mentioned resources, which one was the most vital to the success of the project? * O Amount of funds O Equipment O Manpower O Knowledge Neurodiversity Neurodiversity can be common in the field of software engineering. To understand how organizational policies and involvement affect neurodiverse software engineers we would like to understand how you view the effort put into accommodating neurodiversity by your organization. Do you consider yourself to be on the autism spectrum, have ADHD, dyslexia or another type of neurodiversity? O Yes O No Please rate how much the following measures have been implemented in your organization: Physical work changes: O None 1 O O O O 2 3 4 5 A lot O O O O 2 3 4 5 A lot O O O O 2 3 4 5 A lot O O O O 2 3 4 5 A lot Policies: O None 1 Work times adjustments: O None 1 Social interaction (unwritten) rules: O None 1 How effective would you rate the measures taken by your organization? O Not effective 1 O O O O O O 2 3 4 5 6 7 Very effective Which of these measures do you find the most effective to your personal preferences? _________________________________________________________________________ If you could, what measures would you change or add? _________________________________________________________________________ Please select 7 (Very effective), this is to ensure the validity of the responses. O O O O O O O Not effective 1 2 3 4 5 6 7 Very effective This is the end of the survey, thank you for taking the time to fill in the questions. If you’d like, please leave your email address to be informed of the research results. Please click ‘continue’ to close the survey, you will also immediately know if you have won a gift card. [ Continue → ]