See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/339323100
Success Factors of Big Data to Achieve Organisational Performance:
Qualitative Research
Article · February 2020
CITATIONS
READS
7
443
2 authors:
Candice Walls
Brian Barnard
University of Pretoria
Wits Business School
3 PUBLICATIONS 14 CITATIONS
71 PUBLICATIONS 124 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Equity valuation View project
Term structure decomposition View project
All content following this page was uploaded by Brian Barnard on 18 February 2020.
The user has requested enhancement of the downloaded file.
SEE PROFILE
Expert Journal of Business and Management, Volume 8, Issue 1, pp.17-56, 2020
© 2020 The Authors. Published by Sprint Investify. ISSN 2344-6781
http://Business.ExpertJournals.com
Success Factors of Big Data to Achieve
Organisational Performance:
Qualitative Research
Candice WALLS and Brian BARNARD*
Gordon Institute of Business Science, University of Pretoria, South Africa
This part of the study continues the literature review conducted, and empirically
addresses the research questions raised, in the preceding part of the study: Success
factors of big data to achieve organisational performance: Theoretical perspectives.
Overall, the focus is on the success factors of big data from an organizational context:
how and when big data analytics capabilities yield benefits and improve
organisational performance. The research questions that were addressed are: 1.
What is the general understanding of big data and its current situation? 2. What
capabilities are required in order to be successful at translating big data insights into
organisational performance? 3. What are the biggest challenges/ risks to converting
big data insights into organisational performance? The methodology was qualitative
and exploratory, using semi-structured interviews to have in-depth discussions with
big data employees at varying job levels to ascertain the success factors and
challenges of the big data environment. These constructs provide companies with the
ability to improve their BDAC capabilities in order to generate value from big data,
thereby improving organisational performance.
Keywords: Big Data, Success Factors, Organisational Performance, Value
JEL Classification: O30, L26
1. Introduction
As the subsequent part of the paper, this part continues the preceding literature review (presented in
the previous paper of this journal, namely Success Factors of Big Data to Achieve Organisational
Performance: Theoretical Perspectives), addresses the previously listed research questions, and contains the
empirical findings.
1.
2. Research Methodology
This paper outlines the chosen research methodology chosen for this research study. Information
*
Corresponding Author:
Brian Barnard, Gordon Institute of Business Science, University of Pretoria, South Africa
Article History:
Received 1 January 2020 | Accepted 30 January 2020 | Available Online 17 February 2020
Cite Reference:
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research. Expert Journal of
Business and Management, 8(1), pp.17-56.
This paper has previously been included in an open access repository – SSRN.
17
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
attained from the literature review, outlined in the previous paper of this journal, namely Success Factors of
Big Data to Achieve Organisational Performance: Theoretical Perspectives, was used as the basis for the
chosen methodology as well as the interview guide used during the semi-structured interviews. The approach
adopted was exploratory in nature with the research method, research design, sampling and analysis supporting
the qualitative approach.
2.1. Choice of Methodology
Philosophy and Approach
The research philosophy contains important assumptions about how the world in which the project is
being conducted is viewed and this has an important impact on how we interpret what is being investigated
(Saunders, Lewis, and Thornhill, 2009). The research philosophy adopted for this study was interpretivism
because it is important to understand what was occurring in the BDAC context within the organisations and
since interpretivism relates to the study of phenomena in their natural environment, this approach provided the
right mindset for this understanding to occur (Saunders and Lewis, 2012). Theories described in the previous
paper of this journal, namely Success Factors of Big Data to Achieve Organisational Performance: Theoretical
Perspectives were used as a lens through which to study the effects of BDAC on organisational performance
in each of the companies. Limitations of interpretivism are subjectivity and research legitimisation. Since
interpretivism lends itself to the main influence being the researchers particular view of the context,
subjectivity and researcher perception bias could have influenced the outcome of the study (Saunders and
Lewis, 2012). Research legitimisation is compromised in the validity, reliability and generalisability of the
research due to interpretivism often providing an outcome too unique to the context (Kelliher, 2011).
The research approach was a combined approach between deduction and induction. This approach is
justified through Saunders and Lewis (2012) and Bloomberg and Volpe (2012) who both state that a combined
approach in the same paper can be a good idea. Initially a deductive approach was used to identify themes
within the literature that emerged within the BDAC space. These themes were then used to determine context
for interpreting the effectiveness of BDAC on organisational performance. Relevant theory with which to
compare these capabilities was also identified from the literature review. This context was used to provide an
outline to the semi-structured interviews as well as a framework for interpretation of the collected data from
these interviews. The inductive approach began when new constructs occurred based on observations during
the interviews. These new constructs were used to improve or adapt the initial theory framework in order to
develop a model to be tested in future research. This combined approach is consistent with the literature being
in its early stages with little commonality and consensus amongst researchers. Further development and
convergence of theory was required through the combined deductive and inductive approach. The advantage
of using the deductive approach first is to provide some structure to the results to ensure relevance, and
alignment to current literature trends. A disadvantage is that outcomes outside of the framework may have
been missed if too much emphasis was placed on the known with insufficient room for unknowns to have
surfaced during the interviews. An awareness not to lead the interviewees into the defined framework was
made.
2.1.1. Type of Study and Strategy
Information was collected at a specific point in time over a three week interview period, leading to a
cross-sectional study having occurred (Saunders, Lewis, and Thornhill, 2009). This was appropriate as the
research was aimed at understanding what organisations are doing now to be successful with BDAC initiatives
and was not to measure how the organisations actually perform over time. Interpretivism interprets social roles
to attain knowledge rather than objectively determining the findings. This philosophy lent itself well to a more
flexible qualitative research method where there was greater personal interaction to attain more meaningful
and in-depth insights (Carson, Gilmore, Perry, and Gronhaug, 2001). Qualitative research does not require the
use of numerical techniques to answer proposed research objectives and therefore left room for interpretations
and elaborations (Zikmund, Babin, Carr, and Griffin, 2013). The topic was allowed to naturally develop
through the process without a predefined path (Patton, 2002).
Qualitative research allowed for greater insights and richer content to be generated since more
interaction with the interviewees was accomplished (Carson et al., 2001). This allowed probing and follow up
questions to be asked, facilitating a conversation rather than a one-way quantitative interaction. The emphasis
of qualitative research was on exploration and discovery (Bloomberg and Volpe, 2012). The aim of the
research was to determine what the organisations are doing themselves with regards to BDAC and this required
a more open-ended qualitative approach as quantitative approaches have a fixed set of variables and further
18
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
exploration outside those predetermined variables cannot take place (Guba, Lincoln, and others, 1994). The
literature on the effects of BDAC on organisational performance was scarce and in its early stages of
development. This situation was particularly suited to qualitative methods as the increased depth in knowledge
attained can add to and help to shape the growing and inconsistent literature. There has been a call for more
qualitative research to take place within the BDAC literature in order to explore what organisations do
practically with regards to strategic management and implementation of big data initiatives in order to achieve
improved organisational performance (Vidgen et al., 2017; Y. Wang and Byrd, 2017) and this methodology
was therefore able to answer this call and began to fill this gap in the BDAC literature.
An exploratory study type took place as it discovered general information about the BDAC topic not
yet known in the literature and gained necessary insights to provide answers to the initial research propositions
(Saunders and Lewis, 2012). Saunders and Lewis (2012) and Zikmund (2000) explain that exploratory research
is required when there is a relatively new and unexplained area in the literature, which was the case in this
study. Interviewing is a well-known qualitative method used within exploratory studies (Saunders, Lewis, and
Thornhill, 2009). A semi-structured interview strategy was chosen because this enabled an exploration of
BDAC more deeply and was suited to the scenario of wanting a contextual understanding of the current reality
of organisations (Vidgen, 2014). This allowed for deeper insights into the emerging BDAC research and it was
therefore widely applicable in research conducted with an interpretivist philosophy (Y. Wang, Kung, Wang,
et al., 2018). Literature precedent for qualitative, exploratory, semi-structured interviews is provided by Liu et
al. (2017); Noblet et al. (2011) and Ram, Zhang, and Koronios (2016) within the BDAC literature. Respondents
were a combination of big data consultants and specialists involved in the BDAC projects or strategic
processes. Interviewees were purposefully chosen based on their expertise, roles or familiarity with BDAC
process and outcomes. Liu et al. (2017) explain that purposive sampling is encouraged in this situation.
2.2.
Proposed Research Methodology and Design
2.2.1. Target population
A population is defined by Saunders and Lewis (2012) as the complete set of cases/group members
from which the sample can be taken. Zikmund (2003) defines a population as “a complete group of entities
sharing some common set of characteristics”. The target population from which the interviewees were chosen
was defined as big data consultants and specialists working with big data or on big data projects within South
African organisations. But, Zikmund (2003) goes further with the definition to define the target population as
“the complete group of specific population elements relevant to the research problem”. The target population
was therefore big data consultants and specialists or directors and managers working with big data or on big
data projects within South African organisations that have access to sufficient volume and variety of data to
require BDAC rather than traditional analytical methods. Assessment of the large volume of data access was
done by looking at organisational size and customer base.
2.2.2. Unit of analysis
The unit of analysis was the big data professionals from owners and managing directors to data
scientists and architects. Organisational capabilities and their effect on organisational firm performance was
being investigated through the semi-structured interviews and the individual opinions of these professionals
provided the necessary insights required to obtain the objective as outlined in the previous paper of this journal,
namely Success Factors of Big Data to Achieve Organisational Performance: Theoretical Perspectives.
Sampling method and size
The sample was selected using non-probability purposive (judgement) sampling in order to have
purposefully selected the professionals at various organisations that were able to provide the necessary insight
while being cognisant of remaining representative of the population (Saunders and Lewis 2012; Patton, 2002).
In qualitative research, purposive sampling is often utilised. People with the right access and understanding of
the big data initiatives and outcomes were required to gain the necessary insights from the interviews. Criteria
for the big data professionals were; level of experience must be a minimum of three years working in the big
data field, and the magnitude and complexity of the projects must constitute big data and be a significant
project in the organisation. The complexity was such that it required big data analytics that traditional analytics
tools could not achieve.
According to Patton (2002), the size of the sample should be applicable to the context of the study and
is therefore determined on a per case basis. Based on previous literature precedent, the sample of individuals
was ideally determined based on theoretical saturation with an aim of at least fifteen interviews. Sixteen
interviews were conducted over a period of three weeks consisting of Heads of Departments, Big Data Project
19
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Management Office Executives, Big Data Managers, Data Scientists, Big Data Specialists, Big Data
Strategists, Big Data Analytics Architects, Big Data Consultancy Owners and Team Leads. Each participant
had a minimum of three years’ experience with data analysis and only three of the interviewees were female.
It must be noted that industry was not a criterion for selection and sampling was not done according
to any particular industry. Participants were sourced through big data industry leaders, consultancies and social
media, such as LinkedIn, which contained groups for big data specialists and professionals with many members
from South Africa and Gauteng. Interviewees were from the following industries; Insurance, Banking,
Telecommunications, IT and Consulting (see Table 1).
Table 1. Industry and Position of Interviewee Sample
Position
Number of Respondents
Executive Head
1
Data Scientist
1
Banking
Executive Head
2
Data Scientist
3
Telecommunications Big Data Operations Manager
1
IT
Big Data Analytics Architect
1
Big Data Strategist
1
Technical Sales Leader
1
Consulting
Owner
2
Managing Director
1
Advanced Analytics Team Lead
1
Software Engineer
1
Total
16
Industry
Insurance
2.2.3. Measurement Instrument
Saunders and Lewis (2012) suggest that one of the best methods for conducting exploratory research
is with the use of in-depth interviews after a thorough academic literature review. The chosen measurement
instrument was therefore semi-structured interviews where there was a list of pre-determined themes but the
order of the themes and questions covered varied. During the interview additional questions were asked and
some were also left out if it made sense to do so (Saunders and Lewis, 2012). Flick (2014) points out the
advantage to semi-structured interviews is that the similar question structure helps make analysis of the
collected data easier. Face-to-face and telephonic interviews were conducted to cover each of the research
questions through a set of pre-prepared questions together with probing. This allowed for a balance between
consistency and flexibility.
McCracken, (1988) introduced the systematic and practical concept of the long interview approach,
which was used during the interviews. A combination of exploratory questions, planned prompts and
unplanned prompts were employed throughout the sixteen interviews. Interviews were between 20 and 70
minutes long but on average the interviews lasted 50 minutes. Interviews took place at the venue most suitable
to the participant where nine interviews were telephonic and seven interviews were conducted face-to-face.
Face-to-face interviews were either conducted at the participants place of work or within a booked room at the
Gordon Institute of Business Science. Interviewees were invited to participate via emails or messages within
the linked-in social media platform. The messages contained pertinent information on what the research was
about, the time required from them as well as the consent form detailing the voluntary nature of the
participation and the lack of identifiers when reporting on the information. Once the individual had agreed to
participate, a time and date was discussed and meeting request sent to each individual with the interview guide
attached for their perusal prior to the interview.
An interview guide was pre-prepared to ensure the topics of culture, value, core competency, technology and
challenges were covered with each participant. These were informed by the literature review and a prior
questionnaire by Vidgen (2014) (see Appendix Error! Reference source not found. for draft discussion guide
with the informed consent front page). The interview questions as presented in the interview guide were used
to provide insight into the three research questions outlined in the previous paper of this journal, namely
Success Factors of Big Data to Achieve Organisational Performance: Theoretical Perspectives. The interview
questions can be mapped to the three research questions (see
Table 2 below) to ensure consistency from the objective, through the literature and research questions
to the analysis and discussion of the findings from the interviews.
20
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Table 2. Mapping of Interview Questions to Research Questions
Research Question (from previous
Interview Questions
paper)
Research Question 1
1. How does your organisation see and understand big data (BD)?
What is the general understanding of big
What is BD to your organisation?
data and its current situation?
2. Who consumes BD – how is BD consumed in the organisation - and
how do BD projects initiate, kick-off, or commence?
To what extent has BD caused organisational redesign in your
organisation, or to what extent is it necessary or becoming
necessary?
4. How much of BD done by your organisation is exploratory in
nature? Is there a bit of freedom to play around?
5. What are the main uses of BD within the organisation?
6. What would you define as advanced BD in the context of your
organisation?
Culture
7. How well do employees within the organisation understand BD?
8. How does the organisation make business and strategic decisions,
and to what extent are decisions based on data?
9. How much feedback on BD and BD practices does the organisation
collect?
Value
10. How is BD adding value within the organisation?
11. How and to what extent is BD linked to business problems or
business targets in the organisation? Are industry or business
experts – people that truly know the business – adequately brought
together with BD and BD scientists?
12. How innovative and agile are your organisation and its culture, and
how does this affect BD and BD success?
Core competency
13. What skills do you believe your organisation needs to develop in
order to have BD as a competitive advantage and core competency?
14. How strong is your organisation on BD implementation – following
through on BD projects?
15. How well are you able to articulate and visualise BD data and
output?
16. How extensive is your data gathering? Do you capture unique
sources of data?
Technology
17. How developed and sophisticated is your BD technology?
18. How open and user-friendly is your BD platform, and to what extent
is it disseminated through the organisation?
19. Do you foresee new BD tools in the future to improve BD?
20. What are the typical factors hindering BD and capturing value
through BD?
21. Do you think BD is under-utilised and under-performing, or not?
Can BD still do or contribute more?
22. Did BD deliver with the projects it was incorporated in?
23. What do people typically complain about when it comes to BD?
24. Are you able to source the skills and competencies you need to
make a success out of BD?
25. Do you find the cost of BD to be prohibitive?
3.
Research Question 2
What capabilities are required in order to
be successful at translating big data
insights into organisational performance?
Research Question 3
What are the biggest challenges/risks to
converting big data insights into
organisational performance?
2.2.4. Ethical Considerations
Ethical Clearance was obtained through the Gordon Institute of Business Science as per the guidelines
before participants were contacted. Before the interview began, each individual was again taken through the
consent paragraph sent prior to the interview and re-informed of the voluntary nature, reporting without
identifiers and recording of the conversation to ensure the data was collected in an ethical manner (Saunders
and Lewis, 2012). The front page of the interview guide was also a declaration of confidentiality.
21
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
All respondents could converse in English and as such, no translator was required. Also, no
information about age or race was recorded as this was not pertinent to the analysis and therefore not necessary
to ascertain. Company sensitive information was provided during the interviews and therefore no identifier as
to which insights come from which company are given. Interview transcripts have been anonymised to ensure
confidentiality of the interviewee and their company. In the report respondents are referred to as interviewee
1 to 16 (see Table 3 below) with only contextual information provided when it was important to add context
for the understanding of the interpretation or analysis.
2.2.5. Data collection
Prior to the interview, adequate information on the participant and the company was gathered through
publicly available online information to facilitate a smooth interview and prevent time wasting during the
interview. Data was collected through semi-structured, in-depth interviews, either face-to-face or
telephonically (Saunders and Lewis, 2012; Zikmund et al., 2013). The design of the interview guide was
purposefully structured to ensure an open dialogue occurred around key aspects as identified through the
literature review process as well as left room for other specific topics to be spoken about by the participants.
Adequate preparation occurred to minimise the effects interviewer experience may have on qualitative data
collection (Agee, 2009).
The interviews began with a background section to establish rapport and attain some general
information about each participant and their job description. Different types of prompts; floating, planned,
category question and contrast, as described by McCracken (1988), were used when the interview required.
During some of the interviews a common understanding of what Big Data actually meant was established with
the participant and further clarification on any of the questions or terms occurred throughout the interviews as
necessary. Open-ended, non-leading questions within five different sections; culture, value, core competency,
technology and challenges occurred in each interview and each participant was asked to answer based on their
current and past experiences with their own perceptions and opinions. The confidential nature of the reporting
allowed the participants to answer openly and mention company information freely, which aided in the
exploratory methodology design. Each participant was given the chance to add anything they felt was
important and had not been covered or to ask further questions at the end of each interview (Saunders and
Lewis, 2012). During each interview, notes were taken and all interviews were recorded using the Microsoft
voice recorder. Seven face-to-face interviews were conducted in private rooms to avoid disturbances (Saunders
and Lewis, 2012) and nine telephonic interviews took place since face-to-face was not possible.
2.2.6. Data analysis
The interviews were transcribed fully from the audio recording and the recordings backed up on an
external hard drive. The transcribed interviews were loaded into Atlas.ti, a tool for coding of information.
Excel was also utilised in the coding process as well as to group information for thematic analysis. The analysis
of the first interviews took place on an ongoing basis throughout the data collection for a more focused
approach but most of the analysis took place post the interview process by finding common themes throughout
the interviews. A combination of content and thematic analysis was utilised. Content analysis assigned codes
to the interview guide constructs and groupings, derived from the literature (Patton, 2002). Searches for
common themes within the data took place for the thematic analysis and these emerging themes were coded
to look for insights and new information. The thematic analysis followed the procedure outlined by Braun and
Clarke (2006) who describe thematic analysis as the process whereby patterns or themes are identified in
qualitative data through six main phases; becoming familiar with the data, initial code generation, theme
searching, theme reviewing, theme defining and naming, and reporting.
The recordings, transcripts and written notes taken during the interview were consulted many times
during the analysis in order to determine the codes and these codes were grouped together to form themes
(Zikmund et al., 2013). These themes represented constructs that occurred commonly throughout the
interviews and added value towards answering the research questions (Saunders and Lewis, 2012). Analysis
took place per question in each of the interviews and frequency analysis determined the number of times each
construct was mentioned throughout the sixteen interviews. A list of success factor constructs that can be used
in a conceptual model was developed based on the thematic findings.
2.2.7. Reliability and Validity
Saunders and Lewis (2012) define reliability as the extent that methods for data collection and analysis
produce consistent findings. This is extended for qualitative research to refer to dependability, confirmability
and consistency. Validity is defined as “the extent to which data collection and methods accurately measure
22
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
what they are intended to and whether or not the findings are really about what they appear” (Saunders and
Lewis, 2012). Qualitative research has a subjective nature which could lead to interviewer bias, interpreter bias
and respondent bias occurring (Zikmund et al., 2013). Since qualitative research in the interpretivism
philosophy relies heavily on your interpretation of what you are observing or hearing, researcher perception
bias or preconceptions decrease the reliability of the research findings. Awareness of the potential biases
throughout the process helped to minimise their effect as a concerted effort was made to mitigate the biases.
Internal validity was improved by conducting pilot studies and ensuring the interview guide was appropriate.
After three interviews it was determined that the interview guide was appropriate for the research and these
three interviews were then included for analysis and interpretation. Reliability was improved by following the
semi-structured interview guide within each interview and ensuring the interviewees understood the questions.
Reliability and validity are also improved by no previous relationship being present between the interviewer
and any of the interviewees. This ensured no perception bias could occur or inferences based on any previous
encounters or discussions on the BDAC topic.
•
•
•
•
•
•
•
2.2.8. Research Limitations
Based on the scope of organisations covered during the semi-structured interviews, the results may not be
generalisable to other industries or geographies
The scope of constructs defining the semi-structured questionnaire only covered the literature reviewed
and this may have missed the effects of other mediators or moderators of the relationship between BDAC
and organisational performance. Integrating more variables may be an avenue for future research
The use of the framework for the semi-structured interviews may have constrained and biased the data
collection and interpretation through interpreter bias
Interviewer bias may have occurred as interpretation takes place through the lens of the interviewer.
Researcher perception bias could have been reduced by following the advice of Bloomberg and Volpe
(2012) and ensuring some of the participants evaluated the findings
Respondent bias could have occurred due to their personal opinions and interpretations, for example, their
understanding of what big data actually means as well as their main role in the company not having a
holistic view of the company’s BDAC
Interviewer inexperience could have affected the information collected during the interview process
The research did not actually measure the improved organisational performance and therefore cannot prove
any causal relationship between BDAC constructs and improved organisational performance
3.
Results
This section presents the results attained from the one-on-one semi-structured interviews. The results
are shown as per the research questions in section 3 to ensure consistency throughout the report.
3.1.
Overview of Sample
A total of sixteen semi-structured, one-on-one interviews were conducted over a three-week period
from 14 September 2018 to 01 October 2018. A total of 13 and a half hours of recordings occurred, which
translated into 178 number of pages transcribed. The average interview time was 50 minutes long. The
interviews were thirteen males and three females, all from Gauteng, that spanned thirteen different companies
in five different industries; Telecommunications, Banking, Insurance, IT and Consulting (see Table 3 below).
Gauteng is the economic hub of South Africa and therefore many more companies are based in Gauteng rather
than other provinces. Interviewees were from a range of job levels with varying job focuses and the interviews
covered from owners and managing directors to data scientists and software engineers in order to get a broad
view on BDAC from multiple perspectives. These industries are generally quite large companies and although
there were some smaller consulting companies, large business in Gauteng in South Africa dominated the
interviews and will be the most relevant area to benefit from the findings.
Interviewee
Interviewee 1
Interviewee 2
Interviewee 3
Table 3. Summary of Participants and Interviews
Industry
Position
Date of
Interview (2018)
Telecommunications Big Data Operations
14th September
Manager
Consulting
Owner
18th September
IT
Big Data Analytics
19th September
Architect
23
Type
Telephonic
Face-2-Face
Telephonic
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Interviewee 4
Banking
Data Scientist
21st September
Telephonic
Interviewee 5
Consulting
21st September
Face-2-Face
Interviewee 6
Interviewee 7
Interviewee 8
Interviewee 9
Interviewee 10
Interviewee 11
Interviewee 12
Interviewee 13
Banking
Banking
Consulting
Insurance
Insurance
Banking
Banking
IT
21st September
25th September
25th September
26th September
26th September
26th September
26th September
28th September
Face-2-Face
Face-2-Face
Telephonic
Face-2-Face
Face-2-Face
Face-2-Face
Telephonic
Telephonic
Interviewee 14
Interviewee 15
Interviewee 16
Consulting
Consulting
IT
Advanced Analytics
Team Lead
Data Scientist
Executive Head
Owner
Executive Head
Data Scientist
Data Scientist
Executive Head
Technical Sales
Leader
Managing Director
Software Engineer
Big Data Strategist
01st October
01st October
01st October
Telephonic
Telephonic
Telephonic
Through purposive sampling and using mostly Linked-in, the interviewees were selected based on the
criteria set ou. Participants were big data consultants and specialists or directors and managers working with
big data or on big data projects within South African organisations that have access to sufficient volume and
variety of data to require BDAC rather than traditional analytical methods. The interviewees were also
previously unknown to the interviewer and this helped avoid any bias that may have occurred due to familiarity
or prior conversations on BDAC. Interpretation of the interview findings are then also more objective as no
prior perception based on familiarity of the interviewer can be used to skew the analysis or provide context
unsaid in the interview. All respondents were alone when interviewed to ensure candid and honest responses
and all interviewees had time before the interview to review the interview guide and prepare for a more
informative discussion.
3.2.
Presentation of Results
All results are presented per research question as posed in previous section. Under each research
question are the presentation of constructs that emerged from the codes generated throughout the data analysis.
3.2.1.
Test for Saturation
Number of Codes
120
109
95
100
80
69
58
60
56
44
41
40
23
18
20
19
9
7
3
10
1
0
0
1
2
3
4
5
6
7
8 9 10 11 12 13 14 15 16
Interview
Figure 1. Graph of the code creation over the course of the data analysis
To test for data saturation, new codes were documented and recorded in the order of the interviews
conducted. The number of new codes being generated decreased, in general, as the interviews continued. It
was seen that by the 16th interview no new codes were generated and from the 13th interview no new codes
added any significant constructs and themes to those already formed from the remainder of the interviews.
This means that theoretical saturation had been reached and the interviews conducted cover the range of
24
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
knowledge required for thorough analysis and interpretation.
3.3.
Results for Research Question 1
Research Question 1: What is the general understanding of big data and its current situation?
The aim of research question 1 was to identify the perceived understanding and current view of big
data. This included the understanding of what big data means, what it adds to the organisation and how
companies go about utilising big data. It was also established what organisations view as the main uses and
application of big data. It was important to understand this context in order to then know what success factors
(research question 2) are required to achieve utilising big data successfully in this way and what challenges
(research question 3) are inhibiting organisations from utilising big data successfully.
3.3.1. Big Data Understanding and Applicable View
Understanding
Participants views varied on what big data means, with some participants taking a more academic
definition while others a more practical definition. Participants in leadership roles tended to look at big data in
terms of what it meant for the business while more technical interviewees saw big data more in terms of the
data and technology itself.
Many participants, when asked about their understanding of big data would go straight into tasks and
uses of big data rather than talk through a holistic understanding of the concept, showing that many people
only understand big data from the perspective of where they have seen it applied. One participant answered,
“So I think big data is at the heart of a lot of our solutions, a lot of the products that we sell to our clients.”
The most common view of big data across the interviews was the sense that big data provided a greater
volume of information on a subject, such as an individual or a business. One participant stated that, “Big data
is quite broad, if I start from as many sets of information that you can put together that are tied to one individual
or one thing is what I would consider big data” and another stated “the volume of data that is coming through
in terms of best motion of data and how do we manage that data”. This point was again emphasised by another
interviewee stating that; “we collect quiet a sizeable amount of data… so what big data means to us is being
able to prep all that information and show sides that we were not previously able to determine or see”.
Participants from a business or consulting viewpoint also then added that big data is equally about more
accurate information and the ability to build insight from data knowledge for better decision-making. One
participant stated that, “typically we would say it refers to a larger volume of data… but I think the most
important thing is what do organisations do with their data”, and interviewee 2 gave an example to represent
this fact; “Eventually you are taking away all sense of ‘what if’ you know exactly who it is. You are not saying
what if Stephen would like to buy a bicycle; big data is now saying to me Stephen is in the market to buy a
bicycle it is just about getting Stephen to the right shop to buy the bicycle he wants”.
Some participants took a more academic view of the definition of big data, showing their knowledge
that big data can be explained in terms of the 5 V’s (Braganza et al., 2017; Sheng et al., 2017; Erevelles et al.,
2016; Gupta and George, 2016; Sheng et al., 2017; Wamba et al., 2015). One participant could only recall two
of these V’s, volume and variety, but this interviewee introduced the fact that data then comes from a variety
of sources and is handled through big data by stating; “data is coming from different sources…when it comes
to big data the advantage is big data is able to accommodate this, it has a variety which means any type of data,
you can be able to handle it”. Another interviewee stated that, “the traditional definition of big data for our
organisation holds it’s the reams of data, the variety, velocity, variability and so forth of data”.
Importance
No matter whether an organisation was progressing well with utilising big data or not, there was an
overwhelming view of the importance of incorporating big data successfully in order to be competitive (see
Table 4 below). There was an emphasis that this is for all types of organisations and especially customer
focused businesses.
Table 4. Quotes substantiating the majority view on big data and competitiveness
Quotes from the Interviews
• “it is very central to our survival going forward”
• “in my own honest opinion, every big organisation that is customer or consumer based without adapting
to big data you will die, it’s as simple as that”
• “it drives the economy, people don’t realise it yet, but the data crunch is definitely on its way.”
• “I think it really does demand it, people are requiring it.”
25
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
•
•
•
•
•
•
•
•
•
“Yes, they do require big data to remain competitive”
“it is fundamentally part of almost every problem that you deal with”
“if we weren’t able to work with big data, we probably would have closed shop a long time ago”
“The biggest asset that the business has is the data”
“Yes. Absolutely it is a competitive advantage. I think for us right now… that it is like our one advantage”
“100%. I have no doubt, if we don’t do it people are going to take away from our business, it’s as simple
as that.”
“there is no doubt in my mind that there is a greater scheme, and the bigger the better. The more insights
we have, the more data we have, the better for us.”
“You know this is where the future of the way organisations are going to be run, so without utilising data
to its fullest, I don’t think organisations are going to be innovative and I think they probably will die type
of thing”
“I think if organisations aren’t thinking about big data to compete in the future and they are not there or
close to there I think they are probably in trouble”
There is also a drive for companies to make BDAC a core competency and this is believed to be the
way forward to ensure competitive advantage. “This is what the company is planning towards, it is going to
be the core competency of the company without a doubt”, is what interviewee 1 said in their interview. When
asking another participant if they believe big data should be a core competency they answered, “Definitely
you’ve got to your know customers, you’ve got to know your employees, you’ve got to know your directors,
you’ve got to know your business, you’ve got to know what the market thinks of your business. You’ve got
to know behavioural patterns of your customers and your employees. You’ve got to know who is dealing with
your entity”.
A participant from an organisation that is considered one of the leaders in BDAC in South Africa
perceived competition and the importance of big data competency at a greater level than the other interviewees.
They expanded on the fact that immediate industry competitors are not their main competition anymore. This
is because of the consolidation and convergence of industries that is occurring due to big data analytics
becoming a core competency. Companies with a BDAC core competency that collect and utilise big data well
are the companies where competition is going to emanate from in the near future.A very impactful quote from
interviewee 7 was; “We need to become Google before Google becomes us”. They continued by saying; “Our
competition is data companies. We are looking at who is going to disrupt us”.
3.3.2. Data Capturing
In general, most organisations use their business transactions and internal systems to capture data. The
predominant sentiment was that companies do not utilise all the data that they generate. One participant stated
that; “we generate up to five terabytes of data every day and we are not through with that data yet, some of
that data we use but most of it we are not using”.
Mostly organisations are capturing and using structured data for insights but there does exist a lot of
untapped potential in unstructured data. Knowing how to utilise the unstructured data is more limited and more
big data mature organisations are starting to drive towards utilising more unstructured data. “There is a certain
amount of unstructured data that we do capture but I think mostly structured” was a comment from one
participant while another stated simply that, “the bulk of our system data is structured but there is a massive
drive to gather unstructured data”. Another participant said; “we have a lot of customers, so we have a lot
unstructured as well. I would say we naturally tend to focus on the structured, but there is unstructured stuff”.
Interviewee 3 clearly stated the gap in unstructured data availability and utilisation; “what they are currently
using, the unstructured would be about 20%. What they have got available for them to use unstructured would
be 80%”.
The move to utilising unstructured data is new, which is supported by this participant who said; “So
we are moving to unstructured, but that is still very much R&D level, it is not operational level” and
interviewee 13 echoed the sentiment by saying; “A lot of unstructured data is in almost all organisations, how
many are deriving insights from that unstructured information or doing something with it? Very few”. One of
the big data consultants summed up the situation nicely by saying; “I think the percentage of unstructured data
in the data lake is still very low, it is probably less than ten percent of the data that we are bringing in, mainly
because it is difficult, it is difficult to tag the data, it’s difficult to get the data out, so people are still analogue,
the structured data is still huge, unstructured data in the outside world vastly supersedes the structured data but
even that structured data is so much that we spend more than enough time just analysing structured data without
getting unstructured data in, so we should be doing it but it’s not as pervasive as we would have thought”.
26
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
3.3.3. Politics
There is mixed feeling around politics within organisations due to big data or switching to utilising
big data. Organisations that are new to big data implementation feel that more politics may arise as big data
becomes more prevalent but more big data mature organisations report the opposite in that politics has either
been resolved or the strong leadership drive leaves no room for politics to interfere. Interviewee 1 stated that;
“There is not much politics around it, all we are focusing on now is to make it function as best as we can so
that we can deliver to the business, so personally politics not much but as it grows and people realise its value
then definitely there will be politics around it”. Interviewee 7 from a more mature big data company said;
“whenever there is something that is big like this, it is getting a lot of investment, a lot of focus, and there is
opportunity to make a name for yourself, there is always politics involved. There are always various
stakeholders in different areas that want to have control of it, they want to take over, and they want to benefit
from the success of it. So yes, there has been quite a lot of politics around it. The people that are in charge of
it now seem to be quite adroit at side stepping the politics. It also helps though that the group CEO is on top
of this thing”. Interviewee 9 from a top big data organisation revealed politics to exist but not be an inhibitor
by saying; “there are always politics with certain things, but the business case always wins and it wins quite
easily, and it is also again because we are all striving for the common thing”. This participant introduced a
common goal and having a good business case process which is respected as a reason for politics not interfering
with big data success.
Some participants revealed that politics around systems exist. Interviewee 11 stating that the politics
is around legacy systems; “as much as they want the result, they are going to stay with whatever software we
have done, so you know there is always politics, people still want their own tools.” and the other around which
new system to adopt and relationships with current technology providers; “the only real politics that we have
are we have got some relationships with Microsoft and then with Oracle and getting set up and then I am
pushing for AWS, so there is a lot of ‘why are you not using this’ and ‘why are you not using that’”.
Interviewee 14 stated; “I think you know the information in itself is a political game, um you know
what I have, what I control, what I know is very difficult to let go if you kind of have got the old school of
thinking around hoarding of information and so forth” and this brought to light the politics around information
sharing and how this can affect power positions and reduce key man dependencies, This can become political
if people want to protect their own interests and positions of power above the benefits to the organisation as a
whole.
3.3.4. Process
Most organisations initiate big data projects by an identified business problem. Participants stated that’
“you start with a problem in mind”, “anyone who has got a problem they would like to solve with data, puts
their data set out there” and “there is a business drive and if we need to understand the problem better then go
into the data to understand it”. Some more mature organisations also look at business needs, objectives or
targets of business units to initiate big data projects or big data project leads.
Interviewee 1 described their process as starting with the identification of a business problem, which
is then assigned a product owner. The data scientists then engage with the big data team to find a solution to
the business problem. Data for the solutioning then needs to be secured and a basic solution or minimal viable
product (MVP) is prototyped to evaluate the solution before a full solution is developed.
Two other participants diverged after the business problem identification stating that a business case
is then provided and vetted by a board as well as a data and analytics exco. Business cases are scrutinised from
many different aspects, such as implementation, privacy and operations, and prioritised. Virtual teams
consisting of volunteers can then be created to solve the problems presented, which empowers people to build
and develop their own models, reducing the demand on the big data team and data scientists. Interviewee 7
stated; “anyone can come with a problem that they believe data will solve. They need to business case it. And
then volunteers from multiple business units would get together and we will create a sort of a virtual team that
will solve that problem”. Interviewee 8 stated that their process is similar to the above process and they view
this as a structured project management process for IT projects; “It is structured in terms of the project portfolio
where projects come up; they get prioritised, they get scaled, they get approved, a business case is put forward,
the project is sponsored, the owner then perhaps signs their commitment prior to getting the resources to work
on a project. So, it comes via a very structured IT project management process.”
Another initiator of a big data project is innovation. The freedom to use big data to innovate rather
than solve presented problems was rare amoungst the interviewees organisations. Interviewee 9 stated how
innovation within business units themselves will drive big data projects rather than just project management
27
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
processes. These projects then have a greater amount of freedom at the idea generation phase but caution is
taken to ensure an idea has merit and substance. More rigour is applied throughout the process with more
checks and balances at the implementation phase. Another big data innovation initiative was revealed by
interviewee 10 as a conference; “which is all the actuaries and data scientists and anyone in similar fields,
where you have to come up with some idea to save the company or make the company money, and then do
like a feasibility study and show some results from that.”
An interviewee from a more immature big data organisation revealed that they do not have a process
to follow when it comes to big data projects stating; “when it comes to projects, we let them try and solve
things their own way and hope that senior people will guide them in their process”.
3.3.5. Value
One of the consulting interviewees described the value of the data itself and stated; “data is like the
black market”, which demonstrates this value. He establishes that data is truly sought after by saying; “It is
very touch and go at the moment as to you don’t want to know that a big corporate is buying millions of records
at a time to update their database to increase sales”. Building data sets and doing data analytics has a positive
impact on the economy in two ways; firstly through selling data; “We do put our data together, but we do
definitely sell data, if we don’t sell data we don’t have a job and we don’t make money and the economy gets
even worse because it puts people out of jobs” and secondly by providing businesses with valuable information
that increases their efficiency, thereby positively contributing to the economy.
Value is also seen in terms of the business value that data can bring to a company. This was the more
common view of value in data and BDAC. Some participants perceive that big data has already delivered
plenty of value within their organisation by stating; “the amount we have spent for what we have made is
nothing; that’s why we have been given almost unlimited budget – not unlimited, but ridiculous budget”,
“everyone is on board with it; they see the value”, “data is a big part of what we do… big data is at the heart
of a lot of our solutions” and “realising slowly but surely the value of the data”. There is also a common
sentiment amoungst the respondents that big data can do more and provide more value. This is seen through
the responses; “we are doing really well but we are not doing well enough”, “I think there is huge scope
probably everywhere for us to do more, right? It’s definitely under-utilised, and it is an opportunity for a lot of
organisations to basically be more successful”, “the company is at a point of knowing that they still can
improve”, “in fact there is more coming, there is more coming” and “A lot more. I would go as far as saying…
I am so excited about stuff that we can do”.
Some participants revealed that companies have struggled to derive value from big data as evidenced
by statements such as; “I think it was a case of lack of vision of where big data could actually help”, “currently
I don’t think they are using it enough to be able to say it is of any value”, “it’s like everybody want to do
something with it, but nobody kind of knows what they want to do with it” and “So for now it is very very
low, very very low because we are still coming up with the proper strategy and more or less use cases for it…
so currently I can’t say it is adding that much value yet”.
3.3.6. Training
Most participants eluded to the self-study nature of a lot of the training required to develop big data
skills. Interviewee 7 said; “Literally some of the stuff we want, there is nowhere that you can study it. You go
online, play with the blasted thing, download it on your machine and use it”. Interviewee 9 echoes this
sentiment by saying; “you can actually teach yourself quickly, if you have the attitude and interest and all that
stuff”. Interviewee 6 mentions that training is through on the job experience; “We don’t have a formal thing
for that, it is on the job training, as you go you adapt” and interviewee 12 says that training is; “more of a selfstudy”.
Some organisations enhance training through online courses, third party training, programs and
seminars. Interviewee 7 stated that; “there are companies and courses all over the place. I mean you can go to
Coursera on line and do stuff”. Interviewee 1 said; “it is an external company that does that so it is not only
about self-training” and interviewee 15 said; “we have got a lot of user group training sessions… that is
basically based around specific platforms”. Companies that are serious about developing their BDAC skills
have a training budget and resources that are allocated to big data skills enhancement and development. One
participant mentioned; “there is a budget available as well on a yearly basis so you certify whoever you need
to certify”.
Interviewee 9 from a more mature big data organisation mentioned how they take training to the next
level by having combined sessions, like grade sessions and hackathons, where everybody comes together to
find big data solutions and can then learn skills from each other. This is a particularly useful method to utilise
28
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
for more complex problems that require more complex solutions and techniques.
3.3.7. Organisation
Interviews with participants from companies at varying big data maturity levels were conducted. The
general trend was that less mature organisations have tended to not restructure business units for better BDAC
incorporation. Interviewee 14 said, “so I haven’t seen reorganisation or redesign, I have rather seen that they
have added to the organisational structure”. Some respondents thought it may happen as they mature and others
believed it is unlikely the business units will restructure. Interviewee 15 stated, “it’s not a necessity that it
needs to change because the aim of big data should be to facilitate what they already have in place”. More
mature and leading big data companies have gone through some organisational restructuring to allow big data
to be the centre for decision making. Both centralised teams and deploying big data professionals within
business units are utilised in more mature organisations. Interviewee 7 said; “two years ago the division I am
in was created from scratch; it was an amalgamation of two different areas, so one was looking at analytics, so
data science and all of that, and the other one was what was our data warehouse. So, we put this together under
a new leadership team, which I am part of, with a specific purpose of having a centralised data repository and
platform that we could use to service all the data scientists across the bank” and Interviewee 9 stated “there is
now a whole new restructure or different structure in the whole system technology space to accommodate
where we are going. At one point they actually put together what they call cross company analytics”.
Interviewee 1 mentioned how they have two division of big data, a business and an IT division, where
IT provides the support and business makes the decisions and sets the direction for IT to follow. This is
supported through the statement; “it has got two divisions, it has got the IT side and it has got the business
side, both of those sides is very well led but it’s the business side that has got the most power and the most
decision making the IT side is there to support the business side”.
3.3.8. Application and Uses
Many applications and uses of big data were mentioned throughout the interviews. Some of the main
uses mentioned were that of marketing opportunities, customer satisfaction, operational efficiencies,
innovation, predictions and forecasts, and increasing revenue. Table 5 below lists the main uses of big data
found throughout the interviews with key quotes to substantiate each use.
Table 5. Main Applications and Uses of Big Data
Quotes
Big Data
Application
Marketing
Use
Customer
satisfaction
Offer customer exactly
what he needs
Sentiment analysis
Operational
Improve efficiency
Lean processes
Improve response times
Compliance
Reporting
Improve systems
Automate transactions
Fraud detection
Evaluate KPI’s
Customer insights
Reducing uncertainty
Improving accuracy
Targeting customers
Understanding
customers better
Trend analysis
Understand markets
Directed or intentional
marketing
Interviewee 2: “So what happens is when you start with a specific set
and you start building on that and that’s how the data gets bigger and
bigger and bigger. Eventually you are taking away all sense of ‘what if’
you know exactly who it is”
Interviewee 6: “our customer information would be the other one, how
well do we know our customers”
Interviewee 7: “Those three offers are not random advertising; the
customer has been pre-scored and there is a propensity model to work
out what is the customer most likely to take up next”
Interviewee 12: “understanding client behaviour and trends and client
trends”
Interviewee 7: “so we did a twitter sentiment analysis and project just to
work out what is happening on twitter in terms of us, other banks,
competitors”
Interviewee 9: “sentiment analysis – is the client happy, and what can
we learn from the clients to aim at the right service consultant of that
client, what can we learn from that client to give them the best benefits,
upselling opportunities, those kind of things”
Interviewee 5: “you can use it for internal processes in terms of HR
hiring practices”
Interviewee 7: “fix their customer information system because it was
always out of date” and “So we are making massive inroads into fraud
detection”
Interviewee 9: “it is like what are we doing as a business and how do we
do that best” and “getting cleverer with using the data to be better at
measuring the data KPIs”
Interviewee 14: “using a future amount of big data in fraud mitigation,
29
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
HR hiring
Products and services
development and
improvement
Manage risk
credit card fraud, those type of things”
Interviewee 1: ‘getting problems and trying to come up with innovative
ideas to resolve those problems”
Interviewee 9: “trying to build the best products and build the best
solutions for the clients”
Predictions and
Forecasts
Customer behaviour
Revenue
generation
Unique offerings
Interviewee 1: “you study the patterns of your customer and try to
predict what they will do next”
Interviewee 13: “it is about trying to do predictive models where you
can correlate certain information”
Interviewee 1: “it’s about generating revenue”
Innovation
3.3.9. Advanced Big Data
Participants mainly believed that advanced big data was about the technology and achieving
automation, AI and machine learning. Interviewee 2 stated; “right now we are not near as sophisticated. We
are building a super hi-tech system that is going to be unbelievable”. Interviewee 7 said; “that requires a lot of
automation… your AI component of it, so it learns and its ability to decide is a lot better”, while Interviewee
3 and 6 concurred by stating respectively; “I would say advanced big data would be getting to the machine
learning and cognitive space. So, where you are starting seeing value out of unstructured data that you weren’t
getting beforehand” and “. I think all of the advanced stuff is people trying to crack in essence artificial
intelligence, or some form of artificial intelligence to communicate with your customer – and that is what I
would consider advanced”.
Interviewee 1 mentioned how they believed advanced big data was also about pervasive big data where
big data is a predominant source and part of revenue generation, improving customer experience and resolving
business problems. Interviewee 2 agreed saying that although advanced big data is about advanced systems
and backends, it is also about being advanced in terms of results, outcomes and value. Advanced big data is
seen as accurate and effective big data and the ability to provide up to date information.
Interviewee 7 also believed advanced big data is when the organisation has the ability to compete
against other data companies and not just industry competitors. Interviewee 16 gave their dream of big data as
advanced big data stating; “Its where we don’t even need to ask a question, all we need to do is wake up in the
morning check your phone and you have got a message telling you yesterday this is what happened and this is
why it happened and potentially this is what you can do to improve this going forward”.
Interviewee 9 gave an integrated view of advanced big data across the organisation where big data
competency has the ability to exploit data and derive exceptional value from it, big data implementation has
the ability to apply and use advanced big data tools and techniques and big data reporting has the ability to get
insights to the clients or departments in user-friendly ways for them to action. “Practically combining the
data… and then it is about the intellectual ability to look for the right things, and then it is about implementing”
3.4.
Results for Research Question 2
Research question 2: What capabilities are required in order to be successful at translating big
data insights into organisational performance?
Research question 1 established which organisations were more mature with regards to big data and
more advanced with their ability to successfully use big data and develop BDAC. The questions in the
interview guide for research question 2 were asked to establish the perceived capabilities and reasons as to
what enables an organisation to successfully use their BDAC for better organisational performance. The
question also sought to confirm whether the factors identified in the literature of the previous paper were
confirmed to be the main drivers of BDAC success and this will be expanded on in the next section.
During the interviews, participants mentioned what they believed to be positive factors driving big
data success. These factors were grouped into constructs and then further grouped into nine main themes that
emerged from the sixteen interviews conducted. Table 6 below shows the frequency count of how many
participants mentioned each theme throughout their interview. The higher frequency does not necessarily mean
that that theme contributes more to BDAC success but rather that it is a more prevalent or known aspect that
is required for big data success amoungst the participants. This section will now outline each of the themes
and their constructs in the order of their frequency.
30
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Table 6. Frequency of Big Data Success Factors
Rank
Theme
Frequency
1
Organisational
15
2
Openness and Awareness
14
3
Competency
9
3
Data Competencies
9
5
Development
8
6
Existing Experience
6
7
Forward Thinking Focus
5
7
Cost and Benefit
5
7
Security and Trust
5
3.4.1. Organisational Strategy
Mature organisations and big data consulting participants agreed that companies have a strategy and
vision for big data and big data implementation with a good idea of the opportunities in big data and the value
than can be derived from it. Interviewee 2 stated; “Yes so we do, we know where we want to be in about threeor four-years’ time”
In order for big data to be successful it needs to form part of the overall, long-term strategy as well as
the digital strategy. This fact was mentioned by the majority of the respondents with interviewee 1 stating; “It
is very very strategic, in fact big data is part of what they call vision 2020”, interviewee 7 said; “they identified
I think between ten and twelve strategic things that the bank wants to build and underpinning that is the data
strategy” and interviewee 9 mentioned that; “data is critical to absolutely everything we do”. Interviewee 3
mentioned; “they view it as a very strategic play as to how organisations become a cognitive organisation; so,
without having big data mechanisms in place, an organisation can’t move to become a cognitive organisation”
Interviewee 10 made the interesting point that big data projects that align with the strategic objectives of the
organization easily get buy-in and support.
When making big data a part of the strategy, organisations need to make decisions based on data
whenever possible and ensure there is a clear goal and vision within the organisation to help define what big
data value and value generation looks like. Interviewee 9 stated; “It doesn’t say that there is not “argy bargy”
and you know, very stressful situations that happen and what’s the best way to implement this, but it is almost
like the business case wins.” And interviewee 4 said; “its core to our business, all our strategies revolve around
big data, 2025 digital strategy, it is part of our innovation and we rely on it heavily to make strategic decisions”.
It is important for organisations to not only have a plan for big data technology and insights determination but
also for big data implementation as well.
Part of a strategy process is benchmarking and best practice and this no different for BDAC. The top
big data companies are benchmarking internationally and against top data companies for their goals when it
comes to BDAC. Interviewee 7 stated; “our exco actually flew over and visited Amazon because what they
are doing is quite fascinating”.
Leadership
A key organisational factor that drives big data success is the support and drive of big data from senior
leadership at an executive level and this is then disseminated down the organisation. If big data is seen as a
vision of leadership, the rest of the organisation will make it a priority to implement. The leadership believing
in and driving big data from a place of personal passion also helps to create buy-in and lead others to be
passionate about big data success. Interviewee 7 stated; “So our group CEO, I think he sold his car a year or
two ago; he only uses Uber. Because his thing is he believes someone is going to disrupt the industry, like
Uber disrupted the taxi industry… it is very centrally driven. So, ja, there is massive buy in, and again, it comes
back to our CEO”. Interviewee 8 was very clear that; “it is supported at top level management downwards”.
Interviewee 14 revealed that the emergence of top leadership for big data focus (chief data officer and chief
analytics officer) has helped propel big data forward in the organisation. Interviewee 1 stated; “driving from
the top down, CE is driving it, the CT, the CIO is driving it, so there is quite a lot of interest in big data to
make sure that it succeeds”.
Some companies require a change in leadership to fully reap the benefits of big data. Leadership may
need to have different skillsets and different cultures in order to embrace big data rather than the legacy systems
and processes. Interviewee 5 mentioned the importance of leadership making big data a priority so it does not
get overlooked ignored for other tasks or current day-to-day operations.
31
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Forming a centralised BDAC department with top leadership is important and something that sets the
two top big data organisations interviewed from the rest of the interviewee’s organisations. This centralised
team also becomes the leader of BDAC in the organisation and drives the forward progressing uptake and
implementation of big data throughout the organisation. Interviewee 7 stated; “we put this together under a
new leadership team with a specific purpose of having a centralised data repository and platform that we could
use to service all the data scientists across the bank”.
It is important that leadership takes place on each big data project itself where someone takes
ownership of the project and holds the accountability for its success. This was demonstrated by interviewee 7
who stated; “because it needs to have a sponsor so if someone is not going to drive it, we won’t use it”. It is
also important that leadership ensures the core structures are correct for fully realising benefit from big data
and are willing to make any necessary structure changes to ensure they make sense for successful big data
implementation. Interviewee 9 said; “making sure the core structures are supported… making sure that
structurally everything makes sense”.
Commitment
Commitment to utilising big data and implementing big data solutions is of utmost importance to show
the employees that the organisation is serious about big data and making big data a central part of decision
making and solution driving. The attitude of, failure not being an option and everyone needs to make big data
work, is important to drive positive and determined employee behaviour and shape the correct culture for big
data success. Interviewee 1 stated; “there is a lot of money and resources we are investing in it… it is a big
deal so there is no turning back” and “there is no margin for failure, no one can invest so much in big data and
then it’s a failure so failure is not an option”. Commitment needs to be shown by spending money on systems
and investing in resources. Interviewee 14 said; “a lot of the organisations are spending a lot of money on big
data implementations”. Interviewee 10 talks about the investment in data collection systems; “since I started
there has been a big push, we need access to this data, and we need all of these different features”. These
resources need to be skilled in big data and visible within the business units.
Investment into the big data agenda is key to its success and growth in maturity. Hiring the correct
human resources with the skillsets required for each aspect of big data is important to show the commitment
to seeing big data from systems to capture data through to implementation and realised improved organisational
performance. Interviewee 5 gave an example of the difference between organisations that dedicate resources
to big data properly and those that do not by saying; “here’s a good example I can give you: you will go to
some companies where there is one person that is responsible for all of this, and you’ll go to other companies
where there is an entire team and structure dedicated…and that in itself says something, so how much resources
are being dedicated to it makes a difference”. The next step to commitment is to develop and enforce new
processes and procedures that will enable big data implementation success. Structural reorganisation may be
required to optimise these processes.
Commitment can be measured through the growth in the number of big data projects being completed
successfully by the business units and the centralised big data team themselves. An increase in the amount of
investment that is required as well as the big data resource skills required will also indicate an increased
commitment by the organisation to utilise big data. Interviewee 7 stated; “the demand in terms of what we are
building is massive, so I am starting to have to juggle resources… I am currently going through a massive
planning phase with everyone and some of the teams have got up to 24 months of work in their current capacity
– booked up! Which is quite crazy!... I have literally doubled the team in terms of running the projects”.
Monitoring, Measurement and Incentivising
It is important to monitor the progress of big data and big data implementations. Putting measurements
in place with clear goals and then ensuring the organisation is tracking the performance. Interviewee 7 revealed
that they set targets on the number of data projects successfully completed for the year. Interviewee 9 stated;
“So we track and monitor pretty much everything so as we implement new stuff, we get the feedback as to
whether it is working or not and make changes based on that”. Interviewee 4 mentioned that performance
evaluation and remuneration and business units are aligned with big data. Interviewee 7 also highlighted the
importance of feedback on big data and told us that; “we have a data and analytics EXCO which spans the
entire group and so in that EXCO we get feedback on what is happening, challenges and issues are also raised”.
Incentivising successful and top achievement in big data performance from data gathering to
implementation is important. Both interviewee 7 and 9 are from top big data organisations and they
differentiate themselves from the rest of the organisations sampled by stating that they both run internal
competitions where data related problems and posed and there are rewards for the best presented solution to
32
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
the problem. Interviewee 5 also said; “there has to be an incentive for everybody else in the organisation to
work towards”.
A common method in most larger organisations for measurement is through the use of Key
Performance Indicators (KPIs). KPIs can be used for big data monitoring, measuring and incentivising. One
participant said; “your KPI’s are quite clearly set and you always do your day job. We are very good at
rewarding and incentivising, and it is very clear that you need to deliver, you get paid to do a job, so they make
quite sure that it’s measured”. Data itself can be used to measure and evaluate KPI’s. Interviewee 9 revealed
the importance of refining the KPIs and ensuring they are clearly understood in terms of contribution.
Human Resources
Not only is it important to hire resources with big data skills but it is also important to hire the correct
type of people with the correct aptitudes in all business units going forward. This will allow people to adapt to
big data implementations and have the correct competencies to view situations in ways that can use big data
as a method to solve problems or be innovative. Interviewee 9 believes employing many highly educated
people who are accustomed to problem solving and data analytics contributes to the success of utilising BDAC
for improved organisational performance. Interviewee 15 concurs with this sentiment by stating; “you need
people with that core critical thinking and problem-solving abilities”. Interviewee 7 believes that hiring people
with the ability to learn a new technology is a more important skill to look for in a potential employee.
Interviewee 12 said; “there are definitely people who are more progressive thinking, forward looking” and
thereby highlights the need to hire individuals with progressive and forward-thinking competencies.
One participant highlights the importance of technical big data employees learning the business
understanding as well by stating; “it is always that the importance is getting them to understand what we are
trying to solve; they need to understand the business processes and things like that. It’s important they spend
time in the business”. They also state that the opposite is also important by bringing in more and more big data
competent resources into the business units to transfer learning of BDAC to the more business focused
employees.
Interviewee 4 reveals that it is important that job descriptions be less confined when it comes to big
data in order to account for necessary agility and adaptations that may be required to successfully implement
a big data project. Interviewee 5 believes that BD implementation and application skills are more important to
drive big data success than technical skills. Interviewee 6 makes reference to the competitive advantage that
can be attained by employing and having access to the best big data skills.
Big Data Champions
Big Data Champions are important to drive enthusiasm and support for big data initiatives and projects.
They are important to create awareness around big data and encourage business units to use big data as their
main source of knowledge to solve problems or complete tasks. During the interviews everyone was in
agreement about champions being required and participants from organisations with struggling big data
initiatives generally did not have big data champions or they did not have the correct type and level of
champion. It was revealed by participants from more successful big data organisations that they have high
level and powerful members of the organisation as their big data champions. Interviewee 1 stated “there is
definitely a champion… the project sponsor of the whole thing, the guy is quite powerful”. Interviewee 16
from an organisation just starting to implement big data said; “yes we do have a champion, in other words we
have got one of the MD’s whose division we are trying it on, is set to be the first champion” and this is a good
step to helping promote their big data drive.
Interviewee 4 revealed that the big data unit rather than a person is the champion. The big data unit is
established enough not to require a champion that much anymore as they are well known throughout the
business and respected. Interviewee 10 and 11 said that in their organisation there are big data champions,
advocates or disciplines within each business unit that truly believe in and are passionate about big data.
Interviewee 13 stated; “you definitely need a champion and you need a champion that understands business
and understands the capability of the technology” and this raised an important point about the competency
required from a big data champion being that of business acumen as well as big data understanding.
3.4.2. Openness and Awareness
The organisation and the culture of the organisation needs to be very open-minded to be successful at
implementing big data initiatives. It is important that there is buy-in and a good sentiment towards big data
across the whole organisation. For this to occur there needs to be a level of awareness of big data; what it is,
what it can do for each business unit and how to go about utilising big data to improve organisational
33
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
performance. Interviewee 7 stated; “I have actually got an engineer I hired. Her job is to go and visit people
and find out what their problems are, and work out if there is data that can be utilised to fix this problem” and
interviewee 4 mentioned the importance of people within units in the organisation to bridge the gap between
the unit and the big data hub and help with understanding throughout the organisation. Interviewee 5 agreed
with this sentiment by saying that the link between big data analytics and the rest of the organisation is
important.
The focus should also be on collaboration, in order to leverage data across business units and be able
to gather the best knowledge and datasets by combining data from each business unit into one database. Each
business unit can then leverage off of the whole organisation’s efforts to collect data and not just their limited
ability. Collaboration should remove siloes and allow the business to work as a whole with big data as the
underlying driver linking all business units together. Interviewee 7 stated; ‘The thing with data though is you
want to be able to leverage each other’s data and you want to be able to collaborate. So, part of the strategy
now is to get collaboration particularly around data. So, what we did is we created a group wide strategy so
anyone can come with a problem that they believe data will solve. They need to business case it and then
volunteers from multiple business units would get together and we will create a sort of a virtual team that will
solve that problem… and the data then gets hosted centrally”. Interviewee 9 revealed the importance of the
central unit encouraging collaboration across the business units and stated; “we share data a lot, across the
company. I think that is important, there is no ‘you can’t have this because it is IP or whatever’ – we really do
share data”.
An organisation needs to create an environment where it is not only the data scientists using big data
but the whole organisation. Interviewee 1 said that part of their success is that anyone can approach the big
data unit with business problems to use big data to help solve; “everybody within the business who has a
problem that they need to resolve or a problem, big data is there for them… there is no cut off point here… as
long as you have a clear defined problem and data available, then we are able to use big data and data science
to resolve whatever challenges which you might have and there is no cut off there… any executive that thinks
big data or machine learning will be able to resolve his particular problem we are to here to help”.
A more mature organisation has realised that it is not only about business units approaching the big
data team to solve problems but about fostering a culture where business units can use the big data platform
themselves in order to solve their own problems. This creates a deeper understanding of utilising big data
throughout the organisation and ensures the limited big data team resources are not a bottle-neck to achieving
success with big data across the entire business. Interviewee 7 agreed with this statement and mentioned how
this importantly allows organisations to increase the speed at which big data can be implemented and said;
“we don’t want to have a dependency on an IT”. Interviewee 2 stated; “by next year Feb we will probably be
at our full mature stage, which means that big clients can have access to our application, put their own data
sets in around scoring models off our database they can do everything themselves. Right now, we do everything
for the client; it becomes a bit tedious and becomes a bit time consuming”.
As not all employees will have the necessary skills to do data science, automation and ease of use of
business platforms is important to enable self-service. Interviewee 7 stated; “part of the journey is to make it
user-friendly. The other big part of the journey though is…the bigger issue here is automation and selfservice”. One participant mentioned the importance of providing basic big data education to the business units
in order to make this possible because they need the awareness to start thinking about big data in the right way
and start asking the right questions for the big data platform to be utilised effectively and successfully
throughout the business.
3.4.3. Competency
Data Insights
A big aim and emphasis is to gain as much insight as possible from data and therefore the emphasis
needs to be on insights attained rather than data itself. This will allow big data to become a core competency
and facilitate competitive advantage if done well. To gain the correct insights it is important that the business
side of the problem is understood and the technical employees have been involved from the start in order to
facilitate this occurring. Interviewee 7 stated; “our strong aim is to gather as much data as humanly possible
and leverage insights off it, because data without insights is worthless”.
Data visualisation
The organisation needs to be aware of how the results of big data problems should be visualised to be
understandable and applicable to business and not just the technical specialists. Interviewee 1 mentioned the
importance of understanding how the results of certain big data problem types should be represented and using
34
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
the way they should be measured to guide visualisation. Interviewee 11 mentions how their organisation
ensures big data results are made very graphic and visual so that it is easy to understand; “you find that it shows
pictures and all those things just for anyone to see, because once it has come through visual graphs and all that
it becomes easy then”.
Implementation
Big data competency is not only about deploying a technology, but in its implementation and
application. This will enable organisations to derive the most value out of their investment in the technology.
Interviewee 7 said; “you can have the best tech in the world, if you are doing arbitrary stuff with it, it is not
going to help; whereas you can have medium tech but you utilise the thing till breaking point and you get a lot
of business value”. Interviewee 5 stated the importance of implementation to derive value from big data by
saying that big data has its most success if there is follow through. Working on implementation of solutions,
as much as solutions themselves is important; “the idea is not that we kind of partner with them and then run
away, the idea is that the delivery is just the first step of the project; after that a consultative process starts and
we keep working with them to make sure that they can make use of what we have delivered them”.
Implementation needs momentum and drive to continue moving forward and being able to act and
implement quickly in important for success. Interviewee 9 stated; “you know this company go, go, go, goes –
so we don’t look back too often but if we implement and it breaks then we fix and move on… we just drive it
forward and improve it”. Organisations must be able to quickly make structural changes required to optimise
and streamline roles for the big data solution implementation. It is also important to review and get feedback
on implementations to understand each time what went wrong and what could be done better. Interviewee 5
concurs with this statement and said that identifying and working on the obstacles to implementation is
extremely important. Many organisations are able to innovate with ideas of what big data can do but
implementation is behind and it is important to foster an implementation culture in order for implementation
competency to catch up with the ability to generate ideas and potential solutions.
3.4.4. Data Competencies
Developing the competencies in order to collect as much of the right data for the best knowledge is
important to big data success. A better knowledgebase from which insights are attainable is a competitive
advantage to companies. Interviewee 1 said that their organisation is aggressive when it comes to data
collection and currently collects more data than it can use but understands the importance of having
information available. Interviewee 5 states the importance of the quality and condition the data is in. Less
mature organisation or participants from organisations where big data as not been that successful often
mentioned that they did not capture data even though they could have and this was knowledge that was lost.
Interviewee 12 said; “it is simply because we do not collect big data, as maybe we should have been doing
because of the new sources of big data or data that has become available”.
Interviewee 2 mentions how data has become a valuable commodity and interviewee 1 mentions that
they are storing more data than they use as they understand they may be able to sell it one day. Interviewee 2
mentions that there can be competitive advantage in data, the data incorporated, how it is incorporated and
what is derived from the data. Interviewee 9 believes that there are two sources of competitive advantage from
data; capturing unique and exclusive data and using more common data in unique ways.
Span
Interviewee 2 mentions the importance of knowing what data to source to where to find the data. Data
can be everywhere and just collecting everything with no plan can hinder an organisation more than help to
achieve better performance. A plan for collecting the right data and knowing where this data can be sourced is
step 1 to ensuring a more organised and direct ability to utilise data. Data can then be collected across all
business units in an organisation and stored in a central depository for data but interviewee 7 mentions that
there is a greater span of available data by increasing collaboration across a group of companies that the
organisation belongs too and not just the organisation itself. Interviewee 2 mentions how this is taken a step
further and there is a lot of buying and selling of data between companies to allow each company to build
better and more updated profiles and they also mention how a lot of data sharing takes place between
companies; “data in exchange for data”.
Sophistication
Data analytics has come a long way and increased in sophistication throughout the years. There are
more sources of data available and the ability to combine sources to have even more knowledge in one place
35
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
about any particular subject has increased exponentially. Interviewee 2 mentions how data analytics started
with basic data capturing and basic profiles’ “captured through credit bureaus” but many research companies,
data providers and data builders have entered the field and they generate a lot more information on business
and consumers. It is important for companies to understand this progression and take advantage by being aware
and utilising the new sources of data.
Sophistication can also be in the form of an organisation being smart in where it looks for data, what
and how it measures and how it implements and this sentiment is mentioned by interviewee 9. Interviewee 7
goes on to say how sophistication in data competency moves into doing tasks such as using meta data and
other methods to improve data and test data quality and integrity; “part of our structure now is it is not enough
to just get data in, you have got to have meta data, you have got to have quality, lineage, know where it comes
from, you need to know if the data shifts or if it gets corrupted or anything like that. So part of our infrastructure
investment is it is not just about getting the data into our central platform and making it available, it has got to
get in there and we have got to know that this data is good data; it has been validated, it has been checked and
so basically it is the purity of our data.”.
Uniqueness
A source of competitive advantage is gathering unique data to your other competitors because this is
exclusive knowledge from which to gather insights for better decision making than the competitors.
Interviewee 9 states; “we are getting information that other companies are not getting because of just our
unique products”. Technology is therefore a means to acquiring unique data and interviewee 9 points out that
they can measure things other companies are not able too and this generates unique data for their organisation.
Interviewee 5 also touches on the importance of being able to have access to exclusive data; “there is no other
company in SA that has such an extensive read on that. So, that’s really unique”.
Innovativeness
In order to gather unique data, you need to be innovative in the thinking around how you are going to
source data, this is demonstrated well by the organisation of interviewee 9. This and then innovativeness with
the way in which more common data is used are the two ways innovativeness can provide competitive
advantage for an organisation. Interviewee 2 stated that data sets and data profiles may involve a lot of
innovation to build, with a lot of indirect ways of gathering the needed information. They also mention how
being able to build very innovative data sets and data bases from scratch, opens the door to new business and
marketing opportunities; “You’ve got to think outside the box, if someone wants information somewhere being
a data expert you never say I can’t get you that because you can always get it”.
Software
The interviews revealed an interesting debate between using vendor-bought software versus open
source software. For example, interviewee 8 and 11 stated that the organisation mostly uses vendor software,
rather than open source software with interviewee 8 adding that the vendor software will be made more user
friendly, more interactive, and faster in terms of getting results. But interviewee 7 and 9 agree that open source
software was the key to allowing big data to be utilised by many more organisations and gives organisations
the freedom to quickly adapt to new methods and technologies required. Interviewee 10 agrees with open
source and states; “embrace open source because one issue that like if you get stuck with vendor lock in then
if you bring new people in then they are not going to know how to use that product and you are also reliant on
that company that is providing the product to give you new features, and then there is like a whole onboarding
process. So I think they need to do a lot more around teaching people to use Python and R and using some of
the machine learning modules that are out there”. Interviewee 14 mentions how their organisation use a
combination of vendor and open source software. Interviewee 12 gives their summary where they state;
“typically I say if you are a SAS user you fall into one category, if you are an open source user you fall into a
different category. And the people who use open source are typically more entrepreneurial and innovative in
their approach; SAS and those kinds of environments do not allow for much.”. SAS is an example of a vendor
software.
Based on the interviews, more mature organisations are moving to open source software due to the
flexibility it provides as well as the skills development being easier than needed third party system training.
This removes reliance on a third party and allows the organisation to use the software in a way suitable to their
organisation and their unique requirements rather than a more standard approach as is found in the vendor
software. Open source is also more cost effective as it does not require a large upfront fee to purchase or large
licence fees per user to utilise. Building data competency in open source software that can meet the
36
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
requirements of the big data project can lead to more successful BDAC.
3.4.5. Development
It is important to develop big data systems and technology as well as big data skills.
Technology
Interviewee 2 said that they believe in the importance of optimising, standardising and automating the
big data platform in order to allow for quicker results and the ability to complete more big data projects at one
time. Automation also facilitates removing reliance on the big data specialists and allowing business units to
use the systems themselves effectively.
One participant mentioned that the development of front-end platforms helps to introduce userfriendliness into the system in order to allow non-technical business unit employees to access and utilise big
data. Interviewee 7 specifically highlighted the development of visualisation for simplicity and userfriendliness. Another participant said it was important to develop in order to improve the response times of the
big data systems to reduce the time taken to solve big data queries and problems. Interviewee 9 said
development was important to keep up with the latest technology.
Exploration
Interviewee 1 stated; “before you can do anything with your data the first thing you have to do is to
do exploratory data analysis otherwise you are not going to get any value implementing any big data
solutions… you have to understand in order to implement, so one of the first things you need to do is to explore
the data”. This highlights the importance of exploration to gain new understanding and develop big data
capability. Interviewee 9 mentioned that their organisation allows a lot of freedom at the idea generation phase
to facilitate exploration and develop skills to be innovative and generate new ideas.
Skills
Training is important to develop skills in utilising big data, from systems through to implementation
and from big data specialists to employees in all business units. Interviewee 7 stated the importance of training
to help business units build solutions; “what we have to do with them though is teach them, the solution has to
fit the platform”. Interviewee 9 mentions how they develop skills through conferences, hackathons and
combined sessions to bring people together to find solutions to big data problems. This is important to further
skills development as people learn from the challenge and from each other. Interviewee 15 states; ‘I think what
is really nice is we have got a lot of user group training sessions and we actually host some of them as well
and that is basically based around specific platforms that we use”.
3.4.6. Existing Experience
Existing experience in analytics can help an organisation develop their BDAC capabilities by forming
a starting point from which to understand the power of data as well as benchmark against to ensure big data
delivers more value than traditional analytics. It also allows organisations to take smaller steps towards using
big data for problem solving by starting small, proving value and then scaling-up the big data operation.
Example of doing this successfully came from interviewee 1 who stated; “customer value management has
been here for quite a while, but as big data grows further and further the intention is for big data to replace that
customer value management” and interviewee 5 who said that analytics preceded big data in their organisation.
Interviewee 9 said; “the big data being this sort of huge thing, that’s quite tricky, but there are a lot of interim
steps that you can take” and they believe that traditional analytics; “the excels and that” are important as you
need to start by understanding what you are doing and build from there.
Interviewee 3 mentioned starting small and gave the example; “I think in their call centre they could
really use it”. They eluded to the fact that it was important to gain some experience first by getting it right in
some area or location and snowballing from there. Interviewee 5 stated; “their exposure to it as well… when
we work with an organisation with stakeholders in the business that have worked with this before and are
familiar and comfortable with it, they will champion it, because they understand how good it is”. This
highlights the fact that existing experience in analytics can provide the necessary proof to get buy-in from
employees that big data analytics will provide value and is worth learning and understanding. Interviewee 5
mentioned that it was less daunting when you already had basic analytics knowledge.
3.4.7. Forward Thinking Focus
It is important to be forward thinking when it comes to developing and implementing big data. There
is a common sentiment amoungst the interviewees that big data will continue to grow in focus and support in
37
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
the future. As a result, organisations will become more dependent on big data as the basis for decision making
in the future. Interviewee 1 said; “big data going forward is still going to be a measurable driver and most of
the decisions will be based on it” and also stated that big data will further shift decision making to be data,
insight and fact based. Interviewee 1 expects big data and the insights it provides to further influence
organisational and business unit strategies, further change how business is done and how the business thinks,
and will further change how and what the business implements. One participant stated that they believe there
is still untapped potential and massive opportunities for big data to add value in the organisation and that big
data has the potential to offer plenty more solutions and improvements. Interviewee 2 mentioned that they
believe big data has the potential to open up new markets for organisations in the future. This was repeated by
interviewee 9 who said that expanding reach and scope was eminent in the future as well as companies entering
new areas, applications and markets. Interviewee 7 stated that organisations that are not data or data analytics
competent are likely to be disrupted and made redundant in the future.
Big data is predicted by the participants to form a greater part of revenue generation as it is better
utilised and implemented as well as more innovative solutions generated as skills and competencies develop.
Interviewee 1 stated; ‘we definitely have to generate revenue going forward and it is quite central from a digital
migration platform point of view”. Participants from more mature organisations have started to become better
at big data implementation but organisations still growing their BDAC have said they expect better
implementation competency in the future. Interviewee 1 mentioned that their organisation is looking at driving
sentiment analysis in the future while interviewee 7 is more focused on the convergence of industries they
believe will occur as more organisations become big data driven. Interviewee 12 stated; “traditionally data was
collected for auditing and accounting purposes, and there is a key change that needs to be made so that data is
a key asset or core competency” and this highlights the shift from data for compliance and management
(auditing and accounting) to data as an asset or core competency.
3.4.8. Cost and Benefit
Organisations in early maturity with regards to deploying BDAC or organisations that have not been
successful in implementing big data solutions, often would refer to the cost of big data technology and skills
as a prohibitive factor. This is mainly because they have not managed to derive the benefit and value from big
data, which the participants from the more mature organisations mention far outweighs the cost and creates
the scenario where a big data business case can easily prove its value and make sure any upfront cost for
hardware or skill is not a barrier to implementation. Interviewee 1 stated; “costs are going to be insignificant
and the return on investment on this thing is going to be massive” and also said that although big data has an
upfront deployment cost, this can easily and soon be negated by the value and benefit big data unlocks and
provides. Interviewee 7 agreed and said; “if I do a cost benefit analysis, the amount we have spent for what we
have made is nothing”. Interviewee 14 stated that truly innovative organisations are less concerned about cost,
because they have benefit in mind.
Open source software is also seeing the decline in upfront costs to big data projects while returns on
investments are increase as organisations gain experience and improve in utilising BDAC. Interviewee 1
echoed this sentiment, as did interviewee 7 who stated; “the costs are high but the return on investment seems
to be even higher”. Interviewee 15 demonstrated the negative effects of not demonstrating the benefit of big
data initially by saying; “you find that if companies have had a previous run in where they were sold something,
‘oh this is going to be brilliant’, and then things didn’t work out or they didn’t quite get what they were
expecting then you know the sentiment is not so great”. These organisations must now first be sold and made
to understand how big data works and how big data will deliver value before they will begin to utilise it
properly.
3.4.9. Security and Trust
Data security is crucial for the accuracy of the data as well as the trust it will build with clients or
within business units. Interviewee’s 2 and 7 stated how imperative data security was to their organisations with
interviewee 7 saying; “the security side of it is massive. So, protecting the data, keeping the data safe, making
sure our data doesn’t get breached”. Interviewee 7 went further to say that this will be part of maintaining and
building customer trust along the way. This was echoed by interviewee 9 who said it was important to build
trust within the community because the believe in the security of their data at that organisation. Interviewee
15 stated that building trust relationships are important because it opens the door wider for further and larger
big data projects.
38
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
3.5.
Results for Research Question 3
Research question 3: What are the biggest challenges/risks to converting big data insights into
organisational performance?
The last section of the interview asked participants for inhibitors of big data performance and
established the main challenges and risks to a company’s big data performance. The question also attempted
to confirm the challenges found within the current literature, and see if there were different challenges that
emerged within a South African context across the five industries sampled.
During the interviews, participants mentioned what they believed to be challenges inhibiting big data
success. These factors were grouped into constructs and then further grouped into fourteen main themes that
emerged from the sixteen interviews conducted. Table 7 below shows the frequency count of how many
participants mentioned each theme during their interview. The higher frequency means that more participants
experienced this challenge or believe this is a challenge to big data success. This section will now outline each
of the themes and their constructs in the order of their frequency.
Rank
1
2
3
4
5
6
7
7
9
9
11
11
11
14
Table 7. Frequency of Big Data Challenges
Theme
Frequency
Skills and Resources
15
Proof of Relevance and Value
11
Maturity and Competency Development
10
Politics and Sentiment
8
Sophistication and Complexity
7
Tasks, Routines and Processes
6
Accuracy of Big Data Outputs
5
Legacy Systems
5
Regulation
4
Management
4
Data
3
Awareness and Involvement
3
Ethics
3
Research and Development
2
3.5.1. Skills and Resources
Skills required for big data is viewed as a challenge by all but one participant. The availability of skills
to hire, the development of skills as well as the cost of skills and diversity of skills required to make a success
of big data are factors making skills and resources a big data challenge.
Human Resource Scarcity
Interviewee 1 said that skills for big data are broad and scarce by stating; “skill is an important factor,
there are a lot of skills required in the big data environment, we think the data scientists, the geniuses, the
production engineers, all those things are quite a skill on their own and to find people like that in South Africa
is almost impossible”. Interviewee’s 5, 7, 9, 14, 15 and 16 agree and say the skills shortage is a challenge and
Interviewee 3 says that big data trained professionals are scarce. Interviewee 13 believes lack of skills hold
back big data implementations.
Training and Experience
Interviewee 1 states the importance of investing in training but that most training is online and as
people then have to be self-disciplined and passionate about upskilling, not everyone improves at the rate they
could if they are not self-motivated to complete the self-training. Training can also be by third parties but there
is a cost involved and when budgets are tight, this may be a deterring factor for driving and supporting BDAC.
Interviewee 2 concurs with the above sentiment in that there is a lot of self-learning and learning through
experience involved with big data by stating “training on data is not anywhere near where we need to be… It
is owner managed”. This participant also mentions how big data timelines may be missed if there is missing
experience and maturity in the big data employees.
Interviewee’s 7 and 9 mention how attaining big data implementation skill is important and this takes
time and experience and can therefore be a challenge. They also agree with the above maturity sentiment and
states that a challenge is novice users lacking expertise. This participant also said that practical experience and
self-learning is common but added that education may take too long because by the time the person is done
studying, the big data environment has moved on from what was taught during the degree.
39
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Retention
Interviewee 1 makes mention of retention being a challenge. This is because skills are scare and
therefore there is a demand which reduces loyalty and increases turnover. Interviewee 7 mentions the difficulty
in securing and protecting scarce big data skill; “one of our main challenges, is to try and get the resourcing
and then trying to keep the resourcing”. And also how traditional HR practices are not appropriate for this
environment due to the fast changes in the big data market and the inability to change the job requirements
and salary offerings quick enough; “traditional HR practices are way behind, you know the whole got a salary
survey, we peg this role at this, and you know our answer is six months ago that was the right salary for that
role but now it’s not”.
3.5.2. Proof of Relevance and Value
A challenge many organisations are having is that big data is still hype and seen as a buzzword. This
occurs due to the challenge of proving the relevance and value of big data through successes when the
organisation is still immature in its big data capability. Interviewee 1 states; “I think it is a buzz word but a lot
of people still don’t understand it”.
Value can only be demonstrated when there is a clear vision for big data implementation and a
challenge is that many organisations still do not know how to use and exploit big data for their needs.
Interviewee 1 said; “they just know that there is big data but they don’t know how to use it or how it feeds into
the network”. Interviewee 2 stated that an application needs to first be found for data before the value will be
discovered and this can be challenging. Interviewee 9 said that there can be an uncertainty about the benefits
and value of big data where people do not know how to make big data work. Interviewee 16 stated; “I think
people generally complain about it being a fad, or for a lot of people they don’t really see how it helps it is
very difficult to explain it to a number of people to say this is what it can do for you”.
Interviewee 7 stated that a lot of effort is required to ensure no bad big data related experiences occur
in the business or this will deter future desire to utilise big data. The same was said by interviewee 9 who
mentioned the challenge of ensuring reliability of implementation and ensuring the big data team delivers on
its promises when it comes to benefits. Interviewee 9 indicated demonstrating value can be challenging because
the big data contribution is not always directly measurable.
Some participants mentioned cost as a challenge to big data but as was established in section 5.4.8
above, cost is only seen as high or a problem when organisations have not yet matured to the point where they
are deriving value and benefit that outweighs the cost. Therefore, this challenge mentioned is more a
consequence of businesses finding it challenging to prove big data value.
3.5.3. Maturity and Competency Development
A challenge with big data is that an immense amount of competency building needs to take place in
order to develop and mature with regards to big data implementation success. This is mentioned by both
interviewee 1 and 2 where they both admit their organisations still have a lot of room for big data growth.
Immaturity often brings immature processes which can make implementing on insights more difficult.
Interviewee 1 states that they are immature with regards to their big data processes and this immaturity is
challenging for the business because it takes too long to implement a big data model and provide answers to
business problems; “we tend to take a fairly long time before we can put a model into production and that is a
big issue”. Interviewee agreed that time to implement is an important factor of competition and therefore
maturity to ensure quick turn around times are important but can be challenging.
One participant mentioned that lack of competency leads to the big data unit being unable to do as
much as it wants to, or at a level that is needed. Interviewee 12 mentioned that there is a lot of competency
that still needs to be developed; “there are some very intelligent, very sharp people out there, but they may not
necessarily have the experience and it will take a while to upskill people into the data and methods the business
needs”. Interviewee 7 mentions that competency of data scientists is challenging because otherwise they do
not always understand the system and can cause errors; “one of the nice analogies I like from data is the whole
water system. So, you have a reservoir, a water purification plant and then it gets consumed all over the place.
If the water purification plant messes up, you can poison the whole city. That is the kind of problem we are
trying to deal with”. Interviewee 4 stated that lack of competency in the country is a challenge and still needs
to be developed.
3.5.4. Politics and Sentiment
Politics can occur in an organisation and be a challenge to big data success. Some people may have a
40
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
negative sentiment towards big data because they do not want to change their way of doing business or risk
losing the power they have if big data plays a larger role. Interviewee 9 describes this as complacency and
bureaucracy challenges; “You know they have been running a successful company for so long, ‘really, am I
going to get benefit from this?’ The market works. That is the sort of complacency. You can’t blame them
because it is established companies and then they have got too much bureaucracy”. Interviewee 15 describes
this as a mindset being a predominant obstacle to big data. More politics challenges are seen around sharing
of data between business units or departments within organisations. Interviewee 7 stated; “so it is politics in
terms of people wanting to own stuff, you know it’s mine, it’s not yours”. Interviewee 2 mentioned the
challenge of fear and mistrust around information sharing because of risk aversion and benefits being
overshadowed by dangers. Interviewee 6 said it is a challenge when key stakeholders withhold data; “people
sitting on data and not wanting to give it to you”.
Another sentiment that challenges big data is the privacy and security of information. Interviewee 9
said that people do not want their information known and are afraid of organisations knowing them to the
extent big data can provide knowledge; “there is a fear of big data”. Interviewee 7 echoes the challenge and
says that you need to produce actionable insights from big data using many sources for the best knowledge but
while respecting privacy. This participant provided a good example of privacy protection; “In the US they sent
a girl a pregnancy something or other and the father freaked out because his daughter was 15 but she had just
bought a pregnancy test” and called this “creepy data”, which is what you need to stay away from and there
can be a fine line.
3.5.5. Sophistication and Complexity
Big data systems can be complex in nature and so can big data outputs. This can be a challenge to the
business when they do not know how to properly utilise the system or interpret the output. Interviewee 7 stated;
“it is guys who don’t know how to use the tools properly… but we are having to put a lot of checks and
balances in to protect the platform from the users of the platform… That is probably our biggest problem at
the moment”. Interviewee 3 said that poor representation and interpretation of results is a challenge because
results are difficult to consume and not user friendly. Interviewee 5 stated; “Usability is one, you know it needs
to be easy and accessible for people to use it”. Interviewee 12 mentioned that the results can be challenging to
grasp; “it’s maybe the results, that are sometimes different to what people have seen before. So it is nontraditional, it doesn’t fall into the normal scope of things, so it is just a bit more challenging, the result – and
sometimes it challenges conventional thinking”.
3.5.6. Tasks, Routines and Processes
Big data tasks that need to be performed in the background in order to ensure accuracy of results are
time consuming and this is challenging when the business wants results quickly. Interviewee 7 said this
challenge is about getting the data ready, accessible and available quickly while ensuring data quality; “I would
say currently data onboarding is probably our most time. So, we spend most of our time on data onboarding,
data preparation…”. This participant also mentioned the time to test or check the model’s users wish to run for
practicality and errors as well as stated; “between getting our data clean and protecting our data I would say
that probably consumes about 50 or 60% of our time”. Another challenge this participant mentioned was the
power and lead time or latency of the system delaying answers. Interviewee 1 also talked about the time
required for data preparation tasks being a challenge.
Interviewee 10 stated that meeting business response time is a challenge because end users expect
quick responses and answers, and don't want to wait long for BD solutions to business problems and queries.
A lack of clearly defined and optimised big data processes is a challenge and this is mentioned by interviewee
1 who says this opens the door for issues to arise and they challenge they are now having is to optimise their
big data processes.
3.5.7. Accuracy of Big Data Outputs
Big data may be relatively accurate but due to the dynamic nature of big data it is not possible to be
100% accurate or flawless (Interviewee 2). A good example is forecast or prediction, which is only as good as
its input data and may not be an exact replica of the actual future. A lack of understanding can then cause
business to distrust big data in the future. A challenge is to try and get big data inputs as well as insights as
accurate as possible to gain business trust and produce the outcomes in organisational performance desired.
Interviewee 2 gave an example; “if you tell someone that this is Stephen and he’s looking for a fibre line (for
example) and they phone Stephen on that basis and Stephen says he’s not interested, you then have a problem”.
Interviewee 7 mentioned an important point for accuracy, which is not that all mistakes can be avoided
41
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
but that the challenge is recognising mistakes in time to avoid implementation on a mistake taking place;
“Sometimes the results are very subtly skewed and it takes you six months to work out that it is skewed and
you have done a whole lot of bad business, or you have made a lot of bad decisions. So, that is the massive
risk, the masses of big data, and getting it wrong”. Interviewee 5 and 6 also mentioned data integrity as a big
challenge of big data.
3.5.8. Legacy Systems
A challenge for big data can be converting or integrating into the legacy systems an organisation uses
(interviewee 1). Interviewee 7 agreed and said this can be challenging, time consuming and require a lot of
new deployment of tools and infrastructure if legacy systems are to be replaced. Interviewee 11 said that the
transition from older, conventional or legacy systems may also be an issue and interviewee 12 agreed this is a
challenge. Interviewee 9 stated; “we are already starting to build up legacy which does complicate things. So,
we are continually looking at ways to reset our structures to get all the data and systems effective” and “their
legacy systems, you have to do something, if you are looking at a big organisation, for them it is very tricky to
get started. So if you have something where there never was anything it is easier, because you can start
collecting the data in the way that you want it; as soon as you have something well then you are a bit stuffed,
they come up with these IT design systems and it is three years to implement, and you think ‘what, like three
years?’ Like an Ipad came in within a year everybody has an Ipad. You know, technology moves so fast, how
can you justify system implementation taking three years, yet in the corporate world that is where you are
sitting”.
3.5.9. Regulation
A few participants, including interviewee 12, seem to be aware that although the data market is not
currently regulated in South Africa, it may become regulated in the future. As a consulting company this
awareness is greater as it may affect the business and cause challenges. This was mainly discussed by
interviewee 2 who mentioned the implication for then needing to register and the restrictions it may place on
what data they can acquire. This participant believes this may hinder big data progression and Interviewee 1
agreed when they said that privacy and regulation may inhibit the freedom an organisation has to explore
within big data and big data implementation. Interviewee 9 stated that they view regulation as a challenge to
embrace and opportunity to find solutions so although they believe it is a challenge, they do not feel it will
hinder their big data ability but rather provide them with an opportunity to discover a competitive advantage.
3.5.10. Management
Management of the big data process from beginning to end is important and if not done correctly can
cause problems. Interviewee 2 mentioned how easily data scientists can “run-away” with big data and so it
needs real management to rein it in and focus on what is necessary for business objectives. This participant
mentioned how big data requires constraint and focus because it is too easy to collect too much data and spend
too much time on models and datasets. This can decrease profitability of the big data project. Management of
big data inputs and outputs is required as well as effort required and the level of output desired. Interviewee 7
mentioned the challenge of needing to manage expectations in terms of what big data can do and used an
anecdote; “No, you can’t have that because Star Trek computers don’t exist yet”. Interviewee 9 mentioned the
managing of structures, processes and speed of decision making.
3.5.11. Data
One of the data challenges was mentioned by interviewee 2 who stated that data is dynamic and
changes continuously. It is difficult to keep updated. This participant used some examples to demonstrate this
point; “People go from being fitness freaks to being super unhealthy to being fitness freaks. People go from
wanting a BMW to buying a BMW to getting into debt selling it and buying a Polo and end up financing a car
again”.
Initially, establishing a basic database can be a challenge and data required for big data can be
expensive to procure through buying or researching. Interviewee 2 also mentioned data misconception as a
challenge; “I think people have a misconception around their data. I think people believe that they have all got
the best data. The only way you can have the best data is to buy the updates from someone who has done the
updates over and over again”. Data is also a challenge in that to remain competitive in the future, it may not
be sufficient to utilise only one data set. Interviewee 2 stated, “You can’t just use one set”. Interviewee 2 and
9 mentioned the challenge of ensuring the data is structured and ordered correctly for the best results.
Interviewee 2 also mentioned the fact that there is still missing and incomplete data where not all data is readily
42
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
available or directly capturable. Finding ways to find and capture this data can be a challenge.
3.5.12. Awareness and Involvement
There is a big data challenge that exists to create awareness and drive involvement from more
employees in all business units or divisions in organisations. Interviewee 1 stated that big data implementation
is predominantly driven by data scientists and business problem owners. Interviewee 7 said a challenge was
the general understanding of big data throughout the organisation because it varies; “Across the organisation,
depending on some divisions we deal with, there is massive buy-in and understanding; others are more focused
on what they are doing”. This participant also said; “a lot of areas of the organisation are benefiting from it,
they are just probably not aware of what is going on behind the scenes”, which shows that data is provided
without the knowledge of where t originates and this creates an awareness and disconnect challenge.
Interviewee 9 revealed that their organisation is similar in that some business units do understand and some do
not understand big data and its application and value; “So to a certain extent yes, but also to a certain extent
no. So, they don’t necessarily see the big data link”. This participant also said that those that do not work
directly with the data or have exposure to it, for example their call centre, do not necessarily follow or
understand big data.
Communication around big data is required to further awareness and interviewee 1 stated that the
organisation is trying to get horizontal as well as vertical communication correct for big data.
3.5.13. Ethics
There is an ethics challenge when it comes to what data to collect and what data to buy and sell.
Interviewee 2 revealed that views on what is ethical are not consistent and many organisations are afraid of
getting into trouble if they are deemed unethical with data collection and usage; “some people disagree with
some things and call it unethical. There is always ethics involved in data”.
Interviewee 8 said; “People keep on bombarding the same subject looking for the same information
because the first company or person that did this information from the same subject, is selling it, or I am made
to pay a price to get it, so let me go back to the same person and try to get it myself. And by the time you do
that you are not getting a willing participant.”. The challenge is trying to all collect the same data and
bombarding customers for data, who then get irritated and do not want to share their data. Clear ethics around
sharing and selling of data could solve this issue.
3.5.14. Research and Development
Conducting research and development within the big data environment is a challenge. Big data
personnel are generally too busy doing tasks to prepare for big data projects and completing the projects
approved to solve a presented business problem, that there is no room left for exploration in order to become
more innovative with big data analytics. Interviewee 1 states that exploration around big data is mostly task
related and part of job descriptions. It is not necessarily exploring new big data avenues. Interviewee 7 has sad
that a challenge is finding time for big data planning and new big data implementation over and above the
running of the big data platform; “The other challenge is we still have to run the organisation and do all of the
existing stuff and there is a lot of work that people are doing, so the challenge is to do all of this and do what
we are currently doing and prepare for the future”.
3.6.
Conclusion
The results of the interview questions were presented in this section per research question. Themes
and constructs found from the coding process were presented and discussed under each research question. The
situation was established in the first research question, followed by main success factors of big data in research
question 2 and lastly challenges presented in research question 3. The next section will discuss these findings
in the context of the literature presented in section 2 and a proposed framework for success factors of big data
will be presented based on these findings.
4.
Discussion of Results
The themes are compared and contrasted to those identified from literature thereby contributing to the
body of knowledge of big data success factors and challenges with regards to achieving organisational
performance.
4.1.
Discussion of Results for Research Question 1
Research Question 1: What is the general understanding of big data and its current situation?
Research question 1 aimed to identify the perceived understanding and current view of big data. This
43
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
included the understanding of what big data means, what it adds to the organisation and how companies go
about utilising big data. It was also established what organisations view as the main uses and application of
big data. It was important to understand this context in order to then know what success factors (research
question 2) are required to achieve utilising big data successfully in this way and what challenges (research
question 3) are inhibiting organisations from utilising big data successfully.
4.1.1. Big Data Understanding and Applicable View
Understanding
Participants views varied on what big data means, with some participants taking a more academic
definition while others a more practical definition. This is consistent with literature where no consensus on a
single definition for big data exists either (Sheng et al., 2017). Participants in leadership roles tended to look
at big data in terms of what it meant for the business while more technical interviewees saw big data more in
terms of the data and technology itself. Many participants, when asked about their understanding of big data
would go straight into tasks and uses of big data rather than talk through a holistic understanding of the concept,
showing that many people only understand big data from the perspective of where they have seen it applied.
Most participants referred to the great volume of data when describing their understanding of big data
but literature tells us that the word “big”, does not imply that big data is about the size or amount of data only,
but rather the complexity of the data and how the data is captured and managed (Caesarius and Hohenthal,
2018). This fact was not articulated by the majority of participants. A few big data consultants or high-level
business executives expanded on their understanding of big data to show that it is equally about more accurate
information and the ability to build insight from data knowledge for better decision-making. This is consistent
with what George et al. (2014) describe in the literature that big data is its ability to produce knowledge and
generate insights that were not possible with traditional methods. Gupta and George (2016) also mention that
the ability of the data to provide smart insights is more accurate of what distinguishes big data from traditional
data and these few participants therefore showed their better understanding of the big data term.
Only a couple of participants took a more academic view of the definition of big data, showing their
knowledge that big data can be explained in terms of the 5 V’s. Whereas one participant was able to describe
the variety component of big data over and above volume the other participant managed to list but not detail
velocity and variability as well. This is a start to understanding what literature describes as the definition of
big data in terms of a number of V’s. Literature describes big data in terms of three V’s of volume, variety
and velocity (Braganza et al., 2017; Sheng et al., 2017; Erevelles et al., 2016; Gupta and George, 2016) or five
V’s, which adds value and veracity (Sheng et al., 2017; Wamba et al., 2015). The Five V’s of big data definition
is extended to include valence and variability and become the Seven V’s definition, which provides an allencompassing view of big data (Erevelles et al., 2016; Günther et al., 2017; Braganza et al., 2017; Janssen et
al., 2017).
In general, this research aligns with literature (Sheng et al., 2017) in that the understanding of big data
is not well articulated or consistent, with the most focus still being on the volume of data. Since most
participants focused on functional aspects of big data when describing their understanding of the concept and
this research is about the functional application of big data initiatives, Comuzzi and Patel (2016) definition of
big data is still the most relevant to the research findings and states what the few higher-level business
participants and consultants began to describe as their understanding, “the ability to harness information in
novel ways to produce useful insights of goods and services of significant value and to extract new insights or
create new forms of value”.
Importance
The overwhelming majority stated the importance of incorporating big data successfully in order to be
competitive. There was an emphasis that this is for all types of organisations and especially customer focused
businesses. Everyone believed that the environment required the use of big data to not only succeed but to
survive and that organisations not utilising big data now are in trouble in the future. There is also a drive for
companies to make BDAC a core competency and this is believed to be the way forward to ensure competitive
advantage. This finding is in line with the literature which explains that there is an exponential rise in
realisation of the importance of big data due to value creation opportunities the improved data collection,
storage, processing and transportation technologies have allowed and the increased digital dependency arising
from technology developments (Akter et al., 2016; Kabir and Carayannis, 2013; Caesarius and Hohenthal,
2018).
One participant added a new aspect, not seen in the previous literature about the consolidation of
industries occurring due to big data becoming a core competency and this changing competitors from other
44
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
industry companies to top big data companies where disruption could occur. Companies with a BDAC core
competency that collect and utilise big data well are the companies where competition is going to emanate
from in the near future. Being mature in big data competency will be key to compete and survive in this new
dynamic.
4.1.2. Data Capturing
In general, most organisations use their business transactions and internal systems to capture data. The
predominant sentiment was that companies do not utilise all the data that they generate. This is inline with the
literature that also says big data generates more data than organisations are able to use and is characterised by
unprecedented large volumes, high velocity and extensive variety (McAfee, Brynjolfsson, Davenport, Patil,
and Barton, 2012; Caesarius and Hohenthal, 2018; Günther et al., 2017; Erevelles et al., 2016).
Mostly organisations are capturing and using structured data for insights but there does exist a lot of
untapped potential in unstructured data. This finding is again in line with the literature where big data is now
a mainstream activity of an organisation since routine business today generates an immense amount of
structured and unstructured information (Janssen, van der Voort, and Wahyudi, 2017; Erevelles, Fukawa, and
Swayne, 2016; Kabir and Carayannis, 2013). Knowing how to utilise the unstructured data is more limited and
more big data mature organisations are starting to drive towards utilising more unstructured data.
The literature mentions how organisations are also able to capture communication data due to the
general adoption of mobile devices, tablets, online social media activity (e.g. Facebook, Twitter) and use of
instruments, machines and transmissions (Kabir and Carayannis, 2013; EY, 2014; Wamba, Akter, Edwards,
Chopin, and Gnanzou, 2015; Liu, Wang, and Lin, 2017; Braganza, Brooks, Nepelski, Ali, and Moro, 2017).
This was not a finding within this research as most organisations had not been able to utilise this ability but
were looking to make use of communication data for sentiment analysis in the near future.
4.1.3. Politics
Organisations that are immature in big data implementation feel that more politics may arise as big
data becomes more prevalent but more mature big data organisations report the opposite in that politics has
either been resolved or the strong leadership drive leaves no room for politics to interfere. Politics around
legacy systems exist due to people not wanting to change their current ways of working or risk losing power
that they may currently possess. Politics can also exist around which new system to adopt and current or
fostered relationships with technology providers.
4.1.4. Process
Most organisations initiate big data projects by an identified business problem but a more mature
organisation will all look at business needs, objectives or targets of business units to trigger a big data project.
After a project is identified a business case is required to determine if the project has value and the projects are
then prioritised for analysis and implementation. More mature organisations are beginning to find methods to
reduce the burden and bottlenecks at the data scientists and data teams by encouraging and enabling methods
for business units to own their own big data projects and complete the analysis through big data platforms
themselves.
Another initiator of a big data project is innovation, which this study found to be rare amoungst
organisations. This is also not found in literature as literature focuses more on big data providing insights that
can be used to innovate (Kabir and Carayannis, 2013; LaValle et al., 2011; Sheng et al., 2017) and not on
innovation in the form of exploring, capturing and utilising the data differently or in innovative ways to find
better procedures or new uses for big data that have not been seen before.
4.1.5. Value
Value is seen in terms of the business value that data can bring to a company. Most companies have
struggled to derive value from big data while a few have managed to delivered plenty of value within their
organisation and continue to do so. The majority of the companies are in line with literature which states that
organisations find it difficult to sustainably use BDAC to achieve value and improve organisational
performance (Comuzzi and Patel, 2016; Elgendy and Elragal, 2016; LaValle et al., 2011) but the few
companies that are exceptions have managed to achieve sustainable value and improved performance from
their BDAC and these companies continue to grow and improve their BDAC because more value generated
allows them more freedom, trust and resource allocation.
There is a common sentiment that big data can do more and provide more value, even in the successful
organisations. This research also highlighted the value of the data itself and that data is truly sought after.
45
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Building data sets and doing data analytics has a positive impact on the economy in two ways; firstly, through
selling data and secondly, by providing businesses with valuable information that increases their efficiency,
thereby positively contributing to the economy. This is a new viewpoint on value that has not been seen in the
literature and needs to be understood due to the increasing awareness of the ability to buy and sell data to
increase the knowledge base from just what is captured from internal systems.
4.1.6. Training
Training for BDAC is mostly self-study and learning by experience. Some organisations enhance
training through online courses, third party training, programs and seminars. Companies that are serious about
developing their BDAC skills have a training budget and resources that are allocated to big data skills
enhancement and development. More mature big data organisations take training to the next level by having
combined sessions where everybody comes together to find big data solutions and can then learn skills from
each other. This is a particularly useful method to utilise for more complex problems that require more complex
solutions and techniques. In general, little is done proactively to facilitate training of big data analytics skills
and this could be a hindrance to the progression of BDAC in organisations. Wang, Kung, and Byrd (2018)
recommend training personnel in big data analytics as a means to success but even the literature is scarce on
talking about training when it comes to big data analytics and no details about what type of training can be
found. This is an area that therefore needs to be addressed and better understood so organisations can be
advised on what types of training will best bring about skills development. A well performing organisation
could be a great example to explore their methods further as they have been successful in developing skills
well enough to sustainably derive value from big data initiatives.
4.1.7. Organisation
Most organisations have not restructured to accommodate the introduction of BDAC. There was no
consensus on whether or not a restructure is likely in the future as BDAC and big data projects increase. It is
found that organisations that had managed to grow in big data maturity had gone through some organisational
restructuring to allow big data to be the centre for decision making. This may provide some evidence that
restructuring to best suit big data as a core competency is important to develop and grow in big data maturity.
This included combining divisions or replacing divisions in order to allow a centralised big data team to form
as well as introducing more big data professionals to each of the business units. It is important for big data
teams to have both an IT and business aspect and for these two divisions to work together closely and begin to
teach and understand each other better for efficient big data analysis. There is no current literature on
restructuring to suit big data capabilities and processes.
4.1.8. Application and Uses
The main applications of big data to surface from this research are that of marketing opportunities,
customer satisfaction, operational efficiencies, innovation, predictions and forecasts, and increasing revenue.
Marketing refers to customer insights, reducing uncertainty, improving accuracy, targeting customers,
understanding customers better, trend analysis, understand markets and directed or intentional marketing.
Customer satisfaction consists of offering the customer exactly what they need and sentiment analysis.
Operational efficiencies include; lean processes, improve response times, compliance, reporting, improve
systems, automate transactions, fraud detection, evaluate KPI’s and HR hiring. Innovation is comprised of
products and services development and improvement, and managing risk. Predictions and forecasts are used
for customer behaviour and increasing revenue is due to unique offerings that can become available.
Most of these applications featured strongly in the data collection process and this research has
demonstrated that there is a strong understanding of what big data can be used for in an organisation. How to
achieve success in each of these applications is less known and the subject of research question two.
There is a vast number of papers in recent literature on big data and its applications, demonstrating the
significance of big data (Chen, Chiang, and Storey, 2012; Chen and Zhang, 2014; Chen et al., 2015; De Mauro,
Greco, and Grimaldi, 2015; Hashem et al., 2015; Wamba et al., 2015; Akter et al., 2016; Elgendy and Elragal,
2016; Erevelles et al., 2016; Ram, Zhang, and Koronios, 2016; Braganza et al., 2017; Gil, Song, Aldana, and
Trujillo, 2017; Lee, 2017; Sheng et al., 2017; Vidgen et al., 2017; Wamba et al., 2017; Saggi and Jain, 2018).
Many of these applications have been mentioned within this literature but there are some uses mentioned that
have not featured strongly within literature and can be attributed to the fact that new uses are found all the time
as technology improves and business units themselves utilise big data to solve their specific problems. Directed
or intentional marketing is an example of one of these uses as it is a fairly new concept that would not be
possible without a large database of knowledge on customers. Fraud detection and HR hiring are also seldom
46
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
mentioned in literature and these are operational efficiencies which come from specific departments within a
company.
4.1.9. Advanced Big Data
The research revealed that the sentiment around what advanced big data is, is still immature in thinking
where the biggest focus was on the advancement of the technology and achieving automation, AI and machine
learning. The term advanced was definitely relative based on the level of maturity and success of big data
within the organisation. Less mature organisations struggled to see the potential for advanced big data that
more mature organisations could. The literature describes organisations maturity and what their capabilities
are at that level of maturity in terms of three maturity levels and at the top maturity level is where they will
conduct the advanced big data analytics (LaValle et al., 2011). The transformed level (top level) is where
organisations now organise their other capabilities to align with the BDAC requirements and use BDAC to
differentiate themselves in the market (LaValle et al., 2011).
A novel view found in this research is viewing advanced big data across the big data analytics process
where big data competency has the ability to exploit data and derive exceptional value from it, big data
implementation has the ability to apply and use advanced big data tools and techniques and big data reporting
has the ability to get insights to the clients or departments in user-friendly ways for them to action. This
viewpoint is important as it draws attention to the importance of the whole big data process from the time data
is collected to the time solutions are implemented and their value and benefit reported for measurement and
feedback. Not completing the whole process well would mean not achieving success in big data projects.
4.1.10. Conclusive findings
The huge importance of having BDAC within an organisation is unanimously understood. There is
generally a very surface level and poor understanding of exactly what big data means but functional
understanding of what big data can be used for is much more comprehensive and organisations know what
tasks can be achieved by utilising big data. The lack of proper understanding of big data may hinder their
ability to properly utilise big data for each of the identified applications of big data and prevent the
organisations from improving their maturity. Organisations generally do not use all the data they capture and
since organisations mostly only use their structured data, there is huge untapped potential for utilisation of the
unstructured data available to organisations. Even though organisations understand what uses can bring value,
actually deriving this value is difficult and many organisations are struggling to show benefit in their big data
initiatives. Training is mostly informal and with no directed methods to properly train competency in big data,
improving could be hindered. Restructuring the organisation to better suit the requirements of the big data
process is important to increase big data maturity. Although a lot is known about the uses of big data, what is
required to make each of the uses successful in improving organisational performance is less explored and
unknown to many organisations. This will be covered in research question two.
4.2.
Discussion of Results for Research Question 2
Research Question 2: What capabilities are required in order to be successful at translating big
data insights into organisational performance?
Research question 1 established the current situation with regards to understanding and uses of big
data for value driving. Research question 2 establishes the perceived capabilities and reasons as to what enables
an organisation to successfully use their BDAC for better organisational performance.
4.2.1. Success Factors of Big Data
The applications of BDAC are well documented in the literature and in section 6.1 above. BDAC has
potential to improve organisational performance and create competitive advantage but there has not been a lot
of success in this plan (LaValle et al., 2011; Wang, Liu, Feng, and Wang, 2014). This research concurs with
that literature sentiment and very few organisations were found to be successful at demonstrating value and
benefit from big data initiatives and as a result were unable to translate big data projects into organisational
performance. The organisations that have managed to sustain organisational performance success from big
data are a great source of understanding as to what they are doing well in order to isolate success factors that
organisations can implements in order to achieve big data success.
The main success factors listed in the literature for big data are; management, people, technology,
culture, strategy, processes, basic resources and governance (McAfee et al., 2012; Davenport and Dyché, 2013;
Wamba et al., 2017; Kiron et al., 2014; Gupta and George, 2016; Wang, Kung, and Byrd, 2018; LaValle et al.,
2011; George et al., 2014; Sheng et al., 2017; Braganza et al., 2017).
47
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Literature refers to the importance of culture (Kiron et al., 2014); Gupta and George, 2016; Wang,
Kung, and Byrd, 2018), but culture is broad and it does not sufficiently expand on what is important about
culture and what aspects of a culture will lead to success in BDAC. The analysis in this report talks to more
specific components that can be a part of a company’s culture, such as the openness and awareness of the
organisation. Openness does seem to be a first step toward starting with big data and not enough to increase
maturity in big data alone. Even organisations just starting with big data mentioned how their companies and
the employees were open to big data and aware of big data. The companies have all bought into the notion of
big data and its potential value and there is positive sentiment about building BDAC to improve organisational
performance. Participants from companies that mainly focused on openness to big data as what is happening
and required for big data are the less mature organisations where big data has either not been successful or
they company has just started to utilise big data. Openness within a company culture and in the minds of all
the business units and employees therefore seems to be an order qualifier for developing BDAC and something
that is predominantly existent in most of the organisations from which a participant was interviewed.
The next success factor, which organisations require in order to take organisations from beginner
maturity or big data out of just the “hype” mentality is commitment. Commitment in terms of actions that show
the organisation is serious about making big data work. These are things such as restructuring to form a
centralised BDAC team as well as deploying big data skills throughout business units. The literature mentions
the importance of skills in BDAC (McAfee et al., 2012; Davenport and Dyché, 2013; Kiron et al., 2014;
Wamba et al., 2017; Gupta and George, 2016; Wang, Kung, and Byrd, 2018) but never in conjunction with
the deployment of such skills in a manner that shows commitment and utilising this to drive behaviour within
business units to better utilising big data for performance. Commitment is a new success factor that has been
derived from this research and has not been tested as a success factor in the literature before.
Organisational success factor constructs determined in this research are strategy, leadership,
commitment, monitoring, measurement and incentivising, human resources and big data champions. This
category covers a few of the mentioned success factors from literature and therefore mostly aligns to what
literature has found to be success factors. The new constructs not found in literature are commitment (as
discussed above), monitoring, measurement and incentivising and big data champions. These constructs are
important for success and this research has demonstrated that in order for organisations to mature and improve
their big data competency, they are necessary and without them, organisations tend to be behind or fail with
regards to big data implementation.
This research found that organisations with clear big data strategy and leadership support, are more
successful when it comes to big data. Companies need a strategy and vision for big data and big data
implementation. For big data to be successful it needs to form part of the overall, long-term strategy as well as
the digital strategy. Organisations need to make decisions based on data whenever possible and ensure there is
a clear goal and vision within the organisation to help define what big data value and value generation looks
like. Part of a strategy process is benchmarking and companies need to do this against top data companies and
internationally when it comes to BDAC. This is consistent with the literature on strategy being important for
BDAC.
Leadership support is extremely important because if big data is seen as a vision of leadership, the rest
of the organisation will make it a priority to implement. This is consistent with literature (McAfee et al., 2012;
Davenport and Dyché, 2013; Wamba et al., 2017) but literature lists more managerial tasks rather than
leadership skill and it is leadership skill that is more important and therefore this research is refining and
improving on this construct from literature. It is important that leadership takes place on each big data project
itself where someone takes ownership of the project and holds the accountability for its success. This is also
an aspect of leadership not mentioned in literature. Big data champions also help leadership to drive big data
and ensure an awareness of the big data existence and capability. The champions help to further implement
big data and disseminate big data knowledge within business units.
Measurement and incentivisation of big data helps to keep track of big data and its progress. This is
essential to properly manage big data and is not a construct that has been mentioned in the BDAC success
factor literature. Skilled individuals with the correct aptitude will provide organisations with an immediate
competency advantage. Introducing experienced big data people to the organisation increases the chances of
finding value from big data and implementing big data successfully. This research on skills is in line with
literature (Gupta and George, 2016; Wang, Kung, and Byrd, 2018) and concurs with literatures findings that
this is a success factor of big data.
Companies that successfully implement big data are able to derive value from big data, so that the
benefit of big data exceeds the cost. To do so, they build on previous analytics experience, proper project
management experience, and may involve outside experts. Demonstrating value will create by-in and this is
48
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
important for involving everyone in the organisation to achieve the openness and awareness needed to begin
with BDAC. The more every business unit and person in the organisation is aware of big data, the greater its
application, and the more worthwhile projects are suggested. It also increases buy-in, if people can practically
see the benefit from big data. In this regard, sending big data champions throughout the organisation to identify
common and everyday problems big data can help with, also has good success.
Development and competency building are important to create success with BDAC. There is a second
level of success activation, when the organisation can identify ways to advance its big data competency. It
unlocks more advanced levels of value generation. Cultivating data competencies for big data is important for
accuracy of analysis and therefore outcomes. Implementation competency is crucial to big data success because
insights mean nothing if the organisation cannot act upon them. Existing data analytics experience is important
because it helps tremendously to be able to derive value from big data projects and to identify where and how
big data can offer value. It forms a starting point from which to understand the power of data. Smaller steps
are taken towards using big data for problem solving by starting small, proving value and then scaling-up the
big data operation.
4.2.2. Big Data Success Factor Model Development
The literature also breaks down BDAC into three dimensions to provide competitive advantage; big
data analytics management capability (BDAMAC), big data analytics infrastructure capability and big data
analytics talent capability (Wamba et al., 2017; Akter et al., 2016; Davenport and Dyché, 2013; McAfee et al.,
2012). Niland (2017) uses these three dimensions and fourteen constructs in a model that he tests quantitatively
to determine the variance in organisational performance and he finds 64% of the variance is attributed to these
constructs (see Figure 2 below). This means that the constructs that make up the success factors of big data
have not yet been fully determined and the constructs found in this research can be used to find constructs to
be tested quantitatively in future research (see # below). It is hopeful that these constructs will account for
more of the variance in organisational performance and add to the literature models to build the comprehensive
model with which to take the research into success factors of big data forward instead of current trends which
is many different researchers testing different models and the research progress is not driving forward.
Figure 2. Model of BDAC versus Organisational Performance from Literature
Source: Niland, 2017
49
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
Figure 3. Constructs and Themes of Success Factors of Big Data Found in this Research
4.3.
Discussion of Results for Research Question 3
Research Question 3: What are the biggest challenges/risks to converting big data insights into
organisational performance?
The last section explores inhibitors of big data performance and establishes the main challenges and
risks to a company’s big data performance. The research question also attempted to confirm the challenges
found within the current literature, as identified in section 2, and see if there were different challenges that
emerged within a South African context across the five industries sampled.
4.3.1. Big Data Challenges
Literature identified the following challenges; Leadership, talent management, technology, decisionmaking, company culture, strategic positioning, infrastructure, data access, skills, decision-making, data
privacy and regulation, processes, lack of understanding of maturity and implementation complexity (McAfee
et al., 2012; Comuzzi and Patel, 2016; Akter et al., 2016; Wamba et al., 2017; Frizzo-Barker, Chow-White,
Mozafari, and Ha, 2016; LaValle et al., 2011; Gupta and George, 2016).
Challenges that were identified in this research which are new and add to the literature are; proof of
relevance and value, awareness and involvement, politics and sentiment, regulation, accuracy of big data
outputs, ethics, research and development, and legacy systems.
Skills and resources: Big data skills are scarce and a lot of training and expertise building must still
occur. The availability of skills to hire, the development of skills as well as the cost of skills and diversity of
skills required to make a success of big data are factors making skills and resources a big data challenge. Skills
scarcity, training and experience as well as retention are problematic with big data human resources. This is
consistent with the literature that lists skills as a big data challenge on many occasions.
Proof of relevance and value: A number of organisations struggle to get off the ground, because they
50
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
are unable to find proper application and use for big data, or they struggle to successfully implement big data.
They make significant big data investments, with little to eventually show for it. They hear of the benefits of
big data, but are unable to realise them. This is taken to be a factor of improper big data project management,
lack of big data competency and skills (experienced BD people), and lack of data experience. In this regard,
external big data consultants can have good results, provided that big data and the outcomes are properly
managed.
Maturity and competency development: In a number of organisations, also more successful ones,
big data is simply still immature, with limited competency. Competency must still be developed. A challenge
with big data is that an immense amount of competency building needs to take place in order to develop and
mature with regards to big data implementation success. Immaturity often brings immature processes which
can make implementing on insights more difficult. This concurs with the literature which states the difficulty
of moving between levels of maturity (LaValle et al., 2011).
Politics and sentiment: Business units may hold back data. Fear may cause people to not want to
participate when it comes to big data and its implementation. Some people may have a negative sentiment
towards big data because they do not want to change their way of doing business or risk losing the power they
have. More politics challenges are seen around sharing of data between business units or departments within
organisations Another big data challenge is the privacy and security of information.
Sophistication and complexity: Big data is simply sophisticated and complex. It is by nature not easy
to implement, and can get very complex. Literature also list complexity of implementation as a big data
challenge.
Tasks, routines, processes: Big data involves a lot of tasks, routines and processes that must be
optimised, and perhaps automated, to free up scarce resources. Big data tasks that need to be performed in the
background in order to ensure accuracy of results are time consuming and this is challenging when the business
wants results quickly. Literature also mentions bad processes as a challenge to big data.
Accuracy of big data results: Big data generally has a high success or accuracy rate (80%), but it is
still not perfect. It may also happen from time to time that the results are skewed, and it takes a significant
while to pick it up. A lack of understanding of accuracy limitations can then cause business to distrust big data
in the future.
Legacy systems: A number of organisations have legacy systems, that make the transition to big data
systems and platforms difficult.
Regulation: Big data faces or may face regulation, that can impact its application and use, and that
can constrain it.
Management: Big data requires proper management. It involves the business context, as much as the
technical aspect of big data. Big data requires constraint and focus because it is too easy to collect too much
data and spend too much time on models and datasets. This can decrease profitability of the big data project.
Data: Issues around data include the availability of data in some cases. Data is dynamic, and changes
all the time. An initial basic database must be developed first. It may be necessary to triangulate and integrate
a number of databases.
Awareness and involvement: It frequently occurs that big data awareness and basic knowledge are
lacking throughout organisations, with very little basic training on big data and big data fundamentals,
principles, and utility for ordinary employees. It means the majority within organisations are left in the dark,
and cannot actively contribute and make use of big data. Communication around big data is required to further
awareness.
Ethics: Big data typically has ethical concerns associated with it. There is an ethics challenge when it
comes to what data to collect and what data to buy and sell.
Research and development: There is too little time, and by extension resources, available to advance
and further big data implementation and competency. This also restricts big data development and growth.
Too much time is spent on merely running platforms and too little time is spent on advancing big data thinking,
competency, and solutions.
5.
Conclusion
5.1.
Principle Findings
The huge importance of having BDAC within an organisation is unanimously understood. There is
generally a very surface level and poor understanding of exactly what big data means but functional
understanding of what big data can be used for is much more comprehensive and organisations know what
tasks can be achieved by utilising big data. The lack of proper understanding of big data may hinder their
51
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
ability to properly utilise big data for each of the identified applications of big data and prevent the
organisations from improving their maturity. Organisations generally do not use all the data they capture and
since organisations mostly only use their structured data, there is huge untapped potential for utilisation of the
unstructured data available to organisations. Even though organisations understand what uses can bring value,
actually deriving this value is difficult and many organisations are struggling to show benefit in their big data
initiatives. Training is mostly informal and with no directed methods to properly train competency in big data,
improving could be hindered. Restructuring the organisation to better suit the requirements of the big data
process is important to increase big data maturity. Although a lot is known about the uses of big data, what is
required to make each of the uses successful in improving organisational performance is less explored and
unknown to many organisations.
Big Data still has tremendous potential that is currently untapped, and yet to be exploited. Even more
successful big data organisations attest to this. There is a lot of development behind big data. A number of
organisations have also identified more advanced ways in which big data can unlock value. There is still a lot
of big data competency to be cultivated but this generally takes time and grows as maturity is developed. Big
data also still requires a lot of optimisation and automation of routines, processes and tasks. Automation will
facilitate openness and allow non-technical employees the opportunity to themselves use big data to solve heir
business problems. Most organisations that get big data off the ground, report benefits that far exceed costs
and this demonstrated value drives interest in big data initiatives and creates the buy-in for the organisation.
When an organisation is immature, it is important to start small and complete small projects successfully to
demonstrate the value and benefit if big data. This will create the support for more resources in order to grow
into bigger projects and develop big data maturity.
Common use cases of big data include using big data to improve customer offerings and satisfaction
as well as marketing efforts and operational efficiencies. Proper strategy, leadership, commitment, champions
and project management (implementation), are some of the organisational factors that promote big data success
and the ability to successfully utilise big data to achieve the uses mentioned above in a way that promotes
improved organisational performance. It also helps if the organisation has prior data analytics experience, or
if the organisation can bring in experienced big data people, that are properly supported and managed. Prior
data analytics experience allows people to better understand the data environment as well as leaves a smaller
gap to bridge to the big data platforms. Some organisations have achieved significant buy-in with regards to
big data and these organisations are maturing quickly and reaping the benefits big data affords. These
organisations can still be performing better but skill shortages and competency level are two of the main
reasons preventing the higher level of performance possible in these organisations.
There is still very basic training on big data analytics, big data implementation and utility training that
the ordinary employee lacks. Training needs to be improved and better managed because self-training ensures
that only some employees with a direct interest will take the time to upskill. Skills are a major limitation when
it comes to big data. Skills are scarce and therefore expensive and retention can be difficult when there are
many opportunities arriving for employees. Organisations follow a number of strategies to attempt to counter
the skills problem, including training people themselves, using open source software that has self—study
online training and can be learnt on the job and it is important to remove the bottleneck on the scarce skill
employees by increasing the ability of the business unit employees to utilise the big data platforms themselves
and solve their own business problems using big data. This will only be possible if there is data and system
enablement where the platforms are made to be user-friendly for non-technical employees and business units.
Organisations typically fail when it comes to big data implementation, because they are unable to
identify how big data can deliver value to the organisation or because they have not developed the
implementation skills. Many organisations are learning how to get insights from big data but converting these
insights into organisational performance through good big data implementation projects is one of the major
competencies lacking within organisations. Focus on improving implementation is important and the success
factors identified under research question two can help organisations to better develop and lead
implementations. Other factors that impact the relative success/ failure of big data include: time, complexity
and sophistication, legacy systems, and proper big data management.
The research achieved the aim of investigating the success factors of big data in order to understand
how and when BDAC yields benefits and improves organisational performance and answered the call for
exploration into the success factors of big data, which was part of the purpose.
5.2.
Implications for Management
The research findings provide an outline of success factors and challenges for management to utilise
when designing and planning around incorporation of big data into their organisation in order to make it a core
52
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
competency. Management can now better understand how to structure themselves, lead the projects, design
processes, capture data and implement with the success factors in mind. This will provide management with a
clearer idea of what the roadmap, planning and change management should be in order to mature a a big data
organisation. Management can now be more confident in their approaches to big data, knowing what is
successful and what challenges they can expect and plan for. The research looked at key management
capabilities which will then directly help management to better lead for big data success. BDAC is a growing
field of interest and is used constantly within the business environment to attain and use knowledge for
organisational performance and competitive advantage. Since organisations are unsure how to capture business
value, research into how to generate value from BDAC is highly desirable and the findings can be directly
applied by management. The success factors developed in this research were more specific and more practical
than most of the current literature and this will help management with implementation.
5.3.
Limitations of the Research
Based on the scope of organisations covered during the semi-structured interviews, the results may
not be generalisable to other industries or geographies. The scope of constructs defining the semi-structured
questionnaire only covered the literature reviewed and this may have missed the effects of other mediators or
moderators of the relationship between BDAC and organisational performance. Integrating more variables
may be an avenue for future research. Respondent bias could have occurred due to their personal opinions and
interpretations, for example, their understanding of what big data actually means as well as their main role in
the company not having a holistic view of the company’s BDAC. The research did not actually measure the
improved organisational performance and therefore cannot prove any causal relationship between BDAC
success factor constructs and improved organisational performance. The environment of data and technology
is constantly changing and developing and therefore this research may only be valid for a specific time period
after the research was conducted.
5.4.
Suggestions for Future Research
Case studies, experiments and longitudinal studies can be conducted to measure the actual
performance improvement of the success factors to test whether each success factor has a significant effect on
organisational improvement as well as the extent to which each success factor contributes to the improved
performance. A quantitative study on a larger number of participants, companies and industries to test if the
success factors are impacting organisational performance and whether this list of success factors is more
comprehensive than the ones currently found in the literature. This will help drive the academic literature
forward in a direction rather than continuing to test newly proposed models and stagnating the literature at this
point. The two best companies with data can be further explored through a case study on each organisation to
see in more detail what they are doing that has put them so far ahead of other companies in South Africa. This
will also bring out more detail around the success factors and challenges and can be the leading South African
benchmark for other companies to learn from and adopt success factors that they are not currently doing to
enhance their BDAC performance. Each of the constructs discussed under each success factor can be further
explored in more detail individually and combined with other fields of study, such as behavioural psychology,
leadership, organisational development and transformation, human behaviour etc. This could give further
detail as to what companies need to do more of specifically to develop each competency to achieve the success
factors and increase their big data maturity and success.
References
Agee, J., 2009. Developing qualitative research questions: a reflective process. International Journal of
Qualitative Studies in Education, 22(4), pp.431–447.
Akter, S., Wamba, S. F., Gunasekaran, A., Dubey, R. and Childe, S. J., 2016. How to improve firm
performance using big data analytics capability and business strategy alignment? International
Journal of Production Economics, 182, pp.113–131.
Bloomberg, L. D. and Volpe, M., 2012. Completing your qualitative dissertation: A road map from beginning
to end. London, UK: Sage Publications.
Braganza, A., Brooks, L., Nepelski, D., Ali, M. and Moro, R., 2017. Resource management in big data
initiatives: Processes and dynamic capabilities. Journal of Business Research, 70, pp.328–337.
Braun, V. and Clarke, V., 2006. Using thematic analysis in psychology. Qualitative Research in Psychology,
3(2), pp.77–101.
Caesarius, L. M. and Hohenthal, J., 2018. Searching for big data: How incumbents explore a possible adoption
53
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
of big data technologies. Scandinavian Journal of Management, 34(2), pp.129–140.
Carson, D., Gilmore, A., Perry, C. and Gronhaug, K., 2001. Qualitative marketing research. London, UK:
Sage Publications.
Comuzzi, M. and Patel, A., 2016. How organisations leverage big data: A maturity model. Industrial Management and
Data Systems, 116(8), pp.1468–1492.
Davenport, T. H. and Dyché, J., 2013. Big data in big companies. International Institute for Analytics, 3
[online] Available at: https://docs.media.bitpipe.com/io_10x/io_102267/item_725049/Big-Data-inBig-Companies.pdf [Accessed on 11 May 2019].
Erevelles, S., Fukawa, N. and Swayne, L., 2016. Big Data consumer analytics and the transformation of
marketing. Journal of Business Research, 69(2), pp.897–904.
EY., 2014. Big Data - Changing the way businesses compete and operate. [online] Available at:
http://www.ey.com/Publication/vwLUAssets/EY__Big_data:_changing_the_way_businesses_operate/%24FILE/EY-Insights-on-GRC-Big-data.pdf
[Accessed on 11 May 2019].
Flick, U., 2014. An introduction to qualitative research. London, UK: Sage Publications.
George, G., Haas, M. R. and Pentland, A., 2014. Big data and management. Academy of Management Journal,
57(2), pp.321–326.
Gil, D., Song, I.-Y., Aldana, J. F. and Trujillo, J., 2017. Big Data. New approaches of modelling and
management. London, UK:Elsevier.
Guba, E. G. and Lincoln, Y. S., 1994. Competing paradigms in qualitative research. In N. K. Denzin and Y. S.
Lincoln (Eds.), Handbook of qualitative research, pp. 105–117. London, UK: Sage Publications.
Günther, W. A., Mehrizi, M. H. R., Huysman, M. and Feldberg, F., 2017. Debating big data: A literature
review on realizing value from big data. The Journal of Strategic Information Systems, 26, pp.191–
209.
Gupta, M. and George, J. F., 2016. Toward the development of a big data analytics capability. Information and
Management, 53(8), pp.1049–1064.
Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A. and Khan, S. U., 2015. The rise of “big data”
on cloud computing: Review and open research issues. Information Systems, 47, pp.98–115.
Janssen, M., van der Voort, H. and Wahyudi, A., 2017. Factors influencing big data decision-making quality.
Journal of Business Research, 70, pp.338–345.
Kabir, N. and Carayannis, E., 2013. Big data, tacit knowledge and organizational competitiveness. Journal of
Intelligence Studies in Business, 3, pp.54–62.
Kelliher, F., 2011. Interpretivism and the pursuit of research legitimisation: an integrated approach to single
case design. Leading Issues in Business Research Methods, 1(2), pp.123–131.
Kiron, D., Prentice, P. K. and Ferguson, R. B., 2014. The analytics mandate. MIT Sloan Management Review,
55(4), pp.1-25.
LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S. and Kruschwitz, N., 2011. Big data, analytics and the
path from insights to value. MIT Sloan Management Review, 52(2), p.21.
Lee, I., 2017. Big data: Dimensions, evolution, impacts, and challenges. Business Horizons, 60(3), pp.293–
303.
Liu, C.-H., Wang, J. S. and Lin, C.-W., 2017. The concepts of big data applied in personal knowledge
management. Journal of Knowledge Management, 21(1), pp.213–230.
McAfee, A., Brynjolfsson, E., Davenport, T. H., Patil, D. J. and Barton, D., 2012. Big data: the management
revolution. Harvard Business Review, 90(10), pp.60–68.
McCracken, G., 1988. The long interview. London, UK: Sage.
Niland, M. J., 2017. Towards the Influence of the Organisation on Big data Analytics. Gordon Institute of
Business Science.
Noblet, J.-P., Simon, E. and Parent, R., 2011. Absorptive capacity: a proposed operationalization. Knowledge
Management Research and Practice, 9(4), pp.367–377.
Patton, M. Q., 2002. Qualitative research and evaluation methods. Thousand Oaks, CA: Sage Publications.
Ram, J., Zhang, C. and Koronios, A., 2016. The implications of big data analytics on business intelligence: A
qualitative study in China. Procedia Computer Science, 87, pp.221–226.
Saggi, M. K. and Jain, S., 2018. A survey towards an integration of big data analytics to big insights for valuecreation. Information Processing and Management, 54 (5), pp.758-790.
Saunders, M., Lewis, P. and Thornhill, A., 2009. Research methods for business students (5th ed.) London,
UK: Pearson Education.
Saunders, M. N. K. and Lewis, P., 2012. Doing research in business and management: An essential guide to
54
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
planning your project. London, UK: Pearson Education.
Sheng, J., Amankwah-Amoah, J. and Wang, X., 2017. A multidisciplinary perspective of big data in
management research. International Journal of Production Economics, 191, pp.97–112.
Vidgen, R., 2014. Creating business value from Big Data and business analytics: organizational, managerial
and human resource implications. Citeseer.
Vidgen, R., Shaw, S. and Grant, D. B., 2017. Management challenges in creating value from business analytics.
European Journal of Operational Research, 261(2), pp.626–639.
Wamba, S. F., Akter, S., Edwards, A., Chopin, G. and Gnanzou, D., 2015. How ‘big data’can make big impact:
Findings from a systematic review and a longitudinal case study. International Journal of Production
Economics, 165, pp.234–246.
Wamba, S. F., Gunasekaran, A., Akter, S., Ren, S. J., Dubey, R. and Childe, S. J., 2017. Big data analytics and
firm performance: Effects of dynamic capabilities. Journal of Business Research, 70, pp.356–365.
Wang, Y. and Byrd, T. A., 2017. Business analytics-enabled decision-making effectiveness through
knowledge absorptive capacity in health care. Journal of Knowledge Management, 21(3), pp.517–539.
Wang, Y., Kung, L. and Byrd, T. A., 2018. Big data analytics: Understanding its capabilities and potential
benefits for healthcare organizations. Technological Forecasting and Social Change, 126, pp.3–13.
Wang, Y., Kung, L., Wang, W. Y. C. and Cegielski, C. G., 2018. An integrated big data analytics-enabled
transformation model: Application to health care. Information and Management, 55(1), pp.64–79.
Zikmund, W. G., 2000. Business research methods (6th ed.) Orlando, USA: The Dryden Press, Harcourt
College Publishers.
Zikmund, W. G., 2003. Sample designs and sampling procedures. Business Research Methods, 7, pp.368–400.
Zikmund, W. G., Babin, B. J., Carr, J. C. and Griffin, M., 2013. Business research methods (7th ed). Mason,
OH: Thomson/South-Western.
Appendices
Interview Schedule
Culture
1. How does your organisation see and understand big data (BD)? What is BD to your organisation?
2. How well do employees within the organisation understand BD?
3. How does the organisation make business and strategic decisions, and to what extent are decisions based
on data?
4. How much feedback on BD and BD practices does the organisation collect?
5. Who consumes BD – how is BD consumed in the organisation - and how do BD projects initiate, kick-off,
or commence?
6. To what extent has BD caused organisational redesign in your organisation, or to what extent is it necessary
or becoming necessary?
Value
7. How is BD adding value within the organisation now and what about in the future?
8. How and to what extent is BD linked to business problems or business targets in the organisation? Are
industry or business experts – people that truly know the business – adequately brought together with BD
and BD scientists?
9. How innovative and agile are your organisation and its culture, and how does this affect BD and BD
success?
10. To what extent is BD contributing to your organisation innovating – being innovative?
11. How much of BD done by your organisation is exploratory in nature? Is there a bit of freedom to play
around?
12. What are the main uses of BD within the organisation?
13. To what extent do you find that conventional data analytics forms an entry point for BD to get a foot in the
door and to build momentum?
14. Do you think your organisation needs BD to successfully compete? Is BD a core competency of the
organisation, or can BD become a core competency of the organisation?
Core competency
15. What skills do you believe your organisation needs to develop in order to have BD as a competitive
advantage and core competency?
55
Walls, C. and Barnard, B., 2020. Success Factors of Big Data to Achieve Organisational Performance: Qualitative Research.
Expert Journal of Business and Management, 8(1), pp.17-56.
16. How strong is your organisation on BD implementation – following through on BD projects?
17. How well are you able to articulate and visualise BD data and output?
18. How extensive is your data gathering? Do you capture unique sources of data?
19. What would you define as advanced BD in the context of your organisation?
Technology
20. How developed and sophisticated is your BD technology?
21. How open and user-friendly is your BD platform, and to what extent is it disseminated through the
organisation?
22. Do you foresee new BD tools in the future to improve BD?
Challenges
23. What are the typical factors hindering BD and capturing value through BD?
24. Do you think BD is under-utilised and under-performing, or not? Can BD still do or contribute more?
25. Did BD deliver with the projects it was incorporated in?
26. What do people typically complain about when it comes to BD?
27. Are you able to source the skills and competencies you need to make a success out of BD?
28. Do you find the cost of BD to be prohibitive?
56
View publication stats