The Practice of Market Research
A01 The Practice of Market Research 31362.indd 1
01/10/2021 14:20
At Pearson, we have a simple mission: to help people
make more of their lives through learning.
We combine innovative learning technology with trusted
content and educational expertise to provide engaging
and effective learning experiences that serve people
wherever and whenever they are learning.
From classroom to boardroom, our curriculum materials, digital
learning tools and testing programmes help to educate millions
of people worldwide – more than any other private enterprise.
Every day our work helps learning flourish, and
wherever learning flourishes, so do people.
To learn more, please visit us at www.pearson.com/uk
A01 The Practice of Market Research 31362.indd 2
01/10/2021 14:20
The Practice of Market
Research
From Data to Insight
Fifth Edition
Yvonne McGivern
Harlow, England • London • New York • Boston • San Francisco • Toronto • Sydney • Dubai • Singapore • Hong Kong
Tokyo • Seoul • Taipei • New Delhi • Cape Town • São Paulo • Mexico City • Madrid • Amsterdam • Munich • Paris • Milan
A01 The Practice of Market Research 31362.indd 3
01/10/2021 14:20
PEARSON EDUCATION LIMITED
KAO Two
KAO Park
Harlow CM17 9NA
United Kingdom
Tel: +44 (0)1279 623623
Web: www.pearson.com/uk
First published 2003 (print)
Second edition published 2006 (print)
Third edition published 2009 (print)
Fourth edition published 2013 (print and electronic)
Fifth edition published 2022 (print and electronic)
© Yvonne McGivern 2003, 2006, 2009 (print)
© Yvonne McGivern 2013, 2022 (print and electronic)
The right of Yvonne McGivern to be identified as author of this work has been asserted by her in accordance with the Copyright, Designs
and Patents Act 1988.
The print publication is protected by copyright. Prior to any prohibited reproduction, storage in a retrieval system, distribution or transmission in any form or by any means, electronic, mechanical, recording or otherwise, permission should be obtained from the publisher or,
where applicable, a licence permitting restricted copying in the United Kingdom should be obtained from the Copyright Licensing Agency
Ltd, Barnard’s Inn, 86 Fetter Lane, London EC4A 1EN.
The ePublication is protected by copyright and must not be copied, reproduced, transferred, distributed, leased, licensed or publicly performed or used in any way except as specifically permitted in writing by the publishers, as allowed under the terms and conditions under
which it was purchased, or as strictly permitted by applicable copyright law. Any unauthorised distribution or use of this text may be a
direct infringement of the author’s and the publisher’s rights and those responsible may be liable in law accordingly.
Pearson Education is not responsible for the content of third-party internet sites.
ISBN: 978-1-292-33136-2 (print)
978-1-292-33134-8 (PDF)
978-1-292-33137-9 (ePub)
British Library Cataloguing-in-Publication Data
A catalogue record for the print edition is available from the British Library
Library of Congress Cataloging-in-Publication Data
Names: McGivern, Yvonne, author.
Title: The practice of market research : from data to insight / Yvonne
McGivern.
Description: Fifth edition. | Harlow England ; New York : Pearson, 2022. |
Includes bibliographical references and index.
Identifiers: LCCN 2021036889 | ISBN 9781292331362 (paperback) | ISBN
9781292331379 (epub) | ISBN 9781292331348 (pdf)
Subjects: LCSH: Marketing research--Methodology. | Social
sciences--Research--Methodology.
Classification: LCC HF5415.2 .M3827 2022 | DDC 658.8/3--dc23
LC record available at https://lccn.loc.gov/2021036889
10 9 8 7 6 5 4 3 2 1
25 24 23 22 21
Cover designed by Michelle Morgan
Cover image: kingwin/iStock/Getty Images Plus
Print edition typeset in 10/12pt Sabon LT Pro by Straive
Printed in Slovakia by Neografia
NOTE THAT ANY PAGE CROSS REFERENCES REFER TO THE PRINT EDITION
A01 The Practice of Market Research 31362.indd 4
01/10/2021 14:20
Brief contents
Foreword
About MRS
Preface
Guide to content for MRS Advanced Certificate syllabus
learning outcomes
Author’s acknowledgements
xvi
xvii
xviii
xxiii
xxv
Part One Introducing market and social research
1
1
Research, data and insight
2
2
The practice of market and social research
28
3
Types of data and research
58
Part Two Planning and designing research
81
4
Business problem and research problem
82
5
Research design
106
6
Writing a research brief
124
7
Writing a research proposal
140
Part Three Secondary or existing data
165
8
Sources of existing data
166
9
Using and evaluating existing data
196
v
A01 The Practice of Market Research 31362.indd 5
01/10/2021 14:20
Brief contents
Part Four Qualitative research
219
10 Qualitative research methods
220
11 Doing qualitative research
250
12 Analysing data from qualitative research
282
Part Five Quantitative research
313
13 Methods of data collection
314
14 Sampling
350
15 Designing questionnaires
390
16 Understanding data
438
17 Analysing quantitative data
466
18 Data mining and data analytics
500
Part Six Bringing it all together
527
19 Managing and reviewing a project
528
20 Communicating the findings
558
Bibliography
Index
Publisher’s acknowledgements
593
613
631
vi
A01 The Practice of Market Research 31362.indd 6
01/10/2021 14:20
Contents
Foreword
About MRS
Preface
Guide to content for MRS Advanced Certificate syllabus
learning outcomes
Author’s acknowledgements
1
2
xvi
xvii
xviii
xxiii
xxv
Part One Introducing market and social research
1
Research, data and insight
2
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
What is research?
The context and use of research
The value of research
The limitations of research
Chapter summary
Exercises
References
Recommended reading
2
2
2
3
4
8
13
17
24
25
25
27
The practice of market and social research
28
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
The research process
Research skills, roles and tasks
The ethical, legal and regulatory context
Chapter summary
Exercises
References
Recommended reading
28
28
28
29
30
32
40
54
55
55
56
vii
A01 The Practice of Market Research 31362.indd 7
01/10/2021 14:20
Contents
3
4
5
Types of data and research
58
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Types of data and research
The size of the data
The format of the data
The source or producer of the data
The nature of the data
The nature of the research
Continuous or ‘one-off’ data collection
The method of data collection
Chapter summary
Exercises
References
Recommended reading
58
58
58
59
60
60
63
64
67
67
70
73
77
78
78
79
Part Two Planning and designing research
81
Business problem and research problem
82
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Business problem and research problem
The nature of the research enquiry
Formulating research objectives
Identifying ethical and legal issues
Chapter summary
Exercises
References
Recommended reading
82
82
82
83
84
90
97
101
103
103
103
104
Research design
106
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
The process of research design
Types of research design
Chapter summary
106
106
106
107
108
110
121
viii
A01 The Practice of Market Research 31362.indd 8
01/10/2021 14:20
Contents
6
7
8
Exercises
References
Recommended reading
121
122
122
Writing a research brief
124
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Definition of a research brief
Roles in the briefing process
Links between the brief and the proposal
Preparing a written research brief
Choosing a research supplier
Checking the brief
The client–researcher relationship
Chapter summary
Exercises
References
Recommended reading
124
124
124
125
126
126
126
127
133
135
135
138
138
139
139
Writing a research proposal
140
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
The purpose of a research proposal
Links between the brief and the proposal
Questioning the brief
The contents of a research proposal
Evaluating a proposal
What happens next?
Chapter summary
Exercises
References
Recommended reading
140
140
140
141
142
142
142
147
161
163
163
164
164
164
Part Three Secondary or existing data
165
Sources of existing data
166
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
166
166
166
ix
A01 The Practice of Market Research 31362.indd 9
01/10/2021 14:20
ontents
C
9
What you should get from this chapter
Using existing data: the context
Sources of existing data
Data storage and retrieval systems
Chapter summary
Exercise
References
Recommended reading
167
168
170
184
192
193
193
194
Using and evaluating existing data
196
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Using existing data: the rationale
Using existing data: types of projects
Using existing data: the process
Evaluating sources
Evaluating suitability and quality
Evaluating technical issues
Developing an analysis plan
Chapter summary
Exercise
References
Recommended reading
196
196
196
197
198
198
201
205
209
214
215
216
216
216
217
Part Four Qualitative research
219
10 Qualitative research methods
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
What is qualitative research?
Ethnography
Semiotics
Interviews and group discussions
Deliberative methods
Online research communities
Chapter summary
Exercise
References
Recommended reading
220
220
220
220
221
222
226
232
234
242
245
246
246
247
249
x
A01 The Practice of Market Research 31362.indd 10
01/10/2021 14:20
Contents
11 Doing qualitative research
250
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Tasks and skills of the qualitative researcher
Sampling and recruiting participants
Designing the data collection guide
Interviewing and moderating skills
Chapter summary
Exercises
References
Recommended reading
250
250
250
251
252
253
260
272
278
278
279
280
12 Analysing data from qualitative research
282
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
What is qualitative data analysis?
Approaches to analysis
Planning the analysis
Doing the analysis
Computer-aided qualitative data analysis
Automated data analysis
Chapter summary
Exercise
References
Recommended reading
282
282
282
283
284
285
289
294
308
308
309
310
310
312
Part Five Quantitative research
313
13 Methods of data collection
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Research designs in quantitative research
Asking questions and observing
Interviewer-administered and self-completion
Self-completion
314
314
314
314
315
316
316
320
323
xi
A01 The Practice of Market Research 31362.indd 11
01/10/2021 14:20
ontents
C
Telephone interview data collection
Online data collection
Mixing or switching modes of data collection
Observational methods of data collection
End note
Chapter summary
Exercise
References
Recommended reading
14 Sampling
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Sampling units and sampling elements
Developing a sampling plan
Sampling theory
Probability or random sampling methods
Semi-random sampling
Non-probability sampling methods
Samples in online research
Chapter summary
Exercise
References
Recommended reading
15 Designing questionnaires
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
The importance of good design
The questionnaire design process
Question content
Question wording
Question structure
Designing questions on attitudes
Reviewing the questions
Question order
Layout and appearance
Questionnaire length
Checking and piloting the questionnaire
Chapter summary
Exercise
330
333
339
342
346
346
347
347
349
350
350
350
350
351
352
352
361
369
378
379
383
386
388
388
388
390
390
390
391
391
392
397
399
405
414
417
425
427
428
430
430
434
434
xii
A01 The Practice of Market Research 31362.indd 12
01/10/2021 14:20
Contents
References
Recommended reading
435
437
16 Understanding data
438
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Understanding data: questionnaire to dataset
Data processing
Making an analysis plan
Data reduction
Chapter summary
Exercise
References
Recommended reading
17 Analysing quantitative data
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Looking for patterns and relationships
Explanatory analysis
Inferential analysis
Chapter summary
Exercise
References
Recommended reading
18 Data mining and data analytics
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Context and process
Data mining and data analytics
Doing a project
Algorithms and machine learning
Data visualisation
Chapter summary
Exercise
References
Recommended reading
438
438
438
439
440
443
448
449
462
463
463
463
466
466
466
466
467
468
480
491
497
497
498
498
500
500
500
500
501
502
503
509
513
522
523
524
524
525
xiii
A01 The Practice of Market Research 31362.indd 13
01/10/2021 14:20
ontents
C
Part Six Bringing it all together
527
19 Managing and reviewing a project
528
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Project management
Getting started: making it happen
Organising and briefing fieldwork
Organising and briefing data processing and analysis
Checking and reporting progress
Reviewing and evaluating the quality of the research
Chapter summary
Exercises
References
Recommended reading
20 Communicating the findings
Introduction
Topics covered
Relationship to MRS Advanced Certificate Syllabus
What you should get from this chapter
Communicating the findings
Managing insight
Communicating findings from a project
Presentations and reports
Writing a report
Presenting data in diagrams, charts and tables
Chapter summary
Exercises
References
Recommended reading
528
528
528
529
530
534
535
546
550
552
554
555
555
556
558
558
558
558
559
560
560
562
564
572
580
590
591
591
592
xiv
A01 The Practice of Market Research 31362.indd 14
01/10/2021 14:20
Contents
Bibliography
Index
Publisher’s acknowledgements
593
613
631
Supporting resources
Visit go.pearson.com/uk/he/resources to find valuable online resources
For instructors
■ An instructor’s manual that includes a list of Industry Insights and boxes,
suggested solutions to questions and exercises in the book, and teaching
notes.
■ A set of PowerPoint slides from the book.
For more information please contact your local Pearson Education sales representative or visit go.pearson.com/uk/he/resources
xv
A01 The Practice of Market Research 31362.indd 15
01/10/2021 14:20
Foreword
Events worldwide, especially those as shattering as the global Covid pandemic, have
thrown the need for understanding people and their behaviours into the spotlight.
Decision makers are questioning their accepted beliefs and are increasingly risk averse
in taking decisions.
Research in this latest period of turmoil has become critical to the C-suite and the
government as well as charities and other stakeholders. Clients are seeking to test
assumptions and choices in a radically different environment.
In this context the need for assurance of the research which is increasingly underpinning difficult decisions is even more vital. Research is evidence that is relied upon,
so it has to be reliable. The stakes are increasingly high, with individual reputations
and the reputation of the sector at stake.
Our clients and suppliers are demanding data competence with the ability to distil
complex merges of data sources, both continuous programmes and ad hoc projects,
into a single motivating narrative. They are asking for agility and creativity. Above
all they are seeking trustworthiness.
The British painter, LS Lowry famously painted ‘matchstick men’. Some have said
this was easy. However, his simplicity was overwhelmingly based on a very sound
technical expertise in drawing, which he studied at art college. Great simplicity comes
from sound understanding.
If trust is important, then training and qualifications are the foundations on which
trust is based. These need to be rigorous, relevant and contemporary, fit for helping
people sort out the fast-changing challenges of the twenty-first century.
Around the world, people are increasingly taking MRS qualifications in recognition of these issues. MRS has been delivering core and advanced qualifications for
over 40 years and is the global leader in this area. As our world internationalises, the
importance of globally recognisable and transferable accreditation and qualification
grows. There are researchers in over 40 countries who have studied for an MRS qualification, and many of the world’s largest global research suppliers have incorporated
MRS qualifications into their graduate professional development programmes.
The MRS Advanced Certificate in Market and Social Research Practice qualification, developed by MRS, draws on best practice across the profession, and is designed
for those who have just entered or are seeking to enter the research profession, working in the areas of commercial market and/or social research. This fifth edition of
The Practice of Market Research: From Data to Insight is most timely; it has been
comprehensively updated to reflect the changes in the MRS Advanced Certificate syllabus providing candidates with the ideal foundation as they undertake their studies.
Best wishes for your journey in becoming a qualified, professional Researcher.
Jane Frost, CBE
CEO
The Market Research Society (MRS)
xvi
A01 The Practice of Market Research 31362.indd 16
01/10/2021 14:20
About MRS
The Market Research Society (MRS) was established in 1946 and represents p
­ roviders
and users of market, social, and opinion research, insight, data analytics and business
intelligence.
With over 600 Company Partners, MRS is the world’s largest association for
research organisations. MRS has individual members in over 60 countries and has a
diverse membership of individual researchers within agencies, independent consultancies, client-side organisations, the public sector and the academic community.
MRS promotes, develops, supports and regulates standards and innovation in
areas from market and social research to data analytics on an evidence-based consultancy. MRS regulates research ethics and standards via its Code of Conduct. All
individual MRS members, Company Partners and International Affiliates agree to
self-regulatory compliance with the MRS Code of Conduct and commit to the highest
research and data standards.
MRS is the leading global supplier for research professional development offering
comprehensive research training, qualifications and CPD to professionals all around
the world. It is also the only awarding body in the UK for vocational qualifications
in market and social research.
To find out more about MRS activities visit the MRS website: www.mrs.org.uk.
xvii
A01 The Practice of Market Research 31362.indd 17
01/10/2021 14:20
Preface
The aim of this book
This book provides a comprehensive, straightforward account of the practice of
market and social research that is both easy to read and easy to understand.
Who should use this book?
This book provides a thorough introduction to the practice of market and social
research. It is suitable for undergraduates on research methods or research skills
courses, and is suitable for undergraduates and postgraduates on courses where there
is a requirement to complete a research project or dissertation. In addition, research
practitioners will find it useful as a reference text and source of information and ideas
on both method and practice.
The book was also designed with the MRS Advanced Certificate in Market and
Social Research Practice in mind. This is a vocational degree-level qualification that
follows the research process from problem definition to reporting the findings. It
aims to help candidates to develop a wide range of research skills. The book covers
the syllabus for this qualification.
New to this edition
In preparing this new edition changes have been made to the structure and content
to reflect changes in practice and methods and in the ethical, legal and regulatory
context in which research operates. The changes are also in response to changes in
the syllabus of the MRS Advanced Certificate in Market and Social Research Practice, and in response to customer feedback. In brief, in terms of structure, there are
now six sections: Introducing Market and Social Research; Planning and Designing
Research; Secondary or Existing Data; Qualitative Research; Quantitative Research;
and Bringing It All Together. In terms of content, all chapters have been updated to
reflect changes in practice and methods and the language used to describe them, and
to provide better explanations of areas that students find difficult. One such area
is that covered in Chapter 4 where there is greater explication of the link between
understanding the business problem and defining the problem to be researched. Other
topics which have significant new content are Chapter 8 on sources of existing data,
which includes an expanded section on databases and data warehouses; Chapter 9
on the evaluation of those sources; and Chapter 18 which covers data mining and
data analytics in much greater detail than the previous edition.
xviii
A01 The Practice of Market Research 31362.indd 18
01/10/2021 14:20
Preface
Revised Chapters 1 and 2 on the practice of market and
social research
These chapters have been updated to reflect changes in what market and social
research involves and the context in which it is operates.
Revised Chapter 3 on types of data and research
This chapter has been revised and updated to include a section on big and small data
and unstructured, semi-structured and structured data.
Revised Chapter 4 on the business problem and the research
problem
This chapter has been revised to include greater emphasis on identifying and understanding the business problem and defining the research problem. It covers material
relevant to designing a project.
Revised Chapter 8 on secondary or existing data
This chapter has been updated to reflect changes in the sources and types of secondary or existing data.
Revised Chapter 9 on using and evaluating secondary or
existing data
This chapter has been updated to cover the use and evaluation of data that were not
originally collected as part of a research project.
Revised Chapters 10, 11 and 12 on qualitative research
These chapters have been revised and updated.
Revised Chapter 13 on quantitative methods of data
collection
This chapter has been revised and updated.
Revised Chapter 15 on designing questions
This chapter has been updated.
xix
A01 The Practice of Market Research 31362.indd 19
01/10/2021 14:20
Preface
Chapter 18 on data mining and data analytics
This is a new chapter that expands the material that was covered as part of a previous chapter.
Chapter 19 on managing and reviewing a project
This chapter has been updated and material on reviewing a project that was previously in the chapter on communicating findings is now here.
Chapter 20 on communicating findings
This chapter has been updated and new material added on insight management.
Ethical and professional practice: the MRS Code of Conduct
Where relevant material on the rules and guidance set out in the MRS Code of Conduct have been updated and new material added.
Industry Insights from real-life research projects
In the previous edition, most chapters contained case studies drawn from real-life
research projects. In this edition these have been replaced by shorter Industry Insights,
although a few of the old case studies remain and are relabelled as Industry Insights.
The aim of the Insights is to show research in action and to illustrate and/or provide
examples of the techniques and practices covered in the chapter. They aim to demonstrate why research was done with an outline of how it was done. They serve to
illustrate the usefulness and value of research, and how it relates to and addresses the
business or decision maker’s problem. Finally, many highlight innovative approaches
to research practice – from design through to dissemination.
Distinctive features
Methods and practice
The book is unusual in that it covers research methods and the practical tasks
involved in planning and running a project. Few other textbooks do this. For this
reason the book is particularly valuable to practitioners as well as students. There is
comprehensive coverage of the following:
Research design
Sampling
● Use of existing data
● Qualitative data collection techniques including ethnography and semiotics
● Quantitative data collection including online methods
●
●
xx
A01 The Practice of Market Research 31362.indd 20
01/10/2021 14:20
Preface
Questionnaire design
Qualitative data analysis
● The basics of quantitative data analysis
● The basics of data analytics and data mining.
●
●
An entire chapter is devoted to the analysis of qualitative data, a topic which few
other market research texts cover in any detail.
In terms of the practical tasks involved in setting up and running a project and
bringing it to completion, there is comprehensive coverage (with examples) of the
following:
how to start an ethical review
how to do a literature review
● how to prepare a brief;
● how to write a proposal;
● how to manage a project – including how to brief interviewers, how to prepare a
coding frame, how to write a data processing specification;
● how to prepare and write a report;
● how to design and give an oral presentation;
● how to evaluate research findings
●
●
Real-life Industry Insights and examples
As noted above, throughout the book there are examples of research in action, most
new to this edition. They cover a wide variety of sectors and topics in market and
social research. On the market research side there are examples of research on wellknown brands from the arts, media, telecommunications, financial services, retailing
and fast moving consumer goods sectors including the BBC, Channel 4, Sky TV, AA,
Asda, Asahi, Chivas Brothers, Centrica, Formula One, GlaxoSmithKline, Gousto,
IBM, innocent smoothies, InterContinental Hotel Group, PayPal, Penguin Random
House, RBS, Royal London, Tesco, Three, TfL, Twinings, Unilever, and Volvo. On
the social research side there are examples of research for government and charities
on complex social issues including the COVID-19 pandemic, anti-social behaviour,
HIV/AIDS, exclusion from school, teenage pregnancy, cancer support and leaving a
legacy to charity.
In addition to the Industry Insights, there are examples of the key documents and
outputs of research. These include examples of a brief; a proposal; project timetables
and costing grids; a sampling summary; discussion and interview guides; questions
and sections of questionnaires; interviewer briefing notes; findings from pilot studies;
a coding frame and list of extractions; an analysis specification; charts and diagrams;
a checklist for preparing a report; and examples of key bits of a report – an abstract
and summary, conclusions and recommendations.
Clear structure
The book is structured largely to follow the research process (and to follow the structure of the MRS Advanced Certificate in Market and Social Research syllabus). The
first section contains three chapters by way of introducing market and social research.
xxi
A01 The Practice of Market Research 31362.indd 21
01/10/2021 14:20
Preface
The first chapter sets out what research is, its value and its limitations. Chapter 2
describes the research process, the roles within it and the ethical and legal framework
within which it operates. Chapter 3 introduces different types of data and research.
The next section is about designing and planning research. The first chapter in this
section, Chapter 4, deals with understanding the business problem and defining the
research problem. Chapter 5 covers research design. Chapter 6 is about how to write
up a research brief; and Chapter 7 is about how to prepare a research proposal. The
next section is about existing data: Chapter 8 covers sources of existing data and
Chapter 9 is about using and evaluating existing data. The next section is on qualitative research and contains three chapters: Chapter 10 on methods; Chapter 11 on
aspects of doing a qualitative project including sampling and designing a fieldwork
guide; and Chapter 12 is about analysing data from a qualitative research project.
The next section deals with quantitative research. It contains six chapters. Chapter 13
focuses on quantitative methods of data collection; Chapter 14 is on sampling; and
Chapter 15 is on designing questionnaires. The next three chapters, Chapters 16 to
18, are devoted to analysis of quantitative data, including so-called ‘big data’. Chapter 19 looks at aspects of managing a project and reviewing the findings. Chapter 20
deals with communicating the findings.
Superb pedagogy to aid learning
Each chapter opens with an Introduction which summarises the aim of the chapter. A
list of Topics covered is then presented. Next, there is Relationship to MRS Advanced
Certificate Syllabus, a useful tool that shows how the material in the chapter relates
to the MRS Advanced Certificate Syllabus.
At the end of the chapter you will find Chapter summaries. These help to reinforce
the main points made in the chapter, and are useful as a revision tool. Exercises at the
end of each chapter are designed to test the reader’s knowledge and understanding.
Each chapter ends with References and Recommended reading which provide more
detail on the topics or issues covered in that chapter. Finally, at the end of the book,
is the Bibliography.
Supplementary resources
A range of support materials, including suggested solutions to the questions and
exercises in this are available for lecturers.
xxii
A01 The Practice of Market Research 31362.indd 22
01/10/2021 14:20
Guide to content for MRS
Advanced Certificate syllabus
learning outcomes
The Research Context
Learning outcome
Source of most relevant
content
Evaluate the usefulness of research to a given
setting
Chapters 1, 2, 3 and 4
Identify and define the problem to be researched
and the associated research objectives within a
given setting
Chapter 4
Identify the information needed to address
defined research objectives
Chapter 4 and Chapter 5
Plan and/or evaluate a research brief for a given
research problem
Chapters 4, 5 and 6
Plan a research proposal for a given research
brief
Chapters 4, 5, 6 and 7
(also Chapters 2 and 3
and ­Chapters 8–15 and
Chapters 16–18)
Develop plans for the appropriate use of
resources (people, time and money) during the
delivery of the research project
Chapter 2 and Chapter 19
Guiding Principles:
Validity and reliability
Apply the concepts of validity and reliability and/
or their qualitative equivalents as appropriate
throughout the research process
­­Chapter 1, Chapters 4 and
5, Chapter 9, Chapters 10
and 12, Chapters 13–15,
Chapters 17 and 18,
Chapters 19 and 20
Ethical principles
Identify relevant ethical principles and apply them
appropriately throughout the research process
Chapters 2, 3 and 4,
Chapters 6–20
Topic 1
Understanding the research context and planning the research
project
Topic 2
xxiii
A01 The Practice of Market Research 31362.indd 23
01/10/2021 14:20
Guide to content for MRS Advanced Certificate syllabus learning outcomes
Topic 3
Selecting the research design and
planning the approach
Identify and evaluate possible research designs
Chapters 3, 4 and 5
Select the most appropriate research design and
justify that selection
Chapters 3, 4 and 5
Evaluate sources of data and select the most
appropriate
Chapters 3 and 4, and
Chapters 8 and 9
Identify and evaluate a range of data collection
methods
Chapter 10 and Chapter 13
Select the most appropriate data collection
method/s and justify that selection
Chapter 10 and Chapter 13
Plan all aspects of the data collection including
fieldwork and the design of the data collection
tools
Chapters 10 and 11,
Chapter 13, Chapter 15
and Chapter 19
Identify and evaluate possible sample sources
Chapter 11 for qualitative
research and Chapter 14 for
quantitative research
Identify and evaluate possible sampling
approaches and techniques
Chapter 11 and Chapter 14
Create a suitable sampling plan and plan its
implementation
Chapter 11, Chapter 14 and
Chapter 19
Identify and evaluate possible suitable
approaches for the analysis and interpretation of
data
Chapter 12, and
Chapters 16–18
Create a suitable analysis plan and plan how to
implement it
Chapter 12, and
Chapters 16–18
Identify and evaluate the usability of research
findings
Chapter 1 and Chapter 19
Identify and evaluate the suitability of different
approaches for the reporting of research findings
Chapters 16, 17 and 20
Select the most suitable approach for the
­reporting of findings and justify that selection
Chapter 20
Topic 4
Selecting an appropriate sample
Topic 5
Analysing and interpreting data
and reporting findings
xxiv
A01 The Practice of Market Research 31362.indd 24
01/10/2021 14:20
Author’s acknowledgements
Many people have been enormously helpful to me while I was writing the original
book and when I was preparing the new edition.
I would like to thank Barry McGivern for his help and his insight throughout.
I would like to thank Paula Devine and Katrina Lloyd of Queen’s University Belfast
and Lizanne Dowds for their help and advice and for the pieces they contributed and
for the use of the Life and Times Survey material. I would also like to thank Geoff
Murphy for his help with data science. Thanks are also due to Kevin McGivern.
I am very grateful indeed to MRS for permission to use extracts from the MRS
Code of Conduct, the International Journal of Market Research and MRS Awards
case studies. I am also very grateful to Impact for permission to use extracts from
the magazine. Thanks are also due to the MRS Professional Development team,
past and present, in particular Samantha Driscoll, Melissa Coghlan, Hayley Jelfs,
Debrah Harding and Karen Adams. I also want to thank my colleagues on the MRS
Advanced Certificate assessment team, Michael de Domenici, Daniel Hardwick,
Andrew Farrell, and Frances Wills, Claire Harris, Cathy Bakewell, Kathy Brown,
Helen Clark, Justin Gutmann and Paul Szwarc.
I would also like to thank the reviewers for this fifth edition. Their suggestions
were much appreciated.
Thanks too to the team at Pearson for their patience and their help: Avijit S­ engupta,
Akanksha Marwah, Agnibesh Das, Nikhil Kumar, Hemalatha Loganathan from
Straive, Antonia Maxwell, Jonathon Price, Eileen Srebernik and Rufus Curnow.
Yvonne McGivern
Summer 2021
xxv
A01 The Practice of Market Research 31362.indd 25
01/10/2021 14:20
A01 The Practice of Market Research 31362.indd 26
01/10/2021 14:20
Part One
Introducing market and
social research
M01 The Practice of Market Research 31362.indd 1
27/09/2021 21:44
Chapter 1
Research, data and insight
Introduction
The aim of this chapter is to give you an overview of research, what it is and why
it is done. It is a practice that involves gathering, analysing and interpreting data
to inform decision making. It is about data at one end and insight at the other.
We look at definitions and uses of research in business and society, its value
and its limitations.
Topics covered
What is research?
● The context and use of research
● The value of research
● The limitations of research.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project; and Topic 2: Guiding Principles: Validity and
reliability.
M01 The Practice of Market Research 31362.indd 2
27/09/2021 21:44
What you should get from this chapter
At the end of this chapter you should be able to:
understand the nature of research; and
● recognise the value and contribution of research and its limitations.
●
3
M01 The Practice of Market Research 31362.indd 3
27/09/2021 21:44
Chapter 1
Research, data and insight
What is research?
Research is about enquiry; it is about systematic observation or investigation to find
things out. It is the process by which we produce evidence or knowledge about the
world. It is founded on scientific methods, which are in turn supported by philosophical principles about the nature of knowledge and how we construct that knowledge.
What is sometimes called ‘pure research’ is research undertaken to gain knowledge
or understanding of something without having in mind a specific application of that
knowledge or understanding. ‘Applied research’ is the term given to research undertaken to gain the knowledge or understanding or insight needed to address a specific
need. In this book we focus on the practice of applied research. Applied research
involves questioning, observing, listening to people, gathering data and analysing and
interpreting gathered and existing data in order to build information and knowledge
that can be used to enable, inform, communicate, persuade, design, develop, improve
and predict. From here on, we’ll refer to applied research simply as research.
From data to insight
It is likely – almost certain – that research has played a role in nearly every aspect of your
daily life: for example, in the design of your phone, the apps on your phone, the services
you avail ourselves of, the things you buy, and in how your country works, its policy
and its laws. An organisation needs to make a decision – a well-informed decision –
about how to invest its limited resources to best effect. A well-informed decision
relies on good quality, relevant, timely information or evidence. That information or
evidence starts out as data. Data are ‘observed or recorded facts’ (Allen, 2017). Data
relevant to the decision are brought together, processed, analysed and interpreted
in the context of the decision facing the organisation. In other words, as a result of
‘work’ done on the data – with an end use or purpose in mind – the data become
information. The work done has brought the data together in ‘actionable assemblies’
(Allen, 2017). These ‘actionable assemblies’ should provide the decision-maker with
the information and the insight needed to make a decision.
Industry Insight 1.1 contains examples of the role played by research, data and
insight in organisations in the commercial sector and in the charity sector.
Industry Insight 1.1
The role of research, data and insight
The commercial sector
Nick Rich is vice president, global market and
consumer insights, at InterContinental Hotel
Group (IHG). It is present in 100 plus countries
and owns more than a dozen hotel brands including Holiday Inn, Kimpton, Hotel Indigo, Crown
Plaza and InterContinental Hotels & Resorts. The
focus of the global insights group is mainly on
strategy and brand. The team mantra is knowledge for everyone. Marketing is the number one
partner but the aim is to focus on the whole business. Rich points out that everyone in IHG needs
to know about the customer. An insight group
that delivers business-critical information is vital
4
M01 The Practice of Market Research 31362.indd 4
27/09/2021 21:44
What is research?
because the company operates in a hugely competitive sector. The insight team has to be switched
on to short-term requirements and performance
and be able to see what future needs will look like.
This is important because it takes time to build
hotels in a new place and refurbish existing ones
if tastes change.
Source: Adapted from Gray, R. (2018) ‘Inside knowledge’, Impact,
21, pp. 56–60. Used with permission.
The charity sector
Oxfam was founded in 1942 to help alleviate the
suffering of civilians caught up in conflict. Today it
is a global federation with 19 affiliates working in
more than 90 countries. It aims to help people hit
by disaster and it campaigns for action to tackle
the root causes of poverty. Tina Trythall is head
of insight at Oxfam GB. ‘The insight function at
Oxfam is about putting the audience and market understanding at the heart of public support,
staff engagement, communications and marketing
strategy. We are ensuring our supporter and public
engagement strategies are based on this audience
understanding. It is about identifying who to target, who the audience is, to achieve our objectives.
What do they know, think and feel about Oxfam?’
‘Know, think, feel and do’ are the four watchwords at the heart of the Oxfam insight strategy.
Source: Adapted from Gray, R. (2018) ‘Insight into charity
campaigning’, Impact, 20, pp. 44–8. Used with permission.
Professional research practice
The practice of research to acquire and analyse data and turn them into insight is a
professional activity. The Market Research Society, MRS (www.mrs.org.uk), is the
UK professional body for research, insight and analytics. It was founded in London
in 1946 with 23 members. At the time of writing it has 5,000 individual members and
over 500 accredited Company Partners in over 50 countries. It estimates that around
64,000 professionals work in the research, insight and analytics sector in the UK in
around 4,000 enterprises, according to Research Live Industry Report (2021), with a
worth of around £7 billion. MRS published its first Code of Standards in 1954. In the
most recent version, now called the Code of Conduct (2019), MRS defines research as:
the collection, use or analysis of information about individuals or organisations
intended to establish facts, acquire knowledge or reach conclusions. It uses techniques
of the applied social, behavioural and data sciences, statistical principles and theory,
to generate insights and support decision making by providers of goods and services,
governments, non-profit organisations and the general public.
This definition – with its references to applied social, behavioural and data sciences – gives you an idea of the underpinnings of professional research practice and
the contexts in which it is used – goods, services, governments, non-profits – and so
an idea of its reach. This is further illustrated in how MRS describes itself: as an association for ‘everyone with professional equity in market, social and opinion research
(whether you use it or provide it) and in business intelligence, market analysis, customer insight and consultancy.’
ESOMAR (www.esomar.org), established in 1947, is another important research
association: ‘a diverse community of 40,000 + data, research and insight professionals
from 130 + countries’. Like MRS, it promotes the value of research and data and offers
ethical and professional guidance to its members. In its International Code on Market,
Opinion and Social Research and Data Analytics (2016), ESOMAR defines research as:
the systematic gathering and interpretation of information about individuals or
organisations.
5
M01 The Practice of Market Research 31362.indd 5
27/09/2021 21:44
Chapter 1
Research, data and insight
So the practice of research is informed by a wide range of disciplines. These include
anthropology, sociology, statistics, economics, geography, linguistics, psychology,
neuroscience and artificial intelligence. Other disciplines contain skills transferable
to work in the research sector including languages, history, policy studies, computer
science and engineering.
You’ll see that both MRS and ESOMAR refer to the use of techniques from data
science, and both organisations make reference to (data) analytics. The application
of data science and analytics in a market and social research context has come about
because of the increasing size, speed and complexity of the data we create and the
availability of the means to store and analyse them. Such data are often referred to
as ‘big data’. It is a term that is now in widespread use. Diebold (2019), who used
it in a conference paper in 2000 on statistics and econometrics, believes it was first
used by John Mashey and others at Silicon Graphics Inc. in the mid-1990s. Diebold
credits Weiss and Indurkhya (1998) as the first significant academic reference (in
computer science) and Douglas Laney at Gartner for ‘enrich[ing] the concept significantly’ in an unpublished research note in 2001. The term is used to describe the
huge quantities of digital data that are continuously generated and ‘seek[ing] to be
exhaustive and fine-grained in scope, and flexible and scalable in their production’
(Kitchin, 2014). boyd and Crawford (2012) see it as ‘in many ways, a poor term’
given that size is ‘not the defining characteristic’. They point out that some of the
data classed as ‘big data’ (they give Twitter messages about a particular topic as
an example) are not as big as datasets that were not thought of as ‘big data’, for
example census data. We look in more detail at definitions and ways of describing
big data in Chapter 3.
If there is ‘big data’, then there must be ‘small data’ (Kitchin, 2014). Kitchin
quotes Miller (2010), describing ‘small data’ as data ‘produced in tightly controlled
ways using sampling techniques that limit their scope, temporality and size’. Kitchin
acknowledges that while small data ‘may be limited in volume and velocity, . . . they
have a long history of development, with established methodologies and modes of
analysis. . . [and] small data studies can be . . . tailored to answer specific research
questions and to explore in detail and in depth . . .’.
The practice of research now encompasses ‘big data’ projects including data mining and analytics as well as the more traditional ‘small data’ studies using qualitative
and/or quantitative methods. The fundamental skills of the research practitioner –
how to do research that produces actionable insight – have not changed, however.
The researcher must be able to understand the problem that the research and the
data are to address; formulate research objectives; design a project that will deliver
the required data; analyse and interpret those data in the context of the problem;
and communicate the findings in a meaningful way in the context of the problem.
As Moon (2015) notes, ‘the knowledge, skills and influence of the researcher are
becoming increasingly important in the quest to draw meaningful insights from the
growing sea of Big Data’.
With the size, complexity and speed of big data there have, of course, been changes
in data storage and access infrastructure, as well as processing and analysis technology, and so in developments in the skills associated with these. Most prominent
among these are skills related to the broad field of data analysis: data capture, storage, access, processing, cleaning, transformation, analysis and display. It is these skills
that the professional bodies allude to when they mention data science and analytics.
6
M01 The Practice of Market Research 31362.indd 6
27/09/2021 21:44
What is research?
These have always been important – indeed essential – research skills. However, as
John Tukey noted in 1962, ‘. . . data analysis . . . must adapt itself to what people
can and need to do with data’. Tukey foresaw that the influences on the development of these skills were ‘accelerating developments in computers and display devices
[and] the challenge, in many fields, of more and ever larger bodies of data [and] the
emphasis on quantification in an ever-wider variety of disciplines’. From the examples in this book you’ll see that the practice of market and social research – which
at its heart is about learning from data – has adapted, and is continuing to do so.
As far back as 2016, Christina Jenkins, Global Research Team Leader within GSO
Insights, LinkedIn, reported in a debate entitled, ‘Who will succeed in the new era of
data discovery’, that among market researchers on LinkedIn’s member database, an
increasing number were adding skills such as coding, Python, R, SQL, and Hadoop,
as well as what Jenkins terms ‘whole-brain thinking’ skills such as business strategy,
strategic planning, leadership and management. A study conducted by Cluley et al.
(2020) shows evidence of changing roles and work practices to accommodate use of
digital data.
Market, opinion and social research and data analytics
Both ESOMAR and MRS refer to market, opinion and social research and, as we
have noted, to data analytics. It is important to know to what these categories refer
in order to understand the context in which research is used.
Market research generally refers to research activity conducted in the business
and commercial world: to explore, discover, test, verify, measure, explain, and/or
understand goods, products, services, organisations, consumers, and relationships
between these. Those working in market research draw on a body of knowledge
found in business studies (including business environment, business practice, business strategy, business process, business communication, business technology) and
marketing (including product design, new product development, branding, packaging, pricing, sales, advertising and promotion, competitive strategy, segmentation
and customer satisfaction). The term ‘market research’, however, is not as common
as it once was. An analysis by Nunan (2016) found that although the term is ‘well
established as an industry definition’ few of the leading research firms use it to
describe their core activity. Instead the descriptions they use include ‘research, data
and insight’, ‘business intelligence’, ‘consumer insight’, ‘market insights’, ‘strategic
insight’ and ‘analytics’, among others. Despite changes in the market research marketplace, researchers ‘are still around and are proud to be called “researchers”’ (Flores,
2016). Cluley et al. (2020) in a study of 44 market research practitioners found that
20 had a job title that included ‘research’ (‘researcher’ or ‘research manager’); other
titles included ‘product manager’, ‘brand executive’, ‘insight director’, ‘narrator’ and
‘project engager’. Raben (2015), quoted by Flores, notes that whether in the business
of ‘consumer guidance’, ‘insights’ or ‘marketing intelligence’, all of these disciplines
are grounded in research.
Opinion research is the application of market research techniques to gather information on opinions about political issues. For opinion researchers, the body of
knowledge they draw on includes political science and psephology. Social research
generally refers to research conducted to explore, discover, test, verify, measure,
7
M01 The Practice of Market Research 31362.indd 7
27/09/2021 21:44
Chapter 1
Research, data and insight
Industry Insight 1.2
What’s in a name?
At WPP, a large marketing services organisation,
what was once called market research or consumer
insight is now described as ‘data investment
management’, according to its CEO at the time, Sir
Martin Sorrell. It has, he notes, ‘. . . always been about
gathering and interpreting information, but the
internet has created a new, ever expanding universe
of data, the sheer volume and complexity of which
demands ever more sophisticated approaches, tools
and techniques . . . Data collection and analysis is
nothing unless it produces insights . . . Unearthing
and communicating valuable insights remains the
core purpose . . . the toolkit the researcher can use
to find insights has expanded . . . ’
Source: Adapted from Sorrell, M. (2016) ‘Digital, data and
globalisation’, Impact, 12, pp. 37–8. Used with permission.
explain and/or understand the nature of social phenomena, of groups, of organisations and of people, of the social world. Social researchers draw on knowledge
from sociology, anthropology, criminology, history, linguistics, psychology and
social policy.
Data analytics is a term used to describe the analysis of raw data. It is a process
that includes preparing, visualising, analysing and modelling raw data to provide
knowledge and insight. You may also see it referred to, depending on the question
asked or the output, as descriptive analytics or predictive analytics. The raw data may
be ‘captured data’, that is data gathered via a process designed to gather data, or it
may be ‘exhaust data’ (Kitchin, 2014), data that are a by-product of some other data
generation or capture process – think of the data generated by our daily interactions
with digital technology.
What distinguishes a market researcher, an opinion researcher, a social researcher
or a researcher using data analytics is the area in which they apply the findings or
insight, and their knowledge and expertise in the domain in which they apply it, the
context rather than their general research skills set. They should have many of the same
core skills. However, each is likely to have some specialist skills related to the type of
research they do, and/or to the type and scale of data they use, besides their knowledge of the domain or substantive area in which they work.
Whatever area you work in, the process – the way in which a research project is
structured and run – will be largely the same. It involves defining the problem to be
researched; designing and planning the research; gathering, generating and/or selecting the data; processing and analysing and interpreting the data; communicating the
findings; and applying what you have learned to the problem under investigation.
The context and use of research
‘Market’, ‘opinion’ and ‘social research’ are broad terms for a rich variety of research
conducted in many different contexts. Market research, for example, contains within
it several fairly distinct categories of research characterised either by the type of participants involved (consumer research or business-to-business research); the subject
8
M01 The Practice of Market Research 31362.indd 8
27/09/2021 21:44
The context and use of research
matter of the research (advertising research or product development research or
audience research or customer satisfaction research); and the location of the research
(international or domestic research). We look at some of these in more detail below,
and there are examples throughout the book.
Consumer research
Consumer research, as its name suggests, is conducted among consumers – individuals and households. The purpose is usually to understand consumer behaviour,
attitudes and opinions in relation to products and services and the marketing activity
that surrounds them. Many of the Industry Insights in this book could be classed as
consumer research projects.
Business-to-business research
The purpose of business-to-business (or B2B) research is usually to understand the
behaviour, attitudes and opinions of those businesses involved in marketing and
selling products and/or services to other businesses. The sorts of populations from
which samples are drawn include those in commercial and retail organisations (e.g.
members of the c-suite of executives – CEOs, Chief Operating Officers, Chief Information Officers, Chief Technology Officers, Chief Financial Officers, and so on – IT
managers, procurement officers, human resource managers); professional practices
(e.g. dentists, lawyers, surveyors); opinion formers and opinion leaders (e.g. politicians, community leaders, journalists, bloggers). Getting access to members of B2B
Industry Insight 1.3
Consumer research
Chivas Brothers, owned by Pernod-Ricard, produces alcoholic drinks. In 2018, it planned to
launch a new single malt Scotch whisky called
Allt-A-Bhaine. Before the launch the brand team
wanted to build confidence in the new offering
among the company’s internal stakeholders and
it wanted to secure a marketing budget for the
launch of the whisky in Australia. To do this they
needed to test the brand concept, profile its potential consumers, identify key occasions for which
those consumers would associate this new single
malt, and get a robust estimate of the whisky’s
potential sales volume in its first three years after
launch. They hired insights agency, Skim, to conduct an online quantitative survey of 1,200 whisky
buyers in Australia. This survey tested the concept,
provided profiles of potential buyers, measured the
impact of awareness of the new single malt on buyers’ choices and examined the best price to charge
for the whisky. The agency gave the client feedback on reaction to the name of the whisky, the
visual appearance of the bottle, the taste and the
pricing. Pernod-Ricard Australia gave the agency
three years of sales data on the whisky’s expected
competitive set. The agency analysed both the survey data and the sales data and were able to predict how many cases of the new whisky would sell
every week for three years after its launch.
Source: Adapted from McQuater, K. (2019) ‘Distilling data’, Impact,
25, p. 15. Used with permission.
9
M01 The Practice of Market Research 31362.indd 9
27/09/2021 21:44
Chapter 1
Research, data and insight
Industry Insight 1.4
B2B research
Epicor is a software company supplying technology that runs the businesses of mid-size organisations. It sells to organisations in a wide range
of sectors – manufacturing, distribution, retail
and services – in various countries. To gather the
information it needs the company runs a global
longitudinal survey with its research supplier,
Morar HPI. The research helps the company
understand what roles people have within target
organisations. Those taking part in the research
might be members of the company board or they
might be users of the software. Director of communications at Epicor, Tabita Seagrave, reports
the survey results to the Chief Marketing Officer.
They are used in strategic planning and marketing
and to direct sales.
Source: Adapted from Phillips, T. (2017) ‘A business mindset’,
Impact, 19, pp. 27–36. Used with permission.
populations can be difficult as they are often ‘protected’ by ‘gatekeepers’, personal
assistants or more junior executives. The techniques used in consumer research are
also used in B2B research. The sample size in B2B research projects tends to be
smaller than those found in consumer or social research.
Advertising research
Advertising is often something organisations spend a great deal of their limited
resources on and so they want to know whether or not that money is well spent.
Thus they use research to help them determine what advertisements will work, and
which will work best, and in what medium; and how effective the advertising is
among the target audience, what view it creates of the organisation or its products
and brands, and what effect the advertising is having on the target market in terms
of its buying behaviour.
Industry Insight 1.5
Advertising research
Sky Media is the advertising sales arm of broadcast
entertainment company Sky TV. Sky serves 22.5
million customers across seven countries including
UK, Ireland, Spain, Germany and Italy. A sizeable
portion of its income is from subscriptions to its
‘linear’ or traditional broadcast TV and digital
services; most of it, however, comes from advertising revenue. Sky Media sells advertising across
Sky’s TV, online and on-the-go platforms. It has
an insight and research department with 12 staff.
The department not only commissions research
but it also uses internal data sources and conducts
research across its suite of products such as Sky
AdVance and Sky AdSmart. Using Sky subscriber
data, Sky AdSmart allows brands of all sizes to
target specific households with bespoke messages
at a fraction of the price of traditional TV.
In 2018, supermarket chain Asda ran a Ramadan
range campaign aimed at South Asians. AdSmart
technology combined analyses of Sky customer
10
M01 The Practice of Market Research 31362.indd 10
27/09/2021 21:44
The context and use of research
data and Experian Mosaic data (geodemographic
data). Sky could reach those customers with a
high propensity to purchase and who lived near
a participating store. With research agency BVABDRC, Sky conducted online interviews with Sky
customers, allowing Asda to assess the advertising
campaign’s impact on those the ads reached compared with a control group who were not targeted.
Source: Adapted from Bold, B. (2019) ‘Turn on, tune in’, Impact,
25, pp. 48–52. Used with permission.
International research
International research (including multi-country research) is organised typically in
one of two ways: the entire project is ‘centrally coordinated’ from one country and
only the data collection is carried out ‘locally’; or each country runs its own project
based on a research plan and a standardised data collection tool with results pooled
on completion.
Marketing process
What the examples above illustrate is that a research project is usually closely tied
to some element of marketing. If you are unfamiliar with the marketing process, you
might find the following paragraphs useful in understanding the terminology and the
sort of planning and decision making that are often involved.
Marketing is about identifying, anticipating and satisfying customer needs in a way
that is profitable for the organisation. Although it may not be formalised, or even
recognised, as marketing in some organisations, the task of identifying, anticipating
and satisfying the needs of the customer exists nevertheless. A marketer’s job is to
seek out (business) opportunities – opportunities that will serve the interests of the
organisation. When an opportunity is discovered, the marketer’s role is to develop
a marketing plan to apply the organisation’s resources to achieving measurable
marketing objectives, and so contribute to the organisation’s goals. Marketing
objectives are statements of what is to be achieved. For example, a marketing
objective might be to launch a new current account into the online banking market
and to achieve a 5 per cent market share within a year, or to launch a new cancer
screening service and achieve an uptake of 80 per cent of the target market.
In order to develop a marketing plan and set marketing objectives, marketers need
a clear understanding of the environment in which they operate. They need to understand the wider external environment that is made up of or influenced by social (and
cultural), legal, economic, political and technological factors (you may have come
across these factors under the acronyms SLEPT or PEST), and the internal environment and resources of the organisation. They need a clear picture of both the opportunities and threats posed by the external environment and also of their organisation’s
strengths and weaknesses. The process of examining the external environment and
the resources of the organisation is referred to as a marketing audit. The analysis of
strengths and weaknesses, opportunities and threats is called a SWOT analysis.
Once a marketing audit and a SWOT analysis have been completed and a business opportunity established and evaluated, a marketing plan can be developed and
11
M01 The Practice of Market Research 31362.indd 11
27/09/2021 21:44
Chapter 1
Research, data and insight
marketing objectives set. To achieve the marketing objectives a marketing strategy
is developed – a plan for achieving the objectives. This plan will involve defining the
marketing mix, which consists of the four Ps: the product (or service) – its design, its
features, its packaging; its price; how it will be promoted – advertising, direct mail,
digital marketing, social media, email marketing, search engine marketing, public
relations and so on; and place – the distribution and sales channels, and the level of
customer service. The marketer’s task is to implement the marketing plan and monitor and evaluate its success in achieving the marketing objectives.
Not only are individual products and services marketed to customers, but the
organisation itself is marketed to its customers and to a wider audience of stakeholders, including employees and shareholders, in the case of private sector organisations,
and taxpayers and voters, among others, in the case of public sector organisations.
Think of all the information needs – the research needs – that this marketing
process involves if it is to be done effectively, if the goal of marketing – to identify,
anticipate and satisfy customer needs in a profitable way – is to be met. Market
research can be used to achieve the following:
understand the wider environment and how it affects the organisation;
identify opportunities and threats;
● identify markets, competitors and customers;
● help with priority setting and direct the use of resources;
● build knowledge for longer-term benefit;
● understand customers and market dynamics;
● monitor customer and stakeholder satisfaction;
● understand how to build and enhance customer relationships;
● monitor and evaluate competitors/competitive activity;
● identify or monitor market changes and trends;
● develop marketing strategies;
● test different marketing strategies;
● monitor and control marketing programmes;
● understand how to influence customer attitudes and behaviour;
● understand how best to communicate with customers and stakeholders;
● develop advertising and communication strategies;
● develop and test advertising executions; and
● develop or select a product or service, a brand name, a pack design, a price point,
a distribution channel.
●
●
Social research
Social research is commissioned for much the same reason as market research – to
obtain information, to understand what is going on in the wider environment, to
understand people’s attitudes, opinions and behaviour – in order to provide data for
effective planning and decision making in relation to policy development and implementation. Plans and decisions have to made about how society operates, how we
deal with ‘social issues’, how we allocate scarce resources, what services should be
provided, how they should be designed, to whom they should be targeted and how
they should be implemented. Plans and decisions about policy and public service provision are nowadays subject to scrutiny and often require justification. They should
12
M01 The Practice of Market Research 31362.indd 12
27/09/2021 21:44
The value of research
therefore be based on robust, defensible evidence; the best way of providing that
evidence is via objective research. In social research, the wider external environment
is society, the attitudes of interest are ‘social’ attitudes, attitudes to ‘social issues’, and
the behaviour of interest is how we live and behave in the ‘social’ world.
Social research is commissioned by government departments, public bodies, public
services, local government, non-governmental organisations, charities, policy studies
groups, the media, think-tanks, academia and research institutes. The topic areas are
many and varied and include health and social care, crime, transport, leisure and
the arts, work and family life, housing, labour force participation, and training and
skills needs. A social research project might be commissioned for many of the same
reasons that a market research project is commissioned – for example to achieve one
or more of the following:
to help with priority setting and to direct the use of resources;
to understand the wider environment;
● to identify or monitor changes and trends;
● to build knowledge for longer-term benefit;
● to develop policies and programmes;
● to monitor or evaluate programme delivery;
● to identify relevant stakeholders;
● to understand the beliefs and values and attitudes of stakeholders;
● to understand how to influence stakeholder attitudes and behaviour;
● to understand how to build and enhance stakeholder relationships;
● to monitor stakeholder satisfaction; and
● to understand how best to communicate with stakeholders.
●
●
The value of research
So organisations, not just those in the private sector but those in the public and notfor-profit sectors, rely on research and data to inform and improve planning and
decision making. In all organisations resources are scarce. For an organisation to
survive and prosper it must use its limited resources wisely. To do this effectively it
must understand the behaviour, wants, needs and opinions of both its customers and
other stakeholders (employees and shareholders, for example, in the case of private
sector organisations, and citizens – taxpayers and voters – in the case of public sector
organisations) and it must understand the environment in which it operates.
This is where the value of research lies: in its ability to provide high-quality data,
information and insight for planning and decision making in often very complex,
fast-moving, decision-making environments in which all sorts of other information
sources – whose quality and value are often harder to assess – vie for attention.
Decisions based on robust and credible research and data should lead to better quality decision making, better use of resources, better products and services, better
policies and better relationships with customers and other stakeholders, increased
customer and stakeholder satisfaction and value, and ultimately greater longevity for
the organisation than if research were not conducted. The type of decision making
an organisation uses is sometimes referred to as data-driven decision making (DDD),
the process of making decisions based on research and data rather than on intuition
alone. (However, as you’ll see below, there is research that shows that not everyone
13
M01 The Practice of Market Research 31362.indd 13
27/09/2021 21:44
Chapter 1
Research, data and insight
uses DDD.) Brynjolfsson et al. (2011), in a study of 179 publicly traded companies,
found that DDD ‘is associated with a 5–6% increase in their output and productivity,
beyond what can be explained by traditional inputs and IT usage’.
It is not surprising then that research is a widespread, worldwide activity. According to ESOMAR in 2019, the data, research and insights sector was worth around
£70 billion and employed over 135,000 people. You only have to look through the
Research Buyers’ Guide (www.rbg.org.uk) to see the current scope of organisations
and services available in the UK and Ireland alone.
Research, and the data and insight it provides, influences decision making, influences
what is provided and the way in which it is provided. By doing research, gathering and
analysing data and turning it into insight, an organisation learns about those who use
or buy its products or services, or who are affected by its policies. It gives those people
a voice, a role, a degree of influence. By doing research and analysing data, and thus
gaining knowledge, understanding and insight, an organisation can make better use of
its limited resources to improve its performance. As the MRS observes:
Research, insight and analytics stand at the heart of all well-informed commercial,
social and political decisions. Insight into what makes a product, business initiative
or government policy work is often the hidden – yet defining – factor between success
and failure.
Kollman and Curry (2019) define the sum of what an organisation knows about
its markets, its performance, its customers and the wider context in which it operates, together with its ability to identify and act on that information, as ‘Intelligence
Capital™’.
There is evidence that organisations that spend more on research are more successful in the long run. Research is therefore best viewed as an investment and not as a
cost. However, because of the difficulty in communicating the outcome of research, in
terms of bottom-line value or profitability or return on investment (see, for example,
Wills and Webb, 2006), it is easy to understand why organisations can see it as a cost.
Oates (2019) notes a ‘recurring need to justify the investment in research and data’,
despite the benefits it can bring. In Industry Insight 1.6, Matt Taylor recalls how his
team at mobile phone operator O2 measured the value of its work and how his team
at Twitter, where he is consumer insight lead, measures it.
Industry Insight 1.6
Quantifying value
In my previous role in the insights team at O2
we were expected to report on the value we were
creating, consistent with other teams that were
directly connected to commercial performance.
We were expected to demonstrate the revenue
benefit of our investments into brand tracking
and customer segmentation. We landed on two
routes to quantify the value of our work: cost saving and potential revenue. Demonstrating that we
were saving the company money, or identifying
new price points, dramatically improved our ability to ask for greater investment in insights.
Eight years has passed since then and the challenge in the age of data is even more pronounced.
Simply showing that research helps increase the
company’s level of understanding in customers is
fine, but when the chief finance officer is looking
at that next to a revenue per head model from a
14
M01 The Practice of Market Research 31362.indd 14
27/09/2021 21:44
The value of research
sales team – or the results of a product experiment
that shows a causal link between a new product
feature and user growth – it’s easy to see where
the investment will go first.
With the new audience insights team at Twitter,
we have tried to build this in from the start. Every
project we work on comes in via a request system
that is automatically tied to revenue databases, so
we can analyse the revenue being generated by our
work. We can finally demonstrate that account
teams at Twitter are 24 per cent more likely to
win a pitch if they are supported by our team.
This has dramatically helped our ability to pitch
for resources and grow our function.
Source: Adapted from Taylor, M. (2019) ‘Quantifying the value of
work’, Impact, 26, p. 71. Used with permission.
As Tanner (2005) points out, to see research as an investment, researchers need to
show decision-makers the link between the research objectives and the business or corporate objectives; they need to set the findings into the wider context and show how
it relates to the decision to be taken. This may mean combining so-called ‘small data’
from traditional research with ‘big data’, data such as transactional data, financial data,
social media data. Researchers must also communicate the findings in the language of
the business or policy context and not in the language of research and analytics.
To do research or not?
How do you decide whether or not to proceed with a research project? If you do decide
to proceed, how do you proceed? Remember, resources will be limited. Should you
spend some of these limited resources on research? The answer is, of course, it depends.
Why are you thinking about doing research or analysing data? Will it – and the insight it
delivers – help with whatever problem or issue you have identified or help with the decision you need to take? How important is solving that problem or addressing that issue or
taking that decision for the organisation? If research or data will help, and if the problem
or issue or decision is deemed serious enough, enough of a priority for the organisation,
then the research should proceed. In Industry Insight 1.7, Nick Rich at InterContinental
Hotel Group explains how he decides whether or not to take on a project.
Industry Insight 1.7
The investment decision
InterContinental Hotel Group has more than
5,200 properties worldwide offering more than
three quarters of a million hotel rooms. When it
comes to investing in research, Nick Rich says,
‘There are two objectives for me. One is that we
want to prove the worth of research and insight,
so if we have helped change a decision – what has
the material bottom line impact been? And two,
helping the business prioritise. We have been asked
to do every kind of research from the quality of the
sausages on a breakfast bar right through to new
opportunities in developing markets. When faced
with an abundance of requests, and the need for
support and partnership, we have to have some
way of saying: X is a priority but Y isn’t. By tracking back to our bottom line, our sources of revenue
growth – we can have those conversations . . . and
not have to say yes to everybody.’
Source: Adapted from Gray, R. (2018) ‘Inside knowledge’,
Impact, 21, pp. 56–60. Used with permission.
15
M01 The Practice of Market Research 31362.indd 15
27/09/2021 21:44
Chapter 1
Research, data and insight
Two key questions in deciding how to proceed are: how much time is there in
which to complete the research; and what resources (people and money) are available
with which to undertake it? The answers to these questions will have a bearing on
the type and scope of research that can be conducted. Of course, the two sets of questions are related: the resources available should reflect the importance of the problem
or issue or decision to the business. In deciding on the budget (and to some extent
the time available for the research) you – as the client or internal client researcher –
should consider the value of the information and insight that the research will provide
to the organisation and to the decision to be taken. The value of the insight (the benefit) should be greater than the money spent to get it (the cost). One way of looking
at this is to assess the risk (and the cost) involved in making a decision without the
help of the insight generated by the research: is the risk (and the cost) of making the
wrong decision greater than the cost of the research? If, for example, you are planning
to spend £3 million on the launch of a new service, the decision to spend £50,000
researching the effectiveness of the launch campaign may be a relatively easy one.
The risk is that you spend the £50,000 to find that the launch campaign is effective.
If you don’t spend it you take a bigger risk – the risk of spending £3 million on an
ineffective launch. Determining the value of the insight is, however, not always so
straightforward. In some cases, depending on the nature of the decision, the type of
organisation or the size of the potential investment, more formal risk assessment or
cost–benefit analyses, for example using decision tree theory or Bayesian statistical
theory, might be used.
It is also important to note the wider and longer-term value of the insight to the client organisation. The insights derived from a piece of research tend to have strategic,
long-term value (Wills and Williams, 2004) as well as tactical, short-term value. In
other words, the value of a piece of research may go well beyond what it contributes
to a particular decision; it may contribute to the greater understanding of, for example, a particular area, or a particular customer group, product or market. Industry
Insight 1.8 looks at the decision to do research before a product re-launch and at the
longer-term value of some advertising research.
Industry Insight 1.8
Cost and value
The cost of insight
In 2019, brewing company Asahi re-launched
Asahi Super Dry beer in its ‘authentic form’. The
beer had been brewed under contract but with the
backing of the beer’s parent company in Japan,
Asahi brought it back in-house, switched to the
original recipe and original production processes renowned for delivering its uniquely dry,
crisp taste. These changes allowed the company
to reposition it as a premium brand in Europe.
Before the re-launch the company needed to test
and optimise Asahi Super Dry’s new positioning
and a large number of advertising executions for
TV, out of home, press, digital, events and so on,
in five markets: Argentina, Romania, Canada,
France and Netherlands. One proposal was to
do face-to-face focus groups, each lasting two
hours, in each of the five markets. Paul Thomas,
Asahi’s global head of insight, had two concerns:
how tiring it would be for participants; and the
estimated cost (£150,000). Instead he chose a different approach: online (research) communities in
each market. He consulted them every day over
the course of a week and a half. This approach
16
M01 The Practice of Market Research 31362.indd 16
27/09/2021 21:44
The limitations of research
avoided participant fatigue and cost less money
(£75,000).
Source: Adapted from Gray, R. (2019) ‘A brewing challenge’,
Impact, 24, pp. 47–50. Used with permission.
The longer-term value of insight
2017 was a disappointing year for sales of
Mr Kipling cakes, despite 55 per cent of the UK
having had Mr Kipling in their house in the last
year. An internal review found that its advertising and pack design were underperforming. It is
a well-loved brand but not one that was moving
with the times. Catherine Haigh, insights controller at Mr Kipling parent company, Premier Foods
said, ‘We really wanted to reignite the love for the
brand . . . ’.
The company worked with consultancy, The
Leading Edge, to understand the place of the
brand, and the category it belongs to, ambient
packaged cake. They ran group discussions with
Mr Kipling fans, and frequent and infrequent buyers of packaged cake, asking them exploratory
questions. They found that this type of cake is
not associated with indulgence but rather it is an
everyday treat. After discussing the findings with
the client, the researchers conducted more groups
to co-create and test the new positioning, the concept of celebrating little moments that mean the
most in people’s daily lives. The creative agency,
McCann, was involved in the groups and saw and
heard what the consumers had to say. From that
McCann developed an ad called ‘Little Thief’. Premier Foods worked with research agency Hall &
Partners to test and develop the communications
idea and the scripts for the ad. This research not
only informed the ‘Little Thief’ execution but provided guidelines, which the company called the
brand’s ‘Recipe for Success’, around its creative
script development for future work. Catherine
Haigh says these ‘will live on and continue to
inform all of our creative development going forward’. The ad aired in March 2018. Six months
later, sales of Mr Kipling had risen by almost
10 per cent.
Source: Adapted from McQuater, K. (2019) ‘Treat yourself’,
Impact, 24, pp. 52–54. Used with permission.
The limitations of research
We have seen how the value of research depends on it providing actionable,
insightful, high-quality information that can be used in the decision-making process and longer term to reduce risk and add value to the organisation. What limits
its value?
Research is only of value if it fulfils its purpose – if it provides information and
insight that contribute to the planning and decision-making process. Research is a
means to an end, not an end in itself. It will be of use only if it is based on a clear
understanding of what problem or issue it is to address, if it has clear aims and objectives, if there is a clear understanding of what kind of data and information is needed
for effective decision making. In fact, there are many factors that limit the value of a
piece of research, including the following:
poor definition of the problem;
lack of understanding of the problem (or the brief);
● poor or inappropriate research design;
● limitations of the methods used;
● poor execution of the research itself;
● poor quality of the data;
● problems with the interpretation of the results;
● status of knowledge;
●
●
17
M01 The Practice of Market Research 31362.indd 17
27/09/2021 21:44
Chapter 1
Research, data and insight
time that elapses between commissioning the research and delivering and applying
the findings;
● use or misuse or non-use of research evidence by the decision makers.
●
We look at each of these in turn below.
Poor definition of the problem
This is a key stage in the research process. A clear and accurate statement or definition of the problem is essential if the research is to provide useful information for
the decision-making process (Bijapurkar, 1995). Good quality research is relatively
easy to carry out but it all means nothing if it does not address the problem or issue
under investigation. It will not be able to address the problem if it is not clear what
the problem is. Paas (2019) notes that one of the key reasons projects fail is because
the ‘focal strategic issue’ is poorly defined. A key skill for a researcher is to be able
to define or help the client define the problem to be researched. To do this effectively
the researcher must understand the wider context of the problem and the decision
to be made on the basis of the findings, including the factors that may affect the
implementation or action to be taken as a result. It is important therefore that the
researcher checks that all those in the client’s decision-making unit (DMU) have been
consulted about what it is the research is to investigate so that (at the other end of the
process) they are clear about what can be done – what decisions made, what actions
taken – with the evidence collected by the research.
Lack of understanding of the problem (or the brief)
If the researcher fails to understand what the research must deliver or misinterprets
what is needed, they may design research that is inappropriate and so of little or no
value. The person asking for or commissioning the research has a responsibility to
ensure that the research brief, the document which sets out the business problem
and what the research must achieve, is clear and unambiguous. This does not mean
that the statement of the problem should go unchallenged. You must interrogate the
brief. Whitehill Hayter (2014) notes how doing this can ‘re-prioritise the . . . issues
that can sometimes obscure what lies at the heart of a . . . brief’. The researcher has
a duty to ensure that they understand the problem and the brief (Pyke, 2000), and
understand what evidence is needed from the research.
Poor or inappropriate research design
The value of any research will be limited by the research design – by its suitability in
providing the evidence needed to address the problem. If the research design is poor,
the research will be of little value. Good research design is dependent on a clear understanding of the context or setting, a clear and accurate definition of the problem to be
addressed by the research and a clear understanding of how the findings will be used.
Poor research design is sometimes referred to as ‘design bias’ (Smith and Noble, 2014).
Here’s an example: you need your customers’ views on an important change to the
18
M01 The Practice of Market Research 31362.indd 18
27/09/2021 21:44
The limitations of research
service you offer. You send an email inviting customers on your database to take part in
a survey. Customers on the database are those who have an online account with you and
who have given permission to be contacted for research purposes. You get a response
rate of 5 per cent. How useful will the findings be? Not all your customers are on the
database; not all those on the database are available for research; of those who are, only
5 per cent responded. Given all of this, it is very unlikely that the findings will be of any
use since those who responded are not representative of your population of customers.
Limitations of the methods used
The data you collect will only be as good as the methods you use to collect them. If,
for example, you need a detailed, in-depth understanding of women’s facial cleansing
routines, the data collected using a survey questionnaire may be limited; it may be
more appropriate to use qualitative methods – an ethnographic approach – to get at
the sort of understanding needed. Remember, all research methods have advantages
and limitations for the context in which they are applied. The idea is to choose a
method that is the most appropriate to the research problem and to acknowledge
and try to mitigate any limitations.
Poor quality of the data
The value of research depends on it providing insightful, high-quality information
for decision making. Value will be limited if the quality of the data you use is poor.
Box 1.1
Guiding principles: validity and reliability
There are two concepts you are going to come across throughout the research process:
validity and reliability. These terms are used to describe and to evaluate the quality
and rigour of research. You will see them referred to in the context of research design,
sampling, question design and analysis. They can be difficult concepts to grasp. Validity is about whether you have ‘measured’ what you thought you were measuring. If you
look up the definition of ‘valid’ you’ll see descriptions like ‘sound basis’ and synonyms
including ‘reasonable’, ‘well-founded’, ‘justifiable’, ‘plausible’, ‘authentic’. Those are
the descriptors you want applied to your research and its findings. Reliability is about
consistency or stability: if you were to repeat the research, would you get the same
results? This is sometimes referred to as ‘replicability’. If you look up synonyms of reliable you’ll see ‘consistent’ and ’dependable’, again the sort of words you want used
for your research and its findings. These concepts of validity and reliability are applicable to both quantitative and qualitative methods of research although some qualitative researchers use different terms: ‘truth value’ for validity, for example (Lincoln and
Guba, 1985) or ‘credibility’ (Noble and Smith, 2015) and ‘consistency’ as an alternative
to ‘reliability’ (Lincoln and Guba, 1985). We look in more detail at these concepts in
later chapters.
19
M01 The Practice of Market Research 31362.indd 19
27/09/2021 21:44
Chapter 1
Research, data and insight
If you are using existing data, you will need to ensure that they are of sufficient quality – and relevance – for your purposes. Key questions you should ask of any data
source include the following: Where did the data originate? What is the source? Why
do they exist – for what purpose were they gathered? Who commissioned the data
and/or who produced them? Why were they commissioned/produced? Can they be
used for research purposes? When were they collected – are they current or out of
date? How accurate are they? How valid and reliable? How were the data gathered?
How well do they represent the population? What quality standards do they meet?
What quality checks have been done?
Poor execution of the research itself
Research – even if you are using a suitable design – can be badly executed. Errors
can arise at all stages of the process – translating objectives to questions, designing
the data collection tool, sampling, doing fieldwork, data processing and preparation,
data analysis and communication of findings. In understanding how well research has
been designed and executed it is important to know the assumptions the researcher
or analyst made and their justification or rationale for the approaches they took as
well as to be able to see how the research was executed.
Problems with the interpretation of the results
Data can be misinterpreted, and any misinterpretation limits the value of the research.
The researcher must guard against any possible misinterpretation by making sure
that they clearly understand how to analyse and read the data in an objective and
systematic way, in a way that is free of bias. It is useful to acknowledge potential
sources of bias in order to address them.
Status of knowledge
Research does not produce ‘right answers’ – the findings from any research are
always partial and contingent, and dependent on context (Shipman, 1997). Knowledge is not ‘value free’ – it is influenced by the social and cultural context in which
it is collected, and by the view of the participant, and by the researcher designing
the study and collecting and interpreting the data. Although we strive to conduct
objective research we can never be completely objective – our ways of knowing and
finding out about things are always filtered through our own way of thinking and
our way of seeing and knowing the world. Throughout the process – in designing
and conducting research, gathering data and interpreting and using them – we need
to be aware of these possible sources of bias, and their influence.
Time
The time that elapses between commissioning the research and delivering and applying the findings can limit the value of the research. Data become out of date – the passage of time erodes the value of research simply because the data are time dependent.
20
M01 The Practice of Market Research 31362.indd 20
27/09/2021 21:44
The limitations of research
Use, misuse or non-use of research evidence
The value of research also lies in whether the findings are used and, if they are
used, how they are used. They may be used well or badly, or they may be ignored.
In a 2017 survey of data scientists (Kaggle, 2017) 24 per cent said that one of the
barriers they faced at work was ‘results not used by decision makers’. Results may
not be used for a number of reasons – the decision-makers may simply not believe
them, or may not believe that they are valid or reliable; they may find them hard to
understand, ‘not in their language’ (Berinato, 2019), unconvincing or irrelevant; or
they may fail to see how they could be used or they may have problems integrating
them into the decision-making process. Shah et al. (2012) note that, ‘Investments in
analytics can be useless, even harmful, unless employees can incorporate that data
into complex decision making’. Using what they call the ‘Insight IQ’, described as
a tool for assessing ‘ability to find and analyze relevant information’, they evaluated 5,000 people in 22 global companies. They placed them into three groups:
‘unquestioning empiricists’ who ‘trust analysis over judgment’; ‘visceral decision
makers’ who ‘go exclusively with their gut’; and ‘informed skeptics’, those who
‘balance judgment and analysis, possess strong analytic skills, and listen to others’
opinions but are willing to dissent’. They found that only 38 per cent of employees
and 50 per cent of senior managers fell into this ‘data savvy’ group. Tarka (2018)
reports from a study of 213 managers that despite ‘strong declarations and beliefs
about the . . . usefulness of information [from marketing research] . . . , [managers] in reality prefer solutions based on intuition and irrational thinking’. He also
notes that faced with the difficulty of information processing, limited memory,
and strong reliance on personal experience, among other things, managers look
for much simpler solutions in decision making, and information from research is
neglected.
It is worth noting too that research findings are not always clear-cut – they can
be inconclusive, which may limit their use, or lead to the wrong decision being
taken. An organisation’s culture and/or any internal political issues may affect the
use, misuse or non-use of the research findings; or its structure, systems or level of
skills and resources may get in the way of integration or use of the findings (Wills
and Williams, 2004). Shah et al. (2012) came to similar conclusions. They found
four issues preventing organisations getting a better return on investment in data:
too few employees with analytic skills; IT not spending enough time helping others
get the information they need and/or not enough time helping them articulate the
way in which they need to use the data; lack of a coherent, accessible structure for
data storage, leaving staff not knowing where to find the information they need;
and a lack of information management and under-investment in understanding the
information by the business executives. James Oates (2019), UK analytics director
at Nielsen, notes that showing ‘the true value of research is a constant challenge’. In
Industry Insight 1.9, he sets out the value of a leader who champions the importance
of data and data-based analytics.
The process by which decision makers ‘digest, accept, then “take on board”’ findings is called ‘knowledge mobilisation’ (Brown, 2014). Brown lists the factors that
influence the ‘taking on board’: whether there is a ‘demand’ or need; the perception
of the credibility of the source of the findings; the perception of the quality of the
research; and the decision makers’ level of engagement with the research. Of course,
all of this is contingent on the researcher getting access to the decision makers in order
21
M01 The Practice of Market Research 31362.indd 21
27/09/2021 21:44
Chapter 1
Research, data and insight
Industry Insight 1.9
Data direction from the top
When an organisation has a senior leader championing the importance of data and analytics it
creates an environment where data is seen as a
critical influence on a business. When this leadership is in place, there are consistent traits that we
can recognise.
First, there is an explicit expectation that
the data becomes part of the common language
of the organisation. This creates a culture that
requires employees across all parts of an organisation to both understand the data and explain
performance and decision making through the
data. With a consistency in data use, and ability
to interpret the data, come faster decisions. Less
time is required to explain what the data or the
facts mean, and the focus shifts to getting into
the real business of identifying what to do next.
This is crucial as there is always a department or
individual that is a heavy user of data, but it is
when you see greater organisational reach of data
that its value increases. To drive such an approach
usually requires investment in data training and
education, but also investment in ensuring there
are tools that support the light or infrequent user
to access the data easily and successfully. Data
shouldn’t be pushed by a lone research manager
or data officer against a tide of resistance, there
should be a positive flow of data throughout the
business to willing recipients. When an organisation creates this basic level of data capability, the
data flows through the business and fosters an
environment where there is an increasing need for
more sophisticated data analysis. As companies
move beyond the basics of using data for performance reporting to using it for strategic business
decisions, bigger and more dynamic questions
arise. This is when an agency can add its value by
clarifying the big questions, unlocking new analytic approaches and ensuring the data is delivering growth. Creating a business environment
to drive deeper analytics means being prepared
to invest. This is where budget is crucial and the
transition from data being viewed as an overhead
to an asset begins. Data will only come to life with
senior sponsorship and then we can see its true
value.
Source: Adapted from Oates, J. (2019) ‘Data direction from the
top’, Impact, 26, p. 67. Used with permission.
to have the findings considered. In terms of getting the message across, Brown notes
that decision makers want it to be presented in a clear and accessible way, tailored
to their needs and set in context.
In ensuring that the value of any research and data is ‘taken on board’ and
maximised, it is important to have a clear understanding of the problem which the
research must address, that is, the business problem, and what the decision makers plan to do with the findings – what decisions are to be made or what actions
are to be taken. Having this understanding at the outset will mean that you can
at one end of the process design quality research to address the decision makers’
‘demand’ issue and, at the other end, present the findings in a clear and accessible way in the context of and tailored to the business problem, thus helping the
decision-maker to take whatever action is needed. As a researcher you can influence the decision-makers’ perception of both the quality of the research and the
credibility of the source of the findings. You also have a role to play in managing
the decision-makers’ expectations in terms of what the research and the data can
and cannot provide.
22
M01 The Practice of Market Research 31362.indd 22
27/09/2021 21:44
The limitations of research
Research buyers’ views of the research product
What do research buyers, users and decision-makers think are the weaknesses of the
service provided by researchers? Vriens et al. (2019) found that the main complaints
were the need for more actionable insights and greater actual return on investment
in research. In discussions with research managers and research users, and from a
review of the literature, Bairfelt and Spurgeon (1998) found that research often did
not meet expectations. They found that research buyers perceived that it was not well
managed, that findings were poorly presented, and that it failed to deliver value for
money. Interviews with research users in Spurgeon’s organisation, Shell, revealed a
perception that too few researchers are ‘commercially orientated’ and most have little
knowledge of or interest in the way the client’s business works; they would rather
focus on data than insights and implications. In addition, it was felt that the data
produced are ‘nice to know but not directly actionable’ and tend to be historical,
not future-focused. Research is perceived to be a discipline of ‘black box techniques’
and the research process ‘shrouded in mystery’. It is viewed as lacking creativity in
design and delivery – the output (and the way it is presented) seen as ‘too often dull
and uninspiring’.
Simmons and Lovejoy (2003) believe that research sometimes fails to deliver the
understanding that CEOs need because researchers do not worry enough ‘about what
the research actually means’. In Tanner’s (2005) study of what CEOs want from
research, one said, ‘We want results that demonstrate the hard reality of whatever’s
being researched will lead to profitable results.’ Another remarked, ‘It’s almost as
though they’re talking a different language. You brief them [the researchers] . . . but
the responses are often in terms of intangibles. It’s frustrating. I’m often left thinking
that I’ve paid them to do this and I don’t have anything I can easily grasp . . . I have to
ask, “Have I wasted my money? Would I do this again?”’. Berinato (2019) notes the
same thing: business executives ‘don’t see tangible results because the results aren’t
communicated in their language’. Tanner notes that because most researchers do not
speak the same language as the decision maker, important findings ‘are lost at senior
management levels where they are necessary for strategic business decision making’.
Bijapurkar (1995) suggests that to contribute better to strategic decision making
researchers should improve their problem-definition skills and their understanding
of the business context of the decision, and learn to look to the future rather than
describe the present. Research by Egan et al. (2009) reinforces this: they asked managers where their company market researchers fell short. The top three areas were:
lack of understanding of the business issue; lack of ability to see the ‘so what’ or ‘now
what’; and lack of ability to translate the findings into actionable recommendations.
Chadwick (2005) notes that clients ‘expect more proactivity in the delivery of insight,
more integration of information . . . , more consulting and more senior involvement’.
Edgar and McErlane (2002) argue that researchers should position themselves as the
client’s ‘integrated business partner’, that clients will ‘go back to researchers who
can provide them with business solutions and knowledge not just research data’. As
Morgan (2018) notes, data is the ‘supporting material’.
In conclusion, the value of research and data lies in turning them into information,
knowledge and actionable insight. This means paying attention to drawing out the
implications of the findings and interpreting the findings in the context of the client’s
23
M01 The Practice of Market Research 31362.indd 23
27/09/2021 21:44
Chapter 1
Research, data and insight
business problem and business environment, rather than just presenting data. Not
only do you need to know what Smith (2005) calls ‘the “content” of market research’
but you also need to be able to communicate what it means to those who are going
to use it. You must be what Smith calls a ‘trusted information advisor’. This involves
the following, according to Smith:
being able to work in partnership with clients;
being able to get to grips with and make sense of the data;
● being able to interpret what the data mean by applying ‘contextual understanding’;
● being able to develop robust arguments that help clients make informed judgements;
● being able to present the research evidence in an active, engaged way and not in a
passive, detached way;
● being able to engage in the decision-making process.
●
●
Being able to operate as ‘a trusted information advisor’ depends on several things,
including your skills set, your confidence in your skills and abilities, and your credibility with the client (Smith, 2005). It also depends on the client’s willingness to
engage in this way, which – as Chadwick (2005) above and others (Pyke, 2000) have
found – is not always the case.
Chapter summary
●
●
Research is about systematic investigation to find things out. It is the process
by which we produce evidence or knowledge about the world. It is founded on
scientific methods, which are in turn supported by philosophical principles about
the nature of knowledge and how we construct that knowledge.
Research plays a vital role in providing robust and credible evidence in a wide
range of contexts for planning and decision-making processes in organisations
in the public, private and not-for-profit sectors. Its value can be limited by many
things, including the following:
– poor problem definition;
– lack of understanding of the brief;
– poor or inappropriate research design;
– the limitations of the methods used;
– poor execution of the research itself;
– poor quality data;
– the interpretation of the results;
– the status of knowledge;
– the use, non-use or misuse of research evidence by the decision makers;
– the time that elapses between collecting the data and applying the findings.
24
M01 The Practice of Market Research 31362.indd 24
27/09/2021 21:44
References
Exercises
1 Your client has a new internal research executive and a new marketing executive.
Prepare a guide for the new recruits on what research can and cannot offer
decision makers making use of the information in the Industry Insights 1.1–1.9.
References
Allen, G.D. (2017) ‘Hierarchy of knowledge – from data to wisdom’, International Journal of
Current Research in Multidisciplinary, 2, 1, pp. 15–23.
Bairfelt, S. and Spurgeon, F. (1998) ‘Plenty of data, but are we doing enough to fill the information gap?’, Proceedings of the ESOMAR Congress, Amsterdam: ESOMAR.
Berinato, S. (2019) ‘Data science and the art of persuasion’, Harvard Business Review,
January–February, pp. 126–37.
Bijapurkar, R. (1995) ‘Does market research really contribute to decision making?’ Proceedings of the ESOMAR Congress, Amsterdam: ESOMAR.
Bold, B. (2019) ‘Turn on, tune in’, Impact, 25, pp. 48–52.
boyd, d. and Crawford, K. (2012) ‘Critical questions for big data: Provocations for a cultural,
technological and scholarly phenomenon’, Information, Communication & Society, 15, 5,
pp. 662–79.
Brown, C. (2014) ‘“Who you know and what you have to say”: an alternative look at knowledge mobilization theory’, International Journal of Market Research, 56, 4, pp. 531–50.
Brynjolfsson, E., Hitt, L. and Kim, H. (2011) Strength in Numbers: How Does Data-Driven
Decisionmaking Affect Firm Performance? http://dx.doi.org/10.2139/ssrn.1819486
(Accessed 12 January 2021).
Chadwick, S. (2005) ‘Do we listen to journalists or clients? The real implications of change
for the market research industry’, Proceedings of the Market Research Society Conference,
London: MRS.
Cluley, R., Green, W. and Owen, R. (2020) ‘The changing role of the marketing researcher
in the age of digital technology: Practitioner perspectives on the digitization of marketing
research’, International Journal of Market Research, 62, 1, pp. 27–42.
Diebold, F. (2019) ‘On the origin(s) and development of “Big Data”: the phenomenon, the
term and the discipline’, https://www.sas.upenn.edu/~fdiebold/papers/paper112/Diebold_
Big_Data.pdf (Accessed 1 September 2020).
Edgar, L. and McErlane, C. (2002) ‘Professional development: the future’s in diamonds’,
Proceedings of the Market Research Society Conference, London: MRS.
Egan, M., Manfred, K., Bascle, I., Huet, E. and Marcil, S. (2009) The Consumer’s Voice – Can
Your Company Hear it? Boston Consulting Group, https://www.bcg.com/publications/2009/
markerting-sales-customer-experience-consumer-voice (Accessed 25 May 2021).
Flores, L. (2016) ‘Market research industry, tipping point or no return? International Journal
of Market Research, 58, 1, pp. 15–17.
Gray, R. (2019) ‘A brewing challenge’, Impact, 24, pp. 47–50.
Gray, R. (2018) ‘Inside knowledge’, Impact, 21, pp. 55–60.
Gray, R. (2018) ‘Insight into charity campaigning’, Impact, 20, pp. 44–8.
25
M01 The Practice of Market Research 31362.indd 25
27/09/2021 21:44
Chapter 1
Research, data and insight
ICC/ESOMAR (2016) International Code on Market, Opinion and Social Research and Data
Analytics, Amsterdam: ESOMAR.
Jenkins, C. (2016) Conference Notes, IJMR-hosted debate: Who will succeed in the new era
of data discovery, International Journal of Market Research, 58, 3, pp. 473–84.
Kaggle (2017) https://www.kaggle.com/kaggle/kaggle-survey-2017 (Accessed 10 June 2021).
Kitchin, R. (2014) The Data Revolution, London: Sage.
Kollman, J. and Curry, A. (2019) The Responsive Business: Creating Growth and Value
Through Intelligence Capital, London: Kantar and MRS.
Laney, D. (2001) ‘3-D Data Management: Controlling Data Volume, Velocity and Variety’,
META Group Research Note, 6 February, http://goo.gl/Bo3GS (Accessed 1 September
2020).
Lincoln, Y. and Guba, E. (1985) Naturalistic Inquiry, Beverly Hills, CA: Sage.
McQuater, K. (2019) ‘Distilling data’, Impact, 25, p. 15.
McQuater, K. (2019) ‘Treat yourself’, Impact, 25, pp. 52–4.
Miller, H. (2010) ‘The data avalanche is here. Shouldn’t we be digging?’ Journal of Regional
Science, 50, 1, pp. 181–201.
Moon, C. (2015) ‘The (un)changing role of the researcher’, International Journal of Market
Research, 57, 1, pp. 15–16.
Morgan, B. (2017) ‘Ageing populations’, Impact, 17, pp. 14–15.
MRS (2019) Code of Conduct, London: MRS.
Noble, H. and Smith, J. (2015) ‘Issues of validity and reliability in qualitative research’,
Evidence-Based Nursing, 18, 2, pp. 34–35.
Nunan, D. (2016) ‘The declining use of the term market research: An empirical analysis’,
International Journal of Market Research, 58, 4, pp. 503–22.
Oates, J. (2019) ‘Data direction from the top’, Impact, 26, p. 67.
Paas, L. (2019) ‘Marketing research education in the Big Data era’, International Journal of
Market Research, 61, 3, pp. 236–51.
Phillips, T. (2017) ‘A business mindset’, Impact, 19, pp. 27–36.
Pyke, A. (2000) ‘It’s all in the brief’, Proceedings of the Market Research Society Conference,
London: MRS.
Raben, F. (2015) ‘A call to research arms’, ESOMAR Research World, April.
Research Live Industry Report 2021, London: MRS.
Shah, S., Horne, A. and Capella, J. (2012) ‘Good data won’t guarantee good decisions’,
Harvard Business Review, April 2012, https://hbr.org/2012/04/good-data-wont-guarante
e-good-decisions (Accessed 1 September 2020).
Shipman, M. (1997) The Limitations of Social Research, London: Longman.
Simmons, S. and Lovejoy, A. (2003) ‘Oh no, the consultants are coming’, Proceedings of the
Market Research Society Conference, London: MRS.
Smith, D. (2005) ‘It’s not how good you are, it’s how good you want to be: Are market
researchers really up for “reconstruction”?’, Proceedings of the Market Research Society
Conference, London: MRS.
Smith, J. and Noble, H. (2014) ‘Bias in research’, Evidence-Based Nursing, 14, 4, pp. 100–1.
Sorrell, M. (2016) ‘Digital, data and globalisation’, Impact, 12, pp. 37–8.
Tanner, V. (2005) ‘Using investment-based techniques to prove the “Bottom Line” value of
research and give CEOs what they want’, Proceedings of the Market Research Society
Conference, London: MRS.
26
M01 The Practice of Market Research 31362.indd 26
27/09/2021 21:44
Recommended reading
Tarka, P. (2018) ‘The views and perceptions of managers on the role of marketing research in
decision making’, International Journal of Market Research, 60, 1, pp. 67–87.
Taylor, M. (2019) ‘Quantifying the value of work’, Impact, 26, p. 71.
Tukey, J. (1962) ‘The future of data analysis’, Annals of Mathematical Statistics, 33, 1,
pp. 1–67.
Vriens, M., Brokaw, S., Rademaker, D. and Verhulst, R. (2019) ‘The marketing research curriculum: closing the practitioner-academic gaps’, International Journal of Market Research,
61, 5, pp. 492–501.
Weiss, S. and Indurkhya, N. (1998) Predictive Data Mining: A Practical Guide, MA: Morgan
Kaufmann Publishers.
Whitehill Hayter, C. (2014) ‘Behavioural economics: a model of thinking’ International Journal of Market Research, 56, 2, pp. 145–7.
Wills, S. and Webb, S. (2006) ‘Measuring the value of insight – it can and must be done’,
Proceedings of the Market Research Society Conference, London: MRS.
Wills, S. and Williams, P. (2004) ‘Insight as a strategic asset – the opportunity and the stark
reality’, International Journal of Market Research, 46, 4, pp. 393–410.
Recommended reading
Impact magazine: https://www.mrs.org.uk/resources/impactmagazine
International Journal of Market Research: https://www.mrs.org.uk/resources/ijmr
Research Live: https://www.research-live.com/
GreenBook: https://www.greenbook.org/
Quirk’s Marketing Research Review: https://www.quirks.com/
27
M01 The Practice of Market Research 31362.indd 27
27/09/2021 21:44
Chapter 2
The practice of market and
social research
Introduction
The aim of this chapter is to give you an overview of the practice of research.
We look at the research process and the skills, tasks and roles involved in
commissioning, supplying and using the output. We also look at the ethical,
legal and regulatory framework in which it all operates.
Topics covered
The research process
● Skills, tasks and roles
● Ethical, legal and regulatory issues in research practice.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project; and Topic 2: Guiding Principles.
M02 The Practice of Market Research 31362.indd 28
27/09/2021 21:44
What you should get from this chapter
At the end of this chapter you should be able to:
understand the research process and the stages in a project;
● understand the roles of commissioner, supplier and end user; and
● understand the ethical, legal and regulatory framework within which research operates and the
implications of that framework for the practice of research.
●
29
M02 The Practice of Market Research 31362.indd 29
27/09/2021 21:44
Chapter 2
The practice of market and social research
The research process
The process of conducting a research project is the process of conducting an enquiry
or investigation. It can be divided into the following stages:
Identifying the business problem and the research problem
Designing and planning the project
● Setting up the project
● Gathering the data
● Processing and analysing the data
● Communicating the findings.
●
●
Here is an outline of what is involved in each of the stages:
Problem identification
Identifying the business problem
Understanding the business problem
● Understanding the need for research
● Defining and refining the research problem
● Formulating the research objectives.
●
●
Design and planning
Reviewing the research objectives
Deciding what must be ‘measured’ and how to measure it
● Designing the research
● Selecting suitable approaches and techniques
● Identifying ethical, legal and/or regulatory issues.
●
●
Project set up
Briefing the project team
Allocating roles within the team
● Addressing any ethical, legal and/or regulatory issues.
●
●
Data gathering
Getting the data/doing fieldwork
Managing the data/fieldwork team
● Monitoring and checking the quality of the data
● Managing any ethical, legal and/or regulatory issues
● Storing the data
● Liaising with the wider project team.
●
●
30
M02 The Practice of Market Research 31362.indd 30
27/09/2021 21:44
The research process
Processing and analysis
Managing any ethical, legal or regulatory issues
Cleaning and transforming the data
● Exploring the data
● Displaying and visualising the data
● Modelling and/or analysing the data
● Liaising with the wider project team
● Interpreting the data.
●
●
Communicating the findings
Interpreting and reviewing the findings
Liaising with the analysis team and the client team
● Planning and designing the output/presentation/report
● Delivering the findings, including conclusions, recommendations and actions to
be taken.
●
●
While this is a largely sequential process, it is also an iterative process. You might
move forward to one stage and find that you need to go back to a previous stage
to amend or tweak it. You may realise, for example, in reviewing the research
objectives that you do not understand the business problem, and so you need to
return to it. The boundaries between each stage, and the activities within stages,
can be fluid.
While projects vary in type and scale, the process outlined above should serve any
research project. Other related disciplines – namely, statistics and data science, have
similar processes. A fairly common framework for teaching statistics (Spiegelhalter,
2019), which can also be applied to real-world research problems, including those
that use existing data, is the PPDAC Model of the statistical method defined and
outlined by MacKay and Oldford (2000). PPDAC stands for:
Problem
Plan
● Data
● Analysis
● Conclusion.
●
●
As you can see, the stages correspond roughly to the stages in the research process.
Eremenko (2018) uses a five-stage process (devised by Joe Blitzstein and Hanspeter
Pfister for a data science course at Harvard in 2013), which he says offers ‘a robust
and reliable workflow for any kind of data project’:
Identify the question
Prepare the data
● Analyse the data
● Visualise the insights
● Present the insights.
●
●
Figure 2.1 presents a summary of the research process.
31
M02 The Practice of Market Research 31362.indd 31
27/09/2021 21:44
Chapter 2
The practice of market and social research
Problem
Plan
•Identify the business problem
•Define the research problem
•Design the research
•Plan and set up the project
•Gather the data
Data
Analysis
Conclusion
•Process the data
•Analyse the data
•Interpret and review the findings
•Communicate the findings
Figure 2.1 Summary of the research process
Research skills, roles and tasks
Let’s look at the skills required and the roles and day-to-day tasks in the research
process.
Research skills
The key skills of a research practitioner are set out in Box 2.1. These are the skills
which the MRS vocational course, the Advanced Certificate in Market and Social
Research Practice, aims to develop.
Research roles
There are effectively three main roles in the research and data industry – supplier,
buyer and end user. The supplier, as the name suggests, is the person who supplies
the research and/or data. They are typically responsible for the research design, for
32
M02 The Practice of Market Research 31362.indd 32
27/09/2021 21:44
Research skills, roles and tasks
Box 2.1
MRS Advanced Certificate: research skills
Key research skills include the ability to do the following:
Understand, define and evaluate research objectives for given research problems
Design appropriate research solutions to identified problems based on clear understanding of a range of research approaches and techniques
● Select suitable approaches and techniques for the collection and analysis of the data
needed to inform effective decision-making, and justify the selection
● Plan a research project based on understanding of the research need and the
resources available for the research
● Present the findings of research, reflecting a clear understanding of the analysis and
interpretation of the data collected
● Provide recommendations to support the decision-making process based on clear
understanding of the information gathered during the research process
● Analyse and evaluate choices made at each stage of the process.
●
●
Source: MRS Advanced Certificate in Market & Social Research Practice. Syllabus & Assessment Guidelines,
October 2020. Used with permission.
overseeing its execution and for reporting the findings to the buyer and/or the end
user. The buyer, again as the name suggests, buys or commissions research and/or
data from a source either inside their own organisation – for example from a knowledge, insight or data centre or from a department or team that might be called marketing intelligence, marketing planning or marketing services, or consumer insight,
market research or merely research or data analytics – or from outside the organisation – for example from an agency or a consultancy. In some organisations, the buyer
is also the end user. Below we look at the role of the clientside or in-house researcher,
the different types of supplier, the roles within a research agency and, in particular,
the role of a research executive.
The clientside research or insight role
The clientside role will depend on the type of organisation, the nature of its business
and the way it views research, data and insight. Some organisations have research,
data and insight teams. Some outsource the service; some have integrated it into a
broader function involving, for example, analytics, information or knowledge or
insight management, business or marketing intelligence, marketing services, business
or strategic planning, or policy making. A scan of job advertisements reveals the
variety of clientside roles. There are ads for, among others, project manager; research
executive, research manager and research director; Voice of the Customer (VoC)
research expert; data analyst; marketing analyst; brand strategist; strategic insight
consultant; and insight and analytics manager.
Industry Insight 2.1 illustrates how things work at brewing group Asahi Europe.
33
M02 The Practice of Market Research 31362.indd 33
27/09/2021 21:44
Chapter 2
The practice of market and social research
Industry Insight 2.1
What does a client do?
Asahi is a ‘challenger’ brand with ambitions to
be a ‘global premium beer powerhouse’ in a market that is changing, with declining beer sales in
Europe, and the rise of the microbrewery and craft
beer. The ‘challenger mindset’ is engrained in the
business and is mentioned in major external presentations to explain what the business stands for.
The mindset is built on four pillars:
unleashing the right portfolio for the right
market;
● executing everything to the highest possible
degree;
● employing the right people; and
● insights.
●
Paul Thomas, global head of insight at Asahi says,
‘We are very clear that having a superlative understanding of the consumer and the customer will
lead to bigger and better ideas. If we just get the
same insights as, say, Heineken, they are going to
win, because they have a lot more money to spend
“above the line”. We have to be smarter.’ It is the
quest to be smarter that led to the setting up of the
global insights function within global marketing.
One of its key roles is to ensure that the ethos of
consumer insight permeates the whole organisation. It is a small team, three-strong, but the company has about 20 local insights staff elsewhere
in the organisation. These local teams were in the
past very self-contained but are now part of a network with the global insights team. The insights
team ‘don’t just do the data’, says Paul Thomas,
‘ . . . we do everything: from high level strategic
ideation right down to in-store executions; from
developing a platform idea for advertising to
testing and optimising that final copy and from
understanding customer reality to . . . tracking
sales performance’. The team works with external agencies but as much as possible is completed
in-house – otherwise, Thomas believes, ‘there is
always the danger that the client-side insight function becomes little more than a project management team’.
Source: Adapted from Gray, R. (2019) ‘A brewing challenge’,
Impact, 24, pp. 47–50. Used with permission.
Many organisations are organised to ensure that the customer is the focus of the
business, with research and market planning functions called customer or consumer
insight. In some organisations researchers are involved in the early stages of the
business or marketing or policy-making process, at the development of ideas and
initiatives, as well as at a later stage when formal research or analysis is being commissioned. Industry Insight 2.2 illustrates how it works at innocent, makers of innocent smoothies.
The day-to-day job of an in-house researcher
The in-house researcher may be responsible for research into a particular market or
product or service or area, or they may be a specialist in a particular type of research
or data. The job is likely to involve liaising with or working alongside decision
makers, for example in strategy, marketing, customer service, sales, production or
policy formulation, as the Industry Insights show. The job may involve internal consultancy, advising on the use of research and data, reviewing research conducted or
34
M02 The Practice of Market Research 31362.indd 34
27/09/2021 21:44
Research skills, roles and tasks
Industry Insight 2.2
Where does the insight team fit in?
innocent started selling fruit smoothies at a music
festival in 1999. They famously put up a sign asking people if they thought they should give up their
jobs to make smoothies, with a bin in front of their
stall for ‘Yes’ and a bin for ‘No’. At the end of the
weekend the ‘Yes’ bin was full. The company now
has around 450 employees across 11 countries in
Europe. Its insight team, however, is very small
and has limited resources. It does brand tracking
research and ad hoc projects. Jessica Vara, group
consumer insight manager, describes the set up
and the approach the company takes: ‘We don’t
sit within our teams – we sit across the business,
with other teams . . . Insight is here to help give
us confidence in our gut feel . . . There are certain
projects where we might not think we need insight
because there is enough knowledge in the room.
We look at which projects and business challenges
are our biggest bets and we look at where insight
is going to have the most impact. We partner with
people – whether it’s our ideation team or our
marketing managers – and we are key in the decision making from the beginning . . . We’re here to
champion the consumer voice.’
Source: Adapted from Bainbridge, J. (2018) ‘Smooth operator’,
Impact, 22, pp. 50–4. Used with permission.
data gathered and ensuring that the insights from it are integrated into the business
and into the planning process – in effect making sure that any data are converted to
information, knowledge and insight and applied effectively to move the business or
the issue forward. Richard Ellwood (2011), brand and marketing research director
at The Walt Disney Company (EMEA), says that the role of the researcher within
an organisation is:
‘not purely to facilitate the communication of data from research agency to internal
teams [but] to interpret, dig deeper, make relevant and provide sustained consultancy
by acting as the voice of the consumer’.
Gayle Fuguitt, consumer insights vice president at General Mills, whose brands
include Cheerios, Yoplait and Häagen-Dazs, said this about the clientside research
function (Tarran, 2012):
I have always had a clear sense of how important the function could be in helping
grow the business . . . We start every year in a dialogue with company presidents about
what their biggest business issues are. We don’t say, ‘Right, these are our research
initiatives . . . ’
Fuguitt lists the skills clientside researchers need as ‘listening skills, business acumen, championing action, interpersonal skills and communications skills’ and ‘bifocal
vision’, which she defines as ‘keeping one eye on the here and now and the other on
the threats and opportunities that are on the horizon’.
The role may also involve collecting and analysing data, and providing guidance
and advice to internal data analysts. It may also involve managing and developing
databases and decision support systems. The role is also likely to involve providing and/or commissioning research and/or data analysis, managing the process,
and managing the relationship with research and data suppliers, and attending
fieldwork.
35
M02 The Practice of Market Research 31362.indd 35
27/09/2021 21:44
Chapter 2
The practice of market and social research
Types of research supplier
Research suppliers can be divided into three broad groups: full-service agencies;
specialist suppliers; and limited-service suppliers. Full-service agencies offer a full
research service, supplying everything from research design, data gathering and data
processing, to analysis and reporting of the findings and their implications. Specialist suppliers are those that specialise in a particular type of data or data collection
method, for example analytics or online research; or those that specialise in a particular market sector, for example pharmaceuticals, financial services, media or businessto-business research; or those that specialise in a particular technique or application,
for example online panels, semiotics, mystery shopping or product testing.
There are various kinds of limited-service suppliers – suppliers that specialise in
a particular part of the research process, usually fieldwork or data gathering only
or in data processing, statistical analysis or analytics. There are also those – usually
independent consultants – who provide research advice and consultancy, research
design, project management, qualitative fieldwork, data interpretation and reporting services.
Roles within a research agency
Most full-service research agencies and most specialist agencies will have a client
service or account management department, a field or operations department and a
data-processing or analytics (or even data science) department. These departments
reflect the key stages of a project: liaising with the client and planning the research
(client service and/or account management); collecting or accessing the data (field or
operations); processing and analysing the data (data-processing); and reporting the
findings (client service and/or account management). Each of these services can be
provided as a stand-alone service and, as we saw above, limited-service agencies are
those that specialise in one or more of these. Within each of these departments or
service functions there will be executives at different levels of seniority.
The role of researcher
The job of researcher involves everything from the research briefing stage (and
sometimes before it) through to the delivery of the findings and their implications (and sometimes beyond). In addition to this role, which is typically a mixture of research expertise, project management and client-facing consultancy, the
researcher may be involved in preparing new business sales pitches and may even
undertake internal development work. Also, some of their time may be devoted to
keeping up to date with developments in research practice on the one hand – the
content of the job – and developments in their clients’ business areas on the other –
the context of the job. There are also research jobs that are a combination of
research executive and data analyst roles. This is sometimes described as ‘research
analyst’, but some jobs labelled ‘data scientist’ also fit this description and many
researcher or insight jobs do too. It is a role that requires a mix of research and
data analysis skills.
36
M02 The Practice of Market Research 31362.indd 36
27/09/2021 21:44
Research skills, roles and tasks
The main duties of the researcher in a project involving primary research or
research with existing data are as follows:
liaise with the client;
understand the business problem;
● define the research problem;
● design the research;
● identify and plan for any ethical, legal and/or regulatory issues involved in the research;
● cost the research;
● prepare a timetable;
● write the proposal or research plan;
● discuss the proposal with colleagues and with the client;
● liaise with the client from commissioning to delivery;
● set up the project and manage it;
● liaise with and/or brief colleagues throughout the project;
● design and/or script the data collection tool;
● conduct a pilot study;
● refine the research plan/data collection tool in the light of the findings from the
pilot study;
● brief the data gathering or operations team;
● source existing data and evaluate the quality;
● prepare an analysis plan;
● brief the data processing and/or analysis team and/or do some processing and/or
analysis;
● liaise on progress;
● attend (or conduct, in the case of qualitative research) fieldwork;
● inspect the data;
● analyse and interpret the data;
● design the delivery of the findings;
● communicate the findings to the client, drawing out the implications for the client’s
business;
● take part in follow-up discussions with the client;
● review and evaluate the project; and
● organise archiving and storage of data and project documents.
●
●
In the fast-paced digital world clients can be – and many are – in real-time continuous
dialogue with their customers. Some use a marketing approach called ‘agile’ marketing or ‘scrum’ marketing, terms borrowed from software development. These terms
refer to an iterative process in which you identify a change to a product or service,
work on it, measure its impact, and aim to improve it over time. This approach
requires constant customer feedback. Rather than going to a research supplier with
a brief, getting a proposal, commissioning research and waiting for the findings, the
client needs almost instantaneous and continuous customer insight. As a result, the
relationship between client and researcher is ‘always on’. The researcher is delivering findings on a continuous basis (Cluley et al. 2020). In this scenario the role of
researcher is in effect a combination of field and research and, in some cases, analysis.
Industry Insight 2.3 shows how this operates at supplier, InSites Consulting. It is an
approach that has implications for the research skills set (Nunan and Di Domenico,
2019 and Cluley et al., 2020).
37
M02 The Practice of Market Research 31362.indd 37
27/09/2021 21:44
Chapter 2
The practice of market and social research
Industry Insight 2.3
The ‘always on’ client–researcher relationship
The client and the research supplier must have
a day-to-day relationship, to cater to the client’s needs for agility and to provide a better
understanding of the daily lives of consumers.
At InSites Consulting, interaction with clients
has switched from the classic briefing-planningcommissioning way of working towards a weekly
heads-up meeting at which clients share their projects and discussions. Often this won’t lead to formal commissioning – with briefings and request
for proposals – but it can lead to an explorative
brainstorm, or to a concrete topic guide or questionnaire proposal. InSites Consulting’s researchers have training in broad project management
skills and in narrower implementation skills and
can adapt to different types of work, depending
on what needs to be done that day. The research
teams are structured into fluid ‘squads’. They hold
short, daily meetings to prepare for the job that
needs doing. Every day they must incorporate
feedback from the client and learning from the
research. To make this work, the company uses
automation including artificial intelligence (AI)
and bots. This enables researchers to get rapid
feedback from research participants and turn it
into information for clients quickly.
Source: Adapted from Phillips, T. (2018) ‘Keeping a constant
customer dialogue’, Impact, 21, pp. 32–42. Used with permission.
Roles within the data service
The roles within the DP department or DP service provider will vary depending on
the source of the data. If the agency or provider specialises in survey data then typically the team will consist of data-processing (DP) executives and data analysts. They
may be managed by a data-processing manager, who in turn may report to a DP
or data services director. In addition, the agency may also have staff with specialist
statistical skills relevant to the type of analysis required. DP executives or analysts
write, test and implement the programs or scripts necessary for data capture, data
entry, coding, verification, data cleaning, and those for producing data tabulations
and statistical calculations, queries, reports and dashboards. In some organisations
the research executive prepares an analysis specification (sometimes called a tab spec
or a DP spec) that sets out how the data are to be coded, tabulated and analysed
and what statistical tests or special analyses are to be conducted. On the basis of this
specification, the DP executive writes the program that will produce the tables and
the necessary analysis.
If the agency or supplier deals with existing or ‘exhaust’ data, including big data
(for example, transactional data, social media data or other customer- or usergenerated data) then it is likely that the organisation will have a data science or data
analytics department. Executives in such a department will be involved in various
aspects of data infrastructure and technology, data strategy and data governance and
management. There may be a data architect responsible for designing the organisation’s data infrastructure or ecosystem. There may be data engineers responsible for
creating and/or maintaining the data ecosystem; and database administrators responsible for running and enabling access to databases and other data storage systems
including insight management platforms. There will be data analysts responsible
for data wrangling or munging (cleaning, transforming and mapping data) and displaying or visualising the data; machine learning specialists responsible for building
38
M02 The Practice of Market Research 31362.indd 38
27/09/2021 21:44
Research skills, roles and tasks
and implementing algorithms; and statisticians, data scientists and business analysts
responsible for analysing and interpreting data (akin to the researcher role). A manager will oversee the work of the department, managing the budget and the workload,
preparing costings, and liaising with analysts, clients and end-users about their needs
and have quality control responsibilities. They may be involved in recruiting, training
and supervising staff. They are likely to be responsible for managing the data governance framework. A director is likely to be responsible for the success of the data
operation, for overall quality control, including the design and/or implementation of
a data governance and a data curation strategy, for business development, for keeping up to date with developments in software and technology and for implementing
systems that will deliver to the needs of the analysts, clients and end-users.
Roles within the fieldwork service
When it comes to traditional research approaches, the responsibility for gathering
data lies with the fieldwork service. The field executive’s role involves preparing costings, liaising with research executives on timings, on script or questionnaire design
and on sample design, and setting up and managing fieldwork including dealing with
any ethical, legal and/or regulatory issues, and recruiting participants. For interviewer
administered surveys, fieldwork management involves allocating work, setting quotas
for the number of interviews to be completed per interviewer day, preparing briefing
notes for interviewers, running briefing and training sessions, and checking on the
progress of fieldwork. It may also include attending or supervising fieldwork, training and recruiting interviewers, and generally managing the fieldforce. For ­telephone,
mobile and online research, the tasks of monitoring progress of fieldwork and reporting on it usually fall to the research executive. Depending on the size of the organisation, the size of the fieldforce and the volume of fieldwork conducted, the field
executive’s role may be more differentiated, to the extent that some of these tasks
are conducted by specialists.
If international research is involved, an international fieldwork co-ordinator or
project manager may be part of the field or client service team, or part of a separate international co-ordination or operations unit. The role of a co-ordinator is to
ensure that the fieldwork in each country is conducted to the same standards. This
will involve liaising with in-market fieldwork suppliers, ensuring that those conducting the fieldwork are fully briefed about the project requirements and perhaps even
training local in-market fieldworkers. It will involve checking that the data collection tool is adapted to suit the market and that all relevant materials are translated
accurately. Back translation, that is, retranslating into the original language, is often
carried out to ensure that any meaning is not lost or distorted. The co-ordinator will
also check that data collection tools in different languages are measuring the same
thing. The role may also involve briefing research executives to ensure that they are
aware of the environmental factors – cultural, social, economic, technological, ethical, legal, regulatory and political – that will affect how the research is conducted or
the data obtained, including for example use of recruitment and sampling techniques,
methods of data collection, wording of questions and use of stimulus material. The
co-ordinator may also be involved in overseeing the handling of the data at the end of
fieldwork, including retranslation of responses to open-ended questions and translations of transcripts of focus groups and in-depth interviews.
39
M02 The Practice of Market Research 31362.indd 39
27/09/2021 21:44
Chapter 2
The practice of market and social research
The ethical, legal and regulatory context
Being able to conduct research, and to collect and analyse data, depends on the
willing co-operation of people. It depends too on their belief that this is done in a
professional, legal and ethical way – that is, that it is carried out honestly, objectively
and, as the ESOMAR Code puts it, without ‘infringing their privacy or creating
disadvantages for those whose data is used in research’. There is growing concern
about the use of data and the importance of privacy, and of being able to control
how and for what end personal data are used. As Stanton (2013) notes, ‘If data are
important enough to collect, they are often important enough to affect people’s lives’.
For research to maintain the support and confidence of the public it is vital that there
is clear guidance for researchers on how to conduct research and use data in a professional, legal and ethical way. This is what we look at next: the ethical, legal and
regulatory context of research and the responsibilities of the research practitioner.
First of all, we look at the general ethical principles that underpin research. We
move on from there to the more formal framing of those principles in codes of conduct. These codes aim to provide a comprehensive framework for self-regulation.
They also ‘require strict adherence to any relevant regional, national and local laws
or regulations, and industry/professional codes of conduct that may set a higher
standard’ (ICC/ESOMAR, 2016). In the final section in this chapter, we look at
three pieces of legislation relevant to research practice: data protection; privacy and
electronic communication; and freedom of information.
What are ethics?
Ethics are moral principles that are used to guide behaviour. The study of ethics is
the study of standards of conduct, of the rights and wrongs of the behaviour of a
particular person or group. Ethical principles are used to set standards of conduct
for groups or professions in how they deal with people. The research profession is no
exception. Ethical standards are important in a research context in order that those
involved in research – researchers, research participants, clients and other users of
research and data and the wider community – know what is and what is not acceptable behaviour in the conduct of research and the analysis of data. While an ethical
code extends to the treatment of clients and other researchers and their work, the
primary focus is the setting of standards of behaviour in relation to research participants, on whom research depends.
Getting co-operation
Why do people agree to take part in research or allow their data to be used for
research purposes? As researchers we ask a lot and we give little in return. We intrude
into people’s lives – we observe, record, measure and question their behaviour, their
attitudes and their opinions, and we analyse, interpret and report on their data. We
often ask them to divulge personal, sometimes sensitive, information. There is little
tangible or intangible reward for taking part – it is rare that research directly serves
the interest of the individual research participant. Given these circumstances it is
40
M02 The Practice of Market Research 31362.indd 40
27/09/2021 21:44
The ethical, legal and regulatory context
unlikely that people would willingly co-operate in research if they felt that they could
not trust the researcher. One way of creating trust is to ensure, and to show, that
research is conducted in an acceptable and ethical way. This is done to a large extent
by publishing and promoting a formal code of conduct by which research practitioners agree to abide. Various research industry bodies have such codes with a view to
enhancing the public’s confidence in research. These self-regulatory codes outline the
rights and safeguards to which research participants are entitled, and make it clear
to researchers the need to behave responsibly when conducting research, particularly
research with vulnerable people.
Ethical principles
The ethical principles that are the basis of most standards of conduct in relation to
research participants are as follows:
voluntary participation;
no harm to the participants;
● informed consent;
● anonymity, confidentiality (privacy);
● transparency;
● not deceiving subjects.
●
●
Voluntary participation
Voluntary participation is the cornerstone of an ethical code: it requires that no one
should be forced or deceived into taking part in research. The researcher should
obtain an individual’s or an organisation’s consent and this consent should be based
on a clear understanding of what the research will involve and how the data collected
will be used. The participant should be told that they have the right to withdraw from
the research at any time and are under no obligation to answer any of the questions
asked.
No harm to the participants
At all times during the conduct of research participants should be treated with respect
and sensitivity. The onus is on the researcher to ensure participants’ emotional as well
as physical well-being and to take steps to ensure that participants are not harmed
or adversely affected as a result of taking part in a research project. It should be
relatively easy to recognise what might cause physical harm to participants, and to
avoid it. For example, if the research involves a participant testing a product, the
client and the researcher should take steps to ensure the safe use of the product, for
instance by providing clear instructions about its use. It is more difficult to recognise
what might cause people emotional or moral harm, however. There are many possible causes – the very fact of being researched can cause anxiety and stress (Gordon
and Robson, 1980). Intruding on participants at unsuitable or inconvenient times can
cause annoyance and distress. Asking questions about sensitive topics can embarrass
41
M02 The Practice of Market Research 31362.indd 41
27/09/2021 21:44
Chapter 2
The practice of market and social research
and distress participants. Reporting or publishing the findings of research in which
individuals or research settings are identifiable can cause embarrassment or distress,
and may damage participants’ self-image or public reputation (Lee, 1992).
Informed consent
The principles of voluntary participation and no harm to participants form the basis
of informed consent. Research should not proceed without the informed consent of
the participants. Participants must give consent freely. Consent should be specific and
participants should be clearly and unambiguously informed about what is involved
and how the data they provide will be processed and used. Under the EU General
Data Protection Regulation (GDPR) 2016 and the UK Data Protection Act (UKDPA)
2018, all organisations must have a documented lawful ground to process personal
data. Consent is often the ground used in research projects: it is relied on for panel
research, online surveys, qualitative and quantitative research based on free-found
recruitment, and random-dialled telephone interviews. We look in more detail at
the legal and ethical requirements in the GDPR, the UKDPA and the MRS Code of
Conduct later in this chapter.
Anonymity and confidentiality
Ensuring anonymity and confidentiality of participants and the data they provide
are two ways in which the well-being and interests of participants can be protected.
Anonymity and confidentiality are often confused, and sometimes taken to mean the
same thing. They are different, and it is important to remember this when such assurances are given. If you promise confidentiality it means that, while you can identify a
particular response with a particular participant, you agree not to do so publicly. If,
however, you promise anonymity it means that you cannot identify a response with a
particular participant. Promises of anonymity are not always possible in research. In
most projects personal data are collected for quality control and verification purposes
and they remain attached to the data record at least until quality checks have been
made. Participants are therefore not anonymous. Data records can be anonymised –
by removing all identifying information – and it is good practice to do this as soon as
possible after quality checks on the data have been made. Nunan and Di Domenico
(2016) examined whether, with the growing power of technology to gather and analyse data and, as a result, the risk of re-identification, anonymisation is still an effective tool. They concluded that research participants should be informed about the
risks. Data records can also be ‘pseudonymised’. Pseudonymisation is a term used to
describe the processing of personal data in a way that means the personal data cannot
be attributed to a specific participant without the use of further information (such as
a unique identifier), provided that the further information is held separately and, as
the MRS Code of Conduct (2019) notes, ‘is subject to technical and organisational
measures to ensure that the personal data is not attributed to an identified or identifiable natural person’. In other words, you should not be able to match the data with
the identifier to identify the individual. The key thing here is to avoid the likelihood
of re-identification. If participants are promised anonymity – in some projects it may
be necessary to do this in order to secure co-operation – no personal data should be
42
M02 The Practice of Market Research 31362.indd 42
27/09/2021 21:44
The ethical, legal and regulatory context
recorded on the data record that could identify them with their responses. This will
mean that quality or verification checks cannot be made.
Allied to promises of anonymity and/or confidentiality and consent is data security: it is essential that you ensure that personal data used in research are protected
from unauthorised access; they should never be disclosed without the consent of the
participant.
Transparency
The person or organisation collecting the data must be transparent about the purpose
of the research, the end use of the data and the fact that anonymity or confidentiality
is or is not promised.
Research can be conducted without the promise of either anonymity or confidentiality. For example, data can be collected on an attributable basis. This, however,
can be done only with the consent of the participant and the data can only be used
for the purpose described to the participant at the time of collection.
Not deceiving subjects
Deceiving subjects in order to get them to take part in research is unethical. For
example, it would be unethical to tell a participant that the interview will take 15
minutes if you know that it will take 45 minutes. Deceiving subjects into thinking
that they are taking part in research when they are not is unethical. The reputation
of research has been harmed, and co-operation rates have declined, as a result of a
range of practices, examples of which are described in the MRS Code of Conduct
(2019): ‘sugging’ or selling under the guise of research; ‘frugging’, fundraising under
the guise of research; ‘plugging’, lobbying for political purposes under the guise of
research; and ‘media-mugging’, creating false media content and commentary (including social media) under the guise of research. Under Rule 4, MRS ‘members must
never undertake any activities, under the guise of research, which aim to manipulate,
mislead or coerce individuals’. Subjects should not be misled or deceived in any way;
it should be made clear to them (transparency again) that they are taking part in bona
fide research and they should be informed honestly about what that research involves.
Ethical considerations beyond participants
Researchers’ ethical responsibilities do not rest only with participants; they also have
ethical responsibilities to the clients or the funders of the research (Lovett, 2001), to
fellow researchers and to the wider community. As a result, researchers have an ethical responsibility to behave in a way that does not cause the public or the business
community to lose confidence in research or the profession. They should not recommend or undertake unnecessary research. They should not make claims about their
qualifications or experience that are untrue, for example. In conducting a project,
researchers are entrusted with confidential and commercially sensitive information –
they have an ethical responsibility not to disclose this information. When proposing or conducting research or reporting on the findings, researchers have an ethical
43
M02 The Practice of Market Research 31362.indd 43
27/09/2021 21:44
Chapter 2
The practice of market and social research
responsibility to be open and honest about the way in which research will be or was
conducted, and its limitations or shortcomings. If difficulties are encountered, or mistakes made, these should be pointed out – to allow others to learn from them, and for
the wider research community to benefit, and to allow others to judge the quality –
the validity and reliability – of the research. Researchers also have an ­ethical
­responsibility to ensure that findings – positive or negative – are reported accurately
and honestly and that they are not used to mislead in any way. They have a responsibility to ensure that they do not use or take advantage of the work of another
researcher without that researcher’s permission.
Questions about how to apply ethical principles will always arise. It is to address
such questions, and to ensure a professional and consistent standard of practice, that
professional bodies representing researchers, such as ESOMAR and MRS, among
others, have developed formal codes of conduct.
Professional codes of conduct
The purpose of a formal code of conduct is to establish good practice and to set
minimum standards of ethical behaviour. The codes aim to do this by ensuring
that important ethical issues are identified and addressed and by trying to clear up
any ambiguity in the interpretation of ethical principles. Most codes cover three
areas: the researcher’s responsibilities to research participants, to those who fund
the research, and to other researchers. Members of ESOMAR, the professional
body representing researchers worldwide (www.esomar.org), are bound by the International Chambers of Commerce (ICC)/ESOMAR International Code on Market,
Opinion and Social Research and Data Analytics. Members of MRS are bound by its
Code of Conduct. The Social Research Association (www.the-sra.org.uk) publishes
­Ethical Guidelines.
These codes, although they may incorporate principles that are covered by legislation, do not replace or take precedence over national law. It is essential to remember
that laws, regulations and codes change: the onus is on the researcher to keep up to
date.
ESOMAR International Code
The latest version of the ESOMAR Code is available at www.esomar.org. It is regularly updated to reflect changes in legislation and changes in the practice of research.
MRS Code of Conduct
The MRS Code, first published in 1954, is also revised and updated regularly to take
account of changes in research practice and in legislation. The principles of the Code
are set out in Box 2.2.
The Code covers general rules of professional conduct and of data accountability.
It reflects the research and data collection process with sections on, for example, commissioning and design, client confidentiality, permission and consent, and participants’
rights. A full version with notes about its interpretation is available at the MRS website
44
M02 The Practice of Market Research 31362.indd 44
27/09/2021 21:44
The ethical, legal and regulatory context
Box 2.2
MRS Code of Conduct
The principles of the Code
MRS Members shall:
1 Ensure that their professional activities can be understood in a transparent manner.
2 Be straightforward and honest in all professional and business relationships.
3 Be transparent as to the subject and purpose of data collection.
4 Ensure that their professional activities are not used to unfairly influence views and
opinions of participants.
5 Respect the confidentiality of information collected in their professional activities.
6 Respect the rights and well-being of all individuals.
7 Ensure that individuals are not harmed or adversely affected by their professional
activities.
8 Balance the needs of individuals, clients, and their professional activities.
9 Exercise independent professional judgement in the design, conduct and reporting
of their professional activities.
10 Ensure that their professional activities are conducted by persons with appropriate
training, qualifications and experience.
11 Protect the reputation and integrity of the profession.
12 Take responsibility for promoting and reinforcing the principles and rules of the
MRS Code of Conduct.
Source: MRS Code of Conduct (2019). Used with permission.
(www.mrs.org.uk). For a more detailed framework for the interpretation of the principles of the Code, MRS also publishes a series of Binding Guidelines, Regulations and
Guidance documents, also available at its website. These interpret the Code for the
practice of different types of research. In addition, MRS operates a free confidential
email advice service called Codeline, staffed by its Standards team. It offers practical
advice to members and MRS Company Partners about the Code, and the Regulations
and Guidelines series. You can submit a query at codeline@mrs.org.uk.
In a study of ethical decision making among market researchers in New Zealand,
Yallop and Mowatt (2016) found that ethical codes ‘do not appear to have a significant effect on ethical behaviour’. They found that ethical issues tend to be dealt with
through communication and close, trusting relationships with clients, consultation
with peers and use of the researcher’s personal ethics and values.
Research and data protection legislation
While professional codes take account of data protection legislation, it is important
for research practitioners and data users to be familiar with the legislation in more
detail. The collection and processing of personal data is a key part of research and the
45
M02 The Practice of Market Research 31362.indd 45
27/09/2021 21:44
Chapter 2
The practice of market and social research
protection of that personal data is essential. The European Union (EU) General Data
Protection Regulation (GDPR) 2016 and the UK Data Protection Act (DPA) 2018
require organisations to be accountable in protecting the privacy rights of individuals. The GDPR is directly applicable in all EU Member States (without any further
implementing domestic law). The UK’s national data protection framework reflects
the GDPR but is applied according to the UK DPA 2018. The GDPR, however, still
applies to any organisation in the UK (or elsewhere) that deals with data relating to
EU residents. In relation to the EU, the UK is a ‘third country’ and so the extra territorial provisions in the GDPR apply. For ease of reference from hereon, reference
to GDPR serves to include the UK DPA 2018.
Any organisation monitoring, storing or analysing personal data must comply
with the legislation: failure to do so may result in significant fines. In the UK
the Information Commissioner’s Office (ico.co.uk), an independent body, is the
authority responsible for upholding information rights. It can audit an organisation to determine if it is compliant with the legislation. The organisation must be
able to show that it has a comprehensive data protection compliance programme
with policies, procedures and compliance infrastructure in place. It must have the
following:
documentary evidence of consent;
clear records of all data processing activities, including the purposes for which data
are being processed and the legal basis for processing;
● the categories of data subjects and personal data within the organisation;
● any transfers of data to ‘non adequate’ countries and the safeguards used to protect
the data; and
● a general description of technical and organisational security measures in place.
●
●
In the following sections we look briefly at the principles of data protection, what
constitutes personal data, and the rights of data subjects. We look at the legal bases
for processing personal data in research. We look at the roles of data controller and
data processor. We look at data protection impact assessments (DPIA) and privacy
notices. Please note that the information provided here is for general guidance only
and does not constitute legal advice. It is not intended to address the circumstances
of any particular project.
Principles of data protection
There are six principles of data protection:
1 Lawfulness, fairness and transparency: Personal data is processed lawfully, fairly
and in a transparent way.
2 Purpose limitation: Personal data is obtained for specified, explicit, legitimate
purposes and not further processed in a manner that is incompatible with
those purposes. Further processing is allowed for archiving, scientific, statistical
and historical research purposes.
3 Data minimisation: Personal data processed is adequate, relevant and limited to
what is necessary.
4 Accuracy: Personal data is accurate and, where necessary, kept up to date.
46
M02 The Practice of Market Research 31362.indd 46
27/09/2021 21:44
The ethical, legal and regulatory context
5 Storage limitation: Personal data is not kept longer than is necessary (but data
processed for archiving, scientific, statistical and historical research purposes can
be kept longer subject to safeguards).
6 Integrity and confidentiality: Suitable technical and organisational measures are
put in place to guard against unauthorised or unlawful processing, loss, damage
or destruction.
Personal data
Personal data are information relating to an identified or identifiable natural person
who can be identified directly or indirectly by that data on their own or together
with other data. These can be demographic details, occupation, address, location,
or physical characteristics, and sound and video recordings and still photographs or
images, as they can be linked to a person.
There is a special category of personal data that is sensitive personal data, data
relating to religious or philosophical beliefs, health, racial or ethnic origin, trade
union membership, political beliefs, sex life or sexual orientation, genetic data and
biometric data (where used for identification purposes) and health data. For specific
definitions of genetic data, biometric data and health data, see the ICO website. This
category of special data is subject to greater restrictions than other types of personal
data (although data about criminal convictions and offences is treated separately and
is subject to tighter controls).
Personal data can come directly from research participants, from third parties,
from client customer databases, and from publicly available data. It is important to
note the difference between data that identifies the participant and other data. Identifiable demographic details are personal data; responses given during data collection
are only personal data when they can be linked to the demographic details or if the
responses themselves contain identifiable details. Sound and video recordings and still
images are personal data because they can easily be linked to a person.
Under GDPR personal data should be processed to the point where the data subjects cannot be identified.
Data subjects and their rights
Data subjects are those living individuals to whom the personal data you collect and
hold relates. These subjects have rights that allow them to control and protect the
use of their data. These rights are as follows:
Right to be informed
Right to access data
● Right to data portability
● Right to erasure (to be forgotten)
● Right to object
● Right not to be evaluated by automated decision making
● Right to rectification (of inaccurate data)
● Right to restrict processing.
●
●
47
M02 The Practice of Market Research 31362.indd 47
27/09/2021 21:44
Chapter 2
The practice of market and social research
A system must be in place to ensure that your organisation can address these rights.
You need to ensure, for example, that your organisation’s IT system and its organisational processes can deal with subject access requests in the time frame required –
within one month of the request.
Processing personal data
‘Processing’ personal data in the context of data protection refers to any operation
or set of operations performed on personal data, either automatically or manually.
It includes the following:
gathering;
recording;
● organising;
● storing;
● altering or adapting or combining;
● retrieving;
● consulting;
● using;
● disclosing;
● transmitting;
● disseminating or otherwise making available; and
● erasing or destroying.
●
●
Whatever processing of personal data is done, it must be necessary and proportionate and it must be conducted under the auspices of the data controller as the person
who ensures compliance with data protection policy. Processing must comply with
the principles of data collection outlined above. It must be:
for specified, explicit, legitimate purposes (and personal data should not be further
processed in a way that is incompatible with those purposes);
● processed securely, fairly and lawfully, and accurately;
● kept up to date;
● processed in line with the rights of the data subjects;
● processed in a form that allows identification for no longer than is necessary
for the purposes for which the data were collected or for which they are further
processed;
● retained for no longer than is necessary for the purposes for which they were
­collected (once this retention period has lapsed, you must securely delete the data
or ensure that they are securely deleted).
●
Lawful bases for processing personal data
To ‘process’ personal data you must have a valid, lawful basis. There are six lawful
bases. The most appropriate basis will depend on the purpose of the processing and
the relationship with the individual. Most lawful bases require that the processing is
‘necessary’ for a specific purpose. You must determine your lawful basis before you
begin processing and you should document it. You must provide a privacy notice to
48
M02 The Practice of Market Research 31362.indd 48
27/09/2021 21:44
The ethical, legal and regulatory context
data subjects and this must include your lawful basis for processing and the purposes
of processing. The lawful bases appropriate for most research projects are ‘consent’
of data subjects or ‘legitimate interests’ of the data controller or a third party. For
public sector research projects, the lawful basis can be ‘public task’, and it can be
‘performance of a contract’ for aspects of panel research. According to the MRS
(2019), researchers processing personal data only need to have a processing ground.
However, if you are processing special category data you need a processing ground
for the personal data and a processing condition for the special category data.
Consent
Consent is suitable for a range of research methods including the following:
qualitative and quantitative research based on free-found recruitment;
random-dialled telephone interviews;
● online surveys;
● audience measurement surveys;
● customer satisfaction research that is not done using client databases; and
● panel research.
●
●
Consent is also suitable for tracking-based digital market research. Please note, however, that at time of writing the EU Commission is proposing to reform its ePrivacy
Directive and the proposed ePrivacy regulation will have a significant impact on
online research. Please check for the most update advice with your professional body
and your organisation’s Data Protection Officer. If you are processing sensitive data
or special category data then you must obtain explicit consent and you must follow
any other relevant national laws that apply to that type of data. If you are doing
research using profiling or segmenting and making decisions based on this, then
additional restrictions will apply.
If you use consent for data collection it must be freely given, specific, informed
and unambiguous with a clear, affirmative action or statement. The data subjects or
research participants must be given accurate and full information – in clear, plain
language – about the following:
the nature of the data to be processed;
the purpose of processing;
● details of to whom the data may be transferred;
● time limits on storing the data;
● rights of the data subject; and
● the absence of negative consequences if consent is not given.
●
●
Consent may be given in writing or electronically or verbally. The organisation’s data
controller must be able to show that consent was obtained. Age verification is needed
for parental consent for children.
Consent is also suitable as a lawful basis for scientific and statistical research purposes, including social research projects where the results will be published, public
health research and longitudinal studies. If you are using broad consent for scientific
and statistical research purposes, then you need to be aware that broad consent can
be obtained from participants where it is not possible to identify all purposes of
personal data processing for scientific purposes at the time of data collection. You
49
M02 The Practice of Market Research 31362.indd 49
27/09/2021 21:44
Chapter 2
The practice of market and social research
must follow ethical standards for scientific research and implement technical and
organisational safeguards.
Legitimate interests
Researchers can also carry out research based on the legitimate interests of the data
controller (or third parties) except where such interests are overridden by the fundamental rights and freedoms of the data subject. Take particular care in considering
the rights of children. The legitimate interests ground is suitable for the following
types of research with existing data:
customer satisfaction research on existing customer databases;
qualitative or quantitative research using customer databases;
● research using existing datasets or third-party data (for example, social media
analytics); and
● secondary data analysis or data analytics on loyalty card data.
●
●
The legitimate interests ground can also be used to process personal data further –
within the reasonable expectations of the data subjects. If the personal data being
processed have not been collected by consent then processing of that data can only
be for a ‘compatible purpose’. The key issues in deciding whether the purpose is
compatible, according to guidance given in the Efamro/ESOMAR/MRS Guidance
Note (2017), include (but are not limited to) the following:
link between purposes;
context and relationship between the data subject and the data controller;
● nature of the personal data;
● possible consequences of processing; and
● safeguards in place (e.g. encryption, pseudonymisation).
●
●
The roles of data controller and data processor
If you are processing personal data during a research project you need to establish if
you are acting as a data controller, a joint data controller or a data processor. Both
controllers and processors have direct statutory liability under data protection laws.
Identifying your role will help you determine your obligations under GDPR and data
protection legislation. It should be done on a case-by-case basis.
The data controller is the person with greater responsibility. The controller is the
person who determines the purpose and manner for which the data are collected
or used and processed. This could be the researcher, or it could be the client, or it
could be both. If, say, you are the client and you are working with a researcher and
you jointly determine the purposes and manner in which the data are collected and
used, then you may be a joint data controller. If you process data on behalf of a data
controller, for example preparing transcripts or translations or processing data for
a client, then you are a data processor. You may determine the means of processing
the data but you have no role in determining its use. The data controller must be able
to show compliance with data protection principles and policy and be able to implement suitable technical and organisational measures. The processor must be able
50
M02 The Practice of Market Research 31362.indd 50
27/09/2021 21:44
The ethical, legal and regulatory context
to provide sufficient guarantees to implement suitable technical and organisational
measures including data security. Processors act on the instructions of the controller.
If it is the organisation that determines the purpose of processing and the means of
it, then it will be considered to be a controller and have the liabilities of a controller.
Processors can be directly liable to controllers under the terms of the contract as well
as subject to the enforcement regime of the data protection laws. The supervisory
authority such as the UK’s ICO can take action against controllers and processors
and individuals can bring claims for compensation against processors and controllers.
The roles of each party in a research project should be written up in the contract and
on other relevant documents between client and researcher.
Box 2.3
Obligations of controllers and processors
The data controller must do the following:
Conduct data protection impact assessments (DPIAs) when required
Provide a point of contact for data subjects
● Choose and audit appropriate processors
● Enter into suitable contracts with processors
● Register and pay applicable data protection fee to the ICO.
●
●
The joint controller must do the following:
●
●
Determine their respective responsibilities by agreement
Communicate the content of the agreement to data subjects.
The data processor must do the following:
Act on written instructions of the data controller
Co-operate with supervisory authorities such as the ICO
● Ensure security of its processing
● Keep records of its processing activities
● Notify any personal data breaches to the controller
● Seek written approval to appoint a sub-processor
● Seek approval to make data transfers outside of the European Economic Area (EEA)
● Reflect the same contractual obligations it has with the controller in a contract with
any sub-processor
● Be jointly liable with the controller for certain breaches.
●
●
Sub-processors must do the following:
●
●
Incorporate the same contractual obligations the processor has with the controller
Assume the same responsibilities and liabilities as the processor.
Although they have specific statutory responsibilities, controllers and processors also
have significant common responsibilities including:
●
●
Appointment of a data protection officer, as required
Appointment of a representative if based outside the EEA
51
M02 The Practice of Market Research 31362.indd 51
27/09/2021 21:44
Chapter 2
The practice of market and social research
Maintaining detailed records as required
Implementing appropriate technical and organisational measures
● Enshrining privacy by design and default
● Recording and documenting the lawful basis for the data processing activities
● Mandatory notification of a data breach for riskier data breaches
● Ensuring appropriate contracts throughout the supply chain.
●
●
Source: MRS (2018) Data Protection & Research: Guidance Note on Controllers and Processors. Used with
permission.
Data protection impact assessment
A DPIA is an assessment undertaken to identify potential areas of non-compliance
and to act to minimise the risk. It must be completed before starting any new, potentially high risk, processing activity, for example processing sensitive data. The Information Commissioner’s Office (ICO) notes that it is good practice to do a DPIA for
any major project which involves the processing of personal data. Your DPIA should
include the following:
a description of the processing activity:
the purpose of the processing;
● the risks identified;
● the measures taken in response to those risks; and
● the advice, if any, of the data protection officer.
●
●
Privacy notice
A privacy notice or privacy policy is a summary of the organisation’s practices in relation to privacy of personal data. It should describe the ways in which the organisation
collects, uses, discloses and manages personal data. It must contain the following in
clear language:
the identity and contact details of the data controller;
the purposes and legal basis for processing the data;
● if it is legitimate interest, a statement of what the legitimate interest is;
● the recipients or categories of recipient of the data;
● if there are to be cross border transfers of the data and, if so, the basis on which
these are made;
● how long the data will be retained;
● an outline of individual rights;
● whether there is a statutory or a contractual requirement to process the data; and
● the existence of any automated decision making and logic.
●
●
A privacy notice must suit the context in which it is used. For an online research project, for example, it could be a layered document with an overview of the notice with
directions to the full notice. In telephone research it may be necessary to direct the
participant to a website containing further information, or to a particular individual,
52
M02 The Practice of Market Research 31362.indd 52
27/09/2021 21:44
The ethical, legal and regulatory context
should they have any queries, or you might give some of the necessary information
at the start of the call and some at the end.
Research and the Privacy and Electronic Communications
Act 2003
The Privacy and Electronic Communications Act 2003, and amendments to it, which
took effect in May 2011, have implications for research practice. The Act is concerned with two things: unsolicited emails and text messages; and consent for the
use of cookies by websites.
Unsolicited emails and texts
In short, unsolicited commercial emails (spam) and text messages (SMS) to an
individual must have that individual’s prior agreement – they must ‘opt in’ or
give ‘active consent’. An exception to this is where there is an existing customer
relationship – unsolicited communication about similar products or services can
be sent to existing customers unless and until they ‘opt out’. Emails and SMS (and
phone calls) for research purposes are not defined as commercial communications
within the legislation. However, researchers should be prepared for feedback and/
or questions regarding the legislation from participants who are not aware of this
distinction.
Cookies
Since May 2011 when the Act was amended, there is now a requirement to obtain
consent for the placement of cookies on users’ machines or devices (Ryan, 2011).
MRS takes the view that these rules mean the need for greater transparency about
the use of cookies (in invitations and introductions to online research and in terms
and conditions for panels and research communities) so that potential research
participants can make an informed choice about whether or not to take part in
research.
Guidelines on the Privacy and Electronic Communications Regulations are available from the MRS website along with some frequently asked questions about monitoring interviews and the Regulation of Investigatory Powers Act 2000 at https://
www.mrs.org.uk.
Freedom of information
The Freedom of Information Act 2000 applies to information held by or on behalf
of public bodies. Researchers may encounter Freedom of Information Act issues in
two ways: because they have provided information to a public body (for example,
a research proposal or tender document, or a contract or a research report); and
because they may hold information on behalf of the public body (for example, the
53
M02 The Practice of Market Research 31362.indd 53
27/09/2021 21:44
Chapter 2
The practice of market and social research
product of work done for the body that remains the property of the public body).
For information on how to deal with a Freedom of Information Act request, see
the MRS Guidance document available at www. mrs.org.uk.
Chapter summary
●
●
●
●
●
●
The research industry is made up of suppliers and those who buy and/or use
research. There are several kinds of supplier, including the full-service agency, the
specialist agency and the independent consultant.
In-house or clientside researchers are those who work within a client organisation.
The role varies. It can involve internal consultancy, advising on the use of
research, integrating research and other evidence from a variety of sources, and
ensuring that research and other data are converted to information and applied
effectively. The role may also involve providing and/or commissioning external
research.
The role of the agency researcher is to design and manage a project from the
initial client briefing to analysis, interpretation and communication of the findings
and their implications to the client.
Ethics are moral principles used to guide behaviour. Ethical principles are used to
set standards of conduct for groups or professions in how they deal with people.
They are important in a research context so that those involved – researchers,
participants, clients and other users of research and the wider community – know
what is and what is not acceptable in the conduct of research.
Various professional bodies, including MRS and ESOMAR, represent researchers
and the industry to the wider world and aim to ensure that research is conducted
in a professional and ethical manner through the publication of codes of conduct
or practice. While these codes may incorporate principles that are covered by
legislation, they do not replace or take precedence over legislation.
The ethical principles that are the basis of most standards of conduct in relation
to research participants are the following:
– voluntary participation;
– no harm to the participants;
– informed consent;
– anonymity, confidentiality (privacy);
– transparency;
– not deceiving subjects.
●
Research operates in a legal and regulatory environment. Key legislation includes
GDPR 2016, UK DPA 2018, the Privacy and Electronic Communications Act 2003
and the Freedom of Information Act 2000.
54
M02 The Practice of Market Research 31362.indd 54
27/09/2021 21:44
References
●
●
●
An organisation must be able to show that it has a comprehensive data protection
compliance programme in place.
Data subjects are those living individuals to whom the personal data you collect
and hold relates. They have rights that allow them to control and protect the use
of their data.
Processing of personal data must be:
– for specified, explicit, legitimate purposes (and personal data should not be
further processed in a way that is incompatible with those purposes);
– processed securely, fairly and lawfully, and accurately;
– kept up to date;
– processed in line with the rights of the data subjects;
– processed in a form that allows identification for no longer than is necessary
for the purposes for which the data were collected or for which they are further
processed;
– retained for no longer than is necessary for the purposes for which it was
collected (and once this period has lapsed, you must securely delete the data).
●
To process personal data you must have a valid, lawful basis. The lawful
bases appropriate for most research projects are ‘consent’ of data subjects or
‘legitimate interests’ of the data controller or a third party.
Exercises
1 Think about the next project you’ll be involved in and a) list the stages in that
project; b) describe the roles of the project team members, including your
own role; and c) identify any ethical, legal and regulatory issues that you might
encounter and describe how you would handle these.
References
Bainbridge, J. (2018) ‘Smooth operator’, Impact, 22, pp. 50–4.
Cluley, R., Green, W. and Owen, R. (2020) ‘The changing role of the marketing researcher
in the age of digital technology: Practitioner perspectives on the digitization of marketing
research’, International Journal of Market Research, 62, 1, pp. 27–42.
Efamro/ESOMAR (2017) General Data Protection Regulation (GDPR) Guidance Note
for the Research Sector: Appropriate Use of Different Legal Bases under the GDPR,
https://www.esomar.org/uploads/public/government-affairs/position-papers/EFAMROESOMAR_GDPR-Guidance-Note_Legal-Choice.pdf (Accessed 18 May 2021).
55
M02 The Practice of Market Research 31362.indd 55
27/09/2021 21:44
Chapter 2
The practice of market and social research
Ellwood, R. (2011) Conference notes. ‘Not delivering “good enough” but “better than
before”’, International Journal of Market Research, 53, 2, pp. 284–6.
Eremenko, K. (2018) Confident Data Skills, London: Kogan Page.
Gordon, W. and Robson, S. (1980) ‘Respondent through the looking glass: towards a better
understanding of the qualitative interviewing process’, Proceedings of the Market Research
Society Conference, London: MRS.
Gray, R. (2019) ‘A brewing challenge’, Impact, 24, pp. 47–50.
ICC/ESOMAR (2016) International Code on Market, Opinion and Social Research and Data
Analytics, Amsterdam: ESOMAR.
Lee, R. (1992) Doing Research on Sensitive Topics, London: Sage.
Lovett, P. (2001) ‘Ethics shmethics: As long as you get the next job. A moral dilemma’, Proceedings of the Market Research Society Conference, London: MRS.
MacKay, R. and Oldford, R. (2000) ‘Scientific method, statistical method and the speed of
light’, Statistical Science, 15, 3, pp. 254–78. (Accessed 1 September 2020).
MRS (2020a) Advanced Certificate in Market & Social Research Practice. Syllabus & Assessment Guidelines, London: MRS.
MRS (2019) Code of Conduct, London: MRS.
Nunan, D. and Di Domenico, M. (2016) ‘Exploring reidentification risk: Is anonymisation a
promise we can keep?’, International Journal of Market Research, 58, 1, pp. 19–34.
Nunan, D. and Di Domenico, M. (2019) ‘Rethinking the market research curriculum’, International Journal of Market Research, 61, 1, pp. 22–32.
Phillips, T. (2018) ‘Keeping a constant customer dialogue’, Impact, 21, pp. 32–42.
Ryan, B. (2011) ‘Laying down the law’, Research, July, 542, pp. 34–5.
Social Research Association (2003) Ethical Guidelines (http://www.the-sra.org.uk).
Spiegelhalter, D. (2019) The Art of Statistics, London: Pelican.
Stanton, J. (2013) An Introduction to Data Science Conference, Version 3 https://docs.google.
com/file/d/0B6iefdnF22XQeVZDSkxjZ0Z5VUE/edit?pli=1 (Accessed 26 March 2020).
Tarran, B. (2012) ‘Tempus Fuguitt’, in Research, August, 555, pp. 22–3.
Yallop, A. and Mowatt, S. (2016) ‘Investigating market research ethics: an empirical study of
codes of ethics in practice and their effects on ethical behaviour’, International Journal of
Market Research, 58, 3, pp. 381–400.
Recommended reading
For more on research and analytics job roles, have a look at research recruitment websites (e.g.
https://www.researchjobfinder.com).
For more on research ethics, try:
Bulmer, M. (ed.) (1982) Social Research Ethics, London: Macmillan.
Miller, T., Birch, Mauthner, M. and Jessop, J. (eds.) (2012) Ethics in Qualitative Research,
2nd edition. London: Sage.
To look at the UK Data Protection Act 2018, see the UK Government website www.legislation.
gov.uk
To look at the GDPR, see the EU legislation site, see http://eur-lex.europa.eu/legal-content/EN/
TXT/PDF/?uri=CELEX:32016R0679&from=en3
56
M02 The Practice of Market Research 31362.indd 56
27/09/2021 21:44
Recommended reading
The European University Institute has a very useful guide to data protection and research:
European University Institute (2019) Guide on Good Data Protection Practice in Research,
3rd edition. Florence: European University Institute. It can be found here: https://www.
eui.eu/Documents/ServicesAdmin/DeanOfStudies/ResearchEthics/Guide-Data-ProtectionResearch.pdf
57
M02 The Practice of Market Research 31362.indd 57
27/09/2021 21:44
Chapter 3
Types of data and research
Introduction
In Chapters 1 and 2 we looked at what research is, the contexts in which it
is conducted, the uses to which it is put and its value and limitations, and at
the skills and tasks involved. The aim of this chapter is to introduce you to the
terminology used to describe and classify different types of data and research.
Examples illustrate the different types of data and research in context.
Topics covered
Big data and small data
● Structured, unstructured and semi-structured data
● Primary and secondary data
● Primary and secondary research
● Qualitative and quantitative data
● Qualitative and quantitative research
● Continuous and ad hoc research
● Active and passive data collection; interviewing and observation.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project; and Topic 3: Selecting the research design and
planning the approach.
M03 The Practice of Market Research 31362.indd 58
27/09/2021 21:45
What you should get from this chapter
At the end of this chapter you should be able to:
understand the terminology used to describe different types of data and research;
● understand the basics of each type; and
● be aware of the main uses of each type.
●
59
M03 The Practice of Market Research 31362.indd 59
27/09/2021 21:45
Chapter 3
Types of data and research
Types of data and research
In Chapter 1 we saw how research is often classified according to its context: consumer research or business-to-business research (B2B) or social research. There are
several other useful – and very common – ways of classifying or describing data and
research. These are as follows:
the size of the data – big data and small data;
the format of the data – structured, semi-structured and unstructured data, and
simple and complex;
● the source or producer of the data – primary and secondary data (and primary and
secondary research);
● the nature of the data – qualitative and quantitative;
● the nature of the research – qualitative and quantitative;
● the mode of data collection – continuous and one-off or ad hoc research; and
● the method of data collection – active or passive; observation or interview; personal or self-completion; in face-to-face (in person or remote), telephone, online
and postal.
●
●
The size of the data
The practice of research encompasses ‘big data’ projects using data mining and analytics as well as ‘small data’ studies using traditional research methods. Big data
projects use the sort of data that Kitchin (2014) defines as ‘easy to ensnare – data that
are openly expressed (what is typed, swiped, scanned, sensed, etc; people’s actions
and behaviours; the movement of things) – as well as data that are the “exhaust’’, a
by-product, of the primary task/output’. Big data can also be described in statistical
terminology (Spiegelhalter, 2019): they are big in terms of the number of examples,
denoted as n; and big in terms of the number of characteristics or attributes, denoted
as p. So big data will have very large n – in the millions and billions, and very large
p – in the millions. In contrast, in traditional research, a large quantitative survey
might have n in the thousands and p in the hundreds.
Box 3.1
Byte size
One binary digit (0 or 1) is a bit – this is the smallest piece of data you can get. One
byte is made up of eight bits. Big data has been defined as exabyte and terabyte size,
and beyond. Think about the data allowance on your phone – it’s probably something
in gigabytes (GB): 8GB or 16GB or 32GB. A gigabyte is 1,000,000,000 bytes. If you
60
M03 The Practice of Market Research 31362.indd 60
27/09/2021 21:45
The size of the data
have 1,000 gigabytes, you have a terabyte. If you have 1 billion gigabytes you have
an exabyte or one quintillian bytes. With the Internet of Things (IoT), volumes of data
are now measured in zettabytes – 1,000 exabytes is a zettabyte – and yottabytes –
1,000 zettabytes is a yottabyte. Compare this with the size of a survey dataset:
for ­example, a dataset from the Life and Times Survey is about 600 kilobytes or
0.0006GB (www.ark.ac.uk/nilt/).
Big data are also defined in terms of the three Vs: huge in volume; diverse in variety;
and high in velocity (Laney, 2001). Others propose five Vs, adding veracity or
uncertainty of the data and value (Marr, 2014); and others seven, adding variability
and visualisation (McNulty, 2014). Kitchin (2013) proposes a classification that
includes volume, velocity and variety and exhaustivity, resolution, indexicality,
extensionality and scalability. Kitchin and McArdle (2016) found in applying
this classification to 26 datasets from a range of domains that there are ‘different
“species”’ of big data, that not everything that is described as big data in the literature
has the characteristics of the three Vs – volume, velocity and variety. boyd and
Crawford (2012) also believe ‘big data’ to be a ‘poor term’. They argue that size is
not the defining characteristic since some of what is called big data – they cite Twitter
messages about a topic – are not as big as datasets that are not considered big data,
such as census data. Instead they define big data as ‘a cultural, technological, and
scholarly phenomenon that rests on the interplay of technology. . ., analysis . . . and
mythology’. They see it as being about the ‘capacity to search, aggregate, and
cross-reference large data sets’. Murthy et al. (2014) present a similar view with a
six-part classification that reflects not the characteristics of the data but how they
are processed. This is a very useful way of thinking about big data for our purposes.
The categories in their classification are as follows:
1 Data
Data: ‘temporal latency for analysis’ or the time it takes for analysis – real-time,
near real-time and batch
Data structure: structured, semi-structured and unstructured
2 The compute infrastructure – batch or streaming
3 The storage infrastructure – SQL, NoSQL, NewSQL
4 T
he analysis – supervised, semi-supervised, unsupervised or reinforcement machine
learning; data mining; statistical techniques
5 Visualisation – maps, abstract, interactive, real-time
6 Privacy and security and the management of these.
In summary, because of the nature of big data, there are implications for the way in
which they are stored and processed and for the sort of tools and techniques required
to analyse them. In Chapter 8 we look at sources of existing data, including big data;
in Chapter 9 we look at evaluating them for use; and in Chapter 18 we look at mining
and analysing big data. Industry Insight 3.1 gives an example of a research project
that used big and small data to generate insight.
61
M03 The Practice of Market Research 31362.indd 61
27/09/2021 21:45
Chapter 3
Types of data and research
Industry Insight 3.1
The Fragrance Cloud Inspiration Ecosystem
Introduction
The end result
Unilever, a British-Dutch transnational consumer
goods company, and Ipsos, a global market
research and consulting firm, created an intelligent system of insights, called the Fragrance
Cloud Inspiration Ecosystem, a sort of ‘Google
for fragrance’. It has proved very useful in helping
to uncover business opportunities for Unilever.
In terms of content, over 56 insights were uncovered spanning eight opportunity territories in
five weeks, with no primary research. The end
result is a platform that gives access to insights
and content for business action. It is cloud-based
and enables connected, relevant information to
be sent to users. Insights are brought alive with
storytelling in 3D multimedia. Certain features of
this Fragrance Cloud system are special or new.
It integrates multiple datasets including research
reports, sales reports, data tables, infographics,
social data, videos and pictures. It also integrates
big and small data – big data capture for looking
at metrics but also anecdotes, comments, shares,
videos, hashtags and narratives around fragrance
and emotions. It also contains an auto-curation
process in order to have an on-going feed of
internal and external data streams on relevant
topics. It is an open platform and so allows Unilever to engage all its partners – internal (marketing, R&D, research, sales, channel) and external
(research, advertising, digital, PR, other partners).
Where it started
Unilever needed to understand fragrance for
­Comfort, the flagship fabric conditioner, in order
to deliver inspiring and compelling communication
insights, and to go beyond the clichés of flowersclothes-fragrances. To do this meant digging deep
into potentially overwhelming volumes of data
and research to unearth the most valuable stories.
Previous ‘traditional’ research studies showed the
challenge of infusing emotional engagement into a
functional category like laundry. This quest for emotions in the context of fragrance and laundry was
not new. The topic had been researched ­repeatedly –
in various different ways. What could we explore
that had not already been investigated and understood? We had done focus groups, ethnography,
expert interviews, sensitivity panels, even neuroscience . . . what more could we possibly do? Big data
was an obvious source of consumer-generated stories and information. How could we use it?
The initial step was to conduct a pilot study
to bring together insights from a variety of data
and insight sources, and to craft stories, hosting
them on a platform called the Insight Cloud. The
pilot study covered five markets – Brazil, Indonesia, Thailand, UK and Vietnam – and took five
weeks. It involved mining all recent studies (qualitative and quantitative), thought notes, sales data
and desk research to explore themes on fragrance.
It was supplemented with a social listening programme designed to capture themes in local language across the five markets and from different
sources such as social networks, blogs, videos, and
brand communication.
The system in action
Unilever has begun to use the Fragrance Cloud as
a sort of ‘Google for fragrance’ and instant ‘hackathons’ now take place with marketing, advertising and fragrance house stakeholders. It has saved
significant time and cost in terms of exploratory
or developmental research. It has helped inform
new initiatives around concept and communication development. It offers Unilever’s multidisciplinary teams the following advantages:
1 Insights on demand: a network of curated
insights from multiple sources, in one place
2 Collaboration: being an open-source platform,
the Fragrance Cloud has made collaboration
around business tasks easier – partners are able
to collaborate towards a common goal
3 Speed to market: time is saved on rewinds to process multiple reports for synthesis and insights
62
M03 The Practice of Market Research 31362.indd 62
27/09/2021 21:45
The format of the data
4 Zero primary research: an efficient use of existing research, no need to ask questions
5 Early warning system: yields strong, early ideas by
identifying outliers, unique clusters, combinations
6 Auto-curation: organic build up and continuity
of all types of data – all in one approach; excellent for dealing with massive, ever-increasing
quantities of social and other data
7 Flexibility: for use in concept writing, workshops, subject understanding.
Unilever now has eight emotional mega-territories
for fragrance – with directional sizing and market footprints – for communication and product
development. They have begun to work on over
20 business opportunities with extensive execution leads from brands, popular culture and usergenerated content.
Source: Adapted from Ipsos and Unilever Consumer & Market
Insights, ‘The Fragrance Cloud – an Inspiration Ecosystem’, MRS
Awards 2017.
The format of the data
As noted above, data can be structured, unstructured or semi-structured, or a combination (Marr, 2019); and they can be complex or relatively simple. It is important
to understand the format of the data you are working with because it has an impact
on the type of analysis you can do.
Structured data are data that are organised and formatted in a systematic way
in columns and rows. Data held in datasets, databases and data warehouses are
structured data. Machine- and human-generated data can come in structured format. Examples of machine-generated structured data include transactional data from
websites and apps, data from wearables and smart devices including location data,
and data from RFID tags in products. Human-generated structured data includes
data held in customer relationship management (CRM) systems, and survey datasets.
Structured data, depending on their size, can be analysed using standard analysis
techniques and artificial intelligence (AI) techniques.
Unstructured data are data that are not organised in a pre-defined way. As
with structured data, they can be generated by humans or machines. Examples of
unstructured human-generated data include data from communications (phone or
video recordings, images, email); documents produced by business applications
(presentations, reports), books, journals, articles; data from websites and social
media, user-generated content (Verhaeghe and Nola, 2010) or consumer-generated
content (Hardey, 2011); and other text files such as transcripts and responses to
open-ended questions. Many of these examples, however, as you will know yourself
from writing emails and preparing reports and articles, do have some basic elements
of structure within them – headings and key words, for example. Examples of
unstructured machine-generated data include data from sensors (weather stations,
satellites) and surveillance (images and media files from CCTV, from satellites).
Unstructured data are stored in data lakes, particular forms of databases, and
in data warehouses. To analyse this type of data, AI and machine learning (ML)
approaches may be needed.
Semi-structured data are data that are not organised in the formal, standardised
way that you see in a dataset or database but they do have some structure, or they
have elements of unstructured and structured data within them. This structure is typically in the form of markers or tags that identify elements of the data, for example
63
M03 The Practice of Market Research 31362.indd 63
27/09/2021 21:45
Chapter 3
Types of data and research
the time codes or geo-location tags included in photos, or ‘metadata’, the data about
data that are used to describe parts of the data or the whole dataset.
In terms of complexity, data may be simple or complex, or anywhere in between.
At the simple end of the spectrum are structured data from a single source; at the
complex end is a dataset formed of multiple sources of fused or integrated structured,
semi-structured and unstructured data.
The source or producer of the data
Primary research is research designed to generate or collect data for a specific problem; the data collected – primary data – do not exist prior to data collection. Secondary data, on the other hand, are data that were originally collected for a purpose
other than the current research objectives – in revisiting them you are putting the data
to a second use. Searching for and using secondary data is called secondary research.
Analysing secondary data is called secondary data analysis. Sources of secondary data
are wide-ranging and include sources from within an organisation (internal data)
and those beyond it (external data). In the project described in Industry Insight 3.1,
Unilever made use of existing internal data – from previous research and sales – and
external data from social media.
Primary research and primary data
The role of primary research is to generate data to address information needs in
relation to a specific problem or issue. For example, imagine you are interested in
understanding how customers have reacted to changes to the service you provide.
There are no pre-existing data available – you need to conduct primary research.
Primary research may be exploratory, descriptive and/or causal; qualitative or
quantitative; continuous or ad hoc. So primary research produces primary data.
The primary research can be done actively and passively; online, in person face to
face, by telephone, by post; on a one-off or on a continuous basis; and in almost
any market or on any issue. The process starts with identifying a business problem
and information needs and research is designed to collect or gather or capture the
data that will address those information needs. Industry Insight 3.2 gives several
examples of primary research commissioned by Formula One (F1) motor sport.
Industry Insight 3.2
Finding the right formula
Formula One (F1) motor sport is a complex
sport with a set number of teams, race promoters, which are separate companies from F1, and
sponsors. All of these stakeholders need insight
into the sport. This insight must be shared with
everyone so that no team has an unfair advantage
over any another. This is complicated by the level
of competition between the teams on the track
64
M03 The Practice of Market Research 31362.indd 64
27/09/2021 21:45
The source or producer of the data
and in the fact that each team arranges its own
sponsorships.
One of the first pieces of research to be commissioned was an in-depth qualitative study of the
F1 brand. This comprised group discussions in the
UK, the US and Brazil with a mix of avid fans,
casual fans, lapsed fans and non-fans. This qualitative research included semiotics, brand mapping
and looking at social media. The end result was a
‘brand book’ outlining all the things F1 needed to
do to improve the brand.
Next, to help understand the fans and where
the F1 brand sits in fans’ minds, and what
F1 had to do to make it more appealing, the
research team set up a fan tracking study and
established an online community with 53,000
global fans – they called it ‘F1 Fan Voice’. The
team conducts surveys and polls with community members.
To understand fans’ reactions during races,
and which moments viewers find most engaging, the team commissioned a biometric survey
of viewers during a race. The test, a galvanic
skin response test, measured emotional arousal
and stress. The results were overlaid with participants’ reported reactions, which they entered
via an app before and after the event. The findings allowed the team to track and optimise TV
production to improve the fan experience, for
example by changing camera angles and shot
selection to avoid things like showing too many
pit stops.
Source: Adapted from Bainbridge, J. (2019) ‘Pushing the limit’,
Impact, 25, pp. 42–7. Used with permission.
So a primary research project generates primary data with which to address the
specific objectives of that project. Systems and devices also produce or generate
data. Kitchin (2014) refers to these as ‘exhaust data’. The example he gives is of the
checkout till: its purpose is to add up the price of the goods, present a bill and take
payment. However, in doing so it produces data that can be used for other purposes
– stock control, for example, and, if ethically and legally appropriate, it can also be
used for research purposes, for example for shopping basket analysis or analysis of
customer buying behaviour. The exhaust data produced by systems and devices are
extremely valuable to organisations and to researchers and are often re-purposed to
provide insight. In this sense, they are secondary data, data generated by someone or
something else and available for re-use.
Secondary research and secondary data
The process of secondary research involves identifying existing, suitable sources of
data – often referred to as secondary sources; finding those sources and getting access
to them; reviewing them, assessing their quality and suitability for your research
objectives; analysing and learning from them; and using them or assimilating them
into your own research and/or your thinking about your own research or using them –
analysing them – to address your research objectives. This is an area of research that
has expanded in recent years in line with the expansion in the availability of digital
data. Industry Insight 3.3 gives two examples of the use of this sort of data by the
United Nations (UN) to assess how well policy responses are working.
Secondary research can be an invaluable part of any research project. It is used for
exploratory and/or descriptive purposes and it can be used in explanatory or causal
studies. It is useful when you are defining the problem to be researched – it can often
65
M03 The Practice of Market Research 31362.indd 65
27/09/2021 21:45
Chapter 3
Types of data and research
Industry Insight 3.3
Taking the Global Pulse
Global Pulse is a big data initiative of the United
Nations Secretary-General. It recognises that digital
data offers the chance to get a better understanding
of changes in human well-being, and to get realtime feedback on how well policy responses are
working. Here are two short examples:
●
●
Social media sentiment analysis was used to
map the location of hate speech in Europe and
this information was used to manage risk for
refugees migrating through the continent.
Source: Adapted from UN Global Pulse https://www.unglobalpulse.
org/about/
The analysis of mobile phone top-up data in
Rwanda enabled the UN to predict local food
shortages with almost 90 per cent accuracy.
Industry Insight 3.4
Ticketing talks
Three regional audience development agencies
in the UK took box office data from performing
arts venues across London – millions of records
of patron data – and analysed them. From this
box office data they were able to produce sophisticated audience profiling including ticket yield,
product crossover, audience churn, geodemographic information covering Mosaic Group
and Type, Income and Lifestage. This analysis
provided a benchmark of audience information,
looking at markets for each artform, their size,
characteristics (demographic profile, geographic
spread) and behaviour (frequency, value, drive
distance, advance purchase, churn, ticket yield,
programme choice etc). It delivered insight on customer behaviour and market characteristics. The
results had wide-ranging uses, from tactical and
strategic marketing planning to PR, political lobbying, product development, and provision planning on the part of Arts Council England (who
funded the project) and local government.
Source: Adapted from Brook, O. (2004), ‘I know what you
did last summer: arts audiences in London 1998–2002’, MRS
Conference, www.mrs.org.uk. Used with permission.
answer questions which arise during that process and help you narrow the focus of
what you need from primary research. It may even address all of your research objectives without need for recourse to primary research. It is also useful when writing a
proposal – background research using existing sources can give you insight into the
organisation commissioning the research, the business in which it is engaged and the
wider context of that business, and the service or brand that it offers. It is again useful towards the end of a project when it can provide further contextual detail against
which to set your findings.
Sources of secondary data include those from outside the organisation (so-called
external data) and data generated and held within the organisation (internal data).
Examples of external data include government-produced statistics, ‘third party’
data, data aggregated from different sources, and public, consumer-generated
data. Examples of internal data include data from customers, data derived from
66
M03 The Practice of Market Research 31362.indd 66
27/09/2021 21:45
The nature of the research
customer interactions or business processes (exhaust data), and data from previous
research projects. Industry Insight 3.1 showed how Unilever made use of secondary
or existing data from several sources: previous ‘small data’ research studies conducted by the organisation; sales data from within the organisation; and big data
from outside the organisation – consumer-generated data from social networks,
blogs and so on.
It is important to keep in mind that the value or usefulness of any piece of
research or set of data is rarely exhausted on its initial or primary application. The
data and/or the findings may be useful in the same context at a later date, or they
may be useful in a different context. One set of data may be combined or integrated with others – from very different sources – making the combined set more
valuable and of greater use than the individual elements (think of geodemographic
databases, for example). Using existing data or findings can be much cheaper than
carrying out primary research; some secondary data are relatively quick and easy
to get hold of – unlike primary data, they already exist and can be relatively easy
to access.
The nature of the data
Another of the major distinctions is between quantitative data and qualitative data.
Quantitative data consists of numeric records. Numeric data have four levels of measurement: nominal level data; ordinal level data, interval level data; and ratio level
data. Data at the nominal and ordinal level are known as categorical or non-metric
data; data at the interval and ratio level are known as continuous or metric data.
Interval and ratio numbers are also known as cardinal numbers. It is important to
know what level of measurement your data are in order to determine what type of
analysis is appropriate, and the type of statistical test to use when testing hypotheses.
Qualitative data are non-numeric data. This category of data includes text, images
and sounds.
The nature of the research
You will also see a distinction made between quantitative research and qualitative
research. The differences between the two are summarised in Table 3.1.
Quantitative research
Quantitative research involves collecting data from relatively large samples; the data
collected are usually presented as numbers, often in datasets in tables, on graphs
and on charts. (However, qualitative data are also collected in quantitative research
studies in the form of text and images.) Quantitative research is used to address the
objectives of conclusive (descriptive and explanatory) research enquiries; it can also
be used for exploratory purposes. It provides nomothetic description – sparse description of a relatively large number of cases. Qualitative research, on the other hand,
67
M03 The Practice of Market Research 31362.indd 67
27/09/2021 21:45
Chapter 3
Types of data and research
Table 3.1 Differences between quantitative and qualitative research
Topic
Quantitative research
Qualitative research
Research enquiry
Exploratory, descriptive and explanatory
Exploratory, descriptive and explanatory
Nature of questions
and responses
Who, what, when, where, why, how many
Relatively superficial and rational
responses
Measurement, testing and validation
What, when, where, why
Below the surface and emotional responses
Exploration, understanding, and idea
generation
Sampling approach
Probability and non-probability methods
Non-probability methods (purposive)
Sample size
Relatively large
Relatively small
Data collection
Not very flexible
Interviews and observation
Standardised
Structured
More closed questions
Flexible
Interviews and observation
Less standardised
Less structured
More open-ended and non-directive questions
Data
Numbers, percentages, means
Less detail or depth
Nomothetic description
Context poor
High reliability, low validity
Statistical inference possible
Words, pictures, diagrams
Detailed and in-depth
Idiographic description
Context rich
High validity, low reliability
Statistical inference not possible
Cost
Relatively low cost per participant
Relatively high project cost
Relatively high cost per participant
Relatively low project cost
provides idiographic description, that is, description that is rich in detail but limited
to relatively few cases (although technology now enables qualitative researchers to
reach bigger samples and analyse bigger volumes of data).
The most common research designs in quantitative research are cross-sectional
or ad hoc studies, longitudinal or panel studies and experiments (e.g. A/B tests).
Quantitative research is useful for describing the characteristics of a population or
market – for example, household spending patterns, market and brand share, use
of technology, voting behaviour or intention, incidence of illness, levels of economic
activity. It can do this because of the nature of sampling in quantitative research:
it aims to be representative of the population of interest. Quantitative research is
useful for measuring, quantifying, validating and testing hypotheses or theories. It
has some limitations. It is not as flexible as qualitative research – data collection is
structured and standardised (although this offers reliability in return). The structure
and ­standardisation can produce superficial rather than detailed description and
understanding. In a quantitative research interview the use of mostly closed questions means that we do not collect responses in the participant’s own words, and so
we may lose out on ‘real’ responses, and on detail and context; the standardisation
means that we may miss the subtleties, the slight differences in response between participants. Both can contribute to low validity. Industry Insight 3.5 gives an example
of a project that used both quantitative and qualitative methods to address different
aspects of the same issue. The quantitative element of the project was a survey; the
qualitative element was a diary exercise.
68
M03 The Practice of Market Research 31362.indd 68
27/09/2021 21:45
The nature of the research
Industry Insight 3.5
Access all areas
For most people, key issues with public transport
are comfort and convenience. For others, it is a
life-restricting lack of inclusion. Research conducted for Scope, the disability equality charity
in England and Wales, into the obstacles faced by
disabled transport users revealed the size of the
barriers that even basic urban mobility presents.
The research included quantitative and qualitative
methods. A survey gathered data on attitudes to
and experiences of using public transport and
qualitative research, a journey diary exercise,
examined the disconnects between attitudes and
experiences, highlighting travel problems and the
emotions associated with the experience.
Source: Adapted from Phillips, T. (2019) ‘Journey through the
urban jungle’, Impact, 24, pp. 28–38. Used with permission.
Qualitative research
Qualitative research typically involves relatively small sample sizes. In recent years,
however, technological advances in online interviewing, speech recognition and natural language processing mean that it is easier to conduct and analyse larger sample
sizes. In Industry Insight 3.6 we see how PayPal used qualitative research to address
a business problem and develop a new brand positioning.
Industry Insight 3.6
Re-inventing money
In 2015, PayPal was facing two, linked threats:
increased competition, not least from big tech
businesses like Apple and Google; and a market
position as expert in online payments that was
limiting its growth and leaving it vulnerable to
the new tech-driven market entrants. It wanted
to develop a new brand positioning that would
define it in the new, emerging cultural and technological context, one that would resonate with
its target audience and provide a more accommodating platform for innovation in the future.
It had identified some 5.5 billion people worldwide which it felt were ‘underserved’ by money
and financial services. PayPal wanted to appeal to
this group and develop products and services that
would serve their needs. It believed that an ideal
version of money would serve the ‘underserved’
better, and so be an anchor for its new purpose
and positioning. Hence the question, ‘If money
was invented today from scratch, what would it
look like?’
To tackle this complex question, PayPal commissioned qualitative research agency, Firefish. Firefish conducted interviews with experts on aspects
of money. The findings gave them a set of criteria
with which to judge the performance of money in
the lives of the target audience. For insight into the
lives of the target audience they used ethnography.
Each member of the sample wore a FishEye camera for four days. The recordings gave a detailed
visual log of that person’s life over those four days.
Participants were interviewed about each situation
in their lifelog to find out how well money performed. PayPal used the findings to devise a new
brand positioning and a clear brand purpose.
Source: Adapted from Firefish and PayPal, ‘The evolution of
PayPal: New purpose, new money’, Winner, MRS Awards 2016.
Used with permission.
69
M03 The Practice of Market Research 31362.indd 69
27/09/2021 21:45
Chapter 3
Types of data and research
Qualitative research is concerned with rich and detailed description, understanding
and insight rather than measurement. It aims to get below the surface, beyond the
spontaneous or rational response to the deeper and more emotional response. It is
often used to gain insight into and understanding of what people do, what they think,
what they feel, what they want; and why they do and think and feel and want. It seeks
to discover what might account for or contribute to a particular behaviour. It is good
at uncovering a range of responses, and the subtleties and nuances in responses and
meanings. It is both less artificial and less superficial than quantitative research and
can provide highly valid data. It is suitable in exploratory and descriptive research
enquiries. It is more flexible than quantitative research – the researcher has the scope
during fieldwork to modify or adapt the interview guide or the sample to suit the
way in which the research is developing. The less structured and less standardised
approach can, however, mean that it is relatively low in reliability. This is something
that qualitative researchers acknowledge and take steps to address (via training,
addressing one’s own feelings, opinions and biases before undertaking fieldwork,
discussing approach, analysis and findings with colleagues, for example). It is possible
using qualitative research to tackle complex issues, for example understanding the
decision-making process in a crisis pregnancy (Mahon et al., 1998).
Continuous or ‘one-off’ data collection
Continuous research, as its name suggests, is research done on a continuous basis
or at regular intervals in order to monitor changes over time, for example in a particular market or among a particular population. Ad hoc research is research that is
conducted on a ‘one-off’ basis, to provide a snapshot at a particular point in time.
‘Ad hoc’ is Latin for ‘for this special purpose’. Industry Insight 3.7 illustrates how
an organisation – publisher Penguin Random House – makes use of continuous and
ad hoc research.
Industry Insight 3.7
What are you reading?
Penguin Random House (PRH) owns some of the
most famous brands in book publishing. It has a
mission to reach new audiences and its insight
department is at the centre of that. The department has five staff and in 2016 it ran 88 projects
across all of its brands including ­Penguin, Puffin and Ladybird. The central premise for PRH’s
business is to get closer to readers. To do this
requires a broad and varied range of research
techniques. For example, the company has a
research panel of around 4,500 people which has
been running for several years; it also does ad hoc
qualitative and quantitative research projects on
a number of its big authors. One of its busiest
areas of research is children’s publishing. Getting
the children’s publishing team out to meet children is important and they use several approaches
for this including ‘shop-alongs’ with incentivised
participants and ‘detective days’ where members
of the children’s team are sent to a location to
observe and talk with youngsters in their own
environments.
Source: Adapted from Bainbridge, J. (2017) ‘Spotlight on reading’,
Impact, 17, pp. 48–52. Used with permission.
70
M03 The Practice of Market Research 31362.indd 70
27/09/2021 21:45
Continuous or ‘one-off’ data collection
Continuous research
One way of conducting continuous research is to use a panel of participants chosen
to represent the target population. This type of research design is also known as a
longitudinal panel. Data are collected from the same pool of individuals, households
or organisations over time, either on a continuous basis (every day) or at regular
intervals. Panels can be designed to gather all sorts of data. They are best for recording data about what, how many, how much – what people have actually done. Many
panels are set up to gather information about market characteristics to determine
things like brand share or media usage, details of TV viewing, radio listening, newspaper and magazine reading habits – what, where, when, how long for. The data can
be used to monitor changes in the market, short-term changes – for example reaction to price changes or promotions – as well as long-term trends, such as in brand
share. The data can also be used to examine ad hoc issues such as the effect of a new
advertising campaign.
The panel is recruited to be representative of a particular population, for example
all households in Ireland or subscribers to a particular ISP or owners of particular
makes of car or all retail outlets of a particular type. Panels in which individual consumers are the participants are called consumer panels. Panels made up of a sample
of retail outlets are called retail panels and are used to collect retail audit data such
as stock held, brand coverage, rate of sale, promotions, price and so on in order to
determine distribution and sales patterns of different brands, pack sizes by type of
outlet, sales by location/region. As people (or units) drop out of the panel and the
population from which the panel is drawn changes, new members are recruited so
that the panel remains representative over time. This is particularly important in a
new or rapidly developing market, for example users of wearable AI technology.
Newly recruited panel members tend to behave differently from longer-established
members. For this reason, data from new members are usually excluded for their first
few weeks on the panel.
Recruiting and maintaining a representative panel is a relatively expensive
business. Panel owners use a number of techniques to encourage panel members to stay with the panel and to prevent members dropping out before their
time. Incentives include prize draws, competitions and reward points that can
be redeemed against gifts. Panel newsletters are often used as a way of building
on the community feeling of a panel as well as a way of keeping panel members
informed. Panel data can be weighted to bring the sample more in line with
population characteristics. Other errors that can affect panel data apart from
sampling error include pick-up errors, when the participant (or the data collector in a retail audit) omits to record or scan in an item, which can be accounted
for when making estimates of market size in a process similar to weighting the
sample to population estimates.
Continuous data can also be derived from independent samples of the same
population, samples that are recruited anew for each round of fieldwork. For
example, omnibus studies and advertising tracking studies, or product tests where
the same methodology is used on similar or identical samples, can provide continuous data. Examples of this type of continuous or regular research include the
General Household Survey and Living Costs and Food Survey, both conducted on
behalf of the UK Government. Industry Insight 3.8 is an example of such a repeated
cross-sectional survey.
71
M03 The Practice of Market Research 31362.indd 71
27/09/2021 21:45
Chapter 3
Types of data and research
Industry Insight 3.8
Youth in Iceland
Youth in Iceland is a repeated cross-sectional survey that has been conducted since 1997 by the
Icelandic Centre for Social Research and Analysis (ICSRA). The sample comprises children in all
fifth to seventh grade classes in all primary schools
and all eighth to tenth grade pupils in secondary
schools. The survey contains a set of core questions, for example on demographics, parental and
peer support, pastimes and activities, and a set of
additional modules to look at particular issues. The
survey findings are used to advance and disseminate knowledge on the social factors that determine
the health, well-being and behaviour of young people; to enhance the quality of life of young people;
and to create a venue for collaboration for the education and training of young scholars.
Source: For more information, see the article by Katie McQuater
in Impact, 21, pp.16–18, ‘Seeing problems ahead’, Planet Youth
website http://www.planetyouth.org/about and the ICRSA website
http://www.rannsoknir.is/en/youth-in-iceland/
Another form of continuous research is research on a continuous stream of data
that comes in real time from the digital world of sales interactions, web activity, social
media, and from research communities. Learning from these continuous data, and
providing insight based on them, means being able to process them quickly. Given
the speed and volume of data, this means using AI and bots. It also demands a different working relationship between client and researcher, an example of which we
saw in Industry Insight 2.3 where the research supplier was working on a daily basis
to learn from the data and incorporate feedback from the client.
Ad hoc research
Ad hoc research is usually designed to address a specific problem or to help understand a particular issue at a certain point in time. For example, you might commission
ad hoc research among employees to determine satisfaction with their new office
accommodation, or to understand the issues faced by overseas students in their first
few months at university. The types of studies that come under the heading ad hoc
research include advertising pre-tests and communication testing, usage and attitudes
studies, hall tests, store tests, market mix tests and brand/price trade-off research.
Industry Insight 3.9 gives another example of the use of continuous and ad hoc
research, this time by a water supply company based in Bath, England.
Industry Insight 3.9
A flow of ideas
Water companies in the UK operate in a regulatory environment; the regulator for Wessex Water
(WW) is Ofwat. To comply with regulations
water companies must share their business plans
with Ofwat and back them up with evidence. WW
does continuous customer research to gauge and
monitor customer sentiment and it does ad hoc
research to ensure it has the evidence it needs for
its business plan. One such ad hoc project was a
customer evaluation study with questions on what
72
M03 The Practice of Market Research 31362.indd 72
27/09/2021 21:45
The method of data collection
level of service customers are prepared to accept
and how much more or less they would pay for
an increase in services. Another of its obligations
to Ofwat is to speak to future bill payers. So WW
set up qualitative continuous research in the form
of a young people’s panel of 21 sixth-form students. The students were recruited via invitations
to schools in the WW area. The first panel meeting was a one-day event at company HQ where
panel members learned about the water business
and got to meet and question senior staff. The
panel was then divided into four groups and over
the following three months they collaborated on
generating ideas for the business. At the end of the
three months they returned to HQ and presented
their ideas. These ideas formed a key part of the
company’s business plans, contributing to the
company’s report to Ofwat, and there are plans
to implement some of them within the business.
Source: Adapted from Bold, B. (2017) ‘Giving tomorrow’s
adults a say in their futures’, Impact, 17, pp. 54–8. Used with
permission.
Omnibus surveys
Omnibus surveys are surveys that are run by research agencies on a continuous basis.
Clients can buy space on these surveys to insert their own questions – they are usually charged an entry fee and a fee per question that covers fieldwork and standard
data analysis. They can be used to generate continuous data by repeating the same
questions in each round, or they can be used to gather cross-sectional data on an ad
hoc basis – to collect data on specific issues as the need arises.
Depending on the number of questions included, using an omnibus survey can be
very cost effective – fieldwork costs are shared and set-up time is minimised because
of the ongoing, pre-set nature of the survey. The law of diminishing returns, however,
kicks in at about eight to ten questions – it is likely that for this number of questions
a customised survey is just as cost effective. One thing to bear in mind is where your
questions appear on the questionnaire as there may be ‘position effects’.
The omnibus may survey a representative sample of the general public or it may
target a more specialised population or group. For example, omnibus surveys are run
among samples of general practitioners, motorists, teenagers, older people, European
consumers and independent financial advisors. Participants are recruited anew for
each round of an omnibus survey using random or quota sampling techniques. Many
omnibus surveys take place weekly, some twice weekly and others once every two
weeks. Sample sizes vary: for general public omnibus surveys the sample is usually
around 1,000 participants per week but can be up to 3,500; for more specialised
target groups it may be 500 every two weeks. To achieve a robust sample of a low
incidence target group, for example hearing aid users, may mean that questions are
included on more than one round of the omnibus.
The method of data collection
There are several ways of classifying data collection. Data collection can be active
or passive; data can be collected using observation and interviews. Observation
and interviews are used in quantitative and qualitative research. Interviewing can
be classified into interviewer-administered (face-to-face, online and by telephone)
73
M03 The Practice of Market Research 31362.indd 73
27/09/2021 21:45
Chapter 3
Types of data and research
and self-completion (post and online). We look in detail at qualitative methods and
quantitative methods of data collection in later chapters. Here we look at the main
features of active and passive data collection and at the main features of observation
and interviewing, and at the main distinctions between their use in quantitative and
qualitative research.
Active and passive data collection
In most ‘traditional’ research projects you collect data with the active participation
of members of the target population. You can collect data with a researcher present
– for example, in a telephone or in person or remote face-to-face interview. You can
also collect data with no researcher or interviewer present – for example, online or
other self-completion approaches. Whether a researcher is present or not, the sample
members are actively taking part in a research project: they are answering questions
or recording their activities or carrying out tasks. This is active data collection. It
requires that participants have given their informed consent to take part: the purpose
of the research project has been explained to them and the data collected are to be
used for research purposes only (ESOMAR, n.d.). The participants can withdraw
from the research at any time.
Passive data collection, on the other hand, is when the target population does not
actively take part in the data collection; data are collected automatically. Examples of
this type of data collection include the use of cookies and web bugs on websites that
become embedded in your device and allow the website owner to track your visits
to the website and capture your activity; or the location services on your phone that
track your movements. In these examples you, as the subject of the data collection,
are not taking an active part in it – your data are being collected automatically; your
activity is being observed and captured electronically. If the data collected include
personal data – that is, any information relating to an identified or identifiable natural person (including name, address, location, email address, phone number, personal
characteristics, image) – there are legal requirements for data protection, as we saw
in Chapter 2. For any form of processing of personal data, the data subject must give
informed consent: they must know what data are being collected, why they are being
collected and what is to be done with them.
A huge amount of data is collected passively from the whole digital trail of our
daily lives. It may be collected for a primary research project. For example, you might
be invited to take part in a project and asked to download a research app to your
device. The app collects data about you and your activities automatically – passively
– through the app. You don’t have to answer questions or record or enter data. Data
collected passively are also used in secondary research.
The more traditional type of observation used in ethnography, which, as we see
below, can be conducted with or without a researcher present, is active data collection. As we saw in Industry Insight 3.9, members of the children’s publishing team
at Penguin Random House do accompanied ‘shop-alongs’ with readers. Industry
Insight 3.7 is another example of the use of observation in an ethnographic study
and in this case no researcher was present – research participants in this PayPal study
wore cameras to record their interactions. This is nevertheless active data collection:
participants are actively involved in collecting data about their activities.
74
M03 The Practice of Market Research 31362.indd 74
27/09/2021 21:45
The method of data collection
Observation
Observational techniques, based on ethnographic methods used in anthropology and
sociology, are well established in social and market research. The main advantage of
observation over interviewing is that in an interview the participant is recalling and
recounting their behaviour to the researcher whereas in observation the researcher (or
the camera) sees it at first hand – without the filter of memory or selection. Observation is also useful in the following situations:
when you do not know or are unsure about what questions to ask;
when you are starting a project in a setting with which you are not familiar;
● when you want to examine an activity or process in a new way;
● when you want to observe an individual act in detail;
● when you want to see things happen in context;
● when you want to gather data from another perspective;
● when you want greater detail or greater understanding of a process or behaviour;
● when you want to observe unconscious or habitual behaviour;
● when the target audience cannot communicate verbally;
● when you have concerns about the validity or reliability of interview data;
● when you want to observe the behaviour of people en masse.
●
●
Observation can be used to generate both qualitative and quantitative data. Observation methods to collect quantitative data tend to be mechanical or electronic. These
‘surveillance’ methods also tend to be unobtrusive or unseen – that is, those being
observed are largely unaware of it. Also they tend to collect data on the activity rather
than on the person and the activity. Examples of mechanical or electronic observation
devices include traffic counters, devices that record the number of cars or pedestrians
passing a particular point; electronic scanners, including those devices that read and
log the bar code or the Unique Product Code on goods, recording customer purchases
for storage on a database (often referred to as EPOS or electronic point of sale scanners); RFID (radio frequency identification) tags, tiny chips embedded in products or
their labels or packaging enabling them to be tracked; GPS tracker devices in phones
and wearable devices that track location; closed circuit television systems that record
people flow; web counters that count and log visits to a website; ‘cookies’, messages
given to a web browser by a web server that enable it to identify users entering a
website; and visitor recording software, including tracking codes and session replay
scripts, and scroll, click and eye-tracking heatmaps on websites and apps that enable
recording of visitors’ activity. The main advantage of these methods of collecting data
is their thoroughness in counting and/or recording activity. The main disadvantage is
that they can generate high volumes of data that may be difficult to handle, process
and/or analyse.
Observation to collect qualitative data can be done in person by a researcher,
sometimes with the help of a camera or a voice recorder, and, as we saw in Industry
Insight 3.7, it can be done without a researcher, with a camera which the research
participant wears, or a camera at the location of interest. Observation with a
researcher present tends to be more intrusive – the observed are aware that they are
being watched, the aim being to collect data on the individual and the activity. The
example in Industry Insight 3.7 is a less intrusive set-up. Industry Insight 3.10 gives
an example of the observation of a place and its people, Lagos, Nigeria.
75
M03 The Practice of Market Research 31362.indd 75
27/09/2021 21:45
Chapter 3
Types of data and research
Industry Insight 3.10
Living in Lagos
Lagos, Nigeria, is the largest city in Africa. It is
densely populated with more than 20 million
inhabitants. The people in Lagos work long hours
and to beat the traffic many leave home in the
early hours of the morning. From their neighbourhoods on the mainland to the roads that lead
to the islands, navigating the city is part of the
daily grind. As part of a research project for a
client in the pharmaceutical industry, researchers
from agency Naked Eye Research wanted to get
an insight into the day-to-day lives of Lagos people and to map their daily commute to and from
work. The question was, how do you do this in
such a busy, sprawling city? An ethnographer and
a film-maker from the UK-based agency went to
Lagos and teamed up with local researchers. They
captured some information on the ground, using
photography and film, but were constrained by
security issues and the scale and density of crowds,
cars and general commotion. To get around this
they went above it. They set up aerial observation
using drones. With the help of a licensed drone
pilot, they filmed from the sky. They were able to
get incredible visuals of the spaces through which
people moved, worked, shopped and spent time.
‘Once we had downloaded visuals,’ said Nick
Leon, founder of Naked Eye Research, ‘we were
able to compare neighbourhoods. We shared
drone footage with the participants and they were
fascinated. Drone footage gave us new ways of
talking to participants about the spaces that surround them at home, and to and from work. They
began to tell us stories of what had happened in
these different places, and pointed out things from
the aerial shots that were previously hidden from
sight. For example, the respiratory issues they
suffered because of the dust thrown up from the
dirt roads; drainage ditches that became breeding
grounds for malaria in the rainy season; the lack of
play space children had in their neighbourhoods;
the state and repair of homes; where people congregated at weekends; the places they marked as
no-go areas after dark; and the distance from their
homes to pharmacies, clinics and stores.’
Source: Adapted from Leon, N. (2017) ‘Alternative view of Lagos
life’, Impact, 19, pp. 16–17. Used with permission.
Interviewing
Interviewing is a form of primary research. You can collect qualitative or quantitative
data via interviews.
Quantitative interviews
To collect quantitative data researchers use standardised structured or semi-structured
‘forms’ – interview schedules or questionnaires and diaries. There are two ways of g­ etting
a sample to complete these ‘forms’. You get the participant to do it themselves – this is
called ‘self-completion’; or you get an interviewer to ask the questions, either in person
or remotely face to face or via the telephone, and you record their answers on the ‘form’
– this is called ‘interviewer administered’. The option you choose will depend on a number of things. You will need to determine how suitable the method is for the following:
the study and its objectives;
the topic or issues under investigation;
● reaching the right sample;
● achieving the right numbers;
● the time and budget available.
●
●
76
M03 The Practice of Market Research 31362.indd 76
27/09/2021 21:45
Chapter summary
Industry Insight 3.11
Gathering in numbers
Royal London is the UK’s largest mutual life
insurance and pensions company. Its vision is to
be the most trusted and recommended financial
services provider. Understanding what matters
most to customers, recognising the importance
of customer service and delivering products
and services that meet their needs is essential
to achieving their vision. To do this they created the Customer Voice Program with research
agency ORC International. The first step in creating the program was to find out what matters
most to customers. They used qualitative and
quantitative research to do this. They repeated
the research with advisors – the intermediaries
who sell their products to customers – to find
out what advisors value and what support they
need. They put in place an annual telephone survey of around 2,000 customers to monitor progress against goals, to identify gaps, and gauge
change over time. They also run a 90-second
transactional survey to get feedback from customers who recently contacted them by phone;
5,000 customers completed this survey each
month in 2016.
Source: Adapted from ORC International and Royal London,
‘Inspiring change’, MRS Awards 2017. Used with permission.
Industry Insight 3.11 gives an example of the use of a range of quantitative approaches
used by insurance company Royal London to help it understand its customers.
Qualitative interviews
What distinguishes qualitative interviews from quantitative interviews is the style
of the interview. Whereas quantitative interviews are standardised and most of
the questions are structured, closed questions, qualitative interviews are more like
‘guided conversations’ (Rubin and Rubin, 2011) or ‘conversations with a purpose’
(­Burgess, 1984) – less structured and less standardised, making use of open-ended,
non-directive questions.
Chapter summary
There are several useful ways of classifying or describing both data and research. They
are as follows:
●
●
●
the size of the data – big data and small data;
the format of the data – structured, semi-structured and unstructured data, and
simple and complex;
the source or producer of the data – primary and secondary data (and primary
and secondary research);
●
the nature of the data – qualitative and quantitative;
●
the nature of the research – qualitative and quantitative;
77
M03 The Practice of Market Research 31362.indd 77
27/09/2021 21:45
Chapter 3
Types of data and research
●
●
●
●
the mode of data collection – continuous and one-off or ad hoc research; and
the method of data collection – active or passive; observation or interview;
personal or self-completion; in face-to-face (in person or remote), telephone,
online and postal.
The decision about which type to use will depend on the client’s business
problem; the information needs; the research objectives; and the resources
available.
For a given context or project, each type will have advantages and limitations.
Exercises
1 Review Industry Insights 3.1–3.11. For each one, list the type or types of data
and/or research involved and give reasons why you think the use was justified for
the context described.
References
Bainbridge, J. (2019) ‘Pushing the limit’, Impact, 25, pp. 42–7.
Bainbridge, J. (2017) ‘Spotlight on reading’, Impact, 17, pp. 48–52.
Bold, B. (2017) ‘Giving tomorrow’s adults a say in their futures’, Impact, 17, pp. 54–8.
boyd, d. and Crawford, K. (2012) ‘Critical questions for big data’, Information, Communication & Society, 15, 5, pp. 662–79. https://doi.org/10.1080/1369118X.2012.678878
(Accessed 6 June 2021)
Brook, O. (2004) ‘I know what you did last summer: arts audiences in London 1998–2002’,
Proceedings of the Market Research Society Conference, London: MRS.
Burgess, R. (1984) In the Field: An Introduction to Field Research, London: Allen & Unwin.
ESOMAR (n.d.) Passive Data Collection, Observation and Recording, Amsterdam: ESOMAR.
Firefish and PayPal (2016) ‘The evolution of PayPal: New purpose, new money’, MRS Awards.
Hardey, M. (2011) Viewpoint: ‘To spin straw into gold: New lessons from consumer-generated
content’, International Journal of Market Research, 53, 1, pp. 15–17.
Kitchin, R. (2013) ‘Big data and human geography: Opportunities, challenges and risks’,
­Dialogues in Human Geography, 3, 3, pp. 262–7.
Kitchin, R. (2014) The Data Revolution, London: Sage.
Kitchin, R. and McArdle, G. (2016) ‘What makes Big Data, Big Data? Exploring the
ontological characteristics of 26 datasets’, Big Data & Society, 3, 1 https://doi.
org/10.1177/2053951716631130 (Accessed 12 January 2021).
Laney, D. (2001) Application Delivery Strategies 3D Data Management: Controlling
Data Volume, Velocity, and Variety, META Group https://blogs.gartner.com/douglaney/files/2012/01/ad949-3D-Data-Management-Controlling-Data-Volume-Velocityand-Variety.pdf (Accessed 18 September 2020).
Lawes, R. (2018) ‘Science and semiotics: what’s the relationship?’ in International Journal of
Market Research, 60, 6, pp. 573–88.
78
M03 The Practice of Market Research 31362.indd 78
27/09/2021 21:45
Recommended reading
Leon, N. (2017) ‘Alternative view of Lagos life’, Impact, 19, pp. 16–17.
Mahon, E., Conlon, C. and Dillon, L. (1998) Women and Crisis Pregnancy, Dublin: Stationery
Office.
Marr, B. (2019) ‘What’s the difference between structured, semi structured and unstructured data?’ Forbes Magazine, www.forbes.com/sites/bernardmarr/2019/10/18/whats-thedifference-between-structured-semi-structured-and-unstructured-data/#1f378d982b4d
(Accessed 16 March 2020).
Marr, B. (2014) ‘The 5 Vs everyone must know’, https://www.linkedin.com/pulse/
20140306073407-64875646-big-data-the-5-vs-everyone-must-know/ (Accessed 16 March
2020).
McNulty, E. (2014) Understanding Big Data: The Seven Vs, https://dataconomy.com/2014/05/
seven-vs-big-data/ (Accessed 25 May 2021).
McQuater, K. (2017) ‘Seeing problems ahead’, Impact, 21, pp. 16–18.
Murthy, P., Bharadwaj, A., Subramanyam, P., Roy, A. and Rajan, S. (2014) Big Data Taxonomy, Cloud Security Alliance https://downloads.cloudsecurityalliance.org/initiatives/bdwg/
Big_Data_Taxonomy.pdf (Accessed 12 January 2021).
ORC International and Royal London (2017) ‘Inspiring change’, MRS Awards.
Phillips, T. (2019) ‘Journey through the urban jungle’, Impact, 24, pp. 28–38.
Rubin, H. and Rubin, I. (2011) Qualitative Interviewing: The Art of Hearing Data,
3rd ­edition, London: Sage.
Sampson, P. (1967 and 1996) ‘Commonsense in qualitative research’, Journal of the Market
Research Society, 9, 1, pp. 30–8 and 38, 4, pp. 331–9.
Spiegelhalter, D. (2019) The Art of Statistics, London: Pelican.
United Nations, ‘UN Global Pulse’, www.unglobalpulse.org/news and https://www.unglobalpulse.org/project/understanding-perceptions-of-migrants-and-refugees-with-social-media/
(Accessed 26 February 2020).
Verhaeghe, A. and Nola, P. (2010) ‘Keeping up with the conversation’, Research, 534,
­November, pp. 34–5.
Recommended reading
Adams, K. and Brace, I. (2006) An Introduction to Market and Social Research: Planning and
Using Research Tools and Techniques, London: Kogan Page.
Babbie, E. (2020) The Practice of Social Research, 15th edition, London: Wadsworth.
Ritchie, J., Lewis, J., McNaughton Nicholls, C., and Ormston, R. (eds.) (2013) Qualitative
Research Practice: A Guide for Social Science Students and Researchers, 2nd edition, London: Sage.
Strong, C. (2015) Humanizing Big Data: Marketing at the Meeting of Social Science and
Consumer Insight, London: Kogan Page.
79
M03 The Practice of Market Research 31362.indd 79
27/09/2021 21:45
M03 The Practice of Market Research 31362.indd 80
27/09/2021 21:45
Part Two
Planning and designing research
M04 The Practice of Market Research 31362.indd 81
27/09/2021 21:45
Chapter 4
Business problem and research
problem
Introduction
In this chapter we look at the relationship between the business problem
and the research problem and the links with the research objectives and the
research design. We look at the role of background research, including the use
of a literature review. We look at different types of research enquiries and at how
to formulate the research objectives. We look briefly at where the data might
come from to address those objectives. We revisit validity and reliability and
introduce the issues of error and bias. Finally, we look at anticipating the ethical
issues that may arise in the course of a project.
Topics covered
Understanding the business problem and the research problem
● The role of background research
● Establishing the nature of the research enquiry
● Formulating research objectives
● Validity, reliability, error and bias
● Identifying ethical and legal issues.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project; and Topic 2: Guiding Principles.
M04 The Practice of Market Research 31362.indd 82
27/09/2021 21:45
What you should get from this chapter
At the end of this chapter you should be able to:
understand the importance of the business problem;
● define the research problem;
● begin a literature review;
● identify the nature of the research enquiry;
● formulate the research objectives;
● recognise the links between business problem, research problem and
research objectives;
● outline what is meant by validity, reliability, error and bias; and
● begin an ethical review.
●
83
M04 The Practice of Market Research 31362.indd 83
27/09/2021 21:45
Chapter 4
Business problem and research problem
Business problem and research problem
There are two problems to be unravelled in order to commission (and design) goodquality actionable research: first, the business or the decision maker’s problem; and,
second, the research problem. This is the most important stage of the research process. Everything flows from this. If the business problem is not understood, the information needs not clearly identified, and the end use of the information not clearly
established, it will be difficult to design effective research (see Figure 4.1).
The decision maker needs information on which to base a decision – they have
uncovered an issue, they want to take action to address the issue, but need to know
what action to take. This is the business problem, or the client’s or decision maker’s
problem. If it is clear what action they need to take – if the decision maker has a full
understanding of the issues around the problem, if there are no information gaps or
questions in their mind – research will not be needed. If, however, questions do exist,
if there is a lack of understanding, a gap in information, then research is needed. This
is the research problem. What the research must deliver will be formulated as the
research objectives.
What sort of research needs to be done to address the research objectives? This is
a question about research design. We start the process of research design here and
come back to it in the next chapter. The thing to remember now is that all of these
are linked: business problem, research problem, research objectives, research design.
What are business problems?
What are we talking about when we talk about the business or the decision maker’s
problem? Have a look at the examples in Box 4.1. Each one gives you the business
problem and something of the wider context and the research problem that emerges
from that.
Industry Insight 4.1 sets out in some detail a business problem and research problem faced by BBC World Service English (WSE), a broadcast service that reaches
75 million people around the world every week.
From business problem to research problem
How do you get from business problem to research problem? In Industry Insight 4.1
we learn that WSE wants to connect with younger, digital/non-radio audiences in 12
markets using podcasts. This is the business problem. WSE needs to know what is the
best way of achieving this connection. It doesn’t have much information about the
Business
problem
Information
needs
Research
problem
Research
objectives
Figure 4.1 Business problem to research objectives
84
M04 The Practice of Market Research 31362.indd 84
27/09/2021 21:45
Business problem and research problem
Box 4.1
Example: business problems
Selling cars
A car dealership specialising in selling small petrol engine cars has found that although
the number of enquiries it is handling and the number of visitors to the showroom have
remained the same compared with the previous year, sales have fallen dramatically.
It realises that external market forces – largely economic and regulatory issues to do
with emissions and the risk that these cars may attract higher taxes or face bans – may
be affecting things. The dealership wants to understand what is happening among the
car-buying public so that it can take action to at least halt the decline in its business.
Good enough to eat?
The leases of several catering outlets in a railway station with local, national and international services are soon to come up for renewal. The management team sees this as
an opportunity to review current provision. It has key data about the sales and financial
performance of each of the outlets. The team, however, wants to understand two other
things: current trends in food retailing; and how station users view what is currently on
offer. Armed with this information the management team believes that it will be better
placed to make a decision about future provision.
Making use of digital tools
A government department with responsibility for economic development has an action
plan to help small businesses speed up their adoption of digital tools. The plan set
objectives that are to be achieved by a certain date. To develop policy for the coming
year, and to set new objectives, the department wants to review the current situation. It would like to know how successful organisations have been in meeting the
original objectives. It also wants to gauge what effect the use of digital tools has had
on their business. Is it improving revenue per employee? Is it helping to increase revenue growth? Has there been a growth in job numbers? How many businesses have
accessed new markets and/or new customers? Is it improving customer satisfaction
rates? What effect is it having on competition?
Industry Insight 4.1
Broadcasting to podcasting
BBC World Service English (WSE) wanted to connect with younger, digital/non-radio audiences. It
believed that podcasts would be a great way to do
this. It wanted to develop a podcast strategy that
would work across continents and would achieve
‘first mover’ advantage in emerging economies.
First mover advantage is the notion that if you
are the first significant organisation to enter a
new market you will have an advantage over
your competitors. Over time this should lead to
greater revenues and profits. However, it also carries risks, including the risk of failure and the risk
85
M04 The Practice of Market Research 31362.indd 85
27/09/2021 21:45
Chapter 4
Business problem and research problem
of the second mover learning and benefiting from
your first mover mistakes. So, the business problem is: how can WSE successfully connect with
younger, digital/non-radio audiences in 12 diverse
markets using podcasts? In most of the 12 markets WSE has almost no intelligence. Even for the
more developed markets, existing knowledge is
scant.
What is the research problem? Well, the
research problem is the gap in knowledge about
podcasts in the 12 markets. Not only does WSE
not have information on podcasts and podcast
audiences, it does not know where on the ‘hype
cycle’ podcasts are. It is important that WSE
knows this in order to make decisions about how
to approach each market. The hype cycle is a term
coined by research and advisory firm, Gartner. It
is a useful way of analysing or thinking about
what is hype and what is not, what is a real driver
of a product’s commercial promise. Gartner’s
hype cycle has five key stages that describe the
life cycle of the product:
Innovation trigger: a potential technology or
product breakthrough but no useable product
in existence.
● Peak of inflated expectations: some success stories and lots of failures.
● Trough of disillusionment: decline in interest as
products fail to deliver.
● Slope of enlightenment: product and its benefits
are more widely understood and more second
and third generation products appear.
● Plateau of productivity: product applicability and relevance are clear and the product is
adopted by the mainstream.
●
We look at the research WSE went on to commission later in the chapter.
Source: Adapted from Firefish and BBC World Service, ‘Beyond
the hype: shaping a global podcasting strategy’, MRS Awards
2017.
markets, the audiences within those markets or about the position of podcasts within
them. This is the research problem. The aim of the research will be to deliver information that will allow WSE to develop an effective plan to enable it to connect with
younger, digital/non-radio audiences in 12 markets using podcasts. That last sentence
is very important: it tells you about the business impact the research must have. In
other words, the organisation must be able to act on the basis of the research findings.
Thinking about business impact: the action to be taken
The action to be taken depends on understanding the 12 markets WSE has identified,
the audiences that make up those markets and the position of podcasts within each
of them. For WSE to decide what it should do, it needs to understand the setting or
wider context in each of the 12 markets. For example, it needs to know the size of
each market; it needs to know the profile of podcast users in each market; it needs
to know what people understand podcasts to be; it needs to know how people first
came to podcasts; it needs to know which podcasts people prefer and which they
found influential; and it needs to know where podcasts fit in in the wider context of
audio listening. Understanding this ‘podcast landscape’ will help shape the plan and
the actions to be taken and may inform further research.
Who is involved?
Defining the decision maker’s problem and the research problem, and getting to the
specific research objectives, can involve several rounds of discussion with the owner of
86
M04 The Practice of Market Research 31362.indd 86
27/09/2021 21:45
Business problem and research problem
the problem, that is, the decision maker, and the organisation’s (the client’s) internal
researcher or analyst (if there is one). Most of these discussions may take place between
the decision maker and the internal client researcher before an external researcher –
a consultant or an agency researcher – is involved. On the other hand, an external
researcher may be involved and (drawing on their skills and experience in the research
process in addition to sector or market or product knowledge) may be able to provide
valuable insights at this early stage. Some of these discussions may be revisited if, upon
receipt of the brief, the external researcher finds that some elements are not clear.
Learning from informal exploratory research
Many projects benefit from some early-stage, informal exploratory research. This can
take many forms: asking internally if anyone has faced a similar problem and, if so, talking to them and reviewing the work they did; reviewing existing data on the topic including any internal data as well as external material (social media posts, recently published
reports); and having informal interview ‘chats’ with relevant stakeholders or those in the
target market. This exploratory or background research can help you get to grips with
the issues, understand the problem and its context and refine the focus of the research.
Look again at the railway station catering outlet example in Box 4.1. Imagine that
you discover through some informal exploratory research (a review of social media
posts, recently published reports on food trends, informal interview ‘chats’ with
stations users) that external factors, particularly interest in vegan food and a lack
of provision at the railway station, and internal factors, such as speed of service at
existing outlets, are having an impact on sales at the current outlets. You can make
some tentative suggestions – that lack of provision of vegan food is affecting sales,
that speed of service is a key factor in the success of an outlet. These suggestions or
ideas or statements about connections between things are called hypotheses. While in
the early stages of problem definition, when you are trying to get to grips with what
exactly the problem is, these hypotheses are likely to be fairly vague statements. As
you work to uncover the problem these hypotheses are likely to emerge more clearly.
Arriving at a clear understanding of the problem is key to doing good research.
It may be an iterative (back and forth) process rather than a one-off event. It is
important to spend some time doing this. It is a key skill to be able to unpack, even
challenge, and understand the business problem and define the research problem and
show the link between the two. From a review of the literature on the topic and from
the results of a survey among alumni of a US university, Vriens et al. (2019) report
that two of the key gaps in the skills set of research practitioners are understanding
of business issues and the ability to define a marketing research problem.
Let’s look at another example. In Industry Insight 3.6 we looked at the research
PayPal did to address a business problem; Industry Insight 4.2 looks in more detail at
that problem. In Industry Insight 4.2 we learn that PayPal has identified a two-fold
business problem: increased competition; and a problematic market position for its
brand. It wants to take action. It needs to protect itself from competition, secure its
market position and grow the brand. To this end, it has decided that the best course
of action is to create a new brand identity, and a new purpose. It has identified a
potential new target audience, a large group of people who are ‘underserved’ by
money and financial services. It realises that research could help it understand this
audience and its needs. Here is the research question, the question it needs research
87
M04 The Practice of Market Research 31362.indd 87
27/09/2021 21:45
Chapter 4
Business problem and research problem
Industry Insight 4.2
Re-inventing money
PayPal was founded in 1999 as a simple and
secure way of transferring money online. By 2015
the world had changed and PayPal – in the process
of becoming independent of eBay – was looking
for a new identity and a new brand purpose. It
approached research agency Firefish with a question: ‘If money was invented today from scratch,
what would it look like?’
Why was PayPal asking this question? It was
facing two linked threats: increased competition,
not least from big tech businesses like Apple and
Google; and a market position as expert in online
payments that was limiting its growth and leaving
it vulnerable to the new tech-driven market
entrants.
Thus, PayPal wanted to develop a new brand
positioning that would define it in the new,
emerging cultural and technological context, one
that would resonate with its target audience and
provide a more accommodating platform for
innovation in the future. It had identified some 5.5
billion people worldwide which it felt were ‘underserved’ by money and financial services. PayPal
wanted to appeal to this group and develop products and services that would serve their needs. It
needed to know in what way financial institutions
underserve people. PayPal believed that an ideal
version of money would serve the ‘underserved’
better, and so be an anchor for its new purpose
and positioning. Hence the question, ‘If money
was invented today from scratch, what would it
look like?’ It wanted to understand what money
and financial services are supposed to do. Once
it had answers to these questions – and so had
knowledge of the real issues and the unmet needs
of the target group – it could develop a new positioning for PayPal, one that would address the
threats it was facing, and attract investors.
Source: Adapted from Firefish and PayPal, ‘The evolution of
PayPal: New purpose, new money’, Winner, MRS Awards 2016.
to answer: what are money and financial services supposed to do? Put another way,
in what way is this audience underserved by financial services and money? As the
client put it, ‘If money was invented today from scratch, what would it look like?’
What business impact must the research have? If PayPal understands this ‘underserved’ audience and its needs, then it can create a new brand identity that will meet
these needs. If it meets the audience’s needs, it will secure the future of the brand.
Meeting the audience’s needs should guard against increased competition. If the
brand delivers to this audience, this will benefit PayPal and it should mean that the
brand is an attractive proposition for outside investors.
Formal exploratory research: a literature review
At the early stages of a project, particularly in academic and social research projects
but also in some commercial ones, it is common practice to do a literature review. It is
a worthwhile practice, regardless of the type of project you are working on. It can help
increase your knowledge and understanding of the problem under investigation; it
allows you to benefit from the work and the thinking of others, those who have done
research on the same or a similar topic, or those who have tried a particular research
approach, and so it should enhance the quality of your thinking, your research design
and, later, your analysis, interpretation and reporting of the findings. A literature
review is a form of secondary research, an examination of research done by others.
88
M04 The Practice of Market Research 31362.indd 88
27/09/2021 21:45
Business problem and research problem
Box 4.2
How to do a literature review
A literature review should be a synopsis and critical assessment of the relevant literature
on the topic or issue your client is facing and the research you plan to do. In writing
a literature review there are two main pitfalls to avoid: using it to show that you know
‘the area’; and presenting it without any critical thinking about the content. Here is a
guide to what a literature review should aim to show (Silverman, 2005, adapted from
Murcott, 1997):
What is already known about the topic
What you have to say about what is known
● If anyone else has conducted the same research
● If anyone has done research that is related to your topic
● Where the research you are planning fits in with what has gone before
● Why your research is worth doing in the light of what has been done already.
●
●
Keep this in mind when you tackle your literature search, and go back to it when you
write up the findings.
Below are the steps you might follow in doing your search:
Define the purpose of your literature review. Be as precise as possible about what
you want, otherwise when you come to search you may be overwhelmed by the
amount of material you come across.
● Think about what sort of sources might be useful, e.g. books, government publications, commercial research reports, journal articles, conference papers, case studies, websites, blogs, social media. This will determine to some extent where you
search for material.
● Prepare a list of relevant key words, relevant dates and, if you know them, authors’
names. These will form the basis of your search in search engines and online
catalogues.
● Set up a format for recording the sources and the information you find. There will be a
lot and it will be easy to lose track, so make sure you are organised before you start.
You may want to record sources in a spreadsheet or database or in a bibliographic
reference or citation management software package such as RefWorks, Mendeley,
EndNote, ProCite, BibTex or Zotero. You may also want to devise a template on
which to make notes on articles as you read. You may want to set up folders in which
to store relevant, useful material and one for stuff you are uncertain about.
● Once you have found material, you need to decide if it is relevant and useful for your
project. Do a quick ‘scan’ read. Look at the abstract, if there is one. This should tell
you about the entire piece. If not, go to the introduction and look for aims and objectives; look at the methods section; look at the conclusions. Once you have done
this you should know whether this item is worth keeping for a more in-depth, critical
read. If the answer is yes, save it to your ‘relevant/useful’ folder; if you’re uncertain,
save it to your ‘uncertain’ folder; if the answer is no, bin it.
● When the ‘scan’/collection phase is complete, go back to your folders and your
notes and review and critically evaluate each piece. What are its strengths and
●
89
M04 The Practice of Market Research 31362.indd 89
27/09/2021 21:45
Chapter 4
Business problem and research problem
weaknesses? How sound is the research design, the research method, the sample?
How good is the evidence on which the conclusions are based? How does it tie in
with other research in the field? Then ask yourself: What can I learn from it in relation
to my project? Make notes as you do this.
● Prepare an outline of your literature review then write it up, bearing in mind the pitfalls
noted by Silverman (2005) and Murcott (1997) above.
The nature of the research enquiry
Now that you are clear about the business problem and the research problem, the
next question to ask is, what is the nature of my research enquiry? There are three
main types of enquiry: exploratory, descriptive and explanatory or causal. (Sometimes descriptive and explanatory research appear under the heading of conclusive
research.) It is important that you are able to recognise what type of enquiry it is that
you need. It is important as it will guide how you structure your research. Before
we move on to look at the links between the nature of the enquiry and the research
design, here’s a short description of each type of enquiry.
Exploratory research
An exploratory research enquiry aims to explore, to allow you to become familiar
with a topic or the issues around a problem. It should be useful for example in helping you with the following:
‘unpacking’ an issue;
looking for insight into an unfamiliar topic;
● clarifying the nature of a problem;
● defining the scope of an investigation;
● determining the feasibility of conducting further research;
● developing propositions and hypotheses for further research.
●
●
The main disadvantage of exploratory research is that it may not deliver a definitive
or conclusive answer. It may not allow you to see the whole picture.
Descriptive research
Descriptive research is about collecting data to describe people, places, things,
events, situations, experiences – finding the answers to the Who? What? Where?
When? How? and How many? questions. The difference between exploratory
research and descriptive research is that, while exploratory research can provide description, to choose descriptive research you should have a clearer idea of
what you need – you will have a more clearly defined set of research questions or
objectives.
90
M04 The Practice of Market Research 31362.indd 90
27/09/2021 21:45
The nature of the research enquiry
Box 4.3
Example: exploratory research studies
Use of activity tracking devices
Brief: to explore use of activity trackers among 55–84-year-olds.
End use of data: to help understand usage habits, to identify the language used, to help
design a further detailed study.
Data collection methods: secondary research; observation; in-depth interviews; group
discussions.
Political engagement and voter behaviour
Brief: to understand why some young people do not vote and why others are politically
active.
End use of data: to help understand what is involved, to clarify the nature of the issue,
to define more precisely the problem to be researched and the most suitable approach.
Data collection methods: review of the relevant literature (including academic papers,
previous research, blogs, other social media) on the topic; in-depth interviews with
those who do not vote and those who are politically active.
Eating habits
Brief: to get a picture of the use of food outlets in a railway station including level of
awareness of outlets, profile of users of each outlet, users’ opinions of provision in
general as well as opinion of outlet used.
End use of data: to help define the nature and scope of the problem to be researched;
to help design a full-scale study.
Data collection methods: analysis of sales data from outlets; observation; feedback
from social media; mobile survey of station users; small-scale qualitative study with
short, mini-depth interviews.
Box 4.4
Example: descriptive research studies
IT product
Brief: to find out about market size and structure for product A in country X.
Research questions to include the following:
How big is the market for this product (in value and volume)?
What is the nature of the supply chain?
● Who are the main competitors?
● What are their strengths and weaknesses?
● What are their brands?
●
●
91
M04 The Practice of Market Research 31362.indd 91
27/09/2021 21:45
Chapter 4
Business problem and research problem
What are the values of each of the brands?
What are the most popular products?
● What is the annual sales turnover per product?
● What is the level of advertising/promotional spend per brand per year?
● What form of marketing communications do buyers prefer?
● What is their customer profile by brand and by product?
● Who is involved in the buying decision?
● Who makes the buying decision?
●
●
End use of data: to decide whether or not it is feasible to enter this market with product A.
Research approach: secondary research to gather market size, structure and competitor intelligence data; in-depth interviews with specifiers of the product and those
involved in the buying decision (including IT consultants, internal IT specialists, chief
technology officers); semi-structured phone interviews with end users of the product.
Online government service
Brief: to understand who uses an online government service, why they use it and how
they use it in order to improve the user experience.
Research questions to include the following:
Who uses the service?
Why do they use it?
● When do they use it?
● How often do they use it?
● How do they use it?
● How long is each session?
● What problems do users encounter?
● What do they see as the service’s strengths and weaknesses?
●
●
End use of data: to decide whether or not to redesign the service; to get input into a
redesign.
Research approach: web analytics to get a picture of use/traffic and a profile of users;
in-depth interviews with users to understand context of use, motivation to use, needs,
perceptions and experiences; online survey with representative sample of users to
assess key issues; diary study to record interactions and rate them.
Website design
Brief: to determine which design is the most effective at attracting visitors who go on
to make a purchase.
Research questions to include the following:
Who visits?
When do they visit?
● How often do they visit?
● What do they look at?
● How long do they spend on the page?
● How engaged are they?
●
●
92
M04 The Practice of Market Research 31362.indd 92
27/09/2021 21:45
The nature of the research enquiry
What do they do – browse only; browse and buy?
How many times do they visit before buying?
● When do they drop out before purchasing?
●
●
End use of data: to decide which design is the most effective in converting visitors to
buyers.
Research approach: web analytics to get an idea of traffic and profile of users; heatmaps and session recording to get data on time spent, items viewed, scrolling activity
and so on; user survey to get detail on perceptions and experiences and to determine
what issues exist; split test (or A/B test) – testing two versions of the website to determine which performs better; feedback surveys with those seeing each version.
Causal or explanatory research
Descriptive research will give you a picture – of a market, a set of customers, the
users of a product or service, a set of experiences, for example. But you may need to
know more – you may need to know why the patterns you see in the data exist. In
other words, you want an explanation. A descriptive study may have told you that
some customers prefer brand A and some prefer brand B. You will need explanatory
research to tell you why some prefer A and some prefer B. If this is the case then you
need explanatory or causal research, research that will allow you to explain or help
identify causes or even help predict behaviour. Explanatory or causal research allows
you to rule out rival explanations and come to a conclusion – in other words, it helps
you develop causal explanations.
Causal explanations
A causal explanation might be that sales of brand A are affected by advertising
spend (or that income is related to level of educational attainment). In other words,
a causal explanation says that one thing, call it variable Y (sales of brand A, say, or
income), is affected by another thing, call it variable X (advertising spend on brand
A or educational attainment).
Covariance and correlation
It is relatively easy to see if there is a relationship or an association between two
variables. You can do this by examining cross-tabulations of one variable against the
other or by plotting graphs of one variable against the other or by using the statistical techniques of covariance and correlation. (We look at these methods in a later
chapter.) Sales of brand A might indeed increase if advertising spend is increased, for
example, or income may be greater among those with higher levels of educational
attainment. In a relationship between two variables there may be a direct causal
relationship – the change in Y (sales of product A) is caused directly by X (ad spend
on brand A). On the other hand, there may be an indirect causal relationship – in the
link between X and Y there may be an intervening variable or variables that produce
93
M04 The Practice of Market Research 31362.indd 93
27/09/2021 21:45
Chapter 4
Business problem and research problem
the change in Y. For example, occupation may be the intervening variable through
which educational attainment and income are related. If you want to be able to rule
out the possibility that there is another variable involved, the research design must
allow you to do this. If there is an intervening variable at work, the research design
must enable you to examine its effect.
It is, however, important to remember that just because there is a relationship or
an association between two variables, does not mean that that relationship is a causal
relationship. The two things – the variables – might co-vary, that is, one might follow
the other; a change in X is accompanied by a change in Y – ad spend increases, sales
increase. It might be that X and Y – ad spend and sales – are strongly correlated. But
– and this is very important to remember when you come to analysing your data and
interpreting it – it is possible to observe covariation and correlation without there
being any causal relationship between X and Y at all. For example, the correlation
between advertising spend and sales may be spurious (that is, not causally related at
all); it may be that the correlation you see is the result of another variable, an extraneous (or confounding) variable (competitor activity, for instance, or ‘free’ word-ofmouth promotion on social media). The research must be designed – ­structured – in
such a way that you can determine what sort of a relationship there is, and what
variables are involved in it.
Inferring causation
You can see covariance, association and correlation but you cannot see causation –
you have to infer it. To make sound inferences about cause you must make sure that
the research design allows you to do the following:
look for the presence of association, covariance or correlation;
look for an appropriate time sequence;
● rule out other variables as the cause;
● come to plausible or common-sense conclusions.
●
●
Presence of association, covariance or correlation
If there is a causal relationship between X and Y then you should expect to see an
association between them – a change in X associated with a change in Y. In assessing
the evidence for cause you should take into account the degree of association between
X and Y. You might make one inference on the basis of a strong association and
another on the basis of a weak one. But remember, even if there is an association, no
matter how strong, it does not necessarily mean that there is a causal relationship.
Appropriate time sequence
The effect must follow the cause. If you think that X causes Y (increased ad spend
causes increased sales) and you find that Y in fact precedes X (that sales increased
before spend increased), then you have no evidence for causation. If, however, you
observe that Y does indeed follow X, then you have some evidence for causation.
94
M04 The Practice of Market Research 31362.indd 94
27/09/2021 21:45
The nature of the research enquiry
But in real-life research situations, where you are dealing with complex environments
with lots of things going on, it can be hard to establish a time sequence.
Ability to rule out other variables as the cause
Although you see an association between X and Y, it might be the case that a third
variable is responsible for both, and that the relationship you observe exists because
of the effect of this third variable. For example, occupation may be ‘the cause’ of
income levels rather than educational attainment, and occupation may be ‘caused’
– in part at least – by educational attainment. In fact, there may be a whole causal
chain of other variables linking X and Y. The ability to rule out other variables rests
to some extent with your ability to identify what other variables might be involved.
But even if you identify the key variables – we live in a complex world so it is unlikely
that we would ever be able to identify all the variables in the marketing or social
environment – how do we rule them out? For example, it is unlikely that sales of
brand A are determined by ad spend alone – other elements of the marketing mix as
well as competitor activity are likely to have had some effect. Thus a more realistic,
‘real-world’, causal explanation is that variable Y is affected, directly or indirectly,
by a number of variables besides X.
Plausible or common-sense conclusions
You also need to ask: Is it possible that one thing might have caused the other? How
likely is the explanation? Does it pass the common-sense test? What does other evidence tell us?
When you are defining the problem and preparing to write the research brief, knowing that you need evidence to make inferences about cause will allow you to request or
specify research that will deliver that sort of evidence. In order to know what sort of
evidence you need you must of course understand the research problem clearly and in
detail. You need to think about what relationships might exist between variables, and
what the obvious explanations and the alternative explanations for these relationships
might be; and you need to think about what interpretations you might place on the
data. This front-end thinking is crucial. If all of these things are clearly thought out
– possible relationships, explanations and interpretations – it is much easier to commission – and to design – research that will deliver the evidence needed to make sound
causal inferences. At the same time, you need to recognise that you will never be able
to collect ‘perfect information’ and that your inferences will be only that, inferences
and not fact. Research will always be constrained by the complexities of the social and
marketing environment and those of human behaviour and attitudes.
Identifying the problem and the decision that the client or end user needs to take
is the first step in designing research. Once you know what the client needs to do you
can clarify the information needed to help them make that decision. You will know
not only what information is needed but how and in what context that information is
to be used. You have defined the problem and in so doing have uncovered what sort
of evidence you need to address it and thus what type or types of research enquiry
will be involved. This is essential information to have for designing good-quality,
actionable research.
95
M04 The Practice of Market Research 31362.indd 95
27/09/2021 21:45
Chapter 4
Business problem and research problem
While we have made clear distinctions between these three types of research
enquiries, you will likely know from your own experience that research projects do
not usually fall neatly into only one of these categories. It is more often the case that
the purpose of a research project is two- or three-fold: to explore and describe; to
explore, describe and explain; or to describe and explain. You will notice this when
you come to define the problem and plan the research – you may find yourself wording the research objectives in just this way.
Industry Insight 4.1 looked at BBC World Service English and its desire to connect
with a younger digital/non-radio audience. WSE believed that podcasts would be a
great way of doing this. It needed to devise a podcast strategy that would work across
12 diverse markets on which it had little or no existing information. It commissioned
research to address this information gap. But what sort of research enquiry was this?
Have a look at Industry Insight 4.3. It describes the research conducted by research
agency Firefish to address the research problem and so BBC WSE’s business problem.
Industry Insight 4.3
What type of enquiry?
BBC World Service English (WSE) wanted to connect with younger, digital/non-radio audiences via
podcasts. The question was, how to do this successfully? It knew very little about the 12 markets it had identified. It commissioned Firefish to
conduct research. Firefish proposed a three-stage
approach.
market researchers gathered data on behaviour
and attitudes using a range of self-completion
tasks combined with daily diary tasks conducted
digitally over a seven-day period. The information collected helped them get a picture of podcast
consumption from discovery to key occasions and
habits to content and what drives people to listen.
Stage 1: A ‘blast’ of exploratory research
Stage 3: A ‘deeper delve’ – a further
descriptive study
The ‘blast’ stage comprised online qualitative
interviews with 60 participants – a mix of light,
medium and heavy podcast users – in each of the
12 countries. These interviews provided important
information on key issues, including how participants first came to podcasts; listening locations
and occasions; key motivations; preferred and
most influential podcasts; and how they defined
and understood what podcasts are. This information allowed the researchers to get an overview of
the podcast landscape, to create snapshots of the
podcast market in each country and to compare
countries.
Stage 2: A ‘deep delve’ – an in-depth
descriptive study
The ‘deep delve’ focused on four lead markets:
India, USA, Netherlands and Nigeria. In each
The ‘deeper delve’ involved hour-long online
video depth interviews to gather depth and detail
on new, emergent hypotheses.
Other complementary research
To complement this three-tier study, BBC World
Service added questions to nationally representative media surveys in more than 20 countries. The
data from these surveys enabled the research team
to size the podcast audiences and place them into
the context of broader audio (digital and linear)
markets. The BBC WS team also mined existing
podcast analytics to understand responses to BBC
content.
Source: Adapted from Firefish and BBC World Service, ‘Beyond
the hype: Shaping a global podcasting strategy’. MRS Awards
2017.
96
M04 The Practice of Market Research 31362.indd 96
27/09/2021 21:45
Formulating research objectives
Formulating research objectives
You have an understanding of the client’s business problem and a clear definition of
the research problem and you know the sort of research enquiry you need. You can
now move from the broad research problem to the more specific research objectives.
In other words, you can specify what it is you need the research to tell you. This is a
crucial stage – it will clarify the sort of information you need and give you – or the
researcher who replies to your research brief – a framework on which to design the
research. To design effective research the research objectives should therefore be as
specific and precise as possible. Box 4.5 gives some examples.
Here is an example research scenario: imagine that you have been asked to evaluate a ‘healthy ageing’ programme run in gyms across the country. Where do you
start? You might start by exploring the meaning of ‘healthy ageing’, the nature of
the ‘healthy ageing’ programme, what it sets out to achieve, what it involves, who
it is aimed at, who the actual participants are, how many enrolled, how many completed the programme, how many enquired about it but did not enrol in it. Next, you
might want to describe or profile those who completed the programme and those
who enrolled but did not complete it. You might want to understand or explain why
some people completed it and some did not. You might even want to find out why
people enrolled and why others did not.
Box 4.5
Examples: types of objectives
Here are some examples of ‘types’ of research objectives (adapted from Ritchie and
Spencer, 1994). Besides exploratory, descriptive and explanatory, Ritchie and Spencer
also include evaluative and strategic objectives:
Exploratory: discovering or clarifying the form and nature of something
Examples:
What is the nature of the market or subject area?
What products or services are on offer?
Who buys or uses a product or service?
How do people talk about a service or product?
Contextual or descriptive: describing the form and nature of something
Examples:
What is the nature of someone’s experience?
What processes are at work?
What wants or needs does the population have?
What perceptions do people hold of a product or service?
97
M04 The Practice of Market Research 31362.indd 97
27/09/2021 21:45
Chapter 4
Business problem and research problem
Diagnostic or explanatory: examining the causes or reasons for something
Examples:
What factors are involved in a decision?
Why is a particular pathway or action or decision taken?
What factors might explain perceptions or attitudes?
Why is a product or service popular or unpopular?
Evaluative: assessing the value or effectiveness of something
Examples:
How did a product or service perform during its trial period?
How were the marketing objectives achieved?
What factors affected the performance of a product or service?
How does a product or service fit the brand or corporate identity?
Strategic: identifying new plans, actions, services, products
Examples:
What type of product or service will fill the gap in the market?
What actions must be taken to make a product or service successful?
What must be done to improve a process or product or service?
What marketing strategy is needed for success in a new market?
The next steps
At this point in the research process you should have answers to the following
questions:
What is the business or the decision maker’s problem?
What is the research problem?
● How will the research findings be used?
● What is the nature of the research enquiry?
● What kinds of answers are we looking for?
● What sort of evidence do we need?
● What are the research objectives?
●
●
The research objectives tell you what information or evidence you need. Where
do you get that information or evidence? The building blocks of information or
evidence are data: ‘. . . the raw material produced by abstracting the world into
categories, measures and other representational forms – numbers, characters, symbols, images, sounds, electromagnetic waves, bits . . . ’ (Kitchin, 2014). Where do we
find these data? There are two options: look for data that already exist; or generate
what you need.
Data that already exist are often referred to as secondary data. These are data
generated by someone else, or by a system or device. In re-using them, you are putting them to a second (secondary) use. We look in more detail at secondary data in
Chapters 8 and 9. At this point it is important to note that if you plan to use existing
or secondary data then you need to find out as much as possible about them so that
98
M04 The Practice of Market Research 31362.indd 98
27/09/2021 21:45
Formulating research objectives
you can assess their suitability and quality for addressing your research objectives.
Here are the key questions you need to ask:
Are they relevant to our purpose?
Can we get access to them?
● Is it ethical and legal to use them for our purpose?
● How good are they? Are they valid and reliable?
● Are they sufficient for our purpose?
● What is the cost of using them?
●
●
If upon review you find that there are no useable existing data with which to address
your research objectives then you will need to generate your own. When you generate
your own data they are called primary data. Generating primary data via research is
called primary research. To generate primary data you design a research project. In
the design and implementation of that research project, you take steps to ensure that
the data generated are suitable and sufficient for your purposes, that they are valid
and reliable, and ethically and legally robust.
Box 4.6
Guiding principles: validity and reliability
We’ve mentioned validity and reliability here, concepts we introduced in Chapter 1
as indicating the quality and rigour of research and data. Validity refers to how well
a research project, including its design and structure, sample, method of data collection and measures or questions used, delivers accurate, clear and unambiguous
evidence with which to address the research objectives. Reliability is about consistency and repeatability. It is important because it is generally accepted (Kitchin, 2014)
that the more consistent a measure is, the more confidence you can have in it. You
want your research and its findings to be both valid and reliable. Here at the start
of a project, identifying the problem, formulating research objectives and thinking
ahead to the design, it is worth taking a look at two types of validity: internal validity
and external validity.
Internal validity
Internal validity refers to the ability of the research to deliver credible evidence. It is the
job of the research design to ensure that the research has internal validity. In causal or
explanatory research, for example, it is about the ability of the research design to allow
us to make links or associations between variables, to rule out alternative explanations
or rival hypotheses and to make inferences about causality. Internal validity must also
be considered when designing the data collection instrument and constructing questions. In this context, internal validity refers to the ability of the questions to measure
what it is we think they are measuring: do the data collected represent the ‘thing’, the
phenomenon, they are supposed to represent?
External validity
Research is conducted and data are gathered from samples, subsets of items or people, that endeavour to represent the whole population of interest. We rely on samples
99
M04 The Practice of Market Research 31362.indd 99
27/09/2021 21:45
Chapter 4
Business problem and research problem
because it is usually too difficult and too costly to conduct research and gather or analyse data from the whole population of interest. When a piece of research has external
validity, it means that we can generalise from the findings of that research – ­conducted
among the sample or sub-set – to the wider population. The ability to do this is a key
aim in almost all research enquiries.
We have mentioned the term ‘population of interest’. ‘Population of interest’ is the
term used to describe the group (people, organisations, items) about which we want to
report or make generalisations. If the research project is about the awareness and use of
food outlets at a railway station (Box 4.1), the population of interest might well be all
users of the railway station. If it is about recycling habits of households in a particular
geographic or administrative area, the population of households in that area might well
be the population of interest. If it is is to determine the website design most effective
at attracting visitors who go on to make a purchase (Box 4.1), the population of interest might be all visitors to the website within a specified time frame. Knowing who or
what your population of interest is an important piece of information in planning and
designing your research. In Industry Insight 4.4, trainee researchers from research agency
BMRB International describe how they arrived at the population of interest for their
research and how important it is to be precise in defining the population of interest.
Industry Insight 4.4
Excluded from school
The Prince’s Trust is an organisation concerned
with creating opportunities for disadvantaged
young people. The Trust believed that young people were at risk from social isolation as a result
of exclusion from school. A greater understanding of school excludees, their experiences, and the
reasons behind exclusion was needed to enable
the Trust to tailor its schemes to help excludees.
It was concerned with giving school excludees a
voice. Thus, the objectives of the research were
to explore the attitudes and aspirations of young
people who had been excluded from school to
ascertain the problems they faced and to find out
what they believed they needed to succeed. Within
this broader aim, the research aimed to investigate attitudes and opinions of excludees towards
school and exclusion, and to examine how these
attitudes differed from those of non-excludees.
But which type of excludees? There are three
official main types of exclusion: fixed term
(suspension), which can be an exclusion of up to
45 days consecutively; indefinite, an exclusion for
an unspecified number of days; and permanent
(expulsion) whereby the pupil is prohibited from
returning to that particular school. It was decided
to interview only permanently excluded pupils as it
was felt these young people were most at risk and
were more likely to experience difficulties in being
reintegrated into mainstream education. However,
research revealed that in addition to official
exclusions there are also many ‘unofficial’ exclusions
per annum, where a pupil is asked to leave voluntarily
and may not return to that school. Hence, a definition
restricted to official excludees would have missed all
‘unofficial’ excludees and been too narrow.
Therefore, for the purpose of this study, permanent exclusion was defined as ‘those pupils who
have been excluded from school either officially or
unofficially, and are prohibited from returning to
that particular school’.
Source: Adapted from Capron, M., Jeeawody, F. and Parnell, A.
(2002) ‘Never work with children and graduates? BMRB’s class
of 2001 demonstrate insight to action’, MRS Conference,
www.mrs.org.uk. Used with permission.
100
M04 The Practice of Market Research 31362.indd 100
27/09/2021 21:45
Identifying ethical and legal issues
Error and bias
You will also come across references to error and bias in research. Error, put simply,
is about mistakes. You might introduce error into your research project by, say, making a mistake in defining the population of interest; or if you make use of a sampling
frame that does not represent your population of interest accurately; or if the ‘wrong’
participants are selected to take part; or if you word a question ambiguously or in
a misleading way; or if you wrongly code a response or piece of data. When there
is a consistent pattern in error it is called bias. You can introduce bias in the design
of the research, in the sampling design and technique you use, in the way you select
data, in the design of the data collection tool, in the way you process, analyse and
interpret the data and in the way you report the findings.
The way to address issues of error and bias is, first of all, to recognise that they
exist, to know when and where they might occur, and to follow procedures to ensure
that they are minimised, if not eliminated. One of the main ways of addressing error
and bias, and validity and reliability, is research design.
Research design and why it is important
The purpose of research design is to structure the research so that it delivers the evidence necessary to answer the research problem as accurately, clearly and unequivocally as possible. A sound research design is the framework on which good quality
research is built. If you get the research design wrong, you will not be able to provide
credible or even useful evidence to address the research problem; and you will not
be able to make credible claims about what you ‘know’ based on the research, and
so the time and money spent doing the research will largely be wasted. We look in
detail at the process of research design and all it entails in subsequent chapters. The
key thing to remember for now is the need to be clear about the business problem
and the research problem. Without this you will not be able to move forward to plan
and design the research.
Identifying ethical and legal issues
Before moving to that next stage it is important to determine if there may be any
ethical or legal issues that might arise across the life of your project. You need to
identify these so that you can address them in the design and planning stages and
manage them during the project. In fact, once you know what your research problem is and what the project is likely to entail you should conduct an ethical review
(Goddard, 2017).
Doing an ethical review
Doing an ethical review is the process of identifying and recording the ethical
issues you think or know you might encounter in the course of your project. We
looked in detail at ethical issues in Chapter 2 and at the codes of conduct that
101
M04 The Practice of Market Research 31362.indd 101
27/09/2021 21:45
Chapter 4
Business problem and research problem
arise from them, in particular the MRS Code of Conduct (2019). Ethical issues
include issues that relate to data protection and privacy and so intersect with
legal and regulatory issues. At first glance you may not think that your project
has any obvious ethical issues, it may be genuinely low risk. On the other hand,
it may involve a sensitive topic, or participants from a vulnerable group and so
it may be high risk. Whether you view it as low or high risk, or somewhere in
between, it is good practice to do a review: it should benefit you as the researcher,
the research participants and the quality of the end result. Not doing a review has
costs: for example, you fail to identify (and therefore to address) risks to participants, potentially exposing them to harm; negative feedback from participants
and the public can result in damage to the reputation of your organisation and/
or the client organisation.
You might find that a review is required by your organisation, by your client
or by whoever is funding the research. Even if you are a member of a professional
body and you have agreed to abide by its standards and code of conduct, you are
not exempt from doing a review. Box 4.7 gives some questions to get you started. Be
aware, however, that these do make up an entire review. As Salmons (2016) notes,
it is a process not an event.
Box 4.7
Starting an ethical review: some questions
What is the aim of your research?
Who will benefit from it?
● Who will be involved?
● Are there any conflicts of interest?
● Where is the research taking place?
● What does this mean in relation to local ethical and legal requirements, culture and
customs, political issues?
● How will you monitor ethical issues during the project?
● How will you handle unforeseen or adverse events?
● Will you be collecting and/or processing personal data? If so, for what purpose?
● On what grounds will you be processing data?
● What are the potential sources of harm to participants, to researchers, to the
data?
● How will you address any potential risks of harm?
● How will you store the data?
● To whom do you plan to communicate (and disseminate) the findings?
● How do you plan to communicate (and disseminate) the findings?
●
●
102
M04 The Practice of Market Research 31362.indd 102
27/09/2021 21:45
References
Chapter summary
●
The start of a research project typically involves the following:
– identifying and understanding the business problem;
– defining the research problem clearly and accurately;
– establishing the end use to which this information will be put; and
– formulating the research objectives.
●
●
●
●
●
The importance of spending time at this stage cannot be overstated. It is essential
if good-quality, actionable research is to be produced. Decisions about key
aspects of research design rely on having a clear understanding of the business
problem and the research problem.
Conducting informal and more formal exploratory research (such as a literature
review) at this stage can be very helpful. It can help you understand the business
problem, the wider context, and how others may have addressed similar problems
in the past. It may inform the research objectives and/or direct the design of your
research.
There are three main types of research enquiry: exploratory, descriptive and
explanatory or causal. Clarifying the nature of the enquiry will help you clarify your
research objectives, which in turn will help you make decisions about research
design.
It is important at this stage to think about the population of interest. Knowing this
will help you plan and design your research.
When you have some idea of the sort of research you might do, it is useful to
think about the ethical, legal and regulatory issues you might encounter. It is good
practice to begin an ethical review as early as possible to address any potential
issues and plan how to manage them.
Exercises
1 Review the examples in Box 4.1. For each, describe the following: (a) the business
or decision maker’s problem; (b) the research objectives; and (c) the population of
interest.
References
Capron, M., Jeeawody, F. and Pamell, A. (2002) ‘Never work with children and graduates?
BMRB’s class of 2001 demonstrate insight to action’, Proceedings of the Market Research
Society Conference, London: MRS.
103
M04 The Practice of Market Research 31362.indd 103
27/09/2021 21:45
Chapter 4
Business problem and research problem
Firefish and BBC World Service (2017) ‘Beyond the hype: Shaping a global podcasting strategy’, MRS Awards.
Firefish and PayPal (2016) ‘The evolution of PayPal: New purpose, new money’, MRS Awards.
Goddard, M. (2017) ‘Reinforcing research ethics in the digital age,’ Impact, 19, pp. 82–3.
Kitchin, R. (2014) The Data Revolution, London: Sage.
MRS (2019) Code of Conduct, London: MRS.
Murcott, A. (1997) ‘The PhD: Some informal notes’, unpublished paper, School of Health and
Social Care, South Bank University, London.
Ritchie, J. and Spencer, L. (1994) ‘Qualitative data analysis for applied policy research’, in
Bryman, R. and Burgess, A. (eds) Analysing Qualitative Data, London: Routledge.
Salmons, J. (2016) Doing Qualitative Research Online, London: Sage.
Silverman, D. (2005) Doing Qualitative Research, 2nd edition, London: Sage.
Vriens, M., Brokaw, S., Rademaker, D. and Verhulst, R. (2019) ‘The marketing research curriculum: closing the practitioner-academic gaps’, International Journal of Market Research,
61, 5, pp. 492–501.
Recommended reading
Bell, J. and Waters, S. (2018) Doing Your Research Project: A Guide for First Time Researchers, Amacom.
Cresswell, J. (2013) Research Design: Qualitative, Quantitative and Mixed Methods
Approaches, London: Sage.
De Vaus, D. (2001) Research Design in Social Research, London: Sage.
Newton, R. (2009) The Project Manager: Mastering the Art of Delivery, 2nd edition, London:
FT/Prentice Hall. See Chapter 3 ‘What actually is your project?’
Yin, R. (2018) Case Study Research and Applications, 6th edition, London: Sage.
For more on doing an ethical review, see the Consortium of European Social Science Data
Archives (CESSDA): https://www.cessda.eu/Training/Training-Resources/Library/
Data-Management-Expert-Guide/5.-Protect/Ethical-review-process
You might also find the Economic and Social Research Council (ESRC) website helpful: https://
esrc.ukri.org/funding/guidance-for-applicants/research-ethics/
104
M04 The Practice of Market Research 31362.indd 104
27/09/2021 21:45
M04 The Practice of Market Research 31362.indd 105
27/09/2021 21:45
Chapter 5
Research design
Introduction
In Chapter 4 we looked at what you need to know before you design a project:
the business problem, the research problem and the nature of the research
enquiry – exploratory, descriptive, explanatory and causal. Now in this chapter
we look at the process of research design and at the main designs. A number of
examples illustrate the different types in context.
Topics covered
The process of research design
● Types of research design
● Validity and reliability in the context of research design.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project; Topic 2: Guiding Principles; and Topic 3:
Selecting the research design and planning the approach.
M05 The Practice of Market Research 31362.indd 106
27/09/2021 21:45
What you should get from this chapter
At the end of this chapter you should be able to:
understand what is meant by research design and what the process of research design
involves;
● understand key aspects of the main research designs;
● understand the concepts of validity and reliability in the context of research design.
●
107
M05 The Practice of Market Research 31362.indd 107
27/09/2021 21:45
Chapter 5
Research design
The process of research design
Before we look in detail at the different types of research design, it is worth considering what we mean by research design. Some people confuse research design with
the choice of type of data or method of data collection, seeing it as a decision to use
qualitative or quantitative methods, for example, or to collect data using an online
survey rather than face-to-face interviews. All these decisions are part of the research
design process but they are not the whole of it. It is easiest to think of research design
as having two levels.
At the first level, research design is about the logic of the research, its framework or
structure. It is at this level, given what we know about the problem to be researched
and the sort of research enquiry that it demands – exploratory, descriptive or explanatory – that we make decisions about the structure of the research. This structure
may comprise a particular research design – a cross-sectional, a longitudinal or an
experimental design or a case study. Part of the research design process also involves
decisions about the units of analysis – the ‘who’ or ‘what’ to question or observe.
At the second level, research design is about deciding on the ‘mechanics’ of the
research – what type of study or data (primary or secondary, qualitative or quantitative or a combination), what method of data collection, what sampling strategy,
and so on.
So, in summary: the first level is about designing the overall structure of the
research so that it can deliver the sort of evidence you need to answer the research
problem; the second level concerns decisions about how to collect that evidence. The
steps in the research design process are as follows:
First-level design issues
Identifying the business problem
Defining the research problem
● Understanding the nature of the enquiry
● Formulating the research objectives
● Thinking about the end use of the data
● Deciding on the sort of evidence needed
● Deciding on the unit or units of analysis
● Deciding on the logic and structure of the research
● Choosing the research design that will deliver the evidence.
●
●
Second-level design issues
Deciding on the type of data and the method of data collection:
Primary or secondary or both
Quantitative or qualitative or both
● Face to face, telephone, internet/online; groups, in-depth interviews and so on.
●
●
108
M05 The Practice of Market Research 31362.indd 108
27/09/2021 21:45
The process of research design
Designing a sampling strategy
Identifying and defining the population of interest
Identifying the sampling units and the sample elements
● Choosing a sampling approach
● Choosing a sample size.
●
●
Designing the data collection instrument
●
●
Defining the concepts, choosing indicators, operationalising the concepts
Designing the questionnaire or discussion guide.
Why is research design important?
The purpose of research design is to structure the research so that it delivers the evidence necessary to answer the research problem as accurately, clearly and unequivocally as possible. A sound research design is the framework on which good quality
research is built. If you get the research design wrong you will not be able to provide
credible or even useful evidence to address the research problem; you will not be able
to make credible claims about what you ‘know’ based on the research, and so the
time and money spent doing the research will largely be wasted. The key is, of course,
understanding the business or decision maker’s problem, the information needs and
the research problem – without a clear idea of what you need to know you will not
be able to plan the best way of finding out.
Validity and research design
Validity is a key concept in assessing the quality of research. It refers to how well a
research design (and the research method and the measures or questions used) delivers accurate, clear and unambiguous evidence with which to answer the research
problem. It is an indicator of whether the research measures what it claims to measure. There are two types of validity: internal and external validity. We looked at these
in Chapter 4. Here’s a short recap:
Internal validity in the context of research design refers to the ability of the research
to deliver credible evidence to address the research problem. It is the job of the
research design to ensure that the research has internal validity. In causal or explanatory research, for example, it is about the ability of the research design to allow us
to make links or associations between variables, to rule out alternative explanations
or rival hypotheses and to make inferences about causality.
When a piece of research has external validity, it means that we can generalise
from the research conducted among the sample (or in the specific setting) to the wider
population (or setting). The ability to generalise is a key aim in almost all research
enquiries and must be considered at the research design stage as well as at the sample
design stage.
109
M05 The Practice of Market Research 31362.indd 109
27/09/2021 21:45
Chapter 5
Research design
Types of research design
There are four main types of research design:
Cross-sectional study
Longitudinal study
● Experiment
● Case study.
●
●
We look now at what each of them involves, and what sort of evidence each one
can deliver.
Cross-sectional research design
A cross-sectional research design is probably the most common type of design in
market and social research. With a cross-sectional design you collect data from a
cross-section of a population of interest at one point in time. The Census of Population is an example of a cross-sectional study – it describes the make-up of the
population at one particular point in time. Most ad hoc research – research designed
to gather information to address a specific problem – is cross-sectional. Usage and
attitude surveys, for example, use a cross-sectional design, as does an advertising
pre-test. A single cross-sectional design involves only one wave or round of data
collection – data are collected from a sample on one occasion only. It is a snapshot.
A repeated cross-sectional design involves conducting more than one wave of (more
or less) the same research with an independent or fresh sample at each wave. The
use of an independent sample at each round of data collection is what distinguishes
repeated cross-sectional design from longitudinal research. In longitudinal or panel
research, data are collected from the same sample on more than one occasion. This
is analogous to a movie or film rather than to a snapshot.
Uses
A cross-sectional design can be used to provide data for an exploratory or descriptive
research enquiry. It can also be used for explanatory enquiry, up to a certain point –
it can be used to look for and examine relationships between variables; to test out
ideas and hypotheses; to help decide which explanation or theory best fits with the
data; and to help establish causal direction but not to prove cause. With a repeated
cross-sectional design – a snapshot at one point in time, followed by another snapshot
at a suitable interval – you can examine trends over time. Comparison of Census
data from different years is one example; pre- and post-advertising tests are another.
Most tracking studies tend to use a repeated cross-sectional design. A repeated crosssectional design allows you to compare, for example, data from 16–24-year-olds
at one point in time with data from 16–24-year-olds at another point in time, or
COVID-19 infection rates one week and infection rates the next week.
With a cross-sectional design, and this is something that distinguishes it from
experimental research design, we rely on there being differences within the sample
in order to be able to make comparisons between different groups. In experimental
research design, we create the differences within the test sample by manipulating one
110
M05 The Practice of Market Research 31362.indd 110
27/09/2021 21:45
Types of research design
Box 5.1
Examples: uses of cross-sectional design
Single cross-sectional design: buying a car
Aim: to describe and understand the decision-making process involved in buying a car.
Research method: qualitative – mini group discussions, paired (family) and individual
in-depth interviews.
Analysis: comparisons made between those who took a test drive and bought a car
and those who took a test drive and did not buy.
Single cross-sectional design: assessing needs
Aim: to determine the health and social care needs of older people.
Research method: quantitative survey – face to face, in-home interviews.
Analysis: comparisons made (inter alia) between those living alone and those living with
family members; and between those on state pension only and those on state pension
plus private pension or other income.
Repeated cross-sectional design: pre- and post-advertising test
Aim: to determine the effect of an advertising campaign on attitudes to brand A.
Research method: quantitative pre- and post-advertising surveys with independent
samples using face-to-face interviews.
Analysis: comparisons made (inter alia) between regular buyers of brand A, occasional
buyers and rejectors of brand A, pre- and post-advertising.
of the variables – the independent or explanatory variable – in order to see if it causes
a change in another variable – the dependent variable. In a cross-sectional design,
having specified the relevant sample and asked the relevant questions, we examine
the data to see what relationships or differences exist within the sample.
Industry Insight 5.1 gives another example of a repeated cross-sectional survey,
the UK COVID-19 Infection Survey.
Industry Insight 5.1
Research on a pandemic
To help understand the spread of the COVID-19
infection, and so to protect citizens and to
prevent healthcare systems from being overwhelmed, governments needed information on
the spread of the virus over time. In the UK to
gather this information the Office of National
Statistics (ONS) with Oxford University and
the National Centre for Social Research, among
others, designed the COVID-19 Infection Survey, repeated cross-sectional surveys of representative households in the UK (ONS, 2020).
The study was funded by the UK Department
111
M05 The Practice of Market Research 31362.indd 111
27/09/2021 21:45
Chapter 5
Research design
of Health and Social Care. It was designed to
measure the following:
how many people test positive for COVID-19
infection at a given point in time, regardless of
whether they report experiencing symptoms;
● the average number of new infections per week
over the course of the study; and
● the number of people who test positive for antibodies, to indicate how many people are ever
likely to have had the infection.
●
The survey began in April 2020 with a pilot study
phase. Invitations to take part were sent to 20,000
households in England. The researchers assumed
that half (50 per cent) would opt in. It aimed to
enrol these 10,000 households at a rate of 2,500
per week for a month. After this initial round,
about 5,000 additional households were invited
to take part each week.
The survey asked everyone in the household aged
two years and over to have a test, a nose and throat
swab, to determine if they had the virus. All of those
aged 12 years and over were asked to answer some
short questions during a home visit by a trained
survey worker. Some of those aged 16 years and
over were asked to give a blood sample to determine if they had had the virus – except in households where someone had symptoms compatible
with COVID-19 or was self-isolating or shielding.
The pilot study phase of the survey ran between
26 April and 28 June 2020. In this time a total
of 34,992 individuals aged two and over from
16,722 private households took part (Pouwels
et al., 2020).
The findings contributed to the estimates of the
rate of transmission calculated by the UK government’s Scientific Advisory Group for Emergencies
(SAGE). They also provided information on the
socio-demographic characteristics of both people
and households with COVID-19.
You can find out more about the survey at
the ONS website and at the Oxford University
Covid Infection Survey site where results are also
available.
Longitudinal research design
Longitudinal research is common in market and social research, where it is often
referred to as panel design. The main reason for using such a design is to monitor
things – attitudes, behaviour, experiences, perspectives – over a period of time. With a
longitudinal or panel design, you collect data from the same sample (of individuals or
organisations, for example) on more than one occasion. Whereas the cross-sectional
design provides a ‘snapshot’ of a situation, the longitudinal design provides a series
of snapshots (of the same people) over a period of time that can be joined together
to give a moving image. The number and frequency of the snapshots or data collection points depend largely on the research objectives (and the available budget). For
example, if the purpose of the research is to look at the immediate, short-term impact
of an advertising campaign, a relatively small number of data collection points, fairly
closely spaced in time, may suffice; to examine the longer-term impact of advertising on a brand may require a relatively large number of data collection points over
many years.
What distinguishes longitudinal designs from repeated cross-sectional designs is
that in longitudinal designs data are collected from the same sample on more than
one occasion, rather than from independent or fresh samples each time. There is
some overlap in the definitions of longitudinal and repeated cross-sectional designs,
best illustrated by the way in which tracking studies or trend studies are classified. In
some texts, tracking studies are classified as a longitudinal design, the argument being
that the sample at each wave is effectively the same (albeit composed of different
112
M05 The Practice of Market Research 31362.indd 112
27/09/2021 21:45
Types of research design
individuals). In others, and in this one, tracking studies are classed as a cross-sectional
design because although the samples are matched at each wave they are nevertheless
independent samples. Secondary analysis of panel data – in which you re-analyse the
data for a purpose other than the original one – is also commonplace.
Industry Insight 5.2 shows the rationale for choosing a panel approach rather
than ad hoc, cross-sectional surveys to understand consumer behaviour and what
drives it. It highlights the advantages of a panel design and the disadvantages of a
cross-sectional design in the context of Unilever Bestfoods’ needs and the end use of
the data.
Industry Insight 5.2
Seeing the big picture
1 Depth and breadth
Panel data contain multiple layers of information
including information on brand, product category, size, variant, promotion, where purchased,
price, time of purchase, and so on. Over 500 different product categories are measured at any one
time. This level of data is often unachievable in ad
hoc (one-off) Usage and Attitude (U&A) studies
due to the complexity and costs of data collection. In U&As, the scope of the data needs to be
decided up front, and this limits the applicability
of the findings at a later stage. The sample size for
this study using the panel was over 5,700 – ten
times the base sizes used in previous U&As. More
detailed analysis could be completed. There were
sufficient and statistically reliable base sizes on the
detailed sub-samples.
2 Measurability
The panel data contained data on actual products
purchased. These are tracked automatically and
verified. Most U&As rely on the consumer recalling what they did. Even for daily diaries, it is often
difficult to remember accurately exactly what was
done. Most business measurements cannot be reliably and accurately done from ad hoc standalone
studies.
3 Accountability
The nature of panel data is that it is continuous
over time. Thus, things can be accurately measured over different periods of time on the same
consumers. Ad hoc standalone studies are static –
the data are collected at one point in time. Panel
data are collected continuously for the same people (with three years available for analysis). Specific marketing and sales activities can be measured
against the targets for which they were defined.
Product launches can be tracked to compare actual
buyers versus proposed; the impact of promotions
can be measured. To do this using ad hoc studies
would be expensive, time consuming, and lack the
depth and breadth of information needed.
4 Cost-effectiveness
The client, Unilever Bestfoods, needed to invest
in one study only and was able to use the results
for many different brands and product categories. This revolutionised the efficiency of market
research spend.
5 Understanding purchase with
consumption data
The full understanding of the consumer was developed from all the data collected from the panel.
Data from a separate piece of data collection
using a diary and carried out with a sub-set of
panel members enabled detailed analysis of the
relationships between purchase and consumption.
Broader, more useful explanations of behaviour
were developed.
6 Understanding drivers of behaviour
Each part of the panel study was analysed and
interpreted together. Psychographic attributes
113
M05 The Practice of Market Research 31362.indd 113
27/09/2021 21:45
Chapter 5
Research design
(values collected via a questionnaire) were interpreted and understood in respect of who people
were (demographics), actual purchase (panel purchase data), and consumption (diary data). Each
helped to explain the other. Each would have been
less effective on its own. The level of insight we
achieved into consumer behaviour and its drivers
from this one data source allowed us to predict
future behaviour, not just understand what had
happened in the past.
Source: Adapted from Gibson, S., Teanby, D. and Donaldson, S.
(2004) ‘Bridging the gap between dreams and reality . . . building
holistic insights from an integrated consumer understanding’,
MRS Conference, www.mrs.org.uk. Used with permission.
Uses
The main application of longitudinal design is to monitor changes in the marketing
or social environment, changes that occur in the normal course of things and events
that are planned, for example changes as a result of an advertising campaign, a new
product launch or an election. Longitudinal design can be used to provide data for
descriptive research enquiry. Although it cannot be used to prove cause, it can be
used to achieve the following:
explore and examine relationships between variables;
establish the time order of events or changes, and age or historical effects;
● help decide which explanation or theory best fits with the data;
● help establish causal direction (rather than prove cause).
●
●
Drop-out and replacement in panels
As time passes the universe or population from which the panel is recruited changes –
this is especially so in fast-moving markets and in new markets as penetration and use
increases (or declines). Also, the panel members themselves change – for one thing,
they get older. The longer a panel lasts the greater the chance that panel members
will drop out. A key question in panel design and management is whether or not to
replace them. If you do not, you may end up with a very small sample and one that
is likely to be unrepresentative (and so poor in terms of external validity) – those
who drop out being in all likelihood different from those who stay. If you decide to
replace the drop-outs, how do you go about it? There are two approaches to replacement. The first is to find out the characteristics or profile of the drop-outs and recruit
replacements with exactly the same profile. It is of course important to be aware that
you can never know all the characteristics of an individual (there may be characteristics that you do not use in recruiting participants but which nevertheless have a
bearing on other characteristics). The second approach is the use of a rolling panel
design, a technique used in most market research panels, indeed in any long-term
panel which puts a fairly heavy burden on participants and so results in participant
fatigue and high drop-out or attrition rates. In this approach you constantly or at
regular intervals refresh the panel membership by replacing ‘old’ or existing members
with new ones, making sure that the overall panel profile remains the same throughout. The advantage of the rolling panel method is that it smooths out drop-out. It
also smooths out ‘conditioning’ which affects the quality of the data. ‘Conditioning’
is the phenomenon of responding to questions in a way that is ‘conditioned’ by having responded to the same or similar questions in previous rounds of data collection.
114
M05 The Practice of Market Research 31362.indd 114
27/09/2021 21:45
Types of research design
Box 5.2
Examples: uses of longitudinal or panel research
Voters’ attitudes and intentions
Aim: to determine reactions to events; to understand the decision-making processes in
relation to voting intentions in the run-up to an election; to observe the impact of events
on attitudes and intentions; to gain reaction to the content and wording of campaign
messages.
Design: three rounds of fieldwork with the same sample of voters over a three-month
period, each round timed to follow key political events and decisions.
Method: qualitative depth interviews.
Behaviour in the fast food market
Aim: to understand buying behaviour – frequency, brands, type of meal and usage.
Design and method: quantitative self-completion questionnaire and diary recording meal
type, order method, time of day, delivery or collection method, location of consumption,
meal accompaniments and so on; continuous with weekly downloading of data.
Data quality is also an issue with new recruits to the panel: joiners will not have
provided data in the same time period as those already on the panel (which has implications for looking at data at the individual rather than the aggregate level); they are
likely to be more enthusiastic (less conditioned) than established panel members and
so the data they provide will be different and not comparable with that of established
members. The solution here is to ignore data from new panel members for the first
one or even two data collection periods of their membership.
Although panel designs are associated with quantitative research, they can be used
in qualitative research, as Industry Insight 5.3 shows.
Industry Insight 5.3
What happens when you win the lottery?
The overall aim of this research was to understand
the impact over time on small voluntary organisations that receive Charities Board funding from
the National Lottery. The study sample comprised
organisations that had received funding over a threeyear period. The core objectives of the study were to:
establish a clear picture of the goals and
achievements of each participant charity;
● evaluate the impact of the Charities Board
funding over time;
●
examine perceptions of the Charities Board
throughout;
● compare and contrast the Charities Board with
other sources of charitable funding;
● establish useful advice for other small charities
contemplating approaching the Charities Board
for funding; and
● identify aspects of the Charities Board practices and procedures that could be improved to
the mutual advantage of the charities and the
funder.
●
115
M05 The Practice of Market Research 31362.indd 115
27/09/2021 21:45
Chapter 5
Research design
Research design
A longitudinal approach using qualitative data
collection methods was needed to allow analysis
of the long-term outcomes. All participants were
interviewed four times between 1997 and 2000,
coinciding with the period covered by their
Charities Board grant.
The benefits of the approach
This approach proved a most effective means of tapping into the experiences and perspectives of small
charities which were awarded a National Lottery
grant to fund a specific project. It allowed participants to reflect, learn, and discuss, and to allow their
views to mature as the research study progressed.
It provided rich findings. It delivered evidence
that showed how grant recipients coped with the
responsibility of running a project throughout its
life, how they developed and adapted to circumstances, and how they viewed and responded to the
Charities Board – the overseer of the grant and the
client – throughout the term of the grant.
The research helped the Charities Board
focus its thinking and change procedures to
address some of the practical issues that the
study raised.
Source: Adapted from Hall, K. and Browning, S. (2001) ‘Quality
time: cohort and observation combined – a charity case’, MRS
Conference, www.mrs.org.uk. Used with permission.
Experimental research design
You can get evidence for ‘why’ questions from cross-sectional and longitudinal
designs. Depending on the questions you ask or the observations you make, you
may be able to work out the relationship between one variable and another, or one
variable and two or three other variables. However, the complexity of the environment may be such that this may prove difficult, if not impossible – you are unlikely
to be able to prove cause.
The purpose of an experimental research design is to allow you to examine in isolation the effect of one variable (the independent or explanatory variable) on another
(the dependent variable). The idea is that the effects of all other variables will be
removed or controlled in order to see clearly the effect of this one variable. The main
application of experimental research designs is to determine if a causal relationship
exists and the nature of the relationship, to rule out the effects of other variables and
to establish the time order or sequence of events (which is the cause and which the
effect). It is the most effective research design in determining causal connections. It is
used widely in medical and pharmaceutical research, in psychological research studies and in online and marketing experiments, for example, to make decisions about
elements of the marketing mix, to evaluate the effectiveness of one pack design over
another or the effectiveness of advertisement A or B, or website version A or B.
Experimental design works like this. Two identical samples or groups are recruited:
one is known as the test group, the other is the control group. The test and control
groups are matched on key criteria – in other words the two are the same on all key
characteristics. The independent variable – the one that is thought to cause or explain
the change – is manipulated to see the effect that this change has on the dependent
variable. This is referred to as the treatment. The treatment is applied to the test
group but not to the control group. The purpose of the test group is to observe the
effect of the treatment; the purpose of the control group is to act as a comparison.
Since the treatment is not applied to the control group any changes that take place
will not be due to the independent variable but to some other factor(s). The design of
the experiment should be such that the effect of other factors is limited or controlled.
116
M05 The Practice of Market Research 31362.indd 116
27/09/2021 21:45
Types of research design
The groups are monitored and relevant measures taken at specific times to determine
if there is any difference in response between the test and the control group. Comparison of the test and control group allows us to determine the extent of the change
that is due to the independent variable only. This type of experimental design is
called the ‘after with a control group’. There are variations to this design: when the
independent variable and the dependent variable are measured in both groups before
the treatment takes place the design is called a ‘before and after’; if a control group
is used it is called, not surprisingly, a ‘before and after with a control’.
The purpose of the before measurement is to ensure that both the test and control
groups are similar on the key measures. These before measurements, however, do not
need to be taken if we are satisfied that the test and control group samples are the
same on all measures (if, for example, each was chosen using random sampling). The
post-treatment differences between the test and control groups should be sufficient
to determine the change due to the action of the independent variable. We can take
several post-treatment measurements, depending on the objectives of the research –
for example, some effects may take longer to manifest or we may want to observe
the longer-term impact of the independent variable.
The experimental designs described above deal only with the effect of one variable. This can be impractical (and expensive) if we want to look at several variables
and inappropriate if we need to determine how sets of variables might interact or
work together. To look at the effect of more than one variable at a time factorial
design is required. This type of design allows us to examine the main effects of two
or more independent variables and to look at the interaction between the variables
(for example, price and pack size on sales).
The clinical terminology used in experimental design reflects its origin in the
laboratory-based sciences. Experiments can, however, be carried out in the field – A/B
testing, in which two versions of something are tested to determine which performs
better, is one of the most common designs in the online world, for example, where
it is used to optimise digital content – to research ads, emails, apps and web pages.
You have probably been part of many such experiments without knowing it. Here
is a very basic example to show how it works. You decide what it is about your
website homepage that you want to test, say it’s the colour of the banner. Does a red
banner result in more people clicking through than a blue banner? You design two
versions, the only difference between the two being the colour of the banner. When
people visit your website, you assign them at random to one of the two versions.
You let the experiment run until each version is seen by enough people (that is, a big
enough sample) to allow you to make comparisons that are statistically significant.
So an A/B test is a randomised experiment in which two options, A and B, are tested
to identify the best one. A more sophisticated approach is to use a reinforcement
learning algorithm such as upper confidence bound (UCB) and Thomson sampling
(Eremenko, 2018). However, this is beyond the scope of the book.
As you will see from Industry Insight 5.4, simple experiments can be designed to
examine more than two options, A or B.
Experimental designs are difficult (and expensive) to use in full in the real world –
it is not always possible to isolate or account for the complexity of variables. In the
online environment it is possible to do more complex tests. For example, rather than
doing fairly simple simultaneous independent tests on, say, aspects of your website
design (banner colour, size, type of font and so on), you can design multivariate tests
involving a range of combinations. Care must be taken in interpreting the results,
117
M05 The Practice of Market Research 31362.indd 117
27/09/2021 21:45
Chapter 5
Research design
Industry Insight 5.4
Experimenting with incentives
One of the objectives of this experiment was to
examine the relative effectiveness of prepaid cash
incentives, a prize draw for cash and a prize draw
for an equivalent value non-cash prize, as methods
of increasing mail survey response rates.
The sample was made up of 900 New Zealand
residents randomly selected from the 57 electoral rolls representing the main urban centres.
Approximately equal numbers of respondents
from each socio-economic level were assigned
to each of nine groups – one control group and
eight treatment groups; each group contained 100
respondents:
1 Control – no incentive
2 20 cent coin in first mail-out only
3 50 cent coin in first mail-out only
4 $1 note in first mail-out only
5 20 cent coin in second mail-out only
6 50 cent coin in second mail-out only
7 $1 note in second mail-out only
8 Prize draw for $200 cash offered in each mail-out
9 Prize draw for $200 gift voucher offered in
each mail-out.
All respondents were sent the same questionnaire, a covering letter and a reply-paid return
envelope. The letters to the different test groups
varied only in the wording of a single sentence
that drew attention to the incentive. The response
rates were monitored. The results provide qualified support for the claim that monetary incentives are an effective means of increasing response
rates in mail surveys; 50 cents sent with the first
mail-out was very effective in this regard. However, after three mail-outs, this was the only incentive that produced a statistically significant result
when compared with the control group, indicating
that some monetary incentives are not necessarily
any more effective than two reminder mail-outs.
Source: Adapted from Brennan, M., Hoek, J. and Astridge, C.
(1991) ‘The effects of monetary incentives on the response rate
and cost-effectiveness of a mail survey’, International Journal of
Market Research, 33, 3, pp. 229–41. Used with permission.
especially if the experiment has been applied to real-world marketing and social
issues. It is always possible that other uncontrolled external factors may be exerting
an influence. Can you rule out the effect of these variables?
It is useful to be sceptical about the extent to which a causal relationship is proven.
Even with a control group external factors (known and unknown) may influence one
group disproportionately. It is also important to think about the external validity of
the results. The very fact of being studied makes people act differently (a phenomenon known as the Hawthorne Effect, after Elton Mayo’s research into behaviour
at work at the Hawthorne Plant of the Western Electric Company, Cicero, Illinois,
USA, between 1927 and 1932). With online experiments this is often not an issue
as many are done without people knowing they are taking part in a test. It is worth
thinking nevertheless about how artificial the experiment was, and whether you can
generalise from the findings to the wider population.
As with panel design, in a before and after experimental design you need to go
back to the same people for an ‘after’ measure. You might find that some drop out. It
is important to bear in mind the effect this change in the sample will have on pre- and
post- and test and control comparisons. Some problems may be overcome using statistical manipulation of the data. Conditioning is also an issue in experimental design –
participants can become sensitised to the research topic, and they may remember
the answers they gave in the pre-stage and offer their post-answers accordingly.
Timing of the post-stage measure is critical so that you do not miss the effects of the
118
M05 The Practice of Market Research 31362.indd 118
27/09/2021 21:45
Types of research design
test variables (by collecting the data too early or too late); it is possible to under- or
overestimate the length of the effect. Also, you have to bear in mind that the longer
the time lag between the tests the more likely it is that participants will drop out.
Case study research design
A case study is an in-depth investigation of a ‘case’. It is a useful design for examining complex issues within a particular context or setting (Baxter and Jack, 2008).
Yin (2003) describes three types of case study design that relate to the nature of the
enquiry: exploratory; descriptive; and explanatory. A design might be made up of several case studies, not just one – a multiple-case design (Yin, 2003). (It is an approach
associated with the grounded theory of Glaser and Strauss, 1967.) A ‘case’ might be,
for example, a household, an organisation, a situation, an event, or an individual’s
experience. Miles and Huberman (1994) define a case as ‘a phenomenon of some
sort occurring in a bounded context’. Case study research may involve examining all
aspects of a case – the case as a whole, its setting and its constituent parts. For example, imagine you are interested in how recycling operates in households. The phenomenon is recycling and the bounded context is a household. The case study research
might involve, say, observation and interviews with members of the household.
In designing a case study you need to think through (and present) the following: a
framework for the case study that includes all the aspects that you need to examine
(and how they relate to one another), including what is the phenomenon and what
is the context; the research objectives – the client’s information needs, the research
questions; your sampling strategy; your methods of data collection; and what it is
you are going to analyse. A variety of methods of data collection can be used in a
case study, including analysis of documents, observation, qualitative interviewing
(and ethnography) and quantitative interviewing.
Box 5.3
Examples: uses of case study designs
Drug treatment centre
Aim: To understand the reason for the level of success of a particular drug treatment
centre and whether lessons in relation to policy and practice could be transferred to
other centres.
The case study included a review of the brochures and other documentation drawn up
by the centre; a review of policies, procedures and guidelines; and qualitative in-depth
interviews with management, staff, volunteers, and clients and their families.
A university programme
Aim: To evaluate the effectiveness of a pre-admission university programme.
The case study included a review of the documents related to the programme; a
review of the recruitment practice of the programme; qualitative in-depth interviews
with experts involved in the policy-making decisions, programme staff, students on the
programme; and a self-completion survey among students on the programme.
119
M05 The Practice of Market Research 31362.indd 119
27/09/2021 21:45
Chapter 5
Research design
Table 5.1 Summary of key features of research designs
Feature
Single
cross-sectional
Longitudinal
Experiment
Case study
Suitable for exploratory research
+
+
-
+
Suitable for descriptive research
+
+
-
+
Suitable for causal research
+
+
+
+
Exploring relationships between variables
+
+
+
+
Establishing time sequence
-
+
+
+
Establishing association, covariance and
correlation
+
+
+
-
Ruling out other variables/explanations
–
-
+
-
Understanding why one thing causes or affects
another
+
+
-
+
Ability to deal with complexity
-
+
-
+
Making comparisons between groups
+
+
-
+
Ability to detect change
-
+
+
-
Representativeness
+
-
-
-
Ability to look at data at the level of the
individual
-
+
-
+
Relative cost
+
-
-
-
Ease of set-up
+
-
-
+
Ease of management
+
-
-
+
Burden on participants
+
-
-
-
+ indicates a relative strength; – indicates a relative weakness.
Uses
The main application of a case study design is to get the full picture, to achieve an
in-depth understanding and to get a detailed (idiographic) description. It is also useful
in understanding the context of attitudes and behaviour in order to reach a greater
understanding of their meaning. It can be used to establish a sequence of events; to
examine relationships between variables; and to understand which explanation best
fits a hypothesis or theory. In other words, the case study design can used in exploratory, descriptive and explanatory enquiries. Case studies are common in health, educational and organisational research and in evaluation research.
If the findings from a particular case study are to be used to make generalisations
about the wider group or population to which the case belongs, some care must be
taken in ensuring that the particular case is representative of the wider population of
cases. In some instances generalisation may not be the aim of the research – the aim
may be to understand fully the particular case.
Table 5.1 above summarises the key features of each research design.
120
M05 The Practice of Market Research 31362.indd 120
27/09/2021 21:45
Exercises
Chapter summary
●
●
●
●
●
●
●
●
Research design is about deciding on the structure the research will take in order
to deliver the evidence needed to address the research problem clearly and
unequivocally.
There are two levels of research design. The first level involves getting to grips
with the business problem, the action to be taken and the research problem, and
clarifying the nature of the evidence needed to address it; it also involves deciding
on the structure of the research that will deliver the evidence. The second level
involves decisions about how to collect the evidence.
There are four main types of research design: cross-sectional, longitudinal,
experimental and case study.
Cross-sectional research design is probably the most common type of design in
market and social research. In a single cross-sectional design data are collected
once only from a cross-section of a population at one point in time; a repeated
cross-sectional design involves conducting more than one wave of (more or less)
the same research with an independent or fresh sample each time. The use of
an independent sample at each round of data collection is what distinguishes
repeated cross-sectional design from longitudinal research.
In longitudinal research, data are collected from the same sample on more than
one occasion.
The purpose of an experimental design is to examine in isolation the effect of
one variable (the independent or explanatory variable) on another (the dependent
variable). The effects of all other variables are removed or controlled in order to
see clearly the effect of this one variable. The main application of experimental
research design is to determine if a causal relationship exists.
A case study is an in-depth investigation of a case (or cases) – for example
a household or an organisation – for exploratory, descriptive or explanatory
research purposes, or a combination of these.
A research design can use any method of data collection.
Exercises
1 Review Industry Insights 5.1–5.4. For each, describe the research design. Give
reasons why you think its use is justified.
121
M05 The Practice of Market Research 31362.indd 121
27/09/2021 21:45
Chapter 5
Research design
References
Baxter, P. and Jack, S. (2008) ‘Qualitative case study methodology: study design and implementation for novice researchers’, The Qualitative Report, 13, 4, pp. 544–59.
Brennan, M., Hoek, J. and Astridge, C. (1991) ‘The effects of monetary incentives on the
response rate and cost-effectiveness of a mail survey’, Journal of the Market Research
Society, 33, 3, pp. 229–41.
Eremenko, K. (2018) Confident Data Skills, London: Kogan Page.
MRS (2019) Code of Conduct, London: MRS.
Gibson, S., Teanby, D. and Donaldson, S. (2004) ‘Bridging the gap between dreams and reality . . . building holistic insights from an integrated consumer understanding’, Proceedings
of the MRS Conference.
Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory, Chicago, IL: Aldine.
Hall, K. and Browning, S. (2001) ‘Quality time – cohort and observation combined: a charity
case’, Proceedings of the Market Research Society Conference, London: MRS, pp. 65–73.
Miles, M. and Huberman, A. (1994) Qualitative Data Analysis: An Expanded Sourcebook,
2nd edition, London: Sage.
ONS (2020) ‘COVID-19 Infection Survey (Pilot): Methods and further information’, https://www.
ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/
methodologies/covid19infectionsurveypilotmethodsandfurtherinformation (Accessed 21
September 2020).
Pouwels, K., House, T., Robotham, J., Birrell, P., Gelman, A., Bowers, N., Boreham, I.,
Thomas, H., Lewis, J., Bell, I., Bell, J., Newton, J., Farrar, J., Diamond, I., Benton, P. and
Walker, S. (2020) ‘Community prevalence of SARS-CoV-2 in England: Results from the
ONS Coronavirus Infection Survey Pilot’, Coronavirus Infection Survey Pilot, https://www.
medrxiv.org/content/10.1101/2020.07.06.20147348v1 (Accessed 21 September 2020).
Yin, R. (2003) Case Study Research: Design and Methods, 3rd edition, London: Sage.
Recommended reading
De Vaus, D. (2006) Research Design in Social Research, London: Sage.
Kohavi, R., Tang, D. and Xu, Y. (2020) Trustworthy Online Controlled Experiments:
A Practical Guide to A/B Testing, Cambridge: CUP.
Mason, J. (2017) Qualitative Researching, 3rd edition, London: Sage.
Strang, K. (2015) The Palgrave Handbook of Research Design in Business and Management,
Basingstoke: Palgrave.
Yin, R. (2018) Case Study Research and Applications, 6th edition, London: Sage.
122
M05 The Practice of Market Research 31362.indd 122
27/09/2021 21:45
M05 The Practice of Market Research 31362.indd 123
27/09/2021 21:45
Chapter 6
Writing a research brief
Introduction
In Chapter 4 we looked at understanding the business problem and defining
the research problem. In this chapter we look at what is involved in describing
the business problem and the research problem in the form of a research brief.
The brief is one of the most important documents in the research process. It is
largely on the basis of the brief that the researcher designs the research that
aims to deliver the insight the client needs. We also look at the links between the
brief and the proposal. We look at how to choose a research supplier to respond
to your brief. Finally, we look at the client–researcher relationship.
Topics covered
Definition of a research brief
● Roles in the briefing process
● Links between the brief and the proposal
● Preparing a written research brief
● Choosing a research supplier
● The client–researcher relationship.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project.
M06 The Practice of Market Research 31362.indd 124
27/09/2021 21:45
What you should get from this chapter
At the end of this chapter you should be able to:
plan and prepare a research brief;
● understand the links between a brief and proposal;
● evaluate a research brief;
● understand the roles in the briefing process;
● be aware of how to choose a research supplier.
●
125
M06 The Practice of Market Research 31362.indd 125
27/09/2021 21:45
Chapter 6
Writing a research brief
Definition of a research brief
A research brief is a statement that sets out what research a client, a researcher buyer,
needs. It should contain information to place the need for research into the wider
context of the client’s organisation and the market or environment it operates in and
the business problem to which the brief relates. It should also be clear about what the
objectives of the research are and the end use to which the research findings will be
put. It should contain enough information to allow a researcher to design and plan
a project that will address the research objectives. It should include what resources –
time, money – are available to do the work.
Roles in the briefing process
Typically, the role of the person commissioning research – the researcher or decision
maker within an organisation (that is, the client) – is to set out the problem they face
for which they need research. They should identify what information is needed to
address the problem and set out how this information will be used and by whom.
Having done all that, they must communicate it clearly in a research brief.
The role of the research supplier (the researcher) is to design research – on the
basis of the research brief – that will deliver the information, the insight, needed by
the client. The researcher presents their programme or plan in a research proposal.
(There are a few situations in which the same person may be responsible for writing
both the brief and the proposal, for example in applying for research funding in an
academic setting, or in preparing a dissertation or an assignment.)
Depending on the client’s background and experience, the way in which the client
organisation works, or the nature of the relationship between client and researcher,
the client may involve the researcher in the process before the final brief is agreed.
This might happen at the problem definition or information needs assessment stage.
The client can make use of the researcher’s experience and knowledge to help the
client define or refine the problem, and reach a clear understanding of their information needs. The more common situation though is that the client prepares the brief
in-house and sends it out – often to several research suppliers – with a view to getting
an idea of who is best suited to doing the work and who offers best value for money.
Links between the brief and the proposal
The aim of a research brief is to elicit a good-quality research proposal, one that
describes an effective, appropriate research plan. To prepare a good-quality proposal,
the researcher must have a clear understanding of the client’s business problem and
its wider context, the type of information needed to address the problem, and how
this information will be used. Thus the quality of the proposal and the effectiveness
and suitability of the research proposed in it will depend on the quality of the brief.
The brief will also inform the nature of the working relationship between the client
and the researcher.
126
M06 The Practice of Market Research 31362.indd 126
27/09/2021 21:45
Preparing a written research brief
Preparing a written research brief
Verbal research briefings are common but are usually accompanied by or follow the
sending of a written brief. Preparing a written brief is good practice for three main
reasons. Having to commit ideas to paper usually enhances the quality and clarity
of the thinking behind them. A written brief is a valuable aid to communication and
acts as a record for consultation and discussion. It is a document that captures (or
should capture) what is agreed by all stakeholders. In some places it might be called
an ‘engagement letter’ (Eremenko, 2018).
Take time with the project team and other key stakeholders in preparing the brief.
You must reach a clear understanding of the business problem and the wider context in which it sits, and the information needs. Don’t be afraid to ask questions if
anything is not clear or to challenge any assumptions you think are being made. It is
far better that you interrogate things fully at this stage rather than later, when it may
well be too late to make any changes. We looked in some detail at understanding the
business problem and defining the research problem in Chapter 4. Here’s a summary
of what you need to cover in the context of preparing the research brief:
The business problem that you, the client, need to address
The decision to be made and/or the action to be taken
● The importance of the decision or action to the organisation
● Who is to make the decision or take the action
● The information you need to help those people make the decision or take the
appropriate action
● An understanding of whether research evidence will help
● The specific objectives which the research must address
● The amount of time and money available for the work.
●
●
Once you have discussed and interrogated the problem and are happy that you
understand it and understand what needs to be done, you can begin drafting the brief.
Once you start writing it should be apparent whether you have understood both the
business problem and the research problem – it is difficult to write about something
that you do not fully understand or about which you are unclear. Don’t worry if
the drafting process uncovers further questions. You can highlight these in the draft.
This first draft should be circulated to the project team – it can be a useful stimulus
to further thinking and can be used as a focus for further discussion. Several versions
may be prepared before agreement is reached. Once you have the final, agreed version
all involved have a record of what is being sought. This can reduce the chance of a
dispute arising later about what is delivered.
Contents of a research brief
Below is a detailed list of the headings that you might find in a brief:
Title
Discussion and definition of the business problem
● Background to the problem
● Why research is necessary
●
●
127
M06 The Practice of Market Research 31362.indd 127
27/09/2021 21:45
Chapter 6
Writing a research brief
Statement of research objectives
End use of information
● Target population
● Suggested research design and/or approach
● Analysis required
● Outputs
● Liaison arrangements
● Timings
● Budget
● Form of proposal
● Selection criteria.
●
●
Not all briefs contain or need to contain all of this information. In cases where the
research is a repeat of a similar job, or where there is an established relationship
between client and researcher, some of the information may not be included. A more
comprehensive brief is, however, recommended in cases where either party is fairly
new to research, where the relationship between client and researcher is new, or
where the project is unusual or complex. We look at each of these headings in more
detail below.
Title
A title is important – it informs the reader immediately of the main focus of the project and draws attention to the key issue. A title may be obvious immediately or it
may not be obvious until you have thought through exactly what it is you want – so
it may be the last thing you decide upon.
Discussion and definition of the business problem
Get to the heart of the issue immediately with a clear, accurate and precise definition of the business problem. This may be harder to write than you imagine. If you
do have trouble writing it down it may be that you are not clear about it yourself,
and if you are not clear about it, it is unlikely that the researcher will be. Use clear,
jargon-free language. Avoid ambiguous words and phrases. Be as specific and precise
as possible.
Background to the problem
Give some background information about the product, service or issue to which
the problem relates, and its wider setting within the organisation or within the
market or subject area. It might also be useful to provide some information on the
external conditions within which the organisation operates. In addition, especially
if it is the first time that the researcher has been asked to prepare a proposal for
your organisation, you might include some background information about the
organisation’s role, its aims, its responsibilities, its mission statement or business
strategy – something to give a flavour of the organisation’s work. This will not
only help the researcher formulate the most effective research design but will be
useful for interpreting the research findings and understanding the implications
for the organisation.
128
M06 The Practice of Market Research 31362.indd 128
27/09/2021 21:45
Preparing a written research brief
Why research is necessary
State why you think research is necessary and, briefly, how you came to this conclusion. For example: ‘Although existing data tell us that there has been a sharp decline
in sales, we have no evidence as to why.’
Statement of research objectives
State the research objectives – what it is you want the research to tell you. Be as specific and unambiguous as possible. Avoid vague statements such as: ‘To research the
market for electric cars’ or even ‘To conduct a study of the image of the car dealership
among its target audience’.
End use of the information
To ensure that the research is focused and to help the researcher determine the type
and scope of information or evidence needed, and the robustness of it, state what
the information will be used for, who will use it and how it will be used. For each
research objective ask yourself, ‘How will the information I get here be used in the
decision-making process?’ This is a good exercise to ensure you are asking for the
information that you really need.
Target population
Give as much detail as you can about the target audience or the target population.
Specify what it is you want the unit of analysis to be. This information will help the
researcher decide not only on the sampling approach but on the type of research and
the method of data collection. It will also help to cost the project more accurately.
Be as specific and as precise as possible. For example, if you have information on
the incidence of the target market in the wider population, include it. If you have
specific requirements, if there are specific groups within the population that you want
to compare in the analysis – for example, if you want to compare 25–34-year-old
users and 35–44-year-old users; or those in employment and those not; or frequent
users, occasional users and non-users; or those with children and those without, state
this in the brief. This information will guide the researcher in designing the sample,
in determining the number of focus groups or the number of interviews necessary
for these comparisons to be made. In addition, it is important to clarify what you
mean by terms such as ‘frequent’, or ‘in employment’. Does employment mean paid
employment only, for example, or would you include those in voluntary work or
on home duties? Does it include those working part time as well as those working
full time? Does it include those on paternity or maternity leave? Be as specific and
unambiguous as possible.
Suggested research design and/or approach
The amount of detail you give here may depend on your knowledge of research,
or on whether you prefer the research supplier to put forward ideas that are not
influenced by your own. Tell the researcher if the decision makers have a preference
129
M06 The Practice of Market Research 31362.indd 129
27/09/2021 21:45
Chapter 6
Writing a research brief
for a particular type of research or research evidence. If the research needs to be
comparable with a previous piece of research, mention this and give details. If you
want the researcher to suggest a range of possible options and the pros and cons of
each, say so.
Industry Insight 6.1 gives an example of the sort of information you might see in a
brief. In Chapter 7 we look at the research that was conducted to address the issues
set out here.
Industry Insight 6.1
Establishing the facts – the brief
Introduction
Macmillan Cancer Support is a charity. It campaigns to raise awareness of the needs of those
affected by cancer, setting the national agenda
and ensuring funding is spent in the right places.
It has a proud history of providing care to people living with cancer and their families: being
there to provide practical, emotional, financial
and medical help and support to everyone, when
they need it the most. This includes social care,
non-medical support that can be instrumental in
good outcomes. Many healthcare professionals
agree that a lack of social care support can have
a negative impact on the outcomes of people with
cancer. Imagine you live alone and you’re having
chemotherapy. You need nutritious meals to keep
up your strength but there’s nobody to cook for
you. This could have a significant impact on your
physical and emotional well-being. As the population ages, the needs of those living with cancer are
becoming more complex as people have cancer
and existing, age-related conditions, or other complex needs, or their primary carers do.
Knowledge gap
We do not know the full extent of social care
needs among those living with cancer. We do not
have evidence – other than anecdotal evidence –
about how well people are being supported, who
is supporting them and how and what the consequences are when the support isn’t there. Health
and social care organisations including Macmillan have wrestled with this gap. It is a gap that
is becoming more problematic with the focus on
efficiency and increasing integration of NHS and
social care budgets. There is limited official data.
Information needs
Macmillan want to explore and measure the range
of multi-dimensional, subjective, complicated and
deeply personal social care needs among those
with cancer. These include needs ranging from
help taking medication to at-home support after
hospital discharge. What is the extent of those
needs? How are they currently being met? Formally, for example by local authorities or the
NHS? Informally by friends and family? We also
want to map people’s level of need against criteria that local authorities in England use to assess
eligibility for support to see how self-identified
needs are captured in real-world assessment
structures.
Target audience
The target audience is people currently in treatment for cancer; people who have successfully
completed treatment; people with advanced cancer; and those with a terminal diagnosis.
End use of the findings
The study will inform the Macmillan research
team and its policy, campaigning and service delivery workstreams. It is crucial that the research
provides insights that are robust and also communicate the ‘human’ dimension.
Source: Adapted from Bright Blue and Macmillan, ‘Hidden at
home: Establishing the hard facts to set the social care agenda’,
MRS Awards 2015.
130
M06 The Practice of Market Research 31362.indd 130
27/09/2021 21:45
Preparing a written research brief
Analysis required
Set out clearly what type of analysis you need and an idea of the complexity of
the analysis required. In a quantitative study you are likely to want a set of crosstabulations (data tables). Think about what headings or variables you want to include
in these cross-tabulations. Also, think about what sort of statistics you will need –
descriptive statistics (e.g. means, standard deviations, standard error) and/or inferential statistics (significance tests). Will you want to run other more complex analyses
such as factor analysis, cluster analysis or conjoint analysis? The researcher needs
this sort of information in order to make decisions about research design, design of
the sample, sample size, type and level of resources to be assigned to the project, time
needed to complete it and so on.
Outputs
Data tables, summary reports, full reports, dashboards, visualisations, videos and
presentations of findings are often referred to as ‘deliverables’ or outputs – the products of the research. Specify exactly what deliverables you expect during and on
completion of the research. Typically they will consist of a presentation of the findings and either a written summary report or a full report, handed over at the end of
the project. For some projects – especially large-scale ones – you may want interim
reports of the findings or ‘read outs’. You may want to comment on a draft report
before the final report is produced. In a qualitative project you may want a summary of the findings from each group or a film of the ethnography. Whatever your
requirements, mention them in the research brief so that the researcher can plan for
them and cost them.
Liaison arrangements
Set out the contact or liaison arrangements you want. For example, if you have
a project team or advisory group with which the researcher must meet to discuss
progress, give details – frequency of meeting, type and detail of reporting needed – so
that they can build this into the work plan and the costing.
Timings
Give the date by which you need the research to be completed and highlight any
interim deadlines (for completion of fieldwork, say, before a product or service is
launched). This information will not only allow the researcher to plan the work but
it will also affect what sort of research can be done. For example, the time frame may
put constraints on the number of interviews, or the method of data collection. Make
sure the time frame is reasonable (this is not always possible) and make sure you can
meet any obligations you might have – such as to approve the data collection tool,
attend fieldwork, provide product or stimulus material.
Budget
Some research briefs can be so prescriptive that it is not necessary to provide details
of the budget – what is prescribed contains enough information for the researcher to
131
M06 The Practice of Market Research 31362.indd 131
27/09/2021 21:45
Chapter 6
Writing a research brief
cost the work. Even in cases where the client does not specify design and method (the
more common situation) the budget may not be stated. The reason often given is that
the researcher will design the research to use up this budget, whether the problem
calls for it or not. This of course would not be ethical on the part of the researcher. If
you have asked for more than one proposal the absence of a budget can make it more
difficult for you to compare them. Different researchers will interpret a brief in different ways, making different assumptions that will impact on the cost. It is therefore
worthwhile to give at least some idea of the budget, a range or an upper limit, so that
the researcher can avoid proposing research that does not meet it and is better placed
to design research that will maximise value for money. In setting a budget it is useful
to think of the research not as a cost to the organisation but as an investment. There
is likely to be long-term value in the insight the research will deliver; set the budget
accordingly. To paraphrase Professor Sir Claus Moser, research ‘costs money, but
then so does ignorance’.
Form of proposal
Specify clearly the way in which you want the supplier to present the proposal. For
example, you might specify the headings under which the proposal should be written,
the order of the headings, the nature and detail required, even the appearance of the
document, method of delivery (on paper and/or electronically) and the number of
copies to be submitted. This is useful if you need to compare proposals from several
suppliers. Here is an alternative set of headings:
Understanding of the business problem and the client’s requirements
Details of the design and the approach
● Any difficulties that might be anticipated and how these might be overcome
● Timetable
● Separate costing for all options proposed
● Pricing schedule outlining staff inputs and daily rates
● Details of relevant experience of the organisation and proposed project staff.
●
●
Selection criteria
In some research buying contexts, for example in the tendering process for government contracts, the researcher may be told on what basis the research contract will
be awarded – in other words, on what basis the proposals will be evaluated. The
selection criteria might include the following:
suitability of proposed methodology;
relevant experience in this area;
● cost;
● demonstration of understanding of the brief.
●
●
Each proposal is rated on the extent to which it meets these criteria. A weighting or
score may be given to each of them – for example, demonstration of understanding of the brief may be judged to be the most important and cost the next most
important.
132
M06 The Practice of Market Research 31362.indd 132
27/09/2021 21:45
Choosing a research supplier
Choosing a research supplier
Once you have written a brief, you must decide who is to carry out the research.
It may be that you can handle it internally. If you do not have the resources or the
expertise to do so you must select an external supplier.
To choose a supplier, think, first of all, about the type of project it is and the type
of supplier you might need. You may have several options, ranging from a full service
general agency or a specialist agency, to a supplier of fieldwork and tabulation, to
a consultant to write up the findings from data you have collected. You can obtain
information on agencies and consultants from the directories of organisations such
as MRS and the Research Buyer’s Guide (www.theresearchbuyersguide.com/) and
its Top Ten Tips for Buying Research and Insight available at https://www.mrs.org.
uk/ by using search option. Also useful are The GreenBook (www.greenbook.org/),
the directory of the Association for Qualitative Research (www.aqr.org.uk) and the
Social Research Association (www.the-sra.org.uk). MrWeb is a web-based service
that lists suppliers including independent research consultants (www.mrweb.com).
It will also be helpful in choosing a supplier if you are able to keep up to date
with developments in market and social research through the MRS website (www.
mrs.org.uk), the ESOMAR website (www.esomar.org), and research-live.com. It is
a fast-changing sector, and the best solution for your research problem might be a
new approach or technique from a new supplier.
From the suppliers you have identified you may want to draw up a shortlist. The
shortlist can be selected against a number of criteria including the following:
accreditation – does the supplier have a formal measure of quality? For example,
is it an MRS Accredited Company Partner or a Fair Data supplier or does it have
an ISO accreditation? Are its employees members of MRS or ESOMAR or the
Association for Qualitative Research (AQR) or the Social Research Association
(SRA)?
● experience in the general subject area – for example consumer, social or businessto-business issues;
● experience in the particular area – for example pharmaceutical products, older
people’s issues, office equipment; or advertising research, new product development, employee research;
● services available – for example full service or limited service; ethnographic
research, data analytics, or social media analysis;
● expertise in particular research methods or techniques – for example qualitative, quantitative; omnibus, continuous research, mystery shopping, social media
research, analytics.
●
You can determine whether researchers meet your criteria by examining their entries
in directories or their advertising, by reading articles they have published about
their work, by talking to those who have used their services and by talking to them
directly. To reduce the list of those to whom you might send out a research brief you
can invite prospective candidates to make a ‘credentials’ pitch to you – a presentation
outlining their experience and expertise. It is important to remember that people and
the quality of relationships between them are important in the research process and
to take this into account in appointing a supplier.
133
M06 The Practice of Market Research 31362.indd 133
27/09/2021 21:45
Chapter 6
Writing a research brief
Once you have established a shortlist in this way, you can send out the brief. It
is preferable not to ask more than three or four research suppliers to tender for a
project. Proposals take time and money to prepare but are provided free of charge
to clients requesting them, on the understanding that the researcher has a reasonable
chance of winning the job. It is judged unfair to ask suppliers to tender for projects
for which they have less than a one in four chance of success. This guideline aims
to protect research suppliers from being used by clients as a source of free research
advice. The cost of preparing proposals is of course built into the researcher’s overheads and so ultimately affects the cost of research. The more proposals requested,
the more research costs will rise in general. If more than four suppliers are involved
in a pitch, individual suppliers may decline to tender, or may ask for a fee for preparing the proposal. In addition to the cost of preparing the proposal, while you are
considering the proposal, the researcher may turn down other work in the reasonable
expectation of being awarded your project. The opportunity cost therefore to the
researcher is greater if they have little chance of being awarded your project.
The brief usually sets out the deadline for submitting a proposal and offers client
contact details, should the research supplier want to discuss the brief and its contents
further. It is good practice to set up a face-to-face meeting with the researchers to
whom the brief was sent. It gives the researchers a chance to ask further questions
about key elements – the research problem, the business problem and its wider context, the end use of the research findings – and to test the reaction of the client to
some early ideas about the shape or nature of the research. It gives the client a chance
to see whether the researcher understands the issues involved. Thus all parties have
the opportunity to assess if a working relationship is possible or desirable.
A research proposal represents the intellectual property of the researcher and/or
the research agency that produced it. In addressing your research problem it represents the accumulated knowledge and experience of skilled research practitioners. It
is unethical therefore to request a proposal from a practitioner if you are not seriously considering that practitioner as a supplier of the research. It is also unethical to
use the work set out in a proposal without permission, as the following clause from
the MRS Code of Conduct (2019) in the section on General Rules of Professional
Conduct states:
10. Members must not knowingly take advantage, without permission, of
the unpublished work of other practitioners which is the property of those other
practitioners.
Comment: This means, where applicable, that Members must not knowingly carry out
or commission work based on proposals prepared by practitioners in another organisation unless permission has been obtained.
The brief can be used at the end of the research project to review or evaluate
the research, matching what was asked for and what was delivered. However, it is
worth noting that there may remain some ‘unknown unknowns’ in the market or
the environment which you may not be able to foresee when writing the brief. We
live in a fast-moving world and the brief (and the proposal) may capture the business
problem (and the research objectives that address it) as it is at the time of writing.
There may therefore be some uncertainty as things progress. This can be managed to
some extent by meeting to discuss the proposal once it has been submitted; reviewing
the research plan once you give the go-ahead for work to begin; and holding regular
progress meetings.
134
M06 The Practice of Market Research 31362.indd 134
27/09/2021 21:45
The client–researcher relationship
Checking the brief
When the research supplier receives your brief, they will scrutinise it. Listed below are
the sort of things they will examine. It is useful to check how clear these things are
before you send out the brief so that you have the chance to improve or correct them
if necessary. Doing this may increase the chance of the research supplier responding to you. If the supplier, for instance, judges your brief to be poorly thought out,
or sloppy in any way, they may decide it is not worth investing time in preparing a
proposal. Also, checking these things before the brief goes out may also mean that
you save time later answering questions from the supplier.
Here are the sort of things you should check:
Is the business problem clearly defined? What assumptions, if any, have I made?
Why is the research needed?
● Is it clear what the information needs are?
● Will research help?
● Have I set out all the information the researcher will need to design effective
research?
● Are there any gaps in my knowledge about the problem?
● Are there any gaps in my knowledge about what the research is required to
provide?
● If a research approach is suggested, is it feasible? Will it deliver what is needed?
● Are the research objectives clear and unambiguous?
● Are the research objectives relevant to the problem?
● Is it clear what I expect to get from the research?
● Is it clear how the research will be used within our organisation and who will use it?
● Is the budget adequate?
● Is the time frame feasible?
●
●
In effect, what you are doing here is conducting a critical evaluation. Even in a wellprepared brief you may have made some assumptions about what is known or not
known, or you may not have fully explained some points – as a result of being too
close to the problem, for example – and so some gaps or ambiguities might remain.
Hence the importance of reviewing your brief with a critical eye. If there is anything
that is not clear, do not be afraid to go back to the owner of the problem or the decision maker or other relevant stakeholders including whoever is to be the end user of
the research for clarification. The better you understand the business problem and
the research problem, the better the brief and so the better the research.
The client–researcher relationship
To maximise the value of the research you commission it is important at the outset
to establish a good working relationship with the research supplier. A good working
relationship is characterised by a rapport between the parties – a sense of being on
the same wavelength, and each understanding the role of the other (Pyke, 2000). As
the client you will have certain expectations of the researcher: at the very least, the
researcher should be competent in the design and management of a research project.
135
M06 The Practice of Market Research 31362.indd 135
27/09/2021 21:45
Chapter 6
Writing a research brief
Yet you do not want someone who is merely good at the mechanics of research, rather
you want someone who can see the ‘big picture’ and who can put the research findings into this picture. Failure to do this is a common criticism of researchers: clients
often report that researchers are too focused on the research process and the data and
not focused enough on what the data say about the problem (Bairfelt and Spurgeon,
1998). So you should expect the researcher to have a sound understanding of and
an interest in your business and the issues you face, to think about the context of the
problem, the wider issues involved and how the research can help address these. The
researcher should not ignore the question of what is to be done with the information
provided by the research. You should be clear, too, about the level of service you
expect from the researcher and the nature of the working relationship you want with
them. In Industry Insight 6.2, Andrew Pyke highlights the importance of doing this.
As the person who prepared the brief, you must be clear about why you are commissioning research. On the other side, the researcher who undertakes the research
Industry Insight 6.2
‘It’s all in the brief’ — or is it?
What should agencies expect from clients? There
are three key issues for the research world: (a) the
client dilemma – am I a researcher or a planner;
(b) the agency dilemma – are we data providers
or consultants; and (c) how do (a) and (b) impact
where it matters – the marketing decision making.
My hunch is that fundamental questions in the
ideal briefing process – (a) why is this project happening at all; (b) what’s gone before; and (c) what
will happen as a consequence of this project – are
lost in the commissioning process, e.g. haggles
over fees, mad diary scrambles and so on. I suspect that the momentum created by the confirmation of the project – right you’ve won the contract
so let’s get the project up and running in the next
five minutes – leaves these fundamental questions
(e.g. why is this project happening) uninterrogated
or assumed.
Typologies
Why the brief should be at the hub
In data supplier mode, the agency is very keen
to prove their adeptness at supplying data across
the marketing mix, all jobs considered. In business partner mode, the agency is more interested
in developing the contact with the client, supplying more than just the data. We all appear to strive
to the planners/business partners’ modes, at least
that’s what everyone says is a good thing; add
more value, do more than just research.
This is a worry for all of us. Research projects
that search for and successfully apply consumer
understanding to business are our lifeblood. These
projects rely on unity of purpose at the outset,
from the brief. Yet, this does not happen most of
the time. Often, we are involved in projects where
best practice happens to an extent but the ideal
does not happen frequently enough.
I contend that most of the problems lie with the
roles (and resultant interaction of the roles) that
both clients and agencies wish for themselves. The
client modes are classic researcher mode and planner mode. The agency modes are data supplier
mode and business partner mode.
Client modes
The classic client mode is that the client researcher
needs the data: straight answers to straight questions. In classic mode, the client researcher will just
pass the debrief (if there is one) on to the end user.
Alternatively, in the planner client mode the client
needs the agency to use the project as a springboard
to move thinking forward in their organisation.
Agency modes
136
M06 The Practice of Market Research 31362.indd 136
27/09/2021 21:45
The client–researcher relationship
How this impacts on the brief
If the client is in planning mode, using the research
project to drive change in the business, they
need to be clear about this. They need personal
skills beyond research competency, e.g. influencing skills, team facilitation, and entrepreneurial
zeal. They need the agency to be in on the plan –
­otherwise the planner’s initiative will have limited
chance of success. The brief needs to make this
clear. The context section of the brief (definition
and background to the problem, why the research
is needed, use of results) must state the intent.
Otherwise, the research will happen, change will
not and all involved will feel frustrated.
If the client is in classic mode, the research project will make a valuable contribution to the business. However, if the marketing/board end user is
using the classic data in planning mode, the client researcher/agency will suffer/be blamed as the
process unfurls, often at the debrief. The context
section needs to be interrogated for evidence of the
mix of modes. It is remarkable how often it occurs.
To contribute meaningfully to the decisionmaking process, the role of the project needs to
have been established at the outset. Moreover,
the agency needs to know the context from the
client researcher. If the client researcher is not a
reliable source, the agency needs to have interrogated the client organisation to have elicited the
context. Leaving it (literally) to the last minute is
not a good idea. The discussion should take place
at the beginning of the project – at the brief, in the
context section.
How this impacts on the marketing
decision-making process
When client and agency modes are matched, we
are a powerful force:
client classic and agency data supplier –
­delivering exactly what was required, building
a deserved reputation for efficiency and reliability. Demonstrates a clear ownership of consumer insight.
● client planner and agency business partner –
powerful agents of consumer-based process
change. Helps to create a genuine consumer
based marketing culture.
●
When modes are mis-matched for both agency
and client researchers, the dangers are either (a)
failing to meet expectations or (b) not getting the
credit where it’s due.
Moving forwards
Let’s demand that henceforth, in each brief, the
clients proactively say what mode they are in, and
why. Also, clients add in time for interrogation,
saying that the job won’t be confirmed until interrogation has occurred. Let’s make the context section mandatory.
Source: Adapted from Pyke, A. (2000) ‘It’s all in the brief’, MRS
Conference, www.mrs.org.uk. Used with permission.
must always keep in mind why the research is being commissioned. The researcher
will not be able to deliver effective research if you – as the person who prepared the
brief – have not paid attention to defining the problem, if you have not fully understood the nature of the problem, the context of it, or what the end user needs from
the research. If this is the case, there will be problems at the delivery stage. Always
think of the end result at the beginning. This can be difficult – it may not be possible
to get access to the decision maker or the information end user, or information about
them, particularly the sort of information that might help the researcher understand
the decision-making process and the culture and politics of the organisation. From the
researcher’s point of view, for the relationship with you, the client, to work best there
should be no hidden agendas; you should aim to establish an atmosphere in which the
researcher feels able to explore or question the brief and reach a full understanding of
the issues. In order to improve service delivery, the researcher should be able to ask
for (and you should provide) open and honest feedback about the service.
137
M06 The Practice of Market Research 31362.indd 137
27/09/2021 21:45
Chapter 6
Writing a research brief
Chapter summary
●
●
●
●
●
●
●
A research brief is a statement that sets out what research a client, a researcher
buyer, needs. It should contain background information on the wider context; the
client’s business problem; the information needs; the objectives of the research;
the end use of the research; and the resources available for the research. In other
words, it should contain enough information to allow a researcher to design and
plan a research project that will address the objectives.
The role of the person commissioning research is to set out the business problem
and its wider context; define the research problem clearly and precisely, identifying
what information is needed to address the business problem; set out how this
information will be used; and communicate this information clearly in a research brief.
The role of the research supplier is to design research that will deliver the
information needed by the client.
The aim of a research brief is to elicit a good-quality research proposal, one that
describes an effective, appropriate research plan.
To prepare a good-quality proposal the researcher must have a clear
understanding of the client’s business problem and its wider context, the type of
information needed to address the problem, and how this information will be used.
A research brief should contain the following information:
●
the client’s business problem and the background to and context of the problem;
●
the information needs;
●
why research is necessary;
●
end use of the information;
●
research objectives;
●
target population;
●
suggested approach;
●
analysis required;
●
deliverables, timings and budget.
To maximise the value of the research you commission, it is important at the
outset to establish a good working relationship with the research supplier.
Exercises
1 Here are the key elements of a brief from a not-for-profit education and training
centre. You read the brief and decide that you want to meet the client to discuss
it in more detail so that you might make useful suggestions about a way forward.
List the questions that you want to ask with your rationale for asking them.
138
M06 The Practice of Market Research 31362.indd 138
27/09/2021 21:45
Recommended reading
The brief:
We are a not-for-profit education and training provider specialising in the delivery
of IT programmes and courses to those aged 55 and over. We are based on the
outskirts of a city with a population of around 100,000. Demand for our provision
has remained steady over the last five years but we expect it to increase over the
next five to ten years as the population of those aged 55 and over grows. Our
building is now more than 100 years old and although it is in good condition we
are finding it difficult and costly to adapt it to suit our needs and to meet health
and safety and other regulations. We would like to build a new state-of-the-art
training suite on our current site – we have outline planning permission to do so.
We understand that government funding may be available to cover the capital
cost of building the new facility. To make a sound case for this funding we
would like to gather evidence via a programme of research. We should like you
to provide us with costs for the following: a series of focus groups with existing
users of our facilities; a survey of older people within our area; and an audit of
the employment opportunities available to older people who have completed our
training programmes. We have a budget for this research of around £12,000. The
funding application must be submitted in two months’ time. We should therefore
like you to provide a detailed workplan that fits this time frame.
References
Bairfelt, S. and Spurgeon, F. (1998) ‘Plenty of data, but are we doing enough to fill the information gap?’, Proceedings of the ESOMAR Congress, Amsterdam: ESOMAR.
Bright Blue and Macmillan (2015) ‘Hidden at home: Establishing the hard facts to set the
social care agenda’, MRS Awards.
Eremenko, K. (2018) Confident Data Skills, London, Kogan Page.
MRS (2019) Code of Conduct, London: MRS.
Pyke, A. (2000) ‘It’s all in the brief’, Proceedings of the Market Research Society Conference,
London: MRS.
Recommended reading
For examples of research briefs, try: Consumer research brief examples: https://www.­
culturehive.co.uk/resources/research-brief-samples/
For information on working with market research providers, try: MRS/ISBA (2011) A Guide
to Understanding and Working with Market Research Agencies and Consultancies https://
www.mrs.org.uk/pdf/ISBA_Guide_to_Market_Research.pdf
For information and guidance on becoming a government research supplier, try: https://www.
mrs.org.uk/pdf/dps_supplier_guidance.pdf
The MRS also publishes a document on buying and commissioning research within the public
sector. The latest report at time of writing was dated 2017: https://www.mrs.org.uk/pdf/
MRS%20Third%20%20procurement%20report.pdf
139
M06 The Practice of Market Research 31362.indd 139
27/09/2021 21:45
Chapter 7
Writing a research proposal
Introduction
In this chapter we look at preparing a research proposal. This is the document
in which the researcher describes the research that will be conducted; why
that research is a suitable way to address the client’s information needs; the
time frame in which it will be completed; and the costs that it will incur. We
saw in Chapter 6 that preparing a sound proposal relies on being well briefed
about what the client wants. In this chapter, we look at the links between brief
and proposal and at how to question a brief. We look at what a proposal must
achieve and at the sections it should include. Finally, we look at how to evaluate
a proposal and how to respond to it.
Topics covered
The purpose of a research proposal
● Links between brief and proposal
● Questioning the brief
● The contents of a proposal
● Evaluating a proposal
● What happens next?
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 1: Understanding the research context and
planning the research project.
M07 The Practice of Market Research 31362.indd 140
29/09/2021 16:08
What you should get from this chapter
At the end of this chapter you should be able to:
understand the role of a research proposal in the research process;
● understand the relationship between the brief and the proposal;
● design a research proposal; and
● evaluate a research proposal.
●
141
M07 The Practice of Market Research 31362.indd 141
29/09/2021 16:08
Chapter 7
Writing a research proposal
The purpose of a research proposal
When you come to write a proposal you will have received – in most cases – the
research brief; you may even have had a meeting with the client at which you clarified
or discussed in further detail any issues you identified when you read the brief. (If
you work in an academic setting, you may be writing a proposal in order to secure
funding for your own research and so you will have identified an area of research
that interests you and in effect you will have written your own brief.). The purpose
of a research proposal is to show the client the following:
that you understand the problem and the issues involved;
that you understand the implications of the problem in research terms and in terms
of the client’s wider business context;
● that you have the expertise to design, set up, manage and deliver good-quality,
ethically robust research that will provide the evidence and the insight the client
needs to make a decision or to take action.
●
●
Research proposals vary in terms of length and complexity and degree of detail. You
may not receive a written brief. It is, however, good practice to prepare some sort
of written proposal, even if the brief is a verbal brief. Writing a proposal will help
you think through the issues. Having a written document that sets out what is to be
done and why it is being done will avoid confusion and misunderstanding. It can be
used as a focus for further discussion about the research project, helping client and
researcher get to grips with the aims and objectives of the research, and it will act as
a record for future consultation.
Links between the brief and the proposal
The quality of a proposal – and so the effectiveness and suitability of the research
proposed in it – relies heavily on the quality of the brief. To prepare a good-quality
proposal you must have a clear understanding of the client’s business problem and
its wider context, the type of information needed to address the problem, and how
this information will be used. If the brief does not give you this then you must seek a
­meeting with the client to discuss it further. So, first of all, you must question the brief.
Questioning the brief
When you – the research supplier – receive the brief, spend some time reading through
it. The sorts of questions you should ask yourself at this stage are:
Is this a client you want to work with?
Is this a topic or issue you want to work on?
● Is this something that you have experience or expertise in?
● What type of research do you think will be necessary?
● Can you provide this type of research?
●
●
142
M07 The Practice of Market Research 31362.indd 142
29/09/2021 16:08
Questioning the brief
What is the time frame? Can you do the research in that time frame? Do you have
the resources?
● What is the budget? Is the research feasible within that budget?
● What ethical, legal or regulatory issues, if any, does the brief raise?
●
If this is a project that interests you, and you are equipped to deal with it in the time
available, and it poses no insurmountable ethical, legal or regulatory issues, it is likely
to be worth exploring further. The next set of questions to work through are those
that should help you unpack the problem and the client’s information needs:
What is the business problem?
What is the wider context of the business problem?
● Why is the research needed?
● Is it clear what the information needs are?
● Will research help in addressing the business problem?
● Is it clear how the client wants to use the research findings?
● Is it clear who will use the research findings?
● What assumptions, if any, have been made in defining the business problem and
the research problem?
● Are the research objectives clear and unambiguous, and relevant to the
problem?
● If a research approach is suggested, is it feasible? Will it deliver what is needed?
● Is there enough information in the brief to enable you to write a good-quality
research proposal?
●
●
Once you have questioned – or challenged – the brief, you will probably find that to
write a sound, high-quality proposal you need to go back to the client for clarification and/or more information.
Your aim as research supplier is to provide the client with good-quality, ethically and legally robust, actionable research. Such research is typically the result of
a collaborative partnership between client and researcher and, generally speaking,
the earlier this process starts, the better. Here – as in all other stages of the research
process – there should be dialogue and collaboration, and not just for unusual or
complex projects. Your first encounter with the client may well be through the brief.
If you find that the brief is shoddy, what is required is unclear, the budget and ­timings
offered are not feasible, then you may well decide that taking things further is not
worth the time and effort, or you may decide to persevere and contact the client to
ask questions and gather more information.
Clients commission research to address or understand an issue in order, ultimately, to add value to their organisation. The client may need research, for example, to help identify a new market opportunity, or to understand its customers’
online user experience, or to help choose the right service delivery package, or to
test a new product, or to find out how satisfied customers are with an existing
product. They act in expectation that the research findings will provide evidence
and insight that will help them add value. This is what good-quality research
does. In Chapter 1 we looked at the factors that can limit the value of a piece of
research. Let’s revisit the ones you can address at the briefing and proposal stage.
Here they are:
●
●
poor definition of the problem;
lack of understanding of the problem (or the brief);
143
M07 The Practice of Market Research 31362.indd 143
29/09/2021 16:08
Chapter 7
Writing a research proposal
poor or inappropriate research design;
limitations of the methods used;
● poor quality of the data;
● time between commissioning the research and delivering the findings;
● use or misuse or non-use of research evidence by the decision makers.
●
●
Poor definition of the problem
It is essential that you have a clear and accurate definition of the business problem the
research is to address, the wider context of the problem within the client’s business
and the wider environment, and the decision to be made on the basis of the research
evidence. You will be designing the research project around the problem, so if the
problem is not defined clearly and accurately then the research findings may be useless. One of the keys to ending up with good quality evidence is to make sure the
problem is clear and well defined and presented with as much context as possible, so
that it is clear what the research must accomplish. Aim to find out as much as possible about anything that may affect the action to be taken as a result of the research
findings. Check that all the relevant stakeholders have been consulted about what the
research is to investigate so that at the other end of the research process, delivery of
findings, everyone is clear about what can (and cannot) be done on the basis of the
evidence provided by the research.
Lack of understanding of the problem (or the brief)
If you fail to understand what the research must deliver, or you misinterpret what
is needed, you may design research that is not suitable and so of little or no value.
The client commissioning the research has a responsibility to ensure that the business problem (and the brief) is clear and accurate. This does not, however, take any
responsibility away from you as the research supplier: you have a duty to ensure you
understand the problem and the brief (Pyke, 2000), and you must be clear about what
evidence the research must provide.
Poor or inappropriate research design
Research design is a key stage in the research process. We looked at it in some
detail in Chapters 4 and 5. The value of the findings, the evidence, produced by the
research depends on the research design. It must be able to provide the right kind of
evidence with which to address the problem. If the research design is not appropriate
to the problem, the research will be of little or no value. How do you choose a suitable research design? You need a clear and accurate definition of the problem to be
addressed by the research, a clear understanding of the context or setting, and a clear
understanding of how the research findings will be used. Of course, it doesn’t end
there: a suitable design must be accompanied by a well-executed research, analysis
and interpretation process.
144
M07 The Practice of Market Research 31362.indd 144
29/09/2021 16:08
Questioning the brief
Limitations of the methods used
The quality of the data collected will only be as good as the methods used to collect them (and of course how well those methods are executed). Choose the most
suitable method of data collection for the problem at hand. Remember, all methods
have advantages and limitations; the key is to choose the method best suited to the
context, the method most appropriate for the research problem. Ensure that quality
of execution is the best you can achieve with the available resources. Acknowledge
any limitations, and plan for how you will mitigate them.
Poor quality of the data
Whatever the source of the data, and whether they are data that you generate or existing data that you make use of, the value of your findings will be limited if the quality
of these data is poor. You must aim to ensure that the data are of sufficient quality
for your purposes. You must be aware of any limitations or actual or potential bias
within the data. If you are using existing data you should screen them thoroughly
for quality (and relevance) and be explicit about any data preparation or preprocessing that is needed to address quality issues. Whatever data you use or generate,
you should indicate clearly any limitations in the quality. If you have to make any
assumptions about this, you should state them clearly and justify them.
Time
The time that passes between the client commissioning a project and the supplier
delivering the evidence can limit the value of that evidence. Data become out of date
very quickly, so the passage of time can erode the value of the evidence. If you think
this may be the case with your project, discuss it with the client and, if necessary,
design the research to take this into account.
Use, misuse or non-use of research evidence
The value of research also lies in whether the findings are used and, if they are used,
how they are used. You want the client to get maximum value from the research you
are proposing. To achieve this you will need to clarify with the client when you receive
the brief what exactly it is they want from the research; who the decision makers or
end users of the research are; what they think it will deliver; how they would like you
(the researcher) to handle or present the findings; and what they plan to do – how
they envisage using the findings and what decisions are to be made on the basis of the
findings. As researcher, you have a role in managing the expectations of the client in
terms of what the research can and cannot provide. It is important to think about the
end at the beginning: have a plan to help the client understand, communicate, disseminate and integrate the findings into their organisation. Industry Insight 7.1 is about
a project to help the client understand its target consumers – those suffering chronic
pain. It is an example of ‘designing in’ how the findings would be communicated.
145
M07 The Practice of Market Research 31362.indd 145
29/09/2021 16:08
Chapter 7
Writing a research proposal
Industry Insight 7.1
Communicating the pain
Introduction
Pharmaceutical company Bayer markets a range of
pain relief solutions. It wanted to understand better those who suffer chronic pain. The challenge
for research agency Ipsos Mori was two-fold.
First, how to capture data in the moment –
someone with chronic pain may have a pain-free
day and the researcher is relying on their ability to
recall and describe the symptoms and their intensity, some of which they could forget, and thus
valuable information is lost. Second, how to communicate findings about something that is unique
to each individual and invisible to the eye, and
how to communicate them with impact so that the
client understands the person’s world.
Workshops
moment. Researchers asked the participant to discuss five key topics in the film: the condition causing their pain; adjustments made to daily activities
or modifications in the house; tasks they find
hard and movements that trigger pain; solutions
they’ve tried to ease the pain, which were successful and which weren’t; and the overall impact
of chronic pain on their daily life. The film was
made by Gorilla in the Room using a 360-degree
camera, enabling the film to be viewed later by
Bayer staff using a Virtual Reality (VR) headset.
In the edit, the researchers added infrared signals
to show when and where the participant’s pain
flared up during the activity.
Outputs from the workshop
Those suffering chronic pain shared their stories
with Bayer employees so that they might understand what they experience on a daily basis. The
employees were given a short discussion flow
[guide] and a set of cards with key business questions to aid the discussion.
On completing the workshop Bayer staff were
asked to reflect on their experience, think of key
revelations and remaining business challenges and
list initial ideas for marketing, communications,
product, branding and packaging, writing this
up in a workbook. The workshop facilitator also
wrote up the learnings and presented these to the
Chief Executive Officer and the Chief Marketing
Officer. The workshops were filmed and the client was given a three-minute video summary and
a presentation that could be used to show other
teams elsewhere. The approach has been replicated
in other areas of the business. The benefits it offers
are as follows:
Pain simulation
●
The agency designed a series of workshops for
70 Bayer employees. Each workshop consisted of
three elements: a session called ‘Meet Your Consumer’; a pain simulation exercise; and a virtual
reality (VR) video.
‘Meet Your Consumer’
The researchers fitted knee, back and hand pain
simulators to Bayer staff and asked them to
complete tasks that participants had reported as
challenging.
VR video
A research participant told researchers that a
task that gave them the most amount of difficulty
with their pain was unloading the dishwasher and
putting away the dishes and cutlery. The agency
filmed the participant at home doing this task so
as to capture the challenges and symptoms in the
Ability to capture insights in the moment,
avoiding relying on patients to recall their
experiences which can leave out important and
relevant details
● Not limited by pre-existing hypotheses – the
approach leaves room for new insights to be
uncovered through a patient-led approach
● Cost-effective and time saving – the client gets
actionable insights on the day of the workshop
● The insights can be a catalyst for further
research.
Source: Adapted from Ipsos MORI UK and Bayer, ‘The patient
empathy workshop’, MRS Awards 2019. Used with permission.
146
M07 The Practice of Market Research 31362.indd 146
29/09/2021 16:08
The contents of a research proposal
The contents of a research proposal
Set out below is a detailed list of the contents or headings for a proposal. Not all
projects, however, start with the sort of formal, detailed proposal described in the
list. The research being commissioned may be similar to, or a repeat of, a previous
study and so may not warrant a full proposal; or the researcher and client may
have an established relationship and so the client may not require the detail of a full
proposal. Time may be a factor, limiting what can be produced. Ideally, about two
weeks’ notice is needed to prepare a basic, straightforward proposal – giving the
researcher time to fit it into the work plan, arrange for costings to be prepared and
so on. More time will be needed if the research design is complex or if the client has
specified a high level of detail about the design and execution of the research or if
the has asked for detailed information about the agency and its project staff. Several
days will be spent thinking and writing, and again depending on the complexity and
the level of detail required it can take much longer than this. Sometimes a one or two
pager – a short, less formal proposal covering the basics of introduction, a statement
of the problem, the need for research, research objectives, recommended approach,
reporting, timings, costs and relevant experience – is all that is needed.
Here is a guide to what should be covered in a full proposal.
Background to the problem
Show the client that you understand the nature and setting of the problem. Do some
background research – do not just reproduce the background information that the
client gives you in the brief. Add information (from secondary research, for example)
that shows you understand the issues and the client’s business problem and the wider
business setting or context of the problem. This can add value to the proposal, and it
shows the client that you are interested and willing to do that little bit extra.
Research objectives
The research objectives should state what the research will do and so should be relevant to the research problem. They may not be fully or clearly stated in the brief, so
you may need to do some work to draw them out. This will probably mean digging
into or unpacking the business problem in detail, questioning the client about it and
agreeing on it before moving forward. It is crucial that your understanding of the
business problem, the information needs and the research objectives and the client’s
understanding are one and the same; and you both should agree that they will deliver
the necessary information with which to address the business problem. From the
research objectives, and from other information provided in the brief, you may be
able to set out what general questions will be addressed in the research.
Research design
Set out the research design and why this design is the most suitable for collecting the
evidence needed. Whatever you suggest, explain why you have suggested it, and set
147
M07 The Practice of Market Research 31362.indd 147
29/09/2021 16:08
Chapter 7
Writing a research proposal
out the limitations that the design may have and how, if possible, these limitations
can be overcome or mitigated. Describe the evidence you need, the nature of the
research enquiry and the type of data you need to collect.
Sampling
State clearly the target population for the research. For example, it might be all those
aged 17–64 living within the area served by a bus route and who have used the bus
service in the last six months; or all those aged 55 and over living in the community;
or all those within an organisation who are users of a particular software platform.
Note your assumptions about the incidence of the target population in the wider
population and the basis of the assumption. Explain how you intend to draw a sample from this population, for example using quota sampling or random sampling or
a sample drawn from an online panel. State the intended sample size, or the number
of group discussions or depth interviews, and the size of any sub-samples that are
relevant to the research objectives (for example, those who use the bus service during
weekdays). Explain the reasons for these choices, and the implications they have for
achieving the objectives, for data quality, for timing and for costs. Point out if you
envisage any problems in either contacting the sample or achieving the interviews
and explain how you propose to overcome these. Note how you will deal with any
ethical, legal and regulatory issues.
Method of data collection
In this section you must describe the way in which you plan to collect that data. For
example, if it is a qualitative research project, you might describe how you plan to
use online group discussions, or accompanied visits or individual or paired depth
interviews; if it is a quantitative project, indicate if you are using online methods or
in-home, face-to-face interviews, for example. State the reason why you are recommending a particular method. Specify key things about the approach, for example
the expected interview length, its content or coverage and its style. You do not need
to include a fully worked-up data collection tool but you may want to show the client that you understand what topics or question areas need to be covered in order
to address the research objectives. This is important because the length and type of
interview will have an impact on the timings and cost of the project. You should make
any assumptions on which the timings and costs are based clear so that the client can
see how you reached them. Set out the implications of using the method suggested:
what are the advantages and disadvantages? For example, if you have suggested
an online survey it is worth pointing out that this will limit questionnaire length to
about 15 minutes. Will this be long enough to cover what you need to cover? If not,
what do you suggest? Perhaps you need some exploratory research to narrow the
focus so that you can achieve what you need to achieve in a 15-minute online interview. Include information on how fieldwork is to be organized if that is appropriate.
You could explain that participants for the group discussions will be recruited by
specially trained qualitative recruiters; that the fieldforce for the quantitative survey
meets the standards set by the Interview Quality Control Scheme (IQCS); that work
148
M07 The Practice of Market Research 31362.indd 148
29/09/2021 16:08
The contents of a research proposal
is conducted in accordance with the MRS Code of Conduct or the ESOMAR Code
of Practice. Again, note what the ethical, legal and regulatory issues are and how
you will manage them.
Industry Insight 7.2 describes the research conducted in response to the brief in
Industry Insight 6.1. It includes the rationale for some of the data collection methods
the agency used.
Industry Insight 7.2
Establishing the facts – the research
Introduction
Macmillan want to explore and measure the range
of personal social care needs of those with cancer: the extent of those needs; and how they are
currently being met. They also wanted to map
people’s level of need against criteria that local
authorities in England use to assess eligibility for
support. The population of interest is people currently in treatment for cancer; people who have
successfully completed treatment; people with
advanced cancer; and those with a terminal diagnosis. Macmillan commissioned research agency
Bright Blue to do the research.
Scoping the issues
We began by scoping the issues. We conducted a
programme of deliberative cognitive testing of the
terms and question wordings we might use with
different audiences affected by cancer.
Sample
We recruited a representative range of people living with cancer with quotas on the three cancer
journey stages requested by Macmillan as well as
cancer type, age, gender and nation. We weighted
the resulting data using official cancer prevalence
statistics provided by Macmillan.
Data collection methods
The approach comprised a quantitative and a
qualitative phase. For the quantitative phase
we decided on an online survey. This selfcompletion approach would allow people to
take their time, work at their own pace through
the survey, speak openly about very personal
issues from vulnerability and survivor’s guilt to
personal hygiene in an anonymous, ‘safe’ setting. To reassure participants that they could tell
their story, we structured the questionnaire in an
unorthodox way. We used open-ended questions
at the beginning to allow people to share things
that were top-of-mind before moving to closed
questions. The questionnaire had a nested structure in which everyone’s needs were identified
at a high level before allocating participants to
a small number of detailed question loops about
specific needs. We also built in signposts to
sources of help for those needing support on the
issues they were describing. With this approach,
we were able to meet all the objectives. To map
the results to local authorities’ eligibility criteria, we created a set of proxy questions to represent the assessment materials used by the local
authorities. Participants were able to complete
these themselves.
The quantitative phase was followed by qualitative research – in-home depth interviews and
completion of an online diary. The depth interviews were conducted with participants from each
of the segments identified through the quantitative
research. These in-depth interviews helped identify the looping effect that the inter-relationship
between medical and social care needs can cause,
which was not apparent from the quantitative
research. In the online diary people recorded for
a week their care needs and uploaded photos and
videos about their experience.
Source: Adapted from Bright Blue and Macmillan, ‘Hidden at
home: Establishing the hard facts to set the social care agenda’,
MRS Awards 2015.
149
M07 The Practice of Market Research 31362.indd 149
29/09/2021 16:08
Chapter 7
Writing a research proposal
Data processing and analysis
Set out how the data will be processed and analysed. If you are dealing with personal
data you must comply with the relevant data protection legislation and you must
be clear about how you will handle such data in order to be compliant. You should
also set out how you plan to analyse the data to meet the research objectives and so
address the client’s business problem. You may also want to mention technical and
quality aspects, for example, how the data will be cleaned, edited, coded and analysed,
and what software will be used and in what format the data output will be available.
Finally, you may want to note any data storage and data security provisions.
Outputs
Make it clear what outputs you will provide, the format (report, presentation, dashboards, visualisations, films) and the dates on which they will be provided. Set out
the cost of any additional deliverables, for example interim summary reports or briefings, so that the client can take account of the cost implications if these are required.
Ethical, legal and regulatory issues
If you have not already done so in the relevant sections, you must identify any ethical,
legal or regulatory issues in relation to the research that you propose, and you must set
out for the client how these issues will affect how the research is conducted, how the
data are processed and analysed and how the findings are presented, disseminated and
used. You must set out what steps you as the researcher will take to address the ethical,
legal and regulatory issues, and you must set out what the client’s own responsibilities
are. The sorts of issues that may arise may cover all or any aspect of the proper and professional conduct of research in relation to clients and research participants and issues of
data privacy and data security. You must adhere to any relevant regional, national and
local laws or regulations, and to the professional code of conduct to which you belong.
Many organisations, including universities and those funding research in the public
or voluntary sector, require researchers to submit details of their research plans to
research ethical approval committees or ethics review boards. These bodies scrutinise
the plans to ensure that researchers have identified and are taking steps to address
any ethical issues. Typically, among other things, they ask about sample composition and whether it contains people who might be vulnerable (including children);
they ask about how informed consent will be achieved; they ask whether research
participants run a realistic risk of being harmed by taking part in the research; and
they ask about data protection and data security issues. Check whether or not your
organisation has such a process and make sure to comply with it.
As we saw in Chapter 2, there are a number of codes that set out how researchers (and clients) should behave. Researchers who are members of MRS must adhere
to its Code of Conduct when planning and conducting research. For those who are
not members of MRS, the Code of Conduct nevertheless provides a useful guide to
standards and practice. An extract from the MRS Code of Conduct (2019) in Box 7.1
shows some of the points relevant to the research commissioning and p
­ roposal
­writing stage.
150
M07 The Practice of Market Research 31362.indd 150
29/09/2021 16:08
The contents of a research proposal
Box 7.1
Professional practice and the MRS Code of Conduct
General Rules of Professional Conduct
6 Members must act honestly in their professional activities.
8Members must not act in a way which might bring discredit on the profession,
MRS or its Members.
9Members must take all reasonable precautions to ensure that participants are not
harmed or adversely affected by their professional activities and ensure that there
are measures in place to guard against potential harm.
Commissioning and design
10Members must not knowingly take advantage, without permission, of the
unpublished work of other practitioners, which is the property of those other
practitioners.
Comment: This means, where applicable, that Members must not knowingly carry
out or commission work based on proposals prepared by practitioners in another
organisation unless permission has been obtained.
11Members must take reasonable steps to design projects to the specification and/
or quality standards agreed with clients.
12Members must carry out Data Protection Impact Assessment (DPIA) for specified
types of processing prescribed by data and privacy legislation and for any other
processing that is likely to result in a high risk to participants.
13Members must ensure that the rights and responsibilities of themselves, clients, and
sub-contractors as controllers or processors are governed by a written contract.
Comment: See Data Protection & Research: MRS Guidance Note on Controllers
and Processors.
Client confidentiality
14Members must disclose the identity of clients where there is a legal obligation to
do so.
Comment: Transparency is one of the fundamental principles underpinning data
protection laws. In line with this an obligation to name a commissioning client may
arise in three main scenarios:
(a) Client is controller or joint controller
(b) Client is the source of personal data
(c) Client is receiving personal data from a research activity
15Where files of identifiable individuals are used eg, client databases, Members
must ensure that the source of the personal data is revealed at an appropriate
point in the data collection.
Comment: The identity of the client must be revealed when data collection is
­undertaken if clients require personal data from a project.
Source: MRS (2019) Code of Conduct. Used with permission.
151
M07 The Practice of Market Research 31362.indd 151
29/09/2021 16:08
Chapter 7
Writing a research proposal
Timetable
Include a draft timetable or work schedule, highlighting key dates, especially those
that are dependent on input from the client. Two different formats are shown
below. Figure 7.1 is set out as a table showing the dates associated with key tasks
or ‘milestones’ (you could add a third column to show the outputs associated with
the tasks). Figure 7.2, a Gantt chart, shows the individual activities or tasks as
bars with week numbers or days, so that it is clear when different phases begin and
end and where they overlap. At this stage you may not be able to include exact
dates – this will depend on the client giving the go-ahead – but you can put in week
numbers and add in the dates when the details have been confirmed. In drawing up
the timetable think of the practicalities. If possible, and it is not always possible,
build in some contingency time; if the timetable is tight, mention this to the client
and explain why.
Working out precise project times can be difficult – there are so many elements
involved that impact on timings, for example:
nature of the research – primary or secondary, qualitative or quantitative, or a
combination of types;
● size, scope and complexity of the project;
● source of the data or the method of data collection and data capture, e.g. online
or face to face;
● nature of the population under study – ease of access, strike rate or speed and ease
of recruitment;
● length of the interview;
● sample size;
● geographic scope of the research;
● time of year (the impact of holidays);
● size and complexity of the dataset;
● nature of data preparation, processing and analysis needs;
● reporting requirements;
● thinking time – time needed for interpretation, thinking about the implications of
the findings;
● extent of liaison required with the client during the project.
●
Costs
Include details of the cost of conducting the research proposed and the assumptions on which these costs are based. The detail in which you present the costs may
vary depending on custom and practice within your organisation, or on the level of
detail requested by the client, and by the nature of the project. Some clients want to
see an overall cost plus an estimate of expenses; others want to see the number of
hours each staff member will spend on the project and their daily or hourly rate. A
quantitative costing might be presented on a task-by-task basis, or as a client service
cost plus a ‘field and tab’ cost. An example of a project costing grid for client service time for three staff grades is given in Table 7.1. Daily rates are for illustrative
purposes only.
152
M07 The Practice of Market Research 31362.indd 152
29/09/2021 16:08
The contents of a research proposal
Week
Tasks
1
Project start meeting
Discuss project management issues
Agree key ‘milestone’ dates – approval re discussion guide, questionnaire and so on
Agree target population/sample for qualitative work
Agree date of next meeting
2
Choose sample for in-depth interviews
Design discussion guide for in-depth interviews
Get client approval for discussion guide
Brief recruiters
Recruit for in-depth interviews
3
Conduct in-depth interviews
Transcribe interviews
Analyse interviews
Informal telephone debrief with client on findings to date
4
In-depth interviews completed
Transcribe interviews
Analyse interviews
Prepare and send summary report of findings to client
5
Progress meeting
Discuss findings from in-depth interviews
Discuss development of survey questionnaire
Prepare draft survey questionnaire and send to client
6
Meet with client to discuss draft questionnaire
Amend draft questionnaire and send to client for comment and approval
Conduct pilot interviews with approved draft questionnaire
Feed back findings from pilot interviews to internal project team and to client
Agree amendments to questionnaire
7
Finalise questionnaire
Fieldwork planning and set-up
Run survey questionnaire briefing session
8
Fieldwork set-up completed
Fieldwork start
Fieldwork visit with client
9, 10, 11
Draw up specification for analysis of data and send to client
Agree analysis spec
Fieldwork ongoing
Answer field queries re-editing and coding
Data processing set-up
Liaise with client
Agree code frames with client and field
12
Data processing ends
Standard tables produced
Check tables
Prepare topline summary report for client
13
Finalise tables and send to client with datafile
Meet to discuss findings and plan presentation/workshops
14
Write report and design presentation
Send draft summary report and draft presentation of key findings to client
Do presentation to client
Design workshops on findings
15
Answer client follow-up queries
Deliver two workshops to internal client team
Figure 7.1 Example 1 of a draft project timetable
153
M07 The Practice of Market Research 31362.indd 153
29/09/2021 16:08
M07 The Practice of Market Research 31362.indd 154
5 weeks
3 weeks
8 weeks
4 weeks
Project tasks on critical path
Task preparation phase
Key milestones
Project tasks
January
February
March
April
May
June
July
August
3 10 17 24 31 7 14 21 28 6 13 20 27 3 10 17 24 1 8 15 22 29 5 12 19 26 3 10 17 24 31 7 14 21 28
Figure 7.2 Example 2 of a draft project timetable
Conduct project group meetings
Prepare 1st draft of report
Prepare detailed report plan
Write 1st draft
Review 1st draft with core team
Prepare 2nd draft of report
Prepare 2nd draft
Circulate 2nd draft to core team for comments
Prepare final version of report
Prepare final version of report
Final draft to printers for typesetting
Report review and checking
Print and send out report
Report printed
Report sent to client
Follow-up discussions with client
Conduct research
Exploratory research
Clarify and agree secondary research needs
Do secondary research
Write up key findings
Primary research
Agree primary research needs and approach
Design and agree questionnaire
Fieldwork and data processing
Analyse findings and write up
Do follow-up in-depth interviews
Develop research approach
Define research problem
Develop research approach and research plan
Prepare brief for primary research
Project start
Develop project plan
Brief core team
(Week commencing)
Comments on structure,
content and style.
Comments on content and
structure.
Include detailed contents list,
storyline and themes.
Prepare analysis plan and
analytical framework.
Run dynamic pilot. Listen to tapes of
interviews and/or attend fieldwork.
Do back-translations.
List sources of secondary data
for referencing.
Prepare background reading
material for team members.
Notes
Chapter 7
Writing a research proposal
154
29/09/2021 16:08
The contents of a research proposal
Table 7.1 Example of a costing grid for client service time
Staff grade
Staff cost per hour
Director
Manager
Senior Exec.
Research Exec.
£400
£200
£150
£100
Task
Estimated time in hours
Project team briefing
1
1
1
1
General administration
1
2
3
8
Questionnaire design
1
1
7
5
Liaison with client
1
2
2
3
1
Fieldwork briefing
1
1
Liaison with field
1
1
Design of analysis spec
2
1
Liaison with DP
1
1
Checking DP output
1
3
Preparation of topline findings
Discussion of findings with team
1
1
4
2
3
3
Preparation of report
1
2
4
21
Preparation of presentation
1
2
2
7
Delivery of presentation
2
2
Post-presentation follow up
1
1
1
1
Total time per team member
10
16
30
60
Total cost per team member
£4,000
£3,200
£4,500
Total client service cost
£6,000
£17,700
A qualitative costing might be presented in terms of cost per group or depth interview, or as one total cost, or it may be broken down into recruitment costs, fieldwork
costs, moderator’s fee and report writing fee.
Be clear about the length of time for which the costs you quote are valid. The time
that elapses between submitting the proposal and being commissioned to do the
work may be considerable and costs may rise (or fall) in the interim. Be clear also
about the costs for which the client will be liable if they cancel. You incur costs in
preparing a proposal (including the time spent working on the proposal, time away
from other projects, and opportunity costs in not being able to pursue other work).
You incur further costs in setting up and planning the research once the project is
commissioned. You will incur losses if the client decides to cancel either during the
proposal and pre-commissioning stage or once the project has been commissioned.
You should have a provision within the proposal to deal with such cancellations.
Make sure you state whether the cost you have quoted includes or excludes any
relevant sales tax. Costing international projects can be difficult, especially with
fluctuating exchange rates. In costing an international project make it clear in which
currency you are billing the client and, if exchange rates apply, what range of fluctuations in the rates (typically 10 per cent) will be acceptable before it is necessary
to recalculate the cost.
155
M07 The Practice of Market Research 31362.indd 155
29/09/2021 16:08
Chapter 7
Writing a research proposal
The cost of a project is closely related to expenditure of time. In particular, it will
depend on degree of difficulty obtaining participants, length of interview, total number of interviews, method of data collection, project design requirements, analysis
requirements, and project management and reporting requirements. Costing procedures vary. For example, the data processing and client service costs for a quantitative
project may be worked out by those departments in an agency, or by the supplier, on
the basis of an hourly rate for the grades of staff involved and the number of hours
it is likely to entail. Hourly (and daily) rates are calculated on the basis of employee
costs and overheads (e.g. the cost of social insurance and pension contributions
and the cost of office space and equipment) and include a profit margin. The field
department or fieldwork supplier may work out fieldwork costs. The strike rate or
number of interviews achievable in an interviewer shift, the cost of that interviewer
shift and its associated expenses (equipment and venue hire, travel and subsistence
costs and so on) plus the cost of managing the project – the cost of supervisor time
and administrative time – will all be used in reaching a total cost.
It can be difficult to price the service you offer: you do not want to under-value
it or over-price it. You will get some feedback from clients about whether or not
your costs are competitive, especially if you are involved in a competitive tendering
process, and you may get some information informally from other practitioners. You
may find it worthwhile consulting the ESOMAR Global Prices Study, which is carried out on a bi-annual basis with quotations provided by more than 600 agencies in
more than 100 countries. It provides information on costs for seven research projects
(six consumer and one business-to-business) using qualitative, quantitative and social
media approaches and giving charges for staff time and a presentation.
Relevant previous experience and expertise
This is your chance to sell, to show your credentials, to tell the client why you should
conduct the research. Rather than presenting a standard credentials pitch, tailor it to
the particular research brief. Think about what you bring to this subject matter, to
this type of research, to this project. The client may request details of project team
members. Whether this is the case or not, it is useful to include a set of short CVs or
résumés of key staff designed to show relevant experience and expertise.
Terms and conditions of business
The proposal is an important contractual document as well as a selling tool. It is
important to include information on your terms and conditions of business, including
notice of your adherence to the MRS Code of Conduct or ICC/ESOMAR Code of
Practice; your organisation’s rights and responsibilities as well as those of the client
and any sub-contractors in particular in roles as controllers or processors of data;
your data security, data storage and data retention policies; how you plan to bill the
client; an assertion of the right to amend the project cost if the client changes the
specification; and what payment is due if the client cancels the project after commissioning it. It is also worth including something asserting your intellectual property
rights. Although the vast majority of clients and agencies operate in an ethical way,
some are unaware of such guidelines and others ignore them.
156
M07 The Practice of Market Research 31362.indd 156
29/09/2021 16:08
The contents of a research proposal
Writing style
In writing the proposal, do not assume that the intended readers will be research
experts or particularly research literate. Aim to explain things clearly and simply.
Avoid trying to impress with unnecessary jargon or technical language.
Meeting the proposal submission deadline
The research proposal should be submitted on time. It creates a poor impression if
it arrives late. In government tendering a deadline is set – for example 15.00 hours
on 17 October – and no proposal will be accepted after that time, even one minute
after it. Other clients are less precise – close of business on Thursday or Tuesday
morning may be the instruction. A deadline is a deadline and you should use it as an
opportunity to show that you can complete a piece of work on time.
Box 7.2
Example: key sections from a proposal
The proposal below is an ‘outline’ style proposal for a basic two-stage (qualitative
­followed by quantitative) multi-country research project in the B2B area.
Background and introduction
The John Doe Graduate Business School at Wonderland University provides postgraduate and professional business education. The aim of its one-year programme,
Master of Science in Marketing, is to give students – or as it refers to them, marketing
advisors – practical experience in marketing and help them develop management and
professional skills. More specifically, the programme aims to help its marketing advisors
develop a business and marketing knowledge base and build skills in teamwork and
team building, communication, negotiation, goal-setting and leadership. It aims to help
them with their personal as well as professional development. The learning culture is
one of learning by doing. By working on assignments of commercial significance to the
School’s partner organisations, advisors learn ‘hard skills’ that benefit their employers
immediately they start work. The ‘soft skills’ they acquire via the course can help them
progress quickly to senior management level.
As a result of this skills profile, the School has found that marketing advisors find jobs
quickly, and in well-known, high-profile organisations. Many take up positions such as
assistant brand manager, marketing executive and business development executive. A
number of graduates of the programme now hold senior management roles in leading
national and international organisations.
The programme directors are constantly seeking ways of improving the course and the
employability of its graduates. To help do this effectively they want to understand in
more detail how effective marketing education really is in preparing people for senior
positions in major European and international organisations. The research findings will
be used to review and, if need be, improve the marketing education programme. It is
likely that the findings will be published in the relevant marketing education journal.
157
M07 The Practice of Market Research 31362.indd 157
29/09/2021 16:08
Chapter 7
Writing a research proposal
Aims and objectives: What it is you want to achieve with this research
From your brief we understand that you want to ensure that the MSc in Marketing is a
first-rate qualification, one that allows its graduates to obtain jobs in major organisations and to progress rapidly to senior management posts within those organisations.
To do this you need to know what it is that allows this to happen – what makes those
with a postgraduate marketing degree attractive to these organisations and what makes
them move quickly through the ranks to senior positions. What knowledge base do they
need? What hard skills? What soft skills? What sort of experience?
The assumption in your brief is that marketers do not get senior positions within top
companies – or rather they are less likely than those in other disciplines or departments (for example, finance and accounting, or law, or engineering) to secure these
‘C-level’ jobs. This is something that we feel is worth exploring via secondary research,
which we have not costed for here but which we would be happy to discuss further
with you.
This leads us to your three main objectives:
1 To understand why marketers don’t get senior positions within top companies. In
other words, to address the question: Why are marketers less likely than others to
become C-level or board level executives, CEOs or COOs?
2 To determine where the fault lies. Is it within marketing education? If so, what are
the problems with marketing training? What is missing? How could it be improved?
In other words, to understand the role of postgraduate marketing education in
helping marketers achieve senior positions.
3 To get guidance on the course of action for marketing education. In other words,
to address the question, ‘What exactly do you need to do to improve graduates’
employability and ‘promotability’?
The sample: Who to talk to?
Who has the answers to these key questions? In your brief you have identified CEOs.
We would also recommend that you include senior recruitment consultants, those
who recruit people to senior posts within major organisations. We believe that it may
also be worth including senior human resource directors, those who are involved in
recruitment and promotion. We feel that all three groups will be able to offer insight
and understanding.
Suggested approach
We propose a two-stage approach: a qualitative study followed by a quantitative phase.
Location coverage
Since many of your former and current marketing advisors (and future ones) work in
international organisations, and since the findings of this research may have wider
significance, you have decided that the scope of the research should extend outside
Ireland. Specifically, you wish to talk to those in UK, France and Scandinavia. We
would suggest two additions to the list: Germany and the Netherlands. As the biggest
economy in Europe and home to many major brand-owning organisations we feel it
would be important to include respondents from German organisations. We recommend the Netherlands on the same basis – it is home to many well-known brands and
158
M07 The Practice of Market Research 31362.indd 158
29/09/2021 16:08
The contents of a research proposal
English is widely spoken among its business community, making it a possible place of
employment for English-speaking graduates.
In addition, from our own experience researching the views of business leaders across
Europe for other clients, we have found responses to be influenced by culture. Interviewing in the Ireland, UK, France, Germany, Netherlands and Scandinavia will give
you a perspective on that.
At the qualitative stage of the research we would recommend conducting at least two
interviews in each of five countries in order to get as wide a view as possible, cultural
differences and all, to inform the quantitative. At the quantitative stage we would recommend interviewing in up to seven countries.
To help elucidate the research problem more fully, and to understand the key issues in
more depth, we would suggest a programme of exploratory qualitative research. It is
important, we feel, to understand what is going on from the point of view of business
leaders and those recruiting future leaders.
Ideally this type of interview should be conducted face to face. However, both the
groups we have identified are ‘elite’ groups – they are difficult to get access to for
research; they are important people with busy schedules. We would therefore recommend interviewing them by video call at pre-arranged times. We would recommend that
you conduct a minimum of ten interviews (see below). Each interview will last for about
20–30 minutes. From experience, this is the maximum length of time many of the target
groups will be prepared to spend. It will be conducted by experienced researchers with
expertise in this kind of interview with this type of respondent. The interview will be
designed to cover key topics and issues but will be open-ended enough to allow the
researcher to explore relevant issues with the respondent. To encourage respondents
to take part we would advise offering them an incentive – either a thank you payment
to themselves or to a charity of their choice. It can also help secure an interview by
promising to send a copy of the published report to participants.
This exploratory stage should provide insight into and understanding of the key issues,
useful in their own right. It will also help further define and refine the objectives for the
second stage of the research, and it will provide key information for questionnaire
design. A summary of the key elements of this stage are given below.
Summary of Stage 1: Qualitative video call interviews (20 to 30 minutes)
Sample
●
●
CEOs and C-level HR executives of major brand owning organisations
Recruitment consultants (those recruiting senior/C-level executives to major brand
owning organisations)
Location
Ireland
UK
● France
● Germany
● The Netherlands or Sweden
●
●
159
M07 The Practice of Market Research 31362.indd 159
29/09/2021 16:08
Chapter 7
Writing a research proposal
Number of interviews
Ten in total:
●
●
One CEO or C-level HR executive per country (at least two CEOs overall)
One recruitment consultant per country.
Recording/transcription
●
●
Interviews will be recorded with permission
Recorded interviews will be translated (where necessary) and transcribed verbatim.
Analysis
●
●
Transcripts will be analysed and key issues and themes drawn out
Findings presented in a summary report and used as basis for survey questionnaire.
Stage 2: Quantitative telephone interviews (15 to 20 minutes)
The aim of the quantitative phase is to quantify the issues uncovered at the qualitative
stage – to help understand how widespread they are, whether they differ by industry
sector or by country.
The level of detail you want from the analysis, and the statistical precision and accuracy you require, are determined largely by sampling method and sample size. This is
something we would want to talk to you about in more detail. Both have an impact on
costs. For the purposes of costing this proposal we have assumed a sample size of
200. A sample of this size should deliver results that give you clear guidance on your
key issues. It will, however, restrict the level of detailed analysis you can do – for example, it would be difficult to report meaningfully on the findings from CEOs of French
organisations in the travel and transport sector.
We would recommend telephone interviewing as the most cost-effective and feasible
method of data collection among this group. To cover the issues in some detail we
believe will take 15 to 20 minutes.
A summary of the key elements of this stage is given below.
Summary of Stage 2: Quantitative telephone interviews
Number of interviews:
●
200
Sample
Respondents (N.B. Final split may depend on level of insight of each group as shown
in in-depth interviews):
CEOs
C-level HR directors
● Recruitment consultants.
●
●
Location and suggested sample size by country
Ireland and UK (50 interviews)
Sweden, Denmark and/or Norway (60)
● France (30)
● Germany (30)
● The Netherlands (30)
●
●
160
M07 The Practice of Market Research 31362.indd 160
29/09/2021 16:08
Evaluating a proposal
Top brand-owning companies in the following sectors (spread per country)
Financial services
Telecoms/electronics
● Consumer goods
● Retail
● Pharmaceuticals
● Travel and transport.
●
●
Length of interview
●
●
15–20 minutes maximum
Mostly closed questions apart from one final open ended.
Literature review
As we noted earlier, although we have not costed for it here we would strongly recommend a comprehensive review of the literature in this area. This would certainly be
helpful prior to the start of the primary research – before the qualitative work begins –
but it would be advisable before writing up the findings for any academic or business
journal. It would help locate the issue in its wider context.
Evaluating a proposal
When the client receives your proposal they will evaluate it, and probably compare it
against proposals from other researchers. They may ask to discuss the proposal with
you or they may invite you to present it to them. Make sure you are well prepared.
Before you send the proposal, put yourself in the shoes of the client and evaluate it
from their point of view. Below is a set of questions to guide you.
Business problem, research problem and research objectives
Has the researcher demonstrated a clear understanding of the business problem?
Has the researcher shown a clear understanding of the context of the problem and
the decisions to be made on the basis of the research?
● Has the researcher clearly identified the need for research, the key research objectives and the research questions?
● Has the researcher made any (incorrect) assumptions?
●
●
The research design
●
●
Will the research design or approach suggested deliver the right kind of evidence?
Has the researcher made a solid and credible case for the design suggested?
161
M07 The Practice of Market Research 31362.indd 161
29/09/2021 16:08
Chapter 7
Writing a research proposal
Has the researcher identified any limitations of this design? If so, have they
described how these will be mitigated?
● Will the data produced be credible?
● Has the researcher clearly and precisely identified the target population?
● Has the researcher made a plausible case for the proposed sampling strategy?
● Is the sampling strategy appropriate to the aims of the research?
● Is the method of data collection suitable?
● Has the researcher identified any limitations of this method? If so, how will these
be addressed or mitigated?
● Has the researcher addressed quality control issues?
● To what standard is fieldwork conducted?
● Is it clear how the data are to be analysed and presented? Is this approach suitable?
● Has the researcher suggested a framework for interpretation of the findings?
●
Ethical, legal and regulatory issues
Have all ethical, legal and regulatory issues been identified and dealt with
appropriately?
● Is the research to be conducted in a manner compliant with the relevant professional standards, for example The MRS Code of Conduct?
●
Timing and costs
Is the timetable suggested in line with your own and is it feasible?
Is the cost justified and is it clear where the costs arise?
● Is the project adequately resourced?
● Is it value for money?
● Have any/all contingencies been allowed for?
●
●
Experience and expertise
Has the researcher the right level and kind of experience and expertise to deliver
the research specified?
● Is the staffing suggested appropriate? Do the team members have the right amount
and type of experience and expertise?
●
Added value
●
Has the researcher added any value, provided useful insights, done more than you
expected?
Presentation
●
Is the proposal clearly set out, well written and easy to follow?
162
M07 The Practice of Market Research 31362.indd 162
29/09/2021 16:08
Chapter summary
What happens next?
Once you have sent the proposal to the client and the client has evaluated it, they
should contact you. They may have queries or they may want to discuss it further.
For example, it may be that they have concerns about an element of the approach or
about the budget. They may want to discuss with you what changes could be made,
for example, to ensure that the budget is not exceeded (a smaller sample size, for
example) without of course sacrificing or compromising the objectives. Depending
on the likely time involved in these post-proposal discussions, you may need to make
it clear to the client that you have already spent considerable time preparing the proposal and that while you are willing to do some further work, it must be reasonable
otherwise you will need to charge them for your time.
If the client has judged the proposal to meet their needs, if they believe it offers a
sound, effective, value for money solution, then they may commission you to do the
research. If they do, ask for confirmation in writing. Once you have this, contact the
client to tell them whether or not you agree to do the work (things may have changed
since you wrote the proposal or since they confirmed that they want to go ahead
and you may no longer be in a position to take on the work). If you are going ahead
with it, then the next step is to arrange a meeting. The first meeting about a project
is sometimes called a Project Start meeting. It gives all parties the chance to bring up
any issues of concern or interest, to talk through how the research will happen and
to clarify what each expects of the other. Any amendments to the original brief or the
proposal or subsequent research plan should be agreed and put in writing.
If your proposal has been unsuccessful the client should inform you of this as soon
as possible, though this does not always happen in practice. If you have not heard
from the client within a reasonable amount of time after the submission deadline
then you should contact them. Ask them why your proposal was not successful. This
will help you learn – about the market, about the client, about your own abilities,
about your ‘product’. It should give you information that may help you to address
any weaknesses and so improve the service you offer.
Chapter summary
●
The purpose of a research proposal is to show the client the following:
– that you understand the client’s business problem and its wider business
context;
– that you understand the need for research;
– that you are clear about how the research findings will be used by the client;
– that you have the experience and expertise to do the work.
●
The quality of a proposal relies to a great extent on the quality of the brief. You
should question the brief to ensure that you understand fully what is required.
163
M07 The Practice of Market Research 31362.indd 163
29/09/2021 16:08
Chapter 7
Writing a research proposal
●
A research proposal should include:
– an introduction to the client’s business problem and its wider setting;
– a statement of the information needs and the research objectives;
– the research design including sample and methods of data collection;
– a description of how the data will be analysed;
– the outputs (e.g. dashboard, report, presentation);
– a timetable and costs;
– ethical, legal and regulatory issues and how they will be handled;
– relevant previous experience and project team CVs;
– terms and conditions of business.
●
●
You should review the proposal before it goes to the client to ensure what is
described will deliver the information required.
Once the proposal has been accepted and the work commissioned, researcher
and client should meet to discuss in detail how the research and the working
relationship is to be conducted.
Exercises
1 Industry Insight 7.2 is a description of the research that was conducted in
response to the brief in Industry Insight 6.1. Review both 6.1 and 7.2. List what
was requested in 6.1 and from 7.2 state what was done to achieve that.
References
Bright Blue and Macmillan (2015) ‘Hidden at home: Establishing the hard facts to set the
social care agenda’, MRS Awards.
ESOMAR (2018) Global Prices Study, Amsterdam: ESOMAR.
Ipsos MORI UK and Bayer (2019) ‘The patient empathy workshop’, MRS Awards.
MRS (2019) Code of Conduct, London: MRS (http://www.mrs.org.uk).
Pyke, A. (2000) ‘It’s all in the brief’, Proceedings of the Market Research Society Conference,
London: MRS.
Recommended reading
Lock, L., Spirduso, W., and Silverman, S. (2013) Proposals That Work: A Guide for Planning
Dissertations and Grant Proposals, 6th edition.
Punch, K. (2016) Developing Effective Research Proposals, 3rd edition, London: Sage.
164
M07 The Practice of Market Research 31362.indd 164
29/09/2021 16:08
Part Three
Secondary or existing data
M08 The Practice of Market Research 31362.indd 165
27/09/2021 21:46
Chapter 8
Sources of existing data
Introduction
Secondary research is conducted using secondary or existing data – data that
were collected for another purpose. The re-purposing of data for use in a new
project means that they are being put to a second or ‘secondary’ use. In this
chapter we look at sources of existing data, including how they are stored.
Topics covered
Using existing data: the context
● Sources of existing data
● Data storage and retrieval systems.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 3: Selecting the research design and planning
the approach.
M08 The Practice of Market Research 31362.indd 166
27/09/2021 21:46
What you should get from this chapter
At the end of this chapter you should be able to:
understand why existing data might be of interest;
● demonstrate knowledge of sources of existing data;
● understand basic aspects of data storage.
●
167
M08 The Practice of Market Research 31362.indd 167
27/09/2021 21:46
Chapter 8
Sources of existing data
Using existing data: the context
The availability and accessibility of a vast array of sources of existing data offers
a wealth of opportunities for organisations to make use of that data in research to
address key business issues. Existing data can be small data, for example data gathered in a previous research study or data from external, public and ‘open’ sources,
or big data from internal and external sources. It may be machine-generated data
or human-generated data; it may be structured, semi-structured or unstructured, or
a combination. Regardless of how they were generated, if it is ethically and legally
appropriate, the data can be reviewed, queried, manipulated and/or re-analysed and
reported for a different purpose or different set of objectives than those for which
they were originally collected. In Industry Insight 8.1, Matt Taylor from Twitter
offers tips on how to make the most of existing research data.
At the simplest level, the re-use of existing data can take the form of a review
of the published top-line statistics about a product category, a market, a group of
Industry Insight 8.1
Getting more out of it
Upcycle your data
Most research studies are focused on a single
topic or problem, but almost all of them will
carry the same demographic questions each time.
That gives researchers a significant opportunity
to combine and re-use existing studies to identify
entirely new insights about demographic groups
(with GDPR and other relevant legal guidelines).
Here’s an example. At a previous company, our
Chief Marketing Officer (CMO) asked us to
evaluate a potential new target demographic
but with a budget of zero to commission new
research. We gathered all our recent studies, and
rather than look at them ‘vertically’ – through
the lenses of their individual objectives – we
looked at them ‘horizontally’. We took that
audience definition, recreated it from the demographic questions attached to all our surveys, and
looked to see what we could understand, more
broadly, about that audience across our different studies. Say you have a survey about how
video consumption is changing over time and
another about shopping habits. They’ll tell you
a lot about each individual topic, but why not
just look at 45–65-year-olds across both datasets
and see what you can learn about their lives more
broadly?
Learn to ‘question storm’
Data-heavy companies will often have all the
answers, but researchers need to ask the right questions to realise fully the benefit of that ocean of
information. Asking great questions is a core skill
for writing effective surveys – but building on that
to think of theories and questions to explore outside
the original scope of a dataset is a very valuable
capability in which any research team should invest.
Broaden access to research
So many of the best questions and hypotheses
will come from people outside the original stakeholder set. Invest in or build a process for allowing anyone to access research whenever they like
and promote it across your business. By ensuring
that anyone can find, browse and comment on
research, you maximise your chances of finding
new questions and angles through which you can
interrogate past data.
Source: Taylor, M. (2019) ‘How to avoid wasting research’,
Impact, 25, p. 73. Used with permission.
168
M08 The Practice of Market Research 31362.indd 168
27/09/2021 21:46
Using existing data: the context
people or a country; or it can be a review of the published reports of other researchers, including reviews of the scholarly literature on a theory or on previous research
studies on a topic. It can also involve further or ‘secondary’ analysis of an existing set
of data, in whole or in part. It may even involve the merging and analysis of several
different data sources, for example, previous research reports, sales data and social
media posts. We saw an example of this in Industry Insight 3.1. In that example,
Unilever used existing data from internal sources – previous research studies, notes
and company sales data – and data from external sources including social media, to
derive insights about fragrance for a household product. The use of existing small
data, typically from traditional research approaches, is usually referred to as secondary research. When the existing data are big data, it tends to be called data science
or data mining or data analytics.
Whatever the term, the important thing to remember is that the value or usefulness of existing data is rarely exhausted on their initial or primary application. Data
can be useful in the same context at a later date or useful in a different context. One
set of data may be combined with others – from very different sources – making the
combined set more valuable and of greater use than the individual elements. This is
what makes the use of existing data such a potentially fruitful approach. Industry
Insight 8.2 gives another example of how a large organisation used existing data from
multiple internal sources to generate insight.
Industry Insight 8.2
Energising with insight
Centrica is an international energy and services
company. Its main brand in the UK is British Gas.
There are 65 energy companies in the UK; British
Gas is the biggest. However, growing competition
means it has had to change direction to differentiate itself. As a result, its focus has switched from
customer acquisition to customer retention and
gaining greater value from more loyal customers.
Paul de Laat is head of data, insight and customer
value management. He and the data, insight and
customer value management team played a key
role in the shift. When de Laat joined Centrica in
2016 all profit and loss accounts (P&Ls) operated
separately – energy supply, boiler installation and
servicing, insurance, and so on. These product and
service areas were moved into a single P&L to
remove duplication, including duplication of data
and insight. De Laat says, ‘My role was to bring
all the data together from across the business to
create a single view of the facts and a single view
of the customer across all products and channels.’
De Laat’s team is now at the centre, ‘feeding . . . the organisation with data, management
information and insight’. The data strategy and
operations part of the team pulls together data
from multiple sources to understand, at household level, what products and services are being
used. This includes names and addresses captured
through products and services, and rough information on the state of equipment in the house,
based on installation dates and service engineers’
reports. This is combined with socio-economic
profile data, based on where people live. Combining these allows the team to do three things:
1 Give British Gas management information,
such as metrics on the number of people who
visit the website and then call the contact
centre.
2 Do ‘deep dive’ analytics to understand why
customers buy the products; they can profile groups of similar customers, and look at
169
M08 The Practice of Market Research 31362.indd 169
27/09/2021 21:46
Chapter 8
Sources of existing data
differences between groups, to create more
meaningful interactions with them.
3 Develop propositions and products that will
better appeal to different groups of customers.
‘What are the unmet needs of our customer
base? Which products aren’t doing the right job,
and what could we change – could we bundle
products together to help customers understand
their value better?’ asks de Laat. Products include
‘intelligent home’ products such as smart meters,
smart thermostats, smart appliances and lights
marketed under the British Gas sub-brand, Hive.
Source: Simms, J. (2019) ‘Energising the market’, Impact, 26,
pp. 47–50. Used with permission.
Making use of existing sources has the added benefit that, since there is no need to
generate data, it can be a relatively time-efficient and cost-effective approach.
A key part of any project is to find out what data, if any, already exist that might
help you. This is the focus of this chapter.
Sources of existing data
There is a wide variety of sources of existing data. A useful way of classifying
the sources is whether they arise from within the organisation – internal sources
such as data from previous research studies, data from across the operation of the
­organisation – or data from sources external to the organisation. We’re going to look
first of all at external sources.
External sources
External sources tend to be put into one of two categories: official sources and unofficial
sources. Those produced by government departments, its agencies and related bodies,
including national statistics organisations, are the official sources and the output
referred to as official statistics. Those produced by other public- and private-sector
organisations – unofficial sources – are unofficial statistics. Upadhyaya (2016) notes
that there is ‘no common understanding . . . about the distinction between official and
unofficial sources’. Statistics reported by international organisations use official sources
as their primary source – those provided by national statistical organisations or ‘other
sources of authoritative data’ (Durand and Gennari, 2016). Durand and Gennari note
that there are ‘exceptional cases’ where non-official statistics are used, for example,
‘to fill gaps’ or ‘ensure comparability’. Sources are chosen ‘on the basis of professional
independence, scientific methods and impartiality’. A body called the Committee for the
Coordination of Statistical Activities (CCSA) publishes recommendations on the use of
non-official sources for these statistics and its members adhere to a set of principles ‘in
order to provide high quality and comparable international statistics’ for ‘policy needs’.
There is a huge number of unofficial sources. As Our World In Data
(www.­ourworldindata.org) notes:
The web allows us to publish in a way that was unimaginable just a few years ago: distribution is free and research and data can be explored through interactive documents.
We look at examples of official and unofficial sources below.
170
M08 The Practice of Market Research 31362.indd 170
27/09/2021 21:46
Sources of existing data
Government published data
Governments and related bodies collect a wide range of social, economic and business data, from the Census of Population and the demographic characteristics of
the population through their spending habits, lifestyle and attitudes, to information
about different market sectors, from agriculture to tourism, and information on
domestic and international trade and key economic indicators. The quality and usefulness of government-produced data should be assessed in the same way and with
the same rigour as data from any other source. We look at how to evaluate sources
in the next chapter.
Box 8.1
Examples: data from governments and related bodies
In the United Kingdom, the Government Statistical Service (https://gss.civilservice.
gov.uk), ‘a community for all civil servants working in the collection, production
and communication of official statistics’ is a cross-government network led by the
National Statistician. It works ‘to provide advice, analysis and a statistical evidence
base to help people make better decisions. Within the GSS is the Office for National
Statistics (https://www.ons.gov.uk/). It is the ‘UK’s largest independent producer of
official statistics and the recognized national statistical institute of the UK’.
● Through the UK Data Service (https://www.ukdataservice.ac.uk/) you can get access
to thousands of data collections, qualitative and quantitative, as well as administrative and business data collected by the government, for secondary research
purposes as well as access to publications on business, industry and trade; the
economy; employment and labour market; and people, population and community.
● You can get a fairly comprehensive list of government statistical services at the
University of Michigan website (http://www.lib.umich.edu/).
● In Ireland the body responsible is the Central Statistics Office (https://www.cso.ie/en/)
● In Australia it is the Australian Bureau of Statistics (https://www.abs.gov.au/)
● In the United States it is the US Census Bureau (https://www.census.gov/)
● In the European Union it is Eurostat (https://ec.europa.eu/eurostat/).
● Government departments for trade and foreign affairs and the embassies of foreign
governments are useful sources of data on international business environments,
providing information on political, legal, economic and cultural aspects of doing
business or research.
● Other sources of international data include the following:
●
– United Nations (www.un.org), which has a wide range of links to online catalogues, bibliographic databases and directories relating to social, economic and
market data – for example, UNIDO (https://www.unido.org);
– World Bank (https://data.worldbank.org);
– World Trade Organization (https://www.wto.org/);
– OECD (https://www.oecd.org);
– World Health Organization (www.who.int); and
– World Economic Forum (https://www.wef.org).
171
M08 The Practice of Market Research 31362.indd 171
27/09/2021 21:46
Chapter 8
Sources of existing data
Open data
There is a category of data known as ‘open data’, data that are ‘open’ for anyone to
access, use and re-use. It includes mostly government data but also non-government
data. The Open Knowledge Foundation’s ‘Open Definition’ (opendefinition.org)
summarises them as: ‘Open data and content can be freely used, modified, and shared
by anyone for any purpose’. Several other bodies have set out what they believe the
characteristics of open data should be. The organisation opengovdata.org sets out
eight principles of open government data as follows:
complete;
primary;
● timely;
● accessible;
● machine processable;
● non-discriminatory;
● non-proprietary; and
● licence free.
●
●
The Organisation for Economic Co-operation and Development (OECD) reports on
open government data policies in its member and partner countries. It reviews data availability, data accessibility and government support for data re-use (OECD, 2020). Using
Google Public Data Explorer you can get access to and explore a wide range of public
datasets including those from international organisations such the World Bank and the
OECD, national statistics offices, research institutions and non-governmental bodies.
Non-government published data
Sources of non-government research, data and information abound. Producers
include universities, research institutes and foundations, think tanks, regulatory bodies, and trade and professional associations. Much of the material can be tracked
down via the source organisation’s website, via specialist information host sites that
list catalogues, directories, guides and databases, via portals and from data archives.
Box 8.2
Examples: open data sources
Here are some examples of sources of open data:
World Bank Open Data
United Nations (e.g. the UNICEF Dataset)
● World Health Organization Open Data Repository
● European Union Open Data Portal
● Open Data Institute
● data.gov.uk, the hub for open data in the UK
● Datahub.io
● Datahub Canada
● US Census Bureau
● data.gov, the US government’s open data.
●
●
172
M08 The Practice of Market Research 31362.indd 172
27/09/2021 21:46
Sources of existing data
Box 8.3
Examples: non-government data sources
Data portal: through the Council of European Social Science Data Archives
(CESSDA) portal you can access the metadata records for European social science datasets.
● Data service: through the UK Data Service you can get access to social science
and humanities research datasets funded by the Economic and Social Research
Council.
● Data archives: through the UK Data Service you can also get access to data archives
across Europe, North America, South Africa, Australia and New Zealand, India,
Japan, South Korea, and Taiwan. You can search the catalogues of these archives
and the UK Data Service can help you get hold of the data.
● Registry of data repositories: The registry re3data.org lists hundreds of data repositories worldwide.
● Our World In Data (www.ourworldindata.org): a source of data on poverty, disease,
hunger, climate change, war and inequality based at the University of Oxford and
designed to generate impact and ‘empower colleagues in policy, media and civil
society’. Data are available for download and visualisations and text are licensed
under CC BY.
●
There are many commercial organisations which supply existing market research data
and market and business intelligence. They sell standard packages as well as tailored
ones on a range of topics.
Industry Insight 8.3 gives an example of the use of existing sources of data, data
from the OECD and data published by the market intelligence agency, Mintel, to
present an overview of a market.
Box 8.4
Examples: business intelligence data sources
GlobalData (https://www.globaldata.com)
Mintel (https://www.mintel.com/)
● Informa (https://informa.com/)
● Kompass (https://www.kompassinfo.co.uk/)
● Forrester (https:/goforrester.com)
● ClickZ (https://www.clickz.com/)
● eMarketer (https://www.emarketer.com/)
● Through the Financial Times website (https://ft.com) and business websites such as
http://www.business.com/ and Dun & Bradstreet https://www.dnb.co.uk/ you can
access a wide range of information on organisations, markets, industry sectors and
countries.
●
●
173
M08 The Practice of Market Research 31362.indd 173
27/09/2021 21:46
Chapter 8
Sources of existing data
Industry Insight 8.3
Plant power
Australia is one of the world’s largest meat-eating
populations. According to 2018 data from the
OECD, an average Australian consumes close to
95kg of meat per year. However, for health and ethical reasons – as well as the rise of ‘­flexitarianism’
– meat consumption habits are changing across the
country. Mintel research shows that almost onefifth of urban Australians avoided, or were intending to avoid, red meat in 2017 and one half of them
stated that they believed it is healthier to do so.
Consumer movement into plant-based foods aligns
with Mintel’s 2017 global food and drink trend,
‘Power to the Plants’, which describes how people’s
preference for natural, simple and flexible diets is
driving expansion of vegetarian, vegan and other
plant-focused innovation across multiple product
categories. When it comes to product innovation,
Mintel’s Global New Product Database shows that
food and drink launches featuring the ‘vegan/no
animal ingredients’ claim accounted for 2 per cent
of all Australian food and drink launches in 2013;
in 2017 it has risen to 5 per cent. According to
Mintel research, more than one in three urban Australians prefer to buy products that are produced
using sustainable sourcing methods, while almost
one third are willing to pay a premium for everyday goods that are safe to use – for example, have
no additives, or do no harm to the body. This is
also reflected in Mintel’s trend ‘Buydeology’, which
underscores the need for meat-alternative manufacturers to pay close attention to ingredient purity –
as well as to sourcing ethics – when catering to
­consumers who are increasingly basing their purchasing decisions on a company’s ethical stance.
Source: Adapted from McMillan, S. (2018) ‘Plant power Down
Under’, Impact, 22, pp. 12–13. Used with permission.
Box 8.5
Examples: sources of academic and scholarly material
Jisc Library Hub Discover which offers access to details of materials held in many UK
and Ireland national, academic and specialist libraries (https://discover.libraryhub.
jisc.ac.uk/)
● Directory of Open Access Journals (https://doaj.org)
● Google Scholar (https://scholar.google.com)
● Microsoft Academic (https://academic.microsoft.com)
● Bielefeld Academic Search Engine or BASE (https://www.base-search.net)
● CORE (https://core.ac.uk)
● Semantic Scholar (https://www.semanticscholar.org)
● Baidu Scholar (https://www.baidu.com).
●
Consumer-generated content or data
Consumer-generated content – that is, material published in online discussion forums,
blogs, microblogs, content communities, virtual worlds and other social media – is in
174
M08 The Practice of Market Research 31362.indd 174
27/09/2021 21:46
Sources of existing data
effect published data. This type of material is also known as user-generated content
(UGC) or user-created content, or volunteered data (Kitchin, 2014). It is content
produced by consumers and users that is publicly available. It can be in any format – text, audio, images or video. It gives marketers and other interested parties
direct access to the consumer. It contains information about the users who generated
it, including locations and networks of contacts, as well as attitudes, opinions and
preferences that relate to organisations, products and services. For researchers it can
be a less expensive way of getting insight than traditional qualitative or quantitative
research. Vriens et al. (2019) describe it is a ‘valuable and viable source of consumer
insights’. They note its benefits as an alternative to surveys–the fact that the data
are time-stamped (allowing the evaluation of the impact of a campaign or event);
that it is ‘easy to zoom in and out’; and that it facilitates the study of a large set of
brands. There are ethical, legal and regulatory issues, of course, and it may be that
these eventually restrict the use of user-generated content for research, and with the
use of blockers consumers themselves and the browsers they use may also restrict
data collection (Nunan, 2020). The use of UGC also has other limitations, including lack of wider context to enable understanding of the reason for the opinion or
preference and the ability to distinguish genuine consumers from those pretending
to be genuine consumers.
Many organisations use these data (where ethically and legally appropriate) for
research purposes. The type of research includes social media monitoring or social
media listening (from automated monitoring to ad hoc research), social media mining, crawling, extracting, gathering or ‘scraping’ content (manually or using automated processes) from sites and platforms, mining it and analysing it. When the aim
is to find out what people are saying about your company or your brand, say, or a
topic relevant to your organisation, it is sometimes referred to as ‘buzz monitoring’
and the search for and analysis of those conversations as ‘buzz mining’ and opinion
mining and sentiment analysis. It is useful in identifying issues that are being talked
about (the content), perceptions consumers hold, reactions to events, and views and
opinions being expressed and the way in which they are expressed. It is useful for
uncovering issues or concerns about a product or service, for getting an idea of a
brand’s image or reputation, for tracking the impact of advertising and other marketing activity and for gathering intelligence about competitors. Data mining and data
analytics techniques including machine learning can be used to identify patterns and
to cluster and segment consumers.
Organisations can gather consumer-generated content themselves. For example,
there are platforms that enable you to access and use public Twitter data; to gather
tweets from Twitter users; to download your own tweets; to track uses of key words
or hashtags in real time across Twitter; to connect to Twitter data streams to gather
data; and to search past tweets. Alternatively, you can buy data from authorised data
brokers or resellers.
Industry Insight 8.4 gives an example of the use of social media research.
Mobile network Three and researchers at Mindshare UK analysed social media
data to get a more rounded view of Three’s target audience. The findings were
used to make changes to website content, and to modify brand positioning and
pricing strategy.
175
M08 The Practice of Market Research 31362.indd 175
27/09/2021 21:46
Chapter 8
Sources of existing data
Industry Insight 8.4
What do you think of Three?
The brief
Mobile network operator ‘Three’ wanted to
understand better a segment of the market where
it had previously underperformed versus its competitors. Three had formed a hypothesis that its
brand had a reputation for being ‘cheap for a reason’ amongst this target segment and that this was
limiting customer acquisitions. However, this attitude was not apparent in the findings from brand
trackers and other ‘traditional’ forms of measurement. So, on the basis of this hypothesis, the company put the following three key questions to the
research agency, Mindshare UK:
How do people talk about Three?
What is Three associated with?
● What are the kinds of questions associated with
Three?
●
●
The insights generated by the research would be
used to inform the next phase of research and
strategies relating to target customer, brand, pricing and hardware.
The research
Mindshare suggested using a combination of social
listening and search data to uncover consumer
attitudes towards the Three brand that the surveys weren’t revealing. Due to its large scale (high
number of mentions), and unprompted nature,
analysis of social conversation can reveal deeply
held consumer attitudes and biases in a way that
questionnaires and traditional brand trackers have
been unable to do. Tools used for this analysis
included Brandwatch, Crimson Hexagon, Hitwise
and some bespoke Mindshare analytics tools.
Mindshare gathered a dataset of brand mentions
of ‘Three’ stretching back five years. First, they
analysed the mentions for sentiment, which gave
a broad indication of general attitudes towards
the brand. Next, they layered on emotional analysis and segmentation. After that they analysed
the mentions to examine the vocabulary used
to describe brands. Finally, they segmented the
conversation into six key themes: ‘Network Quality’, ‘Data’, ‘Upgrades’, ‘Roaming’, ‘Customer Service’ and ‘Pricing’. This enabled the researchers to
understand which topics were most frequently discussed in relation to Three. To provide context to
the analysis, they did the same for Three’s key competitive set – O2, Vodafone and EE. The analysis
provided extensive insight into the key hypothesis.
However, the agency felt that the question merited
further exploration and corroboration and so they
decided to look at search behaviour. They wanted
to add to the social findings by digging deeper and
analysing behaviour in relation to the brand at a
layer below active conversation. They did this using
Google data. They mapped associated searches to
Three and competitors – these are searches made in
the same session as brand searches. Mapping these
associations revealed the mindset of the audience
searching for the brand. They also examined website traffic data around Three and competitors. First,
they looked at key websites visited when ‘Three’ or
competitor brands were searched for. Second, they
analysed upstream and downstream traffic from
the Three and competitor website homepages, to
gain insight into the consumer journey. The obvious challenge came from creating a Boolean search
string that effectively scraped mentions of ‘Three’
the brand, without including irrelevant content.
They addressed this in two ways:
Accuracy: They constructed a complex query
string that pulled in relevant ‘Three’ brand
mentions, excluding irrelevant terms, and adding in qualifying terms such as ‘network’, handsets etc.
● Consistency: When gathering datasets for competitors with less ambiguous names, they used
the same approach so that they could make a
fair comparison between brand conversations.
To do this they made sure that limitations
around searches for ‘Three’, such as limiting
brand mentions to those also including handset
or network comments, were applied across the
board to all competitors.
●
176
M08 The Practice of Market Research 31362.indd 176
27/09/2021 21:46
Sources of existing data
Use of social data in research runs a risk of presenting a polarised view – with people actively
posting on social media when particularly angry
or happy. The steps the agency took to mitigate
this included:
Benchmarking: It was imperative to compare Three’s results to those of competitor
brands. Any risk of exaggerated results is
avoided by a shift in emphasis between absolute results, to comparative performance
between brands.
● Inclusion of other datasets: The agency takes
the view that the use of multiple data sources
and research techniques yields the best results.
For this project they felt that combining the
analysis of active conversation with more passive search behaviour would approach the
brief questions from different, complementary
angles of consumer behaviour.
●
By segmenting conversation into topic areas, and
performing more complex linguistic analysis, the
agency gained richer and more actionable insights.
Corroborating social findings with search behaviour worked well to provide a more holistic view
of the audience – a combination of opinion and
behaviour. This was particularly useful for Three
as it allowed them to show how certain brand
perceptions translate into different behaviour for
Three vs. its competitors.
The findings
The findings did not prove the ‘cheapness’ hypothesis as expected: negative pricing-related conversation
was no more prominent for Three than for competitors. This was validated in a subsequent phase
of quantitative research where no strong implicit
or explicit associations of ‘cheapness’ were attributed to Three by its target segment. The research
showed that Three is less discussed than competitors, and overall less associated with either negative or positive adjectives. Three was less considered
than competitor brands, driving less strong feelings.
This indicated lack of brand loyalty was corroborated with upstream and downstream website traffic, which contained a much higher proportion of
independent forums and price comparison websites
than for EE and O2, where customers went straight
to these pages and didn’t compare operators.
The business impact
The research validated some of the Three team’s
long-held suspicions about the brand’s reputation, showing how attitudes people have towards
a brand manifest themselves clearly in social and
search behaviour. The findings gave the team proof
points around its brand reputation compared to
competitors, and this helped push through change.
Source: Adapted from Mindshare UK and Three, ‘Three brand
reputation analysis: Using social conversations and search insight
to uncover implicit attitudes towards Three Mobile Network’,
Winner, MRS Awards 2018. Used with permission.
External sources
● Websites
● Statistical services
● Data services
● Data portals and hubs
● Archives, repositories and libraries
● Social media
Internal sources
● Data lakes, databases, data warehouses
● Insight management or business intelligence system
● CRM system
● Enterprise intelligence system
Figure 8.1 Where to look for existing data
177
M08 The Practice of Market Research 31362.indd 177
27/09/2021 21:46
Chapter 8
Sources of existing data
Internal sources
With the continued rise in the number and use of digital devices, the integration and
networking of devices, including the Internet of Things (IoT), the increase in computing power and the availability of vast amounts of relatively cheap storage, lots of data
are not only generated but they are processed, stored and analysed by organisations
of all types. The data collected are used in core functions to understand and manage the operation of the organisation, its products and/or services, and its users and
stakeholders, in an efficient and effective way. A lot of these data are the sort that
marketers and advertisers, in particular, find useful.
A lot of these data are gathered automatically without the user necessarily knowing
that data gathering is happening; and some data are volunteered, including personal
data when registering for a loyalty card or as a customer or member on a website
or downloading apps. The data collected are aggregated, stored and processed in
data lakes, databases, data warehouses or data management platforms (DMP) from
which they can be retrieved for analysis. These storage and retrieval systems can be
designed to function as enterprise intelligence systems (EIS), management information
systems (MIS) or business intelligence systems (BIS) for planning and control, strategy
development, for example. They can be set up to serve more specific functions such as
marketing (marketing information systems or MkIS) or customer relationship management (CRM systems) or research and insight management. Before we move on
to look at these storage and retrieval systems it is worth looking at how some of the
data particularly relevant in a market research context are gathered from customer
transactions and interactions.
Data from loyalty cards and customer databases
We mentioned ‘loyalty’ cards above. Loyalty cards, sometimes called ‘reward’
or ‘club’ cards, are used by many companies to link personal data with buying
behaviour at the level of the individual customer. It works like this: you apply
for the card to benefit from the organisation’s promotion schemes; when you
apply you give the organisation – a retailer or an airline, for example – your
Box 8.6
Examples: sources of internal digital data
Sensor data: data from sensors embedded in products which allow them to be
tracked from point of origin to consumer
● Scanner data: data from scanners such as electronic point of sale (EPOS) devices
which scan machine readable codes on products and forms
● Interaction data: data interactions and/or communications, for example from cookies
in websites, weblogs, information in email headers, and information on time, date,
duration and location of phone calls
● Transaction data: data from in-person and online transactions.
●
178
M08 The Practice of Market Research 31362.indd 178
27/09/2021 21:46
Sources of existing data
personal details; each time you make a transaction and have your ‘loyalty’ card
scanned, the personal details from the card are recorded and logged against that
transaction and so against the purchases you made. Your individual record can
be updated with these purchases. The organisation has a record of your actual
buying behaviour (that is, not your claimed buying behaviour – which is the sort
of data that is recorded in a survey) in your personal record in its database. The
same process operates in online retail. You register to use the site and your activity on it is recorded and stored.
Your purchasing patterns can be analysed (for example, in a ‘shopping basket
analysis’) and, on the basis of this past behaviour, the retailer can send you recommendations and alert you to offers on the types of items or brands that you buy.
The retailer can also compare your purchase record with that of other customers
and, where they detect similarities (say in the purchase of X), notify you, telling you
that ‘people who bought X also bought Y and Z’. ‘Shopping basket analysis’ can
show what sets of products or brands are bought together, and which groups or
segments of their customers buy which sets of products or brands, allowing them to
target groups of customers with tailored offers, for example, and they can see from
the database the uptake on these offers. Further, by examining trends in behaviour
over time, the company can build models and algorithms to predict behaviour, sales
volumes and revenue. This information can be used to understand, for example, how
profitable different groups of customers or different types of outlet are, and what
type of promotion works best for which group.
Data derived from loyalty cards, however, can be limited. While they give information about customer behaviour in the store, they do not give information on
behaviour outside it (for which data from consumer panels may be useful); the
demographic information provided may not in all cases be accurate; and people may
hold more than one card for the same store (Passingham, 1998). Also, the customer
may not use the card for every transaction. In addition, loyalty card data cannot be
used to build a full picture of a store’s or a vendor’s customer base as some customers may refuse the offer of a card. It is a different matter when it comes to online
buying activity, which does not rely on the loyalty card method although many sites
request registration.
Data from customer interactions can be a source for secondary research, providing
there is a lawful basis under data protection legislation for processing or using the
data. The data can provide detailed current and historic information about actual
(observed) rather than the reported buyer behaviour on which primary research such
as a survey might report. In market research terms, the sort of questions these data are
helpful in addressing are questions related to understanding customers, identifying
emerging patterns and trends, planning and designing promotions and advertising
campaigns, and developing new products, to name a few.
Industry Insight 8.5, although conducted many years ago, shows how the analysis
of basic customer information recorded at the box offices of arts venues provided
greater insight into customer behaviour than did customer surveys at the venues. It
shows, too, how data can be shared and used for decision making by several parties
(when there is permission to do so). We looked briefly at this example in Industry
Insight 3.4 Ticketing talks; here we look in more detail.
179
M08 The Practice of Market Research 31362.indd 179
27/09/2021 21:46
Chapter 8
Sources of existing data
Industry Insight 8.5
‘I know what you did last summer’
Background
Since the development of box office software, performing arts venues in London have been accumulating enormous databases of complex and
increasingly high-quality data on their customers.
When a user-friendly software package called data:
crunch came along it allowed users to produce
sophisticated audience profiles and trend information quickly and easily from any box office software
package. Here it was used in a project called Snapshot London to provide a picture of audiences and
their behaviour at a number of arts organisations.
The project plan
This is what we aimed to do:
To include up to 20 organisations from Central London, where a reasonably representative
sample could be covered.
● To concentrate on those with the best and largest databases in order to make best use of the
resources we had available.
● To examine five years’ worth of box office data
from participating organisations.
● To give each participating organisation benchmark data for London as well as its own data
within the data: crunch software, enabling each
to carry out its own analysis.
●
A confidentiality agreement guaranteed that the
research organisation would only present publicly
results regarding the overall benchmark and not
any figures which pointed directly or indirectly at
individual organisations. Any reference to a venue
by name is made with that venue’s permission.
Coding issues
An issue that had been an obstacle to data-pooling
in the past was that of artform coding: those
working within venues needed to work with subtle categorisations of their product whereas, for an
umbrella study, higher-level categorisations were
required. This was solved by allowing each organisation to use whatever product categorisation
they liked for analysing their own data, but the
benchmark database was coded at a higher level
of categorisation which roughly matched the categories used elsewhere. The product coding took
place at two levels – Artform and Style.
Data capture issues
For some artforms we had a good or excellent representation or coverage of venues – and we found
that their rate of data capture was good. For other
artforms (e.g. Cinema or Visual Arts) coverage of
venues was poor and we found that these venues
had very little data capture. This is important in
making any interpretation of the results; we have
tended to issue figures for which we have at least
good coverage, giving careful qualification elsewhere. There are drawbacks in working with box
office data:
Not all ticket sales are sold with name and
address capture. The venues taking part here all
had very high rates, often exceeding 90 per cent
and in many cases 95 per cent. Many sales
without capture are those taken just before a
performance.
● Only the buyer’s details are captured for an
average of just over two tickets sold per transaction. We have allowed for this as far as possible
in that the benchmark data has been analysed
in most cases at household level. However, it
has to be accepted that not all attenders are
captured and customer records will tend to
under-represent their frequency of attendance.
● Not all venues were open for the entire five-year
period (1998–2002) and they were removed
from any analysis of trend information over the
entire period. (However, this enabled us to find
out what happens to a venue’s audience when
it closes.)
●
Initial key findings
We thought we would find a very small pool
of arts audiences who attended very widely and
180
M08 The Practice of Market Research 31362.indd 180
27/09/2021 21:46
Sources of existing data
frequently. In fact, the reverse proved to be the
case. The pool of arts attenders upon which venues draw is very large – we found that over the
five-year period, some 2 million households had
attended one of the participating venues, 7 per cent
of all households in Britain. Within London,
30 per cent of households had attended. And the
geographic reach was broad – the few postal sectors that had no customers were in the Highlands
of Scotland.
Venue crossover
Audiences in London are not skipping between
venues with the fleetness of foot sometimes
assumed. Most households (73 per cent) attend
only one venue; 13 per cent attend two; and
13 per cent attend three or more. It seems that
the more venues and performances that exist, the
more audiences are drawn.
Artform crossover
The percentage of households attending only one
artform over five years is 69 per cent, slightly
lower than the equivalent figure for venues. In
other words, people are more likely to attend
another artform at the same venue than they are
to attend another venue. Nevertheless, the specificity of audiences’ tastes has been a surprise, challenging long-held assumptions that the general
public share the eclectic cultural habits of the arts
professional and the keen attenders who respond
to questionnaires.
Demographic and geographic distribution
There is a general perception of arts audiences as
ageing and wealthy. We found that the proportion
of younger arts attenders matches the population
very closely. In terms of household income, arts
audiences are wealthier than the population. From
a marketing planning perspective, it is useful to
analyse the relationship between the price people
pay for their tickets and the distance they travel,
which established that the further people travel, the
more they pay, the larger their party-size is, and the
more likely they are to return. So, for example, not
bothering to mail previous attenders who live far
away from your venue is a false economy.
Source: Adapted from Brook, O. (2004) ‘“I know what you did
last summer” – arts audiences In London 1998–2002’, MRS
Conference, www.mrs.org.uk. Used with permission.
Data from cookies and trackers
Your activity on a website – across many websites – can be tracked, recorded and
stored with or without you registering personal information. Tracking your online
behaviour can be done in several ways:
by storing cookies in your device;
by logging your IP (internet protocol) address (a unique number that identifies
your device);
● by registering and recording the ‘web bugs’ or beacons you trigger in the sites you
visit;
● by using browser fingerprinting; and
● by accessing your browsing history.
●
●
The next time you visit a new site, rather than click ‘Accept All’ for the cookies and
trackers the organisation uses, hit ‘Manage Preferences’, and have a look at what you
are allowing the site to track and monitor. Storing cookies – bits of data – in your
device means that the website has a unique identifier for you (an ID cookie) so that
when you visit again you are recognised. A cookie from one website may also allow
you to be recognised on another, related website (or one that is monitored by the same
company). Some cookies (‘third party’ cookies used by advertisers) can track your
181
M08 The Practice of Market Research 31362.indd 181
27/09/2021 21:46
Chapter 8
Sources of existing data
activity on multiple websites and thus they can build a profile of your browsing activity. Your IP address is the address to which website content is sent. As soon as you go
to a website, it sees your IP address. Unless your IP address changes or is hidden, the
website can monitor all activity from that address across multiple devices. It can also
use the IP address to find your geographic location. ‘Web bugs’ are bits of code in a
website’s HTML that are used to track who views the web page, when and from what
IP address. They are a means of following you from one website to another within a
group or network of sites. If a site you visit uses browser fingerprinting it can gather
enough information about your browser and your actions to build a profile, including
information about the http headers from the sites you visit; information about what
time zone you are in; and what your browser platform is, your screen resolution,
your system fonts, any plugins you have, even the configuration of your hardware.
Using all of this information, it can distinguish you from other visitors to the website
and it can store this information. Websites can also use program code to look at the
browsing history on your device. Once they have it, they can use it as the basis for
categorising or profiling you, and they can use that information to ‘personalise’ the
websites you visit.
Geodemographic information systems
As Industry Insight 8.5 shows, customer data can be analysed to identify behaviour
and sales patterns by different outlet types and by different regions and patterns of
buying behaviour. Analysis can also reveal the characteristics, demographic or geodemographic, that are associated with different behaviour patterns. These patterns and
characteristics can be used to build profiles of customers and outlets, and to identify
market segments and gaps in the market. Demographic data and geographic data are
combined into geodemographic data (sometimes called ‘geodems’) and the combined
data are used to build geodemographic databases or geodemographic information systems (GIS). These systems can be expanded to include financial data and lifestyle data.
Such databases tend to be produced by commercial organisations. They are widely used
for marketing purposes, primarily to identify and target different types of consumers.
The basis of most geodemographic information systems is more or less the same:
they relate the demographic characteristics of the residential population, derived from
the Census at the smallest geographical unit within the Census for which data are
available, to geographic information about that area. The sources of information used
to construct the system and build categories and profiles may also include the electoral
register, postcode address files, car registration information, credit rating data, data
from surveys on media use or attitudes, and data from customer databases. The end
product is a classification of neigbourhoods or areas within which people with certain
characteristics live. This classification or segmentation of neighbourhoods is based on
a cluster analysis. Each ‘cluster’ or type of neighbourhood will be different from every
other cluster or neighbourhood – because the type of people – the type of consumers –
living in that neighbourhood will be different. A neighbourhood in one town may be
classified as belonging to the same cluster or type of neighbourhood in another town,
because the characteristics of people in that neighbourhood are the same or similar.
Since they are rooted in a geographic location, knowing a person’s postcode or
postal address is enough to allow you to assign them to a particular geodemographic
group. As a result, each record held on a customer database can be assigned to a
geodemographic cluster; if you know the area from which a sample, sub-sample
182
M08 The Practice of Market Research 31362.indd 182
27/09/2021 21:46
Sources of existing data
or research participant was drawn, individual cases from a survey can be assigned
to a geodemographic cluster. This means that data from different sources – from
a customer database or from any type of survey – can be analysed in terms of its
geodemographic profile. Thus data from a survey on buying behaviour, survey data
on attitudes and values, and data from a customer database can be linked – the geodemographic classification of each unit or case being the common variable for fusing or
integrating the data. The database created by this fusion allows us to examine relationships between different types of consumers, their attitudes and their behaviour.
Using geodemographic information can help organisations to gain a more ­in-depth
understanding of their customers’ habits, preferences, attitudes and opinions. This
information can be used to develop strategy and to target products, services and
marketing communication more effectively. Applying geodemographic codes to, say,
existing sales data on customers derived from loyalty cards will help give you information on their demographic and lifestyle characteristics. If you know the geodemographic profile of your customers, you can use a GIS to find where other people with
similar profiles are located. This information can be useful in planning the location
of a business, deciding where to locate a store, what type of store it should be, what
size, what product mix it should contain and so on. Having this information upfront
lowers the risk in decision making and can help maximise a business’s potential.
Geodemographic profiling can also help in targeting marketing and advertising campaigns, helping marketers choose which neighbourhoods to target.
‘Off the shelf’ geodemographic information systems are available for most E
­ uropean
countries as well as for the United States, Canada and Australia, ­New Zealand,
Hong Kong and major cities in China. Several commercial organisations specialise
in providing them, including CACI, which produces a product called ACORN –
A ­Classification of Residential Neighbourhoods – and Experian, which produces
MOSAIC. CACI’s ACORN system divides the population of the UK into 62 types,
which are grouped together into 18 groups spread across six categories.
We saw in Chapter 1 that one of the main uses of research is to reduce risk,
thus helping decision makers arrive at cost-effective solutions to their organisation’s
problems. Industry Insight 8.6 shows how geodemographic data – readily available,
easy to access and straightforward to use – can be used to this end. Here it was used
to provide information that helped in the decision about where to site a retail store.
Industry Insight 8.6
Which site?
Geodemographics were first used by retailers as
a means of taking some of the risk out of major
investment decisions involved in new store locations. In the more mature retail sectors such as
grocery, clothing, CTNs (Confectioners, Tobacconists and Newsagents), petrol retailing and so
on, sectors where there is a requirement to target to consumer demand, ACORN and Pinpoint
(geodemographic classification and analysis systems) can provide retailers with expenditure estimates for a merchandise sector or site, thereby
placing a monetary value on local markets.
Typically, a site location and a definition of
its catchment area are provided by the retailer
in terms of either postcode sectors or radius of
x kilometres from the site location. The output
183
M08 The Practice of Market Research 31362.indd 183
27/09/2021 21:46
Chapter 8
Sources of existing data
generated by services such as ACORN and Pinpoint compares the population and/or household
profile of the area with some or all of:
The national (GB) average
The retailer’s national profile
● The retailer’s ‘target’ profile if this has previously been identified.
●
●
In addition, it can provide for the location:
●
●
population/household numbers
£ value of the defined market spend available
from the catchment.
When referenced against existing stores or the
chain profile this provides an indicator of the site
potential.
Source: Adapted from Johnson, M. (1997) ‘The application of
geodemographics to retailing: meeting the needs of the catchment’,
International Journal of Market Research, 39, 1, pp. 201–24.
Data storage and retrieval systems
Wherever they are from and however they were generated or captured, data must be
stored if they are to be available for re-use. Research and data collection are expensive, and as we have seen in the Industry Insights above, there is very often value in
data beyond their initial, original use: they may be used again in the same context or
in a different context, on their own or in combination with other data.
In most organisations, there exists a process for gathering, storing and preserving
data as well as the necessary infrastructure to support the search, retrieval and re-use
of data. These infrastructures or data ecosystems are designed to serve the needs of
the organisation and the end users of the data. They include facilities such as data
archives, data lakes, databases and data warehouses as well as insight management
platforms. We look at insight management platforms in Chapter 20. Here we look at
data archives, data lakes, databases and data warehouses as these may be the source
of your data.
Data archives
A data archive is a formal, structured way of storing data and the documentation
associated with a dataset and/or a research project. It is a step beyond simply storing the data in a back-up file. The contents held in the archive are documented and
described with ‘metadata’ labels to facilitate search and access. Among other things,
the metadata might include the following:
the name of the client who commissioned the research;
the name of the researcher;
● the project title;
● the dates on which the project ran;
● methods used; and
● key words about the project and/or the findings.
●
●
There are many formal archives containing academic and government-sponsored
data which can be accessed by any interested party. Box 8.7 opposite contains some
examples.
184
M08 The Practice of Market Research 31362.indd 184
27/09/2021 21:46
Data storage and retrieval systems
Box 8.7
Examples: data archives
The UK Data Archive
A vast amount of data – qualitative and quantitative – relating to social and economic
life in the United Kingdom as well as business and administrative data – is held at the
UK Data Archive at the University of Essex. The Archive contains data collected by
the ONS on behalf of the UK Government and includes some of the most well-known
UK data series (e.g. Living Costs and Food Survey and the Crime Survey for England
and Wales) as well as the most important longitudinal surveys such as Understanding
Society and the Millennium Cohort Study. Besides government produced data, the
Archive holds UK academic research data as well as material from other (international)
archives and data from market research, independent research institutes and public
bodies. The Archive website (https://www.data-archive.ac.uk/) contains full descriptions and documentation of datasets (including qualitative data). It can be searched
and accessed through the UK Data Service catalogue.
The site also offers guides and video tutorials, examples of how data have been
used, and information and resources for teachers and learners. There are also examples
of secondary research projects that made use of the Archive’s data. One such example
is a report by Mateos-Garcia and Bakshi (2016) on where in the UK the creative industries are growing. The authors used several sources of official data from the UK Data
Service including business activity data, job figures and financial data, and combined
those with a geographical analysis of data on the supply of university graduates and
research and data on social network activity.
The Central Archive for Empirical Social Research
This archive at the University of Cologne (https://www.gesis.org/) houses German survey data as well as data from international studies and is the official archive for the
International Social Survey Programme (ISSP), of which the European Social Values
Survey is a part. The ISSP collects data on key social and social science issues in over
30 countries worldwide. The Central Archive provides access to the data collected from
each country and to the file containing data from all participating countries for each year
of the survey. Data from the European Social Survey is archived at Norwegian Social
Science Data Services in Bergen (https://www.nsd.uib.no/nsd/english/index.html).
The Inter-University Consortium for Political and Social Research
The ICPSR archive at the University of Michigan (https://www.icpsr.umich.edu/)
­provides access to social science data from over 400 member colleges and universities worldwide. It also has a series of archives relating to particular topics, for example
the Health and Medical Care Archive and the International Archive of Education Data.
Archive of Market and Social Research
This archive (https://www.amsr.org.uk/) contains research studies, books, journals,
data abstracts and conference papers from sources that include some of the bestknown research agencies in the UK.
185
M08 The Practice of Market Research 31362.indd 185
27/09/2021 21:46
Chapter 8
Sources of existing data
Data lakes, databases and data warehouses
A data lake is a repository for raw data from a range of sources, in any format (structured, semi-structured and unstructured), that is not yet processed and for which an
end use has not been determined. Databases and data warehouses, on the other hand,
contain raw data that have been processed with an end use in mind. An organisation that needs to integrate, store and access data from different sources in order to
analyse these data for decision-making purposes will have a data warehouse. It will
have a database for recording or collecting data, for processing and querying the data
and for getting access to data quickly.
The data in a database or data warehouse should be accessible to anyone who
needs them; the data in a data lake, however, tend to fall within the domain of the
data scientist. The lake and warehouse metaphors are useful in helping to visualise
the state of the data within these entities: the data in the lake are fluid and flexible;
the data in the warehouse are organised, stacked and labelled. Many organisations
are likely to have all three entities – lake, database and warehouse – with connections
between them.
Data lakes
For an organisation that generates a lot of unstructured data from different sources,
e.g. websites, apps, Internet of Things (IoT) and social media, and needs access to
insights from that data in real-time, a data lake is a good storage solution. Such a data
lake may be referred to as an enterprise data lake (EDL) because it stores data from,
and relevant to, all parts of the enterprise or organisation. The data can be formatted,
transformed, indexed, and aggregated (and linked with other data). However, they
are typically left unprocessed, to be prepared only as and when needed. This makes
a data lake a very agile facility. It also keeps preparation and processing costs low in
comparison to the costs involved in preparing and processing data for storage in a
data warehouse. The risk with the lake approach to storage, however, is that the lake
may become a swamp, a dumping ground for all sorts of unprocessed data, with the
result that trying to work through it is hard going and time consuming.
This is where data governance comes in. Data governance is about managing the
data according to a set of rules designed to meet the internal standards and needs of
the organisation. Data governance rules tend to address the following internal and
external issues:
data quality (completeness, accuracy, consistency);
usability;
● security; and
● ethical, legal and regulatory requirements.
●
●
So, the data are sent to the lake directly from content sources (although they may
be held temporarily in a staging repository before being moved into the lake). Once
in the lake they can be held in their original format or they can be transformed and
then stored. A key step at this ‘ingesting’ stage is to add metadata to describe and
label the data. This is important from a data governance point of view in ensuring
usability of the data and in preventing the lake from turning into a swamp: describing and labelling the data makes it possible for users to search and find relevant
material. The metadata labels are stored in one layer of the lake with the raw data
186
M08 The Practice of Market Research 31362.indd 186
27/09/2021 21:46
Data storage and retrieval systems
or lightly processed data sitting in another, core storage layer. Besides metadata,
the data lake should also have a search engine, one that can handle the scale and
type of data in the lake (structured, unstructured, semi-structured) in real time, or
near-real time.
Databases and data warehouses
A database is a data recording, storage and retrieval facility that can support a very
large number of users working at the same time to add to or search or retrieve data
with little effect on processing speed. It uses a database management system (DBMS).
The speed of querying the database and getting access to the data – the response
time – is measured typically in milliseconds. A database tends to contain current
data important to the day-to-day operation of the organisation rather than historic
data that might be useful for decision-making purposes. The processing system used
is called online transaction processing (OLTP). Note this is for processing and not
analysis.
A data warehouse is an integrated central storage facility geared towards business intelligence and management decision making (Inmon, 2005). In fact, it may be
referred to as a business intelligence system (BIS) or a decision support system (DSS)
or an executive information system or enterprise intelligence system (EIS) or even
an insight management system. If it has been designed with this end use in mind,
it is a ‘top down’ warehouse. You may also come across ‘data marts’, databases
designed for specific areas of operation such as marketing or sales. ‘A bottom up’
data warehouse is a warehouse created from a combination of these smaller data
marts (Kimball and Ross, 2016).
The end use of the data should dictate how the database or warehouse is structured
and how the data in it are organised. It should be organised in a way that allows the
analyst or end user to look at the data from relevant perspectives, for example by
customer type, by brand and by market; it may be useful to have the data organised
into data marts for sales or purchasing data; current data and historic data may be
stored in a way that facilitates faster access to the more frequently used current data
but allows the historic data to be called up for comparison, for examining trends or
making predictions.
Data warehouse structure
A data warehouse is in effect a very large analytical database that typically contains
data from more than one source and both current and historic data. It is sometimes
referred to as a consolidated database. A typical data warehouse design, particularly
for storing traditional ‘small’ data, contains three tiers or layers. The bottom layer
is the data source layer. It holds the server that collects or pulls data from content
sources in the operational field – from the organisation’s internal operational sources
or databases including orders, sales, invoicing, customer management and so on as
well as from external sources including sensor data, IoT, apps, web traffic and social
media. The middle layer is the data storage area. It holds the warehouse manager or
OLAP server. This is the server that is used to store and structure the data in a way
that aids analysis. The middle layer of the warehouse will also contain a relational
database management system (RDBMS). It may even have a data warehouse of its
187
M08 The Practice of Market Research 31362.indd 187
27/09/2021 21:46
Chapter 8
Sources of existing data
own and an operational data store and the computing power to process all that is
required. The top layer is the applications area. It holds the access tools, the business
intelligence software, that allow the end users to run reports and queries, mine the
data and use analytics.
Moving data to the database or warehouse
Data are sent to the database or warehouse from the operational field – e.g. from
payment card transactions, from RFID tags, or from the company’s website, app or
social media site. At this point the data may be structured, unstructured or semistructured. Whatever the source or the structure, the data should be relevant and
necessary to the needs of the organisation and of good quality. Irrelevant or unnecessary data will only clog up the system and slow down access and processing time.
With data being sent automatically, and constantly in real time, it can be difficult to
decide what is relevant and necessary. Programs can be added to filter out what is
deemed irrelevant or unnecessary. These are, of course, only as good as the parameters set by the programmer. In terms of quality, it is important to remember that
these data are the raw material that will be used to support key decisions and any
inaccuracies or inconsistencies will impact on the quality of the decision making.
Good data governance should mean that the data are cleaned and checked for accuracy, consistency and completeness. This will involve, among other things, checking
for missing values, checking that definitions of terms and variables, and the coding
procedures used for these, are consistent. The aim is that the data can be fully integrated or fused with other data in the warehouse. This is relatively straightforward
to do with traditional small data and structured data. However, when the data are
high-variety, high-volume, high-velocity big data, it is a more difficult but nevertheless essential task. The techniques available include techniques from natural language
processing (NLP), computer vision and machine learning (ML). These techniques can
turn ‘unstructured, low-density, low-value big data into high-density and high-value
data’ (Kelleher and Tierney, 2018). Once transformed, they can be integrated with
data from other sources. Once in the warehouse the data are referred to as being in
the informational field.
ETL and ELT
There are two ways in which the data can be added to the data warehouse (and
moved between databases within it). The first is ‘Extract, Transform, Load’ (ETL);
and the second is ‘Extract, Load, Transform’ (ELT).
The ETL procedure extracts the data from the source, e.g. the transactions operational database, and holds them in a staging database. While they are in this interim
database the data are transformed into whatever structure is appropriate for the
main data warehouse. Once they are structured, they are sent or loaded to the main
warehouse. In the ELT procedure data are extracted from the source and loaded
directly into the main warehouse; there is no interim warehouse. Once in the main
warehouse the data are transformed. An ELT approach is useful if the sources from
which the data are extracted are many and varied.
188
M08 The Practice of Market Research 31362.indd 188
27/09/2021 21:46
Data storage and retrieval systems
Metadata
As with a data lake, the warehouse contains information in the form of metadata
telling users about its structure and how to find their way around the shelves and
the rooms in the warehouse. Besides being a map of the warehouse, it also acts as a
contents list, providing the user with details of the databases or datasets in the warehouse, the elements contained in them, and how these elements relate to elements in
other datasets in the warehouse. Data in the warehouse may have been transformed
in some way (changes to coding or to format) and they exist in the warehouse at
different levels of detail – from what is called ‘detail’ through ‘lightly summarised’
to ‘highly summarised’. The metadata also give users this information – telling them
how the data were transformed, what changes were made to make them consistent,
and on what basis the data are summarised.
OLAP
The sort of processing a data warehouse uses is called online analytical processing
(OLAP). Note the use of analytical here (compared to OLTP or online transaction
processing in a database). OLAP (Codd et al., 1993) is a category of software tools
for retrieving, manipulating and analysing data in very large databases or data warehouses – those that contain elements that are interrelated and multidimensional.
OLAP is often linked to front-end business intelligence tools that enable those who
are not data specialists to interrogate the data. OLAP enables you to get access to
and perform both simple and complex analysis of the data at speed. It understands
the interrelated and multidimensional way in which the data are organised. It has
functions that allow you to perform analysis at the basic level – request descriptive
statistics, for example – and at a more complex level – for example trend and time
series analysis, factor analysis, pattern searching and modelling. The data warehouse
might contain aggregated data on sales, say; it will also contain a multidimensional
cell with data on sales classified or categorised according to different dimensions
relevant to the needs of the organisation, dimensions such as brand, sales outlet type,
market and so on. A single cell will contain data at the most granular level – for
example, sales of brand X phones in Week 32 in France. Data can be aggregated or
consolidated into larger sets (referred to as ‘roll-up’) – for example weekly sales into
monthly sales, monthly sales to quarterly, quarterly to annual; sales via the website
can be aggregated with sales via traditional retail outlets to produce total sales, or
sales in each EU country can be aggregated into an EU total. Aggregated data can be
disaggregated or broken down into smaller units, even down to the individual level,
in a process known as ‘drill down’. Data can be examined across a range of perspectives, such as by volume, by volume within market, or by volume within outlet type
by market, in an operation known as ‘slicing and dicing’. The response time will be
fast but may not be as fast as in an OLTP system.
Moving or not moving data for analysis
In some systems to run analysis the data are removed from where they sit in the
database or warehouse and integrated with data from another location within the
189
M08 The Practice of Market Research 31362.indd 189
27/09/2021 21:46
Chapter 8
Sources of existing data
Box 8.8
Example: OLAP queries
How many units of the cider brand K did we sell in the United Kingdom, Germany
and the Netherlands in the last financial year?
● How were these sales split between direct sales and agency business?
● What is the gross profit on direct sales and agency sales based on these sales
figures?
● On the direct sales side, how were sales split between wholesalers and retailers?
● Among the retailers in the United Kingdom, what was the split between on-sales and
off-sales accounts?
● Were there seasonal variations in sales between these two types of outlet?
● How do sales per quarter in the last financial year compare with sales per quarter in
the last two years?
●
database or warehouse, and then they are analysed. Once analysis is completed, the
findings are loaded back into the database or warehouse so that they can be accessed
by end users. This flow of data out of the warehouse into an analytics server and back
from the server to the warehouse takes time. To cut down on this time, the analytics
server with all the necessary capabilities (statistical analyses, machine learning) can
be built into the system so that there is no need to move the data for analysis. This
means that whatever analyses are required they can be run in the database using SQL,
the database language.
SQL, NoSQL and NewSQL databases
Many data warehouses have a relational database structure. In a relational database
structure the data are in tables of rows and columns and it may contain different sets
of data. The structure means that elements in one set can be related to elements in
another set. Hence the term relational database. You would choose a relational database structure if you have structured data, you know what you will be storing and you
can predict the sort of queries users will have (Murthy et al. 2014). The simplest form
of this type of database is a star design with a central fact table, for sales, for example,
and several linked or related tables, for product group, sales region, sales period and
so on, as the arms of the star. It is a design that makes it fairly easy to run complex
queries. A variation on it is the snowflake design – similar to the star but with extra
dimensions. Some data warehouses make use of the multidimensional cube design. The
software that enables data integration, storage and access in this structure is called a
relational database management system (RDBMS). The programming language used
to run an RDBMS is called Structured Query Language (SQL). As a result, this type
of database is often called an SQL database. (There is also a type of RDBMS called
a NewSQL which we look at briefly below.) Scalability – increasing the size of the
database – can be an issue for a relational database. You can scale up (increasing the
size of the server) to deal with increasing volumes of data but scaling out across more
190
M08 The Practice of Market Research 31362.indd 190
27/09/2021 21:46
Data storage and retrieval systems
than one server does not work well. This means that as the number of data sources,
the complexity and size of the data from those sources increase, the ability of an SQL
system to continue to support sharing and analysis of data becomes more difficult.
This is where a non-relational database comes in. It is also known as a NoSQL
database. There are several types of NoSQL databases (Murthy et al., 2014): document
oriented; key-value stores; big table inspired; and graph oriented. In a NoSQL database
the data structure is different. In a document-orientated database data are not in table
format but are held in JavaScript Object Notation (JSON) documents. For example, in
a relational database, you might have your customer’s last interaction or transaction in
one table, their account details in another table, and their previous purchase in another
table. To get a picture of the customer you need all of that data, data that are spread
across several tables, brought together. This is quite a big task. In a non-relational
database, all of those data are in one JSON document, thus making the task a much
easier one. A NoSQL database is the type of database used for data from web applications, including social media and the Internet of Things, and for other big data analytics. It can store structured, semi-structured and unstructured data; and it can support
the processing of very large volumes of that data at great speed. It uses a distributed
system – that is, data are distributed over more than one server. This means that it is
relatively easy to scale out a NoSQL database. The distributed system is supported by
software such as Hadoop, software that facilitates parallel and massively parallel processing (MPP) across the distributed system. The type of NoSQL database you choose
will depend on a variety of factors including structure of the data, their complexity,
the amount, the type of processing (batch, near real-time, real-time) and performance.
Hadoop is a framework developed by the Apache Software Foundation (hadoop.
apache.org) to enable distributed high-speed processing of data across clusters of
computers. It uses a function called MapReduce to split up or partition datasets into
chunks. It stores these chunks on different nodes within a cluster. When someone
queries the data, Hadoop runs the query across all the separate chunks in parallel
before combining the findings generated from each chunk. A data storage design
for dealing with batches of static data rather than a data stream is likely to include
Hadoop. If, however, you need to process real-time streams of data, Storm (storm.
apache.org) rather than Hadoop is likely to be the choice of framework. Storm works
by processing the individual elements of the data stream rather than the entire dataset
or batch. There are two other noteworthy frameworks: Flink – designed for stream
processing of ‘bounded and unbounded data’ but can also do batch processing (flink.
apache.org); and Spark – ‘a unified analytics engine’ (spark.apache.org) designed for
batch processing but can also do stream processing.
NewSQL is also a way of dealing with the limitations of an SQL system. A NewSQL
database has a relational database structure and uses SQL but has the scalability of
a NoSQL database system. Although it is a relational database, its internal architecture is different to an SQL system. Among other things, it splits up or partitions the
database into chunks or sets and in this way distributes the processing. This, along
with other features, enables it to cope with high volumes of data and lots of users.
Hybrid and cloud-based data warehouses
Kelleher and Tierney (2018) note that the bigger the organisation and/or the greater
the amount of data, ‘the greater the complexity of the technology ecosystem’.
191
M08 The Practice of Market Research 31362.indd 191
27/09/2021 21:46
Chapter 8
Sources of existing data
They report a range of approaches to creating these ecosystems from investing in an
off-the shelf, commercial integrated system to building a bespoke system using open
source tools and languages, and everything in between. The system may still, however, be called a database or a data warehouse. They also note the need to maintain
existing traditional systems. As an example, they note how older data or data that are
not often used can be moved from an existing traditional warehouse into a separate
Hadoop cluster (Kelleher and Tierney, 2018) creating what they call a hybrid system
design. The Hadoop cluster, with the older or less frequently used data, is linked to
the traditional SQL database or warehouse containing the current data so that both
sets can be queried together. Queries and analyses can be run using SQL as if the data
were in the same place and not split between two locations.
There are several different types of cloud-based (as opposed to on-site) data warehouses. What they have in common is that they use massively parallel processing
(MPP) and columnar storage (rather than rows of records used in a traditional database). While they can be difficult to set up, for example, loading the data may not be
straightforward and there may be issues with certain data formats, the advantages
over traditional warehouses include lower set up costs; increased performance; and
increased scalability.
End note
Whatever the internal system set-up, bear in mind when you are scoping or planning
a project that the information you need may already exist within your organisation,
sitting undisturbed in a data lake, a data warehouse or a data archive. We look next
at evaluating existing data and in later chapters at how to analyse the data.
Chapter summary
●
●
●
●
●
Secondary research involves looking for, using and analysing data that already
exist.
Consulting existing sources – doing secondary research – should be the first step
in answering any query or researching any topic. You may discover that there is
no need for primary research, that the secondary sources contain what you need
to address the business problem.
Secondary sources may provide useful information in the early stages of a project,
in problem identification, design and planning, and, at the later stages, providing a
context for the interpretation of primary data.
Existing data can be small data, for example from a previous research study
or data from external, public and ‘open’ sources, or big data from internal,
organisation sources or external sources.
The main external sources are government departments and related bodies
(official statistics); non government bodies (unofficial statistics); and web and useror consumer-generated content.
192
M08 The Practice of Market Research 31362.indd 192
27/09/2021 21:46
References
●
The main internal sources are data within data lakes and data warehouses or
enterprise or business intelligence systems or insight management systems.
Exercise
1 Your client wants to know about COVID-19 in your country: when it began,
incidence among the population, its impact on social, political and economic life.
Identify sources of existing data on this topic and summarise the information they
cover.
References
Brook, O. (2004) ‘“I know what you did last summer” – arts audiences in London 1998–2002’,
Proceedings of the Market Research Society Conference, London: MRS.
Codd, E.F., Codd, S.B. and Sally, C.T. (1993) Providing OLAP (On-line Analytical Processing)
to User-Analysts: An IT Mandate, Toronto, Canada: E.F. Codd and Associates.
Durand, M. and Gennari, P. (2016) ‘Foreword’, Case Studies: Using Non-Official Sources
in International Statistics, CCSA, https://unstats.un.org/unsd/accsub/2016docs-28th/­
E-publication.pdf (Accessed 13 February 2021).
Inmon, W.H. (2005) Building the Data Warehouse, 3rd edition, New York: John Wiley &
Sons, Inc.
Johnson, M. (1997) ‘The application of geodemographics to retailing: meeting the needs of the
catchment’, Journal of the Market Research Society, 39, 1, pp. 201–24.
Kelleher, J. and Tierney, B. (2018) Data Science, MA: MIT Press.
Kimball, R. and Ross, M. (2013) The Data Warehouse Toolkit, 3rd edition, Wiley.
Kitchin, R. (2014) The Data Revolution, Beverly Hills, CA: Sage.
Mateos-Garcia, J. and Bakhshi, H. (2016) ‘The geography of creativity’, UK Data Service,
www.nesta.org.uk/report/the-geography-of-creativity-in-the-uk (Accessed 2 March 2020).
Murthy, P., Bharadwaj, A., Subramanyam, P., Roy, A. and Rajan, S. (2014) Big Data Taxonomy, Cloud Security Alliance, https://downloads.cloudsecurityalliance.org/initiatives/
bdwg/Big_Data_Taxonomy.pdf (Accessed 12 January 2021).
McMillan, S. (2018) ‘Plant power Down Under’, Impact, 22, pp. 12–13.
Mindshare UK and Three (2018) ‘Three brand reputation analysis: Using social conversations
and search insight to uncover implicit attitudes towards Three Mobile Network’, MRS
Awards.
Nunan, D. (2020) ‘Research in the 2020s: from big data to bigger regulation’, International
Journal of Market Research, 62, 5, pp. 525–27.
OECD (2020) OECD Policy Papers on Public Governance No. 1, https://www.oecd.org/gov/
digital-government/policy-paper-ourdata-index-2019.htm (Accessed 25 May 2021)
Passingham, P. (1998) ‘Grocery retailing and the loyalty card’, Journal of the Market Research
Society, 40, 1, pp. 55–63.
Simms, J. (2019) ‘Energising the market’, Impact, 26, pp. 47–50.
Taylor, M. (2019) ‘How to avoid wasting research’, Impact, 25, p. 73.
193
M08 The Practice of Market Research 31362.indd 193
27/09/2021 21:46
Chapter 8
Sources of existing data
Upadhyaya, S. (2016) ‘The use of non-official statistics for transforming national data into an
international statistical product – UNIDO’s experience’, in Case Studies: Using Non-Official
Sources in International Statistics, CCSA, https://unstats.un.org/unsd/accsub/2016docs28th/E-publication.pdf (Accessed 13 February 2021).
Vriens, M., Brokaw, S., Rademaker, D. and Verhulst, R. (2019) ‘The marketing research curriculum: closing the practitioner-academic gaps’, International Journal of Market Research,
61, 5, pp. 492–501.
Recommended reading
Dale, A., Arber, S. and Proctor, M. (1988) Doing Secondary Analysis, London: Unwin Hyman.
Hakim, C. (1982) Secondary Analysis in Social Research, London: Allen & Unwin.
Kelleher, J. and Tierney, B. (2018) Data Science, MA: MIT Press.
Leventhal, B. (2016) Geodemographics for Marketers: Using Location Analysis for Research
and Marketing. London: Kogan Page.
MacInnes, J. (2016) An Introduction to Secondary Data Analysis with IBM SPSS Statistics,
London: Sage.
Webber, R. and Burrows, R. (2018) The Predictive Postcode: The Geodemographic Classification of British Society. London: Sage.
194
M08 The Practice of Market Research 31362.indd 194
27/09/2021 21:46
M08 The Practice of Market Research 31362.indd 195
27/09/2021 21:46
Chapter 9
Using and evaluating existing data
Introduction
In Chapter 8 we looked at sources of existing data. In this chapter we look
at what is involved in using those sources for research purposes, in particular
how to evaluate the sources. In Chapter 18 we look at data mining and data
analytics using existing data.
Topics covered
Using existing data: rationale, types of projects and process
● Evaluating sources
● Developing an analysis plan.
●
Relationship to MRS Advanced Certificate Syllabus
This chapter is relevant to Topic 2: Guiding Principles; and Topic 3 Selecting the
research design and planning the approach.
M09 The Practice of Market Research 31362.indd 196
27/09/2021 21:47
What you should get from this chapter
At the end of this chapter you should be able to:
understand what is meant by using existing sources;
● evaluate sources;
● begin to develop a plan for the analysis.
●
197
M09 The Practice of Market Research 31362.indd 197
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
Using existing data: the rationale
We looked at the reasons why you might use existing data in the last chapter. You use
existing data because they are useful in a variety of ways, and at various stages in a project. Since there is no need to collect data, using existing sources can be relatively time
efficient and cost effective. However, this is not always the case. Getting hold of good
quality data can be difficult and expensive: if the data are not of a certain standard,
you have the cost of cleaning and preparing them; and there are costs associated with
data storage, processing and computing and the infrastructure that goes with that. In
addition, there are ethical and legal risks – any infringements in these areas will have
costly implications. These can include financial penalties from regulators, legal costs
and damage to reputation as well as causing harm to those whose data are involved.
Using existing data: types of projects
The nature and scale of the re-use of existing sources can vary. Quoting topline statistics from a research report is secondary research at its most basic. The re-analysis
of several combined or integrated sources of data – secondary data analysis or data
analytics – represents a more complex use of existing sources.
Secondary research
Basic secondary research typically involves extracting and reporting what you find
from an existing source with little or no re-analysis of the data. It is done at a level
above the dataset or the raw data. It makes use of research reports and published statistics rather than the actual data. The example in Industry Insight 9.1 illustrates this
type of secondary research. A more formalised and rigorous example of this approach
is the literature review, a review of the scholarly literature on a theory or on previous
research studies on a topic. We looked at how to do a literature review in Chapter 4.
Industry Insight 9.1
Setting the scene
By almost all measures, the United States economy
is going great guns. Unemployment is at its lowest
point in 50 years, consumer spending is strong,
wages are growing and GDP is running at around
a healthy three per cent. Yet according to Gallup,
only 31 per cent of Americans are satisfied with
the way things are going.
One of the lasting effects of the recession [in 2008]
has been an increasing anxiety divide, or ‘uncertainty
inequality’. Looking at data from the CDC, stress
levels for low-income consumers grew noticeably in
2008/2009 and continued to increase at roughly the
same pace up to today. For high-income consumers, there was little to no effect. Today low-income
consumers are 12 times more likely than those with
high incomes to feel psychological stress.
Most Americans also feel that the country’s
place in the world is becoming unclear. A recent
Pew study found that 60 per cent of the country
believes that, over the next 30 years, the US will
198
M09 The Practice of Market Research 31362.indd 198
27/09/2021 21:47
Using existing data: types of projects
be less important in the world. China is perceived
as the big threat, with 62 per cent believing that
the amount of US debt China holds is a serious
problem for the country.
Despite the current administration’s tepid
approach, climate change is casting another
shadow. According to a recent Yale University
poll, 73 per cent of US consumers believe global
warming is happening.
Source: Adapted from Potts, M. (2019) ‘America the unsure’,
Impact, 26, p.17. Used with permission.
Secondary research can make use of different sources and different types of
data. The Industry Insight 9.2, which we first looked at in Chapter 3, is an example of a project that used sales data, previous research reports and social media
posts.
Secondary data analysis and data analytics
Secondary data analysis involves analysis of existing data from internal or external
sources for purposes other than those for which the data were collected. It involves
work at the level of the data. It may even involve merging, fusing or integrating
several datasets and analysing the merged set. We looked at an example of this sort
of secondary data analysis project in Industry Insight 8.5: data collected at the box
offices of performing arts venues in London were combined and analysed to provide a
picture of audiences and their attendance at various theatres (Brook, 2004). Industry
Insight 9.3 outlines another project that involved analysis of existing, publicly available data. You can find more detail on this example and others in the case studies
section of the UK Data Service website.
Industry Insight 9.2
The Fragrance Cloud
Consumer goods company Unilever needed to
understand fragrance for Comfort, its flagship
fabric conditioner, in order to deliver inspiring
and compelling communication insights, and to go
beyond the clichés of flowers-clothes-fragrances.
Previous ‘traditional’ research studies showed the
challenge of infusing emotional engagement into
a functional category like laundry. Working with
research agency Ipsos, the company conducted
a pilot study to bring together insights from a
variety of sources. The pilot study covered five
markets. It involved mining all recent studies
(qualitative and quantitative), thought notes, sales
data and desk research to explore themes on fragrance. It was supplemented with a social listening
programme designed to capture themes in local
language across the five markets and from different sources such as social networks, blogs, videos,
and brand communication. Over 56 insights were
uncovered spanning eight opportunity territories
in five weeks, with no primary research.
Source: Adapted from Ipsos and Unilever Consumer & Market
Insights, ‘The Fragrance Cloud – an Inspiration Ecosystem’, MRS
Awards 2017.
199
M09 The Practice of Market Research 31362.indd 199
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
Industry Insight 9.3
Where is creativity?
The creative industries are a fast-growing sector
of the UK economy but little information was
available on where in the UK they are growing.
To address this, researchers Juan Mateos-Garcia
and Hasan Bakhshi (2016) made use of several
sources of ‘official’ data from the UK Data Service, a source we mentioned in the last chapter.
They used data on business activity, job figures,
and financial data, and they combined that data
with a geographical analysis of data on the supply
of university graduates and with data on social
network activity. From that analysis they were
able to identify 47 creative clusters across the UK.
Source: Based on the information from Mateos-Garcia, J. and
Bakhshi, H. (2016) ‘The geography of creativity’, UK Data
Service. www.nesta.org.uk/report/the-\geography-of-creativity-inthe-uk (Accessed 2 March 2020)
When the data analysed in a secondary research project are big data, the process
is often called data analytics. Whatever the name, the aim – as with all secondary
research – is to extract new findings and insights. Industry Insight 9.4 is an example
of a data analytics project, one we looked at in greater detail in Chapter 3. It is relatively unusual in that it includes analysis of images as well as text.
Secondary data analysis became an important part of social research in the United
Kingdom and elsewhere in the 1970s when the type of data collected by government
changed from statistics derived from administrative records to data collected via
sample surveys. At the same time there was an increase in access to computers for
analysis and storage, both of which became less expensive. Archives were created
to store and preserve computer-readable data, thus making the process of retrieval
and analysis much easier than it had been. Continued developments in hardware and
software mean that there is now an enormous wealth of available data – traditional
administrative and research data (so-called small data) and big data from a huge
range of sources – with the technology to access and analyse them ever more useable.
As a result, the scope for doing secondary research, secondary data analysis and data
analytics to address client problems is vast.
Industry Insight 9.4
Understanding Italian style
Italian furniture designer Arper wanted to
sharpen its digital communications. To understand how the brand is viewed on digital media,
research agency Kantar invented a technique
to mine social media for images. It used algorithms to detect elements within the images. The
elements were tagged according to what they
showed. The tags were used to search the web for
similar images. This information, combined with
text mined from a variety of sources, showed how
Arper is viewed. The researchers were also able
to uncover the kinds of images that had the biggest impact.
Source: Adapted from Kantar TNS and Arper, ‘Can chairs talk?
How image and text mining helped bolster Arper’s brand’.
Winner, MRS Awards 2017. Used with permission.
200
M09 The Practice of Market Research 31362.indd 200
27/09/2021 21:47
Using existing data: the process
Problem
Source
Access
Evaulate
• Identify business problem
• Define research problem
• Do the data already exist?
• Find a source
• Determine if access is possible
• Check ethical and legal issues re use for research
• Evaluate suitability and quality
• Evaluate technical issues
• Devise a plan for processing and analysis
Plan
Figure 9.1 The secondary research process
Using existing data: the process
The process of doing secondary research is summarised in Figure 9.1.
Identify business problem and define research problem
Do not start until you are clear about why you are doing the research. This will
determine what data you need.
Find a source
Look around to see what sources are available and relevant, what data exist, what
other research has been done. Talk to someone who has done work in the same or
a similar area; tap into their knowledge, ask for their advice. How did they tackle
things? What data did they use? Are those data available? Did they write a report?
Can you get the report and/or the data?
As we saw in the last chapter, and as the examples above show, there are two
main sources: sources that exist within the organisation; and sources external to
it. Depending on the nature of your topic or business problem, both might be useful,
or one or other might do. If the business problem relates to, say, customer retention,
you might begin the search with a review of internal sources, for example previous
research reports or data stored in the EIS or CRM system. If the business problem
201
M09 The Practice of Market Research 31362.indd 201
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
relates to the wider market in which the organisation operates, external sources might
be useful, including consumer-generated content, market reports, government data
and so on. Keep in mind at all times the business problem or issue or question you
are trying to address, no matter what your starting point. Finding a source may not
be straightforward, however. It may involve a search of the organisation’s enterprise
intelligence system, or a web search, or a search of a particular website or portal or
archive, or a combination of these. Take a look at Box 9.1 for some tips on starting.
Box 9.1
How to begin a search
What is the best way to approach a search for existing sources? There are two main
approaches:
1 start with a vague idea of what you need and narrow it down as you go; and
2 start with a precise topic or issue.
The approach you choose is likely to be related to the nature of your enquiry:
●
●
if it is exploratory in nature then you may want to go with approach number one;
if it is a descriptive enquiry with relatively clear-cut objectives then you may want to
take approach number two.
Whichever it is, it is worth bearing in mind that in any project time is usually short, and
there is likely to be more data, information and research available than you will be able
to handle so it will be easy to get overwhelmed. It is therefore worth thinking about what
it is you are after before you start the search. Set some search criteria or parameters.
These will certainly help you focus your search and/or re-focus your mind if you get
swamped. Examples include the following:
the area of business or the general topic;
the precise subject or topic (or as precise as you can make it);
● the sort of data or information you want (e.g. text or data; qualitative or quantitative;
research or non-research);
● the source of the information (scholarly, government, commercial, in-house, and so
on);
● the names of any authors that you know have worked on or published on that
topic;
● the titles of any works that you think might be relevant;
● the relevant dates.
●
●
During any search try not to be distracted by material that looks interesting but does
not meet your search criteria. Keep your purpose in mind at all times: remember, you
are looking for existing sources – of data, information, research, insight – that will help
you with whatever your goal is.
202
M09 The Practice of Market Research 31362.indd 202
27/09/2021 21:47
Using existing data: the process
Box 9.2
Searching the web
How to do it
It might be that the information you want can be obtained from the internet. We looked
at sources in the last chapter: organisations in the public, private and not-for-profit sectors; academic bodies and research institutions; government departments and bodies;
trade organisations; commercial research companies; web portals; directories related
to particular subjects; data archives; and so on. You can get access to these via the
web. How?
Say you want to search for a particular organisation, the first step is to go directly to
that organisation’s website. If you do not know its web address, make a guess. Leave
out the http:// part and go straight to the www bit of the address. Use the full name,
short name or acronym of the organisation next (e.g. mrs or cim or esomar). Then
add the appropriate domain: .com for international private sector organisations; or
the relevant country code (see http://www.iana.org/root-whois/index.html for a list of
country codes); or .org for non-private sector organisations; or .gov for government
departments and bodies; or .edu for academic bodies in the USA and .ac for those in
the UK. If this approach does not get you to the organisation you want, use a portal,
directory or search engine and type in the name of the organisation.
Web portals (including research, government, university, and student portals and
open data portals, directories and search engines) are not only useful for searching
for organisations, they are also useful in subject searches. There may be a specialist
portal or directory that gets you directly into the subject area you need. Portals offer
search, directory and other services, including news, access to discussion forums and
basic information on popular topics. A subject directory contains selected websites and
classifies them into subject categories. Search engines work by allowing you to search
the database of indexed words.
It is important to remember than none of the portals, directories or search engines offers
comprehensive coverage of what is available on the web. It is also worth remembering
that there is not always overlap in content between search engines – for this reason it is
useful to use more than one search engine in your search. You will find that some have
more powerful search capabilities than others and this may also influence your choice of
which to use. Multiple search engines or metasearch engines are also available. Examples include Dogpile (https://www.dogpile.com/) and MetaCrawler (https://metacrawler.
com). These work by searching in parallel the databases of several individual search
engines. These multiple search engines may not offer as comprehensive a search as
you might achieve with a search of the individual search engines. To search for search
engines, have a look at https://www.searchenginecolossus.com. This is an international
directory of search engines. Remember, even if you are using multiple search engines
simultaneously there will be items on the web that will remain invisible to the search.
Have a plan
You are unlikely to get what you want in only one click. This is another reason why
working out in advance a search strategy or plan with search criteria or parameters is
203
M09 The Practice of Market Research 31362.indd 203
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
useful. It should mean that you won’t get distracted as you do your search. Your strategy should allow you to narrow the focus of your search – starting with the general or
broad topic and working towards the more specific. It is, however, a good idea to be as
precise as you can be at the start of the search. Rather than searching for a key word,
try a Boolean search (that is, making your search more precise by using the operators and, or, not) or a ‘phrase search’ or a title field search to narrow the list of results.
Almost all the portals and search engines allow you to make these. A phrase search can
often be particularly productive. It is a search for the words entered adjacent to each
other and exactly in the order you give them. Most use double quotation marks to identify a phrase, for example: “geodemographic classification systems”. To get a better
match – to narrow the search further – you can add more words. The Advanced Search
facility that most portals and search engines offer also allows you to exclude words.
If you’re not sure exactly what it is you want, start with your key word or topic, search
for that then scan the results to see the sort of material it has produced. This will give
you information that may help you narrow your search by adding more words or building a phrase. You can also use the Advanced Search facilities to limit your search to a
title or a specific domain or a file type or a date, for example. Some Advanced Search
facilities allow you to limit your search to particular sources, such as scholarly papers.
Remember, no search engine can search the entire web, only its own database of
indexed words from those pages listed with it. You will miss, with any search engine
search, the content in the deep web, sites that need a log-in, including material on
intranets, and, in many cases, very current information (for which the best place to
search are the sites of newspapers, magazines, trade press and television and radio
stations).
Keep track
Keep a log of your search. This is important for several reasons: you may need to
show what you have done in a report or to a client or to other people working on the
project; you need to attribute or cite any material you used from your search; and you
or someone else may need to go back to some of the sources you found at a later date
for further information or for verification purposes. Bookmark the sites you visit or add
them to your reading list or your ‘Favourites’ or share the link with members of your
team via social media. Record the address or URL, or better still, the DOI or Digital
Object Identifier, if there is one, in a log or bibliography (using specialist software such
as Mendeley or Zotero or EndNote) along with the date on which you accessed it (this
is good practice since the content of the site may change over time). Where possible,
you should aim to keep a record of the following information for any source you consult:
Name of the author or creator
Title of the document
● Name of the editor (if appropriate)
● Year of publication and/or last update
● Name of the site
● Publisher, data centre or repository, and/or name of the body associated with the
site
● URL or Digital Object Identifier (DOI)
● Date you accessed the site.
●
●
204
M09 The Practice of Market Research 31362.indd 204
27/09/2021 21:47
Evaluating sources
Evaluating sources
Once you have determined that existing data can help, and you have found the
sources of that data and know you can get access to them, the next step is to evaluate
them. We look at several key aspects of this evaluation below:
evaluating ethical, legal or regulatory issues;
evaluating suitability and quality; and
● evaluating technical issues.
●
●
While they appear to stand alone, all of these issues are in fact inter-linked, as you’ll
see. You might begin your evaluation with the technical aspects or with the issues of
suitability and quality.
Evaluating ethical, legal or regulatory issues
You must determine if there are any ethical, legal or regulatory issues related to the use
of an existing source of research or data for your purpose. Research is conducted with
people, individuals, and data are collected from or about people – individuals – and their
activities. It is important to keep this in mind when you plan to re-use existing research
and data. You have ethical and legal responsibilities as a researcher to the individuals
whose data you use. The risks associated with not acting in a compliant manner are enormous. This is a complex area and you are advised to seek out expert professional advice
to ensure that you act in line with the relevant ethical, legal and regulatory framework.
Ethical principles
In Chapter 2 we looked at the ethical principles that underpin research. Here is a
brief recap:
voluntary participation;
no harm to participants;
● informed consent;
● privacy;
● transparency; and
● not deceiving subjects.
●
●
In deciding whether you can use existing data for further research, review them in
light of these ethical principles. It is useful to think ahead to the wider ethical issues
that might arise with the end use of the data and in dissemination or publication of
your findings. As boyd and Crawford (2011) state, ‘just because it is accessible does
not make it ethical’. They ask us to consider several questions, including ‘who is
responsible for making certain that individuals and communities are not hurt by the
research proposals?’ Here are some other questions you must consider:
Is it appropriate to re-use the data for research purposes?
What are your responsibilities to those individuals whose data they are?
● What must you take into account in relation to the key issues of privacy, consent,
end use of the research, and security of the data?
● Has consent been obtained from the individuals whose data they are for their reuse? If not, how do you obtain consent?
●
●
205
M09 The Practice of Market Research 31362.indd 205
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
What choice do the individuals effectively have in giving permission for their data
to be re-used?
● How do you balance the interests of all parties, data subjects, researchers and end
users?
● How do you assess the quality of the data? What can you tell the end users about
the quality of the data?
● How do you maintain transparency in all processes in a secondary research project?
●
The MRS Code of Conduct has several rules that are relevant in this context. These
are set out in Box 9.3.
Box 9.3
Professional practice and the MRS Code of Conduct
1 Awareness and Adherence with Legislation:
Members must ensure that their professional activities conform to the national and
international legislation relevant to a given project, including in particular the Data Protection Act 2018 in the UK, the EU General Data Protection Regulation 2016, and any
amendments and superseding legislation that may be enacted. This also covers applicable legislation inside and outside the UK.
3 Members must ensure that all of their professional activities, whatever the purpose,
are conducted in a transparent manner and that their activities promote compliance
with privacy ethics and data protection rules.
12 Members must carry out Data Protection Impact Assessment (DPIA) for specified
types of processing prescribed by data and privacy legislation and for any other processing that is likely to result in a high risk to participants.
13 Members must ensure that the rights and responsibilities of themselves, clients,
and sub-contractors as controllers or processors are governed by a written contract.
27 Members must ensure that there is a lawful basis for any collection and processing
of personal data undertaken as part of their professional activities.
42 Members must ensure that there is a lawful basis for the further processing of data for
a secondary purpose. This may include consideration of: a) Links between the original
and proposed new purpose/s; b) The context in which the data was originally collected
(in particular the relationship between participants and the original data collector); c) The
consequences of the proposed secondary processing; d) The existence of safeguards.
Source: MRS Code of Conduct 2019. Used with permission.
The UK Government also offers guidance in its Data Ethics Framework (2020). It
is aimed at anyone working with data including ‘data practitioners (statisticians, analysts and data scientists) . . . and those helping [to] produce data-informed insight’.
Data protection and privacy law
If the data you plan to use contain personal data, then any further use or processing of
these data for research purposes will be subject to the relevant data protection legislation,
206
M09 The Practice of Market Research 31362.indd 206
27/09/2021 21:47
Evaluating sources
as noted in the MRS Code. According to ESOMAR (2016), you ‘must ­comply with
the relevant data privacy legislation and the relevant requirements for notice, consent,
accuracy, security and access when personally identifiable data is collected and stored’.
A key concern is harm to data subjects. A way of evaluating risk of harm as a result of
processing data is to conduct a Data Protection Impact Assessment.
Data protection impact assessment
First of all, if possible, revisit the original research proposal to find out if any
potential risks were identified in relation to data processing and data protection
for that project. The researchers who designed the project may have conducted
and documented a data protection impact assessment or DPIA. Doing a DPIA is a
way of identifying risk so that steps can be taken to minimise it. The DPIA for the
original project should give you some insight into the issues related to that project
and the use of data for that project. In your secondary research project, you are
using the data collected in that primary research project for another purpose. You
should review the risks involved in your use of the data. The regulator responsible
for data protection in the UK, the Office of the Information Commissioner, states
that you must do a DPIA if processing data is ‘likely to result in a high risk to individuals’ and that it is ‘good practice’ to do one for any major project which requires
the processing of personal data. Doing a DPIA involves the following (ico.org.uk):
setting out the nature, scope, context and purposes of the processing;
assessing necessity, proportionality and compliance measures;
● identifying and assessing the risks to individuals; and
● identifying any additional measures to mitigate those risks.
●
●
Consult with your organisation’s data protection officer or other expert about the
data you plan to use and the risks associated with that use. Get advice and guidance
on how to proceed to ensure ethical, legal and regulatory compliance.
Lawful bases for processing personal data
If the data include personal data then you must have a valid, lawful basis for processing
them. According to GDPR 2016 there are six lawful bases. We looked at this in some
detail in Chapter 2. The most appropriate depends on the purpose of the processing
and the relationship with the individual data subject. Most lawful bases require that
processing is ‘necessary’ for a specific purpose. It is essential that you determine your
lawful basis before you begin processing data. You must determine the basis and you
must document it. The lawful bases appropriate for most research projects are consent
of the data subjects or legitimate interests of the data controller or a third party.
For the purposes of secondary research and secondary data analysis, the legitimate
interests ground is perhaps the most appropriate. This ground is suitable for the following types of research with existing data:
customer satisfaction research on existing customer databases;
qualitative or quantitative research using customer databases;
● research using existing datasets or third party data (for example, social media
analytics); and
● secondary data analysis or data analytics on loyalty card data.
●
●
207
M09 The Practice of Market Research 31362.indd 207
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
You can also use the legitimate interests ground to process personal data further –
within the reasonable expectations of the data subjects. If the personal data being
processed have not been collected by consent then the processing of that data can
only be for a ‘compatible purpose’. The key issues here, according to guidance given
in the Efamro/ESOMAR Guidance Note (2017), include (but are not limited to) the
following:
The link between purposes
The context and relationship between data subject and data controller
● The nature of the personal data
● Possible consequences of processing
● The safeguards in place (e.g. encryption, pseudonymisation).
●
●
These issues are also reflected in the MRS Code of Conduct (2019). The Guidance
Note suggests that researchers can use personal data for secondary research purposes
on the grounds of the legitimate interests of clients – because the research purpose is
likely to be compatible with the original data collection and processing purpose.
However, you must always check for the most up-to-date advice.
Terms of Use
If the data you plan to use are from an online or social media platform then you are
subject to the platform or service owners’ Terms of Use (ToU). You must read these.
They may mean that you cannot use the data at all – the terms may not allow any
copying of material for further analysis. Make sure you check what the conditions of
use are before you use any data from any platform. It may be that you need to obtain
permission and, if granted, there may be conditions attached, especially in relation
to privacy. If you do not get permission, you may be able to read the information on
the site and summarise it but not copy it. Whatever you do, you must ensure that you
act within the terms of use for that site or platform, act within the law and conform
to the ethical standards of the research profession.
Copyright
You must also determine what conditions of use apply in relation to copyright for
any material you source. You must ensure that you act within the law in relation to
copyright.
Due diligence in buying data
If you are buying data you must do due diligence checks on the provider to ensure
that they comply with the relevant legislation, including data protection legislation, and with any ToU. Here are some questions you should ask of the data
supplier:
Do they have permission to sell you the data?
Have the data been gathered legally?
● Do they comply with the requirements of data protection legislation in the jurisdictions in which they source, gather, process and share the data?
●
●
208
M09 The Practice of Market Research 31362.indd 208
27/09/2021 21:47
Evaluating suitability and quality
Do they have adequate processes in place to safeguard privacy and ensure use of
the data does not harm the subjects?
● Do they have adequate security to protect personal data?
● To which professional codes of practice do they belong?
● To which industry standards do they adhere?
● Are there any complaints with relevant authorities about their use or handling of data?
●
Evaluating suitability and quality
Existing sources can be divided into two groups: those that are derived from primary
research; and those that are not. This is an important distinction to bear in mind
when it comes to evaluating quality and, later, the technical aspects of a source.
First of all, let’s consider suitability: is the material under consideration suitable
for your purpose? If it is, then you can move on to determine if it is of sufficient
quality to be worth using. Quality and suitability go hand in hand: you do not want
to use material that is of poor quality, even if it is suitable; and you do not want to
use material that is not suitable, even if it is of good quality. Once you have established suitability and quality you can consider the technical aspect of the source, its
structure, complexity, volume and speed. However, remember that there may be
technical aspects of a source that mean it does not meet your suitability and/or quality criteria. These three things are interconnected. You may not be able to decide to
use a source until you have assessed all three in an iterative (back and forth) rather
than a linear process.
Evaluating suitability
You need to establish if the existing data are relevant to and suitable for your purpose. This means that you must be clear about what your purpose is. Your purpose
may be solving a business issue or problem that you have identified, or you may
be further along in your research process and have a set of research objectives or
research questions that you hope existing data will help you address. So, you need to
ask: are the data relevant to my purpose? Do they relate to the topic or issue at hand?
Do they cover similar ground? What population do they cover? In their current form
or if subject to further analysis, will they help me? If the data are relevant, then you
must establish the currency of the material – how up to date is it? If the source is a
fixed or static dataset (rather than a dynamic one), then the key questions here are:
When were the data collected? How long ago?
Have they been overtaken by recent events or discoveries?
● Do they make use of or rely on out-of-date ideas, facts, figures and so on?
●
●
The answers to these questions will help you decide whether or not the data are of
such an age as to be worth using. Remember, there may be a time lapse between
when data were collected and when the dataset or other source comprising the data
was made available for use, or when it was published, so make sure to check. It is
not uncommon for data to become available for further use several years after they
were first collected, when the original researchers or end users have finished working
209
M09 The Practice of Market Research 31362.indd 209
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
with them. You must relate the age of the data to the aims of your project: it may be
that the age of the data is not an issue.
If the source of data is a dynamic one, that is, a source to which more and more data
are being added over time – think, for example, of transaction logs, web traffic monitors,
and social media posts – the following question might arise: if the data are constantly
accumulating, when do you consider the data to be out of date? Again, think back to
why you are interested in using the data – what time period is relevant for your purpose?
Once you are clear that the data are suitable for your purpose (and you know it is
ethically and legally appropriate to use them for that purpose), and you have established their timeliness, then you can move on to evaluating quality. First, we look at
evaluating the quality of data derived from a research project; then we look at how
to evaluate data from a non-research source.
Evaluating quality: data from a research project
It should be fairly straightforward to evaluate the quality of data derived from
research because it is likely that the research was designed and conducted by professional researchers working to a recognised standard or code of conduct such as the
MRS Code of Conduct (mrs.org.uk) or the ESOMAR Code (esomar.org). So, start
by finding out the following:
Who commissioned the research?
Who produced the research? Who conducted it?
● Where did you find the research or the data?
●
●
The answers to these questions will help you determine whether the material has
authority (Gibaldi, 2003) and credibility. Once you have those basic answers you
can explore further to address the following:
Is the source an organisation or a person with a track record and/or reputation in
research and/or in data generation and/or storage and archiving?
● What research accreditations or qualifications do they have?
● To which professional bodies do they belong?
●
Besides various codes of conduct, there is a well-established standard of practice for
market, opinion and social research (including insights and data analytics): the British Standard/International Organization for Standardization, BS ISO 20252:2019.
It is available from the British Standards Institute website (bsi.group.com). You can
check which organisations are certified to this BS ISO 20252:2019 standard via the
Market Research Quality Standards Advisory Board on the MRS website (https://
www.mrs.org.uk/). You can view this by using the search field.
The next step is to examine the research project that produced the data of interest.
If the project was conducted by professional researchers, then it should be documented in a transparent manner.
The following documents should be available for review:
A summary of the project covering background, problem, aims and objectives,
research design and methods, time frame and project team (perhaps even the original brief and proposal)
● Details of where and how project files and datasets are stored
● A copy of the original data collection tool
●
210
M09 The Practice of Market Research 31362.indd 210
27/09/2021 21:47
Evaluating suitability and quality
A technical report on sampling including the sampling techniques used and the
response rate
● A copy of instructions given to interviewers or moderators including any definitions or clarifications or other materials given to research participants
● Information on the structure of the dataset
● A description of the data cleaning and transformation process (data preprocessing
and preparation)
● A codebook containing a list of variables and values and a description of how the
data were coded, including non-response codes and any derived variables that were
constructed and any weighting factors applied to the data
● Details of technical tools used in processing and analysis
● A list of publications and/or a copy of the report and/or presentation.
●
You should be able to review the documents to find out about the following:
the problem the research set out to address;
the research design and sample;
● when the data were gathered;
● how they were gathered;
● how they were processed and analysed; and
● how they were used.
●
●
It is this information that will allow you to determine whether the material is relevant
to your purpose, timely and done to a high standard (ensuring a minimum of error
and a high degree of validity and reliability).
Box 9.4
Error and bias
Error and bias are terms we mentioned previously but are worth noting again in this
context. Error can arise or be introduced at any point in a research project. An error
could include any of the following:
a mistake in defining the business problem and/or the research problem;
the choice of an inappropriate research design;
● a mistake in defining the population of interest;
● use of a sampling frame that does not represent the population of interest accurately;
● mistakes in executing the sampling plan;
● selecting the ‘wrong’ participants to take part;
● a badly-designed questionnaire or script that results in a poor response rate and/or
biased responses;
● poorly worded questions that mean responses are ambiguous or misleading;
● poorly executed data collection;
● wrongly coding or classifying a response or an item of data;
● mistakes in tabulating the data;
● mistakes in analysing the data;
● mistakes in reporting and interpreting the data.
●
●
When there is a consistent pattern in error it is called bias. The way to address issues of
error and bias is to follow procedures to ensure that they are minimised, if not eliminated.
211
M09 The Practice of Market Research 31362.indd 211
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
We looked at the concepts of validity and reliability in previous chapters. They are
terms used to describe – and to evaluate – the quality and rigour of research. Validity is about how well a research project – including its design and structure, sample,
method of data collection and measures or questions used – delivers accurate, clear
and unambiguous evidence with which to address the research objectives. It is an
indicator of whether the research measures what it claims to measure. Reliability is
about consistency or stability and repeatability: if you were to repeat the research,
would you get the same results? In reviewing an existing source of research or data
you are assessing its validity and reliability. Here are the things to look out for in
your review to assess the quality of a dataset – the validity and reliability, and the
extent to which error and bias have been minimised:
Research design: is it appropriate for gathering the evidence needed for the project?
If there are flaws in the research design, if it was not suitable for the problem it
aims to address, the data will not be valid and so not worth using.
● Sample: what is the population of interest? How is it defined? Is this appropriate?
Is it accurate? What sampling approach was used? Was this suitable for the aims
of the research? What are the limitations? How were they mitigated? What was
the sample size? Is it robust enough for the level of reporting needed? What was
the response rate? Is it sufficient to ensure validity? Is the sample representative of
the population of interest? If there are problems with the sample, e.g. if the sample
is not representative of the population of interest, you cannot generalise from the
sample to the wider population and so the data are not valid.
● Method of data collection: how were the data gathered? Is that a suitable
approach? What were the disadvantages or limitations? How were they mitigated?
What quality control procedures were in place? What issues arose? How were they
addressed? What data collection tool was used? How well designed was it? These
questions should help you assess validity and reliability.
● Measures: what did the research set out to measure? How did it do this? Examine
the proposal, if it is available, to see what was being investigated; look again at the
data collection tool to see what was actually asked; and look at the report to see
what was analysed and reported. This should help you assess whether the research
and the data and findings it produced are valid and reliable.
● Processing and analysis: how were the data recorded, prepared, cleaned and
coded? What processes were involved? What quality control measures were in
place? These questions will help you assess reliability. How accurate are the data?
How complete are the data?
● Findings: what were the findings of the research? How credible are they? Check
the publications produced from the research – this will give you an insight into the
project, including how the data were interpreted and applied, and it will show you
the ground already covered by the original research team and point to interesting
questions waiting to be answered.
●
Evaluating quality: data from a non-research source
Many secondary data analysis and data analytics projects make use of non-research
data, for example data gathered via social media monitoring, or big data from an
organisation’s data warehouse, or data bought from a data broker. It is important
212
M09 The Practice of Market Research 31362.indd 212
27/09/2021 21:47
Evaluating suitability and quality
that, whatever the source, the data are suitable for re-use for research purposes and of
sufficient quality. The key questions you need to ask to evaluate them are as follows:
What are the sources? Where do the data come from?
How were the data generated or collected?
● Who collected the data or produced them?
● Why were they generated or collected?
● What are they ‘about’?
● What population do they represent?
● How well do they represent that population?
● What are they measuring?
● How accurate are they?
● What biases exist?
● How much data is there?
● How were they processed?
● How clean are they?
● How are they stored?
● What format are they?
● How are they documented?
●
●
Remember, these data are not data from a research encounter and as such, as
Branthwaite and Patterson (2011) argue, they may be limited in that they lack context
and are disconnected from the individuals who produced them. They may have been
gathered or produced in a way that is not familiar (they may come from multiple
sources), and/or they may be presented in a format that is unfamiliar (for example
unstructured data), and so may need some work in order to be able to support further
analysis. Thus you need to find out as much as possible about them in order to be
able to use them in a way that fulfills your ethical and legal responsibilities as a bona
fide researcher (Baker and Wirth, 2018). Baker and Wirth note that it is ‘vital that
the quality be assessed and documented’. They set out the importance of establishing
three things:
data governance;
data curation; and
● data provenance.
●
●
By data governance they mean ‘the rules by which the organisation that collected
the data manages the integrity, usability, security and availability of the data it collects’. They suggest that ‘these should be at least equivalent’ to the standards used
by researchers in collecting and processing primary data but that ‘too often this is
not the case’.
By data curation they mean a process akin to archiving, documenting the data and
preserving them so that re-use and analysis is possible. They note that non-research
data are less likely to be documented or preserved in a way that is useful for secondary data analysis.
For data provenance they cite Buneman et al. (2000): ‘the process of tracing and
recording the origins of data and its movement between databases’. It is particularly
important to know about data provenance when a source has been built from multiple sources ‘where a number of merging, linking, transforming or aggregating steps
have already been done’.
213
M09 The Practice of Market Research 31362.indd 213
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
For non-research data, particularly dynamic data, the volume and speed at which
they amass raise issues of age of the data and timeliness. We noted this above in terms
of deciding when data are considered to be out of date. The following further issues
arise in the case of dynamic data:
how to sample the data;
how much data to sample; and
● over what time period.
●
●
These issues touch on the technical challenges in using non-research data, which we
look at next.
Evaluating technical issues
The technical issues in using existing data are related to format and complexity,
volume, and speed. Assessing the structure, complexity and size of a data source is
important in enabling you to evaluate its quality and usability, and to choose the
best approach and the most appropriate tools and technologies for accessing and
analysing the data within it. In addition, the technical issues also have implications
for your ethical and professional obligations, to those whose data they are and to
end users of the findings.
Format of the data
We saw in Chapter 3 that data – and this includes existing data – can be structured,
semi-structured or unstructured data, or a combination (Marr, 2019); and they can
be complex or relatively simple. In terms of complexity, at the simple end of the spectrum are structured data from a single source; at the complex end is a dataset formed
of multiple sources of fused or integrated structured, semi-structured and unstructured data. Structured data can be analysed using standard analysis techniques and
machine learning and AI techniques. Analysis of unstructured data tends to require
ML and AI approaches.
Volume and speed
In addition to structure and complexity, you need to assess the size of the data source
and the speed at which data are accumulating. Bear in mind that the data source
may not be a static entity but a dynamic one, with more and more data accumulating at high speed, and at an ever-increasing rate. It may be so big that regardless of
its structure it cannot be analysed by standard analysis tools or approaches and may
require the tools and techniques of AI and ML.
Remember, quantity is not quality. Just because you have a huge dataset does
not mean that it is a valid or a useful one. A key question is how representative is it
of your population of interest? Even with big data what is gathered or captured is
a sample (Kitchin, 2014). If it is not representative, if there is bias within it, it does
not matter how big it is. You must also think about what the data are measuring. If
214
M09 The Practice of Market Research 31362.indd 214
27/09/2021 21:47
Developing an analysis plan
they are not accurately measuring what they claim to be measuring, that is, the things
relevant to your purpose, then despite their size, they are not worth using.
Understanding the data – where they come from, how they were collected, how
they are constructed, how they were processed and analysed, how they are stored
and documented – has implications for your planned re-use or re-analysis as well as
for communicating and using the findings drawn from them. Think ahead to how
you might explain the data and how they support your findings in a transparent
way (Baker and Wirth, 2018) to the client and the end users. Transparency is a key
principle of the ICC/ESOMAR Code and the MRS Code of Conduct, as we saw
above: members ‘shall ensure that their professional activities can be understood in
a transparent manner’. In addition, the MRS Code states that members must do the
following:
provide sufficient information to enable clients to assess the validity of results of
projects carried out on their behalf;
● ensure that data [and reports] include sufficient technical information to enable
reasonable assessment of the validity of results; and
● ensure that findings disseminated by them are clearly and adequately supported
by the data.
●
To meet these obligations, you must therefore evaluate the data thoroughly, establishing all the relevant aspects of data governance, data curation and data provenance
(Baker and Wirth, 2018) set out above so that you understand the data and the
techniques used to create, build and analyse them. This may not be an easy task with
some sources, in particular those that are made up of data from a range of sources,
but it is a vital one in upholding your ethical and professional standards. To achieve
this you may need to work closely with your data science team. Your job will be to
explain clearly to them what it is you need the data for, what issue you need them to
address, how you plan to use the data and what your concerns are in relation to that
use in the context of your professional, legal and ethical responsibilities. Be prepared
for them to tell you that the data do not meet your requirements, or that to meet
your requirements the data must undergo preprocessing and preparation. We look
at aspects of data preparation in Chapter 18.
Developing an analysis plan
It is important at all stages of the research process – including finding and evaluating
sources of existing data – not to lose sight of what you are aiming to do. You want to
use the data to address the client’s business problem. You must evaluate the data with
the end use in mind. Once you have evaluated the data, re-visit the business problem,
the research problem and the research objectives. Take some time to think about the
data you have in the context of the business problem, the research problem and the
research objectives. This is the foundation of your analysis plan. The next step is to
get to know the data with the business problem, the research problem and the objectives in mind. Once you are familiar with the data, you can build more detail into
your analysis plan so that you can work through and with the data in a systematic
and rigorous way. We look in more detail at what is involved in this analysis process
in Chapters 16, 17 and 18.
215
M09 The Practice of Market Research 31362.indd 215
27/09/2021 21:47
Chapter 9
Using and evaluating existing data
Chapter summary
●
●
●
●
●
Consulting existing sources – doing secondary research – should be the first
step in answering any query or researching any topic. You may discover that
there is no need for expensive primary research, that the secondary sources
answer the research or business problem. Secondary sources may provide useful
information, especially in the early stages of a project, helping with problem
definition and research design and planning, and, at the later stages, providing a
context for the interpretation of primary data.
Existing sources can be sources within the organisation – internal sources such
as those in an enterprise intelligence system or CRM system. They can also be
found outside the organisation. They can be sources derived from research and
those from non-research activities.
Before using an existing source for research purposes, it is important to establish
whether it is ethically and legally appropriate to do so.
You must also evaluate and document the suitability and quality of the source
for your purposes, including its validity and reliability, and you must evaluate the
technical challenges. These are essential tasks in fulfilling your obligations as a
researcher.
Once you have evaluated the data, re-visit the business problem, the research
problem and the research objectives: these are the basis of your analysis plan.
You must bear in mind at all times the business problem, the research problem
and the research objectives when working with the data.
Exercise
1 You have received a report based largely on data collected from existing sources.
It includes data from research and non-research sources. Describe the steps you
would take to review the suitability and quality of the sources and explain why
each is important.
References
Baker, R. and Wirth, N. (2018) Discussion Paper: Use of Secondary Data in Market, Opinion,
and Social Research and Data Analytics, ESOMAR. Esomar.org/uploads/public/knowledgeand-standards/codes-and-guidelines/ESOMAR-GRBN_discussion-paper_Use-of-SecondaryData_20180225.pdf. (Accessed 16 March 2020).
boyd, d. and Crawford, K. (2011) ‘Six provocations to Big Data’, A Decade in Internet Time:
Symposium on the Dynamics of the Internet and Society, September 2011. Available at
SSRN: http://ssrn.com/abstract=1926431 (Accessed 17 September 2012).
216
M09 The Practice of Market Research 31362.indd 216
27/09/2021 21:47
Recommended reading
Branthwaithe, A. and Patterson, S. (2011) ‘The power of qualitative research in the era of
social media’, Qualitative Market Research: An International Journal, 14, 4, pp. 430–40.
Brook, O. (2004) ‘“I know what you did last summer” – arts audiences in London 1998–2002’,
Proceedings of the Market Research Society Conference, London: MRS.
Buneman, P., Khanna, S. and Tan, W. (2000) ‘Provenance: some basic issues’. Lecture Notes
in Computer Science. Foundations of Software Technology and Theoretical Computer Science, 1974, pp. 87–93.
Efamro/ESOMAR (2017) General Data Protection Regulation (GDPR) Guidance Note for the
Research Sector: Appropriate Use of Different Legal Bases under the GDPR. ESOMAR.
https://www.esomar.org/uploads/public/government-affairs/position-papers/EFAMROESOMAR_GDPR-Guidance-Note_Legal-Choice.pdf (Accessed 16 March 2020).
ESOMAR (2016) The ICC/ESOMAR International Code on Market, Opinion, and Social
Research and Data Analytics. Amsterdam: ESOMAR.
Gibaldi, J. (2003) MLA Handbook for Writers of Research Papers, New York: The Modern
Language Association of America.
Ipsos and Unilever Consumer & Market Insights (2017) ‘The fragrance cloud – an inspiration
ecosystem’, MRS Awards.
Kantar TNS and Arper (2017) ‘Can chairs talk? How image and text mining helped bolster
Arper’s brand’, MRS Awards.
Kitchin, R. (2014) The Data Revolution, London: Sage.
Marr, B. (2019) ‘What’s the difference between structured, semi structured and unstructured data?’ Forbes Magazine, www.forbes.com/sites/bernardmarr/2019/10/18/whats-thedifference-between-structured-semi-structured-and-unstructured-data/#1f378d982b4d
(Accessed 16 March 2020).
Mateos-Garcia, J. and Bakhshi, H. (2016) ‘The geography of creativity’, UK Data Service,
www.nesta.org.uk/report/the-geography-of-creativity-in-the-uk (Accessed 2 March 2020).
MRS (2019) Code of Conduct, London: MRS.
Potts, M. (2019) ‘America the unsure’, Impact, 26, p.17.
UK Government Data Ethics Framework (2020) https://www.gov.uk/government/publications/
data-ethics-framework/data-ethics-framework-2020#overarching-principles (Accessed 13
February 2021).
Recommended reading
Baker, R. (2017) ‘Big data: a survey research perspective’, in P. Biemer, E. de Leeuw, S. ­Eckman
B. Edwards, F. Kreuter, L. Lyberg, N. Tucker, and B. West (eds) Total Survey Error in
Practice, New York: Wiley.
Dale, A., Arber, S. and Proctor, M. (1988) Doing Secondary Analysis, London: Unwin Hyman.
Goodwin, J. (ed.) (2012) SAGE Secondary Data Analysis, London: Sage.
For more examples of case studies using secondary data, try: The UK Data Service, https://
beta.ukdataservice.ac.uk/impact/case-studies/
217
M09 The Practice of Market Research 31362.indd 217
27/09/2021 21:47
M09 The Practice of Market Research 31362.indd 218
27/09/2021 21:47
Part Four
Qualitative research
M10 The Practice of Market Research 31362.indd 219
30/09/2021 18:25
Chapter 10
Qualitative research methods
Introduction
In Chapter 3 we looked briefly at the nature of qualitative research and the way
in which it differs from quantitative research. The purpose of this chapter is to
describe some of the methods used in qualitative research and the applications
of these methods with examples to illustrate their use in context.
Topics covered
What is qualitative research?
● Ethnography
● Semiotics
● Interviews and group discussions
● Deliberative methods
● Online research communities.
●
Relationship to MRS Advanced Certificate Syllabus
The material covered in this chapter is relevant to Topic 1: Understanding the
research context and planning the research project; Topic 2: Guiding Principles;
and Topic 3: Selecting the research design and planning the approach.
M10 The Practice of Market Research 31362.indd 220
30/09/2021 18:25
What you should get from this chapter
At the end of this chapter you should be able to:
demonstrate awareness of a range of qualitative methods; and
● choose the most appropriate method/s for a given research proposal.
●
221
M10 The Practice of Market Research 31362.indd 221
30/09/2021 18:25
Chapter 10
Qualitative research methods
What is qualitative research?
Qualitative research is about rich, detailed description, understanding and insight
rather than measurement. It is both less artificial and less superficial than quantitative
research and it can provide highly valid data. It aims to get below the surface, beyond
the ‘top of mind’ response. It tends to be sensitive to the wider context in which it
is conducted; it is good at uncovering the subtleties and nuances in responses and
meanings as a result. Branthwaite and Patterson (2011) identify three key features
that make it ‘a unique and invaluable tool’ in a consumer market research context: it
is ‘a conversation – a direct dialogue with consumers’; the dialogue is underpinned by
active listening; it entails rapport between researcher and participant, ‘a “merging of
minds” . . . to achieve insights and possibilities that can be extrapolated to marketing
issues.’ Bailey (2014) notes that qualitative research can be recognised by the methods
it uses, ‘the researchers who offer expertise and knowledge to cover the procedures
they use and the interpretations they derive; a particular objective to answer “why?”
and “how” questions and an agency context . . . through which . . . clients can obtain
such work and services’.
Schools of qualitative research
There are many different approaches to qualitative research based on the philosophical standpoint of the researcher, that is, their ontological view and their epistemological view. Their ontological view is the view they take about the nature of
reality and what there is to know about it. They might take the realist view that
there is an external reality independent of our beliefs and understanding; or they
might take the idealist view that there is no such external reality (Ormston et al.,
2014). Their epistemological view is about the way in which they believe knowledge is created. In the context of research, this is about how we find out about the
world, how we get that knowledge – what sort of reasoning and logic we use (e.g
inductive or deductive reasoning); what the relationship is between researcher and
those we are researching; and what we see as ‘truth’. These different ontological and
epistemological views have led to different schools of thought or traditions of practice in qualitative research. Broadly speaking, there are two ‘schools’: the positivist
school (knowledge from empirical reality, objective, unbiased observation); and the
interpretivist school (there is no one objective truth). Gordon (2011) characterises
positivist qualitative research as ‘a rational forum/process of collecting information’
on the basis that ‘participants . . . have information that can be extracted through
asking direct questions.’ The interpretivist approach, which Gordon refers to as
‘the dynamic school of qualitative research’, is more likely to use the techniques of
observation, ethnography and semiotics, and to involve more collaboration between
researcher and participants than the positivist approach. This interpretivist approach
is probably the most common approach in market and social research. Taking this
approach you aim to understand the world through the perspective of the research
participant, gathering data from them about their lives, their view of the world,
their experiences, their opinions and attitudes, in their own words; and you use a
mix of inductive and deductive reasoning, usually informed by existing knowledge
and theory, in analysing the data. Pragmatism also comes into it: Seale et al. (2007)
222
M10 The Practice of Market Research 31362.indd 222
30/09/2021 18:25
What is qualitative research?
argue for a flexible approach in designing research best suited to the objectives of a
project. In other words, you choose the method most appropriate for delivering the
client’s information needs in the context of their business problem.
The stance of the researcher
The qualitative researcher designs the research, conducts the fieldwork and analyses
and interprets the data. The stance of the researcher is therefore important. Ormston
et al. (2014) note that the researcher should aim to ‘achieve an empathetic neutrality’. This entails avoiding ‘obvious, conscious or systematic bias’. This is difficult,
which they acknowledge, as ‘all research will be influenced by the researcher’. The
way they recommend working towards achieving it is to be ‘reflexive’. This requires
you to think about and reflect on your role in the research process and the influence
your beliefs and values have on it. This is important in ensuring the validity – the
credibility – and reliability of the research.
Sampling: representativeness and generalisability
The approach to sampling in qualitative research (we look at this in more detail in
Chapter 11), and the sampling criteria used to choose the sample, are decided by
the researcher responsible for designing the project. Sample sizes are typically small.
The number of interviews or groups or workshops conducted will depend on the
research objectives, the complexity of the topic, the range of views needed, and the
practicalities of time and cost.
There can be confusion about representativeness of a sample and generalisability
from the sample to the wider population in qualitative research. With such small
sample sizes and the use of purposive (rather than random) sampling methods, findings from qualitative research cannot be said to be representative in the statistical
sense, and they are not meant to be. Statistical representativeness is not a goal in
qualitative research, but generalisability is. The aim of most projects is to generalise
from the sample to a wider population, to be able to apply the findings beyond the
sample. To achieve this in a qualitative research project, you must make explicit
the relationship between the sample and the wider population from which it is
drawn; the sampling approach should be systematic and rigorous; and it should be
described clearly to enable clients and others to judge the reliability and validity of
the findings.
Qualitative methods
The wide range of methods that come under the heading ‘qualitative research methods’ reflects the heritage of qualitative research, its roots in the social sciences, in
particular in sociology, anthropology and psychology (Bailey, 2014), and its application in these and other disciplines. The range of methods fall into two broad groups:
observational methods (ethnography, semiotics), which Lawes (2018) refers to as
‘outside-in’ approaches; and interrogative methods (group discussions, in-depth interviews, workshops), which Lawes refers to as ‘inside-out’ methods.
223
M10 The Practice of Market Research 31362.indd 223
30/09/2021 18:25
Chapter 10
Qualitative research methods
In person or online?
Qualitative research can be conducted in person or online. Salmons (2016) suggests
that you might do online research for several reasons:
as the medium of communication (akin to choosing telephone research rather than
face-to-face);
● as the setting or venue (akin to deciding that the venue should be the participant’s
office or home);
● when the medium or setting are of interest as part of the ‘phenomenon’ or issue
the research is designed to investigate–in other words, if the aim of the research is
‘to analyse activities, experiences, and behaviours on or with ICTs’.
●
Online face-to-face group discussions, using software such as Zoom or Teams, or an
online in-depth interview via a WhatsApp video call, are examples of online as the
medium and the setting. An observation or ethnographic study of specific groups or
communities through observation and the analysis of online talk or conversation, or
other online material (Prior and Miller, 2012) is an example of where medium and
setting are of interest as part of the study (referred to as ‘netnography’ by Kozinets,
2002, and Verhaeghe et al., 2009, and also known as ‘virtual ethnography’). For
example, the study of online gamers of virtual worlds. Puri (2007) sees it as ‘a real
social context’; Coombes and Jones (2020) note its use in providing ‘holistic consumer understanding’. Xharavina et al. (2020) argue that it is ‘a relatively easy, costeffective and time-efficient approach’ for providing ‘qualitative market intelligence’.
Key things to consider in deciding between an online and an in-person approach
besides the nature of the enquiry and the suitability of medium and setting include
access to a connected device; internet access and the quality of the access (bandwidth,
speed and security of connection); and access to a suitable private space in which to
take part in confidential research. Of course, in circumstances such as a global health
emergency, it may not be appropriate on ethical grounds (the principle of no harm
and the safety of researcher and participant) to consider an in-person approach.
The advantages of remote fieldwork compared to in-person fieldwork include the
following:
scope to reach a more dispersed set of participants – geography is not an issue;
increased flexibility – participants can take part from wherever they are and times
can be set to suit their availability more easily than is the case with in-person work
where other factors such as venue and travel play a part;
● more efficient use of time and less expense – with no travel involved there are no
travel costs and no fees for hiring facilities;
● with permission you can record the fieldwork easily.
●
●
The limitations of remote rather than in-person fieldwork include the following:
the need for access to the relevant technology;
the need for technology skills and the confidence in those skills to take part;
● difficulty in verifying that participants are who they say are;
● the need for access to a space conducive to taking part in confidential research
(e.g. it must be comfortable, safe and secure for the participant, and relatively
quiet);
●
●
224
M10 The Practice of Market Research 31362.indd 224
30/09/2021 18:25
What is qualitative research?
poor quality of the interaction as a result of the technology and the ‘at a distance’
feel (e.g. issues with the connection including time lags, screen freeze, and difficulty
in building rapport because of a lack of or limited face-to-face contact);
● the risk that the technology might fail.
●
Limitations can be overcome or mitigated by having in place a rigorous and systematic research process. This should include providing clear guidance for researchers; making checks during sampling and recruitment; adhering to ethical, legal and
regulatory requirements; establishing and building trust and rapport with participants; and reviewing findings against other research. For example, you might do
the following:
learn from others who have done similar projects;
list potential problems and plan how to address them;
● specify minimum standards for the technology of the potential participants;
● brief participants and send relevant materials in advance;
● help participants with access needs and with tech skills, for example doing a trial
run and/or taking time at the start of the fieldwork to check that everything is
working and that the participant is comfortable with the set-up (location, technology, research);
● have in place other methods of communication in case the main tech should fail.
●
●
Whatever the method, it will have advantages and limitations. To determine which
is most suitable for the problem you are researching, you need to be clear about the
problem and the sort of evidence you need to address that problem. Once you know
that, you can evaluate the methods available and decide which will best deliver the evidence. You must be able to justify your choice to the client or end user of the research.
Industry Insight 10.1 is an example of a project that used a variety of methods to
understand an aspect of the Fake News phenomenon.
Industry Insight 10.1
What is Fake News?
Introduction
The terrifying consequences of Fake News have
been reported by news organisations in recent
years. Examples include the subversion of democratic elections in Kenya; fuelling inter-ethnic tensions amid genocidal conflict in Myanmar; and mob
violence incited in India via WhatsApp messages.
To move beyond talking to actively fighting global
misinformation, the BBC World Service began a
Beyond Fake News initiative. The BBC World Service Audiences team wanted to understand the phenomenon including why the ordinary citizen shares
Fake News without verification. The team wanted
to understand the psychological and socio-political
reasons behind the sharing behaviour. The research
findings would be used to underpin programming
initiatives, conferences and hackathons.
The research
The research approach had five phases as follows:
Literature review
Auto-ethnography
● In-home qualitative interviews
● Semiotic analysis
● Big data network analysis of Twitter and Facebook ecosystems.
●
●
225
M10 The Practice of Market Research 31362.indd 225
30/09/2021 18:25
Chapter 10
Qualitative research methods
Literature review
Social media network analysis
We conducted a review of existing academic
papers on Fake News and with research agency
Synthesis a media scan of the previous two years
of articles about Fake News across the three countries, Kenya, Nigeria and India.
One of the challenges in analysing Fake News is
that there are few sources that can be classified
as out and out Fake News sources. Our analysis was not about Fake News sources but about
sources which have published Fake News as identified by fact-checking organisations. We used
algorithms to establish a likely list of disseminators on T
­ witter and Facebook Audience Network. We mapped the Twitter network of Fake
News sources and amplifiers in each country and
used cluster analysis to understand agents sharing similar connections. This was harder to do
with Facebook as Facebook public data is harder
to source and is mostly unusable from an ethical
standpoint. Instead, we used Facebook advertising
data to understand strength of affinity between
audiences of Fake News sources and audiences of
legitimate news sources and built a network map
of these various communities.
Auto-ethnography
In collaboration with research agencies Third Eye
and Flamingo we recruited 80 people in India,
Kenya and Nigeria with a mix of social, political, age, gender and economic backgrounds. We
asked them to share what they found interesting
in their WhatsApp and Facebook feeds and what
they were sharing in their networks.
In-home qualitative interviews and semiotics
We visited people at home in a semi-ethnographic
approach to understand the person in full, to
foster trust, to establish how their histories had
brought them to the present point and to find out
how they were contending with societal and cultural forces around them. We showed participants
Fake News messages and asked whether or not
they found these credible and whether they would
share them. At the end of the interview, with
explicit and informed consent, we were permitted
to access and export their WhatsApp chats. This
access would not have been possible without the
rapport developed between moderator and participant and the personhood-centred design of the
interview. We conducted a semiotic analysis of the
Fake News messages from the auto-ethnographies
and the interviews.
Impact of the research
The findings informed the BBC’s Beyond Fake
News season with articles, documentaries and
series commissioned to bring the insights to
audiences around the world. The findings were
discussed in debates attended by policymakers,
activists, MPs, students and ordinary citizens.
The research conclusions were developed into
actionable technology-based solutions using
hackathons.
Source: Adapted from BBC World Service ‘Fake News: Addressing
the global disinformation problem’. Winner, MRS Awards 2019.
Used with permission.
Ethnography
Ethnography is a set of methods for studying and learning about a person or, more
typically, a group of people, in their own environment over a period of time. It can
be done in person and online. It usually involves observation and interviewing. The
researcher observes or participates or becomes ‘immersed’ in the daily lives of those
being studied to get a detailed understanding of their behaviour, circumstances and
attitudes. The overall aim may be to achieve a holistic description of the group or
set of people, or to provide a detailed description of specific issues or situations
or experiences within the wider setting, or to explore an unfamiliar issue or setting
226
M10 The Practice of Market Research 31362.indd 226
30/09/2021 18:25
Ethnography
or group. Desai (2007) cites five areas into which ethnography can provide a great
deal of insight: retail navigation; product development; lifestyles and cultures; urban
ethnography; and habitual actions. It is a widely used approach. Belk (2014) notes
that many ‘large consumer goods companies have . . . anthropologists to conduct
such [ethnographic] research’. Insights 10.2 and 10.3 are examples of the use of
ethnography where, among other things, it provides insights you may not be able to
get from other approaches.
While they can be expensive and time consuming to set up and conduct, ethnographic studies have a number of strengths:
they give you insights that you may not be able to get through interviewing alone –
for example, they can provide insight in situations where people might find it difficult to describe their behaviour;
● they allow you to see the ‘bigger picture’ – the social and cultural context of the
behaviour or activity in which you are interested;
●
Industry Insight 10.2
Experience of persistent pain
Introduction
The Pain Relief team at pharmaceutical company
GlaxoSmithKline (GSK) realised that while they
had a lot of quantitative data on those suffering
persistent pain they had very little knowledge or
any real sense of what it is like to live with such
a condition. They commissioned research agency
Flamingo to capture real-time insight.
The research
The research had three parts:
Digital ethnography with 19 sufferers of persistent pain in the UK and Germany. This
involved participants recording over the course
of a week how pain affects their daily lives and
their ways of dealing with it and completing a
pain diary noting how the pain made them feel
and how they addressed the situation.
● Hour-long online interviews with three relatives of sufferers in the UK and Germany.
● Professionally filmed, in-home ethnographies
in the UK and Germany with nine sufferers,
each lasting three hours, and with two relatives
lasting an hour and a half.
●
Using footage from the ethnographies and
interviews the research agency put together three
45-minute ‘documentaries’ on the following
topics:
Becoming a persistent pain sufferer
Coming to terms with the pain
● Living with persistent pain.
●
●
Communicating the findings
The agency showed the documentaries at an event
for GSK staff that included contributions from
healthcare experts. Almost all attendees agreed
that the event helped them build empathy with and
a better understanding of this group of consumers.
Actions taken
GSK has taken a variety of actions as a result of
the findings including the following:
adapting its clinical study design to explore the
impact of treatment on the needs of consumers
as outlined in the documentaries;
● adjusting its recruitment criteria for survey research with persistent pain sufferers to
include the different emotional stages involved;
● adapting its vision for treatment ‘pillars’ to focus
on consumer needs and condition management.
●
Source: Adapted from Flamingo and GSK, ‘Targeting persistent
pain sufferers’, MRS Awards 2019. Used with permission.
227
M10 The Practice of Market Research 31362.indd 227
30/09/2021 18:25
Chapter 10
Qualitative research methods
they allow you to see things from the point of view of the people involved;
they allow you to hear and see people describe and explain things in their own
words, in their own way;
● they allow you to see things happen – behaviour, activities and so on – in the setting and at the time they normally take place.
●
●
The role of the researcher in ethnography
The extent to which the researcher is involved with participants in an ethnographic
study can vary from complete observer (performing what is often called ‘simple observation’) to participant observer (participant observation) to complete participant.
Simple observation involves watching and recording people and activity, for
example in a supermarket, a bar, a hospital waiting area, or in a virtual world
(Boellstorff et al., 2012), whatever setting is relevant to the research. If the researcher
Industry Insight 10.3
Young men and HIV
Introduction
Of those living with HIV worldwide, almost a
fifth are in South Africa. Young women account
for two-thirds of new infections but men are overrepresented in AIDS deaths. This indicates that
men are not testing as much as women nor initiating treatment as readily. The Bill and Melinda
Gates Foundation (BMGF) approached us to
conduct research and develop interventions to
improve HIV testing and treating rates among
young men.
The research
We wanted to understand, from men’s perspectives, the circumstances of their lives and the barriers to healthcare. We started with ethnography.
Researchers spent ‘a day in the life’ with 18 young
men either at risk of contracting or already living
with HIV. Despite its prevalence, it is still stigmatised and we had to ensure the young men felt
comfortable sharing their homes and stories. The
researchers who conducted the ethnographies had
similar demographics to the young men so as to
encourage communication and rapport. We also
took care to explain how the men’s data would
be protected. The second phase of the research
comprised 58 in-depth interviews with young men
and 64 interviews with healthcare workers. The
third phase was a survey with a representative
sample of men in the two provinces in which the
qualitative research had been conducted.
The findings
The research uncovered a reservoir of trauma,
pain and vulnerability. The men see no benefit in
knowing their HIV status. They fear all they might
lose if diagnosed. Their lives are difficult: unemployment, violence and extensive experience of
bereavement are the norm. Their self-expression is
constrained by narrow gender stereotypes. These
contextual factors are the parameters within
which they operate and are critical for healthcare
providers to appreciate and empathise with. Three
of our main recommendations were as follows:
Adopt a pragmatic, harm-reduction approach
Make services as responsive and relatable as
possible
● Carefully support men in disclosing their status.
●
●
Source: Adapted from Ipsos, Population Services International
and Matchboxology ‘From “villain’’ to “vulnerable’’: re-writing
the story of South Africa’s men and HIV’. Winner, MRS Awards
2019. Used with permission.
228
M10 The Practice of Market Research 31362.indd 228
30/09/2021 18:25
Ethnography
is present, they do not interact with those being observed but make notes about the
behaviour, incidents and routines (and might also record the activity). For example,
in a bar, the researcher might note the demeanour and body language of people
coming into the bar, the way in which the bar staff greet them, the time taken to
choose a drink, the drink chosen, the seat chosen and so on. If the researcher is not
present, the activity may be recorded and this record viewed and analysed later.
Observation allows the researcher to gather data on what people do rather than
what they say they do. To understand why the participant behaves in a particular
way, the recording may be played back to them as a reminder, and the researcher
may ask about the activity, and the participant’s thoughts and feelings at the time.
This is a technique described as a ‘co-discovery interview’ (Griffiths et al., 2004) or
reflective ethnography.
Participant observation is when the researcher is involved in all or part of the activity or task being observed. The extent of participation may vary. The researcher may
adopt the role of ‘observer-as-participant’ (Junker, 1960 and Gold, 1958 quoted in
Hammersley and Atkinson, 1995), limiting the amount of involvement or engagement
with the research subjects and focusing on observing. Alternatively, the researcher
may adopt the role of ‘participant-as-observer’, participating in the activities of the
people being researched. In both cases, those involved are aware of the researcher
and the researcher’s role. The main difference is the ‘stance’ of the researcher: in
the ‘observer-as-participant’ role the researcher is relatively detached and remains
at a distance from the subjects; in the ‘participant-as-observer’ role the researcher is
less detached, more engaged and involved. Accompanied shopping is an example of
‘observer-as-participant’ observation – the researcher goes with the participant on a
shopping trip, listening, observing and/or recording the behaviour on audio or video,
and making notes. The researcher may ask questions for clarification or understanding and to note the participant’s thoughts and feelings – collecting data relevant to
the research objectives.
The researcher may adopt the role of ‘complete participant’. In this case their role
as a researcher is concealed from the subjects of the research. This type of research
is sometimes known as ‘covert observation’. The researcher joins (or is already a
member of) the group under study, posing as an ordinary member but with the aim
of conducting research. This approach is more common in academic (sociological
and anthropological) research studies than it is in commercial research. It has been
used to study secretive or ‘hidden’ groups (Renzetti and Lee, 1993), religious cults
and criminal gangs, for example, or elite groups, those unlikely to let researchers in.
While on the one hand this approach might be the only way to obtain data, and it
offers a way of getting ‘inside knowledge’ untainted by the ‘observer effect’, it comes
with major ethical drawbacks: the nature of the approach means that you cannot ask
for informed consent before research begins.
An ethnographic study, like any other qualitative research method, involves a
systematic and rigorous approach. A study should begin with a clear statement of
what it seeks to achieve, a description of the population or group to be studied, how
this population or group relates to the aims and objectives of the study, and how a
sample of this population or group is to be chosen. A research plan or guide should be
drawn up that sets out what is to be done during fieldwork, what role the researcher
will take (observer, observer-as-participant and so on), and how long it is estimated
that fieldwork will last. During fieldwork the researcher must record (notes, photos,
video) the detail of what they are observing, creating ‘an archive of the context’
229
M10 The Practice of Market Research 31362.indd 229
30/09/2021 18:25
Chapter 10
Qualitative research methods
(Ger, 2014). Once the fieldwork/immersion stage is completed, when the researcher
sees that ‘new data stop adding new information or new questions, and the findings
begin to repeat themselves’ (Ger, 2014), the researcher leaves the field. The data are
then further reviewed and analysed, ‘comparatively, systematically and iteratively’
and a report of the findings prepared. Industry Insight 10.4 contains further examples
of the use of ethnography in producing detailed, context-sensitive understanding of
how consumers behave and make choices.
Industry Insight 10.4
Ethnography in action
Contextual technology – user interface
design
The term ‘contextual inquiry’ is often applied to
this intensive ethnographic exploration of workplaces and home environments whose objectives are better to understand the needs and
work processes around which technology can
be woven. Another use of ethnography in technology product development is to improve the
computer–human interface and thereby enhance
the usefulness, enjoyment and effectiveness of
anything from software to websites. In their
early stages, these research efforts tended to be
laboratory based; however, the limitations of
this rarefied context quickly became evident. The
emerging preferred alternative is to go to homes
and businesses to observe productivity and onscreen navigation in their natural context – on real
consumer-purchased and customised machines. In
this environment, consumers’ expressed attitudes,
observations of their interactions with computers as well as careful examination of surroundings, such as Post-It Notes attached everywhere
and pen and paper resources coexisting with
computers, become redolent with meanings and
opportunities.
Source: Adapted from Mariampolski, H. (1999) ‘The power of
ethnography’, International Journal of Market Research, 41, 1,
pp. 75–87.
Shopping for frozen food
Iceland, the frozen food retailer, was devising
a new brand and communications strategy. It
wanted to know how shoppers behave and what
implications this has for Iceland. It commissioned a multi-method study comprising ethnographic research, online qualitative forums and a
survey. The ethnographic study involved a small
sample of ‘typical’ shoppers in three UK cities.
The researchers visited the shoppers’ homes and
accompanied them on shopping trips. The aim
was to gain insight into what shoppers do and
why they do it.
Source: Adapted from Simms, J. (2018) ‘Changing perceptions of
Iceland’, Impact, 22, pp. 38–42. Used with permission.
A car designed around you
Car maker Volvo claims its cars are ‘designed
around you’. Andreas Strasser, head of research
and automotive strategy at Volvo, says ‘That
means understanding [the customers’] needs,
even if they can’t articulate them properly –
that’s key to us delivering a product that is more
than competitive’. Once the company identifies
an opportunity for a new car, Strasser’s team
and the product definition team start to define
it and understand what customers would want
from it. This involves extensive desk research
followed by extensive exploratory research. The
exploratory research includes ‘ethnographic
immersion’. Those involved in the project,
including engineers, visit customers around the
world in their homes – to look at their lives, talk
about their relationship with their car and drive
around with them. The customer interviews are
led by local moderators. The team learns a great
deal using this approach. On one project they
flagged and avoided a number of design errors
230
M10 The Practice of Market Research 31362.indd 230
30/09/2021 18:25
Ethnography
that Strasser says would almost definitely have
been made if they had relied on other data. ‘We
had certain preconceptions of what was important to consumers, which were ultimately disproved . . . If we hadn’t picked up on that . . . we
wouldn’t have noticed until the first customer
clinic, 18 months in, that we were delivering the
wrong car.’
Source: Adapted from Morgan, B. (2017) ‘Safety in numbers’,
Impact, 16, pp. 39–42. Used with permission.
The observer effect
Knowledge of being observed may alter the behaviour of those being observed to
some extent. It is important to be aware of this observer effect and to plan to minimise it – at the design and fieldwork stage – and take it into account at the analysis
stage. The main way of minimising it is to make the participants comfortable with
the notion of being observed. Here are some ways in which you might do this, at
various stages in the course of the project:
At the fieldwork stage
Briefing participants about the process and the end use of the data, being as transparent and open as possible (see below).
● Giving a general overview rather than a precise description of the purpose of the
research – so as not to influence or bias participants’ behaviour by alerting them
to the activity that you want to observe.
● Allowing time for participants to get used to the idea of being observed – by a
camera or a researcher or both (after a period of time they may revert to their usual
routines and behaviour).
● Giving participants control of the observation – giving them the camera with which
to film themselves or having a camera that they can turn on and off.
● Showing participants your notes – to allay any fears about the sorts of things you
are writing about them.
●
At the fieldwork/analysis stage
Asking them about things you have observed – to get their view about how typical
such things are.
● Showing them the recordings you have made and asking for comment on or evaluation of what they see.
● Observing them in a variety of settings and with different sets of people – to see if
there is any variation in how they behave or the way they approach things.
● Recognising that the observer effect will have had some impact on at least some
of the data you have collected.
● Noting when and where the observer effect occurs or is most prominent and thinking about why this might be the case.
● Thinking about how relevant these effects are in relation to the research
objectives.
●
231
M10 The Practice of Market Research 31362.indd 231
30/09/2021 18:25
Chapter 10
Qualitative research methods
Semiotics
Semiotics is the study of how meaning is created and how it is communicated. Its
foundations (Hervey, 2018) lie in the works of Ferdinand de Saussure (2013) and
Charles Peirce (Bellucci, 2020). It is applied in a research context to explore, understand and interpret or ‘decode’ the meaning of signs and symbols (including language
and music); to understand how these signs and symbols are connected to meaning
(Lawes, 2018) and to ‘shed . . . light on the activity of meaning-making itself and
the stages that are involved in this process’ (Barnham, 2019). It can be applied to
understanding human cultures, what they are made from and how they work (Lawes,
2009). You examine the semiotic signs and this semiotic analysis should show how
the signs are connected ‘with the expectation of being able to discover something
about the system that produced them’ (Lawes, 2018), what they ‘reveal . . . about
the social, cultural, and ideological landscape in which they occur’ (Lawes, 2019).
For example, Kaushik and Sen (1990) describe how the colour yellow is associated in India with ‘life-giving, auspiciousness and . . . vibrancy’; when used in a
sunflower oil ad it ‘becomes the signifier that connects the goodness and the light
quality of the cooking oil with the life and health-giving qualities of the sun’s rays
and the sun-kissed flowers’. Semiotics is often used to decode and understand visual
communications used in advertising (Harvey and Evans, 2001), media (Clough and
Macgregor, 2003) and packaging. You can do semiotic analysis on a synchronic and
a diachronic basis (Lawes, 2009 and 2019). A synchronic semiotic analysis is the
analysis of ‘representations . . . made at the same time but in different contexts, in
other circumstances, or even other countries’; a diachronic analysis is the analysis of
how representations, including the meaning of signs and symbols and linguistic practices, change over time. Lawes (2009) notes that this makes it useful in understanding likely future trends. Barnham (2019) suggests integrating semiotic theory (which
works at the level of culture) and qualitative methodology (working at the level of
the individual consumer) in what he calls ‘Qualitative Semiotics’, an approach which
‘allows us to understand . . . how individual consumers make their own meanings and
how these may . . . differ from the concepts of others’. Industry Insight 10.5 shows
how a semiotic analysis is carried out.
Industry Insight 10.5
What’s wrong with taking the bus?
Introduction
Stage 1
The client had the idea that there are various
prejudices and false beliefs that discourage people
from getting the bus. They wanted to use semiotics to find out what these prejudices are and where
they come from.
Stage 1 begins with a brainstorming session. We
free associated on buses – what kinds of things
they are, who uses them, what sorts of things
happen on buses. We drew on every resource we
could think of: songs, jokes, TV entertainment,
232
M10 The Practice of Market Research 31362.indd 232
30/09/2021 18:25
Semiotics
things in the news, personal experiences. We
looked to see what our pool of ideas had in common and organised them into themes. Then we
did some data searching to find out if there was
any cultural evidence for these themes: were they
just things we had made up among ourselves or
were they recognisably part of the cultural world?
While data searching, we made a collection of
pieces of text and images from a range of sources
including the internet, TV, newspapers, magazines
and even the children’s section of the local library.
The set of themes we were looking for changed
slightly as we grew more familiar with the cultural landscape. We discovered which themes
were culturally prominent and rich in detail, and
which were impoverished and more in the cultural
background.
Stage 2
Stage 2 of the research process is where the collected materials are analysed. Semiotics can be a
fairly technical activity. When you look at a piece
of cultural material – a bit of advertising, say, or
a news report – you have to come equipped with
a set of tools for dismantling and making sense of
what you see. Some of the things in the semiotic
toolkit are as follows:
Visual signs
● Linguistic signs
● Aural signs
● The implied communication situation
● Textual structure
● Information structure
● Visual emphasis
● Genre
● Binary oppositions and contrast pairs
● Communication codes.
●
The semiotic toolkit helps you think in an organised way about what you are looking at, and
notice similarities and differences in the data
within a category or sector. In the buses project,
one of the themes we noticed had to do with the
fear of crime. We collected some data – various
stories and images that articulated this fear – and
our Stage 2 analysis using the semiotic toolkit
revealed some interesting things about the nature
of this concern. If you look at culturally available stories about bus travel, the fear is of a specific type of crime – violent physical attack as
opposed to pickpocketing, say, or deception. The
stories about this type of attack share the same
language . . . and they share some interesting narrative conventions. For instance, it’s interesting
to look at how teenagers are described in these
stories. In a story taking place at the bus stop a
teenager is more likely to be the victim than the
assailant, but in stories where attacks happen on
the bus, teenagers are the assailants while the victims occupy another category, akin to ‘ordinary
British citizens’.
Stage 2 where you deploy the semiotic toolkit
is crucial. We did not just say to our client ‘here
are the prejudices and false beliefs you wanted to
know about’. Because we had done some close
analysis we were able to provide detailed insight
into these culturally available themes and narratives. This was useful because it gave the client an
idea of what they were up against, for instance,
which of these ‘prejudices’ were most amenable
to change.
Another approach
The bus client came to us with a fairly open brief
along the lines of ‘find out this about British culture’. On other projects the client asks a specific
question to do with a particular piece of packaging or an advertising campaign or whatever. In
such a case Stages 1 and 2 are collapsed together.
We analyse the materials in detail to see what we
are dealing with but all the brainstorming and
data searching goes on at the same time so that we
can form an accurate impression of the cultural
context in which the target materials are situated.
Whatever the details of the project, semiotics is
always a formal activity with a distinct set of tools
and a research procedure.
Source: Adapted from Lawes, R. (2002) ‘De-mystifying semiotics:
Some key questions answered’, MRS Conference. Used with
permission.
233
M10 The Practice of Market Research 31362.indd 233
30/09/2021 18:25
Chapter 10
Qualitative research methods
Interviews and group discussions
What distinguishes qualitative interviewing from quantitative interviewing is the style
of the interview. Quantitative interviews are standardised – the questions are worded
in exactly the same way and asked in the same order in each interview – and most of
the questions are structured rather than open ended and non-directive. Qualitative
interviews are more flexible (Sampson, 1967 and 1996), more like ‘guided conversations’ (Rubin and Rubin, 2011) or ‘conversations with a purpose’ (Burgess, 1984).
The choice of interview (or discussion) as the method of data collection rather than
ethnography or observation will be driven by the nature and objectives of the research
and by the practicalities of time and cost.
In-depth interviews
In-depth interviews are conducted by a qualitative researcher on a one-to-one basis
with a participant who has been chosen according to the agreed sampling criteria for
the project. As the name suggests, the aim is to explore a topic in depth, and most
in-depth interviews will be in the range of 45 minutes to 2 hours, depending on the
topic and what has to be covered. In most cases the researcher will use an open-ended,
non-directive interview approach, although this can be adapted to the medium, the
setting and the issue under investigation. Interviews may take place in person in any
suitable venue (the participant’s home, workplace, central location or viewing facility); by phone; using online methods (an app or webchat software, with or without
video); in a text or written format, for example by email or using a messaging app
or a blog or microblogging site such as Twitter.
Face-to-face interviews whether they are in person or done remotely are conducted
in real time, that is, the communication is synchronous. Synchronous communication
can also take place online, on the phone and in writing, for example using instant
messaging or via Twitter. Interviews can also be near-synchronous or asynchronous
using email, messaging or the software developed for bulletin board groups. These
approaches allow the researcher to conduct interviews with several participants separately, in parallel. They have other benefits too. With no pressure to respond immediately, participants can take their time to compose and reflect upon their responses,
and the interviewer has more time to consider them. As a result, the data generated
can be richer and more insightful – and more detailed and in-depth – than might
otherwise be achieved. The anonymity or perceived anonymity of the medium also
offers a number of benefits, enabling research on sensitive topics that may be too
embarrassing to discuss effectively face to face. The anonymity and the remote feel,
together with the almost self-completion nature of the method, can engender a high
level of honesty and openness as well as a willingness to express extreme and less
conventional opinions (Balabanovic, et al., 2003). The email or messaging approach
may appeal to those who prefer to communicate in writing or find it more effective
to do so. In addition, at the end of the interview you have a complete record of the
interaction, ready for analysis. On the limitations side, however, the quality of the
interaction will depend on participants’ ability to articulate their thoughts and feelings and to express these clearly in writing, and to do so in the time available. Having
to think about things and then write them down produces a different sort of data
234
M10 The Practice of Market Research 31362.indd 234
30/09/2021 18:25
Interviews and group discussions
from the sort you get when participants talk. It is likely to be more considered, less
spontaneous.
The decision about which approach to use should be taken in the context of the
research objectives and the population of interest. In-depth interviews are not an alternative to group discussions – they generate different types of data. They are appropriate for more sensitive subjects, for understanding in detail without the views of the
participant being influenced by what members of the group say, or what other members of the group might think of them if they were to report a particular attitude or
behaviour. Of course, similar problems can arise in an individual interview but they are
easier to read and disentangle when there is less ‘contamination’ or ‘noise’ from others.
Industry Insight 10.6 offers an example of the use of in-depth interviews and minigroup discussions on the sensitive topic of leaving money in a will.
Industry Insight 10.6
Leaving a legacy
Introduction
Cancer Research UK (CRUK) relies on its supporters to fund its work. A third of its income is
from legacies, money left to it in people’s wills.
Creating a campaign to encourage legacies is difficult. There is a reluctance to engage with this
sensitive topic which makes it hard to communicate and hard to research. We knew we needed to
devise a respectful approach to explore the topic
and develop creative outputs.
The research
The research comprised two parts: exploration
and development.
In the exploration phase we needed to do two
main things: understand in depth the process and
motivations of legacy giving; and introduce some
campaign positionings to understand what types
of language resonate, what messages and promises
inspire action and how to reframe legacy gifts. We
conducted six in-depth interviews, three with CRUK
legacy supporters and three with general supporters;
and two mini-groups each with six people, one with
CRUK supporters and one with non-supporters.
In the development phase we needed to talk
about attitudes, understanding, motivations and
barriers around the legacy journey. We needed to
do this to support the development of an impactful, effective marketing campaign. We conducted
six mini-groups: two with CRUK legacy supporters;
two with general CRUK supporters; and two with
non CRUK supporters. We introduced each creative
route and execution, exploring responses to understand the different interpretations that emerged.
Aware of our safeguarding responsibility to
help participants through these difficult conversations, and as ambassadors for CRUK, we wanted
to minimise and manage any distress that they
could provoke. We thus adopted three core principles of person-centred therapy in our moderation
style: congruence (being authentic and mentally
present, affirming all input as valid); unconditional positive regard (always holding the participant in a positive light, regardless of what they
share); and empathy (understanding the feelings
that participants are sharing and expressing them
as if they were our own). Using these principles we
were able to navigate these emotional discussions
in a way that we and participants found supportive and rewarding. We did not attempt to offer
advice or guidance beyond referring participants
to other resources if needed.
The new campaign was launched in F
­ ebruary
2019 and significantly outperformed previous campaigns with an increase in legacy pack
requests and in spontaneous consideration of leaving a legacy gift to CRUK.
Source: Adapted from Cancer Research UK ‘The power of
pledging: helping Cancer Research unlock the potential of
legacies’. MRS Awards 2019. Used with permission.
235
M10 The Practice of Market Research 31362.indd 235
30/09/2021 18:25
Chapter 10
Qualitative research methods
Variations on the in-depth interview
There are variations on the standard individual in-depth interview including minidepths, paired depths, triads and family interviews.
A mini-depth is a shorter version of an in-depth interview, lasting usually about
20 to 40 minutes, and is used to explore a specific, bounded topic.
Paired depths are when two people are interviewed together. The pair may consist
of two friends; two family members – partners, siblings, fathers and sons; two work
colleagues – whatever is suitable for the topic being researched. Paired depths are
useful for two reasons. First, some people, particularly children and teenagers, find
it less intimidating and embarrassing to be interviewed with someone rather than
alone. Wright (2015), for example, in a study of religious identity and consumption
among young British Muslims, interviewed men and women aged 17–19 years old
separately in groups of two or three. Secondly, the research objectives may mean that
it is necessary to determine what goes on in a decision-making process that involves
more than one person – for example, buying a car or choosing life insurance. It may
be important to find out who takes on what role, for example who is the influencer
and who is the buyer or the financier.
Triads involve interviewing three people simultaneously and may be suitable for
the same reasons as paired depths.
In-depth interviews are sometimes conducted with all or some of the family group,
either together or separately, or in combinations. The purpose of family interviews
is often to find out about elements of family life, decision-making patterns, rules and
relationships governing food, clothes, holidays and leisure, for example.
Industry Insight 10.7 is an example of the use of paired depth interviews. It
gives the rationale for choosing in-depth interviews and describes the mechanism
for recruiting the pairs of participants. We look at other aspects of this project in
Chapter 11.
Industry Insight 10.7
Talking to teenagers about sex
Introduction
The topics to be covered in this project – sex, contraception and pregnancy – are personal, not subjects for group conversation. Moreover, in a group
setting it is more tempting for respondents to exaggerate their sexual conquests. Equally, they may
hide their true feelings and experiences. In-depth
interviews are a more appropriate environment
in which to discuss sensitive subjects. However,
they can be intimidating for younger respondents.
Many marginalised teenagers (the sample for the
project) have issues with trust and confidentiality
in their lives. It is unrealistic to expect to build a
trusting relationship with a marginalised teenager
in a one-off in-depth interview. That having been
said, there are also drawbacks in conducting communication strategy development research (which
this was) in a series of one-on-one sessions with
the same respondents over time. There is a need
to balance the time taken to understand how people feel about a sensitive subject with the need
to gather data to address the research objective –
how to develop communication that has both
impact and immediacy.
‘Friend get friend’ paired depth interviews
Friendship pairs formed the core of the research.
The value of this method is that respondents feel
236
M10 The Practice of Market Research 31362.indd 236
30/09/2021 18:25
Interviews and group discussions
comfortable in the presence of their friends and
thus open up more easily. Secondly, their friends
act as a safety net, challenging any false statements they may make and, in some cases, even
volunteering information on behalf of their peers.
People have a whole range of different friends. In
order to provide a more sensitive environment for
discussing attitudes to sex, one respondent was
recruited and then asked to recommend the friend
with whom they most felt comfortable discussing relationship issues. ‘Friend get friend’ pairs
proved an open and constructive environment for
in-depth discussion with at risk teenagers.
Source: Adapted from Cohen, J. (2005) ‘Teenage sex at the
margins’, MRS Conference, www.mrs.org.uk. Used with
permission.
Semi-structured interviews
Semi-structured interviews are a sort of half-way house between qualitative in-depth
interviews and more fully structured quantitative interviews (Young, 1966 quoted in
Sampson, 1967). They are often used in business-to-business research. The interview
guide is more structured than is usual in qualitative research and interviews are carried out by interviewers trained in qualitative probing and prompting techniques but
who are not necessarily qualitative researchers.
Group discussions
A standard group discussion or focus group, as they are sometimes called, is usually
made up of 8–10 people (10–12 people in the United States) – small enough for a
manageable discussion and large enough to have a range of views represented. Participants are recruited according to criteria relevant to the topic under investigation. A
skilled qualitative researcher, known as a moderator or facilitator, guides the discussion. In some circumstances, depending on the nature of the topic and the objectives
of the research, the group may consist of 6–8 participants, rather than 8–10. The
smaller group allows the moderator to get a greater depth of response from group
participants. Smaller groups are often used to research sensitive topics, or when the
sample consists of children or teenagers – smaller groups are less daunting for participants and allow the moderator to spend more time on each participant. A group
usually lasts about an hour and a half to two hours, giving enough time to explore
a range of issues related to the research topic in some depth. Should it be necessary
to research the topic in greater depth, the duration of the group may be extended.
Groups can take place in a central location, for example a meeting room in a hotel
or at a viewing facility, or in the home of the person who recruited the participants,
or online using video-enabled software. Industry Insight 10.8 gives an example of the
use of group discussions by an online business.
Online group discussion approaches
Online group discussions can take a number of formats: email groups; online group
discussions; and bulletin board groups, also known as asynchronous online discussion forums (AODFs).
237
M10 The Practice of Market Research 31362.indd 237
30/09/2021 18:25
Chapter 10
Qualitative research methods
Industry Insight 10.8
A recipe for research
Gousto is a recipe box subscription service in the
UK. Customers choose a recipe from the ­Gousto
website or app and a box is delivered containing the ingredients in the right amount for that
recipe. To measure how recipes perform and
to build understanding of customer preferences
the company looks at feedback on its website as
well as data generated when orders are placed.
They combine these data with data based on
attitudes and attributes (e.g. favourite cuisine,
how many people they are cooking for) and this
gives the company information on what individual customers are most likely to want to eat.
However, the company also sees value in using
qualitative research for new product development. Here’s what insights manager Natalia
Paine says, ‘Focus groups are most important in
the early stages, when trying to explore an area
more broadly, . . . exploring particular customer
groups’ motivations and behaviours, or when trying out concepts for a new range of recipes to
understand what people’s actual needs and wants
are, along with how we are meeting them. Focus
groups offer context for what we see in comments
and through other means . . . In the [group] we
can probe the “why”’. In January 2018, the
company launched a range of meals under 600
calories called Boost and Balance. ‘We went out
to customers and tried to find out whether we
should call it “healthy January” or “vegan”, and
we called it “Boost and Balance” based on customer interaction.’
Source: Adapted from McQuater, K. (2018) ‘Relishing a
challenge’, Impact, 21, pp. 50–54. Used with permission.
Email groups
Email groups are group interviews rather than group discussions. There is no direct
interaction between group members; the interaction is with the moderator and with
the moderator’s account of the group’s responses. These ‘moderated email groups’
(a registered trademark of Virtual Surveys Ltd) work like this (Comley, 1999;
­Adriaenssens and Cadman, 1999): the discussion guide is divided into sections (so
that participants are not sent all the questions in one or even two emails); the moderator emails a set of questions to each of the group participants, between 10 and 20
per group, who send back their replies within an agreed time (usually within a day or
two). The moderator collates and analyses these responses (often with input from the
client), sends the next set of questions, collates and analyses those responses, and,
when all the questions have been asked, and responses received, produces a summary
document, which is sent out to the group for comment. There may be a further wave
of questions and interaction with the moderator, depending on the nature of the
project and the time frame, which can be up to two or three weeks. Email groups are
useful if you need to examine in some detail one or two topics or ideas at one time.
Because participants do not have to respond to questions immediately, and have to
type out their responses, there is time for reflection and deliberation.
Online group discussions
Online group discussions, also known as an online focus group (OFG), are conducted
in real time and seek to some extent to replicate a traditional in-person group. The
238
M10 The Practice of Market Research 31362.indd 238
30/09/2021 18:25
Interviews and group discussions
group convenes in a specially set up chatroom or using video-enabled or webchat
software such as Teams or Zoom. It will typically consist of between 6 and 8 participants – although Bruggen and Willems (2009) suggest 3 to 5 – and the discussion may last between an hour and an hour and a half. Participants are recruited
in advance and are sent invitations and log-in details. Participants take part in the
discussion simultaneously. A group can also be split up into breakout rooms for part
of the discussion, to conduct different exercises or view variations on stimulus material, for example. The moderator can share their screen so that participants can see
stimulus material, or material can be sent to them before or during the discussion.
Depending on the complexity of the tasks involved and the number of participants, it
is not uncommon to have two moderators in an online group or a moderator and an
administrator. For example, one moderator might run the main part of the discussion
while the other looks after or monitors participants, sending messages to encourage
those who have not replied to do so, or alerting the moderator to questions a participant might ask in a comments section. Industry Insight 10.9 gives an example of
an online group discussion.
Bulletin board groups
Another way of running a group discussion online is to use a bulletin board approach.
As with OFGs, participants are pre-recruited. This sort of online discussion –
because it takes place at different times and in real time – is sometimes known as an
Industry Insight 10.9
Time online
FlexPaths develops software to support organisations in implementing, managing and promoting
flexible working. It wanted to understand the
many perceived and actual barriers to adopting flexible working ahead of its next software
release. It wanted to use an approach that would
allow senior professionals to discuss the issues
their organisation faced. They needed to interact
with one another to assess current ideas, test new
software live and brainstorm solutions. Working
with research agency DigitalMR and the social
networking site LinkedIn it devised the approach
outlined here.
We set up and moderated six online focus group
sessions with 45 senior executives most of whom
were based in the USA or the UK. There was a
range of expertise in using flexible working. The
platform we used allowed participants to see and
hear the moderator, to type answers to the moderator’s questions and to chat among themselves.
Typed comments were captured by the technology
and transcribed automatically. Because the moderator used audio while the participants typed, they
could all respond simultaneously without having
to wait for each other to finish speaking. The moderator could also prompt those who responded and
get more feedback quickly. We were able to share
graphics and video to test websites and software
applications. The executives were familiar with
video conferencing technologies and found the
medium quite natural to work with. As a result of
the findings, the client prioritised certain features
of its software and the new edition serviced all of
the needs touched on in the research.
Source: Adapted from Michael, M. (2011) ‘A matter of time’,
Research, 543, August, pp. 30–1.
239
M10 The Practice of Market Research 31362.indd 239
30/09/2021 18:25
Chapter 10
Qualitative research methods
asynchronous online discussion forum (AODF). The moderator posts topics, questions and tasks on the bulletin board. Participants take part at their convenience over
an extended period of time (which allows for a greater amount of reflection than in
the OFG method). The discussion can run over several days, weeks or even months
and involve a ‘community’ of 10–30 participants who can reply independently and
take part in collaborative tasks. The moderator briefs participants about frequency
of viewing the bulletin board and responding to questions and comments, which
can vary depending on the nature of the research and the duration of the discussion
group. The platform used should facilitate an open-ended discussion between moderator and participants and between participants. It will include tools that enable the
moderator to design, post and modify the discussion guide; post new questions and
stimulus material for participants to view; send out instructions and information to
participants; and monitor the discussion and participants’ participation in it (including who is logged on, for how long, what comments they have made). Rolland and
Parmentier (2013) report the popularity of bulletin board groups in social media and
brand communities as they are easy to set up; allow access to ‘themed communities’
and the different ‘identities adopted by users of social media’. They advocate their
use as a way of understanding what ‘connects consumers to products or services, the
stability . . . of the connections, what motivates product choice, and the impact of
the views of others . . . ’.
Variations on standard group discussion format
A mini-group, as its name suggests, is a cut-down version of a group, with usually
about 4–6 participants rather than 8–10. It lasts an hour to an hour and a half –
rather than an hour and a half to two hours. Mini-groups are often used if the topic
is a sensitive one, or if it is particularly difficult to recruit participants.
An extended group, again as its name suggests, lasts about four hours (and
sometimes longer) rather than the usual one and a half to two hours. The extra time
means that the topic can be explored in greater detail. A wide range of stimulus
material can be examined and a variety of projective and enabling techniques can
be used. The moderator may also devote a greater amount of time, in comparison
with a standard group, to the group forming process, ensuring that the atmosphere
created is relaxed and safe – this often leads to a greater level of disclosure from
the group.
A reconvened group is one that is recruited to take part in at least two discussions, usually separated by about a week. The first deals with the basics of the topic,
explores the background to it and the more straightforward aspects of it. Participants
are briefed on a task that is to be completed in time for the next meeting. The group
reconvenes for the second discussion to impart their thoughts, feelings and experiences about the topic under investigation.
In the 1960s, Bill Schlackman (1997) devised a type of reconvened group discussion called a sensitivity panel. The idea is that the same group or ‘panel’ of
participants meet many times – Schlackman found that nine or ten sessions were
needed for the group to work productively. The aim is to build trust between moderator and participants, and between participants, and to encourage participants
240
M10 The Practice of Market Research 31362.indd 240
30/09/2021 18:25
Interviews and group discussions
to be themselves. Achieving a high level of trust helps the group reach a greater
level of awareness of and sensitivity to the topic under investigation and allows the
researcher to obtain more in-depth data. Time is taken to train participants in the
use of techniques including projective techniques, free association, stream of awareness and role play, among others. Patterson and Malpass (2015) view Schlackman’s
sensitivity panel as a precursor to online communities in that they share similar
principles.
A friendship group, consisting of pairs or groups of friends or family members, is
another version. This sort of group is often used when researching children or teenagers, or when examining a buying decision in which two or more people are involved
(for example a mortgage or a car).
Industry Insight 10.10 gives an example of a project in which a range of qualitative methods were used to research ideas for an advertising campaign for automobile
recovery company, AA.
Industry Insight 10.10
Testing ideas for ads
Introduction
As part of a complete marketing overhaul the AA
wanted to use an emotional brand-building advertising campaign to help acquire new customers.
The AA recognised that people are busy running
successful careers and families, keeping the show
on the road and relishing every minute of it. The
proposition for the new campaign was that the AA
will do whatever it takes to keep your show on
the road: the advertising would give the audience
a sample of the feeling they would get when their
car is fixed.
The research
Research agency Acacia Avenue designed an
approach to test three creative routes (each with
the same core message) developed to meet the new
marketing strategy. This comprised three elements
as follows:
●
Quick-fire paired interviews: each pair of participants saw only one of the creative routes (to
avoid distinction bias, the focus on details that
separate items when compared). The interviews
lasted no more than 20 minutes and covered
initial, spontaneous responses and an exercise
involving choosing three words from a list that
best fit the idea.
● Comparative quad discussions: short sessions
with four participants to explore the three
creative routes in a relative context. The word
choice exercise was used here too.
● Homework: participants were asked to send
three examples of ads they’d seen recently that
had surprised them or challenged their perceptions of the brand. The idea here was to give us
information on the ‘distinctiveness’ bar which
the AA ad had to reach.
The end use of the findings
The route chosen as a result of the research was
the ‘Singing Baby’. It proved very successful
for the AA, reversing declines in market share,
membership and revenue.
Source: Adapted from Acacia Avenue and The AA ‘From
sparkplugs to singalongs’. Winner, MRS Awards 2018. Used with
permission.
241
M10 The Practice of Market Research 31362.indd 241
30/09/2021 18:25
Chapter 10
Qualitative research methods
Deliberative methods
Several approaches have developed from group discussion: workshops and panels,
and juries and assemblies. What these have in common – and what makes them
different from conventional groups – is the extended amount of time available for
generating data and reflecting on them; and the more collaborative or participative
nature of the process. Some approaches combine different quantitative and qualitative methods to enable participants to reach informed decisions about the topic under
investigation. This might include group discussions, workshops, and/or individual
polling. Participants might be given information, feedback, and/or suggestions during
the process to inform and guide them. These methods are sometimes referred to as
deliberative methods. A deliberative event might be conducted over the course of a
few hours or several days with participants reconvened at the end of each period of
deliberation and polled to gauge the effect of the period of deliberation.
Workshops
Workshops can be used to generate ideas, to explore issues in detail and to solve
problems. Workshops tend to consist of about 15–20 people, sometimes more, and
often include clients as well as consumers. They typically last at least two hours and
may be run over the course of a day. During the workshop session smaller sub-groups
may break away from the main group to work on different aspects of an issue or
problem. Langmaid (2005) uses the term ‘collaborative inquiry’ to describe a workshop approach that he views as an alternative to the traditional group discussion.
He notes that a good summary of the approach is ‘doing research with rather than
on people’.
In Industry Insight 10.11 the client used a workshop to understand the ‘job to be
done’ in keeping children hydrated.
Industry Insight 10.11
Workshopping with kids and parents
Introduction
Twinings, the well-known tea brand, had never
entered the highly competitive market for kids’
drinks. Faced with the challenge of declining tea consumption the company wanted to develop its kids’
‘hydration’ concept, a logical extension, it believed,
of its Cold In’fuse range of drinks. It wanted to
launch a product that was grounded in real needs. It
therefore needed research to do the following:
understand the real-life hydration problems
that the product could address;
● understand how it could solve these problems
better than existing solutions and do so in a
way that was credible to parents and appealing
to kids;
● generate a compelling consumer proposition
and play to the strengths of the Twinings
brand.
●
242
M10 The Practice of Market Research 31362.indd 242
30/09/2021 18:25
Deliberative methods
Time was tight: to align with the product launch
timetable, research had to be completed within
five weeks.
The research
Twinings worked with research agency The
Sounds, the agency acting as part of the product
team. They devised the following approach.
Ethnography
The team conducted an ethnographic exercise
with parents and kids using a mobile app. This
enabled parents to identify and bring to life with
photos and text explanations the key jobs that
drinks perform for their children; it enabled the
children to explain their needs, preferences and
issues; and it enabled Twinings to see what happened as it happened in context.
Generating hypotheses
Based on the data from the mobile ethnography, the team developed hypotheses about the
role the new product, Kids In’Fuse, could best
play.
Hothouse workshop
The workshop approach was inspired by the
­Delphi Technique in which responses of a panel
of experts are used to identify and solve problems. Taking the view in this case that parents are
the experts, some of those who had taken part
in the ethnography were invited to a workshop.
The aim was to help the team understand the ‘job
to be done’ of kids’ hydration. Once the ‘job’ was
identified, the parents worked with the team and
other Twinings’ experts to co-create a concept.
Consumer testing
The team reviewed the output of the co-creation,
clarifying or finessing elements, and agreed the
focus for the concept. The agency used this to
write up the concept for consumer testing. In
tests with Nielsen BASES it scored ‘outstanding’
and a 72 per cent purchase intention among parents, putting it in the top 5 per cent for all drinks
concepts ever tested globally by Nielsen.
Source: Adapted from The Sound and Twinings, ‘How Twinings
infused a new kids’ drink with real-life needs and parents’
expertise’. MRS Awards 2019.
Panels
Qualitative panels are made up of typically around 15–20 people. They meet at
intervals and may stay together for an extended period (weeks or months). The panel
may have a theme – consumer or community consultation, for example. At each session a topic relevant to the theme is discussed. Participants may be briefed about the
topic in advance and/or topics may be revisited, allowing participants the chance to
consider evidence, examine possible options or courses of action, for example, and
the chance to reflect on how they feel before discussing their views with, or presenting
them to, other panel members.
Juries and assemblies
Citizens’ juries and assemblies work in a similar way. In Box 10.1, Deborah M
­ attinson
describes the citizens’ jury model used in the UK.
Citizens’ assemblies can also be conducted online with video-conferencing software, and with large numbers of participants, as Industry Insight 10.12 shows.
243
M10 The Practice of Market Research 31362.indd 243
30/09/2021 18:25
Chapter 10
Qualitative research methods
Box 10.1
How do you run a citizens’ jury?
Opinion Leader Research and the Institute for Public Policy Research jointly developed
the UK model in a series of juries. This model will tend to have the following defining
characteristics:
The jury is made up of 12–16 randomly recruited ordinary members of the public,
selected to match a profile of the local community.
● The jury is asked to consider a question or questions on an important matter of
policy or planning. This may be local or national.
● The jurors sit for four days, with moderators. They usually receive a preliminary briefing session.
● Jurors are fully informed about the question/s, receiving evidence and crossexamining witnesses. They can call for additional information and witnesses.
● Jurors can discuss the issues fully, interrogating witnesses, and deliberating among
themselves in pairs, small groups, and in plenary session.
● On the final day, they draw their conclusions, which are compiled in a report.
● The jurors submit their report to the commissioning body, which is expected to
respond.
●
Source: Adapted from Mattinson, D. (1999) ‘People power in politics’, International Journal of Market Research,
41, 1, pp. 87–95. Used with permission.
Industry Insight 10.12
Deliberating online
Introduction
In 2019, the National Centre for Social Research
conducted an online deliberative poll using videoconferencing software. The aim was to determine
views on the future of Britain and governance
post-Brexit. It was largely an experiment to determine the conditions needed for effective deliberation online. It replicated online what it typically
does in a physical location.
Preparation
It recruited 320 people through a random probability survey among its own panel of 3,000 people
drawn from the British Social Attitudes survey. It
ran a feasibility study on what software to use and
decided on Zoom. It did lots of development work
to build technological capacity in its own team
and participants. It sent a pre-session questionnaire to gauge opinions on the topics being discussed – immigration, food policy and consumer
rights – and followed that with briefing materials
on the topics.
What happened?
The event itself took place over two days, on the
hottest weekend in June. NatCen learned a great
deal. For example, the attrition rate was around
43 per cent, compared with 20–25 per cent typical of face-to-face deliberative polls. The weather
undoubtedly had an effect but they also recognised
that it is easier not to turn up to an online event
than it is to a physical one where hotel and travel
costs have been paid. They found that people
were willing and able to meaningfully deliberate
244
M10 The Practice of Market Research 31362.indd 244
30/09/2021 18:25
Online research communities
online, in a sustained way, with some saying they
enjoyed being involved from the comfort of their
own homes.
What was learned?
NatCen is refining the approach but feels confident to proceed now it has ‘proof . . . that it is a
way of having conversations at scale’. Issues to
be grappled with are digital exclusion and the tiring nature of staring into a screen all day. Shorter
sessions, spread over more days, could be part of
the answer as could asynchronous solutions where
individuals post thoughts over an extended period –
although this challenges the view that all participants need to be in the same ‘room’ at the same
time. While it may in the short term exclude some,
it is likely to include others who find it difficult to
attend a physical forum.
Source: Adapted from Simms, J. (2020) ‘Weighing it up’, Impact,
30, pp.14–17. Used with permission.
Online research communities
Online research communities, also known as market research online communities
or MROCs, are groups of people brought together on an ad hoc basis for a specific project or as a longer-term resource for conducting qualitative and/or quantitative research. In some respects, they are the online equivalent of a workshop or a
panel (Patterson and Malpass, 2015). Some refer to them as ‘Consumer Consulting Boards’ (Willems and De Ruyck, 2013). Community members may be recruited
(and screened) to be representative of a particular target population or they may
be a group of consumers with particular interests – in an issue, a product, a brand,
an organisation (Malinen, 2015). Bang et al. (2018) report their value as a tool for
listening to customers. There are various ways in which data can be collected. There
may be discussion forums (in the manner of those used in AODFs) or real-time conversations; members may be asked to keep a blog or diary, or to take photographs
or films of relevant activities and to upload them; they may be asked to take part in
collaborative exercises; the moderator may run mini-polls or surveys.
Recruitment to a community can take a number of forms: via social networks, via the
client’s website or database, using other marketing databases or email lists, using online
advertisements, and using traditional methods. In terms of size, a small community may
be made up typically of 30 to 80 members and a large community between 100 up to
1,200 members (Poynter, 2010). To operate as a community – with a sense of purpose
and identity – the group must remain together for a reasonable period of time. Longerterm communities typically operate for six months or more; short-term communities for
anything from a few days to three or four months. To be successful it is important that
the community is managed. There should be a community administrator or moderator
with a strategy for communicating with the community. This is likely to include a briefing (welcoming the participant to the community, explaining the purpose, its terms and
conditions, and how it will operate); log-in details; what is expected of members, the
ground rules – what is allowed and what is not allowed, what sort of tasks members
will be expected to complete, how often they will be asked to undertake something and
how quickly a response is expected, and how quickly they should expect a reply or other
feedback; and how the data collected will be used. The moderator should also have a
plan for any project that the community undertakes. For collaborations in MROCs to
be successful, Willems and De Ruyck (2013) suggest ‘empowering members to start
their own discussions’, making them ‘co-researchers’ and ‘co-moderators’.
245
M10 The Practice of Market Research 31362.indd 245
30/09/2021 18:25
Chapter 10
Qualitative research methods
Chapter summary
●
●
●
●
●
●
●
●
●
There are two main ways of collecting data – observation and interviewing. The
main advantage of interviewing is that the participant is recalling their behaviour
whereas in observation the researcher sees it at first hand.
Ethnography is a set of methods for studying and learning about people in their
own environment over a period of time. It typically involves both observation and
interviewing. Ethnography can be expensive and time consuming but it is useful in
providing a detailed and in-depth understanding of how and why people do things,
in the context in which they do them, and how they think and feel at the time.
Semiotics is a form of cultural analysis. It is used to explore, understand and
interpret or ‘decode’ the meaning of signs and symbols.
Qualitative interviews have been described as ‘guided conversations’ (Rubin
and Rubin, 2011). They are less standardised and more flexible than quantitative
interviews. They use a more open-ended and non-directive approach.
The main forms of interviewing in qualitative research are one-to-one in-depth
interviews, lasting about one hour, and group discussions with 8–10 participants,
lasting about one-and-a-half hours.
Individual interviews are used if the topic is sensitive; if you need detailed
information on individual attitudes and behaviour; if you need to get beyond the
socially acceptable view; if you need ‘timeline’ information; or if your sample is
difficult to find.
Group discussions are appropriate if you need to see a wide range of attitudes
and opinions; you need to determine differences between people; you do
not need minority views or views not influenced by the group; you want to
understand social and cultural influences; or you need to draw out creative
thinking/solutions.
Other interview-based approaches include workshops, panels and juries. What
these have in common is the extended amount of time for generating data and
reflecting on; and a more collaborative process.
Qualitative methods can be conducted online in synchronous and asynchronous
modes. Advantages include access to low penetration samples and widely
dispersed populations; and savings in time and costs. Disadvantages include
loss of interaction; and the need for access to a connected device, a good quality
internet connection and a suitable place to take part in research.
Exercise
1 Review Industry Insights 10.1–10.12. Identify the methods used in each and state
the reasons why that method was a suitable choice.
246
M10 The Practice of Market Research 31362.indd 246
30/09/2021 18:25
References
References
Acacia Avenue and The AA (2018) ‘From sparkplugs to singalongs’, MRS Awards.
Adriaenssens, C. and Cadman, L. (1999) ‘An adaptation of moderated email focus groups to
assess the potential for a new online (Internet) financial services offer in the UK’, Journal
of the Market Research Society, 41, 4, pp. 417–24.
Bailey, L. (2014) ‘The origin and success of qualitative research’, International Journal of
Market Research, 56, 2, pp. 167–84.
Balabanovic, J., Oxley, M. and Gerritsen, N. (2003) ‘Asynchronous online discussion forums’,
Proceedings of the Market Research Society Conference, London: MRS.
Bang, J., Youn, S., Rowean, J., Jennings, M. and Austin, M. (2018) ‘Motivations for and
outcomes of participating in research online communities’, International Journal of Market
Research, 60, 3, pp. 238–56.
Barnham, C. (2019) ‘Qualitative semiotics: Can we research consumer meaning making?’
International Journal of Market Research, 61, 5, pp. 47–91.
BBC World Service (2019) ‘Fake News: Addressing the global disinformation problem’, MRS
Awards.
Belk, R. (2014) ‘You are what you can access: sharing and collaborative consumption online’
Journal of Business Research, 67, pp. 1595–600.
Bellucci, F. (ed.) (2020) Charles S. Peirce: Selected Writings on Semiotics, 1894–1912, London:
Routledge.
Boellstorff, T., Nardi, B., Pearce, C. and Taylor, T. (2012) Ethnography and Virtual Worlds:
A Handbook of Method. Princeton: Princeton Press.
Branthwaithe, A. and Patterson, S. (2011) ‘The power of qualitative research in the era of
social media’, Qualitative Market Research: An International Journal, 14, 4, pp. 430–40.
Bruggen, E. and Willems, P. (2009) ‘A critical comparison of offline focus groups, online focus
groups and e-Delphi’, International Journal of Market Research, 51, 3, pp. 363–81.
Burgess, R. (1984) In the Field: An Introduction to Field Research, London: Allen Unwin.
Cancer Research UK (2019) ‘The power of pledging: helping Cancer Research unlock the
potential of legacies’, MRS Awards.
Clough, S. and McGregor, L. (2003) ‘Capturing the emerging zeitgeist: aligning The Mirror to
the future’, Proceedings of the Market Research Society Conference, London: MRS.
Cohen, J. (2005) ‘Teenage sex at the margins’, Proceedings of the Market Research Society
Conference, London: MRS.
Comley, P. (1999) ‘Moderated email groups: computing magazine case study’, Proceedings of
the ESOMAR Net Effects Conference, London.
Coombes, P. and Jones, S. (2020) ‘Toward auto-netnography in consumer studies’, International Journal of Market Research, 62, 6, pp. 658–65.
Desai, P. (2007) ‘Ethnography and market research’, International Journal of Market Research,
49, 6, pp. 691–92.
Flamingo and GSK (2019) ‘Targeting persistent pain sufferers’, MRS Awards.
Ger, G. (2014) ‘The art and science of ethnography’, International Journal of Market Research,
56, 4, pp. 553–56.
Gold, R. (1958) ‘Roles in sociological field observations’, Social Forces, 36, 3, pp. 217–23.
Gordon, W. (2011) ‘Behavioural economics and qualitative research – a marriage made in
heaven?’, International Journal of the Market Research Society, 53, 2, pp. 171–85.
247
M10 The Practice of Market Research 31362.indd 247
30/09/2021 18:25
Chapter 10
Qualitative research methods
Griffiths, J., Salari, S., Rowland, G. and Beasley-Murray, J. (2004) ‘The Qual remix’, Proceedings of the Market Research Society Conference, London: MRS.
Hammersley, M. and Atkinson, P. (1995) Ethnography: Principles and Practice, London: Sage.
Harvey, M. and Evans, M. (2001) ‘Decoding competitive propositions: a semiotic alternative
to traditional advertising research’, Proceedings of the Market Research Society Conference, London: MRS.
Hervey, S. (2018) Semiotic Perspectives, London: Routledge.
Ipsos, Population Services International and Matchboxology (2019) ‘From ‘’villain’’ to ‘’vulnerable’’: re-writing the story of South Africa’s men and HIV’, MRS Awards.
Junker, B. (1960) Fieldwork: An Introduction to the Social Sciences, Chicago: University of
Chicago Press.
Kaushik, M. and Sen, A. (1990) ‘Semiotics and qualitative research’, Journal of the Market
Research Society, 32, 2, pp. 227–42.
Kozinets, R. (2002) ‘The field behind the screen: using netnography for marketing research in
online communities’, Journal of Marketing Research, 39, pp. 61–72.
Langmaid, R. (2005) ‘21st century qualitative research’, Proceedings of the Market Research
Society Conference, London: MRS.
Lawes, R. (2019) ‘Big semiotics: beyond signs and symbols’, in International Journal of Market
Research, 61, 3, pp. 252–65.
Lawes, R. (2018) ‘Science and semiotics: what’s the relationship?’ in International Journal of
Market Research, 60, 6, pp. 573–88.
Lawes, R. (2009) ‘Futurology Through Semiotics, Proceedings of the Market Research Society
Conference, London: MRS.
Lawes, R. (2002) ‘De-mystifying semiotics: some key questions answered’, Proceedings of the
Market Research Society Conference, London: MRS.
Malinen, S. (2015) ‘Understanding user participation in online communities: a systematic
literature review of empirical studies’, Computers in Human Behavior, 46, pp. 228–38.
Mariampolski, H. (1999) ‘The power of ethnography’, Journal of the Market Research Society,
41, 1, pp. 75–87.
Mattinson, D. (1999) ‘People power in politics’, Journal of the Market Research Society, 41,
1, pp. 87–95.
McQuater, K. (2018) ‘Relishing a challenge’, Impact, 21, pp. 50–4.
Michael, M. (2011) ‘A matter of time’, Research, 543, August, pp. 30–1.
Morgan, B. (2017) ‘Safety in numbers’, Impact, 16, pp. 39–42.
MRS (2019) Code of Conduct, London: MRS.
Ormston, R., Spencer, L., Barnard, M. and Sharpe, D. (2014) ‘The foundations of qualitative research’, in Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (eds.)
Qualitative Research Practice, 2nd edition, London: Sage.
Patterson, S. and Malpass, F. (2015) ‘The influence of Bill Schlackman on qualitative research’,
International Journal of Market Research, 57, 5, pp. 677–700.
Poynter, R. (2010) The Handbook of Online and Social Media Research, Chichester: John
Wiley & Sons Ltd.
Prior, D. and Miller, L. (2012) ‘Webethnography towards a typology for quality in research
design’, International Journal of Market Research, 54, 4, pp. 503–20.
Puri, A. (2007) ‘The web of insights: the art and practice of webnography’, International
Journal of Market Research, 49, 3, pp. 387–408.
248
M10 The Practice of Market Research 31362.indd 248
30/09/2021 18:25
Recommended reading
Renzetti, C. and Lee, R. (eds) (1993) Researching Sensitive Topics, London: Sage.
Rolland, S. and Parmentier, G. (2013) ‘The benefits of social media: bulletin board focus
groups as a tool for co-creation’, International Journal of Market Research, 55, 6,
pp. 809–27.
Rubin, H. and Rubin, I. (2011) Qualitative Interviewing: The Art of Hearing Data, 3rd edition, London: Sage.
Sampson, P. (1967 and 1996) ‘Commonsense in qualitative research’, Journal of the Market
Research Society, 9, 1, pp. 30–8 and reprinted in 38, 4, pp. 331–9.
Saussure, de F. (2013) Course in General Linguistics, London: Bloomsbury.
Schlackman, W. (1997) ‘A discussion on the use of sensitivity panels in market research’,
Journal of the Market Research Society, 39, 1, pp. 1–13.
Seale, C., Gobo, G., Gubrium, J. and Silverman, D. (2007) Qualitative Research Practice,
London: Sage.
Simms, J. (2020) ‘Weighing it up’, Impact, 30, pp. 14–17. Used with permission.
Simms, J. (2018) ‘Changing perceptions of Iceland’, Impact, 22, pp. 38–42.
Verhaeghe, A., Schillewaert, N. and Van den Berge, E. (2009) ‘Getting answers without asking questions’, Proceedings of ESOMAR Online Research Conference, Berlin, Amsterdam:
ESOMAR.
The Sound and Twinings (2019) ‘How Twinings infused a new kids’ drink with real-life needs
and parents’ expertise’, MRS Awards.
Willems, A. and De Ruyck, T. (2013) ‘Collaboration with co-researchers in communities’,
International Journal of Market Research, 55, 4, pp. 587–9.
Wright, H. (2015) ‘YBMs: Religious identity and consumption among young British Muslims’,
International Journal of Market Research, 57, 1, pp. 151–63.
Xharavina, N., Kapoulas, A. and Miaoulis, G. (2020) ‘Netnography as a marketing research
tool in the fashion industry in Southeast Europe’, International Journal of Market Research,
62, 4, pp. 499–515.
Recommended reading
Gordon, W. (2016) MindFrames: 6 Enduring Principles from 50 Years of Market Research,
London: Acacia Avenue.
Keegan, S. (2009) Qualitative Research: Good Decision Making Through Understanding People, Cultures and Markets, London: Kogan Page.
Ladner, S. (2014) Practical Ethnography: A Guide to Doing Ethnography in the Private Sector, London: Routledge.
Lawes, R. (2020) Using Semiotics in Marketing, London: Kogan Page.
Kozinets, R. (2015) Netnography: Redefined, London: Sage.
Mariampolski, H. (2006) Ethnography for Marketers: A Guide to Consumer Immersion,
London: Sage.
Mason, J. (2017) Qualitative Researching, 3rd edition, London: Sage.
Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (2014) (eds.) Qualitative
Research Practice, 2nd edition, London: Sage.
Salmons, J. (2016) Doing Qualitative Research Online, London: Sage.
249
M10 The Practice of Market Research 31362.indd 249
30/09/2021 18:25
Chapter 11
Doing qualitative research
Introduction
The purpose of this chapter is to describe the practical aspects of doing
qualitative research, from recruiting or sampling participants to designing the
data collection guide and gathering the data. We look at analysing the data in
the next chapter.
Topics covered
Tasks and skills of the qualitative researcher
● Sampling and recruiting participants
● Designing the data collection guide
● Interviewing and moderating skills.
●
Relationship to MRS Advanced Certificate Syllabus
The material covered in this chapter is relevant to Topic 1: Understanding the
research context and planning the research project; Topic 2: Guiding Principles;
Topic 3: Selecting the research design and planning the approach; and Topic 4
Selecting an appropriate sample.
M11 The Practice of Market Research 31362.indd 250
30/09/2021 18:26
What you should get from this chapter
At the end of this chapter you should be able to:
demonstrate knowledge and understanding of the tasks and skills of the
qualitative researcher;
● demonstrate knowledge and understanding of sampling in qualitative
research;
● design an appropriate interview or discussion guide; and
● demonstrate knowledge and understanding of the skills required to interview
and to moderate group discussions.
●
251
M11 The Practice of Market Research 31362.indd 251
30/09/2021 18:26
Chapter 11
Doing qualitative research
Tasks and skills of the qualitative researcher
The qualitative researcher, unlike their quantitative counterpart, not only designs the
research but also undertakes the fieldwork and the data analysis. This includes briefing someone to recruit participants or recruiting participants themselves; designing
the data collection tool – a research plan for an ethnography or an interview guide
for interviews or group discussions as well as the materials for any tasks or activities
to be undertaken before and/or during fieldwork; and doing the fieldwork. Fieldwork
in qualitative research can mean conducting an ethnography (for example, going on
a shopping trip with the research participant, filming aspects of daily life relevant to
the research), moderating a group discussion (online or in person), running a workshop or doing an in-depth interview. In all of these fieldwork situations the task of
the researcher is to create an atmosphere in which participants feel at ease and are
willing to share. This requires observation and attentive listening; and relating what
is being said (and what is not being said) to the brief and the objectives, deciding
what to explore or follow up, what to clarify or challenge, when to re-iterate or sum
up and when to move on.
In the last chapter we noted Branthwaite and Patterson’s three key features of
qualitative research (2011): that it is a conversation; that it involves active listening;
and that it is interactive and requires a rapport between researcher and participant.
Branthwaite and Patterson elaborate on these features and this elaboration is helpful in
providing further insight into what is involved in being a qualitative researcher. First,
they talk about the importance of the conversation element and the fact that rules
govern conversations in social situations (Harré, 1979; Branthwaite, 1983). These
social situations include ‘everyday conversations’ and the ‘verbal accounts’ that make
up qualitative research. Schlackman (1959), quoted in Patterson and Malpass (2015),
calls it ‘casual conversation’. Rubin and Rubin (2011) refer to qualitative interviews
as ‘guided conversations’. These conversational rules we know intuitively within our
own culture. Outside our own culture, we may find that we are not so sure of them
(hence the reason you may find it difficult to interview in another culture). Next,
Branthwaite and Patterson talk about the ‘power of listening’ and how the necessary
attentive or ‘active’ listening is achieved through the researcher’s skill in the following:
creating an atmosphere of trust and acceptance;
being curious about other people and what they say;
● looking for ambiguities and alternative meanings;
● being aware of non-verbal expressions;
● attuning to the internal conversation the participant is having;
● identifying issues that may need probing or exploring; and
● checking for reliability by asking the same thing in different ways.
●
●
Finally, they talk about empathy. They define empathy as ‘the power of entering into
another’s personality and . . . sharing their experiences. [. . . ] to be able to think and
feel like them’. The skill involved in achieving this is to overcome ‘the constraints
and artificiality of the interview situation’ to get beyond the ‘rhetoric’ to the ‘personal’. Patterson and Malpass (2015) note that ‘if an interviewer has the ability to
understand and share the respondent’s feelings, they can build a feeling of respect
and comfort’. Gordon (2016) also identifies empathy as an important characteristic,
along with authenticity and respect.
252
M11 The Practice of Market Research 31362.indd 252
30/09/2021 18:26
Sampling and recruiting participants
Sampling and recruiting participants
In qualitative research, sample sizes are relatively small in comparison to those in a
quantitative research project. This is because the aims in a qualitative research project are not the same as those in a quantitative project. In a qualitative project you
are aiming to gather rich, detailed data from a small number of cases. Ritchie et al.
(2014a) note that there is a ‘point of diminishing return where increasing the sample
size no longer contributes to the evidence . . . ’. With such small sample sizes, probability theory and notions of statistical representativeness cannot and do not apply.
This does not mean that representativeness is of no interest or importance. This is
where confusion can arise. The aim of most research projects is to be able to generalise from the sample to the wider population or setting, to be able to apply the findings
beyond the sample. In quantitative research, methods of random sampling are used to
achieve a sample that represents the population from which it is drawn with a known
level of accuracy and precision (that’s the statistically representative bit). Qualitative
research projects do not use random or probability sampling methods (the sample
sizes are usually too small to support random sampling). To be able to generalise
from the findings of a sample in a qualitative research project, you use ‘purposive’
sampling. The researcher devises a sampling plan in which they define the population
of interest, make explicit the relationship between the sample and that population and
explain how this relates to the research objectives and to the design of the project.
The aim is to have a sample that includes a range of sample units or elements
with characteristics relevant to the topic under study and to the objectives. Choosing
sample units or elements on the basis of their relevance to the research problem, the
analytical framework or the explanation you hope to develop is a type of purposive
sampling known as theoretical sampling. The best-known version of this sampling
approach is that developed by Glaser and Strauss (1967). Applying the findings
beyond the sample in qualitative research involves three types of inference described
by Ritchie et al. (2014b) as follows:
whether what you find in the sample can be generalised to the population from
which it is drawn, or held to be true of that population – this is what Ritchie et al.
call ‘representational generalisation’;
● whether you can generalise from the findings or make inferences from them about
other settings or contexts, which they call ‘inferential generalisation’; and
● whether from your findings you can ‘draw theoretical propositions, principles or statements’ that can be applied generally, which they call ‘theoretical
generalisation’.
●
Sampling plan
When planning your project you should set out a sampling plan which covers all
aspects of the sampling process for your project and sets out the type or types of
inference you plan to use. To create a sampling plan you must do the following:
Identify and define the population of interest.
Define your sampling criteria, including the use of any quotas.
● Describe the types of inference you will use.
●
●
253
M11 The Practice of Market Research 31362.indd 253
30/09/2021 18:26
Chapter 11
Doing qualitative research
●
●
State the sample size.
Describe your sampling approach including where relevant:
– choice of geographic locations;
– source of sample (e.g. sampling frame such as a list, use of a database);
– use of recruiters and/or gatekeepers;
– method of selection.
As with all choices in your research design, you must be able to justify your sampling
approach to the client or end user of the research. Here are the questions you should
address in preparing a rationale for your choice:
Which sampling approaches did you consider?
Which one did you choose? Why?
● What are its strengths in relation to delivering a sample suitable for the research
objectives?
● What are its limitations?
●
●
– What are the implications of these limitations for the quality of the research and
the end use of the findings?
– How do you plan to/how did you overcome or mitigate these limitations?
What are the sampling criteria?
Why did you choose those criteria? How do they relate to the problem? How do
they relate to the analysis? How do they relate to any inferences you plan to make?
● What is the sample size? Why have you chosen that sample size? How does it relate
to the objectives, and to the analysis needs?
●
●
The practicalities of sampling
You begin the sampling process in qualitative research by identifying and defining
your study population and how your sample relates to the population. You need to
be clear about the links between the client’s business problem, the action or decision
they need to take, the information they need from the research in order to take that
action or make that decision, and who it is that can give you that information. Next
you set out the criteria on which the sample is to be selected. If you are devising a
sample for group discussions, you must set out the composition of each group.
Next you must decide how you are going to select the sample. We look at some
approaches below. In qualitative market research sampling is usually referred to
as recruitment and the specially trained interviewers who undertake it are known
as recruiters. The recruiter is briefed by a fieldwork manager or by the researcher
involved with the project. It is the recruiter who finds the sample based on your
sampling criteria and presents the research to them.
Permission and consent
A key element in any project, including a qualitative one, is the consideration of the
ethical, legal and regulatory issues that impact on how the project can be done. At the
sampling stage the focus is on obtaining permission from participants to take part.
If you need to use a recruiter, or indeed a gatekeeper, to gain access to the sample,
254
M11 The Practice of Market Research 31362.indd 254
30/09/2021 18:26
Sampling and recruiting participants
then potential sample members must give their permission for their details to be
passed on to the researcher. Professional codes of conduct set out rules and guidelines
about project design and sampling including permission.
It is your responsibility as a researcher to be aware of the ethical, legal and regulatory conditions that apply to your project in the country in which you are working.
This is an area in which legislation, regulation and oversight are increasing, and an
area which is increasingly complex. Doing qualitative research is fraught with ethical
dilemmas – things are rarely clear cut (Miller et al., 2012). As Salmons (2016) notes,
dealing with ethical issues is not an event, it is a process. You should consult an ethics
committee/ethics review board or someone with specialist ethics and data protection
knowledge and have your research proposal assessed before you even begin sampling.
This should ensure that your overall project design and approach including sampling
is robust from an ethical and data protection point of view.
At recruitment, a potential participant may be asked to sign a consent form. Before
signing such a form, the participant must be provided with information in clear and
accessible language about – among other things – who is conducting the research,
why it is being done, what it is about, and what taking part involves. You must check
again that the participant has given and continues to give their permission before you
begin data collection. Your organisation may have an information sheet and consent
form template which you can adapt for your project. Box 11.1 contains an example
of such a template for an in-depth interview. Before using any template or form make
sure you have the approval of your ethical review board or your organisation’s ethics
and data protection expert.
Box 11.1
Example: information sheet template
Introduction
My name is . . . . . . . . . . I am doing research on [topic]. . . . . . . . . . for [name of organisation]. I would like to invite you to take part but first of all I want to tell you about the
project and what it involves. This will take a few minutes. Please stop me if you have
any questions or there is anything you are not clear about. You don’t have to decide
now. You can take time to think about whether you want to take part or not. I can give
you this information sheet to take home. Included is the name of a colleague you can
contact if you have any questions.
Purpose of the research
The purpose of the research is to . . . . . . . . . .
Voluntary participation
Your participation is entirely voluntary. You can withdraw at any time, refuse to answer
or request that your information not be used in the project.
Why you have been chosen
You have been chosen because . . . . . . . . . .
255
M11 The Practice of Market Research 31362.indd 255
30/09/2021 18:26
Chapter 11
Doing qualitative research
What the research involves
The research involves taking part in an interview lasting about 45 minutes to one hour.
It covers questions about . . . . . . . . . . It takes place at . . . . . . . . . . Only you and the
interviewer are present. With your permission, the interview will be audio-recorded.
The interviewer will prepare a transcript of the recording but any data or information
that might identify you will be removed. The recording and the transcript will remain
confidential and will be stored securely – no-one except the interviewer will have access
to them. They will be destroyed within four weeks of the date of the interview. The
interviewer may use extracts or quotations from your interview in their report but these
will not be attributed to you – all information that might identify you will be removed.
MRS
The research is being conducted in accordance with the MRS Code of Conduct 2019
and the UK Data Protection Act 2018.
Questions
Is there anything you would like to ask me?
Contact
If you would like more information, or you have any questions later, you can contact
. . . . . . . . . . who knows about this project and will be happy to talk to you.
Thank you
Many thanks indeed for taking the time to consider this.
Sampling approaches
The practice of sampling or recruiting participants can take a number of forms: the
choice will depend largely on the nature of the sample – who the people are, where
they are and/or where you might find them, and what information you have on them
already. Usually a grid or matrix or a detailed list is drawn up using the sampling
criteria you have set, criteria that identify the types of people or organisations
relevant to the research. For example, they might be defined in terms of demographic
characteristics (age, social class, working status and so on) or they might be defined
in terms of usage of a particular product or service, or in terms of their attitudes, or
in terms of their experience, of an event or a process or an organisation – whatever
is relevant to the aims of the research. A combination of factors may be used. Care
should be taken not to over-specify, however, as this is usually unnecessary and
can make sampling difficult and expensive. If necessary, a recruitment or screening
questionnaire can be used to help recruiters find individuals who match the criteria
and to ensure a standard, reliable approach. This questionnaire may be administered
face to face, online or by telephone, by contacting people at home or on the street,
or at a specific place where the incidence of those likely to fit the criteria is relatively
high.
Recruiting at a specified site is a form of convenience sampling, sometimes known
as outcropping. For example, if we were looking for church-goers, we could recruit
256
M11 The Practice of Market Research 31362.indd 256
30/09/2021 18:26
Sampling and recruiting participants
near a church, perhaps at the time of a service. Recruitment could also take place
online – in the church-goers example it might be done via a church’s website or social
networking page. The client may provide a list of possible contacts or you might use
a specialist sample or list provider or a recruiter or fieldwork agency with their own
list or database. Sourcing a sample from a database or a list raises issues in relation
to data protection. The MRS Code of Conduct (2019) states that ‘where files of
identifiable individuals are used, e.g., client databases, members must ensure that the
source of the personal data is revealed at an appropriate point in the data collection’.
Using lists or sampling frames is sometimes referred to as list sampling. The quality of the sample will depend to some extent on the quality of the sampling frame
or list (e.g. how accurate it is, how up-to-date) and on the way in which you choose
people from the list.
If you are buying a research sample from an external source, you have a responsibility to do due diligence to find out how that sample was created. This includes
finding out how the sample provider sources hard-to-reach participants and/or vulnerable people and the measures they have in place for data protection, privacy and
data security as well as those to guard against harm.
Using a network of contacts and asking these contacts to refer you to others is
known as network sampling or snowball sampling. This method is useful if a list or
sampling frame is not available, or if the sample is difficult to find, for example a
low incidence or low visibility group. A disadvantage is that you may end up with a
sample made up of people with similar characteristics.
Piggy-backing or multi-purposing is another way of identifying a sample. It is a
useful strategy if individuals are expensive or difficult to find. At the end of a study
participants are asked if they would be willing to be recontacted to take part in further research. The decision to ask this recontact question should be agreed with the
client during the design of the original research. Recontact should only be made if
the participant gave permission during that initial data collection and the recontact
must be in line with assurances you gave at that time, including assurances about
who would recontact the participant, why they would be recontacted and by whom
(see MRS Code of Conduct, 2019). On recontact at the sampling stage of the new
study you must check the participant’s willingness to take part (and their suitability).
Participants for a qualitative study can also be recruited through advertising in
places, on sites, where the type of people needed are likely to be found. This can
include ads or notices on social media, on websites or search engine pages, in newspapers or specialist magazines or newsletters, or on posters placed in relevant locations
(cafés, museums, offices, fitness centres – with the permission of the owner/manager).
Those responding to an ad can be screened to check that they meet the sampling criteria. This is another sampling strategy that is useful if individuals are hard to find.
Industry Insight 11.1 gives an example of the approach taken to gain access to a
hard-to-reach population using network sampling.
Sample size
A common question in qualitative research is, how many participants, interviews
or group discussions do we need? In their paper, entitled, ‘How many qualitative interviews is enough?’ Baker and Edwards (2012) conclude with the answer
that Harry Wolcott gave them, ‘it depends’. A sample should be large enough to
257
M11 The Practice of Market Research 31362.indd 257
30/09/2021 18:26
Chapter 11
Doing qualitative research
Industry Insight 11.1
Finding teenagers at the ‘margins’
Introduction
To develop an effective communications campaign strategy on teenage pregnancy, the UK
Government Department of Health commissioned
research into teenage attitudes to sex and contraception. Teenagers ‘at the margins’ – a particularly high-risk group – were the target audience
and so the target population for the research.
Sample
The sample to be drawn from this population
was to include boys and girls of a broad range of
ages but with a core of 11–17-year-olds. It had to
include those teenagers identified as being most
at risk including:
those from social grades DE;
those from high-risk areas (from urban inner
city estates to rural seaside resorts);
● the sexually active and non-active;
● teenage parents;
● ‘looked after’ children;
● teenagers from minority ethnic groups.
●
●
Even with considerable experience, we soon found
that our existing recruitment network was not
going to do the job. We used the recruiters closest to the margins within our existing network
to find and train recruiters from ‘lower income’
backgrounds. In other words, recruiters recruited
new recruiters closer to the margins. Those new
recruiters then went on to recruit respondents.
New recruiters were given clear guidance
regarding appropriate codes of conduct. All recruiters were briefed on appropriate and sensitive ways
to approach respondents. Furthermore, the importance of the application of parental/guardian consent for all potential respondents was impressed on
recruiters. No respondents were allowed to take
part in the research without the requisite parental/
guardian consent. The success of the project was in
no small measure due to the enthusiasm and commitment of this dedicated recruitment team. In
addition, the close relationship between recruiter
and respondents enabled us to overcome some of
the particular challenges of recruiting teens to discuss the sensitive area of teenage sex.
The specific recruitment of ‘looked after’ children and teenage mothers required the involvement of experts and professionals in those fields.
Within Local Education Authorities there are
dedicated teams who work with ‘looked after’
children and teenage mothers, and who were
instrumental in recruiting those willing to take
part in the research.
Source: Adapted from Cohen, J. (2005) ‘Teenage sex at the margins’,
MRS Conference, www.mrs.org.uk. Used with permission.
give you the information you need to address the research problem clearly and
unequivocally, and large enough to include sub-groups of relevance to the topic
and to allow you to make meaningful comparisons. In choosing the sample size
you should be guided by your experience (or the experience of others) in similar
types of study or in similar areas or markets. One approach is to take a ‘rolling’ or
dynamic sample – in other words to sample until you reach ‘theoretical saturation’
(Bertaux and Bertaux-Wiame, 1981), until you are seeing or hearing nothing new
in the data. As Harry Wolcott put it to Baker and Edwards, ‘keep asking as long
as you are getting different answers’. This means you must monitor the sample
composition throughout the course of fieldwork to ensure that you are achieving
the sample you need.
258
M11 The Practice of Market Research 31362.indd 258
30/09/2021 18:26
Sampling and recruiting participants
Incentives or participation fees
When participants are being recruited it is fairly common practice, especially in market research, to tell them that they will receive a ‘thank you’ payment or a participation fee. In being told of it up front, it is a form of incentive to take part. Anecdotal
as well as research evidence suggests that this is a useful strategy – it saves money
by ensuring that those recruited turn up and it avoids the need for excessive overrecruitment and rescheduling. Although the size of the incentive does not cover the
cost of taking part, it shows participants that you value their time and realise the
inconvenience they have experienced. It is fairly common nowadays to find that people expect to be paid. According to the MRS Code of Conduct (2019), you must tell
participants who will administer the incentive; what it will be; when they will receive
it; and whether any conditions are attached. Incentives that require participants to
spend money to be redeemed are not allowed. The Code also states that client goods
or services, or vouchers to purchase these, must not be used as incentives for projects
conducted for research purposes. Some clients or funders, particularly in social and
academic research, do not have a tradition of paying attendance fees or incentives.
The venue for the research
In-person qualitative research is conducted in a variety of settings, including the
participant’s home, the workplace, a central venue such as a hotel, a specialised
research venue known as a viewing facility as well as online. The choice of venue will
be determined by a number of factors, including the following:
the nature of the research and the research objectives;
the availability and/or accessibility of a viewing facility or central venue;
● whether the client or funder wants to watch the discussions or interviews live or
remotely;
● the suitability of the venue for the topic under investigation or for the type of
participants – for example, a study of food preparation and cooking practices may be
more suited to a home environment; IT use at work may be better suited to the work
environment or a mock-up of it at a venue such as a viewing facility or a hotel or online;
● the type of interview – participants in individual interviews can feel uncomfortable
in a viewing facility;
● the culture of the country in which the fieldwork is being done – in some cultures
inviting people to a private home or to a viewing facility may not be appropriate;
● the ability to make the venue safe, comfortable and secure for all involved in the
research;
● cost – hiring a viewing facility can add considerably to the cost of a project.
●
●
Whatever the venue, in person or remote, the physical setting is important (Patterson
and Malpass, 2015). The participants must feel safe and comfortable in order to relax
and take part properly in the interview or discussion. For in-person research, you as
the researcher will have some control over this. For example, you can make sure that
it is neither too hot nor too cold, that it has good ventilation and adequate lighting,
that it is not noisy, has comfortable seats and so on. For research conducted remotely,
259
M11 The Practice of Market Research 31362.indd 259
30/09/2021 18:26
Chapter 11
Doing qualitative research
you can suggest to the participant that they choose a venue or space in which they feel
safe and comfortable, one that is private and free of noise or other distractions, if that
is possible. Whatever the venue, make sure participants have no difficulty in getting
to it. If it is a physical location, this may involve organising transport or ensuring
adequate parking facilities. If you are using an online meeting room, make sure the
participant is comfortable with the technology to access the room, has the details of
when and how to log in, and knows how to contact you or someone in your team if
there are problems. You might want to run a trial session to familiarise participants
with the set-up, reassure them and take away some of the fear that things might be
difficult, might not work or might go wrong.
Observers
You should discuss with the client at the research design stage whether there is a
need for the client or other research colleagues to observe fieldwork. The benefits of
observing should outweigh the disadvantages. One of the main disadvantages, which
we noted in the previous chapter, is that the presence of an observer changes the
dynamic and the participants’ responses in what is known as the ‘observer effect’. In
addition, as Rule 41 of the MRS Code of Conduct (2019) notes, ‘There are some situations where observers could adversely affect participants’ interests and/or wellbeing’.
It is therefore good practice to take time to reassure participants about the data collection process and to reassure them about the confidentiality not only of their personal
data but also of any contribution they make during the process. If you have agreed to
observers you should, as Rule 39 of the Code states, inform the observers about their
ethical and legal responsibilities. You should, as Rule 40 states, ‘clarify with participants the capacity in which observers are present; clients must be presented as such,
even if they are also practitioners and/or members of MRS’. If you find that there is
a chance that an observer may know a participant, which can happen in business-tobusiness and employee research projects, then you must tell participants before data
collection begins and allow them to withdraw from the research if they wish.
Designing the data collection guide
A major task in a qualitative research project is the design of the research plan or
fieldwork or data collection guide. The style of this plan or guide – the equivalent of
the quantitative researcher’s questionnaire – varies from a simple list of topics that the
researcher plans to discuss or explore, to one that has more structure, with a series
of questions under headings or topics. The style depends on a number of factors,
including the following:
the objectives of the research, for example an exploratory study may mean a less
structured approach;
● the method of data collection – individual in-depth interview, paired interview, an
interview during an ethnographic exercise, an accompanied activity, a group discussion or any of their variations – and whether face to face, telephone or online;
● the need for comparability between interviews or groups, for example if fieldwork
is shared between a number of researchers or is conducted in a number of countries;
●
260
M11 The Practice of Market Research 31362.indd 260
30/09/2021 18:26
Designing the data collection guide
the experience and knowledge of the researcher/interviewer – for example, an
experienced researcher with an in-depth knowledge of the topic may find it easier
to work from a topic guide, whereas a less experienced researcher might prefer to
have a more detailed guide;
● the house style or preference of the researcher or client – some clients, for example,
prefer a more detailed and structured guide.
●
A research plan or fieldwork guide for an ethnography should set out what is to be
done during fieldwork, what role the researcher will take (observer, observer-asparticipant and so on), and how long it is estimated that fieldwork/immersion will
last. It may also include, where relevant, a participant briefing document, a pro-forma
for note taking, a list of questions or an interview guide, a schedule for recording or
filming (if that is to be done) and a schedule for reviewing material with participants.
Any time plan for an ethnography should allow scope for flexibility as things may
arise during fieldwork which were not anticipated at the planning stage. As in other
forms of qualitative research, data are reviewed during fieldwork which can mean
making amendments to the research plan before conducting further fieldwork.
Research objectives to questions
To write a sound research plan or guide you need to understand what it is the client
needs. Once you are clear about the business problem and the research objectives you
should be able to identify topics and question areas and ways of framing questions
and other methods for eliciting the sort of data and information you need. You might
find it useful to set out the objectives in a grid like the one in Box 11.2 and map topic
areas, questions and tasks against specific objectives. In this way you can develop a
plan or guide that links to your objectives.
Box 11.2
Example: template for planning a guide
Research objective
Topic/question/task
Suitability for sample and method of data collection
Once you have an idea about the content of the plan or guide take some time to think
how it will work with the people in your sample. It must be engaging and involving
for participant/s and it must be suitable for whatever method of data collection you
plan to use.
261
M11 The Practice of Market Research 31362.indd 261
30/09/2021 18:26
Chapter 11
Doing qualitative research
Questioning style
Where most questions in a quantitative research interview are closed or pre-coded,
questions in a qualitative research interview or discussion tend to be open ended and
non-directive; projective and elicitation techniques are also used. The style or model of
interviewing most often used is called the ‘psycho-dynamic’ model – that is, it is ‘based
on the assumption that public statements may be rationalisations dictated by what participants believe interviewers want to hear, or believe they “ought” to say’ (Cooper and
Tower, 1992). The aim of this style of interviewing is to get below the surface, beyond
the rational response, to encourage participants to talk in depth and in detail about their
experiences, their attitudes and opinions and their thoughts and feelings (Cooper and
Branthwaite, 1977; Cooper and Tower, 1992). Open-ended and non-directive questions
allow participants to relate to the topic in their own way, to use their own language (and
not that of the pre-coded response) with little or no direction in how to answer from
the interviewer or moderator. Probing – using follow-up questions to clarify meaning or
to encourage the participant to answer in more depth or detail – for example, ‘Exactly
what happened next?’ and ‘Tell me more about that’ – is used extensively. Closed or
more precise questions can be used to establish clearly the context or ascertain particular
facts – for example ‘How much did you pay for it?’ or ‘How many times did you use it?’
Prompting is another way of encouraging the participant to answer: techniques include
repeating the question or rephrasing it; using non-verbal cues – encouraging looks, nods
of the head and pauses and silence, for example. It is important, of course, to keep a balance between encouraging participants to answer and leading them, or putting words in
their mouths – for example ‘I suppose you are sorry that you bought it’. It is important
to listen to how things are being said so that you can explore or probe or challenge with
follow-up questions. Some examples are set out in Table 11.1:
Table 11.1 Examples of follow-up questions
Feature
Example
Suggested follow-ups
Verb in first or second person;
singular or plural
I recycle as much as I can.
Is it just you who does the recycling?
We recycle as much as we can.
When you say, ‘we’ who do you mean?
You recycle as much as you can, don’t you?
Who does? You do or people do?
Active voice
I take the stuff to the recycling centre.
Does anyone else ever do it?
Passive voice
The stuff is taken to the recycling centre.
Who takes it?
Modality
I should do more.
You feel obliged to? You should
but . . .?
I could do more.
You could? What gets in the way?
Evidentiality
Generalisation
They must do more.
What happens if they don’t?
It’s common sense.
What do you mean by ‘common
sense’?
Obviously. Of course.
Why is it obvious?
Why do say that?
Everyone knows that.
Everyone?
I always buy the one you can recycle.
I never buy single use plastic.
I recycle everything.
Always?
Never? Are there any exceptions?
What do you mean ‘everything’?
262
M11 The Practice of Market Research 31362.indd 262
30/09/2021 18:26
Designing the data collection guide
Method
Simple
questioning
Asking/
remarking
P
U
B
L
I
C
Pressing
Sympathetic
probing
Play, drama
non-verbal
Projective
approaches
P
R
I
V
A
T
E
C
O
M
M
U
N
I
C
A
B
L
E
N
O
N
A
W
A
R
E
N
O
N
Layer
Output
Spontaneous
Immediate,
spontaneous
responses
Reasoned
conventional
Justifications,
explanations
Preconscious
Detailed
elaborations/
introspections
Concealed,
personal
Personal
admissions
Intuitive
Symbols,
imagination,
analogies
Unconscious
Repressed
attitudes, motives
Figure 11.1 A qualitative interviewing model
Source: Cooper, P. and Tower, R. (1992) ‘Inside the consumer mind: consumer attitudes to the arts’, International
Journal of Market Research, 34, 4, pp. 299–311. Used with permission.
Other useful questioning techniques include summarising and restating or reflecting
what the participant has said to clarify meaning, help understanding and build empathy, for example, ‘You feel upset about how they handled the problem.’
Projective and enabling techniques
Projective and enabling techniques – indirect forms of questioning that are deliberately
vague and ambiguous – are used in qualitative research to get beyond the rational
response to the ‘private’ and the ‘non-communicable’ (Cooper and Branthwaite,
1977). Schlackman (1989) notes in interviews where participants may find the material being covered ‘threatening’ and/or want to avoid sharing ‘genuine thoughts and
feelings which might be embarrassing’, they tend to give responses of the kinds he
calls ‘stereotype’ and ‘games/defences’, layers A and B in Schlackman’s model of
consciousness. To get beyond those layers, to gain an understanding of consumer
motivation, Schlackman advocates the use of projective techniques. The idea is that
participants will ‘project’ their ideas, thoughts, feelings, emotions and attitudes in
completing a task. In doing so, you get responses that participants may not have been
able or willing to give via direct questioning. In summary, according to Schlackman
(1989), these techniques can be used to do the following:
to help research participants discover and explore their thoughts, feelings and
experiences;
● to help them express these in a way that reduces anxiety and avoids embarrassment;
● to help them uncover new aspects of a particular issue; and
● to help the researcher interpret the data produced by the participants.
●
263
M11 The Practice of Market Research 31362.indd 263
30/09/2021 18:26
Chapter 11
Doing qualitative research
Pich et al. (2015) found that these techniques ‘can provide a greater understanding
into underlying feelings and deep-seated attitudes’. In a further study to investigate
the reputation of the UK Conservative Party, Pich and colleagues (2018) found them
‘essential’ in capturing, deconstructing and understanding current image and longterm reputation. As is the practice with administering the techniques, the researchers
encouraged participants to annotate the material they produced to provide ‘greater
explanation’. The researchers questioned the participants using a technique known
as ‘laddering’ (Branthwaithe, 2002) or ‘echoic probing’ with the aim of allowing
them to ‘explain their own expressions in their own words rather than lead or interrogate . . . or misinterpret . . . ’.
Types of projective and enabling techniques
There are several types of techniques – techniques of association, completion, construction and expressive techniques. Examples are given in Table 11.2.
Table 11.2 Examples of projective and enabling techniques and their uses
Name
Description
Uses
Word association
Participant asked for first word
that comes to mind when given
a particular word (spoken or
written down).
To explore connections, get
at language used, uncover
product, service or brand
attributes and images.
Picture and concept
association
From a large and varied
collection of pictures or
concepts participants asked to
choose which best suit a brand
or a service or product or its
users.
Thought or speech bubble
completion (balloon drawing)
Participants fill in what the
person depicted in a drawing
or picture might be thinking or
feeling.
To uncover thoughts, feelings,
attitudes, motivations and so
on associated with different
situations.
Sentence completion
Complete incomplete
sentences, for example, ‘If X
[name of organisation] was
really interested in protecting
the environment it would . . . ’.
To uncover top of mind or
dominant ideas or issues
associated with a product or
service or organisation.
Collage
Participants create a collage
from a pile of pictures; or
a collage or picture board
is compiled in advance of
fieldwork.
To uncover a mood, an image
or a style associated with a
product, a service, a brand,
an experience and so on.
Thematic apperception
test (TAT)
Participant given a picture and
asked to create a story about it.
264
M11 The Practice of Market Research 31362.indd 264
30/09/2021 18:26
Designing the data collection guide
Name
Description
Uses
Projective questioning
‘What do you think the average
drinker might think of this bar?’
Stereotypes
Develop a story about a person
or a picture.
To uncover beliefs, attitudes,
feelings, ideas that the
participant may not want or be
able to express directly.
Personification of objects/
brand personality
‘If this brand were a person,
what would he or she look like?
What would he or she do for
a living? What type of house
would he or she live in?’
Mapping
Sort or group brands or
organisations according to key
criteria; sort again on a different
basis.
To see how people view
a market; to understand
positioning; to identify gaps.
Choice ordering techniques
Given the ends of a scale, put
brands or products where they
fall along the scale.
To understand how people
see a range of products or
brands in relation to certain
characteristics and in relation
to each other.
Visualisation
Interviewer guides the participant
in thinking back to the last time
they did X or tried Y, to visualise
the scene in all its details.
To allow participants to recall
in detail an experience or a
situation and to bring to mind
thoughts and feelings about it.
Psycho drawing
Create a drawing.
To express thoughts and
feelings about a brand or
service or process that could
not be articulated in words or
were not top of mind.
Drawing objects
Draw object under study.
To project and examine ideas
about that object.
When choosing a technique it is important to make sure that it addresses the research
objectives and is suitable for the sample. If you plan to use these techniques in international research, make sure to choose those that are suitable for the culture. For a multicountry study, make sure they work in the same way in each country. Those suitable for
multi-country work include collage (make sure that the pictures chosen are relevant to the
country or market and check the meaning of signs, symbols and colours in each country);
word association; bubble and sentence completion; mapping; and personification.
Pre-tasks
Besides tasks that take place during fieldwork there are also those that are done
before researcher and participant meet. These are known as ‘pre-tasks’. Industry
Insight 11.2 describes the use of ‘pre-task’ journals.
265
M11 The Practice of Market Research 31362.indd 265
30/09/2021 18:26
Chapter 11
Doing qualitative research
Industry Insight 11.2
Understanding the lives of teenagers
Introduction
Understanding the lives of teenagers was critical
to the brief. This research was designed to find
concrete examples of what teenagers think, feel
and want. To this end, we asked participants
to fill in journals. The journals focused on four
areas:
respondents in order to generate high-quality preresearch journals.
Simple, visual, fun and cool
To achieve enough journals to represent the lives
of the teenagers, the entire sample was asked to
complete a journal. The expectation was that half
would do so, of which a small proportion would
be creative and imaginative. In fact, over 80 per
cent of the sample completed journals and the vast
majority of them were completed in style. That
someone should care enough about their lives to
pay them to put it down on paper was good for
their confidence and their belief in the value of
what they had to say. They expressed their lives
and attitudes with flair and creativity.
The aim was to make participants feel, from the
moment they first saw the pre-task, that it was
a fun challenge rather than a burden – designed
to enable them to express themselves rather than
something they might be made to do at school.
To achieve this, they received personalised notebooks and sets of stickers printed with simple
questions. Rather than giving them a rigid, predetermined order in which to complete their journal
with allocated spaces for answers, the idea was
to provide something more flexible and intuitive.
The notebooks themselves were carefully chosen
to ensure respondents felt good about the task,
without being of such good quality or so cool
that they didn’t want to mess about with them.
Written instructions were kept to a minimum. All
respondents were given cameras. Respondents
were encouraged to rip things out of magazines
and draw images in their diaries in order to give
the diaries a real sense of them as people and how
they felt. In many cases this was far more involving for respondents, and more interesting for us
at the analysis stage, than the formality of photos.
Approach
Benefits to the client
The approach that we took to getting the sample
to complete the journals was as follows:
Staying close
One of the key benefits of respondent journals is
that they provide a visual insight into the lives
of people that would otherwise remain hidden.
All involved with the client can get a real sense
of who the participants are and what matters to
them. They have a visual reality that a PowerPoint
presentation or written report simply cannot replicate. Because the journals were so powerful, the
client made a significant commitment in terms of
time and money to ensure that the most was made
of them.
It is impossible to overstate the importance of
recruiters staying close and providing support to
Source: Adapted from Cohen, J. (2005) ‘Teenage sex at the margins’,
MRS Conference, www.mrs.org.uk. Used with permission.
Their world: shops, brands, music, TV, magazines, radio
● The people in it: who they most care about,
respect and admire; who they turn to for advice
● Them: likes, dislikes, desires
● Hopes and fears: what they look forward to;
what they want to be doing in ten years’ time.
●
Stay close
Keep it simple
● Make it visual
● Make it fun
● Make it cool
● Involve others.
●
●
266
M11 The Practice of Market Research 31362.indd 266
30/09/2021 18:26
Designing the data collection guide
Group processes
Before we move on to look at how to structure an interview or discussion guide, it is
worth noting something about the phases that tend to occur in research encounters,
in particular in group discussions. Knowing something about these phases should
help you design a more effective data collection tool.
When people get together to form a group they tend to go through a number of
different processes or stages. These stages have been described by Tuckman (1965)
and Tuckman and Jenson (1977) as forming, storming, norming, performing and
mourning and are useful in the context of research group discussions. They usually
occur in this order, although some stages may be repeated during the discussion.
Forming
The forming stage of a group is the inclusion stage. It is important that the group
passes through this stage if it is to function properly as a group. The moderator must
explain the research, set out what is involved and get everyone to speak during the
first few minutes. This helps participants to get rid of their anxiety about speaking
and contributing, and allays their fears about being included and being a useful
member of the group. It is also important that group participants talk to each other
and not just to the moderator and so it can be very useful to run a warm-up or forming exercise. One way of doing this is to ask participants to pair off and introduce
themselves, then introduce each other to the group.
Storming
Storming is the stage the group works through to establish how to relate to one
another, to the moderator and to the task. At this stage participants will be sizing
each other up and trying to establish the boundaries of what is acceptable in the
group. They might challenge the moderator, for example, or another group member. It is at the storming stage that you should be able to recognise (and should deal
with) the dominant participant(s) and the quiet participant(s). Storming can happen
later in the group when new tasks or material are introduced. It is important at this
stage that the moderator reiterates that all views, positive and negative, are valid and
welcome and that they want to hear from all participants.
Norming
Norming is when the group settles down, when participants see that they have something
in common. A sign that it has happened is when there is a general air of agreement, and
when the atmosphere is noticeably relaxed. This is the stage at which the main work can
begin and so it is a good time to introduce or explain the key tasks to be carried out.
Performing
The performing stage is when the work is done. It is the high-energy stage. When the
group reaches this stage it is task orientated, co-operative and happy to get on with
things. This is a good stage at which to introduce more difficult or complex tasks.
267
M11 The Practice of Market Research 31362.indd 267
30/09/2021 18:26
Chapter 11
Doing qualitative research
Mourning
Mourning is the wind-down stage. It is an important stage to work through so that
participants can finish up and let go – of the task and the relationships that they have
formed within the group. To make sure that this phase is worked through properly
the moderator must signal the end of the group clearly and build in a wind-down
period. If participants are not given time to go through this stage they will not want
to finish and/or they can feel used – they may be left with the feeling that the moderator wanted them to complete a task, get information from them and get rid of them.
Signal the end of an hour-and-a-half-long discussion with about 15 to 20 minutes
to go. With about 5 or 10 minutes to go, ask some winding-up questions, such as,
‘Anything you’d like to say that you haven’t mentioned?’
Structure of an interview or discussion
The structure of an interview or group discussion is important in helping you run an
effective interview or discussion. There should be a clear introduction and ‘warm-up’
phase and a clearly signalled ending or ‘wind down’, in line with the group processes
outlined above.
The introduction
A clear introduction is vital from an ethical point of view and to put participants at
ease. It is likely that they will be nervous (Gordon and Robson, 1980), and it is the
researcher’s job to allay any fears they might have. In the introduction you should
do the following:
say something about yourself (your name, the organisation you work for);
tell the participant(s) about the topic and state the purpose of the research;
● tell the participant(s) how long the session is going to take;
● tell the participant(s) about your role as interviewer or moderator (that you are
independent, there to guide the discussion or interview, and to listen);
● tell the participant(s) how and why they were chosen;
● tell the participant(s) what to expect in terms of questions, discussion and tasks;
● give assurances about confidentiality and/or anonymity;
● ask the participants’ permission to record the session;
● inform the participant(s) about any observers and obtain their permission or allow
them to withdraw;
● tell the participant(s) how the information will be used;
● let the participant(s) know that their participation is voluntary and that they are
free to leave and free to refuse to answer any questions;
● mention the ‘ground rules’ (that there are no right or wrong answers, that it is not
a test, that it is the participants’ experiences, feelings, opinions you are interested
in, that all views are valid, and if you are moderating a group, that they can talk
to each other, that they do not have to agree with each other’s views).
●
●
268
M11 The Practice of Market Research 31362.indd 268
30/09/2021 18:26
Designing the data collection guide
The main body of the interview
It is a good idea to start the interview with relatively straightforward, general questions or topics that participants will find easy to answer or talk about – this helps
create a relaxed atmosphere and helps establish rapport between interviewer and
participants. It is possible then to move on to more specific questions or more difficult
topics. This technique is known as ‘funnelling’. The content and order of the questions should, of course, always be guided by the research objectives and there may
be times when you need to take a different approach and jump straight to the main
issue. Here are the types of questions that you might ask in the body of the interview
or discussion:
Broad, open-ended questions: ‘Tell me about shopping’ or ‘Tell me about a really
satisfying shopping experience.’
● Pressing and probing questions: ‘What do you particularly like/dislike about
shopping?’ ‘You mentioned X. Tell me more about that.’ ‘What did you do about
that?’
● Questions narrowing in on particular topics or issues: ‘How did he react when you
made the complaint?’ ‘What happened next?’ ‘What was the end result?’ ‘How
does that compare with the way X handled it?’
● Clarifying questions: ‘What exactly did you do then?’
● Summarising statements or questions: ‘You said that they sent you a letter of apology, explaining what the problem had been and offering you your money back.
I get the impression that the apology meant the most to you.’
●
Introducing projective and enabling techniques
For a projective or enabling task to work well it must be introduced at the right time.
This tends to be when participants are relaxed and at ease with the research situation and what is required of them, and when you have established trust and rapport.
You should introduce the task clearly and explain exactly what is to be done. Make
any instructions clear and explicit – whether the task is to be done individually or in
small groups, whether participants can talk to each other during the task, whether
they can ask questions of each other or of the researcher, the time available, what
will be required at the end of the task and so on. Reassure participants that it is
not a test. As they work on the task, give them plenty of encouragement. Remind
them when the time is coming to an end and reassure them that it does not matter
if they have not finished. Invite them in turn to explain the end product. Follow
this up with a discussion about what conclusions they would draw from what has
been done.
The wind down
It is important to signal the end of the interview or discussion – about 10 minutes
before the end of an hour-long session, and about 15 minutes before the end of a one
and a half to two-hour session. Some useful wind-down strategies include presenting
269
M11 The Practice of Market Research 31362.indd 269
30/09/2021 18:26
Chapter 11
Doing qualitative research
a summary of the main points and asking for final comments; asking participants if
there is anything they have not said that they would like to say, or anything they have
said and wish they had not; and asking what one thought or idea the participant(s)
would like the client to take from the session.
In putting together an interview or discussion guide, check that the questions:
give you the information you want;
are meaningful and relevant to participants (and are within participants’ frame of
reference);
● are in an order that helps the flow of the interview.
●
●
Layout
Think about how you are going to use the plan or guide during data collection and
lay it out accordingly. For example:
Will you need to read it or consult it? If so, you don’t want to be scrambling to see
it or to find your place. Make sure the typeface is big enough to read at a glance;
and lay it out with headings and/or topics and tasks so that you can see easily
where you are.
● Will you need to make notes on it? If so, make sure there is room to write these in.
●
Checking a draft
Once you have a draft, test it out on a colleague or preferably someone in the target
group. Do this using the medium you plan to use for your main fieldwork – that is, if the
interview is to be conducted online, pilot your draft online; if it is an in-person interview,
test it in person. You might find the following checklist useful in reviewing the draft:
●
Are the links between the research objectives, the topics, questions and tasks clear?
– Do the questions/tasks give you the information you need?
– Did the participant understand the questions?
– Did the participant interpret the questions as you intended?
– Is the style of questioning/task suitable for getting the information you need?
– Is the style suitable for the participant?
– Are the questions meaningful and relevant for the participant?
●
Does the guide have a clear structure?
– Is there a good introduction and lead in?
– Does the structure help you establish a rapport with the participant?
– Is the interview well signposted throughout?
– Are the questions and tasks in an order that helps the flow of the interview?
– Are the transitions between questions and/or topics and/or tasks smooth?
– Is the end signalled clearly?
270
M11 The Practice of Market Research 31362.indd 270
30/09/2021 18:26
Designing the data collection guide
●
Did you manage the time well?
– Did you have enough time to cover all the questions/topics/tasks?
– Did you give the participant enough time to respond?
– Did you have enough time to go into depth?
– Were you able to explore and probe as appropriate?
●
●
Is the layout helpful to you in doing the interview/running the group?
Would you want to give up your time to take part in this research?
Box 11.3
Example: an interview guide
Here is the interview guide used to conduct individual in-depth interviews in a study to explore the decision of working adults to live in the parental home,
and to understand the experience of living at home.
Background
Tell me a bit about yourself . . .
Age
Work
Education
Relationship status
Family
Position in the family
Parents’ lives
Brothers and sisters
How long living at home
When did you move back/decide to stay?
The decision to stay at home
Tell me about your decision to live at home or stay
at home . . .
How did it come about? Take me through that . . .
Calculated decision or just happen naturally?
Who influenced it?
Partner, friends, siblings, parents?
Encouraged or pressured?
What influenced it?
Do your friends live at home?
What were the pros and cons?
Economic, financial issues.
Commuting.
Plans for the future . . . move out or stay? Rent or
buy? Saving to buy house, get married, travel . . .
Experience of living at home
How does it work?
Laundry
Take me through a
typical . . . week day
Rent/money
You get up in the . . .
morning
Set-up – behaviour
You come home . . .
after work
Expectations – self,
siblings, parents?
What about a typical weekend?
Rules
Set-up – facilities
Breaking the rules
Use of rooms in house
Conflict
Access to/own a car
Resolving conflict
Set-up – services
Relationships
Responsibilities
Decision making
Who does what for whom?
Support
Cooking
Commitment
Cleaning
Responsibility
Concluding
Sum up
Advantages
Drawbacks
Observations or recommendations to others thinking about staying at home/moving back home? Any
final thoughts you’d like to share, or that came up in
the interview that you’d like to elaborate on?
Source: Adapted from Fleming, P., Ni Ruaidhe, S. and McGarry,
K. (2004) ‘“I shouldn’t be here”: the experiences of working
adults living at home’. Unpublished qualitative research project,
MSc in Applied Social Research. Used with permission.
271
M11 The Practice of Market Research 31362.indd 271
30/09/2021 18:26
Chapter 11
Doing qualitative research
Interviewing and moderating skills
While doing interviews and ‘moderating’ or ‘facilitating’ group discussions or workshops involves asking questions it is just as much, if not more so, about listening
and observing, and about building rapport with participants (see Branthwaite and
­Patterson, 2011). It involves adapting to the needs of the situation, helping participants explore and investigate, deciding what and when to explore, probe, prompt,
or move on. It involves monitoring and managing yourself.
Much of what is communicated is communicated via tone of voice and body language. It is important that you listen not only to what is said but to how it is said – the
words used, the pauses, the style of speech and the tone of voice, and the non-verbal
cues of body language (Colwell, 1990). This will give you a fuller understanding of
the meaning of what is said. However, this is easier said than done. You are having a
conversation with the participant or participants – asking questions, listening, asking
the next question. You are also having a conversation with yourself, in your head.
You are doing the following:
thinking about how what the participant has said or not said fits with the research
objectives, or the ideas you have developed about the issue;
● deciding whether or not you should follow it up, or clarify, or move on;
● formulating the next question;
● watching body language;
● taking account of the dynamics of the interview and what they mean for what
should happen next;
● thinking about the time you have left and what else needs to be covered.
●
Langmaid (2010) suggests that in listening to others there is a ‘foreground’ – what
the speaker is saying; and a ‘background’ – the inner commentary that is going on
in the listener’s (the researcher’s) head. He describes this ‘background’ as having
three modes: ‘listening for safety’ – to manage the conversation; ‘listening about’ –
to interpret the speaker’s meaning; and ‘listening for closure and confirmation’ – to
determine whether you are getting what it is you need.
Listening
Despite this internal conversation or ‘background’, you must listen actively and
attentively to the participant and you must show that you are listening (in a nonjudgemental, empathetic and respectful way) and you must show that you are interested in what is being said. How you do this will differ depending on whether you
are working in person or remotely (online or on the phone).
It is more challenging to indicate that you are listening when you are doing online
(or telephone) interviews or discussions. In these modes of data collection eye contact
is not the same as it is with in-person research and in addition there may be a time lag.
As a result there may be problems picking up the usual visual and auditory cues and
so a loss of interaction. Participants, for example, may miss cues for taking their turn
and so may talk over each other or wait to speak and decide not to. A
­ cknowledge
272
M11 The Practice of Market Research 31362.indd 272
30/09/2021 18:26
Interviewing and moderating skills
these potential issues with participants and explain how you plan to handle them.
You could do this in the form of written instructions sent out in advance, or by doing
a trial run or you could build in some time at the beginning of the session. Here
are some suggestions for enhancing engagement between you and participants and
between participants in an online fieldwork session:
set up the screen so that all involved can see each other;
get everyone to include their names so that these appear on screen;
● make sure you are well-lit so that participants can see you clearly;
● remember that the camera has a wide angle so ensure that there is nothing in the
background to distract participants;
● to achieve eye contact via the screen look at the camera rather than at the faces on
screen;
● use screen-sharing to show materials but make sure that there is nothing on your
screen that you should not be sharing;
● signal clearly that you have heard what the participant has said;
● use encouraging responses as you would in person;
● use the participant’s name to invite them to speak or take their turn;
● ask participants to raise their hand or click the hand raise icon to show that they
want to speak;
● for groups, make use of separate break-out rooms to enable and encourage participants to chat and share.
●
●
You may also want to have a second or assistant moderator working with you to
monitor participants, to answer any queries they might have (either via the software’s
chat facility or on a separate channel, for example, on a phone messaging app) or to
alert you to any issues.
Building rapport
The interviewing/moderating process can be a nerve-racking experience for participants – meeting someone (or several people) they have never met before; being
questioned and asked to talk about subjects that, sometimes, they may not even
discuss with friends. It is vital that the researcher is able to put participants at
ease, to show empathy and respect, to be helpful and encouraging (Patterson and
Malpass, 2015), and to establish rapport – without rapport the quality of the
interaction between interviewer and participants (and the quality of the data)
will be poor (Alison and Alison, 2020). Rapport is about getting the participant’s attention and creating trust. You can build rapport by actively listening, as
described above, by giving the participant your full attention, by being respectful
and by showing the participant that you are interested in understanding their
perspective by going back over what was said. In addition, you can ‘mirror’ or
‘match’ – (subtly and genuinely) adopting aspects of the participant’s verbal and
non-verbal behaviour – the pace and tone of speech, facial expression, posture,
for example.
Industry Insight 11.3 sets out the approach to interviewing teenagers about sex,
contraception and pregnancy.
273
M11 The Practice of Market Research 31362.indd 273
30/09/2021 18:26
Chapter 11
Doing qualitative research
Industry Insight 11.3
Talking to teenagers about sex
Our approach
The approach to the research session itself was
founded upon three principles:
Building a transactional relationship
Funnelling down the discussion
● Visual stimulus and sensitive projective techniques.
●
●
and becoming more personal and specific as the
discussion progressed. This is a familiar research
technique and was especially important here in
generating productive discussion. Here is an outline flow of the discussion:
Their world
Relationships in their world
● Intimate relationships
● Sex
● Teen pregnancy
● Condoms and sexually transmitted infections
● Their sexual experiences
● Their use of contraception.
●
●
Building a transactional relationship
The aim is to build a trusting two-way relationship. The question is how. The participants – marginalised teenagers – rarely have a voice and are
almost never taken seriously. To expect to build a
relationship based on bonds of trust and friendship
in an hour-long interview is unrealistic, particularly
with teenagers who have often been let down. The
participants know they have stuff to say that you
need to know. That is why they are being paid: you
pay, they talk, you listen. If you don’t understand
or you need more, you tell them. It’s a transaction.
And as with any transaction it only works if both
parties get something real out of it.
For the moderator the deal is:
Funnelling the discussion
Typically the interview would start with a general
conversation about their world, their lives, their
homes, their interests, likes and dislikes. This
was largely based on their pre-research journals.
It was important to show that both their views
and their efforts were valued. It was a safe place
to start, allowing them to expand on areas they
felt confident and secure discussing. The journals
were also designed to provide a springboard from
that general conversation into a more focused
conversation about relationships and people in
their lives. Understanding the kinds of relationships they had with those they trust and what
they can talk about to whom proved central to
the development of the communications strategy
that was the reason for the research. The flow
from relationships to sex was relatively natural.
In general, they were happy to talk about sex,
attitudes to contraception and teen pregnancy
as long as the subject had a natural place in the
conversation.
The flow of the conversation was designed to
put respondents at their ease by starting broad
Source: Adapted from Cohen, J. (2005) ‘Teenage sex at the margins’,
MRS Conference, www.mrs.org.uk. Used with permission.
●
You have to want to hear and listen to what
they have to say – no agenda, no predetermined
hypotheses.
For the respondent the deal is:
●
They have to tell it like it is: no making it up,
no pretending.
This simple transaction – this is what is expected of
you and this is what you can expect of me – proved
effective in creating a two-way relationship of equals.
Observing
To build and sustain rapport you need to observe as well as listen. You need to be
aware of and sensitive to body language, including facial expression (Habershon,
2005), in order to interpret what participants are saying correctly and in order to
274
M11 The Practice of Market Research 31362.indd 274
30/09/2021 18:26
Interviewing and moderating skills
run the interview or discussion well. For example, you need to know whether the
participant understands the question or the issue, you need to know whether they are
anxious or interested and so on. Body language will help tell you these things. The
key elements of body language include the following:
movements (of the head and other parts of the body, including hand gestures);
facial expressions;
● direction of gaze (including eye contact);
● posture;
● spatial position (including proximity and orientation);
● bodily contact;
● tone of voice;
● dress.
●
●
If you are involved in international research, remember that gestures and body language may mean different things in different countries (Morris, 1994). In addition
to observing participants’ body language you need to be aware of your own and the
messages that it might be conveying to participants.
Managing yourself
It is important to think about your role and your stance in any qualitative research
encounter, about what assumptions you make about the participants and about the
topic, and to make these explicit to yourself before fieldwork begins. An open mind and
a high degree of self-awareness are important ingredients in qualitative research. At the
outset of a study you should examine your own beliefs, feelings and views on the topic.
Preparing for fieldwork
Whatever type of fieldwork you are doing, take some time to think through everything you need. You may find the following checklist useful:
Researcher materials
Project research brief
Sample information
● Venue/log in details
● Timetable (don’t forget time zones)
● Interview/discussion guide
● Pre-task details
● Stimulus/task material
● Recording equipment
● Identification
● Contact details of key people.
●
●
Participant materials
●
●
Project information sheet
Consent form
275
M11 The Practice of Market Research 31362.indd 275
30/09/2021 18:26
Chapter 11
Doing qualitative research
Interview/discussion introduction
Contact details of key people
● Thank you note
● Incentives.
●
●
Running the fieldwork session
Doing qualitative fieldwork is difficult. There is a lot going on. There is ‘hidden’ stuff
and there is visible stuff. The hidden stuff includes the following:
Actively listening to and thinking about what the participants say and don’t say
Observing body language and thinking about what it means
● Deciding what to do next; ask a question, explore, probe, move on
● Formulating the next question
● Thinking about the technology and whether or not it is working
● Thinking about the observer
● Thinking about the dynamics of the situation and what it means
● Thinking about things from the participant’s point of view
● Wondering if the participant is at ease and comfortable
● Thinking about how much time you have left and what else you need to cover.
●
●
The visible stuff includes the following:
Showing the participant that you’re listening
Asking questions and introducing tasks
● Taking notes
● Looking at the interview/discussion guide.
●
●
Acknowledge all of this to yourself. It can also help if you explain what is going to
happen, what the interview or discussion involves, to the participants.
End of fieldwork review
As soon as the session is over and you have thanked the participants, take some time
to review it. Read over your notes and add to them or clarify them. It can be useful
to write down everything that’s in your head – all your thoughts, ideas, feelings. This
immediate data capture can be very useful for analysis. Reflect on the data you collected and ask yourself what you found that was unexpected or surprising; challenge
any assumptions you might have had. Review and reflect not only on the content but
on how you ran the session. This is important for several reasons:
It helps you to unpack what you did – this will help you to address any issues with
your practice for future sessions.
● It enables you to determine what worked well and what did not at eliciting the
data you need – this will enable you to amend your approach for the next session.
● It helps you to process and embed what you learned – this will help you with the
analysis of the data.
●
Here is a checklist for an after-fieldwork review. It contains suggestions for questions
you might ask yourself to help you reflect on your fieldwork.
276
M11 The Practice of Market Research 31362.indd 276
30/09/2021 18:26
Interviewing and moderating skills
Overall
What went well? What did not go well?
What, if anything, surprised you?
● What was challenging? Why was that?
● What was easier than you anticipated? Why was that?
●
●
Structure
●
●
Did the structure you planned hold up in practice?
If not, why was that?
Introduction
● Was it comprehensive? Did you miss anything out?
● How was your delivery of it? Too fast? Not clear enough?
● Did participants ask any questions?
Flow and pace
● Did the session flow well or not? Why was that?
● How well did you transition from question to question, topic to topic, task
to task?
● How well did you manage the time?
● Did you give participants enough time to answer, to do tasks?
● Which elements, if any, were rushed or slow?
● Did you move on at the right time?
● Did you introduce the tasks at the right time?
Wind down and close
● Did you signal the end in enough time?
● Did you thank participants?
Technique
● Do you think you established rapport? How do you know?
● Were the types of questions you used appropriate?
● Did you elicit any data without direct questioning? How?
● Did you lead participants at any stage? If so, where?
● How did you find listening? How did you find observing?
● How well did you listen? How well did you observe?
● How well did you manage yourself? How was your stance?
Content
● Compare a summary of the session against the research objectives.
● Are there any gaps? What did you miss? Why was that?
● Did you get (enough) relevant data?
● Did you get enough depth and/or detail? If not, why not?
277
M11 The Practice of Market Research 31362.indd 277
30/09/2021 18:26
Chapter 11
Doing qualitative research
Notes for the next time
List what you need to change for the next session.
● List what you need to be aware of for the next session.
●
Chapter summary
●
●
●
●
●
●
The skill and experience of the qualitative researcher is an important element
in the value of any piece of qualitative research. The qualitative researcher’s
role is to design the research, conduct the fieldwork and analyse and report the
findings.
Sampling in qualitative research is a rigorous and systematic process. Purposive
techniques are used and samples are typically small. While notions of statistical
representativeness do not apply, ability to generalise from the sample to the wider
population does.
The style of an interview or discussion guide varies from a simple list of topics to
one that has more structure, with a series of questions listed under headings. The
choice depends on the objectives, the need for comparability between interviews
or groups, experience and house style of the researcher.
The style of questions tends to be open-ended and non-directive.
Listening and observing and the ability to build rapport as well as questioning
skills are vital.
Projective and enabling techniques – techniques of association, completion,
construction and expressive techniques (indirect forms of questioning that
are deliberately vague and ambiguous) – are used to get beyond the rational
response.
Exercises
1 You are planning to do qualitative research among mothers with children under the
age of four to explore the decision to either return to work or not return to work.
a. Describe three approaches that would be suitable for recruiting or drawing a
sample from this population, outlining the strengths and limitations of each
approach.
b. Which approach would you recommend? Give reasons for your answer.
2 You and a colleague must each undertake six group discussions on the same
topic as part of the same project. Describe what you would do to ensure
consistency of approach across all group discussions, giving reasons for what
you suggest.
278
M11 The Practice of Market Research 31362.indd 278
30/09/2021 18:26
References
References
Alison, E. and Alison, L. (2020) Rapport: The Four Ways to Read People, London: Vermilion.
Baker, S. and Edwards, R. (2012) ‘How many qualitative interviews is enough? Expert voices
and early career reflections on sampling and cases in qualitative research’, A National
Centre for Research Methods Review Paper.
Bertaux, D. and Bertaux-Wiame, I. (1981) ‘Life stories in the bakers’ trade’, in Bertaux, D. (ed.)
Biography and Society: The Life History Approach in the Social Sciences, London: Sage.
Branthwaite, A. (1983), ‘Situations and social actions’, Journal of the Market Research Society,
25, pp. 19–38.
Branthwaite, A. (2002). Investigating the power of imagery in marketing communications:
Evidence-based techniques. Qualitative Market Research: An International Journal, 5,
pp. 164–171.
Branthwaithe, A. and Patterson, S. (2011) ‘The power of qualitative research in the era of
social media’, Qualitative Market Research: An International Journal, 14, 4, pp. 430–40.
Cohen, J. (2005) ‘Teenage sex at the margins’, Proceedings of the Market Research Society
Conference, London: MRS.
Colwell, J. (1990) ‘Qualitative market research: a conceptual analysis and review of practitioner criteria’, Journal of the Market Research Society, 32, 1, pp. 13–36.
Cooper, P. and Branthwaite, A. (1977) ‘Qualitative technology: new perspectives on measurement and meaning through qualitative research’, Proceedings of the Market Research
Society Conference, London: MRS.
Cooper, P. and Tower, R. (1992) ‘Inside the consumer mind: consumer attitudes to the arts’,
Journal of the Market Research Society, 34, 4, pp. 299–311.
Fleming, P., Ni Ruaidhe, S. and McGarry, K. (2004) ‘“I shouldn’t be here”: the experiences of
working adults living at home’. Unpublished qualitative research project, MSc in Applied
Social Research, Trinity College Dublin.
Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory, Chicago, IL: Aldine.
Gordon, W. (2016) Mindframes: 6 Enduring Principles from 50 years of Market Research,
London: Acacia Avenue.
Gordon, W. and Robson, S. (1980) ‘Respondent through the looking glass: towards a better
understanding of the qualitative interviewing process’, Proceedings of the Market Research
Society Conference, London: MRS.
Habershon, J. (2005) ‘Capturing emotions’, Proceedings of the Market Research Society Conference, London: MRS.
Harré, R. (1979) Social Being, Oxford: Blackwell.
Langmaid, R. (2010) ‘Co-creating the future’, International Journal of Market Research, 52,
pp. 131–5.
Miller, T., Birch, M. Mauthner, M. and Jessop, J. (eds) (2012) Ethics in Qualitative Research,
London: Sage.
Morris, D. (1994) Bodytalk: A World Guide to Gestures, London: Jonathan Cape.
MRS (2020e) Qualitative Research Guidelines, London: MRS.
MRS (2019) Code of Conduct, London: MRS.
Patterson, S. and Malpass, F. (2015) ‘The influence of Bill Schlackman on qualitative research’,
International Journal of Market Research, 57, 5, pp. 677–700.
279
M11 The Practice of Market Research 31362.indd 279
30/09/2021 18:26
Chapter 11
Doing qualitative research
Pich, C., Armannsdottir, G. and Dean, D. (2015) ‘The elicitation capabilities of qualitative
projective techniques in political brand image research’, International Journal of Market
Research, 57, 3, pp. 357–94.
Pich, C., Armannsdottir, G. and Spry, L. (2018) ‘Investigating political brand reputation
with qualitative projective techniques from the perspective of young adults’, International
Journal of Market Research, 60, 2, pp. 198–213.
Ritchie, J., Lewis, J., Elam, G., Tennant, R. and Rahim, N. (2014a) ‘Designing and selecting samples’ in Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (eds.)
Qualitative Research Practice, 2nd edition, London: Sage.
Ritchie, J., Lewis, J., Ormston, R. and Morrell, G. (2014b) ‘Generalising from qualitative
research’, in Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (eds.)
Qualitative Research Practice, 2nd edition, London: Sage.
Rubin, H. and Rubin, I. (2011) Qualitative Interviewing: The Art of Hearing Data, 3rd edition, London: Sage.
Schlackman, W. (1959) ‘No more ice cream?’ Ice Cream Field, July. Ernest Dichter Papers:
Hagley Museum and Library.
Schlackman, W. (1989) ‘Projective tests and enabling techniques for use in market research’
in Robson, S. and Foster, A. (eds) Qualitative Research in Action, London: Hodder &
Stoughton, Chapter 6, pp. 58–75.
Tuckman, B. (1965) ‘Developmental sequence of small groups’, Pyschological Bulletin, 63,
pp. 384–99.
Tuckman, B. and Jenson, M. (1977) ‘Stages of small group development re-visited’, Group
and Organisational Studies, 2, pp. 419–27.
Recommended reading
Alison, E. and Alison, L. (2020) Rapport: The Four Ways to Read People, London: Vermilion.
Gordon, W. (2016) Mindframes: 6 Enduring Principles from 50 years of Market Research,
London: Acacia Avenue.
Keegan, S. (2009) Qualitative Research: Good Decision Making through Understanding People, Cultures and Markets, London: Kogan Page.
Mason, J. (2017) Qualitative Researching, 3rd edition, London: Sage.
Miller, T., Birch, M. Mauthner, M. and Jessop, J. (eds) (2012) Ethics in Qualitative Research,
London: Sage.
Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (2014) (eds.) Qualitative
Research Practice, 2nd edition, London: Sage.
Salmons, J. (2016) Doing Qualitative Research Online, London: Sage.
280
M11 The Practice of Market Research 31362.indd 280
30/09/2021 18:26
M11 The Practice of Market Research 31362.indd 281
30/09/2021 18:26
Chapter 12
Analysing data from qualitative
research
Introduction
In this chapter we look at analysing the data gathered in a qualitative research
project. We look at converting a mass of raw data – notes, recordings,
transcriptions – into meaningful findings. Gathering data and analysing them
are very closely linked in qualitative research – interwoven almost – and in this
chapter we touch on things you might do at the data gathering stage as well as
covering what you do at the main analysis stage.
Topics covered
What is qualitative data analysis?
Approaches to analysis
● Planning the analysis
● Doing the analysis
● Computer-aided and automated analysis.
●
●
Relationship to MRS Advanced Certificate Syllabus
The material covered in this chapter is relevant to Topic 1: Understanding the
research context and planning the research project; Topic 2: Guiding Principles;
and Topic 5: Analysing and interpreting data and reporting findings.
M12 The Practice of Market Research 31362.indd 282
30/09/2021 18:27
What you should get from this chapter
At the end of this chapter you should be able to:
understand how to approach the analysis;
● understand the links between research objectives and data analysis;
● understand and evaluate the findings from qualitative research;
● undertake and manage the analysis process.
●
283
M12 The Practice of Market Research 31362.indd 283
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
What is qualitative data analysis?
In this chapter we deal with analysis of data from qualitative research projects and
we refer to this as qualitative data analysis. There are three points to note here. First,
the nature or format of the data: the data we refer to are typically non-numerical,
unstructured data. Second, the way in which the data are produced: the data here are
data generated from bespoke research studies. Third, and related to this, these data
are typically small data and not big data. In other words, we are not dealing with
large-scale, unstructured data of the sort generated by social media, online reviews or
customer feedback. Analysis of such data is often automated in a process known as
text mining or text analytics and natural language processing. We look at this briefly
at the end of the chapter. Such software is used to search, identify, retrieve, extract
and summarise; to tag, code and categorise; to classify and cluster; and to identify
patterns, among other things. These processes are also used in analysing data from
qualitative research studies with software designed for that purpose, and many are
still done manually.
The aim of analysis is to extract meaningful insights from the data, to produce
valid and reliable findings that address the research problem and enable the client to
take action. To achieve this, analysis should be disciplined and rigorous. Here are
some suggestions for achieving this:
Bear in mind the context of the research throughout the process – think of the
client’s business problem, the organisational setting of the problem and the wider
social and cultural setting.
● Keep the research objectives under consideration at all times.
● Be explicit and transparent about the analysis process and the approach you are
taking and ensure that it is appropriate for the research objectives.
● Do not formulate conclusions early in the analysis process - take time to examine
all of the evidence thoroughly.
● Separate how you see the issue from how participants see it (to avoid imposing
your views and ways of thinking on the data).
● Do not force the data to fit with what a theory or model suggests.
● Reflect on the credibility of the findings in both the wider context and the narrower
context of the client’s business problem.
● Check what other sources reveal about the topic or issues.
● Review the data and your findings with another researcher.
●
The analysis should be thorough, consistent and comprehensive, systematic without
being rigid, and open to the possibilities and insights that emerge as a result – intuition and creativity are a vital part of it. The process is part mechanical – handling and
sorting the data – and part intellectual – thinking about and with the data. It is not
‘reportage’, a summary of what participants said. It does involve the ‘cataloguing of
remarks’ (Branthwaite and Patterson, 2012) but it is more than that: it is a ‘dynamic’
process that looks for ‘the underlying meanings, together with the connections and
processes that [tie] together the remarks’ with the participant’s beliefs and outlook
as well as the influences and expectations from the wider social and cultural context
(Branthwaite and Patterson, 2012). In the same way that we look for patterns and
relationships in quantitative and numeric data, we examine data from qualitative
research (recordings, text, images) for patterns, themes and relationships.
284
M12 The Practice of Market Research 31362.indd 284
30/09/2021 18:27
Approaches to analysis
Qualitative research analysis is not a discrete phase undertaken once fieldwork
is completed, rather it is ongoing from the start of a project and a lot of the ideas
about what you think is going on in the data will occur to you during fieldwork. It
is once fieldwork is over, however, that you get the chance to organise the data, sort
through them, pull them apart, think about them and with them, and pull together
‘the findings’.
Approaches to analysis
Denzin and Lincoln (1994) refer to qualitative research as ‘bricolage’, the art of
adapting and using a variety of materials and tools, and to the qualitative researcher
as a ‘bricoleur’, someone who is skilled in the use and adaptation of the tools. Data
analysis is one area of qualitative research where this ‘bricolage’ approach is evident.
Techniques for conducting qualitative research and analysing the data have been
drawn from a range of disciplines within the social sciences and the humanities, in
particular from social anthropology and sociology, and language and linguistics. The
approach you take depends on a range of factors and their interaction. These include
your background and training; and your ontological and epistemological view. It may
also depend on the following:
the way your mind works to sort and think about things;
your level of experience;
● your level of knowledge in the area under investigation;
● the availability of relevant theories or models;
● the nature of the research enquiry – exploratory, descriptive, explanatory or a
combination of these;
● the data collection method and mode (groups, depths, workshops; in person,
online);
● the subject matter and how the research participants approach it;
● the scale of the project – the number of participants, the volume of data generated
and/or the number of research locations and the number of researchers involved;
● the end use of the research;
● the resources available – time, money and number of people.
●
●
The approaches you are most likely to hear about in the context of market and social
research are thematic analysis, content analysis, discourse analysis and grounded
theory. Before we look at each of these we look at some of the common processes
involved.
Common processes underlying analysis
Robson and Hedges (1993) in their report on the MRS Working Group on analysis
note that qualitative research practitioners describe analysis as consisting of ‘two
interlocking, interacting processes: data handling and “thinking” (or interpreting)’.
For them, data handling is a thorough and disciplined process that ‘covers the processing, organising and structuring of the evidence collected in order to make sense
of it’. They describe it as a means to an end. The ‘thinking’ element is ‘whereby the
285
M12 The Practice of Market Research 31362.indd 285
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
researchers draw conclusions from all they have experienced from the moment they
were briefed on the project’. They note that this process ‘encompasses the end goal
of the project’. At all times during a project you will be working with the research
objectives in mind. You will have the objectives in mind when you undertake the
‘processing, organising and structuring of the evidence’.
These mental and physical processes are common to most analysis approaches. By
working through them you are preparing for and at the same time doing the ‘thinking’
element that Robson and Hedges describe. We will come back to this in more detail later.
Inductive and deductive reasoning
In a deductive approach you speculate up front, in advance of fieldwork, about
what it is you think you will find and you set out in the research to test this theory
or hypothesis or idea. You design the research and approach the analysis in a way
that allows you to do this. You move from the general to the specific in deductive
­reasoning – from an idea or general hypothesis or theory about what might be happening to specific observations to see if what you expect is actually happening. You
start with pre-existing ideas. It is, after all, difficult not to have any pre-existing ideas
or knowledge about the issue under study.
An approach that is common among some qualitative researchers (Katz, 1983) is
‘analytic induction’ (Robinson, 1951). It works something like this. You have defined
the research problem and have some ideas about what you are looking for. From this,
and using your understanding of the issues and the background to the problem, you
develop working hypotheses about the matter under investigation. You start fieldwork and throughout it you are thinking about how what you are seeing and hearing
and what participants are telling you fits with your initial ideas and hypotheses. You
keep questioning this, asking whether you need to amend or expand the hypotheses,
modify your ideas, explore some issues in greater depth, get more examples of things
that fit and do not fit with your hypotheses and so on.
In many qualitative research projects induction rather than deduction is used. You
do not go into the fieldwork to test out assumptions or existing theories or ideas.
You gather data and from the data you identify general principles that apply to the
subject under study – you move from the specific to the general – theory building
rather than theory testing. One such well-documented approach is grounded theory.
As we noted above, it is difficult to use a purely inductive approach in practice. It is
difficult to keep out all other ideas and to have a completely open mind when tackling
a problem. It is likely that you will have some knowledge of the product field or area
under investigation, or at least some understanding of general patterns of behaviour
and attitudes (from previous research or the literature and from the research brief).
In the real world analysis is an iterative process involving both inductive and deductive reasoning. Ideas and hypotheses emerge from the data and are tested out within
the data; you might revise or change them, collect more data in which to test and
develop ideas and so on.
286
M12 The Practice of Market Research 31362.indd 286
30/09/2021 18:27
Approaches to analysis
Recognising and dealing with your own biases
We all have biases – ways of thinking, opinions and attitudes, conscious and unconscious. Ormston et al. (2014) and Gordon (2016) note the need to recognise these
biases: it is important to do this to prevent or limit them from influencing, skewing
or limiting any part of the research process including the analysis and interpretation
of the data. Your own thinking about an issue may mean that you see only what
you want to see, or only what fits with your view of the problem. It is important
to think about alternative ideas and hypotheses, to be open to different ways of
looking at and interpreting the data, and to question and challenge what you see or
think you see in the data. At the outset of a project, examine what you ‘know’ or
assume, what preconceptions you might be bringing to the fieldwork and analysis.
Before going into the field take some time for reflection. Ask yourself: What do I
think and feel about this? What attitudes do I have about this issue? Make these
explicit, articulate them, challenge them and then leave them to one side as much
as possible.
Some specific approaches to analysis
In this section we look at some of the more common approaches to analysis of data
from market and social research projects.
Thematic analysis
Thematic analysis (Braun and Clarke, 2013) is the identification, description and
analysis of themes within the data. A theme might be a topic, an issue, a question,
an experience relevant to the research objectives that recurs throughout the dataset
or among a particular group within it. Braun and Clarke (2006) argue that it should
be a ‘foundational method’ for qualitative analysis. Nowell et al. (2017) agree, seeing
it as ‘a method in its own right’ but cite others who believe it to be ‘something to
be used to assist . . . in analysis’. Pich et al. (2018) describe a ‘two-stage process of
thematic analysis, coarse-grained followed by fine-grained’ (emphasis in o
­ riginal).
They applied this approach to data from group discussions, data that included outputs from projective techniques of association, construction and completion. For
the ‘coarse-grained stage’ they describe familiarising themselves with all of the findings; noting or ‘cataloguing’ data from each projective activity; ‘assessing emerging
themes’ and reviewing transcripts. For the ‘fine-grained stage’ they describe reviewing the categories they formulated; cross-checking and comparing; re-visiting themes;
and comparing and contrasting the themes with findings of a previous study (Pich et
al., 2015). Sometimes, however, data or findings do not resolve into themes and a
thematic analysis is not useful. Lawes (2017) refers to it as ‘the qualitative equivalent
of descriptive rather than inferential statistics, describing the data sample but not
the world beyond’.
287
M12 The Practice of Market Research 31362.indd 287
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
Content analysis
Content analysis has its origins in communication research (Berelson, 1952). A very
broad definition is that it is a method ‘for classifying, evaluating and studying
recorded communications objectively and systematically’ (Lac, 2016). It has also
been defined as a technique for data reduction: a ‘systematic, replicable technique
for compressing many words . . . into fewer content categories based on explicit rules
of coding’ and as such is a useful way of discovering and describing what the data
contain (Stemler, 2000). Bryman (2008) notes its use in the analysis of documents
and texts as a method ‘that seek[s] to quantify content in terms of pre-determined
categories’. Prior (2014) expands that to include its use in the analysis of speech, in
particular data from interviews.
Discourse analysis
Discourse in its simplest definition is ‘the use of language’ (Chilton, 2004). Since it
‘is implicated in expressing people’s points of view and value systems . . . ’ (Jaworski
and Coupland, 2014) it is of interest to market and social researchers. While it
involves language – speech, text, sign language – it also involves all the other elements of speech that are not words including intonation, stress, voice quality, gestures, and facial expressions. It also goes beyond language to involve other media
(Johnstone, 2017). Martin and Rose (2007) cite image, music, sound, space and
action. Fairclough (1992) contends that, ‘Discourse constitutes the social’ (emphasis
in original). His three dimensions of the ‘social’ – knowledge, social relations and
social identity – ‘correspond . . . to three major functions of language . . . ’. The
analysis of discourse offers a way of ‘exposing or deconstructing the social practices’
(Jaworski and Coupland, 2014) including language and signs and their use. Lawes
(2017) notes the benefit of methods that involve the analysis of language and semiotic signs: they ‘deal with sampling and generalisability by sampling talk rather than
people and generalising . . . to cultural linguistic practices . . . [T]hey do not simply
describe the conversation at hand. They uncover the larger rules of what consumers call normal behaviour’. Johnstone (2017) sums up the usefulness of discourse
analysis like this: ‘To the extent that discourse and discourses . . . are at the center of
human experience and activity, discourse analysis can help in answering any question
that could be asked about humans in society.’
There are many approaches to discourse analysis including speech act theory and
pragmatics; conversation analysis; ethnography of communication; interactional
sociolinguistics; narrative analysis; and critical discourse analysis. Jaworski and
Coupland (2014) note that any classification of approaches ‘probably overstate[s]
the degree of difference between [them]’ and that ‘many researchers have taken an
inclusive view . . . to the extent that their work spans most or all of the traditions
. . . ’. While discourse analysis may focus on language, as with other approaches, it
involves getting to know the data, taking them apart, describing them, questioning
them (sometimes from different points of view or different theoretical perspectives),
and looking for patterns and relationships.
288
M12 The Practice of Market Research 31362.indd 288
30/09/2021 18:27
Planning the analysis
Narrative analysis
Sometimes in a project, the objective is to investigate and/or understand a ‘story’ – for
example, a customer journey or a patient’s experience or a decision-making process or
a ‘day in the life’ scenario or a life history – collected through use of a diary or via interview or on an accompanied visit. If your project has such an objective then you might
find a narrative analysis useful. Using the tools of a narrative analysis can help you
examine and understand the structure, how the story is told, the time sequences, and
the events and actions within it (Allen, 2017). According to Edwards (1997), an analysis
of a narrative can be used to analyse ‘(1) the nature of the events narrated; (2) people’s
perceptions and understanding of events; and (3) the discourse of such understandings
and events’. He notes, too, that these are also ways of approaching any discourse.
Grounded theory
Grounded theory is the approach to analysis described by Glaser and Strauss (1967)
and later by Strauss and Corbin (1998). In the grounded theory approach, data are
examined using the ‘constant comparative method’ in order to identify themes and
patterns; concepts and codes are developed to summarise what is in the data. These
concepts and codes are used to build propositions, or general statements, about relationships within the data. The codes and propositions are tested out in the data to
make sure that they hold up, to make sure that they fit the categories to which they
were assigned and that the propositions help to explain what is being studied. ‘Theoretical sampling’ is used to select new ‘cases’ (participants) that might help develop
the emerging concepts, propositions and theory until saturation is reached – that is,
until the new cases do not add anything new.
Although grounded theory is often cited, particularly in academic research, as the
approach taken in analysis, there is evidence (Bryman and Burgess, 1994) that few use
it in its entirety in the way that Glaser and Strauss and Strauss and Corbin describe.
Citing the grounded theory approach is more likely to mean that the analysis is ‘data
driven’ rather than meaning that the specific approach (the coding procedures, use
of the constant comparative method or theoretical sampling) is followed exactly.
Planning the analysis
In this section we look at what needs to be done before the main stage of analysis,
post-fieldwork, begins.
At the research design stage
Although the main phase of analysis happens at the tail end of fieldwork, that is not
where it begins. Analysis really does start from the moment you get the brief and start
289
M12 The Practice of Market Research 31362.indd 289
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
thinking about the problem. There is no substitute for clear, thorough thinking at this
early stage. The process of analysis will be less painful and the outcome much better
quality if you spend time at the front end understanding the client’s problem and its
wider context and the research problem and its implications for analysis. This may
involve reviewing any relevant literature on the topic, or reviewing the findings of other
research projects on the same or similar topics. The aims and objectives of the research
drive the research design and the choice of sample, method and questions, and all of
this will determine the analysis strategy. Thinking about these things at an early stage
will often give you a way into the analysis, a way of tackling it, helping you to develop
both a strategy and a framework for interrogating the data and presenting the findings.
At the fieldwork stage
There is an overlap between fieldwork and analysis in qualitative research. You collect data, think about them all the while, and collect more – perhaps using a slightly
amended discussion guide or reworked stimulus material as fieldwork sheds light
on the issues. The whole time your thinking about the issues is developing: ideas,
hunches and insights will pop up, hypotheses will emerge that you might want to
test out or explore further.
Making fieldnotes
For this reason it is worth keeping a detailed log of thoughts and insights as they
occur to you during fieldwork. Write them down as soon as possible – you may not
remember them when it comes to the main phase of analysis. Sit down as soon as
possible after an interview or group or workshop is over and ‘braindump’ all your
thoughts, feelings, ideas, impressions and insights in as much detail as possible.
Make detailed notes or maps about what is emerging, what picture is beginning to
build up; write down any particularly relevant or interesting quotations. Ask yourself what was unexpected or surprising; examine and challenge your own assumptions; review your research objectives. Consider what issues need to be explored
in greater depth, what new areas you need to probe. Consider what implications
these early findings have for further fieldwork, and for analysis and interpretation,
and make changes if necessary. Write down what you think are the key themes,
content elements, things that you might want to explore or think about in more
detail later, anything that was said that you did not expect, for example, or weren’t
sure you understood. In other words make a note of anything that occurs to you
that you think might be useful when the analysis process is in full swing. Make sure
to clarify in these notes what are impressions and inferences and what are facts or
more concrete observations (Boulton and Hammersley, 1996).
Reviewing fieldwork with colleagues and clients
If you are working with a colleague, review the fieldwork session together in detail
as soon as possible after it is over and make detailed notes. They may have noticed
something you missed, or they may have understood or interpreted something in a
different way. Record this. If you have client observers, talk to them – ask them what
they thought and note down what they say.
290
M12 The Practice of Market Research 31362.indd 290
30/09/2021 18:27
Planning the analysis
Writing up a summary
It is also useful at the end of a fieldwork session to write up a summary of the main
points made by the participants under each of the topics or questions on the interview
guide or on a ‘contact summary’ form (Miles and Huberman, 1994). Review this
summary against your research objectives to check that you are gathering the data
you need with which to address them. Another useful approach is to ‘mind map®’
them (Buzan and Buzan, 2003). Use whichever approach you think will help settle
and fix things in your memory and will be useful later in the analysis. Having a summary record of some sort will help you think about and develop ideas about the data
and decide on an analysis strategy. It may also be a useful reference source or guide
when it comes to writing up the findings in detail. These notes, summaries and/or
maps can be particularly useful if more than one person is involved in the fieldwork,
and if more than one person is to be involved in the analysis. Other members of the
team can read them in order to get to grips with data across the whole sample.
If you are to work on the analysis but have not been involved in the fieldwork,
make sure, first of all, that the terms under which the data were collected allow you to
be involved. Check with the project leader and review the ethical, legal and regulatory
issues including consent forms and privacy notices. These may restrict what you can
see or work on. If you have clearance to be involved, make sure you are well briefed
about the background to the project, the client’s problem and its wider context and
the research objectives. Get a copy of the sampling plan and the research plan or
interview/discussion guide and other materials used. Find out about the fieldwork –
who did it, when it was done – and get copies of the outputs.
Developing an analysis plan
Having thought through the research problem and completed some of the fieldwork
you will have in your head – and in your notes – the basis of an analysis strategy, a
plan for tackling the analysis. It is worth formalising this plan, making it explicit. It
is easy to feel overwhelmed by the amount of data you have, and by the thought of
having to find a way through them. The possible lines of enquiry in most qualitative
studies are numerous, and time and resources are limited. The analysis plan should
set out a way of approaching the data, and in doing so calm your fears about the size
and complexity of the task and ensure that you tackle it in a systematic and rigorous
way. A strategy that has been developed to suit the aims and objectives of the research
should help you make the most of the time and resources available by prioritising
the lines of enquiry. But having a plan in place does not mean that you have to stick
rigidly to it, whatever the data throws up – it can and should be adapted and modified to fit the circumstances.
In putting together your plan it is useful to think about the following:
●
What are the practical considerations?
– How many are going to be involved in the analysis?
– Is the client or sponsor to be involved in the analysis process?
– How much time do you have for analysis?
– Are you going to work from transcripts, recordings or notes or a combination?
– Will you be using analysis software?
291
M12 The Practice of Market Research 31362.indd 291
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
●
What are the ethical considerations?
– What assurances did you give participants about how their data would be stored
and handled?
– What assurances did you give about who would see the data?
– What assurances did you give about anonymity and confidentiality?
●
What are the research considerations? What are the objectives?
– What decisions are to be taken on the basis of the research findings?
– How detailed does the analysis need to be?
– What outputs are required?
– Are the findings to be published?
●
●
What approach or method are you going to use?
How are you going to start the analysis?
– By research objective?
– Question by question, topic by topic?
– By participant type?
– By country?
– Interview by interview or group by group?
There is no one way of developing a plan – one approach is to use the research brief
or the proposal (if there is one). Start by writing down the big questions that you
have set out to answer – the objectives of the research. List the topics, the questions
and the types of participants that might help throw light on each of these and write
down what it is you will be looking for in the data generated by the questions and the
participants that will help you address the research objectives. This is your analysis
plan. Once you have done some fieldwork, you can add to this by noting what data
were actually generated by those questions. Check that the data you are generating
do indeed address the research objectives.
Making use of theories and models
As your analysis and your ideas develop, you might find (through a search of
secondary or existing data sources) that there is a body of knowledge that supports them or that will give you ideas and alternative ways of looking at the data.
You may have uncovered this knowledge when you were writing the proposal
or planning the research. For example, there may be well-developed models and
theories from management science, marketing, psychology, consumer behaviour,
behavioural economics, sociology or anthropology that could prove insightful and
useful. These models and theories can help you to structure the analysis, suggesting
lines of enquiry, and/or can help you to develop your thinking. They should not
be overlooked as a source of inspiration and help but neither should they be used
uncritically.
In choosing a model or theory you should examine how well founded it is – use
those that are well researched and empirically based.
Industry Insight 12.1 is an example of how theory helped the analysis.
292
M12 The Practice of Market Research 31362.indd 292
30/09/2021 18:27
Planning the analysis
Industry Insight 12.1
Marrying theory and data to get a clearer view
Introduction
My initial strategy in analysing my data (from 11
in-depth interviews with overseas social workers
working in Ireland) was to follow the structure of
my interview guide, question by question. There
was a beginning, middle and end, taking respondents from thinking about working in Ireland –
their first experiences of working here to their
future plans. It made sense to look at the data in
this way, too. I had divided the ‘story’ emerging
from the data into two areas: (1) all the processes
that went on before making the move to Ireland;
and (2) experiences of working in Ireland: first
impressions, the induction period, supervision,
colleagues, clients, future plans.
Analysis
I listened to the recordings of the interviews and
before preparing full transcripts, I wrote up a
summary of each interview, building in the notes
I had made at the time of the interview. These
interview summaries served as a very useful guide
to the whole interview and enabled me to pull out
a number of key topics related to the working
environment in which respondents found themselves. These included:
the lack of structure in Irish social work
induction into social work in Ireland
● supervision
● interaction with colleagues
● interaction with clients.
●
●
While I now had a set or list of topics, I had no
way of drawing the experiences of all respondents together, as not all respondents had the same
experiences or unanimous views on these topics.
Following a suggestion by my research supervisor I began reviewing the literature on culture
and its effects. It soon became apparent to me
that what many of the respondents had experienced when coming to work in Ireland initially
was culture shock. It was clear that the idea of
culture shock and the process of acculturation as
described by Hofstede (1984, 1991) would help
me understand and explain what was going on
in the data I had collected – it would help me
tie all the respondents’ stories together within a
framework.
Hofstede’s dimensions of culture, in particular the values of power distance and uncertainty
avoidance, helped me to question the data and
understand the experiences and feelings respondents were describing in a more coherent way. The
notion of culture shock and the theory or process of acculturation gave me a framework within
which I could explore and explain how non-Irish
social workers felt when they came to work in
Ireland.
Comparing the power distance and uncertainty
avoidance scores for Ireland with those for the
country of origin of each respondent helped me
understand more fully, for example, the uneasy
or unfamiliar nature of the boss/subordinate
relationship many respondents described. Power
distance informs us, among other things, about
the relationship between subordinate and boss. In
large power distance cultures, from which most
respondents came, there is a considerable gap
between boss and subordinate with a culture of
direction-giving from the boss. In small power
distance cultures like Ireland there is a limited
dependence of subordinates on bosses and a preference for consultation between the two, that is,
interdependence between the boss and the subordinate. Not surprisingly, perhaps, some respondents felt that their supervisors in Ireland were too
young and inexperienced to offer them adequate
supervision and that they did not give them the
sort of support they expected: many reported a
lack of regular supervision and some commented
that it lacked structure.
The notion of uncertainty avoidance informs
us about a culture’s tolerance for the unpredictable. Ireland has a lower score on the uncertainty
293
M12 The Practice of Market Research 31362.indd 293
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
avoidance index than the countries of origin of
many of the respondents, indicating a more comfortable relationship with the unpredictable. In
describing their experiences, many respondents
appeared to note this – commenting on, for example, the lack of structure and lack of clear guidelines which they felt characterised social work in
Ireland.
Conclusion
Hofstede’s theory helped me to transform a list of
‘complaints’ and issues in relation to work practices in Ireland into a coherent story about culture
shock and the process of acculturation among foreign social workers working in Ireland. The theory
allowed me to question the data in ways I had not
thought of. It enabled me to infuse my analysis
with an added layer of understanding and explanation, mindful at all times to avoid ‘explaining
away’ respondents’ experiences. Without it I just
had respondents’ perceptions about a list of issues.
References
Hofstede, G. (1984) Culture’s Consequences,
London: Sage.
Hofstede, G. (1991) Cultures and Organisations –
Software of the Mind, London: HarperCollins.
Source: O’Sullivan, D. (2008), written for this book. Used with
permission.
Doing the analysis
The main stage of analysis usually begins when fieldwork is more or less completed.
There are five main steps in this part of the process:
organising the data;
getting to know the data;
● getting to grips with what is going on in the data;
● making links, looking for relationships;
● pulling together the findings.
●
●
Organising the data
Organising the data involves sorting out all the materials you need in order to get on
with the analysis. Depending on the size and complexity of the project, and the way
in which you like to work, you may well have accumulated a lot of ‘raw materials’ – a
pile of recordings from the fieldwork sessions; fieldnotes; films; transcriptions of the
recorded interviews or discussions; and participants’ interpretations of enabling and
projective exercises and notes on these.
It may help you to declutter your mind in readiness for the in-depth analysis
process if you spend some time sorting this material into files or folders, labelling it,
and generally cataloguing it to make it easy to retrieve. It is particularly useful at this
stage to make several copies of transcripts – an unadulterated master copy, a copy for
cutting and pasting (if that is how you like to work), and a copy on which to make
notes. Once this sorting and filing is complete you can review your field notes, listen
to or watch your recordings, read through the transcripts and prepare how you plan
to tackle the analysis.
294
M12 The Practice of Market Research 31362.indd 294
30/09/2021 18:27
Doing the analysis
Common reactions of novice researchers at this stage are panic and anxiety – about
the mass of data and how to get started. In all likelihood you will have more thinking
done than you realise, and sorting and organising your data, reviewing your notes,
reading transcripts and talking to colleagues about the data will help sort things out
in your mind. Do not put off getting started – look at your analysis strategy and get
stuck in. It can be a laborious process – and you must approach it in a systematic
way – but you will soon find that, when you engage your brain with the data, things
fall into place, and a story will start to emerge.
Getting to know the data
It is a good idea in the early stages of your qualitative research career (if resources –
budget, time – and teamwork considerations permit) to listen to the fieldwork recordings and prepare your own transcripts. Not only will you learn a lot about your
interviewing or moderating technique but it will give you the chance to get to know
the data thoroughly. A lot of the data collected in qualitative research is spoken
discourse so it is important that you hear and/or see it in that form more than once,
otherwise you may – when reading the written transcripts – lose some of the richness
and many of the nuances. You can of course use automated transcription or voice
recognition software. If you do not prepare your own transcripts, make sure you
listen to or watch your recordings at least once and then read through the transcripts
in full. Make notes as you do this, putting faces to words, noting how things were
said, what was not said, what interpretations occur to you as you go through, what
ideas strike you and so on.
Box 12.1
How to do analysis – part 1
Below, and later in the chapter, are some of the most common and most useful tasks
in analysing data. You might find them helpful as you move through your analysis or
when you are auditing or reviewing what you have done. While they are presented here
in stages, remember the process of analysis is not always linear or sequential – it is
usually iterative – so you may find yourself working back and forth between tasks and
stages, repeating some of them as you go. You may not do all of them all of the time –
and some of them you may never do.
Taking it in
During and shortly after fieldwork
You do the fieldwork – you hear and/or see and/or feel some of the data.
You make notes at the time and/or shortly afterwards.
● You discuss the fieldwork with team members.
● You prepare a written summary of each fieldwork session.
● You think about what went on – thoughts/ideas occur to you.
●
●
295
M12 The Practice of Market Research 31362.indd 295
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
You listen to or watch the recording – you hear/see/remember more data.
You prepare a transcript.
● You make notes about what you saw and/or heard and/or felt and/or thought.
● You summarise the findings from each interview/discussion/exercise in written form
or as a diagram.
● You list questions and ideas about the data.
●
●
You may find at this stage of the process that one or other of the following statements
describes how you feel:
A. When you finish the fieldwork session and/or read the transcripts you feel that you
sort of understand or can see the big picture but are unclear about how the details
fit in.
B. When you finish the fieldwork session and/or read the transcripts you feel that you
sort of understand or can see the details but are unclear about the big picture.
After fieldwork is completed
If you are working on your own it is useful to find someone with whom you can share
your ideas, someone who will act as a sounding board. Make sure that in sharing
data you observe your ethical and data protection responsibilities, and MRS Code of
Conduct rules, if they apply to you.
You listen to/watch recordings and/or read the transcripts and notes of other team
members’ fieldwork.
● You revisit your ideas about what you think is going on in the data.
● You talk to the team about your fieldwork and what you found, and you listen to
others talking about their fieldwork.
● You discuss all of this with other team members or clients.
●
Although you go into this more intensive phase of analysis with some ideas about
what is going on, and perhaps some ideas about what it all means, it is important not
to jump to conclusions. You may find that until you watch or listen to your recordings or read through the transcripts that the interviews or groups you conducted all
merge into one in your mind. There is a danger that you misremember things, or give
some things more importance in your mind than was actually the case. You need to
protect against the selectivity and decay of your memory. This is why notes made at
the time are particularly important – they are more reliable than notes made some
time after fieldwork – and why listening to or viewing the recordings of the fieldwork
is so important. When reading your notes and transcripts and listening to or watching
your recordings, write down any analytic ideas and impressions that occur to you
and make a note about testing them out right across the data to see if they hold up.
You will need to go back through all of the data systematically and read, listen to
or watch them closely to make sure that you see the whole picture, not just the bits
that stuck in your mind. Test your ideas out by looking at and comparing data from
different types of participants. Do not get too attached to ideas too early – you may
have to ditch them as the analysis develops. Keep your mind open throughout the
process to the possibility of new or alternative explanations and new ideas.
296
M12 The Practice of Market Research 31362.indd 296
30/09/2021 18:27
Doing the analysis
Getting to grips with what is going on
This is the ‘pulling apart’, organising and coding stage of the analysis. Once you
have read your notes and transcripts and listened to/watched the recordings, and
looked at the data by participant, by type of participant or by topic, you will start
to notice patterns and themes. You will see that some things crop up a lot, or at
least more than others, that there are discernible patterns in attitudes, behaviour,
opinions and experiences. You may notice patterns in the way in which people
express themselves about an issue, in the language they use. Record all of these – in a
notebook, on the transcripts, on your data analysis sheet, in the software ­package –
whichever you use.
Coding and summarising
To understand fully what is going on you need to dissect the data, pull them apart
and scrutinise them, bit by bit. This involves working through the data, identifying
themes and patterns and labelling them or placing them under headings or brief
descriptions summarising what they mean. This process is known as categorising
or coding the data. Later in the process, when you have a thorough understanding
of all the elements, you can link the data – all the coded segments – together again.
This coding process is not just a mechanical one of naming things and assigning
them to categories, it is also a creative and analytic process involving dissecting
and ordering the data in a meaningful way – a way that helps you think about and
understand the research problem. Coding is a useful ‘data handling’ tool – by bringing similar bits of data together (Miles and Huberman, 1994) and by reducing them
to summary codes you make the mass of data more manageable and easier to get to
grips with, enabling you to see what is going on relatively quickly and easily. The
process of developing codes and searching for examples, instances and occurrences
of material that relate to the code ensures that you take a rigorous and systematic
approach. Codes are also a useful ‘data thinking’ tool. The codes you develop – and
the way you lay them out – allow you to see fairly quickly and easily what similarities, differences, patterns, themes, relationships and so on exist in the data. They
should lead you to question the data and what you see in them. The coding process
can help you develop the bigger picture by bringing together material related to your
ideas and hunches, thus enabling you to put a conceptual order (Strauss, 1987) on
the data (moving from specific instances to general statements) and to make links
and generate findings.
Generating codes
But how do you generate these codes in the first place? Where do they come from?
You can use the topics or question areas from your discussion or interview guide
(without reference to the data) as general codes or headings. For example, you might
have asked participants to describe their ideal airline flight – you could have a general
code called ‘ideal flight’ and during the coding process bring together all the descriptions under this code or heading, as follows (although in a live project each extract
would be labelled with participant details):
297
M12 The Practice of Market Research 31362.indd 297
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
Ideal flight
‘Good films, plenty of leg room, decent food. You’re sitting on your own for six or eight
hours, you want those things.’
‘You want to feel appreciated by them. You don’t want to be treated like a number.’
‘Plenty of airmiles that I can use to go on holiday.’
‘Nothing to annoy you – no one in front of you in the check-in queue, no delays, a seat
with plenty of leg room and no one sitting beside you, decent food, clean toilets and not
having to wait around for ages before your bags arrive.’
‘Comfort and decent entertainment – that’s it.’
‘The service – the feeling that they’re there to serve you.’
‘There’s never a queue at check-in – it’s hassle-free.
‘A reserved car parking space, close to the terminal, that’s free.’
‘An efficient service from check-in right through to collecting your luggage.’
‘Speed at the check-in, and not having to be there really early.’
‘Comfort and plenty of room – and no one sitting beside you, that’s great.’
‘A fully reclinable chair and plenty of room around you.’
‘Being left alone to get on with some work.’
‘A good entertainment system – good headphones, comfortable ones, and a good selection of films and TV.’
studiom1/123RF
Figure 12.1 Example of a word cloud
298
M12 The Practice of Market Research 31362.indd 298
30/09/2021 18:27
Doing the analysis
‘No delays or hassles – simple things like that.’
‘Being able to get off the plane feeling great, not uncomfortable and exhausted.’
Remember, some people may have talked about a particular topic or answered
a question later or earlier than the topic was mentioned, so you may need to
search the data record for all incidences of it. To do this you could use a word
frequency analyser or word cloud generator on your transcripts and notes. This
will highlight the more commonly used words. Figure 12.1 below is an example.
You can use the output to start to build a coding scheme. Rather than imposing codes from outside the data, you are looking into the data (a bottom up,
data driven approach) to see what words or terms or concepts participants use
to describe things. Remember that different people may use different words to
describe the same thing so make sure that you check the context and source of
the words or phrases.
Box 12.2
How to do analysis – part 2
Sorting and processing the data
There are several ways in which you might tackle this – ‘bottom up’ or ‘top down’, or
a mix of both, in a sort of iterative, back and forth approach.
Order 1 – Bottom up
You think about what participants said/did not say.
You examine the words and phrases they used.
● You note the frequency/strength with which things were said.
● You examine how they said things as well as what they said.
● You look at the context in which they said it.
● You think about what they meant.
● You think about what these things were examples of.
● You create headings, codes or categories to label or describe things.
● You make notes of these headings or codes in the transcript or highlight these bits
of the transcript.
● You cut and paste bits of the transcript under these headings, creating a new document or section for each heading.
● You build up a ‘code frame’ or list of headings.
●
●
Order 2 – Sort of top down
You have in your mind a set of ideas or concepts or headings.
You create a ‘code frame’ based around these.
● You go through the transcripts looking for examples of each of these.
● You make notes of these headings or codes in the transcript, you highlight them in
the transcript.
● You cut and paste bits of the transcript under these headings.
● You go through the material under these headings.
●
●
299
M12 The Practice of Market Research 31362.indd 299
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
You think about what participants said/did not say.
You examine the words and phrases they used.
● You look at the context in which they said it.
● You note the frequency/strength with which things were said.
● You examine how they said things as well as what they said.
● You think about what they meant.
●
●
The coding process
The coding process itself can also be tackled in a number of ways, and different
researchers will have different approaches. A relatively easy way of doing it is
to create a new document for each heading, topic or code. As you work through
your transcript, cut text that relates to the code and paste it into the document
you have created to represent that code. In this way you can build up a store of
relevant material related to that particular code or topic. Take care to label the
source of each bit (participant details, fieldwork details, place in transcript) so that
you know the context from which it came, and can refer back to it if necessary.
Remember too that one bit of data or text may fit under more than one heading
or code. You could, alternatively, go through the transcript and label bits of text
in situ, before gathering the same or similarly labelled bits in one place or under
one heading.
It is likely that you will make several – at least two – coding ‘passes’ through the
data. At the first pass you might keep the codes fairly general and keep the number to
a minimum. For example, you might have identified four or five key themes or topic
areas. As you work through the data a second time you can divide these big, general
codes into more specific ones. In the ‘ideal flight’ example you might, on a second
coding pass, pull apart all the aspects participants include in their ideal flight and
code or group these under headings such as ‘emotional aspects’ (feelings of well-being
and so on), ‘physical aspects’ (leg room and so on), ‘facilities available’ or ‘service’.
In this second pass you might group your data extracts under each of the relevant
codes as follows (note that some appear in more than one category, either because
the participant said more than one thing and you want to maintain the quotation in
full or because in some cases it is not clear in which category to include them):
Emotional aspects
‘You want to feel appreciated by them. You don’t want to be treated like a number.’
‘Nothing to annoy you – no one in front of you in the check-in queue, no delays, a seat
with plenty of leg room and no one sitting beside you, decent food, clean toilets and not
having to wait around for ages before your bags arrive.’
‘Being left alone to get on with some work.’
‘No delays or hassles – simple things like that.’
Physical aspects
‘Good films, plenty of leg room, decent food. You’re sitting on your own for six or eight
hours, you want those things.’
300
M12 The Practice of Market Research 31362.indd 300
30/09/2021 18:27
Doing the analysis
‘Nothing to annoy you – no one in front of you in the check-in queue, no delays, a seat
with plenty of leg room and no one sitting beside you, decent food, clean toilets and not
having to wait around for ages before your bags arrive.’
‘Comfort and decent entertainment – that’s it.’
‘Comfort and plenty of room – and no one sitting beside you, that’s great.’
‘A fully reclinable chair and plenty of room around you.’
‘Being able to get off the plane and feeling great, not uncomfortable and exhausted.’
Facilities
‘Good films, plenty of leg room, decent food. You’re sitting on your own for six or eight
hours, you want those things.’
‘Plenty of airmiles that I can use to go on holiday.’
‘Nothing to annoy you – no one in front of you in the check-in queue, no delays, a seat
with plenty of leg room and no one sitting beside you, decent food, clean toilets and not
having to wait around for ages before your bags arrive.’
‘Comfort and decent entertainment – that’s it.’
‘A reserved car parking space, close to the terminal, that’s free.’
‘A good entertainment system – good headphones, comfortable ones, and a good selection of films and TV.’
Service
‘Nothing to annoy you – no one in front of you in the check-in queue, no delays, a seat
with plenty of leg room and no one sitting beside you, decent food, clean toilets and not
having to wait around for ages before your bags arrive.’
‘The service – the feeling that they’re there to serve you.’
‘There’s never a queue at check-in – it’s hassle-free.’
‘A reserved car parking space, close to the terminal, that’s free.’
‘An efficient service from check-in right through to collecting your luggage.’
‘Speed at the check-in, and not having to be there really early.’
‘Being left alone to get on with some work.’
‘No delays or hassles – simple things like that.’
Alternatively, you can code the other way round – coding everything that occurs
to you as you pass through the data the first time and use the second or third pass to
structure or revise these more detailed codes. There is no right or wrong way –
do what feels best for you and for the data.
During the coding process do not rule out the possibility that bits of data may
have multiple meanings or a meaning different from the one that you are assuming.
Always check the context of comments in order to learn more about the meaning of
what was said; it may also be useful to go back to the recording. Stay open to new
ideas and new ways of looking at and coding the data. Try not to jump to conclusions
or close off avenues of enquiry. Do not think of the codes you have created as static
or fixed – they can be expanded, split apart or even discarded if they no longer seem
useful or if they do not work.
Once you have bits of data together under a heading or code the next step is to
compare all the bits – looking for similarities and differences between them. This
will help you refine the codes, making them more specific, and it will also help you
achieve a greater understanding of the data. You might do this during the second
301
M12 The Practice of Market Research 31362.indd 301
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
pass at coding, or even at a third pass, depending on the time available and the level
of detail and depth you need to achieve with the analysis.
At this stage you may want to extract some verbatim comments – quotations or
vignettes, extended story-like quotations that illustrate a typical experience or event
(Miles and Huberman, 1994) – for use in the presentation or report of the findings.
In selecting these make sure that you do not oversample the responses of the more
articulate participants. You may want to choose a range of responses that illustrate
a particular phenomenon, attitude, feeling or experience, putting together a sort of
database of quotations. Make sure in removing them from the transcript that you
provide enough context so that the meaning is clear, and ensure that they are labelled
with the relevant participant details. Ensure that in doing this you are complying with
the relevant ethical, legal and regulatory guidelines. Check the relevant rules from the
MRS Code of Conduct (2019).
During and after this ‘dissection’ stage you will start to see links and connections between bits of data. The next step is to put things back together in the light
of the understanding you have achieved via the dissection. The summary version
of the data – the coding scheme – can make it easier to see links, connections and
relationships.
Making links and looking for relationships
You should now have a very good grasp of your data. A ‘story’ should be emerging,
and it is likely that you will have some tentative ideas or explanations about what
is going on. As you have read through and/or listened to your data and as you have
coded them you will have made notes about links between different themes or codes
that overlap and you will have been asking questions of the data, testing out ideas
and looking for relationships. For example, you might ask, ‘Does the description of
an ideal flight vary between frequent and less frequent flyers or those who usually fly
club class and those who fly first class?’, ‘Is it only users who think x, or do non-users
hold the same view?’, ‘Is it younger women who say that or is it all women?’ or ‘Is it
life stage rather than age or demographics that might explain a particular pattern?’.
You may be able to develop typologies, categorising participants in terms of similarities in their characteristics. You might be able to isolate several types of business
flyer, for example, characterised by frequency of travel and attitude or delight in the
experience; or different types of customer, characterised by the experiences they have
had with your organisation, the nature of the interaction, the length of time they have
been a customer, the amount of money they spend with you per month. The questions
you ask of the data and the way you develop the data will be driven by the research
objectives.
As you make links and connections, or see relationships, think about what might
explain them and think about more than one explanation. Once you have generated
some possible explanations start looking for evidence to support your ideas and
interpretations as well as evidence that might not support them. At this stage you
may well still be coding the data – and making the codes more detailed or refined.
At the same time you may also find that you can move from the specific codes you
developed to more abstract concepts and from these concepts to a greater degree of
generalisation about what is going on.
302
M12 The Practice of Market Research 31362.indd 302
30/09/2021 18:27
Doing the analysis
Box 12.3
How to do analysis – part 3
Thinking about what’s going on
Words and meaning:
You look for common words and phrases.
You look at the context of words to try to get an understanding of the participant’s
meaning.
● You try to think of alternative meanings in a phrase.
●
●
Frequency, strength and consistency of response:
You examine the data to find out how common particular responses were.
You examine the data to determine if there was a range of response in relation to a
particular topic or question.
● You note how diverse or how similar the responses were.
● You note how strongly opinions or attitudes or beliefs were expressed.
● You examine how consistent opinions or attitudes or beliefs were.
●
●
Piecing it together
At this stage one or other of these statements might describe how you feel . . .
a . You feel that you need to get to grips with all the detail in order to form the big picture.
b . You feel that you need to get an idea of the big picture in order to see how the
details fit in.
You look to see if your codes or categories occur across the whole sample and you
amend your code frame accordingly.
● You pull together all examples from across the sample under these codes or categories.
● You compare and contrast individual cases.
● You summarise the codes.
● You create a diagram, map or flow chart linking the codes, showing how they relate
to each other.
● You create a grid or a table using the main codes to show how they did or did not
vary across the sample.
● You make detailed notes and summaries about what you found out about each code.
● You re-read the transcripts thinking about only one or two codes (or themes or ideas)
at a time.
● You form ideas or hypotheses about what might be going on in the data.
● You test these ideas or hypotheses within the data.
● You go back through your recordings/transcripts/notes looking for evidence to support or refute your ideas or hypotheses.
●
Linking and connecting
●
●
You order and/or map out your ideas or codes.
You look at them to see if there are patterns, and links or relationships between them.
303
M12 The Practice of Market Research 31362.indd 303
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
You go back and forth through the data checking and testing your ideas and
hypotheses.
● You formulate and test new hypotheses.
● You look for or find outliers or anomalies.
● You map out pathways or processes.
● You create vignettes that illustrate typical behaviour or experiences.
● You create typologies.
● You write up a summary of the findings.
● You revisit the literature and/or seek out more literature.
● You use theories or models from the literature to explore or question your data
­further or to help you explain your data.
●
Using charts, diagrams and maps
Using diagrams, tables, flow charts and maps to sort and present data can help you
think and can help to uncover or elucidate patterns and relationships. Some people can
think in and/or express ideas better in pictures and diagrams than they can in words.
Reducing data to fit a diagram or table or mapping things out can focus the thinking
on the relationships that exist in the data (see Figures 12.2, 12.3 and 12.4). The most
suitable format will depend on what it is you are trying to understand. A flow chart
might be suitable to show a detailed chronology of events, for example. A table might
be useful for summarising the reactions of different groups or types of participants to
particular stimulus material – product concepts, for example, or mood boards.
Home and
family life
Personality
School life
Factors influencing
‘political socialisation’
Use of
social media
Social life
Figure 12.2 Diagram summarising key influences on political socialisation
Source: Based on Beattie, D., Carrigan, J., O’Brien, J. and O’Hare, S. (2005) ‘I’m in politics because there’s things
I’d like to see happening’. Unpublished project report, MSc in Applied Social Research.
304
M12 The Practice of Market Research 31362.indd 304
30/09/2021 18:27
Doing the analysis
Parents
Family
Friends
Partner
Self
Siblings
Therapist
Extended
family
Sport,
recreation,
hobbies
Individual
living alone
Education,
training
Local clubs,
organisations
Work social
life
Neighbours
Professional
network
Neighbourhood
Work place
Community
Work
Strong links
Weak links
No link
Key:
Figure 12.3 Map of respondent’s social support network
Source: Breslin, G., Comerford, F., Lane, F. and Ó Gabhan, F. (2005) ‘On and off the treadmill: a typology of work–
life integration for single workers aged 35–44’. Unpublished project report, MSc in Applied Social Research. Used
with permission.
Friendly
Brand M
×
Brand Y
×
Innovative,
fresh
Brand L
×
Brand P
×
Brand S
×
Staid,
stale
Brand R
×
Impersonal
Figure 12.4 Example of a perceptual map
Pulling together the findings
As you work through your data you are likely to reach a point where suddenly it all
seems to fit together and make sense or produce a story. Here are a few ways of helping
this along. When all of the data and ideas are in your head take a break to let things
‘ferment’, to give things time to ‘gestate’. Go and do something unrelated – sleep, walk,
cook or listen to music – and you may find you have that ‘eureka’ moment. Another
way is to talk about the findings out loud to someone not directly involved in the
project. All they have to do is sit and listen and perhaps ask a few questions. Often in
trying to articulate the ideas to speak them out loud and explain them to someone else
you make connections or see a picture that you have not seen before. The other person
305
M12 The Practice of Market Research 31362.indd 305
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
may ask questions that you have not asked yourself. Yet another way is to read the
literature relevant to your project, or the original briefing notes. This may spark off
fresh ideas, suggest further lines of questioning or help you make a useful connection.
Managing yourself
Remember, one of your skills is not only to be aware of the participant’s stance in
relation to the topic but to be aware of your stance, and to be able to stand back, not
impose your own opinion, and to remain non-judgemental. This applies just as much
to the analysis and interpretation stage of the process as it does to the data gathering
stage. An extension of this skill is the ability to take another perspective, to see things
as others might see them – all the time being aware that this is what you are doing.
Box 12.4
How to do analysis – part 4
Pulling together the findings
Talking, writing, reading, listening, drawing/visualising
Talking about it out loud – to yourself and others.
Writing out your ideas, writing out summaries.
● Reading things over – transcripts, notes.
● Listening to or watching the fieldwork recordings.
● Reading the literature or write-ups of other research.
● Mapping things out or drawing diagrams.
●
●
Taking a ‘stance’
Being totally immersed in the data.
Remaining detached.
● Taking different subjective perspectives – your own, the client’s, the participants’,
the literature, the outsider’s.
●
●
Leaving it to gestate in your paraconscious/subconscious
●
●
Leaving it alone and not thinking about it.
Doing something else entirely – sleeping, walking, cooking, listening to music, doing
nothing.
Checking, verifying, developing
●
You think about what the connections and the patterns mean:
1. in the context of the individual interview or group discussion;
2. in the context of the whole sample;
3. in the context of the theme or concept or idea;
4. in the context of the big picture, the overall story.
●
●
You re-read the transcripts for a holistic view.
You discuss the findings with team members.
306
M12 The Practice of Market Research 31362.indd 306
30/09/2021 18:27
Doing the analysis
You re-read the literature and/or seek out more literature.
You compare your findings with what is in the literature.
● You question your findings further having read the literature.
● You go back and forth through the data checking and testing your ideas and
hypotheses in the data.
● You check whether your assignment of things to codes, categories or headings still
makes sense.
● You think about what assumptions you might have made.
● You question whether there could be alternative explanations or interpretations.
● You think about whether the detail really does fit with the big picture.
● You think about whether the big picture really does explain or capture all the detail.
●
●
‘Completing’ it
You are able to see or explain the big picture.
You are able to fit all the details into the big picture.
● You feel that you have explained or accounted for any conflict or lack of fit or discontinuity.
● You feel that you have moved forward towards a more complete understanding of
the issues, the problem.
● You are able to map or set things out in a series of sequential steps or see a narrative
or a coherent story in the data.
● You feel that you have ‘completed the circle’, ‘tied the story together’, arrived at a
‘best fit’, found an ‘internal logic’.
●
●
During the whole of the analysis process it is important that you bear in mind the
objectives of the research – do not lose sight of them as you become immersed in the
data. It can be helpful after you complete the coding stage to start writing things down
in some detail and, as you do so, to be constantly asking yourself how it all ties in with
the research objectives. As soon as you have the story or the elements of the story clear
in your mind, go back again to the objectives. Think about what light the evidence you
have uncovered sheds on the objectives. Think about what implications the findings
have – what is the ‘So what?’ of each of the insights the research has produced?
It is also useful to think about the quality of your findings. Ask yourself the following questions:
How plausible are they?
Do they make sense?
● Are they intuitive or counter-intuitive? Surprising or what you might expect?
● How much evidence is there to support them?
● How credible and plausible is this evidence?
● How does it fit with evidence gathered elsewhere – from other research, from
theory, from the literature?
● Have you thoroughly examined the data for disconfirming evidence?
● Have you checked that other explanations do not fit the data better?
● Have you accounted for contradictions, oddities or outliers?
● Have you introduced any bias?
● Have you given more weight to what the more articulate in the sample have said
at the expense of others?
●
●
307
M12 The Practice of Market Research 31362.indd 307
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
Have you been systematic and rigorous in looking for evidence and taking into
account all views and perspectives?
● Are you seeing in the data what you want to see? Are you overinterpreting things?
● Is there anything you might have missed?
●
Computer-aided qualitative data analysis
There are analysis programs created for use on data from qualitative research projects, including audio and visual data as well as text. NVivo, Atlas.ti and MAXQDA
are examples. These programs are known as CAQDAS (computer-aided qualitative
data analysis) or QDA (qualitative data analysis) packages. The tools they contain
support different approaches to analysis, including grounded theory and content
analysis. The main functions in most packages are as follows:
content ‘search and retrieve’ facilities;
query tools;
● coding, labelling and linking facilities;
● annotations or note-making facilities;
● data visualisation, mapping and charting tools.
●
●
Using these packages is a good way of storing and handling data and making analysis
more accessible. The way the software works means that you can rework coding
schemes as new insights emerge, and revisit segments of the data quickly and easily.
The functions allow you to interrogate the data more easily than you might in a word
processing or spreadsheet package and they enable you to see all the bits of data
plus the whole – you can move back and forth to see the context of extracts. All of
this should mean that you achieve a more in-depth understanding of the data and in
turn greater confidence in your findings. You also end up with a fairly transparent
and traceable route through the data which can be used as an audit trail for review.
Using such packages can also save you time, especially if your qualitative research
project involves a large amount of data. With smaller projects – Morgan (1998)
suggests the cut-off point is six groups or less – it may not be worth the bother. You
must remember, however, that while use of such software might suggest a systematic
approach and added rigour, most of these benefits come from how you use it rather
than from the software itself. It is only as good as your own thinking and analysis
skills.
Automated data analysis
Qualitative research data are largely unstructured data and so can be processed and
analysed using data mining or text mining programs, also knowns as text analytics,
and natural language processing (NLP) using machine learning algorithms. This is
sometimes called automated analysis. Popular packages include IBM Watson Natural
Language Understanding, Google Cloud Natural Language API, Q, Provalis Research
Text Analytics software, SAS Text Miner and Leximancer. Automated text analysis
308
M12 The Practice of Market Research 31362.indd 308
30/09/2021 18:27
Chapter summary
was developed to support the need for analysis of the large volumes of unstructured
data produced by sources such as social media, online reviews and other customer
feedback channels: in other words, ‘big data’ that would be impossible to analyse
at scale any other way. It was not developed to analyse the ‘small data’ gathered via
research projects. However, this approach has been used with open-ended text from
surveys (Downer et al., 2019) and Leeson et al. (2019) has shown how NLP can be
successfully applied to the analysis of interview data from a study on men’s health.
Leeson and colleagues suggest that the automated approach be used as a support or
‘adjunct’, either running an automated analysis and using that output to help create codes, or running it after you have coded the data to check the accuracy of the
codes. Similarly, Berardi et al. (2014), in the context of coding verbatim comments
from surveys, suggest a combination of ‘human inspection’ and automated analysis.
The advantages of automated analysis software are that it can deal with large
volumes of data; it can process and analyse those data very quickly; and it can eliminate human variability, thus offering greater reliability than human coders (Robson
et al., 2013). However, as Downer et al. (2019) report, there is no consensus that
it can achieve ‘the same level of accuracy as human judgement’. The data we get
from qualitative research projects are comprised of natural (and typically spoken)
language. Understanding the complexities and nuances of spoken language is difficult
even for humans. The main limitation of automated analysis is the extent to which it
can ‘reveal the communicative intent of word usage or symbolic meanings of words’
(Robson et al., 2013, citing Gebauer et al., 2008). This is a significant limitation
when it comes to qualitative research where understanding communicative intent
and symbolic meaning are essential.
Chapter summary
●
●
●
●
Qualitative data analysis involves looking for patterns, themes and relationships
in the data. It is an ongoing process that begins at the start of a project and
continues during fieldwork with the main work done at the end of fieldwork. It is
a difficult and time-consuming task. There are many different approaches drawn
from a range of disciplines within the social sciences and humanities.
The aim of analysis is to extract meaningful insights from the data and produce
valid and reliable findings in relation to the client’s business problem. Analysis
should be disciplined and rigorous, systematic without being rigid, and open to
the possibilities and insights that emerge as a result – intuition and creativity are a
vital part of it.
It is important to be aware of your biases so that these are not allowed to skew
analysis and interpretation of the data or limit it in any way. It is important to keep
an open mind, not jump to conclusions and to separate how you see the issue
from how participants see it.
Analysis tends to be an iterative process involving both inductive and deductive
reasoning. Hypotheses and ideas emerge from the data and are tested out within
them.
309
M12 The Practice of Market Research 31362.indd 309
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
●
●
●
●
●
●
A good theory or model can help to develop and expand thinking; speed the
process by giving it a coherence, suggesting lines of enquiry to follow and
providing ideas for developing typologies.
The process of analysis involves organising and sorting the data, getting to
know the data in detail, thinking about them and with them, pulling them apart
to understand them and fitting them together, making links and looking for
relationships, to produce ‘the findings’. Many of these activities will overlap.
Coding or labelling the data is an important analytical tool, helping to summarise
the mass of data and enabling the researcher to think with the data and uncover
patterns, themes and relationships.
Using diagrams, tables, flow charts and maps to sort and present data can also help.
Findings from the analysis, and the evidence on which they are based, should be
checked and tested in the data in a thorough and systematic way.
Specialist software is available for the analysis of data from qualitative research
projects. It can help with the storage, sorting, searching and retrieval of data;
some facilitate theory building. The quality of the analysis is a result of how the
researcher uses it, not the software itself.
Exercise
1 You are working on a project with two other researchers. Each of you has
conducted six group discussions. You are the lead researcher.
a. Prepare a briefing document for the team outlining how you plan to tackle the
analysis.
b. Describe the steps you would take to ensure that the analysis of the data is
thorough and consistent.
References
Allen, M. (2017) ‘Narrative Analysis’ in Allen, M. (ed.) The Sage Encyclopedia of Communication Research Methods, London: Sage.
Berardi, G., Esuli, A. and Sebastiani, F. (2014) ‘Optimising human inspection work in automated
verbatim coding’, International Journal of the Market Research Society, 56, 4, pp. 489–512.
Beattie, D., Carrigan, J., O’Brien, J. and O’Hare, S. (2005) ‘I’m in politics because there’s things
I’d like to see happening’. Unpublished project report, MSc in Applied Social Research.
Berelson, B. (1952) Content Analysis in Communication Research, Glencoe, Ill.: Free Press.
Boulton, D. and Hammersley, M. (1996) ‘Analysis of unstructured data’, in Sapsford, R. and
Jupp, V. (eds) Data Collection and Analysis, London: Sage.
310
M12 The Practice of Market Research 31362.indd 310
30/09/2021 18:27
References
Branthwaite, A. and Patterson, S. (2012) ‘In search of excellence: The influence of Peter
Cooper on qualitative research’, International Journal of Market Research Society, 54, 5,
pp. 635–58.
Braun, V. and Clarke, V. (2006) ‘Using thematic analysis in psychology’, Qualitative Research
in Psychology, 3, 2, pp. 77–101.
Braun, V. and Clarke, V. (2013) Successful Qualitative Research, London: Sage.
Bryman, A. (2008) Social Research Methods, 3rd edition, Oxford: OUP.
Bryman, A. and Burgess, R. (eds) (1994) Analysing Qualitative Data, London: Routledge.
Buzan, T. and Buzan, B. (2003) The Mind Map® Book, London: BBC Worldwide.
Chilton, P. (2004) Analysing Political Discourse: Theory and Practice, London: Routledge.
Denzin, N. and Lincoln, Y. (eds) (1994) Handbook of Qualitative Research, London: Sage.
Downer, K., Wells, C. and Crichton, C. (2019) ‘All work and no play: A text analysis’,
International Journal of Market Research, 61, 3, pp. 266–86.
Edwards, D. (1997) Discourse and Cognition, London: Sage.
Fairclough, N. (1992) Discourse and Social Change, Cambridge: Polity.
Gebauer, J., Tang, Y. and Baimai, C. (2008) ‘User requirements of mobile technology: results
from a content analysis of user reviews’, Information Systems and e-Business Management,
6, pp. 361–84.
Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory, Chicago, IL: Aldine.
Gordon, W. (2016) MindFrames: 6 Enduring Principles from 50 years of Market Research,
London: Acacia Avenue.
Hofstede, G. (1984) Culture’s Consequences, London: Sage.
Hofstede, G. (1991) Cultures and Organisations: Software of the Mind, London: HarperCollins.
Jaworski, A. and Coupland, N. (2014) (eds.) ‘Introduction’, The Discourse Reader, 3rd
edition, London: Routledge.
Johnstone, B. (2017) Discourse Analysis, 3rd edition, London: Wiley.
Katz, J. (1983) ‘A theory of qualitative methodology: the social science system of analytic
fieldwork’, in Emerson, R. (ed.) Contemporary Field Research, Boston, MA: Little, Brown.
Lac, A. (2016) ‘Content analysis’ (pp. 1–5) in Levesque, R. (ed) Encyclopedia of Adolescence,
2nd edition, Switzerland: Springer International.
Lawes, R. (2017) ‘The things you are looking for have names: why brand owners and researchers need semiotics’, International Journal of Market Research, 59, 3, pp. 383–86.
Leeson, W., Resnick, A., Alexander, D. and Rovers, J. (2019) ‘Natural language processing
(NLP) in qualitative public health research: a proof of concept study’, International Journal
of Qualitative Methods, 18, pp. 1–9.
Martin, J. and Rose, D. (2007) Working with Discourse: Meaning Beyond the Clause, 2nd
edition, London: Continuum.
Miles, M. and Huberman, A.M. (1994) Qualitative Data Analysis: An Expanded Sourcebook,
London: Sage.
Morgan, D., quoted in Krueger, R. (1998) Analysing and Reporting Focus Group Results,
Chapter 8, p. 93, London: Sage.
MRS (2019) Code of Conduct, London: MRS.
Nowell, L., Norris, J., White, D. and Moules, N. (2017) ‘Thematic analysis: striving to meet
the trustworthiness criteria’, International Journal of Qualitative Methods, 16, 1.
311
M12 The Practice of Market Research 31362.indd 311
30/09/2021 18:27
Chapter 12
Analysing data from qualitative research
Ormston, R., Spencer, L., Barnard, M. and Sharpe, D. (2014) ‘The foundations of qualitative research’, in Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (eds.)
Qualitative Research Practice, 2nd edition, London: Sage.
Pich, C., Armannsdottir, G. and Dean, D. (2015) ‘The elicitation capabilities of qualitative
projective techniques in political brand image research’, International Journal of Market
Research, 57, 3, pp. 357–94.
Pich, C., Armannsdottir, G. and Spry, L. (2018) ‘Investigating political brand reputation
with qualitative projective techniques from the perspective of young adults’, International
Journal of Market Research, 60, 2, pp. 198–213.
Prior, L. (2014) ‘Content analysis’ in Leavey, P. (ed.) The Oxford Handbook of Qualitative
Research, Oxford: OUP.
Robinson, W. (1951) ‘The logical structure of analytic induction’, American Sociological
Review, 16, pp. 812–18.
Robson, S. and Hedges, A. (1993) ‘Analysis and interpretation of qualitative findings, Report
of the Market Research Society Qualitative Interest Group’, Journal of the Market Research
Society, 35, 1, pp. 23–35.
Robson, K., Farshid, M., Bredican, J. and Humphrey, S. (2013) ‘Making sense of online
reviews: a methodology’, International Journal of Market Research, 55, 4, pp. 521–37.
Stemler, S. (2000) ‘An overview of content analysis’, Practical Assessment, Research and
Evaluation, 7, Article 17, https://scholarworks.umass.edu/pare/vol7/iss1/17 (Accessed 13
December 2020).
Strauss, A. and Corbin, J. (1998) Basics of Qualitative Research, London: Sage.
Recommended reading
Ereaut, G. (2002) Analysis and Interpretation in Qualitative Market Research, London: Sage.
Goulding, C. (2010) Grounded Theory: A Practical Guide for Management, Business and
Market Researchers, London: Sage.
Miles, M., Huberman, A. and Saldana, J. (2018) Qualitative Data Analysis: A Methods Sourcebook,
4th edition, London: Sage.
Ritchie, J., Lewis, J., McNaughton Nicholls, C. and Ormston, R. (2014) (eds.) Qualitative
Research Practice, 2nd edition, London: Sage.
312
M12 The Practice of Market Research 31362.indd 312
30/09/2021 18:27
Part Five
Quantitative research
M13 The Practice of Market Research 31362.indd 313
30/09/2021 18:28
Chapter 13
Methods of data collection
Introduction
A quantitative research project involves collecting data from a relatively large
sample or population in a structured and standardised way. In this chapter we
look at the main methods of collecting this sort of data, their advantages and
limitations and their applications. We illustrate these with examples.
Topics covered
Research designs in quantitative research
● Asking questions and observing
● Interviewer-administered and self-completion
● Face-to-face data collection
● Telephone interview data collection
● Online data collection
● Mixing or switching modes of data collection
● Observational methods of data collection.
●
Relationship to MRS Advanced Certificate Syllabus
The material covered in this chapter is relevant to Topic 1: Understanding the
research context and planning the research project; Topic 2: Guiding Principles;
and Topic 3: Selecting the research design and planning the approach.
M13 The Practice of Market Research 31362.indd 314
30/09/2021 18:28
What you should get from this chapter
At the end of this chapter you should be able to:
demonstrate awareness of the range of methods of data collection in
quantitative research;
● understand the uses of each method;
● understand the limitations of each method;
● select the appropriate method or combination of methods for a given
research proposal.
●
315
M13 The Practice of Market Research 31362.indd 315
30/09/2021 18:28
Chapter 13
Methods of data collection
Research designs in quantitative research
The data collection ‘occasion’ in a quantitative research project will be designed
or structured to suit the objectives of the project. We looked at this in detail in
Chapters 3–5. The most common research designs in quantitative research are crosssectional or ad hoc studies in which data are collected at one point in time; longitudinal or panel studies in which data are collected from the same sample on more than
one occasion; and experiments (e.g. A/B tests) in which two (identical or matched)
samples are recruited and one is subject to a ‘treatment’ and the results are compared.
Within these designs or structures data can be collected in a number of ways. This is
the focus of this chapter.
Asking questions and observing
Quantitative data can be collected by getting people to answer questions; and by
collecting observations. We can put the questions and collect the observations in a
range of ways, including in person, face to face and over the phone, and remotely.
Asking questions
The questions are prepared in advance and appear on standardised, structured or
semi-structured ‘forms’ – an interview schedule or questionnaire, or sometimes in
a diary format or as an assessment form (for mystery shopping research). You get
people to answer the questions in one of two ways. You can ask them to fill in the
form themselves – this is called ‘self-completion’. For this method you need to get the
form to the person who must fill it in and you must get it back from them. You can
do this by post or using the internet, or you can do it in person. You can also place it
where your ‘targets’ are likely to find it (e.g. customer satisfaction surveys in hotels).
The second approach is to get an interviewer to ask the questions, either in person
or remotely, face to face or via the phone. The interviewer records the answers on
the form (from hereon, the questionnaire). This is called ‘interviewer administered’
research. We look at the role of the interviewer in some detail below.
The option you choose will depend on a number of things. You will need to determine how suitable the method is for the following:
the project and its objectives;
the topic or issues under investigation;
● ability to reach the right sample;
● ability to achieve the right numbers;
● the time and budget available.
●
●
It is important to remember that there is no one perfect method for a particular
research problem. Whichever method you choose will have advantages and limitations, strengths and weaknesses. You will be working with limited resources in
terms of time and money. There will be trade-offs and compromises. To decide
316
M13 The Practice of Market Research 31362.indd 316
30/09/2021 18:28
Asking questions and observing
which method is most suitable, you need to know the sort of evidence you need
the research to deliver. Once you know that, then you can evaluate the methods on
offer and determine which will be the best at delivering that evidence. You should
be able to justify your choice and present the rationale for it. Industry Insight 13.1
illustrates the issues involved in choosing – and adapting – approaches to interview
bank customers about a sensitive topic, being encouraged to switch to another
bank.
Industry Insight 13.1
Helping to make the switch
Introduction
As part of the conditions of its financial support
from the UK Government during the financial crisis, bank RBS had to reduce its market share and
increase competition in the small to medium enterprises (SME) banking market. It had to create an
environment where its eligible customers switched
to a challenger bank and at the same time it had to
maintain its own brand and reputation. To help it
achieve this it undertook a two-year programme
of qualitative research and quantitative ad-hoc
and tracking research in partnership with research
agency Savanta.
The research
The programme of research began with exploratory
qualitative research to build a clear understanding
of, among other things, the following:
barriers to switching;
drivers of switching;
● whether customers could be incentivised to
switch and what the best switching offer would
be; and
● customers’ reactions.
●
●
Next came price sensitivity research to find out if
it was possible to encourage customers to leave
with a monetary incentive and, if so, how much
that needed to be. This research comprised 900
ten-minute computer-aided telephone (CATI)
interviews. The researchers chose a telephone survey for two main reasons. First, the subject matter
was sensitive and required experienced interviewers to position it and to build rapport with participants. Second, this approach meant that they
could achieve a representative sample – the bank
had telephone contact details of its customers
whereas coverage of customers’ email addresses
was lacking.
The research revealed that, at the right level,
a cash incentive has impact and would lead to
an increase in switching levels. However, it also
showed that customers were not highly motivated
by a cash incentive. To understand what mix of
incentives would motivate customers to switch,
research was needed to simulate a realistic business
banking switch online marketplace, a comparisonlike website for customers to review offers. To do
this required online research. However, there were
issues with this approach: lack of customer email
addresses; and the low response rates associated
with this method.
The solution the researchers came up with was
a two-stage mixed method approach: a CATI survey of business customers; and a further online
survey with a sub-set of those customers. The
CATI survey comprised 15-minute interviews
with 1,600 eligible business customers. The aim
was to understand their attitudes to switching;
their knowledge of and sentiment about challenger banks; and to recruit them to take part
in a further piece of research online. Those who
agreed to take part were sent an email as soon as
the CATI interview finished. The email contained
317
M13 The Practice of Market Research 31362.indd 317
30/09/2021 18:28
Chapter 13
Methods of data collection
a link to a (conjoint) task that simulated the business banking switch marketplace, enabling the
participants to review realistic offers.
To have a big enough sample for analysis, the
researchers needed a 60 per cent response rate
to this online survey. The initial response, however, was much lower, 25 per cent. They recognised that the time involved in completing both
the telephone interview and the online task was a
challenge. To address this, they developed a new
approach. They sent an email reminder to the participant and an instant message to the interviewer
after the email was sent. The message was to
prompt the interviewer to follow up with the participant by phone to encourage participation. This
approach delivered the required response rate. It
succeeded by maximising the moment when the
research was freshest in the participant’s mind;
it ensured that the email was at the top of their
inbox when the interviewer rang; and it had a personal touch with the targeted and timely call from
the interviewer.
The data from the research was used with data
from the bank’s customer database to predict different customers’ propensity to switch. In addition,
the bank did research to test how best to communicate the sensitive nature of the switching scheme
and to select a name for it; mystery shopping to
ensure that bank branches and call centres were giving the right level of support; and ongoing customer
journey research to track awareness, sentiment and
experiences before, during and after switching.
The end result
The scheme was launched in February 2019. Customer experience research showed that in May 2019,
92 per cent of eligible customers were aware of the
scheme; the bank’s satisfaction rating remained
level; and 87 per cent of customers who successfully
switched between 11 March and 31 May 2019
agreed that switching has been made easy for them.
Source: Adapted from Savanta and RBS, ‘Business banking
switch – business unusual for RBS’, Winner, MRS Awards 2019.
Used with permission.
Observing
Quantitative data can be collected using observational techniques. The sort of observational studies that we deal with in this chapter are the relatively small-scale studies
designed to gather data for a specific set of objectives. We are not dealing with the sort of
data gathered automatically and passively for non-research purposes in the first instance
but data that are later analysed for research purposes using data mining and data analytics.
Observation can be carried out with or without a researcher present. For example,
cameras can be placed at the site of interest to record activity or an existing closedcircuit television system can be used; participants can be recruited to take part in a
project that involves them wearing a camera or a set of eye-tracking glasses. The audit
of stock in retailers, called a retail audit, and gathered together as retail panel data, is
an example of structured data collection using observation; the data are collected electronically. Industry Insight 13.2 gives an example of a project in which two methods
of observational data collection were used: fixed cameras and eye-tracking glasses.
The main advantage of observation over interviewing is that it enables you to
record actual rather than reported behaviour, what people do rather than what they
say they do. This was a benefit noted in Industry Insight 8.5 where data captured
at the box offices of arts venues proved insightful in understanding the behaviour
of arts attenders – more insightful than that reported via survey questionnaires. The
main disadvantage of observational research is that in most cases you are unable to
determine the reason for the behaviour. To overcome this, interviewing is often used
in conjunction with observation, as Industry Insight 13.2 shows.
318
M13 The Practice of Market Research 31362.indd 318
30/09/2021 18:28
Asking questions and observing
Industry Insight 13.2
Capturing the moment of truth
Introduction
capture habitual shopping behaviour;
evaluate in-store influences;
● understand when, where and how people consume cereals;
● create decision trees for cereal types to inform
packaging and merchandising;
● find out what the retailers are doing in store.
●
How do people shop for breakfast cereal? There is
so much choice, so many pack sizes, so many subcategories and so many promotions. Weetabix,
owner of multiple brands including Alpen, wanted
to find out how people navigate the busy retail
environment and make a choice. The company
has a huge amount of information on consumption and trends but it needed to understand that
‘moment of truth’ to be able to enhance the
performance of its brands in store. It needed to
understand how to attract and keep cereal shoppers, and be able to give bespoke guidance on how
to do this to the eight major retailers that stock
its brands.
The research
Weetabix commissioned research agency Trinity
McQueen. The agency, in collaboration with
the Weetabix team, designed a multi-method
approach that focused on the shopper. It aimed
to do the following:
●
The methods used to do this were as follows:
fixed in-store cameras;
in-store observation and intercept interviews;
● interviews with staff;
● accompanied shopping trips;
● eye-tracked shopping trips with post-trip
interviews;
● online survey on cereal usage and attitudes;
● analysis of panel data.
●
●
Table 13.1 shows to which information needs
the methods contributed.
The fixed cameras filmed shoppers over a
72-hour period. Viewing the footage allowed
Table 13.1 Methods and aims
Capture
behaviour
Evaluate
in-store
influences
Understand
consumption
Create
decision
trees
Find out
about retail
set-up
Fixed in store
cameras
✓
✓
✓
In-store
observation
and intercept
interviews
✓
✓
✓
✓
Staff
interviews
✓
✓
✓
Accompanied
shopping
✓
✓
✓
Eye-tracked
trips and post
trip interviews
✓
Online survey
Panel data
analysis
✓
✓
✓
✓
✓
✓
✓
✓
✓
319
M13 The Practice of Market Research 31362.indd 319
30/09/2021 18:28
Chapter 13
Methods of data collection
the researchers to examine customer interactions by cereal type; to identify recurring behaviour patterns; to measure ‘dwell time’; to look
at the number of subcategories shopped; and to
segment shoppers by mission (top up or main
shop). A sample of shoppers was recruited to
recreate their normal shopping trip wearing eyetracking glasses. This allowed researchers to see
what shoppers actually looked at when they were
shopping. They interviewed the shoppers afterwards to ask them about their behaviour. The
team also used an online survey and Kantar panel
data to understand when, where and how people
eat cereals. The panel data provided insight into
long-term purchasing trends and switching.
Use of the findings
The research insight changed Weetabix’s in-store
strategy and tactics. The approach now is to use
simpler packaging, fewer stock-keeping units and
fewer competing promotions. The changes make
it easier to find the Weetabix brands: the new
pack designs are consistent and stand out at the
point of purchase; they are merchandised together
and they prime shoppers as they enter the cereal
aisle; and coordinated promotions optimise choice
architecture so making it easier to shop for cereals.
Here’s what happened with muesli brand
Alpen. Panel data showed that premium muesli
brands had eroded its market share. Eye-tracking
revealed that shoppers found it hard to find on
the shelf. Interviews showed that its heritage was
not clearly communicated. Weetabix refreshed
Alpen’s packaging and proposition, returning to
Alpine imagery and Swiss brand heritage. The
bolder colours and simplified design gave the
packs improved standout on the shelf.
The changes have made it easier for people to
shop for cereal and at the same time improved the
commercial outcomes for retailers and for Weetabix. The better shopping experience has led to
improved sales with Weetabix seeing growth in a
declining market and achieving its highest market
share on record. The insight supports Weetabix’s
position as a category expert with comprehensive
knowledge of the cereal shopper. This has helped
it to build trusted and collaborative partnerships
with retailers. Weetabix now has a new shopper
insight team to help embed the shopper voice and
to work with retailers to create strategies for the
cereals category.
Source: Adapted from Trinity McQueen and Weetabix. ‘The
shopper moment of truth’, Winner, MRS Award, 2017. Used with
permission.
Interviewer-administered and self-completion
In this section we look at the two ways of getting participants to answer questions:
with the help of an interviewer; and without the interviewer (self-completion).
Interviewer-administered
The interviewer has two jobs to do: contact people who match the sample criteria
of the survey and encourage them to take part, and administer the interview. This
is a skilled job. It requires a high level of interpersonal skill, a sound understanding
of the data collection and research processes. Part of the rationale the researchers give for doing a telephone survey in Industry Insight 13.1 includes the skills
of the interviewer in positioning the research and establishing rapport with the
participant.
320
M13 The Practice of Market Research 31362.indd 320
30/09/2021 18:28
Interviewer-administered and self-completion
The effect of the interviewer
Interviewers are not all the same and nor are participants. An interviewer may react
to or interact differently with different participants, and participants will react differently to different interviewers. Much research has been done on the effect an interviewer has on response rates and on the quality of data collected. There is evidence
to show that appearance, age, gender, social grade, ethnic background, religion and
attitude or personality have an effect on the interviewing process and on the outcome
of the interview. This is not confined to face-to-face interviews. Research shows that
participants in telephone interviews make judgements about an interviewer’s characteristics on the basis of their voice. To minimise the effects of interviewer variance
interviewers are trained to carry out interviews according to the instructions provided
and to do so in a professional, courteous and objective way. The MRS provides a very
useful guide for interviewers based on the MRS Code of Conduct, Responsibilities
of Interviewers (2019).
Uniformity of approach
A questionnaire will have been designed to gather data from a relatively large number of people that make up the sample or population under study. Due to the number of interviews needed it is likely that more than one interviewer will be involved.
Uniformity or consistency of approach is a key aim in structured and standardised
quantitative research – data must be collected in the same way across the sample
and any possible bias or errors (part of the family of non-sampling errors) in asking questions or recording responses must be kept to a minimum. It is important
therefore that each participant is asked the questions on the questionnaire in exactly
the same way. This means that the interviewer must read out instructions and ask
the questions exactly as they appear on the script or questionnaire, and in the
way that they were briefed to do (a change of emphasis on a word can change the
meaning). With closed, pre-coded questions the interviewer selects or records the
code that applies to the participant’s answer. For some questions, such as those
with an ‘other’ code in the list of pre-coded responses, or where the participant
says ‘Don’t know’ or ‘Not sure’, the interviewer may need to probe (depending
on what it says in the briefing notes or instructions given during training). Where
there are open-ended questions, questions that require the participant to answer in
their own words, the interviewer must record the answers verbatim. If probing is
needed, to elicit a more detailed response, the interviewer must follow the specified
probing/prompting procedure set out on the questionnaire or specified in training
and in the briefing for the particular study. The interviewer must record the result
of the probing/prompting.
All this means that the interviewer must be familiar and comfortable with the
questionnaire and the interviewing process. Two things are vital here: interviewer
training and project briefing. Questionnaire design also has a role to play: the person
designing the questionnaire has a responsibility to the interviewer to ensure that the
questionnaire is clear, logical, easy to follow and set out in such a way that makes it
easy for the interviewer to record responses. We return to this later (see Chapter 15).
321
M13 The Practice of Market Research 31362.indd 321
30/09/2021 18:28
Chapter 13
Methods of data collection
Training
Typically, interviewers are trained by the research agency or fieldwork company for
whom they undertake work. This training will usually have involved one or two days
of ‘theory’ in the classroom covering the following:
how to find the right participant;
how to obtain and record information to determine the participant’s social grade;
● how to get the participant to agree to an interview;
● explaining the nature of the interview and the time needed to conduct it;
● explaining about confidentiality, and the use of the personal details collected;
● the importance of asking questions and reading out instructions exactly as they
appear on the questionnaire;
● the importance of coding pre-coded responses accurately;
● the importance of recording responses to open-ended questions verbatim or as
close as possible;
● the extent of probing allowed or required and the manner in which probing is to
be done (and how this should be recorded);
● how to use the data collection equipment;
● how to complete all paperwork accurately.
●
●
Office-bound training is followed by some practice interviews in the field under the
supervision of a senior interviewer for face-to-face interviews, on the telephone with
a supervisor or senior interviewer listening. Further on-the-job training takes place
at regular intervals. This is part of the overall quality control procedure that is part
of the management of a fieldforce. Fieldwork quality control also includes checking
and monitoring the interviewers’ completed work.
Interviewer briefing and de-briefing
Interviewers are briefed in detail about the requirements of each particular job. The
aim of a briefing is to ensure overall consistency of approach – by making sure
that they understand clearly how to administer that particular questionnaire, and to
address any concerns or questions that they may have about it. The client service or
field executive or manager will give the briefing, although it may sometimes involve
the person commissioning the work (the client). We look in detail at what is involved
in a briefing later (see Chapter 19). It can also be useful to de-brief interviewers after
a fieldwork session to find out how things actually worked or did not work ‘live’
in the field so that further or future work can be improved. Interviewers can also
provide insights into how participants reacted to the questionnaire or to particular
questions and give their perceptions of how engaged the participant was, and how
honest (Orton and Samuels, 1988 and 1997). Interviewers can also be asked about
the characteristics of those who do not respond or refuse to take part. This can be
useful in understanding non-response and non-response bias.
322
M13 The Practice of Market Research 31362.indd 322
30/09/2021 18:28
Self-completion
Self-completion
A self-completion survey is one of the most cost-effective ways of collecting data,
mainly because no interviewers are involved. It can be administered by post or online,
via email or the web, or presented in person or left in a place where the target group
has access to it. Depending on the population of interest, you may have a comprehensive sampling frame from which to choose a random sample, for example the Postal
Address File or a full list of email addresses, which makes the approach feasible.
A self-completion questionnaire can also be included as part of a personal interview to
collect data on sensitive subjects where the participant might be embarrassed to provide answers to an interviewer. The participant can complete it when the interviewer
is there or it can be left with them to complete and return later. The Life and Times
surveys, which are featured in this book, include a self-completion element in their
in-home interviews. Self-completion is also useful in situations where it is not necessary to have an interviewer ask the questions, during a product or advertising test, for
example. Diaries are a specialised form of self-completion survey – they can be used,
for example, to gather data on product usage or eating or shopping or viewing habits.
Self-completion surveys are an effective method of collecting data if you ensure that:
the nature of the research and the topic are suited to this method of delivery;
the topic is relevant and of interest to the target population;
● the method is a suitable way of reaching and achieving a response from the target
population;
● the questionnaire is well designed – clear, easy to follow and easy to complete and
a suitable length for the medium – and presented in a professional manner.
●
●
Success in encouraging response – on which the representativeness of the sample
relies – depends on all of these. Response rates for self-completion methods tend to
be much lower than for interviewer-administered methods and this threatens representativeness. Many online surveys, for example, achieve response rates of 10 per
cent or less. Before deciding to use this method it is worth asking whether the subject
matter is interesting enough to the sample, and worth finding out (from the literature
or from previous research) the response rate you might expect. If you are prepared
to tolerate a relatively low response rate then you must investigate non response so
that you understand any potential non-response bias in the data.
Techniques to increase response rate
Without an interviewer present response rates in self-completion surveys can be low.
While the match between the research topic and the population of interest, and the
design of the questionnaire or diary or any other materials the participant sees are crucial in trying to maximise response rate, there are other techniques or procedures that
can also help. Industry Insight 13.3 gives an example of the use of an AI chatbot to
encourage participants to record what they eat and why in a food consumption diary.
323
M13 The Practice of Market Research 31362.indd 323
30/09/2021 18:28
Chapter 13
Methods of data collection
Industry Insight 13.3
Help with your diary
Introduction
South African retailer Woolworths wanted to
know what people are eating, when and why. They
knew that traditional food diaries would not be
effective because people stop filling them in when
they no longer enjoy doing so. So Woolworths
worked with agency Kantar to develop a more
effective approach.
The research
Kantar designed a seven-day research study that
used an artificial intelligence chatbot diary. The
aim of the diary was to gather data on when,
what, how and where participants were eating
as well as data on preferences and motivations.
Participants were recruited through Facebook
ad targeting. They received a daily incentive for
completing each day of the diary and an additional incentive if they completed the full seven
days. New weekly waves of participants submitted diary entries over a fieldwork period of one
month. In total, the team collected data on behaviour by hour, day, week and month from 1,100
people – more than 10,000 meal occasions and the
when, what and why of each one.
To help with participation Kantar develop a
chatbot named ‘Serena’ (female to reflect the fact
that 70 per cent of the customer base are women).
Serena acted as a coach, prompting participants
to share photos and updates on their meals and
snacks every three hours. She asked questions
about whether there were any specific reasons
why they had chosen that particular meal or
snack, aside from hunger and the benefits over
something else. Serena spoke in the first person
with a script and questions designed to show an
outgoing ‘persona’. Participant engagement with
Serena was evident in the detailed responses captured in the diary.
Kantar used Natural Language Processing
(NLP) to analyse the data, uncovering the more
than 800 ingredients participants mentioned in
the food they ate. They also coded the reasons
people gave for eating particular meals. The
themes that emerged included healthy eating and
indulgence. The researchers were able to pinpoint
the ingredients related to a particular theme –
chocolate, yoghurt and fruit for indulgence – and
map the motivation of indulgence, seeing how
it changed by time of the week or month. This
helped Woolworths understand the trends it was
seeing in its transaction data. For example, it
saw a lack of interest in promotions for salad or
other healthy options on a Friday; the diary data
revealed that people were in a healthy mindset
early in the week and more interested in indulgence on a Friday.
The end use of the findings
The insight from the diary data, as Woolworths
noted, has significant strategic and practical applications across many areas of its business including marketing, pricing, product development and
trade teams.
Source: McQuater, K. (2020) ‘Decoding cravings’, Impact, 29,
pp. 16–17. Used with permission.
Other techniques for encouraging response include use of a personalised covering
letter or email, sponsorship, advance or pre-notification of the survey, reminders to
complete and return the questionnaire, incentives and, for postal surveys, a return
envelope. We look at each of these in turn.
324
M13 The Practice of Market Research 31362.indd 324
30/09/2021 18:28
Self-completion
Invitation
Postal and online questionnaires may be accompanied by a ‘covering’ letter or email
or SMS, an invitation that is personalised if possible, as this has been found to increase
response rates (Yu and Cooper, 1983). The aim of the invitation is to do the following:
explain the nature of the survey and why it is being conducted;
explain why and how the recipient was chosen;
● reassure the recipient about the confidentiality and/or anonymity of the information they provide;
● state that participation is voluntary and that they can refuse to answer any question;
● give details of any incentive for completing the questionnaire;
● give details of the date by which the questionnaire should be completed (and
returned); and
● for postal surveys, details of how it should be returned (a pre-paid envelope is
usually included).
●
●
Advance or pre-notification
Depending on the sample and the nature of the research, it may help to inform the
sample in advance of the arrival of the questionnaire. This has also been found to
improve the response rate (Yu and Cooper, 1983). Pre-notification can take the form
of a letter, an email, a message or a telephone call.
Sponsorship
It can be helpful in encouraging participation to include on the questionnaire or mention
in the invitation the name of the organisation sponsoring or involved in the research.
Reminders
In most postal and online surveys at least one reminder is sent – usually only to those
who have not returned the questionnaire after a specific period of time. With postal
surveys a reminder is usually sent after two to three weeks; with online surveys it
depends on the time frame for the survey. The reminder should be carefully worded
to encourage response and not deter it. In most cases a second copy of the questionnaire is attached or a link provided to the questionnaire, in case the first has been
misplaced, destroyed or deleted.
Incentives
Incentives are used to encourage response and to thank participants for the time
taken to complete a survey. Industry Insight 5.4 (Brennan, Hoek and Astridge, 1991)
examines the effects of incentives in a postal study: of the incentives offered, a small
325
M13 The Practice of Market Research 31362.indd 325
30/09/2021 18:28
Chapter 13
Methods of data collection
monetary incentive with the first mailout proved most effective. In their research on
incentives among online access panel members in six countries (UK, USA, Canada,
Australia, France and Germany), Dubreuil and Murray (2012) found that cash
was by far the most popular, followed by vouchers then charity donations. In their
research on the factors that motivate panel members to take part in surveys, Brosnan
et al. (2019) discovered that incentive payments are the main driver (others included
short completion time, interest in the topic and ease of completion).
Return envelopes
For postal surveys, a stamped or reply-paid envelope is usually included to encourage
and facilitate response.
Strengths of self-completion methods
Postal and online surveys have a number of strengths. They are relatively easy to
set up and administer. They enable you to reach a widely dispersed population, and
one that may not be amenable to research by other methods. They are an effective
way of asking questions that need time for consideration or involve the participant
in checking or consulting documents. They are relatively cost effective as there are
no interviewers to pay, and with no interviews they are free of interviewer bias or
error. Also, having no interviewer, they offer participants a high degree of perceived
anonymity – which means they are effective in collecting data on sensitive topics
and effective for reducing the risk of social desirability bias. Response rates can be
monitored relatively easily. Turnaround times from end of fieldwork to production of
data tables for online surveys is short compared with postal surveys: data (including
verbatim responses) are captured directly, which also reduces data processing time
and DP errors. Online surveys can be designed to skip automatically to relevant questions; and you can control how much of the questionnaire the participant sees before
filling it in. In addition, you can set up the questionnaire in different languages and
allow participants to choose the language in which they wish to answer.
Elkasabi et al. (2014) ran a series of experiments to evaluate the feasibility of
moving data collection for a consumer survey in the US from a telephone landline
survey with the sample achieved using random digit dialling (RDD) to a postal (mail)
survey with address-based sampling. They found that response rates were similar
but that with the mail survey coverage of the population of interest improved. They
found ‘no apparent measurement or reporting differences’ between the two modes.
Weaknesses of self-completion methods
There are drawbacks to self-completion approaches. Although postal surveys can
be relatively cheap in comparison with other methods, the cost per completed
interview, especially if a survey of non-responders is conducted, may be greater.
Response rates vary – they can be as low as 5 to 10 per cent. With a poor response
rate, or one that is hard to predict, there is a chance that the sample will not be
326
M13 The Practice of Market Research 31362.indd 326
30/09/2021 18:28
Self-completion
representative of the population: those who respond may differ from those who
do not – non-response bias, hence the need for a survey of non-responders. A key
aim should be to reduce non-response bias. In addition, the sample is self-selecting.
Although you might choose a sample relevant to the research and send questionnaires and reminders to that sample, you have no control over who completes the
questionnaire (or how many do so). The recipient decides whether or not to take
part; they may pass the questionnaire to someone else, or someone else other than
the intended recipient may complete it or help the intended recipient complete it.
The lack of control over the data capture process has a knock-on effect in terms
of data quality:
The participant can consult with others before answering the questions.
Participants may not answer all the questions they were supposed to or in the way
required.
● You may get little detail in open-ended questions.
● There is no opportunity to probe or clarify answers – you must accept the response
given by the participant.
● Questions requiring spontaneous answers do not work well.
● The participant can (in some formats) skip ahead or indeed read the whole questionnaire before filling it in, so any ‘funnelling’ of questions and topics does not
work.
● There is no opportunity to observe, for example, or to read body language or hear
tone of voice.
●
●
Pilot studies are essential in any self-completion methods where no interviewer is
present to help mediate the participant–questionnaire experience. It is vital to know
if the questions and the instructions make sense to the participant. In online surveys
they are essential to ensure that the questionnaire works on different platforms and
devices – the browser type or screen size can affect the format of the questionnaire,
how it looks to the participant and how easy or difficult it is to complete.
Face-to-face methods of data collection
Depending on the nature of the survey, face-to-face interviews may take place in the
participant’s home, in the street, in a central location, for example in a hall or in a
shopping centre or mall, or at the participant’s place of work. Thus, if you need a
quota sample of consumers, the interview is about product preferences and is likely
to last no more than about ten minutes, you could recruit and conduct interviews in
the street or in a shopping mall. If, however, you are conducting a random sample
survey on household spending that lasts up to 35 minutes, a face-to-face, in-home or
doorstep interview may be more appropriate.
Street interviews
Street interviews are conducted in busy streets, mostly in town centres where there
is a lot of pedestrian traffic. The interviewer approaches people who seem to fit the
sampling criteria, if the research is being conducted using a quota sample; if a random
327
M13 The Practice of Market Research 31362.indd 327
30/09/2021 18:28
Chapter 13
Methods of data collection
sample is required, the interviewer approaches the nth passer-by and requests an
interview. Street interviews usually last no more than ten minutes – people will not
stand around answering questions for any longer. The topic of the interview must be
one that most people are content to talk about on the street. The amount of stimulus
material that can be shown is limited. Industry Insight 13.4 is an example of a project
that included street ‘intercept’ interviews.
Industry Insight 13.4
Shining a light on print ads
Introduction
Supermarket chain Tesco spends a large amount
on print advertising every year. It is a good way
to communicate weekly offers to a lot of people at
speed. But there was limited understanding of how
print ads actually work in communicating these
offers, and the marketing team and its agencies
spend a lot of time creating the ads and discussing
them before they are finalised. If Tesco had a commonly accepted set of design principles, principles
that it knew worked, it could design more effective ads in a shorter period of time. Tesco commissioned Lumen Research to help. Lumen runs a
print ad omnibus study and has data on thousands
of ads including supermarket ads.
The research
First, we analysed the Lumen print ad database to
understand the reality of attention to these ads. We
found that it is finite and rare but once it passes the
two-second threshold, extremely valuable. Next we
designed a series of A/B experiments on creative
designs to isolate the factors that most effectively
drive attention, awareness and sales. We used the
print omnibus to do this. Each week we recruit a
sample of 80 participants using in-street intercepts
and ask them to read a pdf of a tabloid newspaper
on a 27” screen while having their eyes tracked
using specialist tracking cameras. After they read
the paper, we ask them to complete a short recall
and brand perception questionnaire. We can insert
ads into the newspaper pdf and so test ads in context before they go live. We can benchmark the
performance against a set of norms from several
years’ worth of tests using the omnibus.
The use of the findings
The findings from this research laid the foundation for creative excellence. We developed ‘rules
of thumb’ to help Tesco and its agencies save time
and effort at the idea stage of creative development. We also changed our processes to ensure
that we provided insight from tests to the Tesco
team within 48 hours. In the light of the findings
Tesco has simplified its advertising and improved
its media buying strategy. Research findings show
that its ads are performing much better, and Tesco
knows why. It now has a deep understanding of
the reality of attention which allows it to design
ads that replicate the drivers of success and create
predictable results.
Source: Adapted from Lumen Research and Tesco, ‘A three-part
strategy to optimise Tesco’s trade driving print’, Winner, MRS
Awards 2019. Used with permission.
Shopping centre/hall tests
The main advantage of interviewing in shopping centres or malls (or rooms or
halls off busy shopping streets) compared with interviewing in the street lies in the
comfort of the environment – interviewer and participant are protected from the
weather and the centre is traffic free. This allows a slightly longer interview, up to
328
M13 The Practice of Market Research 31362.indd 328
30/09/2021 18:28
Self-completion
about 15 minutes. In addition, the layout of the centre may be such that it is possible to set out an interviewing station with tables and chairs at which to seat the
participant. As a result the shopping centre or hall may be used for what are known
as ‘hall tests’ – longer interviews that would not be feasible in the street, lasting up
to about 30 minutes. This format also allows scope for exposing the participant to
stimulus material – for example tasting a product. Shopping centres and halls are
private property and permission must be obtained in order to conduct fieldwork;
a fee for their use is usually payable. Where necessary and relevant, interviews can
also take place inside shops, but permission must again be obtained. MRS publishes
a checklist (2016) on its website of the sort of issues you need to address if you
are planning on conducting a hall test which involves the testing of food or drink
products.
In-home interviews
In-home interviews are conducted in the home of the participant or on the doorstep.
In-home interviews may be used for several reasons. It may be necessary to recruit the
sample by going door to door to specific addresses (for example addresses chosen at
random from a sampling frame such as the Electoral Register or the Postal Address
File) or by going to specific areas or streets identified under a geodemographic classification system as containing the type of people likely to meet the sampling criteria.
It may be that the home environment is the most suitable place for the interview – it
may be necessary to refer to products used in the home or it may be a sensitive topic
and the home may be the most relaxed environment for asking such questions. It may
require the interviewer to record observations, for example the brand and model of
computer or television – something that the participant is unlikely to remember in
detail. Interviewing in-home allows a longer interview to take place, usually about
45 minutes to an hour.
Workplace interviews
Workplace interviews are suitable when the subject matter of the interview is related
to the participant’s work. The interview is conducted in the participant’s office or in
a suitable meeting room or in a quiet area; somewhere, if possible, where interruptions can be kept to a minimum.
Strengths of face-to-face data collection
In a face-to-face encounter the interviewer has the chance to build rapport with
the participant, which can help achieve and maintain co-operation and increase the
quality of the data. Response rates can be relatively high in comparison with other
methods (around 55 to 60 per cent compared with 5 to 10 per cent for online). In
fact, face-to-face surveys tend to achieve the highest response rate and the best data
quality of any mode and are sometimes referred to as ‘the gold standard’ (Mouncey,
2014 quoting Krosnick). Krosnick et al. (2015) note that they achieve ‘the most
cognitive effort . . . in generating answers accurately, and the most honesty when
329
M13 The Practice of Market Research 31362.indd 329
30/09/2021 18:28
Chapter 13
Methods of data collection
providing reports regarding sensitive topics’. Face-to-face methods allow for a relatively high degree of flexibility in the interviewing process – the interview can last up
to an hour; stimulus material can be used; complex questions can be explained and
administered; and probing and prompting can be carried out. In central location or
hall tests the environment of the interview can be controlled.
Weaknesses of face-to-face data collection
There are some disadvantages with face-to-face data collection, particularly in relation to in-home interviews. It is expensive and time consuming. It is difficult to cover
remote or rural locations. Cluster sampling methods, which serve to reduce travel
time and costs, risk introducing sample bias. Representativeness of the sample can be
affected in other ways: interviewers may find it difficult to obtain interviews in some
areas, or may be reluctant to interview in some neighbourhoods, from a safety point
of view, for example; indeed potential participants in any neighbourhood may be
unwilling to open the door to a stranger. Finding participants at home (or at work) at
a suitable time (and willing to take part) can be difficult. To overcome this it is often
necessary (and essential for business-to-business interviews) to make an appointment to set a suitable time, which further adds to the cost and the time needed for
the survey. Interruptions from other members of the household or workforce or the
presence of someone else in the room during the interview can impact on the quality of the data collected. There is a greater tendency in face-to-face methods for the
participant to give socially desirable responses – to appear in the best possible light.
With quality control procedures at a greater distance than in telephone interviewing
(where interviewers’ work can be continuously monitored) there is greater scope for
interviewer bias or cheating.
Telephone interview data collection
Most data collection using telephone interviewing is conducted from specialist
­telephone units or centres with computer-aided telephone interviewing (CATI)
systems to landline and mobile phones. Multi-country studies can be conducted from
a central telephone unit, enabling greater control over administration and delivering
the sort of consistency such projects demand. Using a centralised facility allows
face-to-face briefings for interviewers to be conducted, and executives are on hand
to answer queries during fieldwork.
There are differences between fixed line and mobile telephone interviews in terms
of response rates, sample composition and data quality (Vicente et al., 2009) that
it is important to take into account when planning fieldwork. A key issue in using
mobile numbers – a fundamental one – is, according to Vicente (2017), to find ‘ways
to minimise the effort required to make contact and gain cooperation in order to
achieve high response rates and minimise non-response bias’.
Vicente (2014) found in a study conducted in Portugal that establishing contact
with a potential participant on a mobile phone is harder than getting a completed
interview. (The same applies in fixed line surveys.) Across 19 different calling periods,
Vicente reports the contact rate to range from 13 to 33 per cent and the interview rate
330
M13 The Practice of Market Research 31362.indd 330
30/09/2021 18:28
Telephone interview data collection
to be between almost 40 per cent to 60 per cent. The contact rate was influenced by
day of the week and not by time: better days for contact were Saturday, Friday and
Monday. The day or the time on their own did not significantly affect the chance of
obtaining an interview. Vicente found that the best days and times were Thursdays
and Fridays from 5pm to 9pm; Saturdays from 10am to 2pm and Mondays from 7pm
to 9pm. This pattern, Vicente notes, is different from that found with fixed line phone
surveys where weekday evenings and not afternoons are good. However, this may
vary by country, as Vicente notes. It is nevertheless important to have ‘an efficient
calling schedule’ in order to achieve a good response rate in a mobile phone survey.
Fixed line telephone interviews typically last about 15–20 minutes. If the subject
matter is of interest to the participant longer interviews are possible. Interviews on
mobile phones tend to be shorter, more in the order of 10–15 minutes. As Vicente
et al. (2009) note, people can be under special time constraints when using a mobile
phone – to do with their location, the cost of receiving the call or concern about battery failure, for example. If the mobile numbers in the sample are randomly generated, you will know nothing about the person who answers the phone, not even their
location. This has a number of implications which you need to take into account in
planning and designing research that contacts people on mobile phones. You will
need to determine whether the participant is eligible to take part in the survey (e.g.
in terms of age, or location); whether they are in a safe environment in which to do
so (e.g. that they are not driving, that the interview cannot be overheard if personal
and/or sensitive data are being collected); and that taking part in the call will not
cost them money.
Strengths of telephone interview data collection
Telephone interview data collection has a number of strengths. From a methodological point of view, a telephone interview offers the participant a greater degree
of perceived anonymity than does a face-to-face interview. As a result, it is a useful method for collecting data on sensitive or intimate subjects, and it is useful in
reducing social desirability bias. A geographically dispersed sample (including those
in remote and rural areas) can be obtained more easily. With mobile phones, many
groups of people are more accessible than ever. A telephone survey may make it
easier to reach a wider spectrum of participants. It may be the only way of reaching some populations such as the business community. Van der Heijden (2017)
describes CATI as the ‘gold standard’ approach in terms of ‘non-cost criteria’ for its
customer satisfaction research for a longstanding client, a top service provider. She
lists these criteria as ‘impact on the customer; attribution, confidentiality and data
protection issues; potential biases leading to skewed and unrepresentative scores
(non-response, age, gender, geographic); representing outlet and organisation structure; and scalability’.
The method also offers a number of practical, logistical advantages. It is possible
to record answers to open-ended questions in full. Greater quality control is possible
(and so cheating is minimised), with interviewers monitored ‘live’ rather than after
the event via back-checking. Clients and researchers can listen in, enabling them to
get a feel for the findings. It is relatively easy to monitor interview length and the time
taken for individual questions – this can facilitate a dynamic or rolling pilot study and
questions can be altered if necessary. It is also relatively easy to determine the strike
331
M13 The Practice of Market Research 31362.indd 331
30/09/2021 18:28
Chapter 13
Methods of data collection
rate and refusal rate and so monitor the sample and control quotas. Call-backs are
easily managed so that bias towards those more often at home is reduced. Telephone
interviewing is faster than face-to-face interviewing – more questions can be asked
in a shorter period of time and project turnaround times can be shorter. A telephone
survey can therefore be more cost effective than a face-to-face survey.
Weaknesses of telephone interview data collection
The key issue in deciding to use the telephone as a method of data collection is to do
with getting access to the population of interest. In the past, using methods such as
Random Digit Dialling (RDD) and RDD + 1, it was possible to reach a random sample of the population of households in many countries because fixed line telephone
ownership was widespread. The proportion of the target population that was missing – because they did not own a fixed line – was relatively small (although it was
different from the rest of the population). Nowadays, the proportion of households
with a fixed line is in decline while the proportion of those with mobile phones has
increased. More than that, the proportion of mobile-only households is increasing.
The demographics of the two are different – mobile phone users tend to be younger
and are more likely to be in employment.
These changes in the nature of telephone ownership have had an impact on
telephone data collection (Elkasabi et al., 2014). There are no valid, reliable sampling
frames for mobile phone numbers. RDD is possible, of course, but it runs the risk
of finding many non-allocated numbers. There is the issue, too, of not knowing
the location of the phone user (and where they are at the time of the call), which
can be problematic if you need to conduct a survey in a particular geographic area.
Extensive screening may be required to determine eligibility. With fixed line numbers
and the area codes that go with them, this is not an issue. Directories are, of course,
available for fixed line numbers and were used as sampling frames in the days of
almost universal fixed line ownership and use but they no longer represent sufficient
coverage of fixed lines to be useful. Not all fixed line telephone numbers (and so
individuals and households or businesses) are listed. There is also the issue of more
than one fixed line per household and more than one mobile phone or SIM card per
person. In addition, the use of telemarketing has made people suspicious of bona
fide telephone research and this has impacted on response rates, as has the incidence
of voicemail, caller ID and call screening, and fixed lines being used to access the
internet. All have had an impact on the ability of researchers to access samples by
phone and conduct good-quality telephone research.
Mouncey (2014), reporting on work by Krosnick, notes that data quality may
be compromised because of ‘productivity pressures’ that lead to ‘rushed interviews,
causing satisficing by participants’. A further limitation of the telephone interview
is that some of the benefits of social interaction and the chance to build rapport
with a participant are lost. It can be easier for the participant to refuse an interview
or end it early and harder for the interviewer to encourage the person to take part.
Long and complex questions are best avoided. It is difficult to include stimulus
material, although this can be overcome by sending material out to participants
in advance of the interview. These limitations are prominent in surveys that use
interactive voice response or IVR. Van der Heijden (2017) describes pilot studies
with IVR to assess its viability as an alternative or as an addition to a CATI survey
332
M13 The Practice of Market Research 31362.indd 332
30/09/2021 18:28
Online data collection
on customer satisfaction. Invitations to take part were issued by SMS (text message) with a choice of how to complete the survey: a link to an online survey; and
a toll-free number to do the survey using IVR. For the IVR option, the participant
calls the number and a recorded voice reads out the survey questions; the participant answers using the numbers on their phone keypad. The questions used were
simplified versions of those used in a CATI version asked by an interviewer. These
simplified questions limited the amount of information that could be gathered:
questions with multi-coded answers, for example, had to be converted to binary,
yes/no responses. The pilot studies achieved very low response rates. The profile of
those responding to the IVR option was ‘highly skewed to older participants’. For
these reasons, among others, Van der Heijden concluded that the approach was
not viable as an alternative or addition to CATI.
Online data collection
There are several ways of collecting data online. We look at three approaches:
email surveys; web surveys; and mobile surveys. The source of the sample for many
online surveys is an online access panel. We look at what this is and at aspects of its
operation.
Email surveys
Email surveys are those sent to a sample of email addresses with the survey questionnaire either embedded in the email or provided as an attachment. It is important to
bear in mind the size of the mailing – the number of addresses to which the survey
is sent – and the size of return traffic and the effect this will have on server capacity. With a large mailing it can be a good idea to spread it out over a period of time
in order not to swamp the server. Some ISPs block mailings over a certain size and
most organisations have firewalls which can stop large-scale mailings. It may be
necessary to encrypt the questionnaire or email to comply with data security and
data protection requirements. If the individual email or the attachment is too big,
the participant’s email provider may reject it. Pre-notification is important in email
surveys. A questionnaire sent to participants who have not agreed to take part may
be rejected, and the sender blocked from sending others. The sample may be notified
about the survey by email, telephone or post.
All the good practice recommended for a covering letter or email outlined above
also applies here. The sample for an email survey may be recruited via traditional
routes or via the web or from a sampling frame – an organisation’s internal mailing
list or staff directory or a client’s customer database, for example. If a sampling frame
is to be used then it is important that it is accurate. Email does not tolerate wrong
addresses. In addition, people tend to change email addresses more often than they
change postal addresses and tend to use more than one email address so the sampling
frame must not only be accurate but it must be up to date. Conducting effective email
surveys can be difficult because of problems obtaining a suitable sampling frame and
as a result the data they produce lack reliability and/or validity. Van der Heijden
(2017) notes how an online survey has limited application for a client holding very
333
M13 The Practice of Market Research 31362.indd 333
30/09/2021 18:28
Chapter 13
Methods of data collection
few customer email addresses and for whom digital customers are not the only customers. In Industry Insight 13.2, the researchers also highlighted this as an issue and
chose instead to conduct a telephone survey. Email surveys are, however, a useful
way to conduct research where there is an accurate, up-to-date and easily available
sampling frame.
As with other forms of online surveys, an email survey should take no more than
15 minutes to complete, and, as with all self-completion data collection methods, it is
essential that the questionnaire looks good, is well-designed, with questions appropriate to the format, and is easy to fill in.
Web surveys
Web surveys are those conducted on the web, usually at a specially designed private
web address to which a sample is directed and/or given log-in details and a password.
They are a popular method of data collection because they are relatively quick and
cost effective to run.
Sampling for web surveys can happen in several ways. A sample can be recruited
offline, using a sampling frame, for example or it can be recruited online. Traffic
(people browsing the web) can be intercepted on a website – it is useful here to
think of the analogy of the interviewer stopping people in the street – by alerting
them via advertising banners or pop-ups. This approach is sometimes referred to
as river sampling or random web interviewing (Comley, 2003). A sample can also
be recruited from an online access panel. We look at these online panels below.
Another approach is that described by Haikel-Elsabeh et al. (2019). Here a survey
was sent from a Facebook brand page to examine factors that lead to the sharing
of brand content. The researchers created a Facebook application with a link to the
survey. The application enabled them to capture the personal data of those taking
part in the survey. The personal data included number of Facebook friends, number
of brands liked, number of comments and likes in general and on the brand of interest to the researchers. The first page of the survey explained, among other things,
that the data collected were being used for research purposes and participants were
asked for their consent for the data collection. They could refuse and still take part
in the survey.
The issue with web surveys lies in the use of non-probability sampling methods. If
non-probability methods are used then they are not suitable for providing population
estimates. This is evidenced by Erens (2014): Erens and colleagues compared findings from four volunteer web panel surveys on sexual behaviour and attitudes with
the findings from the British National Survey of Sexual Attitudes and Lifestyles, a
probability sample survey with interviews conducted in the participants’ home. The
most sensitive questions in the in-home interview were answered using computeraided self-interview (CASI). Erens reports that the web surveys gave less accurate
results than the probability sample survey; that some of the differences could not be
explained by mode and concluded that selection bias must also be present; and that
while there was no consistent pattern, some results showed that the web surveys
‘obtained higher reports of sensitive behaviours’.
Revilla and Ochoa (2017) asked participants who belong to an opt-in (nonprobability) panel in Mexico to tell them the ideal and the maximum length of a
survey they would want to answer: the median for ideal length was 10 minutes; the
334
M13 The Practice of Market Research 31362.indd 334
30/09/2021 18:28
Online data collection
maximum 20 minutes. They note, however, that these lengths were ‘significantly
linked to the fact that [they] liked answering the survey and . . . trusted that their
data are treated in an anonymous way’. Revilla and Höhne (2020) examined the issue
with participants on two online panels in Germany, a probability-based panel and a
non-probability panel. In this instance they found the ideal length was between 10
and 15 minutes and the maximum between 20 and 28 minutes.
The questionnaire in a web survey must be simple and straightforward – easy to
follow and easy to fill in. Brosnan et al. (2019) used a qualitative study to investigate
what influences online access panel members to take part in a survey. They found
ten things: incentive payments; speed of completion; ease of completion; topic interest; software functionality; benefit to others; topic knowledge; impact; relationship
with brand/organisation; and participant’s opinions being valued. In a stated choice
experiment they discovered that incentive payments are the main driver with short
completion time in second place, then interest in the topic, followed by ease of completion and ease navigating with the software.
It can be hard to know what device a participant will use to view a survey: it might
be a PC, a laptop, a tablet or a smartphone. (You can find out after the event by looking at the ‘paradata’ from the survey. Paradata are data collected about data collection. They include data about what type of device the participant used, what browser,
how long the interview took, how long the participant took on each question, how
they moved through the survey, whether they went back to questions and so on. They
are used to review the data collection process and to understand how it might be
improved.) It is the participant who decides and therefore it is up to the researcher
to ensure that the questionnaire is suitable for viewing on a range of devices. Many
surveys are now designed to be ‘device agnostic’. This is important in terms of quality
of the research output. Survey completion rates vary across devices and, as Brosnan et
al. (2017) found in analysing several datasets of online access panel members, use of
a particular device is associated with socio-demographic characteristics. As a result,
Brosnan et al. advise that you should not restrict participants to a specific device:
it ‘may compromise the quality of the survey completion experience, increase nonresponse error’ and it will have an adverse impact on representativeness. If a survey
is not suitable for the participant’s device then there is the risk that they will not take
part, or not complete the survey (known as a ‘break-off’). This will have implications
for fieldwork completion times and cost (including the cost of finding and presenting
the survey to other potential participants), and for representativeness, and so for the
validity and reliability of any research findings.
Mobile surveys
Surveys are designed for completion on mobile devices and online surveys are completed on mobile devices while not necessarily being designed for that mode of data
collection. As Wells (2015) notes, mobile surveys ‘offer many appealing features and
possibilities for market research’. He cites the opportunities it affords to capture
real-time, ‘in-the-moment’, point of experience activity (and so minimising recall
bias); to capture visual data (photos and videos); to capture GPS location data; and
the ability to scan barcodes. Poynter (2014) includes the use of geofencing, ‘triggering a research activity when somebody enters or leaves a specific location’, and the
ability to track such things as online access and use of mobile commerce and online
335
M13 The Practice of Market Research 31362.indd 335
30/09/2021 18:28
Chapter 13
Methods of data collection
entertainment. Mavletova and Couper (2016) add to this the ability to capture movement and biometric data such as heart rate and blood pressure. Industry Insight 13.5
gives an example of how researchers used smartphone data collection to understand
television viewing choices.
Industry Insight 13.5
How do you pick what to watch?
Introduction
How, why and when do viewers choose what
to watch? UK broadcaster Channel 4 Television
invests millions every year in marketing to ensure
that it attracts as many viewers as possible to its
award-winning content. How does it optimise
that investment? To address this issue it needed to
know how and why viewers pick one programme
over another and what Channel 4 can do to nudge
them away from the competition and towards
Channel 4. It worked with research agency Craft
to find this out.
The research
We took the view that those viewing choices
are longitudinal, non-linear journeys subject to
multiple influences. Previous research had looked
at how decisions are made on the sofa once
the television is on. We wanted to understand
the subconscious factors that nudge people to
watch TV and other video content. Throughout
the project we worked with a behavioural
psychologist to understand choice and the mood
and contextual factors that impact on it.
First, we conducted a two-week smartphone
survey with 100 participants aged 16–44 years.
The sample consisted of those with ‘future-facing
viewing behaviours’ and those who were more ‘traditional’. We designed two ‘micro-questionnaire’
diaries. In the first one participants recorded every
encounter with audiovisual (AV) content, be that
in conversation online and offline and exposure
to any kind of press or marketing material. We
captured title, genre and measured likelihood to
watch. To understand ‘point of sale’, we used the
second questionnaire diary to log every viewing
occasion, capturing what was being watched,
how and where participants had encountered the
content before (if at all), and their ‘need state’.
We adapted an academically-recognised questionnaire for determining need states into a one-screen
graphical interface. Using it we captured levels of
mental, ego and physical depletion. Analysing
these measures we could ascertain the need-state
in which a viewing occasion took place and we
could map those outcomes to other factors such
as genre and time of viewing.
The smartphone data collection yielded 5,000
entries. It provided a rich and nuanced dataset in
which we could track journeys to content over
time and understand when there was no journey
at all and when an impulsive decision had been
made. By tying journeys to occasions we were able
to ascertain the influence of marketing channels,
word-of-mouth (WOM) and other factors. The
insights from this phase were used to shape the
sample and the focus of the qualitative phase.
In the qualitative phase eight participants
wore ‘EyeCams’, lightweight glasses that capture
HD video and audio from the wearer’s perspective, for one week. We briefed them to film every
video content decision they made, giving their
in-the-moment perspective of how, when and
why they chose what to watch. Vocalising their
internal monologue helped us to understand
their conscious thought processes. Timecoding
and analysing the footage allowed us to observe
subconscious behaviours and patterns. The passive and in-the-moment data collection avoids
problems with recall and post-rationalisation.
Once we had analysed the data we visited the
eight participants and discussed with them
their behaviour and together analysed their
footage and filmed observation. This allowed
participants to contextualise and explain their
decisions.
336
M13 The Practice of Market Research 31362.indd 336
30/09/2021 18:28
Online data collection
Use of the findings
The findings provided Channel 4’s marketing department with insight that has changed the way it promotes its content. The insight has also been adopted
by the wider business, influencing commissioning
and scheduling decisions, informing product development and steering business strategy.
Source: Adapted from Craft and Channel 4 Television ‘Remote
access – understanding Gogglebox Britain’, MRS Awards 2015.
Used with permission.
In a review of the literature on mobile surveys, Wells (2015) reports the following:
response rates are low with a mobile device;
survey responses from a mobile device are received sooner after a survey is fielded
than those completed on a computer;
● sending invitations via an SMS increases the share of participants completing on
a mobile device (where it is ethically and legally possible to do this);
● responses are received in a shorter time with an SMS invitation compared to an
email invitation.
●
●
Use of a mobile device, in particular a smartphone, to complete surveys appears
to be related to age. Bosch et al. (2019) examined participation in surveys using a
dataset that comprised over 1.5 million members of an opt-in online panel in eight
countries in Europe, North America and Latin America. They found that while the
Millennials cohort (that is, those born between 1982 and 2003) had lower participation rates than older age groups, they represented a significantly greater proportion
of surveys answered using smartphones.
Online panels
An online panel is a pool of people recruited to take part in research. There are two
main types: those recruited with a specific research purpose or task in mind – for
example, a panel made up of a client’s customers, and used to monitor satisfaction;
and those recruited to work as a source of participants for a range of research projects. The first type tends to be a panel owned by the client and used only by the client. The second type tends to be owned by a research agency or fieldwork supplier,
is sometimes referred to as an online access panel, and is available to any third party
who wants to use it. Panel members can be recruited using traditional methods or
online methods (for example, via sign-up pages advertised on banners or pop-ups or
by email registration). Those recruited online may be contacted for a follow-up to
verify that they are who they say they are.
A sample drawn from an online panel is a popular way of doing ad hoc research –
the panel is the primary source, in effect it is the population of interest and you recruit
your sample from within the population of panel members. If you decide to use an
online panel to generate a sample for your research you need to be clear about what
sort of a sample you are getting. Online access panels do not deliver samples that
are representative of the general population (AAPOR, 2010 and Baker et al., 2013).
First of all, they do not cover the entire population but only those who have internet
access, and only a proportion of them. In 2011, Baker estimated this coverage error
for the UK at 99 per cent: a population of around 50 million adults and internet
337
M13 The Practice of Market Research 31362.indd 337
30/09/2021 18:28
Chapter 13
Methods of data collection
access at around 75 per cent, the UK online population is about 37 million thus 25
per cent of UK adults are excluded from online panels; if a UK panel has around
400,000 members – about 1 per cent of the UK online population and 0.8 per cent
of the total population then the panel’s coverage error is more than 99 per cent.
While things have undoubtedly changed since then, internet coverage is not universal.
Research published on the ONS website at time of writing shows that in January to
February 2020, 96 per cent of households in Great Britain had internet access; the
figure is 80 per cent in households where one adult is aged 65 years and over.
People with internet access are different from those without it on many characteristics (demographic, geographic, psychographic, attitudinal, behavioural) and those
who agree to sign up to a panel and take part in research are different again. Also,
the people who sign up to one panel may be different on a whole range of characteristics from people who sign up to another panel. Next, there is the issue of response
rate. For many panels the response rate is 10 per cent or less. This further erodes the
representativeness of a panel sample. For both these reasons, if you use a panel sample you cannot use your data to make inferences about the population as a whole.
As Baker (2014) notes, we often don’t know what the biases in the data are. If you
are going to address them, you need to be able to identify them. It is not enough to
use just the demographic variables in selecting a quota sample or adjusting it. There
will be other variables, ‘auxiliary variables’ or ‘covariates’ (Baker, 2014) that are
having an effect – the variables that are ‘correlated with the behaviour or attitude
that the survey was designed to measure’. A key question then is, is the distribution
of those variables in your sample the same as their distribution in the population of
interest? If it is not the same, the chance of your survey findings being accurate is
reduced. So, you need to know what those variables are. As Baker puts it, you need
to find ‘the secret sauce’ and, once you have it, it can be ‘added to the mix’ and used
to adjust the algorithm or model that chooses the sample. (Barber et al. (2013) set
out how they determined the covariates in a media tracking study in order to help
in the decision about whether to switch from a face-to-face to an online approach.)
Many online access panels are actively managed to ensure and maintain a standard
of quality. This means making sure that a participant does not take the same survey
more than once, that the number of surveys per panel member within a specified
period is kept to a minimum, that participation in surveys is maximised, that the
quality of the participant’s survey responses is reviewed. Issues around retaining
participants on the panel, minimising attrition or drop-out, are similar to those in
traditional longitudinal panels. To this end it is essential to make sure that there are
clear lines of communication between panel members and the panel administrator,
including telephone contacts as well as email contacts.
In 2012, Revilla and Saris conducted research comparing the quality of findings
from a probability online panel sample to those from a face-to-face survey and found
the quality to be similar. The panel sample they looked at was one that was set up
‘by and for academic research . . . in a country with high internet coverage’. In 2015,
they conducted another study, this time assessing the quality of answers from a nonprobability panel sample in a country with a lower internet coverage compared to
those from a face-to-face survey. The panel they chose was a Netquest panel involved
in surveys related to marketing. The panel provider has ‘arrangements’ with websites
on which it runs user satisfaction surveys. Those who have taken part in a survey
are invited to join the panel if they match the panel’s targets at that time. The survey
participant can decide whether or not to register with the panel. There are incentives
338
M13 The Practice of Market Research 31362.indd 338
30/09/2021 18:28
Mixing or switching modes of data collection
for registering and incentives for each survey completed. The findings from this
panel were compared to findings from the European Social Survey (ESS). The ESS
uses a probability sample and is conducted face to face in the participant’s home. It
is widely regarded as producing high quality data. Revilla and Saris (2015) report
that the quality from the non-probability online panel was similar to that from the
probability face-to-face sample, which, they note, is ‘reassuring’.
It is worth remembering that not all panels are the same. The Advertising Research
Foundation (ARF) examined data from 17 online panels and found a great deal of
variability which data cleaning and weighting did not eliminate (Walker and Petit,
2009). The ESOMAR Guideline on Online Sample Quality (2015) refers to some
online samples as convenience samples ‘lacking the necessary statistical properties
assumed to be needed to accurately represent the intended target population’.
You should be cautious in deciding to use a panel and in the panel you choose:
remember there is coverage error; there is self-selection bias; there are concerns
about ‘professional’ participants, those who sign up to panels and take as many
surveys as possible; and there are issues with panel recruitment, management and
maintenance. Make sure you do due diligence on all aspects of any panel you plan
to use so that you can be clear and transparent with the client about how the sample
was achieved, the implications the sample has for the quality of the findings and
the inferential risk, the ‘chance of getting an inaccurate or even completely wrong
answer’ (Baker, 2014).
Mixing or switching modes of data collection
Using more than one method of data collection in a survey is called mixed-mode
research. You might use different modes within one interview with the same participant (interviewer-administered and self-completion) or you might offer participants
a choice of mode if you think this will help you reach your target sample and achieve
the necessary response rate. There are, however, some issues you need to consider
if you plan to use a mixed-mode approach. As Eva and Jowell (2009) note, these
include mode effects and so an increase in survey error. Mode effects arise from
the fact that different modes have different strengths and weaknesses – differences
in their ability to cover the population of interest, different selection biases and
different types of measurement error. This makes it difficult to compare data collected using one mode with that collected using another. The differences you see
may not be real differences but rather differences arising from the method used to
collect the data.
If, for some reason – squeezed budgets, tighter timescales, poor coverage of the
population of interest, falling response rates – you need to switch your survey from
one mode to another, or you need to move from a single-mode to a mixed-mode
approach, then you need to be aware that this will have an effect on your data and, as
Jowell and Eva note in Industry Insight 13.6, you will need to assess ‘the gains and the
losses in terms of cost, data quality, equivalence, response rates and representativeness.’ This Insight deals with a multi-country survey. It sets out why a mixed-mode
approach was under consideration and outlines the research conducted to assess the
mixed-mode approach.
339
M13 The Practice of Market Research 31362.indd 339
30/09/2021 18:28
Chapter 13
Methods of data collection
Industry Insight 13.6
Mixing modes across Europe
Introduction
The European Social Survey (ESS) is an attitude
survey time series carried out in over 30 countries across Europe. The survey began in 2001
and fieldwork is carried out from September to
December every two years. The questionnaire consists of two parts: a core section repeated every
round and two rotating modules repeated less frequently. The core modules include subjects such
as media consumption, political and social trust,
and religious identification. The rotating modules
each cover a substantive topic in more detail (e.g.
well-being, work–life balance.)
To help ensure equivalence of outputs, the
hour-long survey interview is conducted face to
face in all countries. This mode was initially chosen not only because it tends to get the highest
response rates and is the only mode that offers
complete coverage in all countries but also
because the questionnaire is particularly well
suited to face-to-face interviews. In particular, it
involves a long interview, many showcards and
some complex routing.
However, it became clear that using face-toface interviewing as the sole mode of data collection might need to be reconsidered in view of
its rising cost and diminishing response rates. In
addition, different countries have different experiences and expertise as well as different penetration of modes, which means they may be better
equipped to use a different mode. Research was
needed to shed light on whether a mixed-mode
design would be feasible for ESS data collection
and, if so, what would be the gains and losses in
terms of cost, data quality, equivalence, response
rates and representativeness.
The research
The first two phases of this work investigated
mode effects on measurement and their likely
causes. The overall findings suggested that data
from the telephone interviews differed most from
data derived from other modes, in particular
displaying greater social desirability bias. On the
other hand, while abstract and sensitive questions
in all modes generated the largest mode differences, it was heartening that the overall proportion of items showing mode effects was small.
Phase III of the work investigated existing survey practices across Europe in order to collect
empirical evidence on the existing demand and/
or capacity for different modes of data collection.
The impetus to abandon face-to-face interviewing
as the sole mode of data collection arises from its
high or very high relative cost, which applies in
almost all countries. Practitioners in some countries also believe that mixed-mode data collection
is likely to increase response rates. On the other
hand, we found no other single mode (with the
possible exception of telephone interviewing in
one or two countries) that had sufficient coverage to replace face-to-face interviewing. Most
agencies we consulted had at least some practical experience of having used mixed-mode data
collection.
Our findings suggested that telephone interviewing was the most popular mode to replace
or complement face-to-face interviewing. So we
carried out an experiment to investigate the feasibility of conducting a full hour-long ESS interview
on the telephone, and to test possible alternatives.
We tested three different approaches via a split
sample: one-third of respondents were asked to
complete the full one-hour questionnaire, another
third were asked a 45-minute version, and for the
final third the full questionnaire was split into two
parts. As expected, a higher response rate was
achieved for the shorter questionnaire, though the
difference was not as appreciable as we had anticipated. But we also found some evidence that data
quality deteriorated somewhat with questionnaire
length.
Phase IV was a full mixed-mode experiment
testing a web-based self-administered questionnaire and a telephone interview alongside the faceto-face interview. A ‘concurrent model’, where
340
M13 The Practice of Market Research 31362.indd 340
30/09/2021 18:28
Mixing or switching modes of data collection
respondents were given a choice to complete the
survey by telephone, web, or face-to-face, and
a ‘sequential design’, where the different modes
were offered to respondents in the order of their
costs for the survey agency, were tested.
The importance of assessing the evidence
It is important to assess all the evidence before
introducing a mixed-mode design into a time series
based until now on a uni-mode design. Such a
decision is even more difficult for a cross-national
survey, not just because different countries have different methodological habits and preferences but
also because the effects of mixing modes may differ
across countries. So in addition to the difficulties
of disentangling differences over time and differences by country, there will be differences in data
collection mode. However, although arguments
against change are always persuasive, they have to
be weighed against the empirical evidence that survey costs are increasing fast while response rates are
falling. We need to consider the worrying implications of resisting changes to forms of data collection
that may help to mitigate some of these problems.
Source: Adapted from Eva, G. and Jowell, R. (2009) ‘Prospects
for mixed-mode data collection on cross-national surveys’,
International Journal of Market Research, 51, 2, pp. 267–9.
Used with permission.
Box 13.1
How to reduce survey error
No matter which approach to data collection you choose, as we’ve seen there will be
advantages and limitations. At every stage in the process there is also an opportunity
for error. Error in surveys is often categorised as sampling error and non-sampling error.
Together they are referred to as total survey error. Non-sampling error is a large part of
total survey error. It includes error that arises from the design and the administration of
the questionnaire and from everyone involved in that process. For example, a question
may be badly worded, or it may be leading; the interviewer may not follow the prompt
instructions; the participant may not answer, or may under- or over-report, or the selfcompletion questionnaire may be completed by the wrong participant; the participant’s
response may be recorded wrongly, or only partially, or not at all.
The aim in any project is to reduce error. To help reduce error, the key things to pay
attention to are as follows:
Suitability of the research design (including the sample) for gathering the evidence
needed
● Quality and coverage of the sample in relation to the target population
● Training, briefing and monitoring of interviewers
● Quality of the fieldwork
● Quality of the interaction with participants
● Response rate
● Use of effective measures to improve response rate
● Non-response bias
● Measures to understand and reduce non-response bias
● Use of appropriate incentives
● Quality of the questionnaire design
●
341
M13 The Practice of Market Research 31362.indd 341
30/09/2021 18:28
Chapter 13
Methods of data collection
●
●
Cognitive testing and piloting of the questionnaire
Adherence to ethical, legal and regulatory requirements.
In reporting on a project, you must be transparent about all aspects of the research
design, the sample design, survey implementation including fieldwork, and any work
done on the sample or the data after fieldwork.
Observational methods of data collection
In this section we look at two methods of collecting data using observation: eye
tracking; and mystery shopping.
Eye tracking
Observational exercises are conducted using eye-tracking devices, often in retail settings where researchers are interested in understanding influences on purchasing
decisions close to the point of sale. It works like this. The research participant puts on
a pair of eye-tracking glasses (ETG) in which there is a camera/recorder. The device
records and stores what the participant looks at within the research setting – which
items on a shelf in a shop, which point-of-sale marketing material, for example.
Once the eye-tracking session is completed, the recording from the ETG can be
downloaded and analysed. In many cases a researcher will interview the participant
to find out, among other things, what they thought of the things they looked at
and how they came to a purchase decision. We saw an example of this approach in
Industry Insight 13.2 above. Industry Insight 13.7 gives another example of an eyetracking study in a difficult-to-research environment.
Industry Insight 13.7
What are you looking at?
Introduction
London Underground (LU) is a challenging place
in which to do research. Exterion Media was preparing a bid for an ‘out-of-home’ advertising contract on the Underground (including on trains and
platforms). It wanted to understand LU as an ad
medium, how and why it works, in order to guide
better creative and strategic use of it and for use in
selling advertising in the Underground.
The research
To get this information it commissioned a project
with COG Research and Bournemouth University
that involved following passengers wearing eyetracking glasses (ETG) and skin conductance
recorders (SCR) as well as desk research and other
qualitative and quantitative work.
For the eye-tracking and SCR element of the
project 100 participants were chosen to cover 100
typical tube journeys, some of which included car,
train or bus travel. The study was conducted over
a two-week period and covered all parts of the
day, and all tube lines and areas of London.
For each participant, members of the research
team reviewed the video footage from the ETG
and data from the SCRs which measures implicit
342
M13 The Practice of Market Research 31362.indd 342
30/09/2021 18:28
Observational methods of data collection
reaction times (IRT) to stimulus such as advertising. Combining the ETG and the SCR data they
were able to identify where people were, what
they could see, what they could hear, what they
were doing or looking at and how long and hard
their brains were working. They provided analysis
and highlights to a team of qualitative researchers who conducted follow-up in-depth interviews
with participants no more than ten days after their
journey. The aim of the interview was to get a
clear picture of the person’s journey, experiences,
and ad exposure and engagement.
Use of the findings
The researchers were able to map five zones of
engagement based on context, focus, brain activity
and distraction. This gave insight into the strengths
of different ad formats on the Underground and
provided data for comparison with other work
on TV, Video on Demand and digital media. It
also enabled Exterion Media to show the value of
advertising on the Underground.
Source: Adapted from Exterion Media and COG Research
‘Immerse and engage on London Underground’, Winner, MRS
Awards 2017. Used with permission.
Mystery shopping research
The aim of mystery customer or mystery shopping research is to collect data to give
feedback to an organisation on the quality of its services. The MRS Guideline on
Conducting Mystery Shopping (2020) defines it as ‘the use of individuals trained to
experience and measure any customer service process, by acting as potential customers and in some way reporting . . . on their experiences in a detailed and objective
way’. The information collected might include, for example, length of time in the
queue; number of service points or tills open out of the total available; details of the
greeting and exchange with the member of staff; handling of questions; information
or advice offered and so on. The Guideline notes that the mystery shopping interaction is not a traditional research interview. This means that if personal data are
to be collected, participants must be informed in advance of the mystery shopping
exercise. Industry Insight 13.8 gives an example of mystery shopping in the form of
a longitudinal mystery traveller study.
Industry Insight 13.8
Mystery in accessible London
Introduction
The research
In late 2012 Transport for London (TfL) published its commitment to making it easier for those
with a disability to travel in the capital. To ensure
that it was meeting this commitment it developed a large-scale longitudinal study called the
Accessibility Mystery Traveller Survey (ATMS)
with research agency GfK.
The survey is led by a team of 45 assessors, people
living with a disability including those with sight
impairments and those who use a wheelchair. The
assessors and those who accompany them capture
data on travel experiences across the TfL network
which includes buses, London Underground and
Overground, Tramlink, taxis and the Dial-a-Ride
343
M13 The Practice of Market Research 31362.indd 343
30/09/2021 18:28
Chapter 13
Methods of data collection
service. The assessors receive one-to-one training
with Field Quality Mentors (FQMs) who themselves are trained to know when to instruct and
when to encourage and guide. Each quarter the
assessors conduct 1,300 assessments. Data capture
is adapted to each person’s needs and those who
accompany the assessor either help with guidance
or record the journey. The data the assessors collect
and their insights are shared with stakeholders to
enhance understanding, and they contribute directly
to decisions made and actions taken. To ensure that
the research remains relevant and accessible, assessors and FQMs are involved in the research design
through focus groups and pilot exercises and are
consulted on any potential changes in the process.
Source: Adapted from GfK and TfL ‘Accessible London – The
Accessibility Mystery Traveller Survey’, Winner, MRS Awards
2015. Used with permission.
The MRS Code of Conduct contains rules relevant to mystery shopping. Its
Guideline document (2020) gives guidance for those conducting mystery shopping
exercises with a competitor organisation. This can be summarised as follows:
Keep the length of time spent as short as possible and do not waste the competitor’s resources
● Make a purchase that reflects the type of business
● Consider the frequency of assessment of any one outlet and do not target individuals so that normal business is not disrupted or slowed
● Report data at the aggregated level and not at the level of the individual contact
● Keep the transaction required by the competitor to a minimum and ensure that it
reflects a normal transaction.
●
In Box 13.2, Morrison, Colman and Preston (1997) set out ways of improving the
quality of mystery shopping research. This is useful as in not all mystery shopping
exercises is it possible to use devices such as cameras or eye-tracking glasses.
Box 13.2
How to improve validity and reliability in mystery customer
research
The demands of memory
There are potential threats to the accuracy, validity and reliability of mystery customer
surveys. Some of these arise from the memory demands placed on the assessors (the
mystery shoppers or mystery customers), who normally record the attainment or nonattainment of various standards that they have observed some time after making the
relevant observations.
Suggestions for minimising errors of memory
Omissions and distortions of memory can arise at all three stages of the memory
process: encoding, storage, and retrieval. In the light of this, a review of findings from
cognitive psychology suggests a number of steps that could be taken in designing and
carrying out mystery customer surveys to minimise errors arising from memory failures:
344
M13 The Practice of Market Research 31362.indd 344
30/09/2021 18:28
Observational methods of data collection
To reduce the memory burden on assessors, it might be possible to restrict their task
to checking the attainment of personal and interactive standards of service delivery –
for example, ‘Was I served within two minutes?’, ‘Did the bank teller smile?’ rather
than checking whether the impersonal and relatively fixed, ‘physical’ standards were
attained – for example, ‘Were the toilets in working order?’, ‘Was the company logo
prominently displayed?’ This would reduce the memory demands, thereby helping
to minimise errors arising from memory.
● It is essential that recording of observations should take place during or immediately
after the visit to reduce the problems of decay and reconstructive memory distortion.
Recording should probably be done in writing and the questions on the assessment
forms should be carefully designed to give maximal retrieval cues and above all to
minimise the use of suggestive or leading questions.
● It may be possible to reduce memory problems by using event recorders. The assessors’ memory task would then be restricted to remembering what standards to
check and in what order to check them.
● Assessors should be encouraged to make their visits at a time of day when they
are alert and not tired and when the ambient lighting gives them the best chance of
seeing what needs to be seen.
● Video recordings of a few typical service encounters, including common problems
and difficult distinctions, may be useful for training future mystery customers and
establishing common standards.
● The training of assessors should include a suggestion that, if they have difficulty
remembering certain details . . . , they should try shutting their eyes and vividly
imagining themselves back in the place where their observations were made (visualisation). In addition, assessors should attempt to retain a neutral emotional state
throughout the assessment visit and when recording the results.
● Assessors should be warned about the problem of social pressure and the tendency
to prefer giving favourable reports rather than unfavourable ones, especially if the
people working in the target establishments seem pleasant or easy to empathise
with. They should also be encouraged to assess each establishment objectively on
its own merits rather than consciously or unthinkingly making direct comparisons
between different establishments.
● The standards that form the basis of mystery customer surveys should be as objective as possible. For example, ‘Was I served within two minutes?’ is completely
objective, but ‘Was the bar tidy?’ or ‘Was the shop tidy?’ requires a subjective
judgement, which is likely to undermine the reliability and validity of a survey. The
client should be asked wherever possible to specify exactly what they mean by, for
example, ‘tidy’, ‘clean’, and so on, to enable objective standards to be defined.
● Buyers and users of mystery customer research should establish a ‘best practice’ protocol for conducting mystery customer surveys. Changes in procedure
can have unpredictable and unknown effects on the validity and reliability of the
findings.
●
Source: Adapted from Morrison, L., Colman, A. and Preston, C. (1997) ‘Mystery customer research: cognitive
processes affecting accuracy’, International Journal of Market Research, 39, 2, pp. 349–61. Used with
permission.
345
M13 The Practice of Market Research 31362.indd 345
30/09/2021 18:28
Chapter 13
Methods of data collection
End note
Whichever method or approach you choose, it will have advantages and limitations.
To determine which is most suitable for the problem you are researching, you need,
first of all, to be clear about the problem and the sort of evidence you need to address
that problem. Once you know that, you can evaluate the methods available and
decide which will best deliver the evidence. You must be able to justify your choice
to the client or end user of the research.
Chapter summary
●
●
●
●
●
●
Quantitative data can be collected via interviewing and observation. The method
chosen depends on its suitability for gathering the sort of evidence required
and achieving the research objectives; the topic or issues under investigation;
its ability to reach the sample and achieve the right numbers; and the time and
budget available.
The interviewer has a vital role to play in collecting good-quality data. Interviewing
is a skilled task requiring a high level of interpersonal skill and a sound
understanding of data collection.
Face-to-face data collection has a number of advantages over other methods. It
enables the interviewer to build rapport with the participant, which has positive
effects on data quality; and it allows for a degree of flexibility in the interviewing
process. It is, however, relatively expensive and time consuming.
Telephone interviewing (fixed line and mobile) has a number of advantages over
face-to-face methods. Geographically dispersed and other samples that are hard
to reach can be obtained more easily; it is possible to use a random sampling
approach, thus reducing sampling error; greater quality control is possible with
interviewers being monitored ‘live’; and it is faster and more cost effective. There
are some disadvantages related to sampling and representativeness, including
the increased incidence of mobile rather than fixed line phones.
Self-completion surveys – postal, online – are effective if the topic is relevant and
of interest to the target population and the method is a suitable way of reaching
the target and achieving a response. They can be cost effective as there are no
interviewers to pay, and are suitable for reaching widely dispersed and otherwise
hard to reach samples but they can suffer from poor response rates and thus
problems with representativeness; and there is lack of control over data capture.
Response rates may be increased in a number of ways.
Using an online access panel to obtain a sample for an online survey can be
time and cost effective. There are, however, methodological issues in relation to
coverage of the population, self-selection bias, ‘professional’ participants and
data quality.
346
M13 The Practice of Market Research 31362.indd 346
30/09/2021 18:28
References
●
●
Using more than one method of data collection in a survey is called mixed-mode
research. Mode effects arise from the fact that different modes have different
strengths and weaknesses – differences in their ability to cover the population of
interest, different selection biases and different types of measurement error. If you
need to switch a survey from one mode to another, or you need to move from a
single mode to a mixed-mode approach, then you need to be aware that this will
have an effect on your data.
Observational techniques can be used to collect quantitative data, in person or
electronically. Examples include mystery customer or mystery shopping research,
fixed cameras, use of eye-tracking devices, glasses in which there is a camera/
recorder which records and stores what the participant looks at within the
research setting.
Exercise
1 Review each of the Industry Insights 13.1–13.8. For each one, note the method/s
of data collection and list the advantages and limitations for the use of that
method in that context.
References
AAPOR (2010) ‘Report on online panels’, Public Opinion Quarterly, 74, pp. 711–81.
Baker, R. (2014) ‘We can do better’, International Journal of Market Research, 56, 1,
pp. 11–13.
Baker, R., Brick, J., Bates, N., Battaglia, M., Couper, M., Dever, J., Gile, K. and Tourangeau, R.
(2013) ‘Summary report of the AAPOR Task Force on non-probability sampling’, Journal
of Survey Statistics and Methodology, 1, 2, pp. 90–143.
Baker, R. (2011) Conference notes, ‘Uses and misuses of online panels’, International Journal
of Market Research, 53, 2, pp. 275–8.
Barber, T., Chilvers, D. and Kaul, S. (2013) ‘Moving an established survey online – or not?’
International Journal of Market Research, 55, 2, pp. 187–99.
Bosch, O., Revilla, M. and Paura, E. (2019) ‘Do Millennials differ in terms of survey participation?’ International Journal of Market Research, 61, 4, pp. 359–365.
Brennan, M., Hoek, J. and Astridge, C. (1991) ‘The effects of monetary incentives on the
response rate and cost-effectiveness of a mail survey’, Journal of the Market Research
Society, 33, 3, pp. 229–41.
Brosnan, K., Kemperman, A. and Dolnicar, S. (2019) ‘Maximizing participation from online
survey panel members’, International Journal of Market Research, 61, 1, pp. 1–20.
Brosnan, K., Grün, B. and Dolnicar, S. (2017) ‘PC, phone or tablet? Use, preference and completion rates for web surveys’, International Journal of Market Research, 59, 1, pp. 35–55.
Comley, P. (2003) ‘Innovation in online research – who needs online panels?’, Proceedings of
the Market Research Society Conference, London: MRS.
Craft and Channel 4 Television (2015) ‘Remote access – understanding Gogglebox Britain’,
MRS Awards.
347
M13 The Practice of Market Research 31362.indd 347
30/09/2021 18:28
Chapter 13
Methods of data collection
Dubreuil, C. and Murray, M. (2012) ‘Cash for questions’, Research, 555, August, pp. 19–21.
Elkasabi, M., Suzer-Gurtekin, Z. T., Lepkowski, J., Kim, U., Curtin, R. and McBee, R.
(2014) ‘A comparison of ABS mail and RDD surveys for measuring consumer attitudes’,
International Journal of Market Research, 56, 6, pp. 737–56.
Erens, R. (2014) ‘Using non-probability web surveys to measure sexual behaviours and attitudes in the British general population: a comparison with a probability sample interview
survey’, International Journal of Market Research, 57, 2, pp. 300–05.
ESOMAR (2015) Guideline on Online Sample Quality, Amsterdam: ESOMAR.
Eva, G. and Jowell, R. (2009) ‘Prospects for mixed-mode data collection on cross-national
surveys’, International Journal of Market Research, 51, 2, pp. 267–9.
Exterion Media and COG Research (2017) ‘Immerse and engage on London Underground’,
MRS Awards.
GfK and TfL (2015) ‘Accessible London – The Accessibility Mystery Traveller Survey’, MRS
Awards.
Haikel-Elsabeh, M., Zhao, Z., Ivens, B. and Brem, A. (2019) ‘When is brand content shared
on Facebook? A field study on online Word-of-Mouth’, International Journal of Market
Research, 61, 3, pp. 287–301.
Krosnick, J., Presser, S., Husbands Fealing, K. and Ruggles, S. (2015) The Future of Survey
Research: Challenges and Opportunities, National Science Foundation.
Lumen Research and Tesco (2019) ‘A three-part strategy to optimise Tesco’s trade driving
print’, MRS Awards.
Mavletova, A. and Couper, M. (2016) ‘Device use in web surveys: The effect of differential
incentives’, International Journal of Market Research, 58, 4, pp. 523–44.
McQuater, K. (2020) ‘Decoding cravings’, Impact, 29, pp. 16–17.
Morrison, L., Colman, A. and Preston, C. (1997) ‘Mystery customer research: cognitive processes affecting accuracy’, Journal of the Market Research Society, 39, 2, pp. 349–61.
Mouncey, P. (2014) ‘Editorial’, International Journal of Market Research, 56, 6, pp. 695–98.
MRS (2020d) Guideline for Conducting Mystery Shopping, London: MRS.
MRS (2019) Responsibilities of Interviewers, London: MRS.
ONS (2021) ‘Internet Access – Households and Individuals, Great Britain: 2020’ https://www.
ons.gov.uk/peoplepopulationandcommunity/householdcharacteristics/homeinternetandsocialmediausage/bulletins/internetaccesshouseholdsandindividuals/2020 (Accessed 2 March
2021).
Orton, S. and Samuels, J. (1988, 1997) ‘What we have learned from researching AIDS’, Journal
of the Market Research Society, 39, 1, pp. 175–200.
Poynter, R. (2014) ‘Mobile market research, 2014’, International Journal of Market Research,
56, 6, pp. 705–7.
Poynter, R. and Kaylor, K. (2012) ‘Communities in 2017: A prediction of where communities
will be in five years’ time’, Proceedings of the MRS Conference, London: MRS.
Revilla, M. and Höhne, J. (2020) ‘How long do respondents think online surveys should
be? New evidence from two online panels in Germany’, International Journal of Market
Research, 62, 5, pp. 538–45.
Revilla, M. and Ochoa, C. (2017) ‘Ideal and maximum length for a web survey’, International
Journal of Market Research, 57, 5, pp. 557–65.
348
M13 The Practice of Market Research 31362.indd 348
30/09/2021 18:28
Recommended reading
Revilla, M. and Saris, W. (2015) ‘Can a non-probabilistic online panel achieve question quality
similar to that of the European Social Survey?’ International Journal of Market Research,
57, 3, pp. 395–412.
Savanta and RBS (2019) ‘Business banking switch – business unusual for RBS’, MRS Awards.
Trinity McQueen and Weetabix (2017) ‘The shopper moment of truth’, MRS Awards.
Van der Heijden, P. (2017) ‘The practicalities of SMS research’, International Journal of
Market Research, 59, 2, pp. 157–72.
Vicente, P. (2017) ‘Exploring fieldwork effects in a mobile CATI survey’, International Journal
of Market Research, 59, 1, pp. 57–76.
Vicente, P. (2014) ‘The best time to call in a mobile phone survey’, International Journal of
Market Research, 57, 4, pp. 555–70.
Vicente, P., Reis, E. and Santos, M. (2009) ‘Using mobile phones for survey research: a comparison with fixed phones’, International Journal of Market Research, 51, 5, pp. 613–33.
Walker, R. and Petitt, R. (2009) ARF Foundations of Quality: Results Preview, New York:
Advertising Research Foundation.
Wells, T. (2015) ‘What market researchers should know about mobile surveys’, International
Journal of Market Research, 57, 4, pp. 521–32.
Willems, A. and De Ruyck, T. (2013) ‘Collaboration with co-researchers in communities’,
International Journal of Market Research, 55, 4, pp. 587–9.
Yu, J. and Cooper, H. (1983) ‘A quantitative review of research design effects on response rates
to questionnaires’, Journal of the Market Research Society, 20, 1, pp. 36–44.
Recommended reading
Callegaro, M., Baker, R., Bethlehem, J., Goritz, A., Krosnick, J. and Lavrakas, P. (2014)
Online Panel Research: A Data Quality Perspective. London: Wiley and Sons.
Dillman, D., Smyth, J. and Christian, L. (2014) Internet, Phone, Mail, and Mixed-mode
Surveys: The Tailored Design Method. Hoboken: Wiley & Sons.
349
M13 The Practice of Market Research 31362.indd 349
30/09/2021 18:28
Chapter 14
Sampling
Introduction
In this chapter we examine the ideas behind sampling in quantitative research
and the issues involved in developing a sampling plan and choosing a sampling
technique. We look at sampling theory and what it tells us about samples and
the data derived from them. We look in detail at what’s involved in probability
and non-probability sampling methods. We also look at samples in online
research.
Topics covered
Sampling units and sampling elements
Developing a sampling plan
● Sampling theory
● Probability or random sampling methods
● Semi-random sampling
● Non-probability sampling methods
● Samples in online research.
●
●
Relationship to MRS Advanced Certificate Syllabus
The material covered in this chapter is relevant to Topic 2: Guiding Principles;
and Topic 4: Selecting an appropriate sample.
M14 The Practice of Market Research 31362.indd 350
27/09/2021 21:51
What you should get from this chapter
At the end of this chapter you should be able to:
demonstrate knowledge and understanding of sampling theory and practice;
● develop and implement an appropriate basic sampling plan;
● understand the implications of the sampling plan for the quality and
generalisability of research findings.
●
351
M14 The Practice of Market Research 31362.indd 351
27/09/2021 21:51
Chapter 14
Sampling
Sampling units and sampling elements
The elements of the sample – the people, the organisations – may be ‘contained’ in a
sampling unit. For example, imagine you are commissioned to gauge the attitudes of
the general public to a range of social issues. To achieve the sample you decide to use
a sample source (sampling frame) that provides you with details of households. You
select a sample of households from this sampling frame and from each household you
select an individual. In this case the household is a sampling unit and the individual
is the sample element. You may have decided, on the other hand, to select a sample
of individuals directly, and not from within households. In this case the individual is
both the sampling unit and the sample element.
Developing a sampling plan
Sampling is about selecting, without bias and with as much precision as resources
allow, the ‘items’ or elements from which or from whom you wish to collect data.
In market and social research projects these elements are usually people, households
or organisations, although they may be places, events or experiences. Drawing up
a sampling plan is one of the most important procedures in the research process. It
involves the following:
defining the target population;
choosing an appropriate sampling technique;
● deciding on the sample size;
● preparing sampling instructions.
●
●
The sample choices you make are an integral part of a project’s research design. They
are not independent of other aspects of the research project. In deciding on your sample and how you select it, you must take into account the research objectives (what it
is you want to find out and how you will use it will influence decisions about target
population, choice of technique and sample size); the nature of the target population
and how you identify them (the availability and/or selection of a sample frame or
source); how they can be reached (the method by which you will collect data from
them); and how much of your cost and time resources you can devote to it.
Defining the population
In research terminology, the term ‘population’ has a broader meaning than its common usage in reference to human populations of particular countries. In a research
context it refers to the ‘universe of enquiry’ or – put another way – to the people,
organisations, events or items that are relevant to the research problem. It is important
to define the population of interest or target population as precisely as possible. Any
flaws in the definition of the population will mean flaws in the sample drawn from it.
For example, if you are investigating the health and social welfare needs of
older people, then you might say that older people are the target population. But
what do you mean by ‘older people’? What is the definition of an ‘older person’?
352
M14 The Practice of Market Research 31362.indd 352
27/09/2021 21:51
Developing a sampling plan
In some contexts, it is anyone of 50 years of age and older; in others, it is people aged
over 65. What age limits do you impose? Should you impose an upper age limit or
not? Do you include only older people living independently or do you include those
living in sheltered or residential care accommodation or those in nursing homes or
hospitals? If you decide that it is only those living independently, how do you define
that? Should you include those living in the home of a relative or only those living
in their own home?
The way in which the population is defined depends on the issues the research
aims to address. If, for example, the study of the health and social welfare needs of
older people has been commissioned to help develop policy in relation to community
health initiatives, you may decide that those in residential care, nursing homes or
hospitals are not part of the relevant population. In defining the population, think
of the aims of the research.
Box 14.1
Example: criteria used in defining the population
Organisations and employees
Type of organisation – for example private sector (privately owned or stock market
listed), public sector or not for profit; those selling mainly to consumers or mainly to
other businesses or both; or those selling to more than one country or to one country
only.
● Geographic area – for example all organisations with a head office (or any office) in
a particular region or country.
● Market or industry sector – for example all organisations in the financial services
sector or those in the financial services sector selling to private individuals only.
● Size of organisation – for example in terms of annual turnover or number of
employees.
● Type of experience and/or time – for example all organisations involved in an Initial
Public Offering (IPO) on the stock market in the last financial year.
● Type of department or office within the organisation.
● Job title or role or responsibilities of an individual employee.
● Type of experience of an employee – for example all those receiving merit pay
awards or promotions in the last six months.
●
Households and people
Geographic area – for example all households within a particular region or country
or telephone area code.
● Demographic profile – for example age, sex, social class, presence of children.
● Geodemographic profile – those living within a particular geodemographic cluster
or type of residential neighbourhood.
● Time – for example all those visiting a pharmacy between 10 am and 1 pm on weekdays; all those who bought a new car in the last three months.
● Type of experience and/or time – for example women who gave birth in the last six
months in a private hospital; men who made a purchase in the last month from a
particular website; regular users of brand X.
●
353
M14 The Practice of Market Research 31362.indd 353
27/09/2021 21:51
Chapter 14
Sampling
Target population and survey population
Moser and Kalton (1971) make the distinction between the target population and the
survey population. The target population is the population from which the results are
required; the survey population is the population actually covered by the research.
As Moser and Kalton point out, in ideal circumstances the two should be the same
but, for practical reasons, they may not be. For example, people or organisations in
places that are remote or difficult to access using a face-to-face survey, such as those
on islands, may not be included in a survey population. In a survey of older people’s
health and social welfare needs it may be difficult to get permission to interview
those living in sheltered or residential accommodation. So, although you may have
identified them as part of the target population, they may be excluded for the sake
of expediency from the survey population.
If there is a difference between the target and the survey population, to avoid
misrepresentation of the research and its findings it is important that the difference
is made clear to all involved with the research and in any documents relating to the
research. For example, say that you set out to survey the population of households
in the United Kingdom. Your target population is all households in the United Kingdom. You decide, for various reasons, to conduct the survey using a managed online
panel. Your survey population will not then be the same as your target population
since not all households in the UK use the internet and so will not be represented
in your sample. A proportion of certain types of households will be excluded from
your survey population (e.g. older person households, less well-off households).
You must not then claim when presenting your findings that they are representative
of the population of households in the United Kingdom. You must make it clear that
there was a difference between your target population and your survey population.
This is something known as a coverage error – an error which arises when the sampling approach used does not deliver a representative sample of the target population.
We look at other kinds of error associated with sampling later in the chapter.
Census or sample?
Once the population is clearly defined you must decide whether to collect data from
every member or element of that population (usually referred to as a census) or from a
representative sub-set or sample of it. In most market and social research the population of interest is too large for a census to be practicable, either in terms of the time
it would involve or the cost. There are some circumstances, for example research
among members of a professional body or employees of an organisation, where the
population may be small enough, and accessible enough, for a census to be feasible.
In other cases it may be necessary or desirable to collect data from all elements of a
population. For example, in research to help with a decision about changes in working practices it may be important (and politic) to ensure that all employees’ attitudes
and opinions are surveyed.
There are other disadvantages in conducting a census besides those of time and
cost. The level of non-response may mean that the results are less representative
than might have been achieved with a well-designed sample of the same population. Non-response is where participants invited to take part in a survey do not
take part. It introduces bias to the sample. You do not know if those who did not
354
M14 The Practice of Market Research 31362.indd 354
27/09/2021 21:51
Developing a sampling plan
respond differ in any way from those who did respond. Furthermore, the size and
scope of the census undertaking may result in an increase in the amount of nonsampling error – that is, error arising from sources other than sampling. In a census
scarce administrative, field and data processing resources are likely to be stretched
to the limit, leading to errors in survey handling and administration prior to, during and after the survey. In the end, a census may deliver data of poorer quality
than a well-designed sample. Some of the cost and time savings that arise from
using a sample rather than a census could be directed to reducing non-response
and non-sampling error.
The argument for using a well-designed sample rather than a census rests on
two issues: on the practical issue of the time and cost involved in administering it;
and on the methodological issue of the ability of a sample to be representative of
the population (to deliver external validity). By ‘representative’ we mean that the
results provided by the sample are similar to those we would have obtained had we
conducted a census. Of course it is unlikely, no matter how carefully we choose a
sample, that it will deliver results that match exactly the values in the population.
Sampling theory tells us that a sample design is sound if it delivers results each time
it is repeated that on average would have been achieved with a population census.
Producing representative results is an important aspect of actionable research. It
would be pointless if a study of a sample of older people’s health and social welfare
needs could not be used to generalise about the health and social welfare needs of all
older people; or if, from a study of the brand preferences of a sample of 18–24-yearolds we could not make reliable and valid inferences about the brand preferences of
all 18–24-year-olds.
Sampling techniques
How do you design a sample that is representative of the population from which it is
drawn? It is important to restate what we mean by ‘representative’. When a sample
is representative of the population it should deliver results close to the results we
would have obtained if we had surveyed the entire population. The results are not
biased in any way – the sample estimates of the characteristics we set out to measure
(for example the use of music downloads among 16–24-year-olds) closely match the
value of these characteristics in the population. So what kind of sampling technique
produces a representative sample?
Types of sampling technique
There are two categories of sampling techniques:
●
●
random or probability sampling;
purposive or non-probability sampling.
Random or probability sampling is where each element of the population is drawn
at random and has a known (and non-zero) chance of being selected. The person
choosing the sample has no influence on the elements selected. The random selection process should ensure to some extent that the sample is representative of the
355
M14 The Practice of Market Research 31362.indd 355
27/09/2021 21:51
Chapter 14
Sampling
population. There are certain conditions that need to apply, however, for random
selection to produce a truly representative sample:
For true randomness in the selection process to take effect the sample size must be
at least 100.
● The population should be homogeneous or well mixed – if it is not (if it is stratified
or layered in any way or there is a tendency for similar elements within it to cluster
together) a simple random selection process may not deliver a truly representative
sample.
● The sampling frame, which represents the population from which the sample is
chosen, must be complete, accurate and up to date.
● Non-response must be zero or, put another way, all those selected as part of the
sample must take part in the research.
●
Of course, in real-world research situations, the last three conditions may not hold.
We come back to this in more detail later.
The theory that underpins probability or random sampling allows us to calculate
how accurately a sample estimates a population characteristic and how likely or
probable it is that the sample estimate lies within a certain range of the population
characteristic. This leads us to the concepts of sampling distributions, sampling error,
standard error and confidence intervals, to which we return in more detail in the
section on sampling theory.
In non-probability sampling there is no random selection process, and we do
not know what probability each element has of being selected because the person
choosing the sample may consciously or unconsciously favour or select particular
elements. So how do we ensure that the sample chosen in this way is representative
of the population? We address this issue when we look in detail at non-probability
sampling methods later in the chapter. Suffice to say at this stage that quota sampling, the method of non-probability sampling most widely used in market research,
can produce results that closely resemble those that would have been achieved with
a probability sample.
Choosing a sampling technique
How do you decide which type of sampling technique to use? The decision is complicated. It will be influenced – as we noted above – by both methodological issues,
such as the nature and aims of the study, and by practical concerns, including the
nature and accessibility of the study population, the availability of a suitable sampling
frame, and the constraints of time and budget.
In deciding what sampling technique to use, think first of all about the nature and
aims of the study. If the purpose of the research is exploratory and not conclusive
(that is, neither descriptive nor explanatory), in other words if it is not necessary
to obtain highly accurate estimates of population characteristics in order to make
inferences about the population, then a non-probability sample is appropriate.
If, on the other hand, it is necessary to obtain measurements from the sample of
known accuracy or precision (in order to make statistical inferences or generalisations from the sample to the population), then a probability sampling technique
should be used.
Random sampling, however, does not always produce more accurate estimates
of population characteristics than non-probability techniques. In fact, in certain
356
M14 The Practice of Market Research 31362.indd 356
27/09/2021 21:51
Developing a sampling plan
circumstances, non-probability (quota) sampling may provide a more representative sample. Where there is little variability within a population, that is when the
population is homogeneous, a non-probability sample can be effective in achieving
a representative sample; with a great deal of variability in the population a random
sample is likely to be more effective. When the non-sampling error (errors arising
from question wording, interviewer bias, recording error, data-processing error) is
likely to be greater than the sampling error, non-probability techniques may be just
as good at producing a representative sample.
Industry Insight 14.1 shows the decision-making process involved in choosing a
sampling approach, in this case for survey of social attitudes among young people.
In terms of the practicalities, if there is no suitable sampling frame from which
to select the sample, then random methods are not feasible. We look at sampling
Industry Insight 14.1
Sampling 16-year-olds
Introduction
From 1998–2000, the Young Life and Times
(YLT) Survey recorded the attitudes of young people aged 12–17 years living in the same household
as an adult respondent to the Northern Ireland Life
and Times Survey. YLT involved a paper questionnaire containing a sub-set of questions from the
adult survey, and one complete module of particular relevance to young people. One rationale for
this methodology was that the responses of the
adult and those of the younger respondents could
be linked and subsequently analysed.
However, by 2000, the response rate had
dropped from 74 per cent (in 1998) to 62 per cent.
In addition, while many researchers were using
the data from the adult and young people’s surveys separately, few were actually making use of
the link. In the light of this, in autumn of 2001 the
YLT team undertook a review of the future and
format of the Young Life and Times Survey. The
review consisted of three strands:
a review of other surveys of young people and,
in particular, their sampling methodologies;
● a review of postal and online surveys;
● a discussion forum, involving users and potential users of the surveys from the academic and
voluntary sectors.
●
The outcomes of this review were that there was
unanimous support for having some sort of Young
Life and Times Survey. Having a time series component was useful, especially when monitoring
the impact of policies on young people’s attitudes.
However, the link between the adult and young
person’s survey was not seen as important and
so a standalone Young Life and Times was seen
as acceptable. The age-range of respondents was
an issue. In particular, interviewing younger people (under 16 years) requires parental permission.
There was a concern that questions suitable for
17-year-olds were not always suitable for 12-yearolds (and vice versa) and that this was restrictive.
Finally, consultation with young people themselves, in terms of developing question themes
and/or interviewing, was suggested. Consultation
was also thought to be important for ‘selling’ the
survey to young people.
Population and sample: deciding on a
sampling frame
Thus, in 2003, the YLT team planned to run a
revised version of the survey, among 16-yearolds only. The population of interest therefore
was all 16-year-olds living in Northern Ireland.
The question was how to find a sampling frame
for this group. We ruled out using schools as a
357
M14 The Practice of Market Research 31362.indd 357
27/09/2021 21:51
Chapter 14
Sampling
way of sampling for several reasons including the
following:
not all schools would agree, and only particular
types of schools might participate;
● the problem of privacy among pupils;
● the effect of having a teacher in the room;
● the omission of pupils excluded from school.
●
While there are also problems associated with
household sampling (for example, the exclusion
of young people not living in a private household
and parental influence on response), on balance
this may be the best methodology for obtaining a
more robust sample.
The sample frame
We knew that every child is eligible for Child Benefit, a government benefit for people bringing up
children. The Child Benefit Register contains information on all children for whom Child Benefit is
claimed. This Register would be a very useful sampling frame for our population. Getting access to it
was now the issue. The Child Benefit Register was
maintained by the Social Security Agency (SSA) of
the Department for Social Development (DSD) in
Northern Ireland, who kindly agreed to facilitate
drawing the sample. We decided to approach all
young people who celebrated their 16th birthday in
February of that year, accounting for approximately
2000 young people. However, in 2004, while DSD
still maintained the database, the responsibility for
the payment of Child Benefit transferred to Inland
Revenue. Thus, it was necessary to negotiate access
to this Register from Inland Revenue. This process
of negotiation took five months, culminating in
the preparation of an explanatory memorandum
relating to the Tax Credits (Provision of Information) (Evaluation and Statistical Studies) (Northern
­Ireland) Regulations 2004.
Sample selection
With access now available, we were able to select
all those young people who celebrated their 16th
birthday during February 2004. However, under
data protection regulations relating to use of personal data, the survey team could not contact these
young people directly. Thus, all documentation
relating to the survey was processed by an independent research organisation.
Each eligible young person received a letter from
DSD inviting him or her to take part in the survey.
The initial letter was addressed to the relevant person and provided an introduction to the survey.
It also explained the role of DSD in the project
and confirmed that the YLT project team did not
have access to names and addresses of the young
people in the sample. This letter contained a unique
identifier (with a check letter) under the address,
which was highlighted as ‘Your identification number’. A non-personalised letter from the university
project team provided more information about the
survey, including the aims of the project, the three
possible methods of completing the questionnaire,
and details of a prize draw of £500 for which all
respondents completing the questionnaire were eligible. The pack also contained a paper questionnaire and a pre-stamped return envelope.
Fieldwork
The fieldwork period lasted from 25 August to
24 September 2004. While every eligible young person received a paper questionnaire, each was able
to choose one of three methods for completing it.
1 They could take part by phone, having quoted
their identification number and check digit.
2 They could complete the questionnaire online
– quoting their personal identifier to enter that
part of the Young Life and Times website.
3 They could complete the paper questionnaire
that was sent to them in the initial pack and
post it back in the pre-stamped envelope.
After one week, a reminder postcard was sent out to
addressees who had not made contact of any kind.
Response rate
1,983 questionnaires were sent out; 824 completed questionnaires were received, representing a
response rate of 41.6 per cent. The response rate has
been as follows: 2005 – 40 per cent; 2006 – 39 per
cent; 2007 – 33 per cent; 2008 and 2009 – 23 per
cent; 2010 – 21 per cent; and 2011 – 37 per cent.
Source: Dr Paula Devine, Deputy Director, ARK, Queen’s
University Belfast, written for this book. Used with permission.
358
M14 The Practice of Market Research 31362.indd 358
27/09/2021 21:51
Developing a sampling plan
frames in more detail later. In addition, probability sampling, especially for faceto-face research, can be difficult, time consuming (not only in terms of drawing the
sample but in conducting the fieldwork) and expensive; it is more straightforward
and easier to manage in a telephone survey. If time and budget are limited in a faceto-face study, it is likely that a non-probability method such as quota sampling will
be used. We look in more detail at various random or probability and non-probability
techniques later in the chapter.
Choosing a sample size
The sample size is the number of elements that will be included in the sample. The
size of the sample is important, particularly in terms of the precision of the sample
estimates, but on its own does not guarantee that the results will be accurate or
unbiased; the way in which the sample is chosen (the sampling technique used, the
sampling frame) will affect this.
Deciding on the sample size involves thinking about the nature and purpose
of the research enquiry, and the importance of the decisions to be made on the
basis of the results. In exploratory research, the sample size (for qualitative or
quantitative methods) may be relatively small in comparison with that used in
a conclusive study. In conclusive research enquiries the aim is often to provide
precise estimates of population characteristics (also called population parameters)
– for example the proportion of 25–34-year-olds using brand X. The sample size
therefore needs to be big enough to provide such estimates. The research may be
commissioned to provide conclusive evidence that, for example, a greater proportion of 16–24-year-olds compared with 25–34-year-olds prefer brand X, and to
provide that evidence with a certain degree of confidence that the findings are
an accurate reflection of the situation in the wider population. The sample size
in this case needs to be large enough to provide the evidence with the specified
degree of confidence. If we know the level of precision required of the sample
estimates, or the size of the confidence level or interval required, we can work
out the sample size needed to achieve these. We look at this in more detail later,
in the section on sampling theory.
It is also important to consider the way in which the findings will be analysed. You
may need to look at (and compare) the findings among particular sub-groups within
the sample, for example particular age groups, or organisations of different sizes or
in different industry sectors. It is therefore important to consider how big these subgroups need to be in order to provide precise estimates of their characteristics and to
allow robust analysis. Also, you need to think about the type of analysis needed – if,
for example, you plan to use multivariate statistical techniques, you need to think
about what implications this has for sample size. In planning the sample it is helpful
to know the incidence in the population of any groups of interest, as this may affect
the decision about the overall sample size and the choice of sampling technique.
Finally, and arguably in practice the most important factor in the choice of sample size, we must take into account the time, budget and other resources available.
Generally speaking, for any given sampling approach, the bigger the sample size the
greater the cost.
359
M14 The Practice of Market Research 31362.indd 359
27/09/2021 21:51
Chapter 14
Sampling
Preparing sampling instructions
Once a sampling approach and a sample size have been agreed it is important to set
out how the actual sampling process is to be conducted. This will involve drawing
up a sampling plan that should include the following:
the definition of the target and/or study population;
the sample size required;
● the sampling method to be used, including the way in which the units and elements
are to be selected;
● details of the sampling frame, if one is available.
●
●
Checking the sample achieved
As the fieldwork progresses the sample is monitored to ensure that the units and
elements selected meet the sample criteria. Once sampling and fieldwork are
completed, the sample achieved is checked to ensure that it matches the sample
Box 14.2
Example: sample details from the Life and Times Survey 2018
Target population: adults aged 18 and over in Northern Ireland.
Method: pre-selected addresses; advance letter notification; in-home, face-to-face
interviews using CAPI plus a self-completion questionnaire.
● Required sample size: 1,201.
● Sampling frame: Postcode Address File (PAF) with private business addresses
removed from database before sample selection.
● Sampling units: households.
● Sample elements: persons aged 18 and over; interviewers listed birthdays of all
members of the household eligible for inclusion (i.e. all persons aged 18 and over)
at each address. The person with the next birthday at the time of the call was the
person with whom the interview was to be conducted. Where the selected person
was not available, an appointment was made to call back to interview them at a more
suitable time.
● Sampling technique: systematic random sampling.
● Number of sampling units selected: 2,250 selected and throughout the survey period
a further 1,000 addresses were pulled. Not all addresses pulled were issued to interviewers. In total, 2,296 were issued.
● Number of sampling units in scope: 2,161 (135 were found to be ineligible – vacant,
derelict or commercial properties).
● Response from 2,161 addresses: 1,201 interviews; 1,201 fully co-operating; 0 partially co-operating; 653 refusal to co-operate; 253 non-contacts; and 54 ‘other’. This
is a response rate of 58 per cent, 30 per cent refusals, 12 per cent non-contacts and
2 per cent ‘other’. The percentage of self-completions achieved was 100 per cent.
●
●
Source: Adapted from Devine, P., Technical Notes (https://www.ark.ac.uk/nilt/2018/tech18.pdf).
360
M14 The Practice of Market Research 31362.indd 360
27/09/2021 21:51
Sampling theory
requirements. If any discrepancies are found (high rates of non-response, under- or
over-representation of particular elements and so on) it will be necessary to address
them (for example by conducting further sampling and fieldwork, or statistical
manipulation). It is also important to check key sample statistics against the relevant
population parameters, if that information is available (for example from a recent
census) or against sample statistics from other surveys. This serves as a validation
check on the representativeness of the sample.
At the end of the project you should record all the key information about the
sample you planned and the sample that was achieved. For an example of this, see
Box 14.2 above. If you commissioned the research from an agency or from an online
panel provider, you should be given a similar sample report. It should tell you how
the total population was defined, how the sample selected was drawn, what the gross
sample was, what the start and participation or co-operation rates were, and what the
drop-out rate was. Where relevant, you will also want to have a copy of the invitation or contact text, and you will want to know the details of the fieldwork process
including the number of reminders sent or call-backs made, the quality checks made
and so on. Recording all of this information on the sample, the sampling process
and the fieldwork procedures is good practice and will be useful to those reviewing
or revisiting the research project (it will give them an idea of the overall quality of
this element of the project), and useful to those who may want to repeat the research
at some point in the future. For an online sample, ESOMAR (2015) suggests you
provide the following information:
Sampling frame, sources and sampling method used
Fieldwork dates
● Average or median survey length
● Total number of interviews completed
● Any quotas or other specifications used in sample selection
● Questionnaire and other relevant data collection documents
● The number of survey participants whose identity was successfully validated
● A description of any de-duplication methods used and the numbers removed as a
result
● Measures of participant engagement used and an account of any participants
removed or replaced
● Exclusion information
● Participation rates
● Completion rate
● Details of any subcontractors used.
●
●
Sampling theory
Before discussing the details of the various sampling techniques, we need to look at
the theory that underpins probability sampling. This is important because it will help
you to understand better a number of related issues, including those of precision,
accuracy and bias, and the rationale behind confidence intervals and inferential statistical tests. It will also help you distinguish between probability sampling techniques
and non-probability sampling.
361
M14 The Practice of Market Research 31362.indd 361
27/09/2021 21:51
Chapter 14
Sampling
Terminology
First of all, we need to introduce some more terminology. The things that we
want to talk about in the population, for example the proportion of 18–24-yearolds who drink brand A, or the average income of a particular group, are known
as population parameters. The corresponding figure derived from the sample is
an estimate of this population parameter and is known as a sample statistic. For
example, in a survey of the brand preferences of 18–24-year-olds, the proportion
who drink brand A is the sample statistic, or the estimate of the proportion who
drink brand A in the population. Here is another example: you are conducting a survey among organisations in the financial services sector to determine
the average pay of women. The average obtained from the sample is called the
sample statistic. It is an estimate of the population parameter, the unknown
value of average pay among women in the wider population of financial services
organisations.
The purpose of a survey may be to provide such estimates. The important thing to
remember is that the findings provided by a sample are only estimates of the population values. Statements based on findings from a random or probability sample are
always probability statements. We cannot make claims about the value of population parameters based on sample data with absolute certainty. What we do is rely on
an effective sample design to ensure that the sample estimates accurately reflect the
population values most of the time, and with a known margin of error. This brings
us to sampling theory.
Sampling distribution of the mean
You are interested in knowing the weekly food spend of single person households
in Sweden. You select a sample at random from the population of all single person
households and from the sample data you note the average (or mean) of the particular value that interests you – weekly spend on food. You then select another
sample at random and note the value of weekly food spend from this sample. You
continue this process ad infinitum; you plot the value of the average weekly spend
on food from each sample on a graph. Once you have plotted this value for your
infinite samples you should have a graph like the one in Figure 14.1, the bell-shaped
curve of a normal distribution. This graph is known as the sampling distribution
of the mean.
Sampling variability
The graph shows that each sample does not produce the same value: a range of samples produces a range of values for the same measure (in this case the average weekly
food spend). This variation is known as sampling variability. In real-world research,
however, we do not take repeated samples from a population to measure a value; usually we take only one sample and we estimate the population value on the basis of this
one sample. But given the amount of variability between samples that the sampling
distribution shows, how can we know how accurately our sample measure reflects
the true population value? We can do this with a fairly simple calculation – called
362
M14 The Practice of Market Research 31362.indd 362
27/09/2021 21:51
Sampling theory
the standard error of the mean – from one randomly selected sample made up of at
least two sampling units.
Standard error of the mean
The standard error of the mean is a measure of the variability within the sampling
distribution – the variability or spread in the values of the measures we have taken
from each sample. It is the standard deviation of the sampling distribution. We can
use it to measure the probable accuracy or precision of a particular sample estimate.
To work out the standard error of the mean we need to know the standard deviation
of the population (S) and the size of the sample (n). There is a small complication – it
is very unlikely that we will know the value of the population standard deviation. In
its place we use the standard deviation of the sample (s).
From the information needed to calculate the standard error of the mean we can see
that what it measures – the precision of a sample estimate – depends on two things:
sample size and the level of variability in the population, which is measured by the
standard deviation. It makes sense that these two factors have an impact on precision.
If you think about sample size, it makes sense that a bigger sample will deliver results
that are more precise. The formula for calculating the standard error shows the relationship between precision and sample size: to increase the precision of an estimate by
a factor of two – in other words, to halve the standard error – you need to increase the
sample size by a factor of four. It also makes sense that variability within the population will have an impact on the precision of a sample estimate. If, for example, there is
very little variability – say, for example, that the average weekly food spend of all single
person households in Sweden is €200, then the standard deviation and the standard
error would be zero. We can say that the sample provides a precise estimate of the
population value. If, however, the average weekly food spend varies from, say, €50 to
€500, the standard deviation will be relatively large and so will the standard error. As
a result, the sample will provide a less precise estimate of the population value.
Confidence intervals
You saw in Figure 14.1 that the sampling distribution of the mean closely resembles a
normal distribution. In fact, the larger the sample, the closer the sampling distribution
Number of samples
Y
Estimates of mean
X
Figure 14.1 Sampling distribution of the mean
363
M14 The Practice of Market Research 31362.indd 363
27/09/2021 21:51
Chapter 14
Sampling
Box 14.3
Formula for calculating the standard error of the mean
For numerical data:
Standard error (x) =
For % data:
Standard error (p%) =
A
s
2n
p%(100 - p%)
n
will be to a normal distribution. The normal distribution has a number of useful
properties that can be applied to sampling. It is symmetrical in shape, with 50 per
cent of observations or measures lying above the mean and 50 per cent lying below
the mean. If we divide the normal curve up into segments delineated by standard
deviations, we find that about 68 per cent of all observations lie within 1 standard
deviation either side of the mean; 95 per cent lie within 2; and 99 per cent are within
2.6 standard deviations.
If a sampling distribution closely resembles a normal distribution then we can
use the properties of the normal distribution to obtain some very useful information
about our sample estimates. The first thing we need to do is to convert the standard
deviations into standard errors. This allows us to say that 95 per cent of our sample
estimates lie within 1.96 standard errors of the population mean; and 99 per cent lie
within 2.58 standard errors. To put it another way, we can say that a sample mean
or sample statistic has a 95 per cent chance of being within 1.96 standard errors of
the population mean or the true mean; or a 99 per cent chance of being within 2.58
standard errors.
Calculating the accuracy of the sample estimate
An example makes all this a bit less abstract and a bit more real: imagine that you
have completed your survey on weekly food spend among single person households
in Sweden. You have found that the average weekly spend is €250. The first question
you ask is, how accurate an estimate is this of the population value? In other words,
how big is the standard error? To work this out you need to know the standard
deviation and the size of the sample.
The first step in working out the standard deviation is to calculate the variance
(which is a fairly simple measure of the spread of values within the sample). To
do this, you subtract the sample mean from each of the individual observations,
which in this case are amounts of money spent on food. Next, you square each of
the deviations from the mean (to get rid of any negative values), then add them all
up and divide by the sample size. The number you get is the variance of the sample.
Take the square root of the variance to get the standard deviation. To calculate the
standard error, divide the standard deviation by the square root of the sample size.
The calculations are slightly different if you have proportions or percentages rather
364
M14 The Practice of Market Research 31362.indd 364
27/09/2021 21:51
Sampling theory
Box 14.4
Formulae for variance and standard deviation
Using means
a (x - x)
n
2
Variance s2 =
Standard deviation (s) = 2s2
Using percentages
To calculate the standard deviation using percentages:
Standard deviation (s) =
A
p%(100 - p%)
n
than means, for example if you want to look at the proportion of buyers of brand
A in the sample.
Of course, you would use a computer program to calculate these figures – in a
real research project it would not be practicable to calculate them by hand. From
the formulae, however, you can get some idea of the underlying logic. The second
column in Table 14.1 shows the weekly food spend from the ten households in the
sample (in reality, of course, the sample would be much bigger). Column three shows
the average spend across all ten households; column four shows the deviation of the
actual spend from the average spend; and the final column shows the square of that
deviation.
The sum of the squared deviations – the total of the figures in the final column
of Table 14.2 – is 280. To calculate the variance you divide by the sample size,
which in this example is 10. The variance therefore is 28. The standard deviation,
that is the square root of the variance (28), is 5.29. The standard error, which is the
standard deviation (5.29) divided by the square root of the sample size (3.16), is
50%
50% of values lie to
one side of the mean
and 50% to the other
Mean
50%
68% of values lie
within 1
21sd
96% of values lie
within 1
22sd
99% of values lie
within 1
23sd
68%
96%
99%
]3sd
]2sd
]1sd
0
11sd
12sd
13sd
Figure 14.2 Normal curve with standard deviations
365
M14 The Practice of Market Research 31362.indd 365
27/09/2021 21:51
Chapter 14
Sampling
Table 14.1 Symbols for population and sample values
Value
Population
Sample
Mean
Proportion
Variance
Standard deviation
Size
m
p
s2 or S2
S
N
x
p
s2
s
n
1.67. What does this figure tell you? You can say that you are 68 per cent confident
that the true (population) value of average weekly food spend lies within the range
:250 { :1.67 (the mean plus or minus 1 standard error). In other words, you are
68 per cent confident that the average weekly spend among the population is somewhere between €248.33 and €251.67. You can say that you are 95 per cent confident
that it lies within the range 250 { 1.96 standard errors (1.96 * 1.67) – that is,
between €246.73 and €253.27. You can be 99 per cent confident that it lies within
the range 250 { 2.58 standard errors (2.58 * 1.67), that is, between €245.69 and
€254.31. These limits on the range of a value are called confidence limits. The size
of the difference or the margin of error is called the confidence interval.
You can look at this another way – in terms of the probability that the claims you
make about your findings are correct. This is where significance levels come in. If
you claim that the average weekly food spend among the population lies somewhere
between €246.73 and €253.27, the probability that you are right in this assertion
is 95 per cent (the confidence limit is 95 per cent). There is a 5 per cent or 1 in 20
chance that you are wrong (this is known as the significance level). If you want to
make sure that there is less chance that your assertion is wrong, say, 1 in 100 or 1 per
cent chance (a greater significance level), you are setting a wider confidence interval,
which means the margin of error will be larger.
Table 14.2 Calculations involved in determining the standard deviation
Sampling unit
Household
1
2
3
4
5
6
7
8
9
10
Total
Observation (x)
Weekly food spend (€)
247
253
247
248
259
242
250
252
244
258
2,500
Sample mean (x)
Average spend (€)
250
250
250
250
250
250
250
250
250
250
Deviation from the
sample mean (x - x)
Square of the
deviations (x - x)2
-3
+3
-3
-2
+9
-8
0
+2
-6
+8
Total
9
9
9
4
81
64
0
4
36
64
280
366
M14 The Practice of Market Research 31362.indd 366
27/09/2021 21:51
Sampling theory
In conducting a piece of research you may want to specify at the outset how precisely you want the sample measures to reflect the population values – in an opinion
poll, for example. In other words you may want to specify the confidence limits and
the margin of error that will be acceptable. For example, in the survey of weekly food
spend among single person households, you may want to set the confidence limits at
the 95 per cent level (the level most commonly used in market and social research)
and you might want the estimate of average weekly spend to be accurate to within
€1 of the population values. The question is, what sample size do you need to achieve
this? The formula for calculating the sample size is given in the example in Box 14.5.
So you need a sample of 108 to ensure that you can be 95 per cent confident that
our estimate of average weekly food spend is within €1 of the population value.
In research, we deal with percentages as well as averages. To work out the sample
size necessary to ensure that a particular percentage is within an acceptable margin
of error, the formula is similar. For example, in the survey of weekly food spend,
imagine that you asked whether people had bought fresh fruit. You expect that about
60 per cent will have done so and you want a confidence interval of 2 per cent and a
confidence level of 99 per cent. The calculation is shown in Box 14.6.
Significance levels and the risk of error
Significance levels are the level of probability at which you accept that a difference is statistically significant or real – that is, that it is not due to chance. They
are sometimes referred to as the p or (alpha) value. The level of significance is the
point at which the sample finding or statistic differs too much from the population
expectation for it to have occurred by chance – the difference cannot be explained
by random error or sampling variation and is accepted as a true or statistical difference. Decisions about whether a null hypothesis is accepted or rejected are based on
these significance levels.
Which significance level to use?
The three significance levels used most often are the 5 per cent or the 0.05 level of
probability (sometimes written as p = 0.05 or p 6 0.05); the 1 per cent or 0.01
level; and the 0.001 or 0.1 per cent level. At the 5 per cent significance level there is
a 5 per cent probability or a 1 in 20 chance that the result or finding has occurred
by chance. This is the lowest acceptable level in most market and social research
projects. At the 1 per cent significance level you are setting a higher standard by saying that there is a 1 per cent or 1 in 100 probability that the finding has occurred by
chance. The 0.1 per cent level indicates that there is a 1 in 1,000 probability that the
finding has occurred by chance. As the significance level falls, in other words, the
more confident you can be in the results (the confidence level is greater). So in using
significance levels to judge results, you are giving a probability that the results are
sound and at the same time saying that there is a chance that they may not be. The
significance level you choose will depend on the amount of risk you are prepared
to tolerate in drawing the wrong conclusions from the research. If, for example, the
research involves evaluating a product, it might be best to set the significance level at
367
M14 The Practice of Market Research 31362.indd 367
27/09/2021 21:51
Chapter 14
Sampling
Box 14.5
Calculating the sample size for a given level of precision
The 95 per cent confidence interval means that the sample estimate will lie within 1.96
standard errors of the mean. So 1.96 = z
The standard deviation of the sample is 5.29. So s = 5.29
The margin of error we want is : { 1. So d = {1 where d = the confidence
­interval required
Formula for working out sample size:
z2s2
d2
(1.96 * 1.96) * (5.29 * 5.29)
=
(1 * 1)
= 107.5
n =
= 108 (rounded up to the nearest whole number)
p = 0.001, as the impact on the client’s business of launching a product that might
be rejected 5 per cent of the time (if p = 0.05) could be costly.
Type I and Type II errors
Every time you make a decision to accept or reject a null hypothesis you risk making an error. There are two types of error – Type I or (alpha) and Type II or (beta)
errors. If you make a Type I error you reject the null hypothesis when in fact it is true
and you should have accepted it. An example of a Type I error is when an innocent
person is found guilty. You make a Type II error when you accept the null hypothesis
when in fact it is false and should have been rejected. A Type II error is when a guilty
person is acquitted.
The chance of committing a Type I error is no greater than the level of significance
used in the test (which is why the significance level is sometimes called the alpha
value, the value associated with an alpha error). If you use the 5 per cent level you
can only make a Type I error 5 per cent of the time. You can reduce the probability
of making a Type I error by setting the significance level at 1 per cent or 0.1 per cent.
If you drop the significance level (in effect increasing the stringency of the test and
raising the confidence limits to 99 per cent or 99.9 per cent) you increase the chances
of making a Type II error.
In setting significance levels, therefore, you need to reach a compromise between
the types of error. If making a Type I error (accepting as true something that is really
false) is deemed worse than making a Type II error (accepting something that should
be rejected and is not), then you should set the significance level low (say 0.1 per
cent). If, however, the risks associated with a Type II error are greater, then it might
be best to set the significance level at 5 per cent. To lower the risk of either type of
error arising, you increase the sample size.
368
M14 The Practice of Market Research 31362.indd 368
27/09/2021 21:51
Probability or random sampling methods
Box 14.6
Sample size calculations using percentages
Confidence interval d% = {2 per cent
Standard error for 99 per cent confidence level z = 2.58
Estimate of percentage being measured p = 60
Formula
n =
z2p%(100 - p%)
d2
(2.58 * 2.58) * 60(100 - 60)
=
2 * 2
15,975.36
=
4
= 3,993.84
If you were to reduce the confidence level from 99 per cent to 95 per cent, what
effect would this have on the sample size needed to achieve the same confidence interval of 2 per cent? Lowering the confidence level would mean that a sample of 2,305
would deliver a 2 per cent confidence interval. If you were to reduce the confidence
interval by half to 1 per cent, keeping the confidence level the same, what would this
mean for sample size? You would need a sample size of some 9,220 – in other words,
to halve the confidence interval you need a fourfold increase in sample size.
In deciding on sample size it is not just the total sample that is important; you need
also to think about the size of sub-groups within the main sample. For example, it
may be crucial to the research objectives to examine the views of women and men
separately, or to look at regular users of a service or occasional users. You need to
make sure that these sub-samples are large enough to allow you to comment at the
chosen confidence level and within an acceptable confidence interval.
Caveat – For ease of explanation, all of the above is based on the use of a simple
random sampling approach. Most sample designs in real-world market and social
research are more complicated than this, with the result that calculating margin of
error and confidence intervals is also more complicated. We have also made assumptions about using standard deviation of the sample rather than the standard deviation
of the population.
Probability or random sampling methods
A random or probability sample is one in which each member of the population has
a known and non-zero chance of being selected. There are several kinds of random
sampling methods, from the fairly straightforward simple random sampling approach
to the more complex cluster sampling methods.
369
M14 The Practice of Market Research 31362.indd 369
27/09/2021 21:51
Chapter 14
Sampling
Simple random sampling
Simple random sampling works like this: imagine we have a population of 1,000
(denoted N = 1,000). The population might consist of people or organisations,
whatever is relevant to the research investigation. Before making any selection from
the population, we know that each item in it has a 1 in 1,000 chance of being selected.
Once an item is selected as part of the sample we do not return it to the population.
This is known as sampling wit
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )