Course Development Team
Head of Programme
: Associate Professor Kang Soon-Hock
Approving Faculty
: Associate Professor Ng Wei Ting
Course Developer(s)
: Dr Yao Shih-Ying
Technical Writer
: Chat Inoue, ETP
Video Production
: Danny Chin, ETP
©
2021 Singapore University of Social Sciences. All rights reserved.
No part of this material may be reproduced in any form or by any means without
permission in writing from the Educational Technology & Production, Singapore
University of Social Sciences.
ISBN 9789814873093
Educational Technology & Production
Singapore University of Social Sciences
463 Clementi Road
Singapore 599494
How to cite this Study Guide (APA):
Yao, S. Y. (2021). HBC201 Research methods for the social and behavioural sciences (study
guide). Singapore: Singapore University of Social Sciences.
Release V1.0
Build S1.0.5, T1.5.21
Table of Contents
Table of Contents
Course Guide
1. Welcome.................................................................................................................. CG-2
2. Course Description and Aims............................................................................ CG-3
3. Learning Outcomes.............................................................................................. CG-5
4. Learning Material................................................................................................. CG-6
5. Assessment Overview.......................................................................................... CG-7
6. Course Schedule.................................................................................................... CG-9
7. Learning Mode.................................................................................................... CG-10
Study Unit 1: Introduction to Social and Behavioural Science
Research
Learning Outcomes................................................................................................. SU1-2
Overview................................................................................................................... SU1-3
Chapter 1: Research: Why It Matters.................................................................... SU1-4
Chapter 2: From Philosophical Foundations to Real-Life Applications........ SU1-10
Chapter 3: Research Ethics................................................................................... SU1-19
Summary................................................................................................................. SU1-29
Formative Assessment.......................................................................................... SU1-30
References............................................................................................................... SU1-38
i
Table of Contents
Study Unit 2: Research Orientation, Design, and Measurement
Learning Outcomes................................................................................................. SU2-2
Overview................................................................................................................... SU2-3
Chapter 1: Research Orientations: Numbers or Words...................................... SU2-4
Chapter 2: Research Design.................................................................................. SU2-18
Chapter 3: Measurement....................................................................................... SU2-32
Summary................................................................................................................. SU2-39
Formative Assessment.......................................................................................... SU2-40
References............................................................................................................... SU2-50
Study Unit 3: Quantitative Research Methods I
Learning Outcomes................................................................................................. SU3-2
Overview................................................................................................................... SU3-3
Chapter 1: Key Considerations in Quantitative Research................................. SU3-4
Chapter 2: Types of Quantitative Research....................................................... SU3-12
Summary................................................................................................................. SU3-23
Formative Assessment.......................................................................................... SU3-24
References............................................................................................................... SU3-33
Study Unit 4: Quantitative Research Methods II
Learning Outcomes................................................................................................. SU4-2
Overview................................................................................................................... SU4-3
Chapter 1: Introduction to Survey Research....................................................... SU4-4
ii
Table of Contents
Chapter 2: Major Concerns in Survey Research............................................... SU4-13
Chapter 3: Ethics in Survey Research................................................................. SU4-18
Summary................................................................................................................. SU4-21
Formative Assessment.......................................................................................... SU4-22
References............................................................................................................... SU4-31
Study Unit 5: Qualitative Research Methods I
Learning Outcomes................................................................................................. SU5-2
Overview................................................................................................................... SU5-3
Chapter 1: Main Considerations in Qualitative Research................................. SU5-4
Chapter 2: Types of Interview in Qualitative Research................................... SU5-11
Chapter 3: Politics of Research............................................................................ SU5-16
Summary................................................................................................................. SU5-18
Formative Assessment.......................................................................................... SU5-19
References............................................................................................................... SU5-28
Study Unit 6: Qualitative Research Methods II
Learning Outcomes................................................................................................. SU6-2
Overview................................................................................................................... SU6-3
Chapter 1: Major Types of Qualitative Research................................................ SU6-4
Chapter 2: Ethnography........................................................................................ SU6-11
Summary................................................................................................................. SU6-16
Formative Assessment.......................................................................................... SU6-17
References............................................................................................................... SU6-26
iii
Table of Contents
iv
List of Tables
List of Tables
Table 1.1 Focus of Ontology and Epistemology.................................................... SU1-11
Table 1.2 General Ethical Principles........................................................................ SU1-21
Table 1.3 Common Ethical Issues Concerning Research Participants................ SU1-22
Table 1.4 Common Ethical Issues Concerning Reporting of Research Findings
and Protection of Intellectual Property Rights....................................................... SU1-26
Table 2.1 Examples of Hypotheses in Various Forms............................................. SU2-7
Table 2.2 Purpose of Data Analysis........................................................................... SU2-8
Table 2.3 Four Possible Consequences of a Decision in a Hypothesis Test........ SU2-11
Table 2.4 Features of Qualitative Research............................................................. SU2-12
Table 2.5 A Comparison of Quantitative and Qualitative Research................... SU2-16
Table 2.6 Various Types of Triangulation................................................................ SU2-19
Table 2.7 Description of Cross-Sectional Research and Longitudinal
Research........................................................................................................................ SU2-21
Table 2.8 Different Types of Longitudinal Research............................................. SU2-22
Table 2.9 Benefits and Challenges of Longitudinal Research.............................. SU2-24
Table 2.10 A Comparison of Internal and External Validity................................ SU2-27
Table 2.11 Common Threats to Internal Validity................................................... SU2-27
Table 2.12 Common Threats to External Validity.................................................. SU2-29
Table 2.13 Four Levels of Measurement................................................................. SU2-32
v
List of Tables
Table 2.14 Common Types of Validity Evidence................................................... SU2-34
Table 2.15 Common Types of Reliability Evidence............................................... SU2-36
Table 3.1 Probability of a Possible Consequence in a Hypothesis Test................. SU3-5
Table 3.2 Primary Factors That Influence the Power of a Test............................... SU3-6
Table 3.3 Main Types of Probability Samples.......................................................... SU3-8
Table 3.4 Main Types of Nonprobability Samples................................................... SU3-9
Table 3.5 Common Misconceptions about Quality of Samples........................... SU3-10
Table 3.6 Elements of an Experimental Design..................................................... SU3-12
Table 3.7 Three Types of the Experimental Design............................................... SU3-14
Table 3.8 Various Types of the Quasi-Experimental Design................................ SU3-16
Table 3.9 Three Common Types of the Nonexperimental Design....................... SU3-20
Table 4.1 Types of Survey Questions......................................................................... SU4-4
Table 4.2 Types of Survey Administration............................................................... SU4-6
Table 4.3 Methods to Evaluate a Survey.................................................................. SU4-8
Table 4.4 Pros and Cons of Survey Research........................................................... SU4-9
Table 4.5 Types of Measurement Error................................................................... SU4-13
Table 4.6 Common Sources of Measurement Errors in Survey Research........... SU4-14
Table 4.7 Possible Explanations of Nonresponse................................................... SU4-16
Table 4.8 Common Ethical Issues in Survey Research.......................................... SU4-18
Table 5.1 Criteria to Evaluate the Rigor of Qualitative Research.......................... SU5-7
Table 5.2 Politics of Research.................................................................................... SU5-16
vi
List of Tables
Table 6.1 Key Concepts in Ethnography................................................................. SU6-11
Table 6.2 Key Techniques in Ethnography............................................................. SU6-13
vii
List of Tables
viii
List of Figures
List of Figures
Figure 1.1 Primary Philosophical Positions within Quantitative and Qualitative
Research........................................................................................................................ SU1-12
Figure 1.2 Deduction.................................................................................................. SU1-14
Figure 1.3 Hypothetico-Deductive Method............................................................ SU1-15
Figure 1.4 Induction................................................................................................... SU1-16
Figure 2.1 Key Stages of a Quantitative Study........................................................ SU2-4
ix
List of Figures
x
List of Lesson Recordings
List of Lesson Recordings
Fundamentals of Social and Behavioural Science Research................................... SU1-4
Research Ethics............................................................................................................ SU1-19
Quantitative versus Qualitative Research............................................................... SU2-15
Research Design.......................................................................................................... SU2-18
Measurement............................................................................................................... SU2-32
Key Considerations in Quantitative Research.......................................................... SU3-4
Types of Quantitative Research................................................................................ SU3-12
Survey Research............................................................................................................ SU4-4
Main Considerations in Qualitative Research.......................................................... SU5-4
Qualitative Interviews & Politics of Research........................................................ SU5-11
Major Types of Qualitative Research......................................................................... SU6-4
Ethnography................................................................................................................ SU6-11
xi
List of Lesson Recordings
xii
Course
Guide
Research Methods for the Social
and Behavioural Sciences
HBC201
Course Guide
1. Welcome
Presenter: Yao Shih-Ying
This streaming video requires Internet connection. Access it via Wi-Fi to
avoid incurring data charges on your personal mobile plan.
Click here to watch the video. i
Click here for the transcript.
Welcome to the course HBC201 Research Methods for the Social and Behavioural Sciences, a 5
credit unit (CU) course.
This Study Guide will be your personal learning resource to take you through the course
learning journey. The guide is divided into two main sections – the Course Guide and
Study Units.
The Course Guide describes the structure for the entire course and provides you with an
overview of the Study Units. It serves as a roadmap of the different learning components
within the course. This Course Guide contains important information regarding the
course learning outcomes, learning materials and resources, assessment breakdown and
additional course information.
i
https://d2jifwt31jjehd.cloudfront.net/HBC201/IntroVideo/HBC201_Intro_Video.mp4
CG-2
HBC201
Course Guide
2. Course Description and Aims
This course provides you with necessary knowledge about both quantitative and
qualitative research methods that are commonly used in the social and behavioural
sciences. Through this course, you will develop the appreciation of the research process
in the field of social and behavioural sciences, the ability to critically evaluate the quality
of a research study, and most importantly, the awareness of the importance of conducting
ethical research.
Course Structure
This course is a 5-credit unit course presented over 6 weeks.
There are six Study Units in this course. The following provides an overview of each Study
Unit.
Study Unit 1 – Introduction to Social and Behavioural Science Research
In this unit, we will introduce social and behavioural science research from three aspects.
First, we will outline the definition and importance of social and behavioural science
research. Then, we will compare different types of philosophical foundations in the social
and behavioural science research. Lastly, we will discuss common ethical issues in the
field of social and behavioural sciences.
Study Unit 2 – Research Orientation, Design, and Measurement
In this unit, we will discuss three important aspects a researcher commonly considers
when planning for a research study. First, we will describe two research orientations, that
is, quantitative and qualitative research respectively. Secondly, we will introduce various
methodological matters a researcher usually considers in designing a study. Lastly, we
will discuss measurement issues in social and behavioural science research.
CG-3
HBC201
Course Guide
Study Unit 3 – Quantitative Research Methods I
In this unit, we will discuss quantitative research in greater details. First, we will discuss
major considerations researchers typically have in conducting quantitative studies.
Secondly, we will outline different types of quantitative research in social and behavioural
sciences.
Study Unit 4 –Quantitative Research Methods II
In this unit, we will focus on survey research. Survey is one of the most popular
quantitative methods used in social and behavioural science research. First, we will outline
the fundamentals of survey research. Secondly, we will describe concerns researchers
commonly have in conducting survey research. Lastly, we will discuss potential ethical
issues in survey research.
Study Unit 5 – Qualitative Research Methods I
In this unit, we will focus on qualitative research. First, we will describe major
considerations qualitative researchers typically have in conducting their studies.
Secondly, we will outline different types of interview commonly used in qualitative
studies. Lastly, we will discuss politics of research.
Study Unit 6 – Qualitative Research Methods II
In this unit, we will discuss qualitative research in greater details. First, we will outline
various types of qualitative studies. Secondly, we will focus on ethnography, which is one
of the most well-known qualitative research methods.
CG-4
HBC201
Course Guide
3. Learning Outcomes
Knowledge & Understanding (Theory Component)
By the end of this course, you should be able to:
• Explain the importance of social and behavioural science research to everyday life.
• Discuss the key elements involved in the research process, including problem
definition, measurement, sampling, research instruments, and data collection
methods.
• Examine the advantages and disadvantages of quantitative versus qualitative
methods.
• Recognise and appreciate the ethical issues within social and behavioural science
research.
Key Skills (Practical Component)
By the end of this course, you should be able to:
• Analyse existing published research and critique the research methods.
• Apply the relevant concepts, theories, and research designs to study social
phenomena.
• Propose an appropriate research design and method to address different research
questions.
CG-5
HBC201
Course Guide
4. Learning Material
All of the required learning materials for this course are drawn from the SAGE Research
Methods and the SAGE Research Methods Cases databases. The required learning
materials are listed for each chapter in each Study Unit. You can access the required
materials through the following steps:
1.
Click the URL link provided for a required reading.
2.
If an article is available from multiple sources, please click on the logo of “SAGE
research methods”.
3.
Click “Login” if prompted. Then click “Log in via your institution”. Fill in the
required fields and click “CONTINUE”.
4.
Click “OPENATHENS/SHIBBOLETH” if prompted. Fill in the required fields
and click “Sign in”.
5.
If you keep the browser open, you should be able to access other required
materials in the SAGE databases by simply clicking the corresponding URL links
in each Study Unit.
Required Textbook(s)
There is no required textbook for this course.
Other recommended study material (Optional)
The following learning materials may be required to complete the learning activities:
Website(s):
1.
APA citation guide available via SUSS Library (see https://
libguides.suss.edu.sg/apa).
2.
Google Forms (see https://www.google.com/forms/about/).
CG-6
HBC201
Course Guide
5. Assessment Overview
The overall assessment weighting for this course is as follows:
Assessment
Continuous Assessment
Description
Weight Allocation
Pre-Class Quiz
10%
Tutor-Marked Assignment
20%
(TMA01)
Tutor-Marked Assignment
20%
(TMA02)
Examination
End of Course Assessment
50%
(ECA)
TOTAL
100%
The following section provides important information regarding Assessments.
Continuous Assessment:
There will be continuous assessment in the form of one pre-class quiz and two tutormarked assignments (TMAs). In total, this continuous assessment will constitute 50
percent of overall student assessment for this course. The two assignments are compulsory
and are non-substitutable. It is imperative that you read through your Assignment
questions and submission instructions before embarking on your Assignment.
CG-7
HBC201
Course Guide
Examination:
All topics covered in the course outline will be examinable. To prepare for the exam,
you are advised to review Specimen or Past Year Exam Papers available on Learning
Management System.
Passing Mark:
To successfully pass the course, you must obtain a minimum passing mark of 40 percent
for each component. That is, students must obtain at least a mark of 40 percent for the
combined assessments and also at least a mark of 40 percent for the final exam. For
detailed information on the Course grading policy, please refer to The Student Handbook
(‘Award of Grades’ section under Assessment and Examination Regulations). The Student
Handbook is available from the Student Portal.
Non-graded Learning Activities:
Activities for the purpose of self-learning are present in each study unit. These learning
activities are meant to enable you to assess your understanding and achievement of the
learning outcomes. The type of activities can be in the form of Formative Assessment,
Quiz, Review Questions, Application-Based Questions or similar. You are expected to
complete the suggested activities either independently and/or in groups.
CG-8
HBC201
Course Guide
6. Course Schedule
To help monitor your study progress, you should pay special attention to your
Course Schedule. It contains study unit related activities including Assignments, Selfassessments, and Examinations. Please refer to the Course Timetable in the Student Portal
for the updated Course Schedule.
Note: You should always make it a point to check the Student Portal for any
announcements and latest updates.
CG-9
HBC201
Course Guide
7. Learning Mode
The learning process for this course is structured along the following lines of learning:
a.
Self-study guided by the study guide units. Independent study will require at
least 3 hours per week.
b.
Working on assignments, either individually or in groups.
c.
Classroom Seminar sessions (3 hours each session, 6 sessions in total).
iStudyGuide
You may be viewing the iStudyGuide version, which is the mobile version of the
Study Guide. The iStudyGuide is developed to enhance your learning experience with
interactive learning activities and engaging multimedia. Depending on the reader you are
using to view the iStudyGuide, you will be able to personalise your learning with digital
bookmarks, note-taking and highlight sections of the guide.
Interaction with Instructor and Fellow Students
Although flexible learning – learning at your own pace, space and time – is a hallmark
at SUSS, you are encouraged to engage your instructor and fellow students in online
discussion forums. Sharing of ideas through meaningful debates will help broaden your
learning and crystallise your thinking.
Academic Integrity
As a student of SUSS, it is expected that you adhere to the academic standards stipulated
in The Student Handbook, which contains important information regarding academic
policies, academic integrity and course administration. It is necessary that you read and
understand the information stipulated in the Student Handbook, prior to embarking on
the course.
CG-10
Study
Unit
Introduction to Social and
Behavioural Science Research
1
HBC201
Introduction to Social and Behavioural Science Research
Learning Outcomes
By the end of this unit, you should be able to:
1.
Describe key components in the process of social and behavioural science
research.
2.
Address the importance of social and behavioural science research in the
scientific community as well as amongst the general population.
3.
Compare different types of philosophical positions underlying quantitative and
qualitative research.
4.
Compare different types of logics of enquiry, particularly induction and
deduction.
5.
Discuss key ethical issues concerning research participants.
6.
Discuss key ethical issues concerning the reporting of research findings and the
protection of intellectual property rights.
SU1-2
HBC201
Introduction to Social and Behavioural Science Research
Overview
I
n this unit, we will introduce social and behavioural science research from three
aspects. We will commence by outlining the definition and importance of social and
behavioural science research. We will then proceed to compare the different types of
philosophical foundations in the social and behavioural science research. Finally, we will
discuss common ethical issues in the field of social and behavioural sciences.
SU1-3
HBC201
Introduction to Social and Behavioural Science Research
Chapter 1: Research: Why It Matters
1.1 What is Social and Behavioural Science Research?
Lesson Recording
Fundamentals of Social and Behavioural Science Research
Social science research studies “experiences, events, and facts in social reality”(Neuman,
2014, p.8). Examples of social science disciplines include anthropology, psychology,
sociology, political science, and economics. Knowledge from basic social science research
is commonly applied to research in related applied fields such as criminal justice,
education, social work, and so forth (Neuman, 2014).
Among social science research, behavioural sciences focus on the study of human
behaviour. Psychology is one of the major disciplines in the field of social sciences that
involves behavioural research. In addition, studies of voting behaviour in political science
or studies of consumer behaviour in economics, are also examples of behavioural research
(Lewis-Beck, 2011).
A social science study typically involves several key elements, as listed below. It is
important to note that the sequencing of these elements may vary across studies with
respect to the strategies or methodologies that researchers adopt (Bryman, 2012).
• Philosophy of research: Philosophical choices, such as the form of reasoning, are
involved in the process of a research study, whether a researcher realises it or not. In
the next chapter, we will discuss philosophical issues pertaining to a research study.
• Literature review: A research study typically starts with a general topic or issue
that a researcher finds interesting or important to investigate. Through reviewing
relevant literature, a researcher obtains critical information about past research on
SU1-4
HBC201
Introduction to Social and Behavioural Science Research
the topic of interest, such as findings by previous researchers on the topic, strategies
used by previous researchers to study the topic, etc. (Byrne, 2017a).
Read
Byrne, D. (2017a). Reviewing the literature. Project Planner. SAGE Research Methods
http://methods.sagepub.com/project-planner/reviewing-the-literature
• Research question: Five types of questions are commonly used to frame a specific
research question based on the general topic of interest to a researcher: Why? When?
Who? How? Where? (Byrne, 2017b)
Read
Byrne, D. (2017b). Types of research questions: Why? When? Who? How? Where? Project
Planner. SAGE Research Methods
http://methods.sagepub.com/project-planner/developing-a-researchable-question
• Research design: This element concerns the design of a study to address a research
question. In Study Unit 2, we will discuss common issues a researcher will
encounter in the stage of research design, such as data collection methods, time
points for data collection, etc.
• Research ethics: Conducting and reporting research ethically is critical. A researcher
is obligated to ensure ethical conduct through the entire research process. Ethical
issues common in social and behavioural science research will be discussed in
Chapter 3 of this study unit.
SU1-5
HBC201
Introduction to Social and Behavioural Science Research
• Data collection: This element concerns the securing of data that are suitable to
answer a research question. Study Units 3 to 6 will discuss the choices and issues
concerning data collection.
• Data analysis, interpretation of results, and dissemination of research findings: We
will touch on some selected concepts related to these elements as it is beyond the
scope of this course to cover a more comprehensive discussion of data analysis,
interpretation of analysis results, or dissemination of research findings.
1.2 Importance of Social and Behavioural Science Research
Researchers in the field of social and behavioural sciences adopt rigorous methods and
procedures to study the social reality. Findings of social and behavioural science research
can benefit our society in many ways, (Byrne, 2017c), for example:
• Enhance our knowledge about the social reality we live with, such as personal
identity, human behaviours, and so forth.
• Suggest a solution to address an issue.
• Evaluate the effectiveness of a proposed solution in solving a given problem.
Read
Byrne, D. (2017c). Why do research? Project Planner. SAGE Research Methods
http://methods.sagepub.com/project-planner/philosophy-of-research
SU1-6
HBC201
Introduction to Social and Behavioural Science Research
Case Study: Occupational Commitment
Clements, A. J. (2014). A mixed-methods approach to occupational commitment
in student nurses. SAGE Research Methods Cases. https://
doi.org/10.4135/978144627305013508503
Retaining qualified nurses and social workers in their professions has been an issue faced
by various countries. In this case study, the researcher described a series of research efforts
to investigate the commitment of students in the fields of social work and healthcare
toward their chosen professions in the United Kingdom. The researcher explained the
philosophy, design, and implementation of these research studies clearly. We will discuss
several of the topics the researcher described in this course. For now, please focus on the
following (Clements, 2014):
• Description of how these research studies were motivated and informed by
personal experience, literature, and practical issues in the studied fields [see
Context section].
• Illustration of how the researcher planned a series of studies to investigate the issue
of interest (i.e. occupational commitment of students in care professions, with a
special focus on student nurses) and his/her aim to further develop an intervention
to address the studied issue [see Project Overview section].
SU1-7
HBC201
Introduction to Social and Behavioural Science Research
Case Study: Cognitive Training Research
Sim, S. K. Y., & Chee, C. S. (2018). Implementing cognitive training research in older adults:
Randomized controlled trial, ground issues, and considerations. SAGE Research
Methods Cases. https://doi.org/10.4135/9781526435231
The proportion of the elderly in the overall population has been increasing over the years
in many countries. As such, there is an increasing need to develop cognitive training tools
that can help to slow down the possible cognitive decline due to ageing. In this case study,
the researchers described their efforts to evaluate a cognitive training programme for the
elderly in Singapore (Sim & Chee, 2018). You will encounter some technical terms (such as
randomized controlled trial) in this case study. Just keep calm and continue reading. We
will learn more about these technical aspects in subsequent study units. For now, please
focus on the following:
• The cognitive training programme under study is adapted from a programme
that is originally developed in Japan. The researchers described the importance of
modifying the programme and evaluating its efficacy in the Singapore context [see
Overview and Context section].
• The researchers’ discussion of the challenges and lessons they have learnt in
conducting their research [see Practical Lessons Learned section].
SU1-8
HBC201
Introduction to Social and Behavioural Science Research
Case Study: Impact of the Olweus Bullying Prevention Programme
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases. https://
doi.org/10.4135/978144627305013516576
School bullying can cause serious suffering and has received more attention in recent
years. The goal of the Olweus Bullying Prevention Programme is to develop a
positive environment and reduce violence in school. In this case study, the researcher
described a multi-year effort to evaluate the effectiveness of this programme, which was
originally developed and tested in Norway, to reduce bullying in various schools in the
United States. We will learn more about some of the methodological matters (such as
triangulation) the researcher described in subsequent study units. For now, please pay
special attention to the following (Kuykendall, 2014):
• Discussion of why it is important to evaluate the effectiveness of a programme [see
Introduction section].
• Information about theoretical and practical considerations in evaluation research,
also the description of the evaluation project design [see Research Design section].
Reflect 1.1
1.
How does social and behavioural science research influence your life?
2.
In the case studies presented above, how did these researchers identify and
form the research questions for investigation purpose?
3.
Can you think of a research topic that you are interested to study?
SU1-9
HBC201
Introduction to Social and Behavioural Science Research
Chapter 2: From Philosophical Foundations to Real-Life
Applications
It is inevitable that a researcher in social sciences will encounter the need to make
philosophical choices in the process of conducting a research study, whether he or she
explicitly acknowledges it or not. As Blaikie (2011, para. 3) suggested, the philosophy of
social sciences is mainly involved in the following aspects of research:
• Different types of ontological and epistemological assumptions that serve as the
foundation of a research study.
• Different “logics of enquiry” that can be adopted to answer a research question.
• Different methods that can be used for data collection and analysis.
This chapter discusses the first two aspects of philosophical choices; we will first discuss
common ontological and epistemological assumptions, and then discuss logics of enquiry
widely used in social science research. Different data collection methods will be discussed
in subsequent study units.
Read
Blaikie, N. (2011). Philosophy of social science. In M. S. Lewis-Beck,
A. Bryman, & T. Futing Liao (Eds.), The SAGE encyclopedia of
social science research methods. SAGE Publications, Inc. https://
doi.org/10.4135/9781412950589.n711
SU1-10
HBC201
Introduction to Social and Behavioural Science Research
2.1 Ontology and Epistemology
2.1.1 Definitions of Ontology and Epistemology
Every single research study in social sciences is based upon certain assumptions in both
ontology and epistemology. Table 1.1 compares the focus of ontology and epistemology.
Table 1.1 Focus of Ontology and Epistemology
Ontology
Epistemology
What is the nature of social entities?
What is, or should be, regarded as
acceptable knowledge in a discipline?
(Source: Barron, 2011)
2.1.2 Philosophical Positions Underlying Two Research Orientations:
Quantitative and Qualitative Research
A variety of philosophical terms have been proposed to describe the ontological and
epistemological positions in social sciences. It is beyond the scope of this course to
introduce all the philosophical positions. Instead, we will focus on the major philosophical
terms that are most commonly referred to when researchers discuss the orientations of
their research, i.e. quantitative or qualitative research.
Note that philosophical positions underpinning each research orientation will be
described based on its nature, and will not be classified into an ontological or
epistemological position distinctively for the following reasons:
• Different researchers may use the same term in different ways in reference to an
ontological or an epistemological position. For example, Barron (2011) used the
term “positivism” in a discussion of one type of ontological principles, while Bryman
(2012) used the term “positivism” in reference to one type of epistemological positions
(in recognition that the use of the term “positivism” may vary across researchers).
SU1-11
HBC201
Introduction to Social and Behavioural Science Research
• Although a term (e.g. positivism) may be used by researchers differently, the term
is typically associated with a set of definitive features. For example, objective is
a definitive feature of positivism in Barron’s discussion (Barron, 2011). Bryman
(2012, p.28) describes positivism as “an epistemological position that advocates the
application of the methods of the natural sciences to the study of social reality and
beyond” and objective is presumably the nature of scientific methods. While being
used in different manners, the term “positivism” is essentially associated with the
feature of objective.
Figure 1.1 Primary Philosophical Positions within Quantitative and Qualitative
Research
(Sources: Barron, 2011; Bryman, 2012; and Staller, 2012)
Figure 1.1 shows philosophical positions commonly referred to within the quantitative
and qualitative research. A general comparison of quantitative and qualitative research
orientations will be discussed in detail in Study Unit 2. In this section, we will only focus
on their philosophical positions:
• Objectivism: Objectivism considers society as an entity that is independent of social
actors in society (Barron, 2011). For example, an organisation in a society has its
own missions, rules, hierarchy, and so on. People employed in an organisation
must follow the rules and do the jobs they are hired for. From the perspective
SU1-12
HBC201
Introduction to Social and Behavioural Science Research
of objectivism, an organisation can be viewed as an entity that is separated from
people comprising it (Bryman, 2012).
• Constructivism: Constructivism rejects the idea that researchers can discover truths
about phenomena of interest objectively. Instead, constructivism suggests that
reality is constructed through social interaction and is continually revised (Bryman,
2012; Staller, 2012).
• Positivism: Positivism asserts that there is a stable reality and researchers should
study the reality in an objective way (Staller, 2012).
• Interpretivism: Interpretivism is associated with several related philosophical
traditions. Taking symbolic interactionism as an example, it asserts that “reality is
not stable and pre-existing but rather, it is socially constructed and given meaning
only through ongoing interactions.” (Staller, 2012, para. 4).
You may feel overwhelmed in the beginning if you are encountering these philosophical
terms for the first time. Not to worry though as you will get more familiar with these
philosophies and research traditions that underpin this course as you learn more about
social science research. Keep calm and carry on with your learning journey!
Read
1.
Barron,
L.
(2011).
Ontology.
In
V.
Jupp
(Ed.),
The
SAGE
dictionary of social research methods. SAGE Publications, Ltd. https://
doi.org/10.4135/9780857020116.n134
2.
Read the Positioning Qualitative Inquiry section in Staller (2012):
Staller, K. M. (2012). Qualitative research. In N. J. Salkind (Ed.),
Encyclopedia
of
research
design.
SAGE
doi.org/10.4135/9781412961288.n350
SU1-13
Publications,
Inc.
https://
HBC201
Introduction to Social and Behavioural Science Research
2.2 Logics of Enquiry
There are three major forms of reasoning: deduction, induction, and abduction (Shank,
2012). Deduction and induction are the most common forms of reasoning used and
referred to in social science research. Abduction is considered as “broadly inductive in
approach” (Bryman, 2012, p. 401) and it is distinguished mainly by its basis on the views
of research participants (Bryman, 2012). Deduction and induction are of particular interest
here.
We will first explain deduction, followed by induction. We will also discuss the use of both
deductive and inductive reasoning in the research practice.
2.2.1 Deduction
A researcher adopting the deductive reasoning makes an implication from a premise
(Shank, 2012). In social sciences, a premise is typically a theory, and an implication
deduced from a theory is called a hypothesis. Figure 1.2 shows the relationship between
theory and hypothesis in the deductive reasoning.
Figure 1.2 Deduction
(Source: Shank, 2012)
The so called “hypothetico-deductive method” is commonly used in social sciences
(particularly quantitative research studies) to test a hypothesis and the corresponding
SU1-14
HBC201
Introduction to Social and Behavioural Science Research
theory from which the hypothesis is deduced. As shown in Figure 1.3, the process of the
hypothetico-deductive method typically involves the following steps (Shank, 2012):
• A hypothesis is deduced from an existing theory.
• With the hypothesis in mind, a researcher proceeds to collect empirical data.
• The researcher uses empirical data to examine the hypothesis of interest.
• The finding from empirical data can either (1) support the hypothesis and provide
empirical evidence for the existing theory or, (2) reject the hypothesis and suggest
implications for theory revision.
Figure 1.3 Hypothetico-Deductive Method
(Source: Shank, 2012)
SU1-15
HBC201
Introduction to Social and Behavioural Science Research
Read
Shank, G. (2012). Deduction. In L. M. Given (Ed.), The SAGE encyclopedia
of qualitative research methods. SAGE Publications, Inc. https://
doi.org/10.4135/9781412963909.n105
2.2.2 Induction
In contrast to the use of the deductive reasoning, a researcher using the inductive
reasoning generalises from a specific set of empirical data to a general statement, such as
a theory (Fox, 2012). Figure 1.4 shows the relationship between theory and empirical data
in inductive reasoning. Induction is an important form of reasoning used in qualitative
research studies to establish or extend a theory (Fox, 2012).
Figure 1.4 Induction
(Source: Fox, 2012)
SU1-16
HBC201
Introduction to Social and Behavioural Science Research
Read
Fox, N. J. (2012). Induction. In L. M. Given (Ed.), The SAGE encyclopedia
of qualitative research methods. SAGE Publications, Inc. https://
doi.org/10.4135/9781412963909.n212
2.2.3 Combining Deductive and Inductive Reasoning
As Fox (2012) discussed, philosophers have argued that all theories must be considered
as tentative. A theory based on observations (e.g. all observed birds can fly) can be
overturned by a single contradictory observation (e.g. a flightless bird is observed).
As such, practitioners or researchers should exercise caution in using research findings
to explain a phenomenon or event, and should always be mindful of alternative
explanations. Particularly, practitioners or researchers should bear in mind that findings
of research conducted in one context may not be generalised to another context, as an
event or phenomenon in social sciences may be influenced by many different factors.
While deductive and inductive reasoning appear to be opposite forms of reasoning, these
two types of reasoning can both be incorporated in the research practice to advance the
knowledge of social sciences. Specifically, inductive reasoning can be used to develop new
theories based on empirical data. Deductive reasoning can be used to deduce hypotheses
from these new theories for testing, findings of which can then inform the further
development of the theories (Fox, 2012; Shank, 2012).
SU1-17
HBC201
Introduction to Social and Behavioural Science Research
Activity 1.1
Clements, A. J. (2014). A mixed-methods approach to occupational
commitment in student nurses. SAGE Research Methods Cases. https://
doi.org/10.4135/978144627305013508503
As described in Chapter 1, this case study discussed a series of research efforts
to investigate the commitment of students toward their professions in the United
Kingdom, with a special focus on student nurses (Clements, 2014). Please discuss the
following questions:
1.
How did the researcher use qualitative as well as quantitative methods to
investigate the research questions of interest?
2.
What are the philosophical foundations underlying these research efforts?
SU1-18
HBC201
Introduction to Social and Behavioural Science Research
Chapter 3: Research Ethics
Lesson Recording
Research Ethics
Professional associations in social sciences have formulated codes of ethics to guide their
members in respective disciplines on ethically adequate practices. For an example, refer to
the website of American Psychological Association (https://www.apa.org/ethics/code/)
for its code of ethics (American Psychological Association, 2017).
A research organisation, university, or college, typically has its own institutional review
board (IRB) or research ethics committee (REC) to oversee ethical aspects of research
projects undertaken by members of the organisation. Members of an organisation should
always consult their IRB or REC, if applicable, to determine any requirement to obtain an
approval for their proposed research before conducting any research activity.
3.1 Ethical Issues Concerning Research Participants
3.1.1 Historical Background on Research Ethics
The advocate for establishment of ethical principles to ensure proper protection of
research participants resulted from inhumane experiments and research, as evidenced
from history.
• Nazi physicians and scientists had conducted cruel biomedical experiments on
concentration camp prisoners. During the Nuremberg War Crime Trials, the
Nuremberg Code was formulated after World War II as a response to these
inhumane experiments, which specifies the rights of human subjects in research.
SU1-19
HBC201
Introduction to Social and Behavioural Science Research
While the focus of the Nuremberg Code is on medical research, it also informs
ethical principles for social research (Neuman, 2014).
• The Tuskegee Syphilis Study, which took place in the United States, is another
infamous research project that seriously violated human rights. In this study,
researchers used disadvantaged African American men in Alabama to investigate
symptoms that appeared in the untreated progress of the disease. This research
started in 1929 and only ended in 1972. Subjects in this study were deprived
of treatment and suffered from syphilis, long after the treatment of the
disease was already available in order to allow researchers to continue their
study (Neuman, 2014). The revelation of this unethical study prompted the
establishment of the National Commission for the Protection of Human Subjects
of Biomedical and Behavioural Research in the United States, which published the
Belmont Report (https://www.hhs.gov/ohrp/regulations-and-policy/belmontreport/index.html) to identify the ethical principles for the protection of human
subjects in research.
3.1.2 General Ethical Principles
The following three general principles form the common basis of ethical guidelines on
the protection of human subjects in research: respect, beneficence, and justice (Fisher &
Anushko, 2012). Table 1.2 summarises these three principles as described in the Belmont
Report (National Commission for the Protection of Human Subjects of Biomedical and
Behavioural Research, 1979):
SU1-20
HBC201
Introduction to Social and Behavioural Science Research
Table 1.2 General Ethical Principles
Principle
Respect for Persons
Description
• A researcher is obligated to acknowledge a person’s
autonomy. A researcher should provide a potential
research subject with adequate information about the
research of interest, and the subject should participate in
the research voluntarily.
• A researcher is obligated to protect people who are
immature (e.g. young children) or incapacitated (e.g.
people who lose the capacity for self-determination,
entirely or partly, due to illness).
Beneficence
• A researcher should not harm one person regardless of
the potential benefits to others.
• A researcher is obligated to maximise possible benefits
and minimise potential harm that may result from a
research study.
Justice
• A researcher is obligated to ensure that the benefits of
research are received by people fairly and the burdens
of research are not imposed upon people unduly (e.g.
researchers should not target certain groups of people
when recruiting research subjects simply because of
their compromised positions).
(Source: Belmont Report (National Commission for the Protection of Human Subjects of Biomedical and
Behavioural Research, 1979)
3.1.3 Common Ethical Issues Concerning Research Participants
In this section, we will focus on common ethical issues concerning research participants
in social research. These are: conflict of interest, informed consent, selection of research
SU1-21
HBC201
Introduction to Social and Behavioural Science Research
participants and measures, use of incentives, and anonymity and confidentiality. A
researcher should always consult guidelines produced by relevant organisations and
professional associations to ensure ethically proper conduct of social research (Fisher &
Anushko, 2012; Neuman, 2014).
Table 1.3 Common Ethical Issues Concerning Research Participants
Ethical Issues
Conflict of
Interest
Description
• A researcher in social research should strive to avoid any
potential conflict of interest that may impair the objectivity
of research. Below are examples of possible conflicts of
interest:
◦ A researcher has financial interests in, or receives
gifts from, an organisation that will be affected by the
findings of his or her research.
◦ The organisation that sponsors a research study
favours certain findings, which may place pressure
on the researcher conducting the study.
◦ A researcher or his/her affiliated institution will hold
the patent for the research product.
◦ A researcher is reviewing a grant application or
manuscript submission from a competitor.
• A researcher is typically obligated to disclose any potential
conflict of interest in a grant application or manuscript
submission.
• It is critical to note that a potential conflict of interest is
not always unethical or avoidable. The obligation of a
researcher is to be careful of any potential conflict, avoid
a conflict if possible, and honestly disclose and effectively
manage a conflict when necessary.
SU1-22
HBC201
Ethical Issues
Informed
Consent
Introduction to Social and Behavioural Science Research
Description
• A researcher should provide prospective participants with
all the necessary information that may influence their
decisions to participate in the research. Such information
includes, and is not limited to, the following:
◦ The research purpose and plan (e.g. procedure,
timeline, etc.).
◦ The right to decline to participate, the right to
withdraw from participation, and the corresponding
consequences.
◦ Any potential risk or discomfort, or in contrast, any
potential benefits to participants or the public.
◦ Issues regarding confidentiality.
◦ Any incentive for participation in the study
◦ Contact persons for the research participants to direct
their enquiries.
◦ Additional information to address any question a
prospective participant might have.
• Obtaining informed consent in qualitative research can be
complicated. For example, unanticipated information that
is beyond the scope of the initial consent may emerge in
the progress of the research. One possible solution is for a
qualitative researcher to implement a re-consent strategy if
needed.
• Researchers may use data that are collected and archived
by third parties. The purpose of such an archival research
study or secondary analysis may not be consistent with the
original information provided to participants when data
were initially collected. In such an instance, the validity
SU1-23
HBC201
Ethical Issues
Introduction to Social and Behavioural Science Research
Description
of informed consent to the ongoing secondary analyses
may be questionable. One possible solution is to set up an
advisory board to evaluate whether each new secondary
analysis may violate the initial intent of the informed
consent obtained originally.
• A researcher may consider using deceptive methods
in research, whereby participants will be intentionally
misinformed about the purpose or the procedure of the
study in order to yield participants' authentic responses
in research activities (e.g. experiments). Deception should
be used with great caution in research, as no follow-up
procedures (e.g. debriefing) can completely address the
threat to participants’ right to make a truly informed
decision to participate in the research study.
Selection
• A researcher should ensure that participants are recruited
of Research
with respect to clearly defined population characteristics
Participants
and avoid the following:
and Measures
◦ Intentionally (or unintentionally) excluding certain
groups of people from participation in research.
◦ Inappropriately generalising research findings from
one population to other populations.
• A researcher should select measures (e.g. a survey or an
assessment) that are suitable for studying the population
of interest. For example, items on a test that is designed
and validated for one population may not work well for a
different population. In such a situation, test results from
these two populations cannot be compared fairly.
SU1-24
HBC201
Ethical Issues
Use of
Incentives
Introduction to Social and Behavioural Science Research
Description
• A researcher may provide non-coercive incentives for
participants under some conditions. It is important
that incentives do not harm a prospective participant’s
autonomy in deciding whether to take part in the study,
or lead to unfair burdens being imposed on economically
disadvantaged people.
Anonymity and
Confidentiality
• A researcher is obligated to protect the privacy of research
participants, specifically:
◦ Anonymity: A researcher should keep research
participants anonymous and protect their identity
from disclosure. For example, a researcher may
discard participants’ personal information (e.g. name)
and assign each participant a unique code instead to
ensure their anonymity.
◦ Confidentiality: A researcher should keep
participants’ data in confidence and avoid releasing
any information that may allow someone to link
specific research data to individual participants. For
example, a researcher can report aggregated results
(e.g. proportion, mean, etc.).
(Sources: Fisher and Anushko, 2012; and Neuman, 2014)
SU1-25
HBC201
Introduction to Social and Behavioural Science Research
Read
Fisher, C. B., & Anushko, A. E. (2012). Research ethics in social science.
In P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The SAGE
handbook of social research methods. SAGE Publications Ltd. https://
doi.org/10.4135/9781446212165.n8
3.2 Ethical Issues Concerning Reporting of Research Findings and
Protection of Intellectual Property Rights
Institutional or professional guidelines typically specify a researcher’s ethical obligation
concerning the reporting of research findings and the protection of intellectual property
rights as well. As an example, Table 1.4 presents common ethical issues concerning
these aspects, as documented in the American Psychological Association’s ethics code
(American Psychological Association, 2017). See their website (https://www.apa.org/
ethics/code/) for details.
Table 1.4 Common Ethical Issues Concerning Reporting of Research Findings and Protection of Intellectual
Property Rights
Ethical Issues
Reporting of Research
Findings
Description
• Psychologists should not fake data (Standard 8.10
(a)).
• “If psychologists discover significant errors in their
published data, they take reasonable steps to correct
such errors in a correction, retraction, erratum, or
other appropriate publication means.” (Standard 8.10
(b)).
SU1-26
HBC201
Introduction to Social and Behavioural Science Research
Ethical Issues
Description
• Psychologists should avoid plagiarism (see Standard
8.11).
• Psychologists should avoid duplicate publication of
data (see Standard 8.13).
• Psychologists may share research data upon the
request of other professionals who intend to verify
the research findings through reanalysis under
some conditions (e.g. the confidentiality of research
participants is protected) (see Standard 8.14).
Protection
Intellectual
Rights
of
Property
• Publication credit (such as authorship credit) should
reflect the relative scientific contribution of each
individual involved, instead of their relative status in
an institution (see Standard 8.12 for details).
• Reviewers should protect the confidentiality of
information in materials submitted for review (e.g.
manuscript, research proposal, grant application,
etc.) and respect the related proprietary rights (see
Standard 8.15).
(Source: American Psychological Association, 2017)
Different professional associations have developed guidelines that instruct scholars
on how to report their research findings. For example, the American Psychological
Association has developed a scientific writing style (commonly referred to as “APA
style”), which is widely used in the field of social sciences. You can refer to the Publication
Manual of the American Psychological Association (American Psychological Association,
2020) for specific writing, citation, or formatting guidelines in the APA style. You may also
refer to the APA style website for more information: https://apastyle.apa.org/.
SU1-27
HBC201
Introduction to Social and Behavioural Science Research
Activity 1.2
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases. https://
doi.org/10.4135/978144627305013516576
As described in Chapter 1, this case study described a research project that evaluated
the effectiveness of a school programme on bullying prevention (Kuykendall, 2014).
Please discuss the following questions:
1.
What were the potential ethical issues in this study?
2.
How did the researcher and schools participating in this project address
these ethical issues?
SU1-28
HBC201
Introduction to Social and Behavioural Science Research
Summary
In this unit, we have covered three fundamental topics in the social and behavioural
science research as follows:
• In Chapter 1, we first outlined the major elements of a social and behavioural
science research study. Subsequently, we discussed the benefits of research findings
in the field of social and behavioural science to our society.
• In Chapter 2, we first compared different types of philosophical positions
underlying quantitative and qualitative research. This was followed by a
comparison of the different types of logics of enquiry, specifically induction and
deduction.
• In Chapter 3, we discussed common ethical issues related to different aspects of
a research study, such as research participants, reporting and dissemination of
research findings, and so on.
SU1-29
HBC201
Introduction to Social and Behavioural Science Research
Formative Assessment
1.
Which of the following statements about social science research is correct?
a. Only basic research is conducted in the field of social sciences.
b. Behavioural science studies human behaviour. A study that investigates
Singapore citizens’ voting behaviour is an example of behavioural research.
c. Philosophy of research is only relevant to researchers who conduct basic
research.
d. Research ethics is mainly considered by researchers at the process of data
collection.
2.
Which of the following is the benefit of conducting social science research?
a. Find a possible solution to tackle a social issue.
b. Broaden our knowledge of our society.
c. Evaluate whether a proposed solution solves a problem effectively or not.
d. All of the above.
3.
According to Blaikie (2011), which of the following involves philosophy of research?
a. Ontological and epistemological assumptions underlying a given study.
b. Logics of enquiry a researcher chooses to investigate a research question.
c. Data collection and analysis methods a researcher chooses for his or her study.
d. All of the above.
4.
The question “What is the nature of social entities?” is concerned with _________.
a. Ontology
b. Epistemology
c. Induction
d. Deduction
SU1-30
HBC201
5.
Introduction to Social and Behavioural Science Research
Which of the following are philosophical positions underlying quantitative research?
a. Constructivism, Interpretivism
b. Objectivism, Interpretivism
c. Objectivism, Positivism
d. Constructivism, Positivism
6.
What is the form of reasoning whereby a researcher derives a hypothesis from an
existing theory?
a. Induction
b. Deduction
c. Ontology
d. Epistemology
7.
What is the typical procedure of a hypothetico-deductive method?
a. Theory → Hypothesis → Empirical data → Hypothesis accepted or rejected.
b. Empirical data → Hypothesis → Hypothesis accepted or rejected → Theory.
c. Empirical data → Theory
d. Theory → Hypothesis
8.
A researcher should ensure that a potential participant agrees to join a research study
voluntarily. This requirement reflects which of the following principles described in
the Belmont Report?
a. Beneficence
b. Justice
c. Respect for Persons
d. None of the above.
9.
Which of the following statements is incorrect?
SU1-31
HBC201
Introduction to Social and Behavioural Science Research
a. It is inappropriate for a researcher to generalise findings of a study to
populations that the study participants are not sampled from.
b. A researcher should disclose any conflict of interest honestly as needed (such
as in a grant application).
c. A researcher should protect the privacy of every research subject.
d. A researcher can decide the value of incentives freely within his or her own
research budget.
10. Which of the following statements is incorrect?
a. A researcher should consult the institutional review board (IRB) of the relevant
institution (such as the institution where the researcher works or studies)
before starting any research study.
b. Researchers’ academic status determines authorship credit for a paper they
work on together.
c. Plagiarism is unethical.
d. Reviewers should keep information in the documents under review (such as
grant applications, manuscripts, and so on) confidential.
SU1-32
HBC201
Introduction to Social and Behavioural Science Research
Solutions or Suggested Answers
Formative Assessment
1.
Which of the following statements about social science research is correct?
a.
Only basic research is conducted in the field of social sciences.
Incorrect. Social science research includes both basic and applied research.
b.
Behavioural science studies human behaviour. A study that investigates
Singapore citizens’ voting behaviour is an example of behavioural research.
Correct. See section 1.1.
c.
Philosophy of research is only relevant to researchers who conduct basic
research.
Incorrect. All research studies inevitably involve philosophical choices,
regardless of whether researchers explicitly recognise these choices or not.
d.
Research ethics is mainly considered by researchers at the process of data
collection.
Incorrect. Researchers should follow ethical principles and guidelines
throughout the entire process of conducting research and reporting research
findings.
2.
Which of the following is the benefit of conducting social science research?
a.
Find a possible solution to tackle a social issue.
Incorrect.
b.
Broaden our knowledge of our society.
Incorrect.
c.
Evaluate whether a proposed solution solves a problem effectively or not.
Incorrect.
SU1-33
HBC201
Introduction to Social and Behavioural Science Research
d.
All of the above.
Correct. See section 1.2.
3.
According to Blaikie (2011), which of the following involves philosophy of research?
a.
Ontological and epistemological assumptions underlying a given study.
Incorrect.
b.
Logics of enquiry a researcher chooses to investigate a research question.
Incorrect.
c.
Data collection and analysis methods a researcher chooses for his or her
study.
Incorrect.
d.
All of the above.
Correct. See Chapter 2.
4.
The question “What is the nature of social entities?” is concerned with _________.
a.
Ontology
Correct. See Table 1.1.
b.
Epistemology
Incorrect. Epistemology is concerned with what knowledge is considered
acceptable.
c.
Induction
Incorrect. Induction is a form of reasoning. See section 2.2.
d.
Deduction
Incorrect. Deduction is a form of reasoning. See section 2.2.
5.
Which of the following are philosophical positions underlying quantitative research?
a.
Constructivism, Interpretivism
SU1-34
HBC201
Introduction to Social and Behavioural Science Research
Incorrect. These are philosophical positions underlying qualitative research.
b.
Objectivism, Interpretivism
Incorrect. Objectivism is a philosophical position underlying quantitative
research. However, interpretivism is a philosophical position underlying
qualitative research.
c.
Objectivism, Positivism
Correct .
d.
Constructivism, Positivism
Incorrect. Positivism is a philosophical position underlying quantitative
research. However, constructivism is a philosophical position underlying
qualitative research.
6.
What is the form of reasoning whereby a researcher derives a hypothesis from an
existing theory?
a.
Induction
Incorrect. Via inductive reasoning, a researcher typically develops a theory
from empirical data instead.
b.
Deduction
Correct.
c.
Ontology
Incorrect. Ontology is concerned with the nature of social entities.
d.
Epistemology
Incorrect. Epistemology is concerned with what knowledge is considered
acceptable in a field.
7.
What is the typical procedure of a hypothetico-deductive method?
a.
Theory → Hypothesis → Empirical data → Hypothesis accepted or rejected.
SU1-35
HBC201
Introduction to Social and Behavioural Science Research
Correct.
b.
Empirical data → Hypothesis → Hypothesis accepted or rejected → Theory.
Incorrect. See Figure 1.3.
c.
Empirical data → Theory
Incorrect. This is inductive reasoning.
d.
Theory → Hypothesis
Incorrect. This is deductive reasoning.
8.
A researcher should ensure that a potential participant agrees to join a research study
voluntarily. This requirement reflects which of the following principles described in
the Belmont Report?
a.
Beneficence
Incorrect. See Table 1.2.
b.
Justice
Incorrect. See Table 1.2.
c.
Respect for Persons
Correct. See Table 1.2.
d.
None of the above.
Incorrect. See Table 1.2.
9.
Which of the following statements is incorrect?
a.
It is inappropriate for a researcher to generalise findings of a study to
populations that the study participants are not sampled from.
Incorrect. This is a Correct statement.
b.
A researcher should disclose any conflict of interest honestly as needed (such
as in a grant application).
SU1-36
HBC201
Introduction to Social and Behavioural Science Research
Incorrect. This is a Correct statement.
c.
A researcher should protect the privacy of every research subject.
Incorrect. This is a Correct statement.
d.
A researcher can decide the value of incentives freely within his or her own
research budget.
Correct. This is the Incorrect statement. An incentive with a strong appeal
raises concerns about a potential research subject’s autonomy in deciding
whether to participate in a study or not.
10. Which of the following statements is incorrect?
a.
A researcher should consult the institutional review board (IRB) of the
relevant institution (such as the institution where the researcher works or
studies) before starting any research study.
Incorrect. This is a Correct statement.
b.
Researchers’ academic status determines authorship credit for a paper they
work on together.
Correct. This is the Incorrect statement. Authorship credit is supposed to
reflect the level of scientific contribution of each researcher involved.
c.
Plagiarism is unethical.
Incorrect. This is a Correct statement.
d.
Reviewers should keep information in the documents under review (such as
grant applications, manuscripts, and so on) confidential.
Incorrect. This is a Correct statement.
SU1-37
HBC201
Introduction to Social and Behavioural Science Research
References
American Psychological Association. (2017). Ethical principles of psychologists and code of
conduct (2002, amended effective June 1, 2010, and January 1, 2017). Retrieved from
https://www.apa.org/ethics/code/
American Psychological Association. (2020). Publication manual of the American
Psychological Association (7th ed.). American Psychological Association.
Barron, L. (2011). Ontology. In V. Jupp (Ed.), The SAGE dictionary of social
research methods. SAGE Publications, Ltd. Retrieved from https://
doi.org/10.4135/9780857020116.n134
Blaikie, N. (2011). Philosophy of social science. In M. S. Lewis-Beck, A. Bryman, & T.
Futing Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n711
Bryman, A. (2012). Social research methods (4th ed.). Oxford University Press.
Byrne, D. (2017a). Reviewing the literature. Project Planner. SAGE Research Methods.
Retrieved from http://methods.sagepub.com/project-planner/reviewing-theliterature
Byrne, D. (2017b). Types of research questions: Why? When? Who? How? Where?
Project Planner. SAGE Research Methods. Retrieved from http://
methods.sagepub.com/project-planner/developing-a-researchable-question
Byrne, D. (2017c). Why do research? Project Planner. SAGE Research Methods. Retrieved
from http://methods.sagepub.com/project-planner/philosophy-of-research
Clements, A. J. (2014). A mixed-methods approach to occupational commitment
in student nurses. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/978144627305013508503
Fisher, C. B., & Anushko, A. E. (2012). Research ethics in social science. In
P. Alasuutari, L. Bickman, & J. Brannen (Eds.), The SAGE handbook of
SU1-38
HBC201
Introduction to Social and Behavioural Science Research
social research methods. SAGE Publications Ltd. Retrieved from https://
doi.org/10.4135/9781446212165.n8
Fox, N. J. (2012). Induction. In L. M. Given (Ed.), The SAGE encyclopedia of
qualitative research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963909.n212
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases. Retrieved from
https://doi.org/10.4135/978144627305013516576
Lewis-Beck, M. S. (2011). Behavioral sciences. In M. S. Lewis-Beck, A. Bryman, & T.
Futing Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n52
National Commission for the Protection of Human Subjects of Biomedical and
Behavioral Research. (1979, April 18). The Belmont Report. HHS.Gov. Retrieved
from https://www.hhs.gov/ohrp/regulations-and-policy/belmont-report
Neuman, W. L. (2014). Social research methods: Qualitative and quantitative approaches (7th
ed.). Pearson Education Limited.
Shank, G. (2012). Deduction. In L. M. Given (Ed.), The SAGE encyclopedia of
qualitative research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963909.n105
Sim, S. K. Y., & Chee, C. S. (2018). Implementing cognitive training research in older
adults: Randomized controlled trial, ground issues, and considerations. SAGE
Research Methods Cases. Retrieved from https://doi.org/10.4135/9781526435231
Staller, K. M. (2012). Qualitative research. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n350
SU1-39
HBC201
Introduction to Social and Behavioural Science Research
SU1-40
Study
Unit
2
Research Orientation, Design, and
Measurement
HBC201
Research Orientation, Design, and Measurement
Learning Outcomes
By the end of this unit, you should be able to:
1.
Describe and compare characteristics of quantitative and qualitative research.
2.
Discuss the use of quantitative, qualitative, and triangulation approach in
research design.
3.
Discuss the use of cross-sectional and longitudinal research design.
4.
Describe and compare different types of threats to internal and external validity
in research design.
5.
Explain different levels of measurement.
6.
Differentiate different types of validity evidence for a measure.
7.
Differentiate different types of reliability evidence for a measure.
SU2-2
HBC201
Research Orientation, Design, and Measurement
Overview
I
n this unit, we will discuss three important aspects that a researcher would
commonly consider when planning a research study. First, we will describe two
research orientations – quantitative and qualitative research respectively. Thereafter, we
will introduce various methodological matters that form a regular part of a researcher’s
considerations when designing a study. Finally, we will discuss measurement issues in
social and behavioural science research.
SU2-3
HBC201
Research Orientation, Design, and Measurement
Chapter 1: Research Orientations: Numbers or Words
1.1 Quantitative Research
Quantitative research studies typically adopt the deductive reasoning approach as
described in Study Unit 1. This is where research questions are usually driven by
existing theories, specific hypotheses and variables that are formulated accordingly. Data
are collected subsequently to test the hypotheses. Figure 2.1 shows the key stages a
quantitative study generally has, each of which will be explained below (Kraska, 2012).
Figure 2.1 Key Stages of a Quantitative Study
(Source: Kraska, 2012)
SU2-4
HBC201
Research Orientation, Design, and Measurement
1.1.1 Research Question
A quantitative research study commonly starts with a general topic, from which a
researcher then develops one or more specific research questions that are commonly
informed by existing theories. For example, a researcher may be interested to study
whether five-year-old children enrolled in an innovative mathematics programme in
Singapore demonstrate better mathematical ability than their counterparts who are not
enrolled in the programme.
1.1.2 Operational Definition
A researcher may refer to a concept or construct (e.g. mathematics ability) in framing a
research question. As a construct may be defined and measured differently, it is critical for
a quantitative researcher to provide an operational definition for each construct studied.
The following two steps are usually needed (Mueller, 2011):
• Conceptualisation: A researcher should clearly define a construct to allow others
to understand what the researcher means in using the word. For example, the
researcher may define the concept “mathematics ability” in the research question
as a specific set of mathematical concepts or skills.
• Operationalisation: In addition to a definition, a researcher typically specifies how
a construct will be measured in a quantitative study. For example, a researcher
may measure a child’s mathematics ability as his or her score on the Test of Early
Mathematics Ability-Third Edition (TEMA-3).
From this stage onward, a researcher will move from abstract concepts to empirical
variables (Mueller, 2011). A “variable” is “a characteristic that can vary in value” among
people (Agresti & Finlay, 1997, p. 13). A variable may be defined as one of the following
(Agresti & Finlay, 1997):
• Dependent variable: The outcome variable of interest is called the dependent
variable. In the example of comparing children’s mathematics ability, a child’s
TEMA-3 test score is the dependent variable.
SU2-5
HBC201
Research Orientation, Design, and Measurement
• Independent variable: The variable that is believed to influence a person’s value
on the outcome variable is called the independent variable. In the example of
comparing mathematics ability between children who are enrolled in an innovative
mathematics programme and those who are not, a child’s group membership is the
independent variable.
• Control variable: When studying the relationship between two variables, a
researcher may want to remove the influence of a third variable, which is called
a control variable. For example, in examining the association between children’s
mathematics ability and enrolment in an innovative mathematics programme, the
researcher may want to take a child’s gender into account. In this case, a child’s
gender is a control variable.
1.1.3 Formulation of Hypothesis
A hypothesis can be formulated for each research question; it specifies the researcher’s
expectation about the relationship between studied variables. A hypothesis can be
specified in the null or alternative form. An alternative hypothesis can be either
nondirectional (i.e. a researcher does not specify the predicted direction of a relationship)
or directional (i.e. a researcher specifies the predicted direction of a relationship). In
practice, researchers typically specify a hypothesis in the null form. Table 2.1 presents
examples of hypotheses in different forms.
SU2-6
HBC201
Research Orientation, Design, and Measurement
Table 2.1 Examples of Hypotheses in Various Forms
Form
Example
Null Hypothesis
There is no difference in the average mathematics score
on the TEMA-3 between five-year-old children who are
enrolled in an innovative mathematics programme in
Singapore and those who are not.
Nondirectional
There is a difference in the average mathematics score on the
Alternative Hypothesis
TEMA-3 between five-year-old children who are enrolled
in an innovative mathematics programme in Singapore and
those who are not.
Directional Alternative
Five-year-old children who are enrolled in an innovative
Hypothesis
mathematics programme in average score higher on the
TEMA-3 than those who are not enrolled in the programme
in Singapore.
(Source: Kraska, 2012)
1.1.4 Data Collection
Quantitative researchers typically need to define the population of interest (e.g. fiveyear-old children in Singapore), decide on how to select a sample of participants that is
representative of the population, and determine the number of participants to sample. The
goal is to generalise research findings from the sample to the population of interest.
The objectivity and neutrality of a researcher in the research process is important in
quantitative research. Data collection should be a value-neutral process and should not
be affected by researchers’ values or subjective judgements (Staller, 2012). After data are
collected, researchers will clean and prepare data for subsequent analysis.
SU2-7
HBC201
Research Orientation, Design, and Measurement
1.1.5 Data Analysis
Table 2.2 Purpose of Data Analysis
Purpose
Description
Summarise and
Descriptive statistics are calculated to summarise and describe
Describe Sample
the characteristics of sample data. For example, an average
Data
mathematics score (statistically termed as mean) can be
calculated for participants who join the new mathematics
programme in a study.
Note that a statistic (e.g. the average mathematics score of
participants who join the new programme in a study) is a
characteristic of a sample, while a parameter is a characteristic of
a population (e.g. the average mathematics score of all students
who join the new programme in the target population).
Statistical Inference:
Statistical inference uses statistics from sample data to predict
Estimation
corresponding parameters in population in two ways:
• A point estimate is “a single number, calculated
from the data, that is the best single guess for the
parameter” (Agresti & Finlay, 1997, p.121). For
example, the average mathematics score of participants
who join the new programme in a study.
• An interval estimate is “a range of numbers around the
point estimate, within which the parameter is believed
to fall” (Agresti & Finlay, 1997, p.121). For example,
a range of scores where the population mean score is
predicted to fall.
SU2-8
HBC201
Research Orientation, Design, and Measurement
Purpose
Description
Statistical Inference:
Statistical inference can also be used to test a hypothesis,
Hypothesis Testing
which is typically the goal in many quantitative studies.
(Source: Agresti and Finlay, 1997)
We will now focus on the use of statistical inference for hypothesis testing. The choice
of a statistical test can depend on various factors, such as the type of hypothesis to be
tested, the number of variables, the scale of variables (we will discuss this topic later in
the Measurement chapter), and so on.
From a statistical test, a researcher typically obtains a test statistic and a P-value (Agresti
& Finlay, 1997):
• Test statistic is “a statistic calculated from the sample data to test the null
hypothesis” (Agresti & Finlay, 1997, p.157). Different statistical tests usually use
different test statistics (e.g. a F test uses F test statistic, while a t test uses t test
statistic).
• P-value is the probability of obtaining the observed data if the null hypothesis
were true. The P-value is essentially derived from the test statistic. As it is easier to
interpret the P-value than the test statistic, the P-value is usually of primary interest
when one uses the statistical test result. A small P-value suggests that “the observed
data would be unusual” if the null hypothesis were true (Agresti & Finlay, 1997,
p.173).
1.1.6 Conclusion
Often, after obtaining the test statistic and P-value from a statistical test, a researcher is
interested to further decide whether the empirical evidence is strong enough to reject
the null hypothesis (which typically assumes no association or no effect). A researcher
makes such a decision by comparing the P-value to a predetermined cutoff value, which
is called the α-level or the significance level of the test. It is most common to set α-level at .05
or .01(Agresti & Finlay, 1997).
SU2-9
HBC201
Research Orientation, Design, and Measurement
Suppose a researcher sets the α-level as .05, the conclusion will be one of the following two
scenarios:
• The researcher will reject the null hypothesis, if the P-value is smaller than or equal
to .05. For example, let’s suppose the observed average TEMA-3 math score of
children attending the mathematics programme is higher than that of those who do
not, and the P-value is .02. The researcher can then conclude that children enrolled
in the mathematics programme on average score higher on the TEMA-3 than those
who are not; this difference in the average TEMA-3 score is statistically significant
at the .05 level.
Or,
• The researcher will conclude that the evidence is not strong enough to reject the null
hypothesis, if the P-value is larger than .05. For example, let’s suppose the observed
average TEMA-3 math score of children attending the mathematics programme is
higher than that of those who do not, but now the P-value is .08. The researcher can
then conclude that the difference in the average TEMA-3 score between children
who are enrolled in the mathematics programme and those who are not enrolled is
not statistically significant at the .05 level.
It is important to note that we can never be sure whether a decision made on the basis
of a statistical test is accurate or not, as the results are based on sample data rather than
population data, and there may be unknown sampling error. In fact, a decision made on
the basis of a test may lead to four possible consequences, as shown in Table 2.3 (Agresti
& Finlay, 1997).
SU2-10
HBC201
Research Orientation, Design, and Measurement
Table 2.3 Four Possible Consequences of a Decision in a Hypothesis Test
Decision in a Hypothesis Test
Reject
null Do
not
reject
hypothesis
hypothesis
Correct decision
Truth
Null hypothesis is
Incorrect decision
(Unknown)
true
(Type I error)
Null hypothesis is
Correct decision
false
Incorrect decision
(Type II error)
(Source: Adapted from Table 6.5 in Agresti and Finlay, 1997, p.175)
Read
Read the Introduction and the Elements section in Kraska (2012)
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n352
SU2-11
null
HBC201
Research Orientation, Design, and Measurement
Activity 2.1
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526449399
In this case study, the researcher described a quantitative study that investigated
whether preschool children’s verbal play was associated with their language or social
development.
Please discuss the key stages of this quantitative study (Read, 2018).
1.2 Qualitative Research
While a quantitative study usually has a structured and predetermined research design
at an early stage of the study, a qualitative study is typically more flexible. Table 2.4
summarises the general features of qualitative research in various aspects (Staller, 2012).
Study Units 5 and 6 will discuss the different aspects of qualitative research, such as types
of qualitative studies, in detail.
Table 2.4 Features of Qualitative Research
Aspect
Feature
Research
A researcher typically starts a qualitative study with a broad
Question
research question, which may evolve as the study progresses.
Usually no specific variables or hypotheses are defined or generated
in the framing of a qualitative research question.
Research
Research design is typically flexible and can be adapted during
Design
the process of a study. Inductive reasoning is usually adopted in
qualitative studies.
SU2-12
HBC201
Aspect
Role of a
Researcher
Research Orientation, Design, and Measurement
Feature
• A researcher is regarded as an instrument for data collection
(e.g. observations are made through a researcher’s eyes)
in qualitative studies. This feature suggests that “all
interpretations and observations are filtered through the
researcher, who brings his or her own values and identity to
the process”(Staller, 2012, Role of the Researcher section).
• A researcher may share some characteristics with research
participants. Also, a researcher’s role can range from an
unobtrusive observer to a participant in the research process.
As such, a qualitative researcher is expected to reflect on and
disclose his or her role in a study.
Study Site
A qualitative study is usually conducted in a real-world setting, as
opposed to a laboratory.
Sample
In contrast to quantitative studies, randomly selecting a large
sample of participants that represents the population of interest is
rarely done in qualitative studies. Instead, participants are typically
selected intentionally, and the sample size of a qualitative study is
typically smaller than that of a quantitative study.
Data
Data in a qualitative study are usually in non-numeric form (e.g.
Collection
audio recording) and are commonly collected through interviews,
observations, or documents/artifacts.
Data
Analysis of qualitative data generally requires a researcher to
Analysis
closely examine the structure and patterns in the data (e.g. interview
transcripts, field notes, etc.). Qualitative researchers may check
their preliminary findings with research participants to ensure the
accuracy of their interpretations of the data.
SU2-13
HBC201
Research Orientation, Design, and Measurement
Aspect
Feature
Writing
• A qualitative researcher commonly uses the first person
perspective in writing up research findings (as opposed to the
detached third-person perspective in quantitative reports).
This reflects the active role of the researcher (as opposed to a
neutral and objective role) in the process of the study.
• While quantitative researchers typically report aggregated
data (e.g. mean) in a concise manner, qualitative researchers
commonly provide rich and contextual details when
putting together their findings in an effort to stay close
to the meanings of the words or actions of their research
participants.
Rigour
in
Although qualitative research is usually more flexible, it does not
Qualitative
mean that qualitative research is inferior to quantitative research.
Research
Indicators to evaluate the rigour and quality in qualitative research
will be discussed in Study Unit 5.
(Source: Staller, 2012)
Read
Staller, K. M. (2012). Qualitative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n350
SU2-14
HBC201
Research Orientation, Design, and Measurement
Case Study: Identity and Mixed Race
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526429872
In this case study, the researcher described a qualitative study that investigated the
identity of mixed-race people in the Singapore context. We will discuss the methodological
aspects of qualitative research in more detail in Study Unit 5. For now, please focus on the
following (Rocha, 2018):
• Depiction of the background of the study [see Project Overview and Context
section].
• Description of the characteristics of narrative research, which is a type of qualitative
research [see Research and Narrative section].
1.3 Comparison of Quantitative and Qualitative Research
Lesson Recording
Quantitative versus Qualitative Research
For a detailed discussion of the philosophical positions underlying quantitative and
qualitative research, please review Section 2.1.2 in Study Unit 1. Following the discussion
in Study Unit 1, let’s now focus on the common philosophical positions in quantitative
and qualitative research.
SU2-15
HBC201
Research Orientation, Design, and Measurement
Table 2.5 A Comparison of Quantitative and Qualitative Research
Quantitative Research
Qualitative Research
Associated with philosophical positions
Associated with philosophical positions
such as objectivism and positivism.
such as constructivism and interpretivism.
Based on the belief that reality is stable.
Based on the belief that “reality is generated
through social interaction and iterative
processes…” (Staller, 2012, para. 3).
Deductive
approach:
A
researcher
Inductive approach: A researcher may have
defines specific hypotheses at the
a general idea, but typically does not have
beginning of a study, and proceeds to
specific hypotheses at the beginning of a
data collection and hypothesis testing
study. Instead, research question(s), or other
subsequently.
aspects of the study, may evolve through the
research procedure.
A researcher typically defines variables
A researcher is considered as an instrument
and uses instruments (such as surveys,
for data collection (e.g. the researcher
tests, etc.) to collect data. Objectivity
observes or conducts interviews with
and neutrality of the research process
participants), and the research process is
are deemed important.
considered as value-laden.
Data collected are typically in numeric
Data collected are typically in non-numeric
form.
form (e.g. audiotapes, words, etc.).
Larger sample size
Smaller sample size
SU2-16
HBC201
Research Orientation, Design, and Measurement
Quantitative Research
Qualitative Research
The goal is typically to generalise
The goal is typically to understand a specific
research findings to larger populations.
issue in a context.
Context is usually ignored.
(Sources: Brewer and Headlee, 2012) and Staller, 2012)
SU2-17
HBC201
Research Orientation, Design, and Measurement
Chapter 2: Research Design
Lesson Recording
Research Design
2.1 Quantitative, Qualitative, or Triangulation Approach?
The choice of quantitative or qualitative research methods may depend on various
factors, such as a researcher’s philosophical position (e.g. researchers who believe in
interpretivism typically prefer qualitative methods), the purpose of research studies (e.g.
researchers who are interested in generalising research findings to populations usually
opt for quantitative methods), and so on.
A researcher may sometimes decide to use more than one research method or have more
than one source of data; such a practice is called triangulation. It is common to use
triangulation in studies that adopt both quantitative and qualitative methods; such a
practice is usually referred to as mixed methods (Hastings, 2012).
A researcher may use the triangulation approach for different reasons, such as (Hastings,
2012):
• As each type of data or research method has its own strengths and weaknesses,
using a triangulation approach provides researchers an opportunity to verify
findings from different sources or research methods and thus build up the
credibility of the research results.
• Some researchers are less concerned with whether findings from different sources
or methods are consistent with each other. Instead, these researchers view the use
of a triangulation approach as a means to obtain multiple perspectives to expand
one’s understanding of the phenomenon of interest.
SU2-18
HBC201
Research Orientation, Design, and Measurement
Table 2.6 Various Types of Triangulation
Type
Description
Data Triangulation
Use multiple sources of data in a study. For example, a
researcher may choose to interview not only teachers, but also
parents and students in a study evaluating the impact of an
educational programme.
Investigator
Use multiple investigators in research activities, such as
Triangulation
observing or coding participants’ behaviours, in order to
reduce potential bias.
Theory
Use multiple theoretical perspectives in conducting a study or
Triangulation
interpreting findings.
Methodological
The most commonly used type of triangulation. Use multiple
Triangulation
methods to study a research question.
• Within-methods triangulation: use multiple
quantitative or qualitative methods.
• Between-methods triangulation: use both quantitative
and qualitative methods. It is typically regarded as a
stronger approach than within-methods triangulation.
(Source: Hastings, 2012)
While triangulation has several benefits as discussed, it is no panacea and has its
limitations, such as (Hastings, 2012):
• Use of a triangulation approach does not always guarantee the quality of data.
• Data from different sources may not be suitable to address the same research
question. For example, information collected from a person’s diary may be very
different from information collected through an interview with the person as a diary
is written privately, while an interview is conducted in a social setting. A researcher
should have a clear rationale when using triangulation.
SU2-19
HBC201
Research Orientation, Design, and Measurement
• Use of triangulation typically demands more time and resources from a researcher.
As such, researchers need to evaluate time and financial constraints in deciding on
the use of triangulation strategy.
Read
Read the Introduction and the Elements section in Kraska (2012)
Hastings, S. L. (2012). Triangulation. In N. J. Salkind (Ed.), Encyclopedia of research
design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n469
Activity 2.2
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305013516576
As discussed in Study Unit 1, this case study described a research project that
evaluated whether the Olweus Bullying Prevention Programme reduced school
bullying effectively in the United States (Kuykendall, 2014).
Please discuss the use of triangulation in this research project as follows:
1.
What types of data did the researcher collect?
2.
What finding did the researcher obtain from each type of data?
3.
What were the benefits of triangulation in this project?
4.
What were the possible limitations of triangulation in this project?
SU2-20
HBC201
Research Orientation, Design, and Measurement
2.2 Data Collection Time Periods
Another issue a researcher needs to consider when designing a study is whether data
should be collected at a single or one time period, or over multiple time periods.
Depending on the number of time periods, a study can be classified as either a crosssectional or a longitudinal study. Note that the exact definition of longitudinal research
may vary across disciplines. Nonetheless, the terms “cross-sectional research” and
“longitudinal research” are commonly understood as fitting the descriptions in Table 2.7
(Menard, 2011).
Table 2.7 Description of Cross-Sectional Research and Longitudinal Research
Type
Description
Cross-Sectional
These research projects collect data from individuals or
Research
aggregates (such as households, cities, or countries) on
variables of interest only at a single point in time, i.e. a one time
period.
Longitudinal
These research projects collect data from individuals or
Research
aggregates on variables of interest over extended time, i.e.
multiple time periods, which allow investigations of change
within individuals or aggregates over time.
Note that it is possible to collect retrospective data of an
extended time within a single period (e.g. asking an individual
to recall and report every crime he or she has committed over
the past years in one sitting). Here the broad definition of
longitudinal research is used, and such retrospective studies
are considered as longitudinal research too.
(Source: Menard, 2011)
SU2-21
HBC201
Research Orientation, Design, and Measurement
2.2.1 Different Types of Longitudinal Research
Table 2.8 Different Types of Longitudinal Research
Type
Total
Description
Population
Design
Data are collected for the entire population at different time
periods. In principle, individuals enter the study by birth and
exit the study by death over time. An example is the Uniform
Crime Report of the Federal Bureau of Investigation (FBI),
which collects data on the arrests and crimes that the police is
aware of annually. The purpose of the FBI report is to monitor
the aggregate pattern of those arrests and crimes that the
police is aware of over time.
Repeated
Cross-
Sectional Design
Data are collected for different samples at different time
periods. An example is the General Social Survey (GSS; see
https://gss.norc.org/) conducted by the National Opinion
Research Center in the United States. The GSS emphasises the
administration of the same questions to individuals sampled
at different periods in order to allow the investigations of
trends in various aspects (such as attitudes, behaviours, and
so on) of the American society across time.
Revolving
Design
Panel
Data are collected for sampled individuals for more than one
period. The original sample is replaced by a new sample at
some point as planned. Data collection then continues for
the new sample, and the process continues. An example is
the National Crime Victimization Survey (NCVS; see https://
www.bjs.gov/index.cfm?ty=dcdetail&iid=245) conducted by
the Bureau of Justice Statistics in the United States, which
SU2-22
HBC201
Research Orientation, Design, and Measurement
Type
Description
is one of the most widely used data sources on aggregate
trends in the rates of crime victimisation over time at the
national level in the United States. The NCVS samples
households rather than individuals. Each sampled household
is interviewed seven times over a three-year period. The NCVS
adopts a staggered replacement strategy, such that about one
sixth of the households in the sample are replaced with new
households every six months.
Longitudinal
Design
Panel
Data are collected for the same sample at more than one
time period, i.e. over an extended time. Unlike the other
designs described previously, the sample does not take new
participants after the first period. Also, the purpose of such
a design is typically more concerned with intraindividual
developmental change than aggregate historical change.
An
see
example
is
the
National
Youth
Survey
(NYS;
https://www.icpsr.umich.edu/icpsrweb/NAHDAP/
series/88) conducted in the United States. NYS collected data
on deviant behaviours by youth across years. The participants
interviewed over the years were from the original sample,
although some participants might have dropped out of the
project over time. While the use of the GSS, FBI, or NCVS data
typically focuses more on aggregate historical trends over time
at the national level, the use of the NYS data is typically for
the study of developmental trends over time at the individual
SU2-23
HBC201
Research Orientation, Design, and Measurement
Type
Description
level, and also for the observation of causal relationships
between variables in order to test theories of crime.
(Source: Menard, 2011)
2.2.2 Benefits and Challenges of Longitudinal Research
Table 2.9 Benefits and Challenges of Longitudinal Research
Benefits
Challenges
Describe patterns of change for
The cost of conducting a longitudinal
individuals or aggregates.
study is typically higher than that of a
cross-sectional study.
Establish causal relationships between
Various measurement issues may
variables with respect to direction (such
result from repeated administration of
as whether two variables are positively
questions over time.
or negatively associated, whether
• For example, participants may
variable X causes variable Y to change or
realise that providing certain
the other way around) and strength (i.e.
answers to a question will
whether the relationship between two
lead to one or more follow-up
variables is strong or weak).
questions, and thereby avoid
giving certain answers to the
question in subsequent data
collection period(s) in order to
avoid follow-up questions.
• Some measures may become
inappropriate at a later point of a
study. However, changing these
measures will make subsequent
SU2-24
HBC201
Research Orientation, Design, and Measurement
Benefits
Challenges
data incomparable to the previous
data that were collected using the
original measures.
Can depict patterns of change in an
Data may not be collected for all
attribute or phenomenon of interest
participants at all time periods as
across time.
planned due to various reasons (such
as unavailable participants, withdrawn
participants, and so on), which may lead
to following issues:
• Participants withdrawing from
the study (i.e. attrition) may
raise the concern about whether
the reduced sample is still
representative of the population of
interest, which in turn may affect
the appropriateness of the findings
for generalising to the population
of interest.
• Missing data can pose challenges
in the analysis of longitudinal
data.
(Source: Menard, 2011)
SU2-25
HBC201
Research Orientation, Design, and Measurement
Read
Menard, S. (2011). Longitudinal research. In M. S. Lewis-Beck, A. Bryman, &
T. Futing Liao (Eds.), The SAGE encyclopedia of social science research
methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n516
Activity 2.3
Clements, A. J. (2014). A mixed-methods approach to occupational commitment in
student nurses. SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305013508503
As discussed in Study Unit 1, this case study described a series of research efforts to
examine the occupational commitment of students (particularly student nurses) in the
care professions in the United Kingdom (Clements, 2014).
Please discuss the following questions:
1.
What benefits and limitations did the researcher experience in using the
cross-sectional design in part of these research efforts?
2.
What benefits and limitations did the researcher experience in using the
longitudinal design in part of these research efforts?
2.3 Threats to Internal or External Validity
In designing a research project, a researcher also needs to consider potential threats to both
internal validity and external validity, which are described in Table 2.10.
SU2-26
HBC201
Research Orientation, Design, and Measurement
Table 2.10 A Comparison of Internal and External Validity
Type
Description
Internal
Suggests “the confidence with which researchers can make causal
Validity
inferences” (Brewer, 2011, para. 1) from research results; that is,
an independent variable of interest causes the variation in the
dependent variable.
External
Suggests the confidence with which researchers can generalise
Validity
findings to larger populations.
(Sources: Brewer, 2011, and Ondercin, 2011)
2.3.1 Threats to Internal Validity
Note that the existence of additional variables (other than the particular independent
variable of interest) that can cause the studied dependent variable to change does not
necessarily threaten internal validity. However, if the effects of the independent variable
of interest cannot be separated from the effects of other variables that may also influence
the dependent variable, then internal validity is challenged. Table 2.11 shows common
extraneous factors that are not intentionally under investigation in a study and yet may
pose threats to internal validity. In cases where the effects of these extraneous factors
cannot be separated from the effects of the studied independent variable, these extraneous
factors are referred to as confounding factors (Brewer, 2011).
Table 2.11 Common Threats to Internal Validity
Threat
Description
Self-selection
For example, students involved in a study to find out if a new
mathematics programme improves their mathematics test
score are free to decide if they want to join the mathematics
programme. Students who choose to join the programme may
have higher motivation to learn mathematics and therefore are
SU2-27
HBC201
Threat
Research Orientation, Design, and Measurement
Description
more likely to perform well on mathematics. In this case, the
potential difference in the average mathematics score between
students who choose to join the programme and those who do
not may have nothing to do with the mathematics programme
per se. Rather, it could be due to the differing motivation levels
of students who join or do not join the programme.
In addition to participants’ self-selection issue, there could be
other issues in the selection and group assignment of participants.
History
Differences in the dependent variable measured at two different
time points may be caused by an event that occurs between the
measure, instead of being due to the independent variable of
interest.
Maturation
Differences in the dependent variable measured at two
different time points may be caused by changes in participants’
conditions (e.g. participants are growing older), instead of being
due to the independent variable of interest.
Testing
If the same test is administered to participants more than once,
the test scores on the second administration may be affected due
to the participants’ previous exposure to the test.
Measurement
Issues
Issues such as the following can produce inappropriate measures:
• Lack of consistency in data collection (e.g. the procedure
of test administration differs across measurement
occasions), or data coding activities (e.g. failure to code
participants’ behaviours strictly according to the coding
scheme).
SU2-28
HBC201
Research Orientation, Design, and Measurement
Threat
Description
• Instruments (such as surveys, tests, and so on) used do
not produce reliable measures.
Experimental
Different proportions of participants dropping out of the study
Mortality
in the different conditions may affect group comparison.
(Source: Brewer, 2011)
Read
Brewer, M. B. (2011). Internal validity. In M. S. Lewis-Beck, A. Bryman, & T. Futing
Liao (Eds.), The SAGE encyclopedia of social science research methods.
SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n436
2.3.2 Threats to External Validity
Table 2.12 Common Threats to External Validity
Threat
Description
Non-
Researchers should ensure that “the underlying causal
representative
process is the same for both the sample and the population
Sample
of interest” (Ondercin, 2011, para. 2). For example, in a study
about the effect of campaign advertising on voters’ opinion of
the campaign candidate, using a sample of young participants
all in early 20s may provide results that are very different from
using a sample of participants from all age groups that are
eligible to vote, as young people’s opinions may be affected by
the advertisements more easily.
SU2-29
HBC201
Research Orientation, Design, and Measurement
Threat
Description
Artificial Research
Participants’ behaviours in an artificial setting, such as in a
Setting
laboratory, may be very different from their behaviours in a
natural environment.
Testing Effects
Experience of being tested or observed may influence
participants’ behaviours.
(Source: Ondercin, 2011)
In considering options of research design, researchers may need to evaluate the trade-off
between internal and external validity. For example, in order to study the associations
between the specific variables of interest (ensuring internal validity), researchers may
decide to conduct the study in an artificial setting in a laboratory instead of a natural
environment (somewhat compromising external validity) (Ondercin, 2011).
Read
Ondercin, H. L. (2011). External validity. In M. S. Lewis-Beck, A. Bryman, & T.
Futing Liao (Eds.), The SAGE encyclopedia of social science research methods.
SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n318
SU2-30
HBC201
Research Orientation, Design, and Measurement
Activity 2.4
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305013516576
Please discuss the following questions based on the above case study (Kuykendall,
2014):
1.
What were the possible threats to internal validity in this study?
2.
What were the possible threats to external validity in this study?
SU2-31
HBC201
Research Orientation, Design, and Measurement
Chapter 3: Measurement
Lesson Recording
Measurement
3.1 Levels of Measurement
In a study, each participant is expected to have a specific value (e.g. male or female) for
each variable (e.g. gender). The nature of these values indicates the level of measurement
for a variable. Table 2.13 summarises four possible levels of measurement from the lowest
(nominal level) to the highest (ratio level) (Neuman, 2014).
Table 2.13 Four Levels of Measurement
Level of
Description
Measurement
Nominal
The possible values for a variable are a set of distinct and
unordered categories. Examples include gender (male or female)
or marital status (single, married, divorced, widowed). Each
observation can only be classified into one of the categories.
Ordinal
The possible values for a variable are a set of distinct categories
that have a natural ordering, but the distance between any two
of these values cannot be determined. For example, a person’s
political philosophy may be classified as liberal, moderate, or
conservative. Each observation can only be classified into one of
the categories.
SU2-32
HBC201
Research Orientation, Design, and Measurement
Level of
Description
Measurement
Interval
The possible values for a variable are numerical; one can
determine the range between any pair of these numerical values.
An example is temperature.
Ratio
The only difference between interval and ratio levels is that
ratio level has a true zero such that values for a variable can
be compared in the form of proportion or ratio. An example is
income.
(Source: Neuman, 2014)
The choice of statistical methods for data analysis is largely affected by the level of
measurement of the studied variables. A researcher should try to measure a construct at a
higher level of measurement if possible due to the following practical reasons (Neuman,
2014):
• A higher level of measurement can be converted to a lower level, but the other
way around is not possible. For example, let’s say we have collected information
on participants’ age (ratio level). We can always convert the participants’ age to an
ordinal variable (e.g. classify a participant as either below 20, 20- to 39-year-old, 40to 49-year-old, 50- to 59-year-old, 60-year-old and beyond). However, we cannot
know a participant’s actual age if we only ask each participant to select a category
where his or her age falls in at the stage of data collection.
• A higher level of measurement allows more options of statistical methods to be
employed at the stage of data analysis.
3.2 Validity
Validity evidence provides information on the degree to which a measure assesses a
construct of interest as intended (Mueller, 2011). Table 2.14 summarises common types of
validity evidence (Mueller, 2011; see also Maul, 2018).
SU2-33
HBC201
Research Orientation, Design, and Measurement
Table 2.14 Common Types of Validity Evidence
Type
Description
Face Validity
Suggests whether a measure appears to assess a construct of
interest properly. For example, using only income from the male
head of the household appears to be a poor measure of a family’s
household income in Singapore, as it is common to have more
than one income earner in a household in Singapore.
Content
Evaluates whether a measure assesses all aspects of a construct
Validity
of interest as defined. For example, a researcher may define a
child’s mathematical ability at a given age as a set of specific
mathematical knowledge and skills. The mathematics test used
in the study should then properly cover the set of knowledge and
skills as defined.
Criterionrelated Validity
• Predictive validity evaluates the association between the
measure of interest and a future criterion. For example,
a measure assessing voting preference before election
demonstrates good predictive validity, if it is strongly and
positively associated with actual voting behaviour.
• Concurrent validity evaluates the association between the
measure of interest and another measure that has been
established as a good indicator of the construct of interest.
For example, let’s assume the goal of a study is to develop
a new mathematics test. A positive and strong correlation
between students’ test scores on the new mathematics test
and an existing mathematics test that has been shown to
demonstrate excellent validity and reliability provides good
concurrent validity evidence for the new mathematics test.
SU2-34
HBC201
Research Orientation, Design, and Measurement
Type
Description
Construct
Let’s say we have a study that measures two constructs,
Validity
extraversion and dominance. Each construct is assessed with the
following two indicators respectively: a research participant’s selfreport and the report of the research participant’s close friend.
• Convergent validity evaluates the extent to which different
indicators that measure the same construct relate to one
another. Following the above example, a strong and
positive correlation between participants’ self-reports and
friends’ reports of a studied construct (e.g. extraversion)
provides evidence for convergent validity.
• Discriminant validity evaluates the extent to which
different indicators that measure distinct constructs relate
to one another. Following the above example, a weak
association between self-reports of extraversion and selfreports of dominance provides evidence for discriminant
validity.
(Sources: Mueller, 2011, and Maul, 2018)
Reflect 2.1
Can you describe the meaning of each of the following validity concepts that we have
discussed in Chapter 2 and this section?
• Internal validity
• External validity
• Measurement validity
SU2-35
HBC201
Research Orientation, Design, and Measurement
3.3 Reliability
Reliability evidence provides information on the consistency of the measurement results
of interest (Mueller, 2011). We will now focus on some of the reliability evidence commonly
used in social sciences, as shown in Table 2.15. For a more detailed description of reliability,
you may refer to Fan and Randall (2018).
Table 2.15 Common Types of Reliability Evidence
Type
Description
Test-Retest
This is typically evaluated with the correlation between two
Reliability
measurement results obtained from repeated administration
of the same instrument (e.g. the correlation between two
scores obtained from the administration of the same
mathematics test to the same participants through identical
testing procedure at two time points).
Cronbach’s Alpha
Cronbach’s alpha is one of the methods commonly used
to evaluate internal consistency (i.e. the consistency of
measurement results from multiple indicators assessing the
same construct). For example, we can compute Cronbach’s
Alpha using students’ scored responses to items in a
mathematics test.
Interrater Reliability
This concerns the consistency of measurement results
obtained from different observers or raters. For example,
we can ask two raters to rate children’s behaviours using an
identical scoring guide and after that, to calculate the degree
of agreement between their ratings.
(Sources: Mueller, 2011, and Fan and Randall, 2018)
SU2-36
HBC201
Research Orientation, Design, and Measurement
It may be tempting at times for researchers to develop a new measure for the research
questions under investigation. However, it is better to identify and use existing measures
in a field that have been proven to be valid and reliable, rather than develop a new measure
from scratch for the following reasons (Mueller, 2011):
• It is risky to use a new measure with unknown validity and reliability as the
evaluation of validity and reliability only occurs after data collection. If the new
measure turns out to have unacceptable validity and reliability, the quality of the
collected data will be questionable. This may result in added cost for additional
data collection, or it may cost the study to be abandoned.
• Using existing measures of key constructs with acceptable validity and reliability
benefits the accumulation of knowledge across research studies in a field.
Read
Mueller, C. W. (2011). Conceptualization, operationalization, and measurement. In
M. S. Lewis-Beck, A. Bryman, & T. Futing Liao (Eds.), The SAGE encyclopedia
of social science research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n150
SU2-37
HBC201
Research Orientation, Design, and Measurement
Reflect 2.2
Figure 1 in Maul (2018) illustrates the concepts of validity and reliability
visually with various scenarios, in each of which multiple attempts are
made to shoot the bull’s-eye of a target. See “Basic Concepts in Validity”
section
in
Maul
(2018)
for
the
figure
and
explanations
through
the
following link: https://methods.sagepub.com/reference/the-sage-encyclopedia-ofeducational-research-measurement-and-evaluation/i21833.xml
SU2-38
HBC201
Research Orientation, Design, and Measurement
Summary
This unit examines the various aspects a researcher would usually consider when
designing a research study.
• In Chapter 1, we discussed and compared features of quantitative and qualitative
research.
• In Chapter 2, we outlined a number of methodological considerations a
researcher may face in research design. These include the considerations for
adopting quantitative, qualitative, or triangulation method, also considerations for
employing cross-sectional or longitudinal design. We also looked at the potential
threats to internal or external validity.
• In Chapter 3, we discussed various measurement issues, such as level of
measurement, validity, and reliability.
SU2-39
HBC201
Research Orientation, Design, and Measurement
Formative Assessment
1.
A researcher plans to study the difference in voting behaviour between males and
females in Singapore after taking into account the participants’ age. Which of the
following statements is correct?
a. Voting behaviour is the independent variable, gender is the dependent
variable, and age is the control variable.
b. Voting behaviour is the dependent variable, gender is the control variable, and
age is the independent variable.
c. Voting behaviour is the control variable, gender is the independent variable,
and age is the dependent variable.
d. Voting behaviour is the dependent variable, gender is the independent
variable, and age is the control variable.
2.
A researcher specifies the hypothesis of a study as follows: There is no difference in
voting behaviour between males and females in Singapore. What is the form of this
hypothesis?
a. Null hypothesis
b. Nondirectional alternative hypothesis
c. Directional alternative hypothesis
d. None of the above.
3.
The same researcher in question 2 obtained a P-value of .02 from the statistical test of
the hypothesis. What does this mean?
a. The small P-value suggests that there is no association between voting
behaviour and gender in Singapore.
b. The small P-value suggests that it would be very unlikely to obtain the
observed data if there were no association between voting behaviour and
gender in Singapore.
SU2-40
HBC201
Research Orientation, Design, and Measurement
c. The difference in voting behaviour between the genders is statistically
significant.
d. The difference in voting behaviour between the genders is not statistically
significant.
4.
Which of the following statements is incorrect?
a. Qualitative researchers usually use inductive reasoning in conducting their
research.
b. Qualitative researchers are usually considered as instruments for data
collection in their research.
c. Qualitative researchers usually conduct their studies in real-world settings.
d. It is common for qualitative researchers to randomly recruit a large sample of
subjects for their studies.
5.
Which of the following statements about triangulation is incorrect?
a. Triangulation allows a researcher to compare results obtained from different
methods which can foster the credibility of the research findings.
b. Triangulation allows a researcher to obtain data from different sources which
can broaden the researcher’s understanding of the studied issue.
c. Researchers should always use triangulation strategy as it has many benefits.
d. Using triangulation strategy does not necessarily lead to a high-quality
research study.
6.
A researcher plans to annually collect data on the social and emotional skills from
a sample of Singaporean children for a period of three years. What is this type of
design?
a. Cross-sectional design
b. Total population design
c. Repeated cross-sectional design
d. Longitudinal panel design
SU2-41
HBC201
7.
Research Orientation, Design, and Measurement
Which of the following is a threat to internal validity?
a. The sample is not representative of the population of interest.
b. Research participants’ behaviours are observed in a laboratory, rather than in
a real-world setting.
c. The procedure for the test administration is not standardised and differs across
test administrators involved in the same study.
d. Research participants intentionally behave in a more socially desirable manner
during observations.
8.
What level of measurement is the variable income at?
a. Interval
b. Ratio
c. Nominal
d. Ordinal
9.
The finding by a researcher shows that students’ scores on a short mathematics test
have a high correlation with their scores on a long mathematics test that measures
the same set of knowledge and skills, and the measure used has been shown to be
valid and reliable. What type of validity evidence does this finding provide for the
short mathematics test?
a. Predictive validity
b. Content validity
c. Concurrent validity
d. Face validity
10. Which of the following evaluates the internal consistency of a test?
a. Test-retest reliability
b. Interrater reliability
c. Cronbach’s Alpha
SU2-42
HBC201
Research Orientation, Design, and Measurement
d. None of the above.
SU2-43
HBC201
Research Orientation, Design, and Measurement
Solutions or Suggested Answers
Formative Assessment
1.
A researcher plans to study the difference in voting behaviour between males and
females in Singapore after taking into account the participants’ age. Which of the
following statements is correct?
a.
Voting behaviour is the independent variable, gender is the dependent
variable, and age is the control variable.
Incorrect. Voting behaviour is the outcome of interest and is thus the
dependent variable.
b.
Voting behaviour is the dependent variable, gender is the control variable,
and age is the independent variable.
Incorrect. The influence of gender on voting behaviour is the interest of the
study. As such, gender is the independent variable.
c.
Voting behaviour is the control variable, gender is the independent variable,
and age is the dependent variable.
Incorrect. Voting behaviour is the outcome of interest and is thus the
dependent variable.
d.
Voting behaviour is the dependent variable, gender is the independent
variable, and age is the control variable.
Correct.
2.
A researcher specifies the hypothesis of a study as follows: There is no difference in
voting behaviour between males and females in Singapore. What is the form of this
hypothesis?
a.
Null hypothesis
SU2-44
HBC201
Research Orientation, Design, and Measurement
Correct. A null hypothesis specifies no association between variables
under investigation.
b.
Nondirectional alternative hypothesis
Incorrect. A nondirectional alternative hypothesis specifies that there is an
association between variables under investigation.
c.
Directional alternative hypothesis
Incorrect. A directional alternative hypothesis specifies the expected
direction of an association between variables under investigation.
d.
None of the above.
Incorrect
3.
The same researcher in question 2 obtained a P-value of .02 from the statistical test of
the hypothesis. What does this mean?
a.
The small P-value suggests that there is no association between voting
behaviour and gender in Singapore.
Incorrect. The small P-value suggests that it would be very unlikely to obtain
the observed data if null hypothesis were true (i.e. no association between
voting behaviour and gender in Singapore).
b.
The small P-value suggests that it would be very unlikely to obtain the
observed data if there were no association between voting behaviour and
gender in Singapore.
Correct.
c.
The difference in voting behaviour between the genders is statistically
significant.
Incorrect. It depends on the significance level the researcher sets. The
difference in voting behaviour between the genders is NOT statistically
significant at the .01 level.
SU2-45
HBC201
Research Orientation, Design, and Measurement
d.
The difference in voting behaviour between the genders is not statistically
significant.
Incorrect. It depends on the significance level the researcher sets. The
difference in voting behaviour between both genders is statistically
significant at the .05 level.
4.
Which of the following statements is incorrect?
a.
Qualitative researchers usually use inductive reasoning in conducting their
research.
Incorrect. This is a Correct statement.
b.
Qualitative researchers are usually considered as instruments for data
collection in their research.
Incorrect. This is a Correct statement.
c.
Qualitative researchers usually conduct their studies in real-world settings.
Incorrect. This is a Correct statement.
d.
It is common for qualitative researchers to randomly recruit a large sample
of subjects for their studies.
Correct. This is the Incorrect statement. Qualitative researchers typically
select participants for their studies purposely, and the sample size is
typically not large.
5.
Which of the following statements about triangulation is incorrect?
a.
Triangulation allows a researcher to compare results obtained from different
methods which can foster the credibility of the research findings.
Incorrect. This is a Correct statement.
b.
Triangulation allows a researcher to obtain data from different sources which
can broaden the researcher’s understanding of the studied issue.
Incorrect. This is a Correct statement.
SU2-46
HBC201
Research Orientation, Design, and Measurement
c.
Researchers should always use triangulation strategy as it has many benefits.
Correct. This is the Incorrect statement. It is typically more expensive and
time-consuming to use triangulation strategy. As such, researchers should
consider time and funding available for the use of triangulation.
d.
Using triangulation strategy does not necessarily lead to a high-quality
research study.
Incorrect. This is a Correct statement.
6.
A researcher plans to annually collect data on the social and emotional skills from
a sample of Singaporean children for a period of three years. What is this type of
design?
a.
Cross-sectional design
Incorrect. A study using cross-sectional design only collects data at a single
time period. Here the researcher plans to collect data over three time periods.
b.
Total population design
Incorrect. The researcher plans to collect data from a sample instead of the
entire population.
c.
Repeated cross-sectional design
Incorrect. A study using repeated cross-sectional design collects longitudinal
data from different samples. Here the researcher plans to collect data from
the same sample instead of from different samples across years.
d.
Longitudinal panel design
Correct.
7.
Which of the following is a threat to internal validity?
a.
The sample is not representative of the population of interest.
Incorrect. This is a threat to external validity.
SU2-47
HBC201
Research Orientation, Design, and Measurement
b.
Research participants’ behaviours are observed in a laboratory, rather than
in a real-world setting.
Incorrect. This is a threat to external validity.
c.
The procedure for the test administration is not standardised and differs
across test administrators involved in the same study.
Correct. The inconsistency in test administration can have undesirable
influence on participants’ test scores.
d.
Research participants intentionally behave in a more socially desirable
manner during observations.
Incorrect. This is a threat to external validity.
8.
What level of measurement is the variable income at?
a.
Interval
Incorrect. One can compare two persons’ incomes in the form of ratio, which
is not a characteristic of variables at interval level (such as temperature).
b.
Ratio
Correct. One can compare two persons’ incomes in the form of ratio.
c.
Nominal
Incorrect. Income is not a categorical variable.
d.
Ordinal
Incorrect. Income is not a categorical variable.
9.
The finding by a researcher shows that students’ scores on a short mathematics test
have a high correlation with their scores on a long mathematics test that measures
the same set of knowledge and skills, and the measure used has been shown to be
valid and reliable. What type of validity evidence does this finding provide for the
short mathematics test?
a.
Predictive validity
SU2-48
HBC201
Research Orientation, Design, and Measurement
Incorrect. Predictive validity evaluates the relationship between the test
under study and a future indicator.
b.
Content validity
Incorrect. Content validity evaluates whether a test assesses a construct of
interest comprehensively as defined (e.g. the specific set of mathematics
knowledge and skills).
c.
Concurrent validity
Correct. Concurrent validity evaluates the relationship between the test
under study and an established test assessing the same construct.
d.
Face validity
Incorrect. Face validity evaluates whether a test appears to measure a
construct of interest appropriately.
10. Which of the following evaluates the internal consistency of a test?
a.
Test-retest reliability
Incorrect. Test-retest reliability evaluates the consistency of test scores from
repeated administration of the same test.
b.
Interrater reliability
Incorrect. Interrater reliability evaluates the consistency of test scores from
different raters.
c.
Cronbach’s Alpha
Correct.
d.
None of the above.
Incorrect
SU2-49
HBC201
Research Orientation, Design, and Measurement
References
Agresti, A., & Finlay, B. (1997). Statistical methods for the social sciences (3rd ed.). Prentice
Hall, Inc.
Brewer, E. W., & Headlee, N. (2012). Proposal. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n341
Brewer, M. B. (2011). Internal validity. In M. S. Lewis-Beck, A. Bryman, & T. Futing Liao
(Eds.), The SAGE encyclopedia of social science research methods. SAGE Publications,
Inc. Retrieved from https://doi.org/10.4135/9781412950589.n436
Clements, A. J. (2014). A mixed-methods approach to occupational commitment
in student nurses. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/978144627305013508503
Fan, F., & Randall, J. (2018). Reliability. In B. B. Frey (Ed.), The SAGE encyclopedia of
educational research, measurement, and evaluation. SAGE Publications, Inc. Retrieved
from https://doi.org/10.4135/9781506326139.n584
Hastings, S. L. (2012). Triangulation. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n469
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n352
Kuykendall, S. (2014). Measuring the impact of the Olweus Bullying Prevention
Program: An evaluation study. SAGE Research Methods Cases. Retrieved from
https://doi.org/10.4135/978144627305013516576
SU2-50
HBC201
Research Orientation, Design, and Measurement
Maul, A. (2018). Validity. In B. B. Frey (Ed.), The SAGE encyclopedia of educational research,
measurement, and evaluation. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781506326139.n731
Menard, S. (2011). Longitudinal research. In M. S. Lewis-Beck, A. Bryman, & T. Futing
Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n516
Mueller, C. W. (2011). Conceptualization, operationalization, and measurement. In
M. S. Lewis-Beck, A. Bryman, & T. Futing Liao (Eds.), The SAGE encyclopedia of
social science research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412950589.n150
Neuman, W. L. (2014). Social research methods: Qualitative and quantitative approaches (7th
ed.). Pearson Education Limited.
Ondercin, H. L. (2011). External validity. In M. S. Lewis-Beck, A. Bryman, & T. Futing
Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n318
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526449399
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526429872
Staller, K. M. (2012). Qualitative research. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n350
SU2-51
HBC201
Research Orientation, Design, and Measurement
SU2-52
Study
Unit
3
Quantitative Research Methods I
HBC201
Quantitative Research Methods I
Learning Outcomes
By the end of this unit, you should be able to:
1.
Explain the issue of power in quantitative research.
2.
Distinguish different types of sampling in quantitative research.
3.
Describe experimental design.
4.
Describe quasi-experimental design.
5.
Describe nonexperimental design.
SU3-2
HBC201
Quantitative Research Methods I
Overview
I
n this unit, we will discuss quantitative research in greater details. We will first
look at major considerations that are typical for researchers in their conduct of
quantitative studies. We will next outline the different types of quantitative research in
social and behavioural sciences.
SU3-3
HBC201
Quantitative Research Methods I
Chapter 1: Key Considerations in Quantitative Research
Lesson Recording
Key Considerations in Quantitative Research
Reflect 3.1
Can you outline the key stages of a typical quantitative study, as discussed in Study
Unit 2?
1.1 Power in Quantitative Research
As discussed in Study Unit 2, a decision made on the basis of a hypothesis test can lead
to one of the four possible consequences (as shown in the square brackets in Table 3.1).
A researcher can never know for sure whether the decision made in a hypothesis test is
correct or incorrect. Nonetheless, a researcher can consider the probability of having a
specific consequence, such as the following (Vo & James, 2012):
• Power, which is the probability that a statistical test will correctly reject the null
hypothesis if the effect of a treatment or association of variables under investigation
indeed exists.
• Probability of a Type II error, which is sometimes referred to as β. Power is equal
to 1− β.
• The significance level of a test, which is commonly referred to as α. This is the
probability of a Type I error.
SU3-4
HBC201
Quantitative Research Methods I
Table 3.1 Probability of a Possible Consequence in a Hypothesis Test
Decision in a Hypothesis Test
Reject null hypothesis
Do not reject null
hypothesis
Truth
Null hypothesis is
(Unknown)
true
Significance level
1−α
α
[Correct decision]
[Type I error]
Null hypothesis is
false
Power
β
[Correct decision]
[Type II error]
(Sources: Adapted from Table 6.5 in Agresti and Finlay (1997, p.175); Vo and James, 2012)
The power of a statistical test is usually of greatest interest to researchers. A researcher
will certainly wish that a statistical test can correctly indicate the existence of an effect or
association as hypothesised if it does exist. Table 3.2 summarises the primary factors that
will influence the power of a test (Vo & James, 2012).
SU3-5
HBC201
Quantitative Research Methods I
Table 3.2 Primary Factors That Influence the Power of a Test
Factor
Description
Significance
Power will increase if a researcher chooses to use a larger
Level
significance level. However, using a larger significance level
inevitably increases the risk of getting a Type I error. In practice, the
significance level is typically set at .05.
Effect Size
Effect size indicates the “magnitude of the effect of interest in
the population”(Vo & James, 2012, Primary Factors That Influence
Power section). As the magnitude of effect size increases, so does
power.
Sample Size
Sample size is the number of observations, which in social sciences is
typically the number of participants in a study. Power will increase
if sample size increases.
(Source: Vo and James, 2012)
Researchers typically conduct a power analysis before data collection to determine the
sample size needed in order to ensure that adequate power will be achieved. A power
of .80 is typically considered acceptable (Vo & James, 2012).
Read
Vo, H. T., & James, L. M. (2012). Power. In N. J. Salkind (Ed.), Encyclopedia of research
design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n324
SU3-6
HBC201
Quantitative Research Methods I
Activity 3.1
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526449399
Please discuss the following questions based on this case study (Read, 2018):
1.
What hypotheses did this study test?
2.
What factors might influence the power of each hypothesis test in this study?
1.2 Sampling in Quantitative Research
Researchers usually recruit a subset of members from a population of interest (i.e. a sample)
for their studies instead of collecting data from members of the entire population, due to
constraints in resources (Huck et al., 2012). As described in Study Unit 2, the goal of a
quantitative study is typically to generalise findings from sample data to the population
of interest, such as using statistical inference to predict a population parameter or to test
a research hypothesis (Agresti & Finlay, 1997 ; see also Huck et al., 2012).
1.2.1 Types of Samples
There are various sampling techniques, which mainly fall into two categories: probability
sampling and nonprobability sampling.
• Probability Sampling: In probability sampling, a researcher recruits a subset of
members from a population of interest in such a way that “each member of
the population has a precise probability of being included in the sample”(Huck
et al., 2012, Probability Sampling Techniques section). Researchers typically use
SU3-7
HBC201
Quantitative Research Methods I
probability sampling with the aim of generating a sample that is representative of
a population of interest (Neuman, 2014).
Table 3.3 Main Types of Probability Samples
Type
Simple
Description
Random
Sample
Each individual in a target population has an equal chance of
being selected into a study.
• The sampling frame is first generated and it consists of all
individuals in a target population.
• A sample is generated using an unbiased method such
as the use of a random number generator.
• Once an individual is selected and included in the
sample, he or she can either be kept in the sampling
frame (sampling with replacement) or be excluded from
the sampling frame (sampling without replacement).
Systematic Random
If the number of individuals in a target population is N and the
Sample
desired sample size is n, the sampling interval (k) is N divided
by n. A researcher will generate a sample in n stages using k
as follows: an individual located between positions 1 to k in
the sampling frame is first selected randomly, then k is added
to that position to identify the second individual for inclusion
in the sample, and so on.
Stratified
Sample
Random
A researcher first specifies a set of mutually exclusive and
exhaustive categories (known as strata; e.g. marital status)
in the population of interest. Then, the researcher randomly
SU3-8
HBC201
Type
Quantitative Research Methods I
Description
selects cases within each category (known as stratum) to
generate a sample.
Cluster Sample
A researcher first identifies a list of clusters (e.g. all of the
classes within a school) in the population of interest, and
then randomly selects a subset of the identified clusters. The
researcher can then either include all cases in the selected
clusters (single-stage cluster sampling) or randomly select a
subset of cases from the selected clusters (two-stage cluster
sampling).
(Sources: Huck et al., 2012, and Neuman, 2014)
Nonprobability Sampling: Unlike probability sampling, nonprobability sampling does
not randomly select individuals from a predetermined sampling frame to form a sample
(Huck et al., 2012).
Table 3.4 Main Types of Nonprobability Samples
Type
Description
Convenience Sample
Members of a population are selected for inclusion in a sample
mainly because they are readily available to participate in a
study.
Quota Sample
Researchers will first determine the categories (e.g. male or
female category) and the desired number of participants for
each category (e.g. 80 males and 90 females). Researchers
will then recruit cases to meet the predetermined number of
participants in each prespecified category accordingly.
SU3-9
HBC201
Quantitative Research Methods I
Type
Description
Snowball Sample
A researcher will identify one or a few cases first before asking
the selected cases to refer or identify additional cases that are
qualified for inclusion in the sample.
Purposive Sample
A researcher uses specific criteria to recruit participants.
(Sources: Huck et al., 2012, and Neuman, 2014)
1.2.2 Misconceptions about Quality of Samples
Table 3.5 summarises common misconceptions about quality of samples and the
corresponding correct concepts (Huck et al., 2012).
Table 3.5 Common Misconceptions about Quality of Samples
Misconception
A
will
random
have
sample
Due to sampling error, a random sample and the population
identical
from which the sample is drawn rarely have the exact same
characteristics
(e.g.
mean, variance, and
so
on)
population
as
the
from
which the sample is
drawn.
Correct Concept
characteristics.
Nonetheless, a well-planned probability sample can provide
information that is highly accurate. If a population is very
large, data from a good probability sample may even provide
more accurate information than data collected from every
member in the entire population as systematic errors may
easily occur in the collection and processing of the enormous
amount of population data (Neuman, 2014).
SU3-10
HBC201
Quantitative Research Methods I
Misconception
Correct Concept
A larger sample is
What influences the quality of a sample is the absolute size of
needed if its target
the sample, instead of the relative size of the sample to its
population is larger.
target population.
(Sources: Huck et al., 2012, and Neuman, 2014)
Read
Huck, S. W., Beavers, A. S., & Esquivel, S. (2012). Sample. In N. J. Salkind (Ed.),
Encyclopedia of research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n395
Activity 3.2
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526449399
Please discuss the following questions based on this case study (Read, 2018):
1.
What sampling strategy did the researcher use in this study?
2.
What was the implication of the sampling strategy on the interpretation of
the study findings? For example, could the researcher generalise findings of
the study to the population?
SU3-11
HBC201
Quantitative Research Methods I
Chapter 2: Types of Quantitative Research
Lesson Recording
Types of Quantitative Research
2.1 Experimental Design
The experimental design is typically considered the most rigorous form of quantitative
research. It is commonly used to examine causal relationships among variables of interest
(Kraska, 2012). Studies implementing the experimental design usually have several key
elements, as summarised in Table 3.6 (Kraska, 2012).
Table 3.6 Elements of an Experimental Design
Element
Description
Randomisation
Each research participant has an independent and equal chance
of being assigned into either a treatment or a control group.
Random assignment of participants is a critical element of an
experimental design. It eliminates self-selection bias and hence
enhances internal validity (Brewer, 2011).
Experimental
All other variables are under control, such that the outcome
Control
of interest (dependent variable) is expected to vary only with
respect to the independent variable of interest.
Experimental
A treatment (e.g. a new teaching method) is essentially the
Treatment
independent variable of interest.
SU3-12
HBC201
Quantitative Research Methods I
Element
Description
Comparison
This element is essentially the hypothesis testing described in
between Groups
the previous study units. For example, in a study to investigate
the impact of a new teaching method on students’ mathematics
ability, the researcher can design the study by first randomly
assigning participating students to either a treatment group
(where students will be taught by the new method), or a control
group (where students will be taught by traditional method).
At the end of the experiment, the researcher will administer
a mathematics test to participating students and compare the
average test score between the treatment and control groups.
Standardised
A standardised instrument is considered as a measure that
Instruments
“meets certain standards or criteria for technical adequacy
in construction, administration, and use” (Kraska, 2012, True
Experimental Designs section).
(Source: Kraska, 2012)
The table below presents three types of the experimental design (Kraska, 2012).
SU3-13
HBC201
Quantitative Research Methods I
Table 3.7 Three Types of the Experimental Design
Type
Description
Pretest-Posttest
A pretest is administered to participants before the experiment.
Control
A posttest is administered to participants after the experiment.
Group
Design
The difference between pretest and posttests are compared
between the treatment and control groups.
Only posttest is administered to participants after the
Posttest-Only
Control
Group
experiment.
Design
Solomon
Four-
Group Design
This design is a combination of the other two designs and have
the following four groups:
• Group 1 (treatment): receives both pretest and posttest.
• Group 2 (control): receives both pretest and posttest.
• Group 3 (treatment): receives only posttest.
• Group 4 (control): receives only posttest.
(Source: Kraska, 2012)
Read
Read the True Experimental Designs section in Kraska (2012):
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n352
SU3-14
HBC201
Quantitative Research Methods I
Case Study: Cognitive Training Research
Sim, S. K. Y., & Chee, C. S. (2018). Implementing cognitive training research in older
adults: Randomized controlled trial, ground issues, and considerations. SAGE
Research Methods Cases.
https://doi.org/10.4135/9781526435231
Presented previously in Study Unit 1, researchers in this case study described several
efforts to evaluate the efficacy of a cognitive training programme for the elderly in
Singapore. We will focus on the research design of their studies. The researchers
specifically used randomised controlled trial, an experimental design, in their evaluation
of the cognitive training programme (Sim & Chee, 2018):
• The researchers described clearly the randomised controlled trial design (see Figure
1 for the flowchart of one intervention study) [see Research Design section].
• The researchers discussed implementations and challenges of three intervention
studies [see Research Practicalities section].
2.2 Quasi-Experimental Design
When it is not feasible for a researcher to randomly assign participants to either a treatment
or a control group, the researcher may consider quasi-experimental design. It is common
for researchers in social sciences to conduct studies in the field (e.g. many educational
research studies are conducted in school settings), where researchers cannot randomly
assign participants and usually do not have full control over all possible variables that
may influence the outcome of interest (Kraska, 2012).
Due to the lack of randomisation in quasi-experimental studies, researchers may
use strategies, such as the following, to tackle the concern about the equivalence of
participants assigned to the treatment and control groups prior to the experiment (Kraska,
2012):
SU3-15
HBC201
Quantitative Research Methods I
• A researcher may try to ensure that participants in the treatment and control groups
are as similar as possible on variables that may influence the outcome of interest,
yet are not the primary interest of a study.
• A researcher may use a pretest to evaluate the similarity of the treatment and control
groups (similar pretest scores provide evidence that the two groups are similar with
respect to the outcome of interest prior to any treatment).
Table 3.8 Various Types of the Quasi-Experimental Design
Type
Description
Pretest-Posttest Design
As participants are not randomly assigned to a group, a
(Nonrandomised
comparison of the pretest score is made by researchers to
Control Group)
examine the equivalence of the treatment and control groups
prior to the experiment. Subsequently, the experiment is
carried out. The same posttest is administered to both groups
after the end of the experiment.
Time-Series Design
Over time, a series of measures are taken on one group
periodically before and after an intervention is introduced.
Single-Subject Design
This design concerns the effect of a treatment on one
or a limited number of behaviours of a single research
participant. The design consists of two components:
measures of the participant’s behaviour(s) of interest (A)
and treatment (B). A researcher typically measures the
participant’s behaviour(s) of interest (A) before a treatment
to establish baseline data and then implements the treatment
(B) later on. A researcher can repeat components A and B
in order to study the effect of a treatment on a participant’s
behaviour(s).
SU3-16
HBC201
Quantitative Research Methods I
Type
Description
Factorial Design
A researcher may consider factorial design if there is more
than one independent variable in a study. For example,
in a two by three (2 × 3) factorial design, there is one
independent variable with two levels (e.g. a variable about
mathematics teaching that has the following two categories:
new teaching method vs. traditional teaching method),
and another independent variable with three levels (e.g. a
variable about students’ motivation in mathematics learning
that has three categories: low, medium, or high).
(Source: Kraska, 2012)
Read
Read the Quasi-Experimental Designs section in Kraska (2012):
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n352
Case Study: Effect of Cooperative Learning Strategies
Bataineh, M. Z. (2018). Using a quasi-experimental quantitative design to understand
the effect of cooperative learning strategies on undergraduate academic
performance. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526444196
This case study described a research project that examined whether the use of cooperative
learning strategies in an undergraduate course improved students’ academic performance
SU3-17
HBC201
Quantitative Research Methods I
in Saudi Arabia. The researcher delved into the choice and practical issues of using a quasiexperimental design in the project as follows (Bataineh, 2018):
• The researcher described a number of hypotheses to be tested in this study [see
Hypothesis Development section].
• The researcher described the reason for using a quasi-experimental design.
Importantly, the researcher also discussed potential threats to internal validity [see
Research Design section].
• The researcher described the implementation of the quasi-experimental study [see
Method in Action section].
• The researcher discussed the results of the study [see Result section].
Case Study: Legal Impact
Koehle, G. (2018). A legal impact study using an interrupted time series design. SAGE
Research Methods Cases.
https://doi.org/10.4135/9781526441829
This case study described a research project that investigated whether the enactment of
an ordinance reduced criminal violations in nuisance rental properties. The researcher
illustrated the use of the interrupted time series design in this project as follows (Koehle,
2018):
• The researcher described the design of the study [see Research Design section].
• The researcher described the procedure of data cleaning and the results of data
analysis [see Method in Action section].
• The researcher discussed practical lessons learnt, such as how empirical data could
show patterns that were unexpected [see Practical Lessons Learned section].
SU3-18
HBC201
Quantitative Research Methods I
Case Study: Job-Related Behaviors
Mackey, M. (2017). Single-subject research design: Utilizing video feedback to improve
job-related behaviors of twins with autism spectrum disorder. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526411259
This case study outlined a project that investigated whether using video feedback
could effectively improve the targeted job behaviours of young twin adults with autism
spectrum disorder. The researcher looked into the following aspects and use of the singlesubject design in this project (particularly in The Ins and Outs of Single-Subject Research
section), (Mackey, 2017):
• Major components of the study, such as the decision to use the within-participant
multiple-probe design, measures of the targeted behaviours, and so on [see
Research Method subsection].
• Different types of single-subject design and the particular design implemented in
this project [see Research Design Phases subsection].
• Findings and unexpected influence of the study [see Results and Anecdotes
subsection].
2.3 Nonexperimental Design
Nonexperimental design is used if researchers are not able to manipulate variables of
interest. Table 3.9 presents three common types of the nonexperimental design (Kraska,
2012).
SU3-19
HBC201
Quantitative Research Methods I
Table 3.9 Three Common Types of the Nonexperimental Design
Type
Description
Causal
In this design, changes in independent variables of interest
Comparative
are beyond researchers’ control and have occurred prior to the
Research
research studies.
Survey Research
While census surveys collect data from the entire population,
sample surveys collect data from a sample of subjects that are
supposed to be representative of the target population. We will
discuss survey research more thoroughly in Study Unit 4.
Correlation
Correlation research investigates relationships between two or
Research
among multiple variables of interest. The method to investigate
correlation depends on the level of measurement and the
number of variables.
• Simple correlation indicates the association between two
variables.
• Partial correlation indicates the association between two
variables, after the potential influence of a third variable
is controlled.
• Multiple correlation coefficient indicates the association
between one dependent variable and a number of
independent variables.
• Canonical correlation indicates the association between
a number of dependent variables and a number of
independent variables.
(Source: Kraska, 2012)
SU3-20
HBC201
Quantitative Research Methods I
Read
Read the Nonexperimental Designs section in Kraska (2012):
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n352
Case Study: Children’s Verbal Play
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526449399
Previously discussed in Study Unit 2, the researcher in this case study described
a correlation research project that investigated whether children’s verbal play was
associated with their language development or social development (Read, 2018).
Activity 3.3
Please go to the Google Forms website (https://www.google.com/forms/about/)
and complete the following:
SU3-21
HBC201
Quantitative Research Methods I
1.
Create a survey using Google Forms. As this is a practice, feel free to try
different item formats available on Google Forms (e.g. short answer, multiple
choice, etc.). Please limit the number of items to five.
2.
Invite five people to complete your survey.
Please be prepared to share your experience (e.g. questions you may have in
generating your survey or taking surveys created by others) when we discuss survey
research in Study Unit 4.
Reflect 3.2
In this unit, we discussed the different types of quantitative research design. Is there a
particular design that is suitable for investigating a research question of your interest?
SU3-22
HBC201
Quantitative Research Methods I
Summary
We delved into quantitative research in this unit through our study of the following:
• A discussion of factors that might influence the power of a hypothesis test in
Chapter 1, which also includes an outline of the different types of sampling
strategies.
• A description of three main types of quantitative research design, i.e. experimental
design, quasi-experimental design, and nonexperimental design in Chapter 2.
SU3-23
HBC201
Quantitative Research Methods I
Formative Assessment
1.
What does “power” mean in a statistical test?
a. The probability that the null hypothesis will be falsely rejected in a statistical
test when the null hypothesis is true.
b. The probability that the null hypothesis will NOT be rejected in a statistical
test when the null hypothesis is true.
c. The probability that the null hypothesis will NOT be rejected in a statistical
test when the null hypothesis is false.
d. The probability that the null hypothesis will be correctly rejected in a statistical
test when the null hypothesis is indeed false.
2.
Which of the following will affect the power of a statistical test?
a. Significance level
b. Effect size
c. Sample size
d. All of the above.
3.
Suppose a researcher randomly selected participants within each of the four preidentified ethnicity categories (i.e. Chinese, Malay, Indian, and Other) in Singapore,
what would the generated sample type be?
a. Simple random sample
b. Systematic random sample
c. Stratified random sample
d. Cluster sample
4.
Suppose a researcher recruited family members and friends who were easy to
approach and readily available for a research project, what would the generated
sample type be?
SU3-24
HBC201
Quantitative Research Methods I
a. Quota sample
b. Convenience sample
c. Snowball sample
d. Purposive sample
5.
Which of the following is correct?
a. As long as participants are randomly selected, a sample should have the same
characteristics (such as mean) as the population of interest.
b. A researcher should recruit more participants if the population of interest is
larger.
c. Both options a and b are incorrect.
d. Both options a and b are correct.
6.
What type of experimental design does Figure 1 in the Cognitive Training Research
case study (Sim & Chee, 2018) represent?
a. Pretest-posttest control group design
b. Posttest-only control group design
c. Solomon four-group design
d. None of the above.
7.
What is the main difference between a quasi-experimental design and an
experimental design?
a. Randomisation
b. Control group
c. Pretest
d. Treatment
8.
What is the type of design used in this scenario – a researcher collected data
on a sample of senior citizens’ mental health twice per day before and after the
SU3-25
HBC201
Quantitative Research Methods I
implementation of lockdown measures in Singapore for eight months in the time of
COVID-19?
a. Pretest-posttest (nonrandomised control group) design
b. Time-series design
c. Single-subject design
d. None of the above.
9.
What is the type of design used in this instance – a researcher measured a participant’s
smoking behaviour before the participant joined an intervention programme and
after the participant completed the intervention programme?
a. Pretest-posttest (nonrandomised control group) design
b. Time-series design
c. Single-subject design
d. None of the above.
10. Correlation research is a type of _________ research.
a. Experimental
b. Quasi-experimental
c. Nonexperimental
d. None of the above.
SU3-26
HBC201
Quantitative Research Methods I
Solutions or Suggested Answers
Formative Assessment
1.
What does “power” mean in a statistical test?
a.
The probability that the null hypothesis will be falsely rejected in a statistical
test when the null hypothesis is true.
Incorrect. This is the meaning of significance level.
b.
The probability that the null hypothesis will NOT be rejected in a statistical
test when the null hypothesis is true.
Incorrect. Power is concerned with the probability of rejecting the null
hypothesis when it is false.
c.
The probability that the null hypothesis will NOT be rejected in a statistical
test when the null hypothesis is false.
Incorrect. This is the meaning of probability of a Type II error.
d.
The probability that the null hypothesis will be correctly rejected in a
statistical test when the null hypothesis is indeed false.
Correct.
2.
Which of the following will affect the power of a statistical test?
a.
Significance level
Correct.
b.
Effect size
Correct.
c.
Sample size
Correct.
d.
All of the above.
SU3-27
HBC201
Quantitative Research Methods I
Correct.
3.
Suppose a researcher randomly selected participants within each of the four preidentified ethnicity categories (i.e. Chinese, Malay, Indian, and Other) in Singapore,
what would the generated sample type be?
a.
Simple random sample
Incorrect. The researcher selected participants within each pre-identified
ethnicity category. As such, the probability of being selected is NOT identical
for all prospective participants in the population.
b.
Systematic random sample
Incorrect. The researcher did not generate the sample using a sampling
interval.
c.
Stratified random sample
Correct. The researcher randomly selected participants within each
stratum (i.e. each ethnicity category in this example).
d.
Cluster sample
Incorrect. Ethnicity categories are strata, instead of clusters.
4.
Suppose a researcher recruited family members and friends who were easy to
approach and readily available for a research project, what would the generated
sample type be?
a.
Quota sample
Incorrect. The researcher did not select participants with respect to
prespecified categories.
b.
Convenience sample
Correct. The researcher selected the participants just because they were
easy to approach and readily available.
c.
Snowball sample
SU3-28
HBC201
Quantitative Research Methods I
Incorrect. The researcher did not ask recruited participants to refer other
prospective participants.
d.
Purposive sample
Incorrect. The researcher did not use any sampling criterion.
5.
Which of the following is correct?
a.
As long as participants are randomly selected, a sample should have the same
characteristics (such as mean) as the population of interest.
Incorrect. It is common that population and a random sample do not have
identical characteristics because of sampling error.
b.
A researcher should recruit more participants if the population of interest is
larger.
Incorrect. What matters is the absolute size of a sample, rather than the
proportion of a sample to its population.
c.
Both options a and b are incorrect.
Correct. See explanations for options a and b.
d.
Both options a and b are correct.
Incorrect. See explanations for options a and b.
6.
What type of experimental design does Figure 1 in the Cognitive Training Research
case study (Sim & Chee, 2018) represent?
a.
Pretest-posttest control group design
Correct. A pretest was administered to treatment and control groups before
treatment. A posttest was administered to treatment and control groups
after treatment.
b.
Posttest-only control group design
Incorrect. A pretest was administered prior to the treatment in this study.
SU3-29
HBC201
Quantitative Research Methods I
c.
Solomon four-group design
Incorrect. This study only had two groups, both of which received pretest
and posttest.
d.
None of the above.
Incorrect.
7.
What is the main difference between a quasi-experimental design and an
experimental design?
a.
Randomisation
Correct. In a quasi-experimental study, a researcher cannot randomly
assign participants to either a treatment or a control group.
b.
Control group
Incorrect. A quasi-experimental study can include a control group as well.
c.
Pretest
Incorrect. A quasi-experimental study can include a pretest as well.
d.
Treatment
Incorrect. A quasi-experimental study can include a treatment as well.
8.
What is the type of design used in this scenario – a researcher collected data
on a sample of senior citizens’ mental health twice per day before and after the
implementation of lockdown measures in Singapore for eight months in the time of
COVID-19?
a.
Pretest-posttest (nonrandomised control group) design
Incorrect. There was no control group in the study.
b.
Time-series design
Correct. The researcher collected data at a series of time points before and
after the implementation of the intervention from the same sample.
SU3-30
HBC201
Quantitative Research Methods I
c.
Single-subject design
Incorrect. The researcher did not focus on a single participant.
d.
None of the above.
Incorrect.
9.
What is the type of design used in this instance – a researcher measured a participant’s
smoking behaviour before the participant joined an intervention programme and
after the participant completed the intervention programme?
a.
Pretest-posttest (nonrandomised control group) design
Incorrect. There was no control group in the study.
b.
Time-series design
Incorrect. The researcher did not collect a series of data points across time.
c.
Single-subject design
Correct. The researcher focused on the effect of the intervention
programme on a single participant’s smoking behaviour.
d.
None of the above.
Incorrect.
10. Correlation research is a type of _________ research.
a.
Experimental
Incorrect. Correlation research does not investigate whether a causal
relationship exists between two variables.
b.
Quasi-experimental
Incorrect. Correlation research does not investigate whether a causal
relationship exists between two variables.
c.
Nonexperimental
Correct.
SU3-31
HBC201
Quantitative Research Methods I
d.
None of the above.
Incorrect.
SU3-32
HBC201
Quantitative Research Methods I
References
Agresti, A., & Finlay, B. (1997). Statistical methods for the social sciences (3rd ed.). Prentice
Hall, Inc.
Bataineh, M. Z. (2018). Using a quasi-experimental quantitative design to understand
the effect of cooperative learning strategies on undergraduate academic
performance. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526444196
Brewer, M. B. (2011). Internal validity. In M. S. Lewis-Beck, A. Bryman, & T. Futing Liao
(Eds.), The SAGE encyclopedia of social science research methods. SAGE Publications,
Inc. Retrieved from https://doi.org/10.4135/9781412950589.n436
Huck, S. W., Beavers, A. S., & Esquivel, S. (2012). Sample. In N. J. Salkind (Ed.),
Encyclopedia of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n395
Koehle, G. (2018). A legal impact study using an interrupted time series design. SAGE
Research Methods Cases. Retrieved from https://doi.org/10.4135/9781526441829
Kraska, M. (2012). Quantitative research. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n352
Mackey, M. (2017). Single-subject research design: Utilizing video feedback to improve
job-related behaviors of twins with autism spectrum disorder. SAGE Research
Methods Cases. Retrieved from https://doi.org/10.4135/9781526411259
Neuman, W. L. (2014). Social research methods: Qualitative and quantitative approaches (7th
ed.). Pearson Education Limited.
Read, K. (2018). Making sense of nonsense: Quantifying children’s verbal play for
correlational research. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526449399
SU3-33
HBC201
Quantitative Research Methods I
Sim, S. K. Y., & Chee, C. S. (2018). Implementing cognitive training research in older
adults: Randomized controlled trial, ground issues, and considerations. SAGE
Research Methods Cases. Retrieved from https://doi.org/10.4135/9781526435231
Vo, H. T., & James, L. M. (2012). Power. In N. J. Salkind (Ed.), Encyclopedia
of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n324
SU3-34
Study
Unit
4
Quantitative Research Methods II
HBC201
Quantitative Research Methods II
Learning Outcomes
By the end of this unit, you should be able to:
1.
Illustrate the use of survey in social sciences.
2.
Describe different types of survey questions.
3.
Compare different types of survey administration.
4.
Compare different methods to evaluate the quality of a survey.
5.
Discuss pros and cons of survey research.
6.
Discuss the issue of measurement error.
7.
Discuss the issue of nonresponse.
8.
Discuss ethical issues in survey research.
SU4-2
HBC201
Quantitative Research Methods II
Overview
W
e will focus on survey research in this unit. Survey is one of the most popular
quantitative methods used in social and behavioural science research. We will
begin by outlining the fundamentals of survey research. We will next look at the various
concerns that researchers commonly have in conducting survey research. At the end of
this unit, we will discuss potential ethical issues related to survey research.
SU4-3
HBC201
Quantitative Research Methods II
Chapter 1: Introduction to Survey Research
Lesson Recording
Survey Research
1.1 Use of Survey
Surveys are commonly used in social science research through which “individuals answer
specific questions about their behavior, attitudes, beliefs, or emotions” (Mrug, 2012, para.
1). Although sometimes researchers may use surveys in experimental research, surveys
are often used in nonexperimental research that does not permit an examination of causal
relationships between variables (Mrug, 2012).
1.2 Types of Survey Questions
A survey usually consists of multiple questions that measure a construct of interest, such
as primary school students’ motivation for learning mathematics (Mrug, 2012).
Table 4.1 Types of Survey Questions
Type
Description
Open-
When answering an open-ended question, a participant needs to
Ended
construct responses in his or her own words.
Question
Close-
When answering a close-ended question, a participant needs to choose
Ended
the most suitable answer from a given set of responses. With respect
Question
to the format of response categories, close-ended questions can be
classified into one of the following types:
SU4-4
HBC201
Type
Quantitative Research Methods II
Description
• Dichotomous question, which only provides two response
categories (such as Yes or No category).
• Nominal question, which provides more than two unordered
response categories.
• Ordinal question, which provides more than two categories that
can be ordered from low to high or small to large.
• Continuous question, which allows a participant to place his or
her answer at any point on a continuum that has descriptors on
either end.
• Rank-ordered question, which asks a participant to rank a given
number of response categories with respect to a criterion (e.g. the
participant’s preference).
(Source: Mrug ,2012)
Every type of survey question has its pros and cons. For example, although open-ended
questions provide the opportunity for some participants to elaborate their own thoughts,
other participants may find it too time-consuming to take open-ended questions and
decide not to answer them. Similarly, with closed-ended questions, some participants
may find it easier and thus be more willing to answer such questions while others may
find none of the close-ended response categories applicable. A possible alternative is a
partially open-ended question, where a set of close-ended response categories are given
and followed with an open-ended response category for participants to specify their
answers if they wish to (Mrug, 2012).
SU4-5
HBC201
Quantitative Research Methods II
1.3 Types of Survey Administration
Researchers can administer surveys in various ways, as summarised in Table 4.2 (Mrug,
2012).
Table 4.2 Types of Survey Administration
Type
Description
Major Concerns
Mail
Each participant completes a
survey (that is either received
via mail or from a researcher
directly) at his or her own
• The
researcher
cannot
control how surveys are
completed.
• Nonresponse bias
convenience and then returns
the completed survey to the
researcher by mail.
Telephone
A survey is administered to
each participant via telephone,
• Interviewer bias
• Sampling bias
either by interviewers or by
interactive voice response (IVR)
systems where questions are prerecorded.
Internet
Each participant completes a
• Sampling bias
survey either on a website or by
• The
e-mail.
researcher
cannot
control by whom or under
what
condition
a
survey is completed.
SU4-6
given
HBC201
Type
Quantitative Research Methods II
Description
Major Concerns
• Biased
results,
participants
if
some
complete
the
survey more than one time.
In Person
(Individually)
An interviewer administers a
• Interviewer bias
survey to a participant in person.
• Biased results, if participants
feel
It is possible to use computerassisted personal interviewing
(CAPI)
or
audio
uncomfortable
truthfully
about
answering
sensitive questions in person.
computer-
assisted self-interview (ACASI)
to assist with the interview
process.
In Person
(Group)
A
group
of
simultaneously
participants
complete
• Place pressure on individuals
in the group to participate.
surveys on their own at a given
• Confidentiality
setting.
• Some participants may be
less
willing
truthfully
in
to
a
answer
group
setting than in individual
interviews.
(Source: Mrug, 2012)
SU4-7
HBC201
Quantitative Research Methods II
1.4 Methods to Evaluate the Quality of a Survey
After a survey is drafted, it is important to test the survey to identify any potential problem
(such as unclear wordings, confusing instruction for survey administration, etc.). Table 4.3
presents various methods a researcher may use to evaluate a survey (Holyk, 2011).
Table 4.3 Methods to Evaluate a Survey
Method
Description
Cognitive
The purpose of a cognitive interview is for a researcher to
Interview
understand respondents’ thinking process when they answer
survey questions. Typically, a small number of respondents are
interviewed individually during which each respondent may be
asked specific questions about his or her own thinking process
(e.g. an interviewer may ask the respondent “how did you
interpret ‘effective’ when you answered the question ‘My yoga
instructor’s teaching is effective?’”). A respondent may be asked
to say what is in his or her mind when answering a survey
question too.
Focus Group
A researcher finds a small group of respondents to discuss and
critique survey questions.
Expert Panel
A researcher asks experts in the content area of interest and in
methodology to evaluate the quality of a survey and identify
potential problems.
(Source: Holyk, 2011)
SU4-8
HBC201
Quantitative Research Methods II
1.5 Pros and Cons of Survey Research
Table 4.4 Pros and Cons of Survey Research
Pros
Cons
• It is relatively easy and efficient to
use surveys to collect data.
• Participants may not provide
valid answers to survey questions
• Some information can best be
due to various reasons, such as
collected by surveys (for example,
misinterpretations of questions
a person’s religious belief).
or response categories, low
• There are various options for both
the format and the administration
method of a survey.
motivation to answer survey
questions, incorrect memories or
self-perceptions, and so on.
(Source: Mrug, 2012)
Read
Mrug, S. (2012). Survey. In N. J. Salkind (Ed.), Encyclopedia of research design. SAGE
Publications, Inc.
https://doi.org/10.4135/9781412961288.n449
SU4-9
HBC201
Quantitative Research Methods II
Case Study: Work Experiences Questionnaire
Sloan, M. M. (2018). The work experiences questionnaire: Using mail survey research to
collect data from a large sample of workers. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526430755
In this case study, the researcher described the design and administration of a mail survey
to study the association between people’s emotional management and well-being in the
work place (Sloan, 2018). Among other lessons, the researcher shared valuable experiences
particularly in the following aspects of conducting a mail survey study:
• Formulating the research question [See Project Overview and Context section].
• Deciding on the sampling strategy and the administration method of the survey
[See Research Practicalities section].
• Developing the survey [See Creating the Questionnaire section].
• Collecting survey data [See Designing the Implementation Strategy section].
• Piloting and revising the survey [See Pilot Testing section].
Case Study: Scottish Health Survey
Sharp, C., & Rutherford, L. (2014). The Scottish Health Survey. SAGE Research Methods
Cases.
https://doi.org/10.4135/978144627305013501440
This case study described the efforts to develop and administer the Scottish Health Survey
which is used by the Scottish government to collect information on the health of the
population over time. The researchers outlined various elements of the survey research,
particularly the following (Sharp & Rutherford, 2014):
• Procedure and practical issues of sampling [see Research Design section].
• Major issues in survey development and administration, such as the following [see
Questionnaire Design, Testing, and Administration section]:
SU4-10
HBC201
Quantitative Research Methods II
◦ Actions taken to evaluate the performance of survey questions at the stage
of survey design and testing.
◦ Different modes of survey administration.
◦ Special concerns with how to measure certain health aspects (such as
biological conditions or alcohol consumption).
◦ Strategies used to improve survey response.
◦ Quality control strategies.
Case Study: EU Kids Online II Project
Livingstone, S., Haddon, L., Görzig, A., & Ólafsson, K. (2014). EU Kids Online II: A
large-scale quantitative approach to the study of European children’s use of the
internet and online risks and safety. SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305014533936
This case study described an international research project that designed and conducted
child and parent surveys to examine children’s internet use and related risks in
various European countries. The researchers described the following key components of
designing and conducting this cross-nation survey research (Livingstone et al., 2014):
• Features of cross-national comparisons in this study [see Rationale for Research
Design section].
• Process of testing the surveys [see Rationale for the Research Methods section].
• Process and practical issues of data collection and cleaning [see Data Collection and
Analysis section].
• Concerns with the use and interpretation of the findings by different parties or
audiences [see Interpretation and Dissemination of Findings section].
SU4-11
HBC201
Quantitative Research Methods II
Reflect 4.1
In Study Unit 3, you created and administered a survey via Google Forms. Please
reflect on the following questions based on that experience:
1.
What type(s) of items did you use in your survey? Will you choose to use a
different item type given your experience and what we have learnt in this
chapter?
2.
What were the benefits or challenges you discovered in administering
your survey online? Will you choose to use a different mode of survey
administration given your experience and what we have learnt in this
chapter?
3.
What method(s) will you consider using to evaluate the quality of your
survey?
SU4-12
HBC201
Quantitative Research Methods II
Chapter 2: Major Concerns in Survey Research
2.1 Measurement Error
Throughout the measurement process, a researcher typically needs to first define a
construct of interest (e.g. mathematical ability) prior to determining how to measure the
construct, and thereafter proceed to data collection and data processing. Errors can occur
at any stage of the measurement process (Miller, 2011).
2.1.1 Types of Measurement Error
Every measure is subject to errors to a certain degree. Assuming a researcher is able to
measure the same participants under the same conditions repeatedly, a measurement error
can be classified as either systematic error or random error (Miller, 2011).
Table 4.5 Types of Measurement Error
Type
Description
Systematic Error (Bias)
If a measure systematically provides scores that deviate
from the true score in one direction in repeated trials (e.g.
a mathematics test systematically overestimates a student’s
mathematical performance in repeated tests), the measure is
biased.
Random
(Variance)
Error
If a measure provides scores that randomly deviate from
the true score in repeated trials (e.g. a mathematics test
sometimes overestimates and sometimes underestimates
SU4-13
HBC201
Quantitative Research Methods II
Type
Description
a student’s mathematical performance), the measure is
influenced by random error.
(Source: Miller, 2011)
2.1.2 Measurement Error in Survey Research
Table 4.6 shows typical sources of measurement errors that may occur at the stage of data
collection or at the later stage of data processing (Miller, 2011).
Table 4.6 Common Sources of Measurement Errors in Survey Research
Source
Phrasing
Description
of
Survey
Question
Respondents may misunderstand a survey question
due to improper phrasing of the question, such as
using terminologies or sentence structures that impede
respondents’ comprehension of the question.
Construct-irrelevant
Survey questions may pose additional burdens unrelated
Burden
to the construct of interest on respondents, such as asking
respondents to report events that happened a long time ago
and thus are difficult to recall, or asking respondents to do
calculations that are time-consuming and unnecessary, and
so on.
Social Desirability
Respondents may be inclined to provide answers that are
socially approved.
Question Format
Respondents may have difficulties in providing articulate
answers to an open-ended question. On the contrary, those
SU4-14
HBC201
Quantitative Research Methods II
Source
Description
given close-ended questions may be forced to choose from
a set of response categories that do not accurately describe
their situations.
Context
Where a specific question is placed in a survey may affect
respondents’ responses to it.
Administration
For example, respondents may have low motivation to
Method
answer survey questions if they dislike the interviewers or if
a survey is self-administered.
Data Processing Errors
Data entry or data coding errors may occur. For
instance, responses to open-ended questions may be coded
inaccurately.
(Source: Miller, 2011)
Read
Miller, P. V. (2011). Measurement error. In P. J. Lavrakas (Ed.), Encyclopedia of survey
research methods. Sage Publications, Inc.
https://doi.org/10.4135/9781412963947.n291
2.2 Nonresponse
Nonresponse refers to the cases (such as individuals, households, and so on) that
researchers have sampled and yet do not have data collected for/from (Goyder, 2011).
Table 4.7 shows possible explanations of nonresponse, the distinction of which is not
always clear though (Goyder, 2011).
SU4-15
HBC201
Quantitative Research Methods II
Table 4.7 Possible Explanations of Nonresponse
Explanation
Description
Noncontact
Researchers fail to reach cases that are sampled.
Refusal
Cases that are sampled refuse to take the survey.
Ineligibility
Cases that are sampled are not eligible to take the survey.
(Source: Goyder, 2011)
While the reason of nonresponse may not always be straightforward, there can be
uncertainty about the predictability or the impact of nonresponse too as explained below
(Goyder, 2011):
• It is typically challenging to predict the probability of a person responding to a
survey, as different reasons (or even a mix of reasons) may influence the person’s
behaviour.
• The nonresponse rate does not necessarily indicate the quality of survey data. For
example, a survey study may have a low percentage of nonresponse. However, if
the people who refuse to take the survey are exactly what the study needs, the low
nonresponse rate may still bias the results of the study. On the contrary, if the people
who do not take the survey are nearly a random subset of the sample, nonresponse
rate will be less threatening to the validity of the study results.
There has been rich research in nonresponse. In addition to theories that attempt to explain
people’s nonresponse behaviours, researchers have also provided practical advice on how
to increase the response rate (Goyder, 2011):
• Increase the number of attempts to contact non-respondents. For example,
researchers conducting telephone surveys may try to call non-respondents again at
different times.
• Provide incentives for people to complete surveys.
SU4-16
HBC201
Quantitative Research Methods II
Read
Goyder, J. (2011). Nonresponse. In P. J. Lavrakas (Ed.), Encyclopedia of survey research
methods. Sage Publications, Inc.
https://doi.org/10.4135/9781412963947.n339
Activity 4.1
Sloan, M. M. (2018). The work experiences questionnaire: Using mail survey
research to collect data from a large sample of workers. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526430755
Please discuss the following questions based on this case study (Sloan, 2018):
1.
What actions did the researcher take in this study to reduce potential
measurement errors?
2.
What actions did the researcher take in this study to reduce nonresponse?
SU4-17
HBC201
Quantitative Research Methods II
Chapter 3: Ethics in Survey Research
In Study Unit 1, we discussed the background and importance of ethical practices in
social science research. Researchers conducting survey research are obligated to protect
research participants from the start of the studies to the dissemination of results. Table
4.8 highlights common ethical issues survey researchers should be aware of (Valerio &
Mainieri, 2011).
Table 4.8 Common Ethical Issues in Survey Research
Issue
Description
Informed
A researcher should provide potential participants with critical
Consent
information about a survey study, such as:
• Purpose of the survey.
• Description of each research activity (e.g. completing a
survey in the paper-and-pencil format, completing a face-toface interview, and so on).
• Commitment expected from participants (such as time
commitment) and any potential concerns.
• Benefits for the participants or for society.
Voluntary
Participation
• Every person’s participation in a survey study should be
voluntary.
• Even after agreeing to join a study, every participant can
decide not to answer any specific survey question and can
withdraw from the study at any time.
SU4-18
HBC201
Quantitative Research Methods II
Issue
Description
Confidentiality/
• Every person involved in a survey study (such as
interviewers) is obligated to protect the confidentiality of
Privacy
participants.
• Participants may find some survey questions intrusive and
should be able to refuse to answer these questions.
Incentives
Incentives with a strong appeal may damage the nature of
voluntary participation and thus raise ethical concerns.
Vulnerable
Researchers should take additional actions as needed to protect
Population
vulnerable populations, such as children, people with health
conditions, and so on. For example, researchers conducting studies
of young children need to get parental consent.
(Source: Valerio and Mainieri, 2011)
In addition to the protection of research participants, survey researchers should also
follow ethical guidelines for the reporting of their research, including truthfully describing
their research methods and results, disclosing information about sponsorship, and so on
(Valerio & Mainieri, 2011).
Read
Valerio, M. A., & Mainieri, T. (2011). Ethical principles. In P. J. Lavrakas (Ed.),
Encyclopedia of survey research methods. Sage Publications, Inc.
https://doi.org/10.4135/9781412963947.n167
SU4-19
HBC201
Quantitative Research Methods II
Case Study: EU Kids Online II Project
Livingstone, S., Haddon, L., Görzig, A., & Ólafsson, K. (2014). EU Kids Online II: A
large-scale quantitative approach to the study of European children’s use of the
internet and online risks and safety. SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305014533936
In the Professional and Ethical Standards section of this case study, the researchers
described clearly the actions taken to address ethical issues in this cross-national survey
research (Livingstone et al., 2014).
Activity 4.2
Sloan, M. M. (2018). The work experiences questionnaire: Using mail survey
research to collect data from a large sample of workers. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526430755
What efforts did the researcher make to address the potential ethical issues in this
study (Sloan, 2018)?
SU4-20
HBC201
Quantitative Research Methods II
Summary
We learnt about the different aspects of survey research in this unit through our study of
the following chapters:
• Chapter 1, where we outlined various topics on survey development and
administration (e.g. types of survey items, modes of survey administration, etc.).
We also compared benefits and limitations of survey research.
• Chapter 2, where we looked into the two concerns researchers usually have in
survey research, i.e. measurement error and nonresponse.
• Chapter 3, where we discussed common ethical issues in survey research.
SU4-21
HBC201
Quantitative Research Methods II
Formative Assessment
1.
A restaurant’s survey form ask its diners about the quality of its food with ratings
from 1 (Very low quality) to 5 (Very good quality). What type of survey question is
this?
a. Dichotomous question
b. Nominal question
c. Ordinal question
d. Rank-ordered question
2.
Which of the following would be considered as major concern for a researcher getting
participants to complete a survey on a website?
a. Some participants may take the survey more than once.
b. The researcher cannot control the condition in which each participant
completes the survey.
c. The potential participants are limited to those who are capable of completing
online surveys.
d. All of the above.
3.
A survey question that asks about a respondent’s ethnicity with four response
categories (i.e. Chinese, Indian, Malay, and Other) is a ___ question.
a. Dichotomous
b. Nominal
c. Ordinal
d. Rank-ordered
4.
Which of the following mode of survey administration may raise concern about
sampling bias?
a. Mail
SU4-22
HBC201
Quantitative Research Methods II
b. Telephone
c. Individual interview
d. Face-to-face group administration
5.
A researcher interviewed a small number of respondents individually to understand
the thinking process behind each of them when answering survey questions. This is
the _____ method to evaluate the quality of a survey.
a. Focus group
b. Cognitive interview
c. Expert panel
d. None of the above.
6.
Which of the following statements about measurement error is correct?
a. If a scale overestimates my weight consistently, the scale is biased.
b. A nurse used a thermometer to check a patient’s body temperature five times
in a row and obtained the readings of 38 degrees Celsius, 36 degrees Celsius,
40 degrees Celsius, 35.5 degrees Celsius, and 37 degrees Celsius, respectively.
The patient’s body temperature was normal and hence, the thermometer was
biased.
c. If a measuring tape consistently underestimates the width of my office door,
the difference in the estimates suggests variance in measurements.
d. None of the above.
7.
Which of the following is NOT considered a source of measurement error?
a. A participant did not answer survey questions truthfully, as he was worried
that the interviewer would judge him.
b. A student failed a mathematics item as he did not know the key mathematical
concept the item was designed to assess.
c. A student failed a mathematics item as she was confused about the item asked
due to the complex sentence structure used.
SU4-23
HBC201
Quantitative Research Methods II
d. A respondent randomly chose a response category as she found none of the
given response categories suitable.
8.
If a researcher cannot reach a potential participant that is sampled, this is called _____.
a. Refusal
b. Ineligibility
c. Noncontact
d. None of the above.
9.
Which of the following statements about nonresponse in survey research is correct?
a. A researcher should try to predict how likely a participant will not respond to
the survey and exclude that participant to avoid nonresponse.
b. A low nonresponse rate suggests good quality of data.
c. The nonresponse rate is not an absolute indicator of the quality of survey data.
d. A researcher should never provide incentives for participants to take a survey.
10. Which of the following statements about ethics in survey research is correct?
a. A participant is obligated to answer all questions after he or she agrees to take
the survey.
b. Only researchers are responsible for protecting participants’ confidentiality.
c. A participant can choose to discontinue his or her own participation in a survey
study.
d. Researchers can freely decide the amount of incentives as long as they can
afford.
SU4-24
HBC201
Quantitative Research Methods II
Solutions or Suggested Answers
Formative Assessment
1.
A restaurant’s survey form ask its diners about the quality of its food with ratings
from 1 (Very low quality) to 5 (Very good quality). What type of survey question is
this?
a.
Dichotomous question
Incorrect. This item has more than two response categories.
b.
Nominal question
Incorrect. This item has ordered response categories.
c.
Ordinal question
Correct. This item has more than two response categories that are ordered
(i.e., from very low to very good quality).
d.
Rank-ordered question
Incorrect. This item does not ask a respondent to rank response categories.
2.
Which of the following would be considered as major concern for a researcher getting
participants to complete a survey on a website?
a.
Some participants may take the survey more than once.
Correct. See Table 4.2.
b.
The researcher cannot control the condition in which each participant
completes the survey.
Correct. See Table 4.2.
c.
The potential participants are limited to those who are capable of completing
online surveys.
Correct. See Table 4.2.
SU4-25
HBC201
Quantitative Research Methods II
d.
All of the above.
Correct.
3.
A survey question that asks about a respondent’s ethnicity with four response
categories (i.e. Chinese, Indian, Malay, and Other) is a ___ question.
a.
Dichotomous
Incorrect. There are more than two response categories.
b.
Nominal
Correct. There are more than two response categories that cannot be
ordered.
c.
Ordinal
Incorrect. The response categories cannot be ordered.
d.
Rank-ordered
Incorrect. Respondents are not asked to rank response categories.
4.
Which of the following mode of survey administration may raise concern about
sampling bias?
a.
Mail
Incorrect. See Table 4.2.
b.
Telephone
Correct. Some potential participants may not have a listed phone number.
c.
Individual interview
Incorrect. See Table 4.2.
d.
Face-to-face group administration
Incorrect. See Table 4.2.
SU4-26
HBC201
5.
Quantitative Research Methods II
A researcher interviewed a small number of respondents individually to understand
the thinking process behind each of them when answering survey questions. This is
the _____ method to evaluate the quality of a survey.
a.
Focus group
Incorrect. The interview was conducted individually, and not in a form of
group discussion.
b.
Cognitive interview
Correct. See Table 4.3.
c.
Expert panel
Incorrect. The researcher interviewed respondents, instead of experts in the
field of interest.
d.
None of the above.
Incorrect.
6.
Which of the following statements about measurement error is correct?
a.
If a scale overestimates my weight consistently, the scale is biased.
Correct. See Table 4.5.
b.
A nurse used a thermometer to check a patient’s body temperature five times
in a row and obtained the readings of 38 degrees Celsius, 36 degrees Celsius,
40 degrees Celsius, 35.5 degrees Celsius, and 37 degrees Celsius, respectively.
The patient’s body temperature was normal and hence, the thermometer was
biased.
Incorrect. The different temperature readings suggest random measurement
errors.
c.
If a measuring tape consistently underestimates the width of my office door,
the difference in the estimates suggests variance in measurements.
Incorrect. The tape measure is biased.
SU4-27
HBC201
Quantitative Research Methods II
d.
None of the above.
Incorrect.
7.
Which of the following is NOT considered a source of measurement error?
a.
A participant did not answer survey questions truthfully, as he was worried
that the interviewer would judge him.
Incorrect. Social desirability is a source of measurement error.
b.
A student failed a mathematics item as he did not know the key mathematical
concept the item was designed to assess.
Correct. The student’s incorrect answer accurately reflected his lack of
understanding of the mathematical concept of interest. As such, this is not
a source of measurement error.
c.
A student failed a mathematics item as she was confused about the item
asked due to the complex sentence structure used.
Incorrect. Improper wording of items is a source of measurement error.
d.
A respondent randomly chose a response category as she found none of the
given response categories suitable.
Incorrect. Inappropriate question format is a source of measurement error.
8.
If a researcher cannot reach a potential participant that is sampled, this is called _____.
a.
Refusal
Incorrect. See Table 4.7.
b.
Ineligibility
Incorrect. See Table 4.7.
c.
Noncontact
Correct. See Table 4.7.
d.
None of the above.
SU4-28
HBC201
Quantitative Research Methods II
Incorrect
9.
Which of the following statements about nonresponse in survey research is correct?
a.
A researcher should try to predict how likely a participant will not respond
to the survey and exclude that participant to avoid nonresponse.
Incorrect. It is challenging to predict a person’s behaviour. In addition,
excluding potential participants simply based on such a prediction poses
threats to the representativeness of a sample.
b.
A low nonresponse rate suggests good quality of data.
Incorrect. If the people who did not complete the survey share certain
characteristics, the results may be biased even though the nonresponse rate
is low.
c.
The nonresponse rate is not an absolute indicator of the quality of survey
data.
Correct. See explanation for option b.
d.
A researcher should never provide incentives for participants to take a
survey.
Incorrect. A researcher can provide incentives for participants if he or she
follows ethical guidelines properly.
10. Which of the following statements about ethics in survey research is correct?
a.
A participant is obligated to answer all questions after he or she agrees to
take the survey.
Incorrect. A participant can refuse to respond to any specific question.
b.
Only researchers are responsible for protecting participants’ confidentiality.
Incorrect. Everyone involved in a study has the obligation to protect
participants’ confidentiality.
SU4-29
HBC201
Quantitative Research Methods II
c.
A participant can choose to discontinue his or her own participation in a
survey study.
Correct.
d.
Researchers can freely decide the amount of incentives as long as they can
afford.
Incorrect. Incentives that have a strong appeal can be unethical. See Table 4.8.
SU4-30
HBC201
Quantitative Research Methods II
References
Goyder, J. (2011). Nonresponse. In P. J. Lavrakas (Ed.), Encyclopedia of survey
research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963947.n339
Holyk, G. G. (2011). Questionnaire design. In P. J. Lavrakas (Ed.), Encyclopedia of
survey research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963947.n425
Livingstone, S., Haddon, L., Görzig, A., & Ólafsson, K. (2014). EU Kids Online II: A
large-scale quantitative approach to the study of European children’s use of the
internet and online risks and safety. SAGE Research Methods Cases. Retrieved from
https://doi.org/10.4135/978144627305014533936
Miller, P. V. (2011). Measurement error. In P. J. Lavrakas (Ed.), Encyclopedia of
survey research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963947.n291
Mrug, S. (2012). Survey. In N. J. Salkind (Ed.), Encyclopedia of research design. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412961288.n449
Sharp, C., & Rutherford, L. (2014). The Scottish Health Survey. SAGE Research Methods
Cases. Retrieved from https://doi.org/10.4135/978144627305013501440
Sloan, M. M. (2018). The work experiences questionnaire: Using mail survey research
to collect data from a large sample of workers. SAGE Research Methods Cases.
Retrieved from https://doi.org/10.4135/9781526430755
Valerio, M. A., & Mainieri, T. (2011). Ethical principles. In P. J. Lavrakas (Ed.),
Encyclopedia of survey research methods. SAGE Publications, Inc. Retrieved from
https://doi.org/10.4135/9781412963947.n167
SU4-31
HBC201
Quantitative Research Methods II
SU4-32
Study
Unit
Qualitative Research Methods I
5
HBC201
Qualitative Research Methods I
Learning Outcomes
By the end of this unit, you should be able to:
1.
Explain key concepts in qualitative research.
2.
Describe typical sampling strategies in qualitative research.
3.
Discuss various types of criteria that can ensure the rigour of qualitative research.
4.
Define unstructured interview.
5.
Define semi-structured interview.
6.
Discuss pros and cons of using interview in research.
7.
Discuss politics of research.
SU5-2
HBC201
Qualitative Research Methods I
Overview
F
ocusing on qualitative research, we will look at major considerations that
qualitative researchers typically have when they conduct their studies. We will
also give attention to the different types of interviews that are commonly used in
qualitative studies. A discussion of politics of research will conclude this unit.
SU5-3
HBC201
Qualitative Research Methods I
Chapter 1: Main Considerations in Qualitative Research
Lesson Recording
Main Considerations in Qualitative Research
Reflect 5.1
Can you describe the features of a typical qualitative study, as discussed in Study Unit
2?
1.1 Key Concepts in Qualitative Research
We have discussed the overall features of qualitative research in Study Unit 2. The purpose
of this section is to highlight two specific characteristics: (a) the emic perspective versus
the etic perspective, and (b) thick description.
1.1.1 Emic Perspective versus Etic Perspective
In qualitative research, there are two types of perspectives (Fetterman, 2012):
• Emic perspective, which is the “insider’s view of reality” and is critical
for a researcher to understand “how people perceive the world around
them” (Fetterman, 2012, para. 1). Note that a person’s view may not reflect what
has happened. Also, different people in the same community may have different
perceptions of the reality.
• Etic perspective, which is the “external social scientific perspective on
reality” (Fetterman, 2012, para. 2).
SU5-4
HBC201
Qualitative Research Methods I
It is common for a qualitative researcher to first collect data from the emic (that is, the
insider’s) perspective and subsequently interpret data from both the emic and etic (that
is, the scientific) perspectives (Fetterman, 2012).
Read
Fetterman, D. M. (2012). Emic/etic distinction. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412963909.n130
1.1.2 Thick Description
The interpretation of an observed behaviour may rely on the context in which the
behaviour occurs. For example, a behaviour of a person blinking one eye can be regarded
as a reflex instead of a wink. Therefore, a qualitative researcher commonly describes
observed behaviours or events in a social setting in details (Hammersley, 2011).
Read
Hammersley, M. (2011). Thick description. In M. S. Lewis-Beck, A. Bryman, & T.
Futing Liao (Eds.), The SAGE encyclopedia of social science research methods.
SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n1013
SU5-5
HBC201
Qualitative Research Methods I
1.2 Sampling in Qualitative Research
In Study Unit 3, we have seen the various types of sampling typically used in quantitative
research. In qualitative research, it is common for researchers to use purposive sampling
to deliberately select participants who meet the specified criteria (Morse, 2011). The major
characteristics of sampling in qualitative research are detailed as follows (Morse, 2011):
• It is common for qualitative researchers to continue recruiting participants in the
process of research studies.
• In qualitative studies, researchers select participants intentionally rather than
randomly due to various concerns such as:
◦ Some participants may not have the characteristics needed for a qualitative
study (e.g. some participants are not as articulate as needed to complete
interviews).
◦ Researchers may need to collect data for specific aspects (e.g. researchers may
need to select negative cases that are exceptions to the patterns emerging
from the existing data).
• Researchers may perform “secondary selection” (Morse, 2011, Convenience Sample
section) and exclude data from certain participants that are considered not useful.
Read
Morse, J. M. (2011). Sampling in qualitative research. In M. S. Lewis-Beck, A.
Bryman, & T. Futing Liao (Eds.), The SAGE encyclopedia of social science
research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412950589.n885
SU5-6
HBC201
Qualitative Research Methods I
1.3 Validity and Reliability Concepts in Qualitative Research
In qualitative research, there is more than one conceptualisation of validity or reliability
(Smith, 2011). Instead of using the validity and reliability concepts commonly used to
evaluate the quality of quantitative research as described in Study Unit 2, qualitative
researchers typically use other criteria. Table 5.1 shows various criteria that are commonly
used to evaluate the rigour of a qualitative study (Saumure & Given, 2012).
Table 5.1 Criteria to Evaluate the Rigor of Qualitative Research
Criteria
Description
Transparency
A researcher should clearly describe each research step taken in
a qualitative study, such that:
• Other researchers may attempt to replicate the research
procedure and see if similar findings can be obtained.
• Others can evaluate whether the study is designed and
implemented appropriately.
Credibility
A researcher can strengthen the credibility of a study through the
following strategies:
• A researcher should be attentive to negative cases rather
than just cases that are consistent with his or her
expectations.
• A researcher should double check findings of the
study with research participants from whom data were
originally collected.
SU5-7
HBC201
Qualitative Research Methods I
Criteria
Description
Dependability
A researcher can check the dependability of a study with the
following approaches:
• Ask more than one person to code the data collected and
check the similarity of the results from different coders.
• Discuss the interpretation of the results with others.
Comparativeness
A researcher should compare findings from different cases in
a given study, or compare findings across research studies,
to develop a more comprehensive picture of the studied
phenomenon.
Reflexivity
A researcher should reflect on how he or she may have influenced
the results of the study (e.g. the presence of the researcher may
influence the participants’ behaviours).
(Source: Saumure and Given, 2012)
The concept of dependability in qualitative research is analogous to the traditional idea
of reliability commonly used in quantitative research (Smith, 2011). Specifically, when a
quantitative researcher considers reliability, the focus is on whether the results can be
reproduced. However, the philosophical foundation of qualitative inquiry is essentially
different from that of quantitative inquiry (e.g. qualitative researchers view reality as being
constructed through social interaction while quantitative researchers view reality as being
stable; review Study Unit 2 for a detailed comparison). In view of that, some qualitative
researchers have argued that “replication is not possible for qualitative inquiry; all that
can or should be expected from a researcher is a careful account of how she or he obtained
and analyzed the data” (Smith, 2011, Reliability section).
SU5-8
HBC201
Qualitative Research Methods I
The concept of credibility in qualitative research is related to the traditional definition of
validity in quantitative research (Smith, 2011). When evaluating validity, a quantitative
researcher typically considers whether the phenomenon of interest is correctly
represented (e.g. whether an instrument measures a construct of interest accurately).
Based on the philosophical foundation of qualitative inquiry (e.g. reality is constructed
socially), some qualitative researchers have argued that whether a researcher accurately
depicts the reality or not is not of major concern. What matters more when evaluating
a qualitative study is whether a researcher faithfully depicts the phenomenon he or she
studies (Smith, 2011).
Read
Saumure, K., & Given, L. M. (2012). Rigor in qualitative research. In L. M. Given
(Ed.), The SAGE encyclopedia of qualitative research methods. SAGE Publications,
Inc.
https://doi.org/10.4135/9781412963909.n409
SU5-9
HBC201
Qualitative Research Methods I
Case Study: Identity and Mixed Race
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526429872
As presented in Study Unit 2, this case study described a qualitative research project that
examined the identity of mixed-race individuals in the Singapore context. Please reflect
on the following aspects of the case study (Rocha, 2018):
• The various components of research design, such as choosing a research method,
recruiting participants, and so on [see Research Design and Theory section].
• The interview process [see Narrative Enquiry: Interviews in Action section]. We
will further discuss interview in qualitative research in the next chapter.
• The influence of the researcher’s background (being a mixed-race individual
herself) on this research [see Locating Myself in Narratives section].
• The various issues in the research process, such as factors that might influence
interviews, limitations in recruitment, and so on. Also, the strategies used by the
researcher to ensure the validity and reliability of the research findings [see Practical
Lessons Learned section].
• The findings and also the emphasis that generalising the findings of the study to
the population was not the goal of the study [see Conclusions section].
SU5-10
HBC201
Qualitative Research Methods I
Chapter 2: Types of Interview in Qualitative Research
Lesson Recording
Qualitative Interviews & Politics of Research
Interview is commonly used by qualitative researchers for data collection. An interview is
usually classified as one of the following, depending on how structured it is (Firmin, 2012):
• Unstructured interview, which is sometimes called “open interview” (Firmin, 2012,
para. 2).
• Semi-structured interview, which combines the open and structured formats and is
sometimes called “in-depth interview” (Cook, 2012, para. 2).
• Structured interview, which resembles the administration of a predetermined set
of questions verbally.
The structured interview is similar to the survey method described in Study Unit 4. Here
we will focus on unstructured and semi-structured interviews.
2.1 Unstructured Interview
In an unstructured interview, a researcher typically starts with a general and open-ended
question in order to learn the interviewee’s experiences or thoughts on a chosen topic.
Usually, the direction of an unstructured interview is led by an interviewee’s responses
rather than by a fixed set of questions a researcher prepares in advance. A researcher
typically uses unstructured interview to explore a new field or to understand a chosen
topic of interest in depth (Firmin, 2012).
SU5-11
HBC201
Qualitative Research Methods I
Read
Firmin, M. W. (2012). Unstructured interview. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412963909.n475
2.2 Semi-Structured Interview
In a semi-structured interview, a researcher typically asks an interviewee a series of
questions related to the topic of interest. There is more control over the direction and scope
of the conversation as compared to an unstructured interview. A researcher typically asks
open-ended questions and uses different probes in a semi-structured interview that allows
the interviewee to elaborate his or her thoughts or experiences more freely, as compared
to a structured interview that is rigidly based on a fixed set of predetermined questions
(Ayres, 2012).
A researcher usually uses the semi-structured interview format when he or she already
has some knowledge of the studied topic. It is critical that a researcher should avoid
using questions that lead an interviewee toward certain type of answers (Ayres, 2012). For
example, a leading question “How did you benefit from your education in this college?”
was used in a research study to understand the experiences of first-year students in the
college. Such leading questions not only direct students to focus only on the benefits (as
opposed to an evaluation of both benefits and limitations) of their educational experiences,
but also limit the scope of college experiences to educational experiences only.
SU5-12
HBC201
Qualitative Research Methods I
Read
Ayres, L. (2012). Semi-structured interview. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412963909.n420
2.3 Pros and Cons of Interview
The semi-structured interview is one of the most popular data collection methods
implemented in qualitative studies as it is relatively easy and cost-effective for researchers
to use (Cook, 2012). Nonetheless, the semi-structured interview, along with other
interview techniques, has the following limitations (Cook, 2012):
• Using the interview method, researchers do not have the opportunity to observe
actual behaviours of one or various member(s) involved in a phenomenon of
interest and can only rely on the recall of interviewees to obtain information.
• The success of an interview also depends on a researcher’s ability to elicit rich and
relevant information from an interviewee, as well as an interviewee’s ability to
accurately recall and clearly describe his or her experiences related to the topic of
interest.
SU5-13
HBC201
Qualitative Research Methods I
Case Study: Local Food Movements
Kennedy, E. H. (2018). Using semi-structured interviews to identify the place and
prominence of shopping for change in local food movements. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526429391
In this case study, the researcher described how the semi-structured interview is used to
study the consumer and civic activities, as well as participants’ motivations, in the local
food movements in various Canadian cities. The researcher provided an excellent account
of the research activities in this qualitative study, particularly the following (Kennedy,
2018):
• Formulation of the research questions, specifically how this study fills gaps in the
existing literature and why the topic of this study is on food [see Making a Better
Food System in Canada: Project Overview and Context section].
• Criteria for selection of research sites for data collection, such as selecting sites
where the researcher has personal connections, selecting sites that vary in the
phenomenon of interest, and so on [see Research Design section].
• The use of purposive and snowball sampling techniques [see Research Design
section].
• The development and evolvement of interview questions [see Method in Action
section].
• Step-by-step instructions on how to set up an interview, how to conduct a semistructured interview, and what can be done right after an interview [see Practical
Lessons Learned section].
SU5-14
HBC201
Qualitative Research Methods I
Activity 5.1
Kennedy, E. H. (2018). Using semi-structured interviews to identify the place and
prominence of shopping for change in local food movements. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526429391
What actions did the researcher take to obtain data that were suitable to study the
phenomenon of interest in this case study (Kennedy, 2018)?
SU5-15
HBC201
Qualitative Research Methods I
Chapter 3: Politics of Research
It has been argued in social sciences that “all research is political insomuch as it comes
out of a particular view of the world, makes claims about reality, and supports or refutes
existing knowledge claims” (Joshee, 2012, para. 1). Table 5.2 summarises three interrelated
topics that typically occur in discussions of politics of research (Joshee, 2012).
Table 5.2 Politics of Research
Topic
Qualitative
Description
versus
Quantitative Debate
Qualitative and quantitative research differ in various
aspects, such as underlying philosophy about what reality
is, approach to study a question, and so on. Such differences
have raised debates about the benefits and limitations of each
research orientation.
Relationship between
The discussion of this topic is sometimes referred to as “the
Researcher
politics of the gaze,” which is concerned with questions like
Researched
and
who has more power, what influences the dynamic may have
on individual(s) with less power, and so on in social science
research (Joshee, 2012, Relationship between Researcher and
Researched section).
Evidence-Based
Evidence-based research is a term that has been used by
Research
policy makers and funding agencies in various Western
and Northern countries since the 1990s. In the advocation
for evidence-based research, quantitative studies are often
SU5-16
HBC201
Qualitative Research Methods I
Topic
Description
regarded as more valuable and superior than qualitative
studies by policy makers and funding agencies.
(Source: Joshee, 2012)
Read
Joshee, R. (2012). Politics of qualitative research. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412963909.n326
Activity 5.2
Kennedy, E. H. (2018). Using semi-structured interviews to identify the place and
prominence of shopping for change in local food movements. SAGE Research
Methods Cases.
https://doi.org/10.4135/9781526429391
What research politics issues could the researcher possibly encounter in the process
of this study (Kennedy, 2018)?
SU5-17
HBC201
Qualitative Research Methods I
Summary
We have gained more understanding about qualitative research methods from the three
chapters in this unit:
• In Chapter 1, we have outlined common considerations researchers have
in conducting qualitative studies: the fundamental concepts (the emic/etic
perspectives and the thick description), sampling strategies, and criteria to evaluate
the rigour of qualitative studies.
• In Chapter 2, we have described the unstructured interview and semi-structured
interview in details.
• In Chapter 3, we have discussed concerns about politics of research in social
sciences.
SU5-18
HBC201
Qualitative Research Methods I
Formative Assessment
1.
Which of the following statements is correct?
a. Etic perspective shows a research participant’s view of reality.
b. Emic perspective shows a researcher’s external view of reality.
c. A qualitative researcher typically collects data from the emic perspective.
d. A qualitative researcher should analyse data solely from the emic perspective.
2.
A qualitative researcher typically delineates his or her observation of a behaviour or
event in great details. This is called ________.
a. Emic perspective
b. Etic perspective
c. Thick description
d. None of the above.
3.
Which of the following statements is correct?
a. Qualitative researchers typically finish recruitment of participants before data
collection starts.
b. Typically, qualitative researchers randomly select participants for their
research.
c. Qualitative researchers may need to intentionally recruit special cases that
deviate from the pattern identified in the analysis of the existing data.
d. Qualitative researchers should use data from all participants.
4.
Which of the following is the correct description of the transparency criterion in the
evaluation of the rigour of a qualitative study?
a. A qualitative researcher should delineate how he or she conducts a study.
b. A qualitative researcher should double check findings of his or her own
research with the research participants.
SU5-19
HBC201
Qualitative Research Methods I
c. A qualitative researcher can ask a different person to code the data, and check
if similar findings are obtained.
d. A qualitative researcher should reflect on his or her impact on the findings of
a study.
5.
Which of the following is correct?
a. The same reliability concepts apply to both quantitative and qualitative
research.
b. The same validity concepts apply to both quantitative and qualitative research.
c. The dependability criterion in qualitative research is similar to the reliability
concept in quantitative research.
d. The credibility criterion in qualitative research is similar to the reliability
concept in quantitative research.
6.
An in-depth interview typically refers to a ______ interview.
a. Unstructured
b. Semi-structured
c. Structured
d. Group interview
7.
Which of the following statements about unstructured interview is incorrect?
a. A qualitative researcher usually begins with a broad question in the openended form in the unstructured interview.
b. Interviewee’s responses typically determine the direction of an unstructured
interview.
c. Unstructured interview is commonly used by researchers to explore a new and
unfamiliar topic.
d. None of the above.
8.
Which of the following statements about semi-structured interview is correct?
SU5-20
HBC201
Qualitative Research Methods I
a. A researcher typically follows a list of pre-determined questions strictly in a
semi-structured interview.
b. A researcher typically follows the lead of the interviewee freely in a semistructured interview.
c. A researcher should not ask questions that prompt an interviewee to provide
a certain type of responses in a semi-structured interview.
d. A researcher typically uses semi-structured interview to explore an unfamiliar
topic.
9.
What is the limitation of a semi-structured interview?
a. A researcher does not observe a participant’s actual behaviours and only relies
on the participant’s responses.
b. The quality of an interview is influenced by a researcher’s interview skill.
c. The quality of an interview is influenced by how accurately and articulately
an interviewee describes his or her experiences or thoughts.
d. All of the above.
10. Which of the following statements is correct?
a. All research is objective.
b. Quantitative research is superior to qualitative research.
c. In the call for evidence-based research, third parties (such as funding agencies)
prefer qualitative research.
d. Quantitative and qualitative research both have their benefits and limitations.
SU5-21
HBC201
Qualitative Research Methods I
Solutions or Suggested Answers
Formative Assessment
1.
Which of the following statements is correct?
a.
Etic perspective shows a research participant’s view of reality.
Incorrect. This is the definition of emic perspective.
b.
Emic perspective shows a researcher’s external view of reality.
Incorrect. This is the definition of etic perspective.
c.
A qualitative researcher typically collects data from the emic perspective.
Correct.
d.
A qualitative researcher should analyse data solely from the emic
perspective.
Incorrect. A qualitative researcher typically analyses data from both emic and
etic perspectives.
2.
A qualitative researcher typically delineates his or her observation of a behaviour or
event in great details. This is called ________.
a.
Emic perspective
Incorrect. Emic perspective is a research participant’s (i.e. an insider’s) view
of reality.
b.
Etic perspective
Incorrect. Etic perspective is a researcher’s external view of reality.
c.
Thick description
Correct.
d.
None of the above.
SU5-22
HBC201
Qualitative Research Methods I
Incorrect.
3.
Which of the following statements is correct?
a.
Qualitative researchers typically finish recruitment of participants before
data collection starts.
Incorrect. Qualitative researchers oftentimes continue selecting participants
as needed in the process of their research projects.
b.
Typically, qualitative researchers randomly select participants for their
research.
Incorrect. Typically, qualitative researchers intentionally select participants
for their research.
c.
Qualitative researchers may need to intentionally recruit special cases that
deviate from the pattern identified in the analysis of the existing data.
Correct.
d.
Qualitative researchers should use data from all participants.
Incorrect. Qualitative researchers may decide to exclude data from specific
participants.
4.
Which of the following is the correct description of the transparency criterion in the
evaluation of the rigour of a qualitative study?
a.
A qualitative researcher should delineate how he or she conducts a study.
Correct.
b.
A qualitative researcher should double check findings of his or her own
research with the research participants.
Incorrect. This is the credibility criterion.
c.
A qualitative researcher can ask a different person to code the data, and check
if similar findings are obtained.
SU5-23
HBC201
Qualitative Research Methods I
Incorrect. This is the dependability criterion.
d.
A qualitative researcher should reflect on his or her impact on the findings
of a study.
Incorrect. This is the reflexivity criterion.
5.
Which of the following is correct?
a.
The same reliability concepts apply to both quantitative and qualitative
research.
Incorrect. Due to differences in philosophy underlying quantitative and
qualitative research, some qualitative researchers stated that certain
reliability concepts (such as replicability of results) that are established for
quantitative research are not suitable for qualitative research.
b.
The same validity concepts apply to both quantitative and qualitative
research.
Incorrect. Due to differences in philosophy underlying quantitative and
qualitative research, some qualitative researchers stated that certain validity
concepts (such as accuracy in capturing a construct of interest) that are
established for quantitative research are not suitable for qualitative research.
c.
The dependability criterion in qualitative research is similar to the reliability
concept in quantitative research.
Correct.
d.
The credibility criterion in qualitative research is similar to the reliability
concept in quantitative research.
Incorrect. The credibility criterion in qualitative research is similar to the
validity concept in quantitative research.
6.
An in-depth interview typically refers to a ______ interview.
a.
Unstructured
SU5-24
HBC201
Qualitative Research Methods I
Incorrect. An unstructured interview is sometimes referred to as an open
interview.
b.
Semi-structured
Correct.
c.
Structured
Incorrect. An in-depth interview uses a combination of structured and open
forms.
d.
Group interview
Incorrect. An in-depth interview is typically an individual interview.
7.
Which of the following statements about unstructured interview is incorrect?
a.
A qualitative researcher usually begins with a broad question in the openended form in the unstructured interview.
Incorrect.
b.
Interviewee’s responses typically determine the direction of an unstructured
interview.
Incorrect.
c.
Unstructured interview is commonly used by researchers to explore a new
and unfamiliar topic.
Incorrect.
d.
None of the above.
Correct.
8.
Which of the following statements about semi-structured interview is correct?
a.
A researcher typically follows a list of pre-determined questions strictly in a
semi-structured interview.
SU5-25
HBC201
Qualitative Research Methods I
Incorrect. A researcher typically follows a list of pre-determined questions
strictly in a structured interview.
b.
A researcher typically follows the lead of the interviewee freely in a semistructured interview.
Incorrect. A researcher typically prepares some questions in advance and has
some control on the direction of the interview in a semi-structured interview.
c.
A researcher should not ask questions that prompt an interviewee to provide
a certain type of responses in a semi-structured interview.
Correct.
d.
A researcher typically uses semi-structured interview to explore an
unfamiliar topic.
Incorrect. A researcher typically uses unstructured interview to explore an
unfamiliar topic.
9.
What is the limitation of a semi-structured interview?
a.
A researcher does not observe a participant’s actual behaviours and only
relies on the participant’s responses.
Correct.
b.
The quality of an interview is influenced by a researcher’s interview skill.
Correct.
c.
The quality of an interview is influenced by how accurately and articulately
an interviewee describes his or her experiences or thoughts.
Correct.
d.
All of the above.
Correct.
10. Which of the following statements is correct?
SU5-26
HBC201
Qualitative Research Methods I
a.
All research is objective.
Incorrect. Third parties may interfere with the process of research projects.
b.
Quantitative research is superior to qualitative research.
Incorrect. Each research orientation has its benefits and limitations.
c.
In the call for evidence-based research, third parties (such as funding
agencies) prefer qualitative research.
Incorrect. In the call for evidence-based research, third parties (such as
funding agencies) prefer quantitative research.
d.
Quantitative and qualitative research both have their benefits and
limitations.
Correct.
SU5-27
HBC201
Qualitative Research Methods I
References
Ayres, L. (2012). Semi-structured interview. In L. M. Given (Ed.), The SAGE encyclopedia
of qualitative research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963909.n420
Cook, K. E. (2012). In-depth interview. In L. M. Given (Ed.), The SAGE encyclopedia of
qualitative research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963909.n209
Fetterman, D. M. (2012). Emic/etic distinction. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc. Retrieved from
https://doi.org/10.4135/9781412963909.n130
Firmin, M. W. (2012). Unstructured interview. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc. Retrieved from
https://doi.org/10.4135/9781412963909.n475
Hammersley, M. (2011). Thick description. In M. S. Lewis-Beck, A. Bryman, & T.
Futing Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n1013
Joshee, R. (2012). Politics of qualitative research. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods. SAGE Publications, Inc. Retrieved from
https://doi.org/10.4135/9781412963909.n326
Kennedy, E. H. (2018). Using semi-structured interviews to identify the place and
prominence of shopping for change in local food movements. SAGE Research
Methods Cases. Retrieved from https://doi.org/10.4135/9781526429391
Morse, J. M. (2011). Sampling in qualitative research. In M. S. Lewis-Beck, A. Bryman, &
T. Futing Liao (Eds.), The SAGE encyclopedia of social science research methods. SAGE
Publications, Inc. Retrieved from https://doi.org/10.4135/9781412950589.n885
SU5-28
HBC201
Qualitative Research Methods I
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526429872
Saumure, K., & Given, L. M. (2012). Rigor in qualitative research. In L. M. Given (Ed.),
The SAGE encyclopedia of qualitative research methods. SAGE Publications, Inc.
Retrieved from https://doi.org/10.4135/9781412963909.n409
Smith, J. K. (2011). Reliability and validity in qualitative research. In M. S. LewisBeck, A. Bryman, & T. Futing Liao (Eds.), The SAGE encyclopedia of social
science research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412950589.n846
SU5-29
HBC201
Qualitative Research Methods I
SU5-30
Study
Unit
Qualitative Research Methods II
6
HBC201
Qualitative Research Methods II
Learning Outcomes
By the end of this unit, you should be able to:
1.
Describe narrative research.
2.
Describe focus group research.
3.
Describe action research.
4.
Explain key concepts in ethnography.
5.
Compare different types of techniques used in ethnography.
6.
Discuss ethical issues in ethnography.
SU6-2
HBC201
Qualitative Research Methods II
Overview
I
n this unit, we will discuss qualitative research in greater details. We will first
examine the various types of qualitative studies. This will be followed by a
deliberation on ethnography, which is one of the most well-known qualitative research
methods.
SU6-3
HBC201
Qualitative Research Methods II
Chapter 1: Major Types of Qualitative Research
Lesson Recording
Major Types of Qualitative Research
There are various types of qualitative research as shown in the Methods Map on the SAGE
research methods website (see http://methods.sagepub.com/methods-map/qualitativedata-collection). In this chapter, we will focus on three types of qualitative research that are
common in social sciences: narrative research, focus group research, and action research.
1.1 Narrative Research
In narrative research projects, researchers typically ask participants to “describe in
detail―tell the story of―either a particular event or a significant aspect or time of life
(e.g. a turning point)” or to “narrate an entire life story” (Josselson, 2012, Life is a Story
section). Like other types of qualitative research, narrative research has the following main
characteristics (Josselson, 2012):
• Researchers are interested in how participants perceive and organise their life
events (which may be subjective) instead of pursuing factuality or objectivity.
• The focus of narrative research is to understand the process through which
participants construct their experiences in certain contexts. Unlike studies that
employ hypothesis testing, generalising findings to a larger population is not of
interest in narrative research.
• Researchers’ interpretations, instead of statistical analysis, play an important role
in the analysis of narrative research data.
SU6-4
HBC201
Qualitative Research Methods II
Narrative researchers typically conduct individual interviews with participants who
meet the study criteria. Depending on the research purpose, researchers may sometimes
use group interviews or include secondary data (such as diaries) as well. During
interviews, participants are encouraged to elaborate on experiences related to the topic
of interest. Researchers record and subsequently transcribe interviews for subsequent
analysis (Josselson, 2012).
Read
Josselson, R. (2012). Narrative research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n259
Case Study: Identity and Mixed Race
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526429872
As shown in previous study units, this case study outlined a narrative research project
that investigated the identity of mixed-race people in the Singapore context. Please reflect
on the following (Rocha, 2018):
• The features of narrative research and why narrative enquiry was suitable for
studying the researcher’s question [see Research and Narrative section].
• The interview process [see Narrative Enquiry: Interviews in Action section].
• The transcription and analysis of the data [see Moving from Conversation to Text
section].
SU6-5
HBC201
Qualitative Research Methods II
1.2 Focus Group Research
In a focus group study, a researcher typically moderates a discussion among a group of
selected participants about the topic of interest. The focus group method has been widely
applied in different settings, such as in informal book club meetings where participants
get together to discuss their thoughts about a chosen book, in marketing research where
researchers gather feedback from selected participants on a specific product of interest,
and so on (Grumbein & Lowe, 2012).
A focus group study usually involves various elements as follows (Grumbein & Lowe,
2012):
• A researcher first determines the purpose of the focus group study.
• The researcher determines the moderator and selects participants of the focus
group.
• The researcher determines the number of focus group sessions needed as well as
other logistics (such as venue(s) for the focus group session(s)).
• The researcher prepares a guide for the moderation of the focus group session(s).
• The researcher conducts and records the focus group session(s) for subsequent
analysis.
Read
Grumbein, M. J., & Lowe, P. A. (2012). Focus group. In N. J. Salkind (Ed.),
Encyclopedia of research design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n156
SU6-6
HBC201
Qualitative Research Methods II
Case Study: Meaning of Colors in Nutrition Labeling
Styśko-Kunkowska, M., Wąsowicz, G., & Grunert, K. G. (2018). Gaining wealth and
deep insight into meaning of colors in nutrition labeling using an extended focus
group interview. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526455970
In this case study, the researchers described their research on people’s interpretation of
colours in nutrition labelling of different types of healthy and unhealthy food products.
Specifically, the researchers described the use of focus group method in their qualitative
study as follows (Styśko-Kunkowska et al., 2018):
• The researchers described their rationale for conducting this qualitative study first
before they proceeded to quantitative studies, and explained their choice of the
extended focus group interview method [see Research Practicalities section].
• The researchers then described the criteria and process used to recruit participants,
the preparation for the focus group session (such as the setup of the facility, the
development of a discussion guide, etc.), how the moderator conducted the focus
group session, and how they made sense of the data from the focus group session
[see Methods in Action section].
• The researchers also discussed benefits (such as collecting data efficiently) and
challenges (such as analysing the rich amount of data) of the method [see Practical
Lessons Learned section].
SU6-7
HBC201
Qualitative Research Methods II
Activity 6.1
Please discuss the following questions:
1.
According to Grumbein and Lowe (2012), what are the benefits and
limitations of the focus group method?
2.
What strategies did the researchers use in the “Meaning of Colors in
Nutrition Labeling” study (Styśko-Kunkowska et al., 2018) to address the
potential limitations of the focus group method?
1.3 Action Research
Action research is listed under “qualitative data collection” in the Methods map on
the SAGE research methods website (see http://methods.sagepub.com/methods-map/
qualitative-data-collection); it is thus classified as qualitative research here. In practice,
action researchers may use triangulation strategies (see Study Unit 2 for the description
of triangulation) in their studies (Adams, 2012).
In action research, researchers typically collaborate with participants to find solutions to
problems of interest. Action research usually has the following characteristics (Adams,
2012):
• Reflection: Both researchers and participants ruminate on various aspects of action
research projects, such as the problems at hand, possible interventions, and so on,
• Iterancy: Action research typically consists of multiple iterations, during which
“participants engage in successive cycles of collecting and making sense of data
until agreement is reached on appropriate action” (Adams, 2012, Iterancy section).
• Collaboration: Researchers and participants in action research are typically
perceived as collaborative partners. While external researchers are expected to
contribute their professional expertise to the projects, participants are expected to
SU6-8
HBC201
Qualitative Research Methods II
engage in the research procedure as well, rather than merely serve as the source
of data.
• Role of the Researcher: Researchers may take on various roles in action research,
depending on the degree of their involvement in the activities under investigation.
Nonetheless, researchers are typically “full participants” as well in action research
(Adams, 2012, Role of the Researcher section).
Read
Adams, P. (2012). Action research. In N. J. Salkind (Ed.), Encyclopedia of research
design. SAGE Publications, Inc.
https://doi.org/10.4135/9781412961288.n3
Case Study: Disadvantaged Citizens as Co-Researchers
Rogers, D., Arthurson, K., & Darcy, M. (2014). Disadvantaged citizens as co-researchers
in media analysis: Action research utilising mobile phone and video diaries.
SAGE Research Methods Cases.
https://doi.org/10.4135/978144627305013509351
In this case study, the researchers described an action research project in which
disadvantaged citizens worked as co-researchers with academic scholars to study the
depiction of disadvantaged social housing tenants in a television series in Australia.
Specifically, the researchers delineated the following aspects of their research (Rogers et
al., 2014):
• Description of the television series and the background of this action research
project [see Introduction section].
SU6-9
HBC201
Qualitative Research Methods II
• The purpose and process of action research in theory and how this project was
implemented in practice [see Action Research: An ‘Expert’ Panel Discussion to Set
the Scope of the Research section].
• The rationale and practice of asking participants to respond to a number of research
questions in the form of a written, audio, or video diary after watching each weekly
episode of the television series [see Mobile Phone Video Diaries: The Remote
Capture of Real-Time Data by Co-Researchers section].
• The use of the focus group method for project participants to make sense of the
collected data and the use of the findings from this project [see Focus Groups:
Verifying the Research Findings with the Tenant ‘Experts’ section].
SU6-10
HBC201
Qualitative Research Methods II
Chapter 2: Ethnography
Lesson Recording
Ethnography
Ethnography is one of the most prominent types of qualitative research. In this chapter,
we will learn about ethnography in three main aspects: its key concepts, key techniques,
and ethics.
2.1 Key Concepts in Ethnography
The focus of ethnography is to understand and describe a particular culture or group.
Ethnographers typically conduct their research through fieldwork. Table 6.1 shows key
concepts in ethnographic research (Fetterman, 2012).
Table 6.1 Key Concepts in Ethnography
Concept
Description
Culture
An ethnographer needs to understand behaviours and
knowledge in the specific culture under study and be able to
describe his or her observations from the perspective of the
studied group.
Holistic Perspective/
An ethnographer should strive to capture the studied culture
Contextualisation
or group as comprehensively as possible.
Emic/Etic
An ethnographer typically adopts the emic (insider’s)
Perspective
perspective in the data collection stage, and subsequently
SU6-11
HBC201
Qualitative Research Methods II
Concept
Description
both the emic and the etic (external social scientist’s)
perspectives in the data analysis stage.
• Nonjudgmental orientation: An ethnographer
Nonjudgmental
should “suspend personal valuation of any given
Orientation/
Intercultural
and
Intracultural
Diversity
cultural practice” (Fetterman, 2012, Nonjudgmental
Orientation and Inter- and Intracultural Diversity
section).
• Intercultural diversity: Differences between any two
cultures of interest.
• Intracultural diversity: Differences between any two
subcultures that exist within a specific culture.
Symbol and Ritual
Symbols are defined as “condensed expressions of meaning
that evoke powerful feelings and thoughts”, while rituals are
defined as “repeated patterns of symbolic behaviour that play
a part in both religious and secular lives” (Fetterman, 2012,
Symbols section).
(Source: Fetterman, (2012)
2.2 Key Techniques in Ethnography
Ethnographers typically study cultures or groups of interest in the natural setting for an
extended period of time. Table 6.2 summarises techniques that ethnographers commonly
use in their research (Fetterman, 2012).
SU6-12
HBC201
Qualitative Research Methods II
Table 6.2 Key Techniques in Ethnography
Technique
Description
Participant Observation
Ideally, ethnographers should stay in the field for extended
time to learn the language and observe behaviours of the
groups under study.
Interview
Ethnographers use informal (unstructured) interviews
most of the time.
Informant Interview
Informants (or key actors) are “more articulate and
culturally sensitive” individuals in the culture or group
under study (Fetterman, 2012, Key Actor or Informant
Interviewing section). Informants are important source of
information for ethnographers.
Questionnaire
Questionnaires can be used to collect data from a group of
people within a relatively short period of time.
Unobtrusive Measure
Ethnographers can collect data through unobtrusive ways,
such as observing physical evidence (e.g. graffiti) in the
natural settings.
(Source: Fetterman, (2012)
2.3 Ethics in Ethnography
In general, ethnographers follow the ethical guidelines discussed in Study Unit 1. Specific
ethical issues that are commonly addressed in ethnographic research include the following
(Fetterman, 2012):
• Ethnographers should respect the culture or group under study.
• Ethnographers should obtain informed consent to conduct their research.
SU6-13
HBC201
Qualitative Research Methods II
Fieldwork plays an important role in ethnographic research. Respecting the culture and
building trust with the group of people under study is not only a matter of research ethics,
but also influences the quality of data in ethnographic research (Fetterman, 2012).
Read
Fetterman, D. M. (2012). Ethnography. In L. M. Given (Ed.), The SAGE encyclopedia of
qualitative research methods. SAGE Publications, Inc.
https://doi.org/10.4135/9781412963909.n150
Case Study: Mobile Virtual Ethnography
Molz, J. G. (2018). Mobile virtual ethnography: Studying interactive travelers online and
on-the-move. SAGE Research Methods Cases.
https://doi.org/10.4135/9781526427236
In this case study, the researcher described the use of mobile virtual ethnography to study
interactive travel in three forms: couchsurfing, travel blogging, and worldschooling. The
researcher described the following aspects of mobile virtual ethnographic research (Molz,
2018):
• The paradigm that inspires mobile virtual ethnography [see The Mobilities
Paradigm section].
• The characteristics of traditional ethnography and mobile virtual ethnography [see
Mobilizing Ethnographic Techniques section].
• The research process to study each of the three forms of interactive travel
(i.e. couchsurfing, travel blogging, and worldschooling) respectively [see Mobile
Virtual Ethnography in Action section].
• The nature of mobile virtual ethnography [see Following Connectivity: Rethinking
the Ethnographic Field section].
SU6-14
HBC201
Qualitative Research Methods II
• The various issues in conducting mobile virtual ethnographic research. The
discussions about ethical issues and researcher safety are worth special attention
[see Practical Considerations section].
Activity 6.2
Based on the case study (Molz, 2018), compare the similarities and differences between
mobile virtual ethnographic research and traditional ethnographic research.
Reflect 6.1
We have discussed different types of qualitative research in this unit. Is there a
particular type of research design that is suitable for studying your research question?
SU6-15
HBC201
Qualitative Research Methods II
Summary
We have addressed qualitative research in greater details in the following chapters in this
unit:
• In Chapter 1, we have described the three types of qualitative research methods that
are commonly used: narrative research, focus group research, and action research.
• In Chapter 2, we have focused on ethnography, one of the most important
qualitative research methods. We have examined various aspects of ethnographic
research, including its key concepts, main techniques, and ethical issues.
SU6-16
HBC201
Qualitative Research Methods II
Formative Assessment
1.
Which of the following describes narrative research correctly?
a. The goal of narrative research is to pursue factuality.
b. Researchers are typically interested in generalising findings of narrative
research to larger populations.
c. Statistical analysis is typically used in the analysis of data in narrative research.
d. Individual interview is typically used in narrative research.
2.
A marketing researcher gathered a group of people who were selected from the
target customers and collected their thoughts on a new cellphone through a group
discussion; what is the type of research used by the researcher?
a. Narrative research.
b. Focus group research.
c. Unstructured interview.
d. Semi-structured interview.
3.
In ______, a researcher usually works together with research participants to
understand an issue or identify a solution.
a. Narrative research.
b. Focus group research.
c. Action research.
d. Ethnography.
4.
Which of the following about action research is incorrect?
a. In action research, both researchers and participants reflect on an issue of
interest.
b. In action research, participants are more than data providers; both participants
and researchers are collaborators.
SU6-17
HBC201
Qualitative Research Methods II
c. In action research, researchers can participate in the events under study.
d. None of the above.
5.
If a researcher would like to obtain a deep understanding of the youth culture in
Singapore, which type of research will be most suitable?
a. Action research.
b. Narrative research.
c. Focus group research.
d. Ethnography
6.
In ethnographic research, a researcher should avoid criticising participants’
behaviours or cultural events under study. This is the concept of _________ in
ethnography.
a. Holistic perspective.
b. Emic perspective.
c. Nonjudgmental orientation.
d. Etic perspective.
7.
Which of the following statements is correct?
a. Intercultural diversity refers to potential difference(s) between subcultures of
a given culture.
b. Intracultural diversity refers to potential difference(s) between cultures.
c. An ethnographer can freely describe a culture or a group from his or her own
perspective.
d. It is common for an ethnographer to conduct his or her own study via
fieldwork.
8.
Common technique(s) used in ethnographic research include:
a. Participant observation.
b. Informant interview.
SU6-18
HBC201
Qualitative Research Methods II
c. Unobtrusive measure.
d. All of the above.
9.
Which of the following is correct?
a. In ethnographic research, a researcher usually does not spend much time on
the fieldwork.
b. It is common for an ethnographer to use structured interview in the research.
c. It is common for ethnographers to obtain important data from informants.
d. Observing graffiti is considered an obtrusive way to collect data in
ethnographic research.
10. Which of the following is correct?
a. An ethnographer does not need to respect a culture he or she studies.
b. An ethnographer is not obligated to obtain informed consent from
participants.
c. An ethnographer should build trust with a group of people he or she studies,
in consideration of research ethics.
d. An ethnographer should build trust with a group of people he or she studies, in
consideration of research ethics as well as in the interest of the quality of data.
SU6-19
HBC201
Qualitative Research Methods II
Solutions or Suggested Answers
Formative Assessment
1.
Which of the following describes narrative research correctly?
a.
The goal of narrative research is to pursue factuality.
Incorrect. Narrative research is about how participants under study perceive
their experiences, which can be subjective.
b.
Researchers are typically interested in generalising findings of narrative
research to larger populations.
Incorrect. Generalising findings of a study to a larger population is not the
goal of researchers conducting narrative studies.
c.
Statistical analysis is typically used in the analysis of data in narrative
research.
Incorrect. Data analysis in narrative research is largely based on researchers’
interpretations.
d.
Individual interview is typically used in narrative research.
Correct.
2.
A marketing researcher gathered a group of people who were selected from the
target customers and collected their thoughts on a new cellphone through a group
discussion; what is the type of research used by the researcher?
a.
Narrative research.
Incorrect. Narrative research focuses on participants’ perceptions of their life
experiences and is usually conducted via individual interview.
b.
Focus group research.
Correct.
SU6-20
HBC201
Qualitative Research Methods II
c.
Unstructured interview.
Incorrect. Unstructured interview is typically conducted individually rather
than as a group.
d.
Semi-structured interview.
Incorrect. Semi-structured interview is typically conducted individually
rather than as a group.
3.
In ______, a researcher usually works together with research participants to
understand an issue or identify a solution.
a.
Narrative research.
Incorrect. In narrative research, researchers and participants do not work
together as partners.
b.
Focus group research.
Incorrect. In focus group research, researchers and participants do not work
together as partners.
c.
Action research.
Correct.
d.
Ethnography.
Incorrect. In ethnography, researchers and participants do not work together
as partners.
4.
Which of the following about action research is incorrect?
a.
In action research, both researchers and participants reflect on an issue of
interest.
Incorrect.
b.
In action research, participants are more than data providers; both
participants and researchers are collaborators.
SU6-21
HBC201
Qualitative Research Methods II
Incorrect.
c.
In action research, researchers can participate in the events under study.
Incorrect.
d.
None of the above.
Correct.
5.
If a researcher would like to obtain a deep understanding of the youth culture in
Singapore, which type of research will be most suitable?
a.
Action research.
Incorrect. The focus of action research is for researchers and participants
to work together on understanding an issue or identifying a solution to a
problem.
b.
Narrative research.
Incorrect. In narrative research, researchers do not observe participants’
actual behaviours or contexts of interest.
c.
Focus group research.
Incorrect. In focus group research, researchers do not observe participants’
actual behaviours or contexts of interest.
d.
Ethnography
Correct. Researchers commonly conduct ethnographic studies to obtain a
deep understanding of a culture or group.
6.
In ethnographic research, a researcher should avoid criticising participants’
behaviours or cultural events under study. This is the concept of _________ in
ethnography.
a.
Holistic perspective.
SU6-22
HBC201
Qualitative Research Methods II
Incorrect. Holistic perspective is about delineating a culture or group under
study as comprehensively as possible.
b.
Emic perspective.
Incorrect. Emic perspective is the perspective of an insider.
c.
Nonjudgmental orientation.
Correct .
d.
Etic perspective.
Incorrect. Etic perspective is the perspective of an external researcher.
7.
Which of the following statements is correct?
a.
Intercultural diversity refers to potential difference(s) between subcultures
of a given culture.
Incorrect. Intercultural diversity refers to potential difference(s) between
cultures.
b.
Intracultural diversity refers to potential difference(s) between cultures.
Incorrect. Intracultural diversity refers to potential difference(s) between
subcultures of a given culture.
c.
An ethnographer can freely describe a culture or a group from his or her own
perspective.
Incorrect. It is important for an ethnographer to delineate a culture or group
from the perspective of insiders.
d.
It is common for an ethnographer to conduct his or her own study via
fieldwork.
Correct.
8.
Common technique(s) used in ethnographic research include:
a.
Participant observation.
SU6-23
HBC201
Qualitative Research Methods II
Correct. See Table 6.2 in this unit.
b.
Informant interview.
Correct. See Table 6.2 in this unit.
c.
Unobtrusive measure.
Correct. See Table 6.2 in this unit.
d.
All of the above.
Correct.
9.
Which of the following is correct?
a.
In ethnographic research, a researcher usually does not spend much time on
the fieldwork.
Incorrect. An ethnographer usually spends a long time on the fieldwork.
b.
It is common for an ethnographer to use structured interview in the research.
Incorrect. It is common for an ethnographer to use informal or unstructured
interview in the research.
c.
It is common for ethnographers to obtain important data from informants.
Correct.
d.
Observing graffiti is considered an obtrusive way to collect data in
ethnographic research.
Incorrect. Observing graffiti is considered an unobtrusive way to collect data
in ethnographic research.
10. Which of the following is correct?
a.
An ethnographer does not need to respect a culture he or she studies.
Incorrect. An ethnographer needs to respect a culture he or she studies.
b.
An ethnographer is not obligated to obtain informed consent from
participants.
SU6-24
HBC201
Qualitative Research Methods II
Incorrect. An ethnographer is obligated to obtain informed consent from
participants.
c.
An ethnographer should build trust with a group of people he or she studies,
in consideration of research ethics.
Incorrect. An ethnographer should build trust with a group of people he or
she studies, not only in consideration of research ethics but also for the sake
of data quality.
d.
An ethnographer should build trust with a group of people he or she studies,
in consideration of research ethics as well as in the interest of the quality of
data.
Correct.
SU6-25
HBC201
Qualitative Research Methods II
References
Adams, P. (2012). Action research. In N. J. Salkind (Ed.), Encyclopedia of
research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n3
Fetterman, D. M. (2012). Ethnography. In L. M. Given (Ed.), The SAGE encyclopedia of
qualitative research methods. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412963909.n150
Grumbein, M. J., & Lowe, P. A. (2012). Focus group. In N. J. Salkind (Ed.),
Encyclopedia of research design. SAGE Publications, Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n156
Josselson, R. (2012). Narrative research. In N. J. Salkind (Ed.), Encyclopedia
of research design. , SAGE Publications Inc. Retrieved from https://
doi.org/10.4135/9781412961288.n259
Molz, J. G. (2018). Mobile virtual ethnography: Studying interactive travelers online
and on-the-move. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526427236
Rocha, Z. L. (2018). Narrative enquiry and analysis: Exploring narratives of identity and
mixed race in Singapore. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526429872
Rogers, D., Arthurson, K., & Darcy, M. (2014). Disadvantaged citizens as coresearchers in media analysis: Action research utilising mobile phone
and video diaries. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/978144627305013509351
Styśko-Kunkowska, M., Wąsowicz, G., & Grunert, K. G. (2018). Gaining wealth and
deep insight into meaning of colors in nutrition labeling using an extended
SU6-26
HBC201
Qualitative Research Methods II
focus group interview. SAGE Research Methods Cases. Retrieved from https://
doi.org/10.4135/9781526455970
SU6-27
HBC201
Qualitative Research Methods II
SU6-28