Scientific Process and Research Design
Anne S. Tsui
Arizona State University
Peking University
Xi’an Jiaotung University
International Association for Chinese Management Research
Management and Organization Review
1
Topics of this Session
1.
2.
3.
4.
The scientific process
Types of empirical studies
Purposes of research design
Contribution to global management
knowledge
5. Research ethics and IACMR values
2
Readings for this session
Required readings:
Wallace, Walter. 1971. The Logic of science in sociology. Chicago:
Aldine, pp. 11-25.
IACMR Statement of “Commitment to Excellent”
Tsui, A.S. 2006. Contextualization in Chinese management research.
Management and Organization Review, 2(1): 1-13.
Supplemental readings:
Tsui, A.S., Jone L. Pearce, Lyman W. Porter, and Angela M. Tripoli.
1997. "Alternative Approaches to the Employee-Organization
Relationship: Does Investment in Employees Pay Off?” Academy
of Management Journal, 40(5): 1089-1121.
Xiao, Z.X. & Tsui, A. S. 2007. “Where brokers may not work: The
culture contingency of social capital”. Administrative Science
Quarterly, 53: 1-31.
Yan, A. & Gray. B. 1994. Bargaining power, management control, and
performance in the United States-China joint ventures: A
comparative case study. Academy of Management Journal, 37:
1478-1517.
3
Four sources of knowledge or truth
 Mythical
 Authoritative
 Logical
 Scientific
4
Goal of Science:
Seek truth or create knowledge
 Science is the pursuit of truth, of explanation,
prediction, and control a phenomenon.
 Truth obtained from the scientific method
contains both logic and evidence that are
consistent with each other.
 Science is about the creation of knowledge, not
the application of knowledge
 Science cannot settle debate about values (good
or bad, right or wrong, e.g. stem cell research) objectivity
 Ultimate goal of science is to better the human
condition
5
Methodology
The conduct of scientific inquiry
 Methodology is the science of finding out
knowledge (truth).
 The standards for determining truth is rigor in
logic and methods
 The scientific community determines the
standards of rigor and professional competence.
 Each scientist is accountable to the scientific
community for adhering to standards of
professional competence and norms.
 The visible part of our scientific conduct and
results is “reconstructed logic”
 The ethics of science is in the conduct of the
scientist.
6
Reality (Epistemology)
Three views in social sciences
 Premodern: Only one reality, no individual
experience matters or allowed.
 Modern: There is an objective reality (e.g.,
the temperature) but we have different
subjective experience of it (e.g., warm, cold).
Agreement in subjective experiences could
indicate objective reality.
 Postmodern: No objective reality, only
images of reality or subjective experiences,
and all are true. Agreement is neither
necessary or desirable to define reality.
7
Normal Science
based on the modern view of reality
 Normal science is empirical science – data, evidences
or observations are necessary to support theory.
 Determination of significant fact, matching of facts
with theory, and articulation of theory – basic
paradigm of normal science that we accept.
 Research based on shared paradigm is committed to
the same rules and standards for scientific practice.
 That commitment and the apparent consensus it
produces are both the genesis and the continuation
of a research tradition.
 Paradigms involve different assumptions of truth or
reality – normal science is based on a modern view of
reality.
8
The Domain of Organization and
Management Research
Inter-firm level: firm to firm, firm to environment
Firm level: Strategy, structure, culture, process
Group level: process, structure, dynamics
Individual level: attitudes, behavior, decision, perception
Cross-level: firm
firm
individual
group
individual
individual
group
firm
9
Commonly Used Research Methods in
Management





Survey Research （调查研究）
Laboratory experiments (实验室研究）
Quasi experiments （准实验）
Secondary data design (二手数据分析）
Qualitative and case methods (定性和个案研究）
10
Scientific Research
Scientific research is a systematic, controlled,
empirical, and critical investigation of natural
or social phenomena (using commonly
accepted methodology) that either (a) is
guided by theory and hypotheses about
the presumed (true) relations among such
phenomena, or b) results in theory and
propositions about the possible (true)
relationships among such phenomena.
11
Two Types of Empirical Research
 Theory testing empirical research




Surveys (interview, mail, internet, phone)
Experiments (university setting)
Quasi experiments (university or company)
Secondary data (financial, operational, personnel)
 Theory building empirical research
 Qualitative research (interview, observation, text)
 Case methods (interviews, text, secondary data)
12
A Simplified Scientific Research Process
and Types of Research
1. Research
Question
研究问题
Deductive research
演绎研究
2. Literature
review文献回顾
4. The empirical
study实证研究
Inductive research
归纳研究
Theoretical research
理论研究
3. Theory and
Hypotheses
理论和假设
Descriptive research
13
Elements of the Scientific Process
Concept formation
Propositions
Induction
形成概念、假设, 归纳
Theories
Logical
Deduction
Logical Inference
演绎逻辑
逻辑推理
Empirical
Generalization
实证概括
Measurement,
sample summarization,
parameter estimation
Accept or
reject Hs
Tests of
hypotheses
Observations
Hypotheses
假设
Research Design,
instrumentation,
scaling, sampling
From W.L. Wallace, The Logic of Science in Sociology, Figure 1
14
The Scientific Process - Review
Curiosity
or puzzle
Research
Question
Literature
review
Yes
Theory?
Deductive
Design
Observation
No
Inductive
Design
15
The Truth @@@###@@#
 Research is “messy”.
 The “reconstructed story” does not
reveal the “messiness” in the
“construction process”.
 Research is “perseverance” through
iterations of induction and deduction
 Thomas Edison – 2000 times before he
succeeded – 2000 step journey
16
Research design:
Purpose 1 - variance control
 Systematic variance系统变异
 Extraneous variance外生变异
 Error variance误差变异
A good research design should:
 maximize systematic variance,
 control extraneous variance, and
 minimize error variance
17
Approaches to variance control
 Maximize systematic variance
 Through good experimental control
 Through a strong theory
 Through systematic sampling
 Control extraneous variance
 Through randomization or matching of subjects
 Through including meaningful control variables
 Minimize error variance
 Through controlled conditions
 Through valid measurement
18
Variance Control in EOR study
(Tsui, et al 1997, AMJ)
 Systematic: Independent variable (EOR)
 Sample selection – multiple industries (5),
multiple firms (10) – 85 jobs
 Measurement of I.V. – 4 types at the job level
 QSC – 31%, UI – 18%, OI – 18%, MI –
33%
 Extraneous: Control variables
 individual, job, company – 8 total
 Error:
 Precision in measurement (EFA and CFA)
19
Variance Control in IJV control study
(Yan & Gray, 1994, AMJ)
 Systematic: Independent variable
(Management Control)
 Sample selection – theoretical sampling using
criteria of IJV age, size, industries, ownership
(4).
 Measurement or data – interviews and archives
 Extraneous:
 Through sampling – match of age, foreign
country, etc.
 Error:
 Triangulation of data from multiple sources
20
Variance control in social capital study
(Xiao & Tsui, 2007, ASQ)
 Systematic variance in the IV –
industry experts nominate firms that
vary on commitment
 Extraneous variance – through 9
control variables
 Error variance – through valid
measurement
21
Research Design:
Purpose 2 - Enhance Validity
Validity is the probable truth or falsity of an
assertion or inference. Four types of validity:
 Construct validity（构想效度）
 Present when measure produces results consistent
with alternative (valid) measures of same construct
 Internal validity（内部效度）
 Absence of alternative explanations
 External validity（外部效度）
 Present when control variables do not interact with
causal variables, i.e., results would hold at other
times, in other settings, and for other individuals
 Statistical conclusion validity（统计推论效度）
 Significant presence of co-variation between variables
22
Construct Validity - Does the variable
measure what it purports to measure?
 Content validity（内容效度）
 When the measure is absent of deficiency or
contamination
 Criterion-related validity（效标关联效度）
 When the measure relates to other constructs
as expected
 Convergent validity（会聚效度）
 When the measure relates to other measures
of the same construct
 Discriminant validity（区分效度）
 When the measure does not relate to other
constructs as expected.
23
Construct validity in EOR study
(Tsui et al., 1997, AMJ)
 1 independent and 7 dependent variables
 Assess internal consistency of multiple items for
the same construct (alpha = .76 to .96)
 Factor analysis to ensure that items load on the
intended factor (CFA  discriminant validity)
 Agreement between multiple assessment of EOR
dimensions in the same job (ANOVA F, R-2
= .33 and .50 for the two dimensions)
24
Construct validity in IJV study
(Yan & Gray, 1994, AMJ)
 Triangulation of data from multiple
sources
 Replication of discovered constructs
across cases
 Case examples to illustrate
constructs
25
Construct validity in the social capital
study (Xiao & Tsui, 1997, ASQ)
 3 main constructs: structural holes, high
commitment organization, career success
 2 of the 3 are based on current measures
 High commitment – firm level construct
measured at the individual level (n=88,
117, 102, 128 employees in four firms)
 EFA, Rwg=.83 to .84 (4 firms), ICC1=.14,
ICC2=.95
26
Internal Validity:
Confidence in inference,
absence of alternative explanations
Strong statistical validity
Strong theory (internal validity)
Strong controls (extraneous variance)
Valid measurement (construct validity)
Appropriate sample (external validity)
Strong inference of causation (internal
validity)
 No artificial covariance due to design or
attribution






27
Internal validity of EOR study
(Tsui et al., 1997, AMJ)
 Strong theoretical foundation
 No serious common method variance
problem
 Inclusion of 8 control variables
 Construct validity of EOR measure –
only 7 items (3 for contribution and 4
for inducement) - content deficiency?
28
Internal validity in IJV study
(Yan & Gray, 1994, AMJ)
 Replication of relationships in different
cases
 Consistency of patterns across cases
 Coders should be unaware of the initial
model or hypotheses
 Competing explanations (e.g., IJV changes)
are incorporated into the revised model
29
Internal validity in the social capital
study (Xiao & Tsui, 1997, ASQ)
Controlling for reverse causality
 Career data (DV) collected 6 months after IV
 Networks ties within performance period
excluded
 Using 9 control variables that may also
influence the dependent variable

3 ways to test the moderating hypothesis
 Subgroup analysis
 Interaction of firm level commitment score
 Interaction of individual level commitment score

30
Contributing to Global Management
Knowledge
(how to publish in international journals)
Address an interesting and important “puzzle”
Contribute to a “current conversation”
Connect to current theories and constructs
Use current normal science method
Select a type of international management
research
 Engage in appropriate “contextualization”





31
The “Puzzle”
Asking interesting questions
“Uninteresting” questions in any study:





Obvious questions
Irrelevant questions
Absurd questions
Definitional questions
Affirmation questions
“What to study in Chinese
Management research”
Tsui, Zhao, and Abrahamson,
MOR3.2
“Uninteresting” Chinese studies:




Replicating a published study
Uncritical application of an existing theory
Affirmation of similarities to Western samples
Outside-in and literature-driven approach in selecting
problems or research questions
(Davis, 1971)
32
The “Conversation” (Huff, 1999)
 What are the old and new topics in the
conversation?
 What are the current debates?
 How familiar are current conversationalists
about your topic and context (sample)?
 How to make the connection to the
unfamiliar?
 Why should “they” listen to you?
33
Connect to Current Theories
 Apply current theories
 Extend or modify
 Create and connect




Making the novel appear familiar
Making the familiar appear novel
Borrow and return something better
Vision limited by borrowed lens
Whetten, 2002; 2008
34
Contextualization and type of global
management knowledge (Tsui, 2004)
35
Use of Current Methods
 Meet criteria of validity – construct,
internal, external
 Contextualization of measurement
and data collection procedure
 Develop new methods of data
collection, measurement, or analyses
to supplement current methods
36
Contextualization (Tsui, 2006)
 The Phenomena – from “outside in” to
“inside out”
 The Theory – from “application” to
“creation”
 The Measurement – from “translation” to
“indigenization”
 The Methodology – from “sharpening old
tools” to “developing new instruments”
37
Types of international management
research
1. Pure international – studies of management in
a multinational context
2. Comparative or cross-cultural management –
studies of the behavior or decisions of firms
or of individuals within firms across different
cultural/national contexts
3. Country specific or indigenous management –
studies of firm or individual behavior within a
nation/culture
Tsui, Nifadker & Ou, 2007, JOM; Werner, 2002, JOM
38
IACMR Commitment to Excellence
Research Code of Ethics







Research integrity
Journal submission policy
Respect for research participants
Reviewer and editor responsibilities
Professional interaction and exchange
Conference participation
IACMR member responsibility
39
IACMR
Source of Inspiration
灵感之源
Spirit of Service
奉献精神
Commitment to Excellence
追求卓越
40
References
Academy of Management Journal award winning papers (book)
Administrative Science Quarterly award winning papers (book)
IACMR Statement of “Commitment to Excellent” on www.iacmr.org
Babbie, Earl R. 2004, 10th edition. The practice of social research. Belmont, CA:
Wadswroth/Thomson Learning.
Davis, Murray S. 1971. That is interesting! Phil. Soc. Sci., 309-344.
Huff, Anne. 1999. Writing for scholarly publications. Thousand Oaks, CA: Sage.
Kaplan, Abraham. 1964. The conduct of inquiry. San Francisco: Chandler.
Kuhn, Thomas. 1996. The structure of scientific revolution, 3rd edition. Chicago: University
of Chicago
Press. Popper, Karl R. 1968, 2nd edition. The logic of scientific discovery. New York: Harper.
Tsui, A.S. 2004 . Contributing to global management knowledge: A case for high quality
indigenous research. Asia Pacific Journal of Management, 21: 491-513.
Tsui, A.S. 2006. Contextualization in Chinese management research. Management and
Organization Review, 2(1): 1-13.
Tsui, A.S., Nifadkar, S. & Ou, Y. 2007. Cross-national cross-cultural organizational behavior
research: Advances, gaps, and recommendations. Journal of Management, 33(3):
426-478.
Tsui, A.S., Zhao, SM & Abrahamson, 2007. “What to study in Chinese management
research. Management and Organization Review, 3(2): 1:171-181.
Wallace, Walter. 1971. The Logic of science in sociology. Chicago: Aldine, pp. 11-25.
41
42