Chapter 2
Research Methods in
Organizational Psychology
SOP6669
Dr. Steve
Research Methods
Methods:
 Experiment / Quasi-Experiment
 Questionnaire/Survey
 Naturalistic Observation
 Case Study
 Meta-Analysis
Research Methods
Experiment
Study conducted in a contrived environment
 Benefits:


Provides more safety
Cause and effect relationships
• Manipulate I.V. (e.g., leadership style)
• Measure D.V. (e.g., task performance)
• Control extraneous variables (e.g., experience)
 Disadvantages:

Time consuming
Quasi-Experiment – not randomized or
unable to manipulate IV (e.g., gender)
Research Methods
Questionnaire/Survey
Self-report to obtain data on attitudes/behaviors
conducted by phone, mail, interviews, electronically
 Benefits:


Can collect a large quantity of data
Disadvantages:



Accuracy of reporting
Representativeness of sample
Return rate
Research Methods
Naturalistic Observation
Observe overt behaviors over time



Benefits:


Systematic sampling at various times
Representative sample
Use to generate hypotheses
Disadvantages:



Experimenter bias
Obtrusiveness
Frequency of behavior occurring
Research Methods
Case Study
In depth view of past events using interviews and
archival records

Benefits:


Detailed account of why particular event occurred
Disadvantages:

Little generalizability
Data Analysis
Meta-analysis
Meta-analysis – statistical procedure that
combines the results of many independent
research findings on a single topic
 Used to estimate true relationship
 Measures effect size of findings
 Uses archival data
Research Steps
Statistical Analysis
Descriptive vs. Inferential Statistics
 Descriptive stats merely describe data




Frequency
Central tendency
Variability
Inferential stats used to test hypotheses





T-Test
Analysis of variance
Correlation
Regression
Non-parametrics
Data Analysis
Central Tendency
example scores = 5, 6, 6, 8, 9, 10, 11, 17
_
1. Mean – average: X = ∑X / N
Mean = 72 / 8 = 9
2. Median – middle score (when placed in order)
- use when outliers exaggerate the mean
Median = 8.5
3. Mode – most often occurring score
Mode = 6
* In a normal distribution, Mean = Median = Mode
Data Analysis
Variability

Range - distance between highest and lowest
score



(Range = High score – Low score)
Range = 17 – 5 = 12
Standard Deviation – average distance from the
mean

S= Σ(x – x)2 / n – 1
S = (5-9) 2 + (6-9) 2 + (6-9) 2 + (8-9) 2 + (9-9) 2 + (10-9) 2 + (11-9) 2 + (17-9) 2 / 7
S = 3.85
Normal or
Bell-shaped
Distribution
Frequency
Data Analysis
Skewed Frequency Distributions
20
18
16
14
12
10
8
6
4
2
0
65- 75- 85- 95- 105- 115- 125- 135- 145- 15574 84 94 104 114 124 134 144 154 164
IQ Scores
Negatively Skewed Distribution
30
20
18
16
14
12
10
8
6
4
2
0
25
Frequency
Frequency
Positively Skewed Distribution
20
15
10
5
65- 67- 69- 71- 73- 75- 77- 79- 81- 82- 8466 68 70 72 74 76 78 80 82 83 85
Professional Golf Scores
0
250- 260- 270- 280- 290- 300- 310- 320- 330- 340259 269 279 289 299 309 319 329 339 349
Weight (lbs) of NFL Lineman
Data Analysis
Correlation
r
A
B
A
1.0
B
.40
1.0
C
.20
.09
Correlation ( r ) – Degree of relationship
between two variables






Used for prediction
Cannot be used to infer causation
Range from –1 to +1
Negative r – as one variable increases the other
decreases
Positive r – as one variable increases so does the
other
Zero r – no relationship between the two variables
C
1.0
Data Analysis
Correlation
600
College GPA
3.5
3
2.5
2
1.5
1
0.5
** **
*
**
**** **
* * ***
*
****** * *
*
*
*
**
*** *
*
**
*
*****
***
**
*
*
*
*
*
***
*
*
*
**
*
*** *
*
*
*
*
*
*
*
*
***** *** *
*
*
*
*
*
*
*
*
*
**
**
***
*
*** * *
*
*
*
***
* **
* *
*
*
**
*
*
* ***** ** *
*
*
*
* ** * **
* **
60
0
80
100
120
20
*
Years of Practice
4
800 1000 1200 1400 1600
15
10
5
0
GRE Scores
Positive Correlation
Golf Scores
Negative Correlation
Correlation Examples






IQ scores of identical twins: r = +.86
Phases of the moon & # acts of violence: r = .00
Economic conditions & # lynchings: r = -.43
Amount of ice cream sold & # drownings: r = +.60
Price of rum in Cuba & priests salaries in New
England: r = +.38
Number of cigarettes smoked per day & incidence
of lung cancer: r = ???
Statistical Methods
Regression
Regression Variables (used for prediction)
Yi = ß0 + ß1Xi1 + ß2Xi2 (Y = a + b1X1)
 Predictor Variable (X) – measure used to predict
an outcome (similar to independent variable)


Criterion Variable (Y) – outcome to be predicted


Example: selection test scores, years of experience,
education level
Example: work performance, turnover, sales,
absenteeism, promotion, etc.
Example: AFOQT scores as predictors of pilot
training performance
Statistical Pitfalls:
Bias
 Representative


Sampling
Selecting a sample that parallels the
population
Might use covariates to account for
differences
 Statistical Assumptions

ANOVA assumes a normal distribution and
independence
• Lack of normality is only minor problem, but may
want to identify distribution shape and why
• Observations may not be independent, may need
to aggregate (e.g., class instead of student)
Statistical Pitfalls:
Errors in Methodology

Statistical Power – probability of detecting a true
difference of a particular size



Type I error – falsely reject null hypothesis when a true
difference does not exist
Type II error – fail to reject null hypothesis when a true
difference does exist
Power affected by
•
•
•
•
Sample size
Effect size (e.g., Cohen’s D)
Type I error rate selected (alpha)
Variability of sample

(F ratio = var between group / var within group)
Statistical Pitfalls:
Errors in Methodology

Multiple Comparisons – if you compare enough
variables, will find a relationship by chance
alone

Bonferroni correction – family-wise adjustment
(alpha = .05 / #comparisons)



Replicate
Cross-validate (holdout sample)
Measurement Errors


Reliability: Consistency of Measure
Validity: Measures what it was designed to measure
Statistical Pitfalls:
Problems with Interpretation
 Confusion

P value does not reflect effect size – could
have a small effect, but a lot of power
 Precision

over significance
vs. Accuracy
More decimals not necessarily more accurate
 Causality

Correlations are not causal, but ANOVA may
not be either
Statistical Pitfalls:
Problems with Interpretation

Graphs

May not provide accurate portrayal of data
84
100
83.5
80
83
60
82.5
Score
Score
40
82
20
81.5
0
81
Group A
Group B
Group A
Group B
Research
Critical Thinking
Always think critically about the research you read






Who were the participants in the study?
How strong of a relationship was found?
Was it causal or correlational?
Was it a field study or laboratory study?
How was data collected and analyzed?
Do you agree with the conclusions based on the
analyses provided?
Ethical Principles of Research

Privacy:


Confidentiality:


Participants have the right to limit the amount of information they
reveal about themselves. If they decide to withdraw from the
experiment at any time, they have the right to do so
Participants have the right to decide to whom they reveal
confidential information. By ensuring confidentiality, researchers
may be able to obtain more honest responses
Protection from Deception:

Deception can only be used if the value of the research must
outweigh the harm imposed on participants and the
phenomenon cannot be measured any other way