What are Peer-Reviewed,
Primary
and Empirical Articles?
Peer-Reviewed (or Refereed)
Before an article, book chapter or book is accepted for publication it is
examined by several (at least 2, usually at least 3) experts on the topic of the
article. These experts are called referees. The referees evaluate the article on
whether or not it should be published. The referees evaluate the article on
whether the article used appropriate methodology for the field, whether the
topic is appropriate for the field and the importance of the findings to the field.
To prevent personal bias, refereed journals use a blind review process where the
referees do not know the name of the article’s author (and often there is a
double blind review process; the article’s author does not know the names of
the referees). The referees will report to the journal’s editor as to whether
accept the article, reject it, or accept it with revisions.
How to identify refereed articles
 Somewhere in the journal (usually on the back of the front cover) will be
instructions to the authors. In the instructions will be a description of the
submission process.
 In PsychInfo, on the abstract page will be a category named, “Publication
Type.” It will say “peer-reviewed journal,” if the journal is, indeed, peerreviewed.
What’s not refereed
Articles written by employees (reporters) of the magazines or journals. E.G. Time,
Newsweek, Psychology Today.
Primary Source
This refers to an article, book chapter or book which represents the author’s own
and original work. Examples include; an article where an author has collected
data herself and presents that data; or a book where an author develops and
presents a new theory.
How to identify primary sources


For empirical articles, the author will describe her data collection process
(Method, Results sections).
For theoretical articles or books, the author will present the theory as her own.
What’s not primary
Articles which describe the work of others. E.G. a literature review, such as a
Psychological Bulletin article or an Annual Review of Psychology chapter; a
Psychology Today article.
Meta-Analyses?
A meta-analysis is a statistical analysis of the results of many studies on the same
topic. The conclusions of a meta-analysis can be considered primary, but
nothing else (such as the descriptions of the studies that made up the metaanalysis).
Empirical Work
In Psychology, empirical work is based upon systematic observation of behavior.
This could be an experiment, a quasi-experiment, a field experiment, an
observational study, a questionnaire or a structured interview. The key is
systematic observations. For example, in an interview, the author will describe a
standard set of questions which were asked of all participants in a standard way
and setting.
How to identify empirical work
 In Psychology, a dead-giveaway is the presence of Method and Results
sections.
 Some I/O and Business articles will be empirical articles but will not have
Method or Results sections. Look at the article and look for evidence that
the author proposed hypotheses, collected data and tested the data to
evaluate the hypotheses. Do not confuse a review article or a theoretical
article with this latter type. A review article will propose hypothesis but not
collect data nor evaluate the hypotheses.
What’s not empirical
A personal observation (Jane Elliot saying that based upon her 30 years of
experience the Blue eye/Brown eye technique reduces racism; Jack Welch
saying that using boundary-less organizational principles earned G.E. billions of
dollars; David Rosenhan describing his experiences as a peudopatient).
Unstructured interviews (a journalist interviews unwed mothers about how much
stress they feel – this is an unstructured interview … how do you tell? if it was a
structured interview the author would present a description of the structure of
the interview … e.g. “all interviewees were asked the following 5 questions in
order … their responses were recorded and analyzed by raters …”).
Note: peer-reviewed-ness, primary-ness and empirical-ness are independent.
That means that an article could be peer-reviewed, but not primary or
empirical (that’s a review article)! Or an article could be empirical and
primary, but not peer-reviewed (many websites fit this category).
Take the Quiz!
For each of the following, determine whether the article is refereed, primary
and/or empirical. You know that, you’ll need to look the articles up on
PsychInfo to find information regarding it’s primary status.
Title: Assessor Cognitive Processes in an Operational Assessment Center.
Author(s): Lance, Charles E., Department of Psychology, University of Georgia,
Athens, GA, US, clance@uga.edu
Foster, Mark R., Institute of Government, University of Georgia,
Athens, GA, US
Gentry, William A., Department of Psychology, University of Georgia,
Athens, GA, US
Thoresen, Joseph D., Cornerstone Systems, Carnegie, PA, US
Address: Lance, Charles E., Department of Psychology, The University of
Georgia, Athens, GA, US, clance@uga.edu
Source: Journal of Applied Psychology, Vol 89(1), Feb 2004. pp. 22-35.
Journal URL: http://www.apa.org/journals/apl.html
Publisher: US: American Psychological Assn
Publisher URL: http://www.apa.org
ISSN: 0021-9010 (Print)
Digital Object 10.1037/0021-9010.89.1.22
Identifier:
Language: English
Key Concepts: postexercise dimension ratings; assessment centers; situational
specificity hypothesis; assessor cognitive processes; general impression
model salient dimension model; job performance
Abstract: The purpose of this study was (a) to provide additional tests of C. E.
Lance, Newbolt, et al.'s (2000) situational specificity (vs. method bias)
interpretation of exercise effects on assessment center postexercise
dimension ratings and (b) to provide competitive tests of salient
dimension versus general impression models of assessor within-exercise
evaluations of candidate performance. Results strongly support the
situational specificity hypothesis and the general impression model of
assessor cognitive processes in which assessors first form overall
evaluations of candidate performance that then drive more specific
dimensional ratings. (PsycINFO Database Record (c) 2004 APA, all
rights reserved)(journal abstract)
Subjects: *Assessment Centers; *Cognitive Processes; *Employment Tests; *Job
Performance; *Personnel Evaluation
Classification: Personnel Evaluation & Job Performance (3630)
Population: Human (10)
Location: US
Age Group: Adulthood (18 yrs & older) (300)
Form/Content Type: Empirical Study (0800)
Quantitative Study (0890)
Journal Article (2400)
Publication Type: Peer Reviewed Journal (270); Print
Format(s) Available: Print
Release Date: 20040209
Accession Number: 2004-10572-003
Number of Citations in 51
Source:
Persistent link to this http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url
record: ,uid&db=psyh&an=2004-10572-003
Database: PsycINFO
Title: Self- versus others' ratings as predictors of assessment center ratings: Validation evidence for
360-degree feedback programs.
Author(s): Atkins, Paul W. B., Australian Graduate School of Management, Australia,
paul.atkins@anu.edu.au
Wood, Robert E., Australian Graduate School of Management, Australia
Address: Atkins, Paul W. B., Australian National U, National Graduate School of Management, Canberra,
ACT, Australia, 0200, paul.atkins@anu.edu.au
Source: Personnel Psychology, Vol 55(4), Win 2002. pp. 871-904.
Publisher: US: Personnel Psychology
Publisher URL: http://personnelpsychology.com
ISSN: 0031-5826 (Print)
Language: English
Key Concepts: self-ratings; ratings; predictors; assessment center ratings; rating validity; 360-degree feedback
programs
Abstract: This study (n = 62 team leaders) had 2 main aims: First, to examine the validity of ratings from a
360-degree feedback program using assessment center ratings as an independent criterion and to
determine which source (i.e., self, supervisor, peers, or subordinates) provided the most valid
predictor of the criterion measure of competency. Second, to better understand the relationship
between self-observer discrepancies and an independent criterion. The average of supervisor,
peer, and subordinate ratings predicted performance on the assessment center, as did the
supervisor ratings alone. The self-ratings were negatively and nonlinearly related to performance
with some of those who gave themselves the highest ratings having the lowest performance on th
assessment center. Supervisor ratings successfully discriminated between overestimators but
were not as successful at discriminating underestimators, suggesting that more modest feedback
recipients might be underrated by their supervisors. Peers overestimated performance for poor
performers. Explanations of the results and the implications for the use of self-ratings in
evaluations, the design of feedback reports, and the use of 360-degree feedback programs for
involving and empowering staff are discussed. (PsycINFO Database Record (c) 2004 APA, all
rights reserved)
Subjects: *Assessment Centers; *Feedback; *Prediction; *Rating; *Statistical Validity
Classification: Occupational & Employment Testing (2228)
Population: Human (10)
Location: Australia
Age Group: Adulthood (18 yrs & older) (300)
Form/Content Type: Empirical Study (0800)
Publication Type: Peer Reviewed Journal (270); Print
Format(s) Available: Print
Release Date: 20030102
Accession Number: 2002-08510-007
umber of Citations in Source: 44
Persistent link to this record:
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=2002
08510-007
Database: PsycINFO
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=200
2-08510-007
Title: A field study of the effects of rating purpose on the quality of multisource ratings.
Author(s): Greguras, Gary J., Louisiana State U, Dept of Psychology, Baton Rouge, LA, US,
ggregu1@lsu.edu
Robie, Chet, Wilfrid Laurier U, Dept of Business, Waterloo, ON, Canada
Schleicher, Deidra J., U Tulsa, Dept of Psychology, Tulsa, OK, US
Goff, Maynard III, Personnel Decisions International
Address: Greguras, Gary J., Louisiana State U, Dept of Psychology, 236 Audubon Hall, Baton Rouge, LA
US, ggregu1@lsu.edu
Source: Personnel Psychology, Vol 56(1), Spr 2003. pp. 1-21.
Publisher: US: Personnel Psychology
Publisher URL: http://personnelpsychology.com
ISSN: 0031-5826 (Print)
Language: English
Key Concepts: reliability; variability; performance ratings; multisource ratings; peers; subordinates; rating
purpose; rating quality; managers; generalizability theory
Abstract: Using a field sample of peers and subordinates, this study employed generalizability theory to
estimate sources of systematic variability associated with both developmental and administrative
ratings (variance due to items, raters, etc.) and then used these values to estimate the
dependability (i.e., reliability) of the performance ratings under various conditions. 454 manager
in a large telecommunications company participated as ratees in this study. Ratings given by peer
and subordinates were analyzed. Results indicate that the combined rater and rater-by-ratee
interaction effect and the residual effect were substantially larger than the person effect (i.e.,
object of measurement) for both rater sources across both purpose conditions. For subordinates,
the person effect accounted for a significantly greater percentage of total variance in
developmental ratings than in administrative ratings; however, no differences were observed for
peer ratings as a function of rating purpose. These results suggest that subordinate ratings are of
significantly better quality when made for developmental than for administrative purposes, but th
same is not true for peer ratings. (PsycINFO Database Record (c) 2004 APA, all rights reserved)
Subjects: *Job Performance; *Personnel Evaluation; *Rating; *Statistical Reliability; *Variability
Measurement; Management Personnel; Peer Evaluation; Theories
Classification: Occupational & Employment Testing (2228)
Personnel Evaluation & Job Performance (3630)
Population: Human (10)
Male (30)
Female (40)
Age Group: Adulthood (18 yrs & older) (300)
Form/Content Type: Empirical Study (0800)
Publication Type: Peer Reviewed Journal (270); Print
Format(s) Available: Print
Release Date: 20030407
Accession Number: 2003-02432-002
umber of Citations in Source: 57
Persistent link to this record:
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=2003
02432-002
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=200
3-02432-002
Title: Assessment centers: What's new?
Author(s): Joiner, Dennis, joinerda@pacbell.net
Address: Joiner, Dennis, joinerda@pacbell.net
Source: Public Personnel Management, Vol 31(2), Sum 2002. pp. 179-185.
Publisher: US: International Public Management Association for Human Resources
Publisher URL: http://www.ipma-hr.org
ISSN: 0091-0260 (Print)
Language: English
Key Concepts: assessment centers; assessment; employment
Abstract: Discusses the definition of assessment centers and how assessment center results are used.
Suggests that variations such as videos, multiple situations exercises, and situational judgment
tests can be use in the assessment center process. The author notes that assessment centers and
the related technology are a constantly evolving area within the discipline of testing and
assessment. It is suggested that all assessment center concepts should be applied differently in
every setting based on the unique requirements of that setting. (PsycINFO Database Record (c)
2004 APA, all rights reserved)
Subjects: *Assessment Centers; *Employment Status; *Personnel Evaluation; *Testing
Classification: Occupational Interests & Guidance (3610)
Publication Type: Peer Reviewed Journal (270); Print
Format(s) Available: Print
Release Date: 20020717
Accession Number: 2002-15219-007
umber of Citations in Source: 4
Persistent link to this record:
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=2002
15219-007
Database: PsycINFO
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=200
2-15219-007
Title: A current review of public sector assessment centers: Cause for concern.
Author(s): Ross, Joyce D., San Diego State U, School of Public Administration & Urban Studies
Source: Public Personnel Management, Vol 8(1), Jan-Feb 1979. pp. 41-46.
Publisher: US: International Public Management Association for Human Resources
Publisher URL: http://www.ipma-hr.org
ISSN: 0091-0260 (Print)
Language: English
Key Concepts: public organizations' use of personnel assessment centers for management personnel selection &
promotion, implications for legal challenges & critical evaluation
Abstract: Public organizations have become increasingly involved in the use of assessment centers for
selection purposes. Standards for such centers are reviewed, possible legal challenges are noted,
and public organizations are asked to evaluate critically the conduct of their assessment centers.
(PsycINFO Database Record (c) 2004 APA, all rights reserved)
Subjects: *Government Personnel; *Management Personnel; *Personnel Evaluation; *Personnel Promotion;
*Personnel Selection; Employment Tests; Legal Processes; Measurement
Classification: Industrial & Organizational Psychology (3600)
Population: Human (10)
Publication Type: Peer Reviewed Journal (270); Print
Release Date: 19800701
Accession Number: 1980-22215-001
Persistent link to http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=1980this record: 22215-001
Database: PsycINFO
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&AuthType=cookie,ip,url,uid&db=psyh&an=198
0-22215-001’
Title: A Year In The Life.
Authors: Kantrowitz, Barbara
Source: Newsweek; 06/04/2001, Vol. 137 Issue 23, p42, 3p, 1c
Document Type: Article
Subject Terms: *EDUCATORS
*FIRST year teachers
*TEACHERS
*MIDDLE school teaching
Geographic Terms: ILLINOIS
UNITED States
EVANSTON (Ill.)
Abstract: Presents the experiences of Evanston, Illinois middle-school language arts
teacher Elizabeth Jackson during one academic year. Jackson's anxieties
over job performance and the success of her students; Her perceptions
about being the youngest teacher in the school; Jackson's description of
her first year of teaching as her first in the real world. INSET: 'There are no
magic words that will motivate every student'.
Full Text Word Count: 1882
ISSN: 0028-9604
Accession Number: 4487394
Persistent link to this http://search.epnet.com/login.aspx?direct=true&db=aph&an=4487394
record:
Database: Academic Search Premier
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&db=aph&an=4487394
Title: 3M: The Heat Is on the Boss.
Authors: Weimer, De'Ann
Source: Business Week; 03/15/99 Issue 3620, p82, 3p, 1 chart, 1 graph, 4c
Document Type: Article
Subject Terms: *CHIEF executive officers
*INDUSTRIAL management
*PUBLIC opinion
Company/Entity: MINNESOTA Mining & Manufacturing Co. -- Officials & employees
Ticker: MMM
NAICS/Industry Codes: 55 Management of Companies and Enterprises
People: DESIMONE, Livio
Abstract: Reports on problems that Livio DeSimone, chief executive officer of 3M
Co., faces over his job performance. Managers' complaints against
DeSimone; Analysts and investors' anger that management at 3M has
missed annual earnings projections for two years; What critics believe his
problems stem from.
Full Text Word Count: 2009
ISSN: 0007-7135
Accession Number: 1610355
Persistent link to this http://search.epnet.com/login.aspx?direct=true&db=aph&an=1610355
record:
Database: Academic Search Premier
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&db=aph&an=1610355
Title: Personnel selection: Looking toward the future--Remembering the
past.
Author(s): Hough, Leaetta M., The Dunnette Group, St Paul, MN, US
Oswald, Frederick L.
Source: Annual Review of Psychology, Vol 51, 2000. pp. 631-664.
Journal URL: http://psych.annualreviews.org/
Publisher: US: Annual Reviews
Publisher URL: http://www.annualreviews.org
ISSN: 0066-4308 (Print)
Digital Object Identifier: 10.1146/annurev.psych.51.1.631
Language: English
Keywords: job analysis & personnel selection & performance criteria & cognitive
ability & personality predictors & measurement issues
Abstract: Reviews personnel selection research from 1995-1999. Areas covered
are job analysis; performance criteria; cognitive ability and personality
predictors; interview, assessment center, and biodata assessment
methods; measurement issues; meta-analysis and validity generalization;
evaluation of selection systems in terms of differential prediction, adverse
impact, utility, and applicant reactions; emerging topics on team selection
and cross-cultural issues; and finally professional, legal, and ethical
standards. Three major themes are revealed: (a) better taxonomies
produce better selection decisions; (b) the nature and analyses of work
behavior are changing, influencing personnel selection practices; (c) the
field of personality research is healthy, as new measurement methods,
personality constructs, and compound constructs of well-known traits are
being researched and applied to personnel selection. (PsycINFO
Database Record (c) 2004 APA, all rights reserved)
Subjects: *Cognitive Ability; *Job Analysis; *Personality Traits; *Personnel
Evaluation; *Personnel Selection; Performance Tests; Prediction
Classification: Occupational & Employment Testing (2228)
Personnel Evaluation & Job Performance (3630)
Population: Human (10)
Form/Content Type: Literature Review (1300)
Publication Type: Peer Reviewed Journal (270); Print
Release Date: 20000501
Accession Number: 2000-15267-021
Persistent link to this http://search.epnet.com/login.aspx?direct=true&db=psyh&an=2000record: 15267-021
Database: PsycINFO
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&db=psyh&an=2000-15267-021
Title: Career Benefits Associated With Mentoring for Proteges: A
Meta-Analysis.
Author(s): Allen, Tammy D., Department of Psychology, University of South
Florida, Tampa, FL, US, tallen@luna.cas.usf.edu
Eby, Lillian T., Department of Psychology, University of Georgia,
GA, US
Poteet, Mark L., Independent Practice, Organizational Research &
Solutions, Tampa, FL, US
Lentz, Elizabeth, Department of Psychology, University of South
Florida, Tampa, FL, US
Lima, Lizzette, Department of Psychology, University of South
Florida, Tampa, FL, US
Address: Allen, Tammy D., Department of Psychology, University of South
Florida, 4202 East Fowler Avenue, PCD 4118G, Tampa, FL, US,
tallen@luna.cas.usf.edu
Source: Journal of Applied Psychology, Vol 89(1), Feb 2004. pp. 127-136.
Journal URL: http://www.apa.org/journals/apl.html
Publisher: US: American Psychological Assn
Publisher URL: http://www.apa.org
ISSN: 0021-9010 (Print)
Digital Object Identifier: 10.1037/0021-9010.89.1.127
Language: English
Keywords: career benefits; mentoring; compensation outcomes; career
satisfaction
Abstract: Meta-analysis was used to review and synthesize existing empirical
research concerning the career benefits associated with mentoring
for the protege. Both objective (e.g., compensation) and subjective
(e.g., career satisfaction) career outcomes were examined.
Comparisons of mentored versus nonmemored groups were
included, along with relationships between mentoring provided and
outcomes. The findings were generally supportive of the benefits
associated with mentoring, but effect sizes associated with
objective outcomes were small. There was also some indication
that the outcomes studied differed in the magnitude of their
relationship with the type of mentoring provided (i.e.. career or
psychosocial). (PsycINFO Database Record (c) 2004 APA, all
rights reserved)(journal abstract)
Subjects: *Career Development; *Job Satisfaction; *Mentor; *Occupational
Success; *Salaries
Classification: Personnel Evaluation & Job Performance (3630)
Population: Human (10)
Form/Content Type: Conference Proceedings/Symposia (0600)
Meta Analysis (1400)
Journal Article (2400)
Conference: Conference of the Society for Industrial and Organizational
Psychology, 17th, Toronto, ON, Canada
Conference Notes: A previous version of this article was presented at the
aforementioned conference.
Publication Type: Peer Reviewed Journal (270); Print
Format(s) Available: Print
Release Date: 20040209
Accession Number: 2004-10572-010
Number of Citations in Source: 87
Persistent link to this record: http://search.epnet.com/login.aspx?direct=true&db=psyh&an=200410572-010
Database: PsycINFO
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&db=psyh&an=2004-10572-010
Title: A New Habit for Trainers.
Authors: Covey, Stephen R.1
Source: T+D; Dec2004, Vol. 58 Issue 12, p14, 2p
Document Type: Article
Subject Terms: *COMMITMENT (Psychology)
*EFFECTIVE teaching
*EMPLOYEE training personnel
*HABIT
*SOCIAL facilitation
*TRAINING
Abstract: The article presents suggestions that would increase effectiveness of
trainers. There should be clarity about outcomes and measures. The trainers
should ask participants to define their personal measures of success and
should challenge them to make an explicit connection with the greater goals
of the organization or team. The trainer's task is to lead people to insight and
inspire them. The trainers should consider what key insights participants
should gain. Translating commitment to action and helping participants
empower themselves by defining barriers to action are some other
suggestions for trainers to improve their effectiveness.
Author Affiliations:
1Co-Chairman,
FranklinCovey Company.
ISSN: 1535-7740
Accession Number: 15308113
Persistent link to http://search.epnet.com/login.aspx?direct=true&db=aph&an=15308113
this record:
Database: Academic Search Premier
* from campus you can see the whole article by following this link
http://search.epnet.com/login.aspx?direct=true&db=aph&an=15308113
Answers
Lance
Refereed, primary & empirical.
Atkins
Refereed, primary & empirical.
Greguras
Refereed, primary & empirical.
Joiner
Refereed, not primary & non-empirical. It was a review article in a refereed
journal.
Ross
Non-refereed, not primary & non-empirical. Even though Ross is a professor,
there’s no evidence this is a peer-reviewed journal. She did not do anything
herself and didn’t collect data.
Kantrowitz
Non-refereed, primary & non-empirical. News magazines like Newsweek are not
peer-reviewed (also, the abstract page does not state the journal is peerreviewed). Kantrowitz wrote the article about her work (interviewing the
teacher), so it is primary. Kantrowitz’s investigation of the teacher is not
systematic data collection.
Weimer
Non-referred, primary & non-empirical. For more or less the same reasons as
Kantrowitz’s article.
Hough
Refereed, not primary, non-empirical. This is from the Annual Review of
Psychology – it’s a literature review.
Allen
Refereed, not primary, non-empirical. This is a meta-analysis. It’s based upon
other people’s works (not primary) and behavior is not observed systematically.
Of course, this is really open to interpretation. Some may feel that metaanalyses are empirical, since they systematically collect data. Maybe – BUT NOT
FOR ANY OF MY CLASSES!
Covey
Non-refereed, primary & non-empirical. Covey talks about his own ideas so it’s
primary.