Leadership of your
PhD
Anders Dysvik, Ph.D.
Professor of Organizational Behavior
BI Norwegian Business School
Spend 3 minutes discussing facets of
research quality
• How rigorous is the study and the report?
• How original and new is the contribution?
• How relevant is it within the field?
• How usefulness is it in society?
• Can we pick and choose among these requirements - or are
they interdependent?
Requirements I
• Prepared for replication
• Empirical support or lack of rejection
• Precision regarding constructs, level of analysis,
form of relationships, and generalizability
• A unifying conceptual framework
• Falsifiable statements and freedom from prejudice
”…when properly performed,
the rain dance will make rain fall
from the skies”.
”…when properly implemented,
incentive motivators are effective
mechanisms for enhancing
individual performance”.
Peterson, S. J., & Luthans, F. (2006).
The impact of financial and
nonfinancial incentives on businessunit outcomes over time. Journal of
Applied Psychology, 91(1), 156-165.
Requirements II
• Generalizability
• theoretical statements are by definition about something more than
single cases
• the less situation specific, object specific and time specific, the
better
• Generalizability in terms of
• the number of different persons and objects
• how well a phenomenon is explained
• how many phenomena that are explained
The research problem/question
• More specific than the research topic
• Not given, they are discovered, found or constructed
• Something we don’t already know and that is possible to explore,
investigate or answer with the means of scientific methodology
• Should solve a problem or gain relevant insight that develops
theory or practice
• Explicitly related to relevant research literature(s)
• Evolving during the early phases
• Expressed in terms of relationships and/or questions
Positioning the research problem
• Novelty
• something we don’t already know
• typical, a lack of knowledge, i.e. “few studies have….”
• Contribution to theory and/or practice
• the reason why we should know something we don’t already know –
lack of knowledge is not sufficient!
• “Accordingly, the purpose of this study was to test the relationship
between …….. with the objective of contributing to …. by way of
….”
Research problem check list
• Is it clearly formulated? (formulation vs. content)
• Is it specific enough to be dealt with within existing time
and resource constraints?
• Is it positioned – is it explained why it is interesting (cf.
research quality)?
• Is it related to existing literature(s)?
• Are the hypotheses related to it?
What does it take to make a scientific
contribution?
• Contribution in terms of
• theory
• empirical
• or both
• That something has not been investigated (well enough)
yet is a necessary, but not a sufficient condition
• as there may be a number of good reasons for lack of research on
a particular phenomenon
Some pitfalls you should avoid
• Not explicitly stating the contribution you set out to make
• Stating it, but not explicitly enough
• exactly how will the study increase our understanding of the phenomenon
• Stating it, but too late in paper
• Stating it in the beginning, but not in the discussion section
• Ending up with a “collection of variables” with several (too)
minor contributions
• minor contributions can mentioned in the discussion section
Some pitfalls you should avoid
• Several (too) small contributions
• ending up with a collection of variables
• adding every variable that may explain variance
• pick one or two main contributions, leave minor ones to the discussion section
• Making a (too) small empirical contribution
• replications in contexts where there are no compelling reasons that you will not
end up with the same findings/observations as in the original study
• need to explicitly state why and how the findings/observations should be
different
The importance of active reading and
writing
• Don’t buy everything you read
• tactics in the business of publish or perish
• Don’t use rules of thumbs as rules without understanding them
• statistical significance and α = 0.7 or 0.69
• choices of statistical methods and analyses (e.g. second order models,
EFA/CFA, lack of discriminant validity)
• Don’t write something you don’t understand as lack of clarity won’t
disappear by itself
• Be an active reader and writer
Normative research problems
• How to conduct performance appraisal that increases
employee performance?
• not suited for empirical testing
• an interesting study contains much more than a discussion of factors (different Xs)
that may influence Y
• e.g. in order for performance appraisal to increase employee performance it is
important to
• Reframe the research problem to contain something we
don’t already know (enough about)
• e.g. whether specific factors of performance appraisal are related to employee
performance
Research problems without variance in
X when we ask whether X can explain Y
• Does an organization’s CSR practice influence
employees’ affective commitment?
• impossible to investigate with only one organization because X is a constant and
not a variable
• Look for differences between groups or units or
significantly increase the number of organizations
• Or, investigate employee perceptions of X, knowledge about X, use
of X etc.
• Does employee perceptions of an organization’s CSR practice influence their
affective commitment?
Research problems where X does not
exist yet
• Will implementing, increasing or changing X increase Y?
• Will increasing pay differences among employees lead to perceptions of
injustice?
• It is better to investigate perceptions of the existing X
(perceptions of or tolerance for pay differences)
• Or, investigate antecedents to X (then X becomes Y), which may be
important to know before an organization implements, increases or
otherwise changes X (e.g. differences between groups of employees)
Purely descriptive research problems
• Do managers and employees vary in their perceptions of an
organization’s HR practices?
• not very interesting
• Extend the research problem and include factors that may
explain a potential variance in perceptions of X
Quality of research publications
• The Harzing-list provides several rankings
• http://www.harzing.com/
• Australian Business Deans’ Council Journal List
• http://www.abdc.edu.au/
• Four levels: A*, A, B, C
• 11 Disciplines: Business and Management
A*-Journals (28/424 = 6.60%)





Best or leading journal in its field
Publishes outstanding, original and rigorous
research that will shape the field
Acceptance rates are typically low and the editorial
board is dominated by leading scholars in the field
or subfield, including from top institutions in the
world
Where relevant to the field or subfield, the journal
has the highest impact factors or other indices of
high reputation
Acceptance rates usually less than 10%
A selection of A*-Journals
Academy of Management Journal
Academy of Management Review
Administrative Science Quarterly
British Journal of Industrial Relations
California Management Review
Decision Sciences
Entrepreneurship: Theory and Practice
Harvard Business Review
Human Resource Management (US)
Industrial Relations
Journal of Applied Psychology
Journal of Business Venturing
Journal of Int. Business Studies
Journal of Management
Journal of Management Studies
Journal of Operations Management
Journal of Organizational Behavior
Journal of Personality and Social Psych
Product Innovation Management
Journal of Vocational Behavior
Leadership Quarterly
Management Science
Operations Research
Organization Science
Organization Studies
Org. Behavior & Human Decision Proc.
Personnel Psychology
Strategic Management Journal
A-Journals (68/424 = 16.04%)
• Highly regarded journal in the field or subfield
• Publishes excellent research in terms of originality,
significance and rigour, has competitive submission and
acceptance rates, excellent refereeing process and where
relevant to the field or subfield, has higher than average
impact factors
• Not all highly regarded journals have high impact factors,
especially those in niche areas
• Acceptance rates 10% - 20%
A selection of A-journals
Academy of Management Perspectives
Academy of Management Learning and
Education
Annals of Operations Research
Applied Psychology: An Int. Review
British Journal of Social Psychology
Business Ethics Quarterly
European Journal of Industrial
Relations
Family Business Review
Group & Organization Management
Human Performance
Human Relations
Human Resource Mgmt Journal (UK)
Journal of Industrial Relations
Journal of Occupational Health
Psychology
Journal of Occupational and Org. Psych.
Journal of Small Business Management
Journal of World Business
Long Range Planning
Management International Review
Management Learning
Organization
Organizational Dynamics
Organizational Research Methods
Supply Chain Management
Work and Occupations
B-Journals (97/424 = 22.88%)
• Well regarded journal in the field or subfield
• Publishes research of a good standard in terms of
originality, significance and rigor and papers are fully
refereed according to good standards and practices but
acceptance rates are higher than for Tiers A* and A
• Depending on the field or sub-field, will have a modest
impact factor and will be ISI listed
• Acceptance rates 20%-50%
A selection of B-journals
Canadian Journal of Adm. Sciences
Employee Relations
European Business Review
European Journal of Work and
Organizational Psychology
Human Resource Development Int.
Human Resource Development Quart.
Human Resource Development Review
Human Resource Management Review
Industrial Relations Journal
Integrated Manufacturing Systems
International Journal of Conflict Mngt.
International Journal of Cross-Cultural
Management
Journal of Applied Behavioral Science
Journal of Business and Psychology
Journal of General Management
Journal of Labour Research
Journal of Management Education
Journal of Management Inquiry
Journal of Organizational Behavior Mngt.
Journal of Org. Change Management
Labor Studies Journal
Labour and Industry
Leadership and Org. Dev. Journal
Management Communication Quarterly
Nonprofit Management & Leadership
Organization Development Journal
C-Journals (231/424 = 54.48%)
• A recognized journal
• Publishes research that is of a modest standard and/or is
yet to establish its reputation because of its newness
• This tier is more inclusive than the others but only
includes refereed journals
• Acceptance rates typically > 50%
• Remember that most journals are not listed!
A selection of C-journals
Advances in Developing Human Res.
Advancing Women in Leadership
Asia-Pacific Journal of Human Res.
Career Development International
Culture and Organization
Development and Learning in
Organizations: An International Journal
Employment Relations Record
Equal Opportunities International
European Business Journal
European Management Journal
European Management Review
Human Systems Management
Intern. Employment Relations Rev.
International Journal of Management
Intern. J. of Mngt. and Decision Making
Intern. Journal of Management Reviews
Intern. Journal of Organization
Behaviour
Intern. J. of Training and Development
Intern. J. of Work Org. and Emotion
Intern. Rev. of Industrial &
Organisational
Intern. Review of Women and
Leadership
Journal of Change Management
Scandinavian Journal of Management
Relevance versus Rigor
Top journals = top research?
• Dissensus among reviewers
• interrater reliability typically less than 0.30
• Low acceptance rates
• “Highly prestigious journals publish quite a few low-value articles, low prestige
journals publish some excellent articles, and excellent manuscripts may receive
successive rejections from several journals”
• Variation in article impact
• 12% of articles published in top journals are not cited
• less than 4% generate more than 100 citations
• reviewers are not able to predict the future impact of an article (r = .14)
Glick, W. H., Miller, C. C., & Cardinal, L. B. (2007). Making a life in the field of organization science.
Journal of Organizational Behavior, 28(7), 817-835.
Conclusions
• Don’t equate top journals with top research
• although usually more rigor (multiple data sources, better
measures, longitudinal designs etc.)
• Don’t buy everything you read
• good journals also publish low quality research
• top scholars sometimes publish low quality research