Best of 2008: Research findings
you may have missed.
Ryan Watkins, Ph.D.
George Washington University
www.gwu.edu/~etl
www.ryanrwatkins.com
1
Session Objectives
Participants will be able to…
 Summarize leading research articles from
multiple disciplines;
 Translate research finding into useful HPT
applications; and
 Apply research findings in their daily practice
of HPT.
2
“Best of 2008”???
Step 1: Surveyed 20 researchers and faculty


Both ISPI and non-ISPI members
Varied disciplines


3
HRD, OD, ISD, Management, Psychology
No specific criteria for “best”
Step 2: Reviewed diverse journals from 2008
Step 3: Based on initial input, sent preliminary list
of articles back to researchers and faculty
Step 4: Used their input to create a final list.
Reminders




4
Only a summary of each research article is being
provided, please read the whole article if you are
interested in the details.
Presenting them in alphabetical order, not order of
importance.
To maintain copyright I can not provide copies of the full
articles.
The “HPT Implications” are my applications of the
findings, you may have your own and the person sitting
next to you may see different applications.
Article One



5
Title: An Examination of the Relationship
Among Structure, Trust, and Conflict
Management Styles in Virtual Teams
Authors: Xiaojing Liu, Richard J. Magjuka,
and Seung-hee Lee
Publication: Performance Improvement
Quarterly (an ISPI publication)
Article One – Liu, Magjuka & Lee


Purpose: Examine “the roles of team structure, trust,
and conflict variables in the effectiveness of virtual
teamwork.”
Sample Hypotheses:
–
–
–
–
6
Nonhierarchical structures outperform hierarchical structures on
their virtual team performance in complex task environments.
Trust will be positively associated with virtual team performance.
A collaboration conflict management style in virtual teams is
positively associated with virtual team satisfaction..
Hierarchical teams have a higher level of trust than
nonhierarchical teams.
Article One – Liu, Magjuka & Lee


Variables: A 22-item questionnaire contained three
dimensions: trust, conflict management resolution, and
teamwork satisfaction.
Methods:
–
–
–
–
7
Students enrolled in a capstone management course as part of
an online MBA program at a large Midwestern university.
Subjects were separated into project groups of four to six people
to work on an online simulation project.
Hierarchical (17 teams) and non-hierarchical (21 teams) team
structures were assigned to the teams.
N = ~208 participants (some were dropped for teams not
following the instructions, but the final N was not given).
Article One – Liu, Magjuka & Lee

Findings:
–
–
–
–

HPT Implications:
–
–
8
Team structure had a significant effect on team performance (with
nonhierarchical leading to greater performance).
Team structure did not have a significant effect on team
satisfaction.
Trust had a significant effect on team satisfaction, but not
performance.
Collaborative conflict management had a significant effect on team
satisfaction.
In virtual teams, structure seems to have a greater influence on
team performance, while social processes (i.e., team trust and
conflict management) has greater influence on member
satisfaction.
“Even in a virtual team that is formed for a short time, it is
worthwhile to facilitate ‘‘swift’’ relationship building through
proactive communicative actions and to develop shared goals…”
Article Two



9
Title: Feedback Acceptance in Developmental
Assessment Centers: The role of feedback
message, participant personality, and affective
response to the feedback session
Authors: Suzanne T. Bell and Winfred Arthur Jr.
Publication: Journal of Organizational Behavior
Article Two – Bell and Arthur

Purpose:
–

Hypotheses:
–
–
–
10
Examine the role of the affective reaction to the feedback session
in mediating the relationship between the assessor ratings and
feedback acceptance.
The relationship between assessor ratings and feedback
acceptance will be mediated by the affective reaction to the
feedback session.
Participant self-ratings will moderate the relationship between
assessor ratings and feedback acceptance.
Three additional hypotheses related to the moderating role of
personality characteristics (extroversion, emotional stability, and
agreeableness).
Article Two – Bell and Arthur

Variables:
–
–
–

Assessor ratings, affective reaction to the feedback session, and
feedback acceptance.
Participants’ self-ratings of their performance.
Participant personality characteristics of extraversion, emotional
stability, and agreeableness as moderators.
Methods:
–
Two organizations:


11
–
First: Career development program intended to help mid-level
managers develop their leadership skills.
Second: Professional school of public administration which was
training individuals for leadership and managerial positions in
government or public service.
N = 141 participants, working with 35 trained assessors.
Article Two – Bell and Arthur

Findings:
–
Assessor ratings were positively related to feedback acceptance
and this relationship was partially mediated by the affective
reaction to the feedback session.



HPT Implications:
–
–
12
Participant’s self-ratings, extroversion, and emotional stability did not
serve as a moderator of this relationship.
Participant’s agreeableness (good-natured, flexible, trusting,
cooperative, and tolerant dispositions) did serve as a moderator.
Performance feedback is essential to improving performance,
and the context in which that feedback is given is very important
to acceptance.
Build rapport before giving feedback (for example, light
conversation before and after).
Article Three



13
Title: Implications of Domain-General
‘‘Psychological Support Skills’’ for Transfer of
Skill and Acquisition of Expertise
Authors: David W. Eccles and Paul J. Feltovich
Publication: Performance Improvement
Quarterly (an ISPI publication)
Article Three -- Eccles & Feltovich

Purpose:
–
14
Little research has been directed at identifying
psychological skills that (1) support learning and
performance and (2) are domain-general and thus can
be applied in a variety of novel learning and
performance settings.
Article Three -- Eccles & Feltovich

Variables:
–
Psychological Support Skills


–
Meta-Cognition


–

15
Domain-general skills that support meta-cognition.
Mental imagery and rehearsal skills, self-talk management skills, goal-setting
skills, relaxation skills, and planning, preparing, and organizing skills.
Ability to regulate one’s psychological state in order to attain and
subsequently maintain a state conducive to learning and performance.
Sustain a high level of self-confidence, sustain a high level of motivation,
control anxiety and cope with stress, control concentration and attention, and
organize the learning environment and the learners’ time and energy.
Learning and Performance
Methods:
–
Model development (early stages).
Article Three -- Eccles & Feltovich

HPT Implications:
–
–
–
16
Experts typically prepare over a 10-year period that
includes upward of 10,000 hours of practice.
Transfer of domain-specific skills gets more
challenging as performance gets more complex (i.e.,
tasks have fewer similarities).
Developing domain-general psychological support
skills may be a good complement to meta-cognitive
skills and “weak” (i.e., non-task specific) problemsolving methods for improving performance.
Article Four



17
Title: Improving the quality of evaluation
participation: a meta-evaluation
Authors: Darlene Russ-Eft and Hallie Preskill
Publication: Human Resource Development
International
Article Four -- Russ-Eft & Preskill

Purpose:
–

Sample Research Questions:
–
–
–
18
Describe the successes and failures in obtaining desired
participation as of one organization attempted to implement
several beta-test evaluations of a newly designed training
program.
Why did some clients not fulfill their commitment to complete all
of the evaluation’s requirements?
What kind of incentives would have made clients more likely to
participate fully in the evaluation?
For those clients who participated more fully in the evaluation
process, what factors contributed to their participation?
Article Four -- Russ-Eft & Preskill

Variables:
–
–

Methods:
–
–
–
19
A newly designed customer service program that replaced two
older customer service programs the organization used to offer.
Pre-test, post-test, post-course reaction survey, 3 to 4 weeks
post-course follow-up survey and telephone interviews.
Eight organizations (received free training in return), all of which
has very low response rates to post-training surveys.
Meta-evaluation (i.e., an evaluation of an evaluation) using a
case study method.
Ten 45-minute telephone interviews were conducted with 24
individuals.
Article Four -- Russ-Eft & Preskill

Findings:
–
–
Individual incentives probably would not have helped,
free training was enough.
Sample reasons for none response:





20
They were too busy.
They left the organization.
They changed jobs within the organization.
The office was shut down.
A lack of manager follow.
Article Four -- Russ-Eft & Preskill

HPT Implications:
–
–
–
21
Respondents must know that they have the “authority”
to respond for the organization. Key sponsors within
an organization can help communicate this authority.
Respondents and their organization must have the
“capacity” to participate fully in the beta-test (including
having time available to participate in all aspects of the
evaluation).
The “motivation” to participate in the evaluation must
be built, including “what is in it for them” and the
“importance of their input”.
Reflection Activity


22
Identify one “take away” from the five
research articles previously presented
Discuss your “take away” with the person
sitting next to you
Article Five



23
Title: Institutional Economics and Human
Resource Development
Authors: Greg G. Wang and Rita L. Dobbs
Publication: Advances in Developing Human
Resources
Article Five – Wang & Dobbs

Purpose:
–
–
Expand the “knowledge of institutional economics and
its foundational role and direct applicability to HRD
research and practice.”
Institutional economics…


24
is the study of relationships between the role of human
institutions (i.e., organizations) and the related economic
behaviors.
considers the complex interactions of various institutions. For
example, individuals, organizations, governments, socioculture and history, rather than a simple demand–supply
equation.
Article Five – Wang & Dobbs

Focus:
–
–
25
Orthodoxy neoclassical economics focuses on the allocation of
scarce resources among alternative ends. It emphasizes equilibria,
where the solutions to individual and firm maximization problems are
reached.
Alternatively key assumptions of institutional economics include:
1. Institutions involve the interaction of agents with crucial information
feedbacks.
2. Institutions have common conceptions and routines.
3. Institutions sustain and are sustained by shared conceptions and
expectations.
4. Institutions have relatively durable, self-reinforcing, and persistent
qualities.
5. Institutions incorporate values and processes of normative evaluation.
Article Five – Wang & Dobbs

Assumptions:
Assumption of Institutional
26
Assumptions of Neoclassical
Labor
market
Imperfect labor market with
underemployment as an outcome
Labor market is competitive and reaches full
employment in the long-run
Jobs
Institutional rules and customs allocate
employees’ jobs within unequal job and
training structures
Job allocation is by market clearing prices or
wages
HRD
HRD increases the value of employee to
the organization, and the interventions is
embedded in the work process as well as
taught by coworkers
Training is differentiated by types of skills,
i.e., specific vs. general, which determines
who pays for the investment and who
receives the return
Article Five – Wang & Dobbs

Assumptions:
Assumption of Institutional
Labor
Market
Focus
Internal labor market – promotion, OJT,
career ladders, internal hirers, training,
and other factors are accounted for
decisions.
Learning Learning is a developmental and
reconstitutive process. Learning typically
takes place through and within social
structures and involves adaptation to
changing circumstances. Such
adaptations mean the reconstitution of
the individuals involved.
27
Assumptions of Neoclassical
External Labor Market - pricing, allocating,
hiring, training, and other decisions are
controlled directly by economic variables.
Learning is the cumulative discovery of
preexisting blueprint information, stimulus
and response, or the updating of subjective
probability estimates in light of incoming
data.
Article Five – Wang & Dobbs

HPT Implications:
–
–
–
28
Both Institutional and Neoclassical economic theories
are at play in most organizations, and their application
plays an important role in the application of HPT and
HRD in organizations.
We should be familiar with both, and understand the
consequences of each as they applied in
organizations (especially in these economic
conditions).
Realize that we may be coming an Institutional
Economics perspective but others in our organizations
may have a Neoclassical perspective.
Article Six



29
Title: Investigating the Success of Decision
Making Processes
Authors: Paul C. Nutt
Publication: Journal of Management Studies
Article Six -- Nutt

Purpose:
–
–
–
30
Effectiveness of two decision making processes…
Idea-imposition which “caters to the interests of
powerful stakeholders by looking for ready-made
solutions they can support. A ready-made solution
allows the DM to visualize a course of action and its
ramifications before commitments are made.”
Discovery (i.e., HPT) which “gathers intelligence about
needs, specify desired results, uncover options,
evaluate options according to their benefits, and
implement by working with interest groups and their
perceptions to install the most beneficial option.”
Article Six -- Nutt

Sample Hypotheses:
–
–
–
31
The prospect of success increases when decision
makers follow a discovery process compared to an
idea-imposition process.
The prospect of success declines when decision
makers use an emergent opportunity process,
compared to a discovery process.
For each of the contingencies considered, such as
high and low urgency, the prospect of success will be
greater when decision makers follow a discovery
process, compared to the three other processes.
Article Six -- Nutt

Variables:
–
–
–
–
32
Discovery
 and the “emergent opportunity” variation.
Idea-imposition
 and the “redevelopment” variation.
Frequency of use in relation to success.
Controlling for context and decision type.
Article Six -- Nutt

Methods:
–
–
–
–
33
Analyzed 202 decisions about: financing, internal
operations/controls, personnel policy, marketing,
buildings, technologies, and reorganizations.
Profit and non-profit; top and mid-level managers
Interviews with decision makers used to determine
actions, and decision making process.
Indicators of effectiveness and efficiency are used to
measure success, with at least 2 raters on each
decision.
Article Six -- Nutt

Findings:
–
–
–
34
–
Discovery had far more success than idea imposition.
Decision makers are drawn to idea-imposition
because it seems timely and pragmatic, the results
however indicate little timeliness, low quality, and very
low adoption rates compared to discovery.
Discovery produced far better outcomes, with 90%
sustained adoptions, 85% complete adoptions, rated
as good to excellent, completed in nearly half the time.
Emergent opportunity processes fell a little short of
discovery in effectiveness, and were inefficient.
Article Six -- Nutt

HPT Implications:
–
–
35
Decision made using a systematic HPT process that
begins with a needs assessment (without a solution)
are more likely to produce desired results, results are
more likely to be maintained, and results will be
achieved more efficiently.
It is tempting, but do not abandon your systematic
processes once you think a good solution is identified
(i.e., emergent opportunity), this is less effective and
significantly delays results.
Article Seven



36
Title: Long work hours: a social identity
perspective on meta-analysis data
Authors: Thomas Ng and Daniel Feldman
Publication: Journal of Organizational Behavior
Article Seven -- Ng & Feldman

Purpose:
–

Sample Hypotheses:
–
–
37
Understand the factors that contribute to longer work weeks and
the relationships among hours worked, job performance, and
employee well-being.
Job security, organizational support, job autonomy, opportunities
for learning, organizational tenure, job level, current salary,
number of promotions, career satisfaction, educational level,
general work experience, international work experience, and
social networking are each positively related to employees’ hours
worked.
Hours worked are positively related to job stress, mental strain,
physical health problems, work injuries, and use of tobacco,
alcohol, and drugs.
Article Seven -- Ng & Feldman

Methods:
–
–
–
–
38
Meta-analysis.
20 years of journal articles and 7 dissertations.
Organizational science, applied psychology, and
sociology journals.
199 relevant articles, containing a total of 222
independent samples.
Article Seven -- Ng & Feldman

Findings:
–
–
–
–
–
–
39
Average number of hours worked was 42.4 (SD is 10.2 hours).
Work hours were positively related to job stress and mental
strain.
Once a very high state of stress is reached, each new hour work
creates marginally less additional stress.
One additional work hour in an already jammed week may
exponentially increase work–family conflict.
The relationship of career satisfaction and educational level with
long work hours were stronger for women than for men.
With some exceptions, the results here provide some support for
the position that working long hours is negatively associated with
work effectiveness and work attitudes.
Article Seven -- Ng & Feldman

HPT Implications:
–
–
40
Long work hours can increase stress, mental strain,
and family-to-work conflict.
The factors leading to long work hours are numerous,
thus if we want to manage work hours (increasing or
decreasing) you will have to account for multiple
variables.
Article Eight



41
Title: Organizational diversity, integration
and performance
Authors: Andrew H. Van De Ven, Russel W.
Rogers, John P. Bechara, And Kangyong
Sun
Publication: Journal of Organizational
Behavior
Article Eight -- Van De Ven, et. al

Purpose:
–

Hypothesis:
–
42
Examine how encouraging diverse perspectives (for
example, around a change) influences the
relationship of varied models of organizing on
performance.
Integrative behavior moderates the relationship
between organizational diversity in perspectives and
performance.
Article Eight -- Van De Ven, et. al

Variables:
–
–
–

Methods:
–
43
Integrative behavior as the degree to which employees perceive
that their organization encourages different perspectives and
synthesizes those perspectives in ongoing day-to-day activities
(e.g., openness to ideas and change involvement).
Models of organizing: System/Bureaucracy Model; Market Model;
Profession Model; and Community Model.
Performance: Clinic productivity, net income, and patient
satisfaction.
–
37 primary healthcare clinics go through restructuring changes.
Surveys and secondary-data analysis.
Article Eight -- Van De Ven, et. al

Findings:
–
–

HPT Implications:
–
–
–
44
Although the evidence is not as strong, results show that the main effect
of diverse organizing models on performance is positive.
Diverse mental models for organizing among employees was positively
related to performance, and this relationship was substantially stronger
when integration processes for managing diversity are taken into
account.
Processes that involve employees in decision making, encourage open
debate and integrative methods of conflict resolution, and diversity of
perspectives are associated with higher performance.
Do not focus solely on developing a single vision of change, but rather
engage people with alternative perspectives.
Supports Framing Organizations by Bolman and Soft Systems Theory by
Checkland.
Reflection Activity


45
Identify one “take away” from the five
research articles previously presented
Discuss your “take away” with the person
sitting next to you
Article Nine



46
Title: Performance Implications Of Delayed
Competitive Responses: Evidence From The
U.S. Retail Industry
Authors: Jens L. Boyd And Rudi K. F.
Bresser
Publication: Strategic Management Journal
Article Nine – Boyd and Bresser

Purpose:
–

Research Question:
–
47
Contribute to ongoing efforts to develop theory on
first mover and follower advantages by revisiting the
consequences of fast versus delayed competitive
responses.
What is the nature of the response timingperformance relationship?
Article Nine – Boyd and Bresser

Hypotheses:
–
–
48
Curvilinear relationship between response delay and
responder performance (i.e., on average, there will
be lower success for fast responders, higher success
for responders with intermediate delays, and lower
success for late responders).
Linear relationship between response delay and first
mover performance (i.e., as response delays
increase, the impact on the performance of first
mover decreases).
Article Nine – Boyd and Bresser

Variables:
–
–
–

Methods:
–
–
–
49
Response delay (number of working days between action and
reaction).
Stock price effects (first mover and responder).
Control variables of imitation, tactical actions, firm size, financial
performance, age, industry concentration, and industry sales
growth.
The 17 largest U.S. department stores from 1994 to 2000.
Structured content analysis of 982 press articles.
370 competitive moves including 105 responses.
Article Nine – Boyd and Bresser

Findings:
–
–
–

HPT Implications:
–
–
50
Most responses were imitations, indicating that many responders
intend to copy to catch up, potentially further escalating
competition.
As responders move too quickly or too slowly, this impacts their
performance.
The performance effects of response delays for responders and
first movers do not mirror each other.
Be aware of the risk of responding too fast to competition.
Response timing requires a balance between the risks of
premature entry against missed opportunity.
Article Ten



51
Title: Task knowledge overlap and
knowledge variety: the role of advice network
structures and impact on group effectiveness
Authors: Sze-Sze Wong
Publication: Journal of Organizational
Behavior
Article Ten -- Wong

Purpose:
–
–
52
Study 1 examines whether advice networks and
group knowledge affect subsequent group
performance.
Study 2 was conducted on 40 knowledge-intensive
work groups from three organizations.
Article Ten -- Wong

Sample Hypotheses:
–
–
–
53
The higher the knowledge overlap and variety in a
group, the higher the group effectiveness.
The higher the density of the internal advice network
(i.e., the more each group member seeks advice
from other members), the higher the degree of
knowledge overlap in a group.
The higher the centralization of the internal advice
network (i.e., a few key influential group members),
the lower the degree of knowledge variety in a group.
Article Ten -- Wong

Variables:
–
–
–

Methods:
–
–
54
Knowledge variety is the range of different types of task
knowledge in a group.
Knowledge overlap is the extent to which group members have
common task knowledge
Internal and external social advice network structures (rather than
social or friendship networks).
Study 1: Sample of 80 undergraduate students groups involved in
a complex business simulation task.
Study 2: 50 work groups from a hospital, an industrial diversified
firm, and a high-technology firm.
Article Ten -- Wong

Findings:
–
–
–
–
55
Knowledge overlap was not significantly associated
with group effectiveness in both studies.
Knowledge variety was positively associated with
group effectiveness in both studies.
Internal network centralization was negatively
associated with knowledge variety in both studies.
Internal network centralization was positively related to
group effectiveness in Study 1, but not Study 2.
Article Ten -- Wong

HPT Implications:
–
–
–
56
Promote knowledge breadth for groups involved in
knowledge-intensive work (such a cross-disciplinary
teams).
Foster ties with many different individuals, both within
and outside of your group, to promote knowledge
variety.
Network with external groups.
Article Eleven (BONUS)



57
Title: The Effect of Performance Support and
Training as Performance Interventions
Authors: Frank Nguyen & James Klein
Publication: Performance Improvement
Quarterly (an ISPI publication)
Article Eleven -- Nguyen & Klein

Research Questions:
–
–
–
–
58
What combination of performance support and
training maximizes user performance?
What combination of performance support and
training do users prefer?
Do users access performance support more if they
have not had training prior to task performance?
What combination of performance support and
training minimizes the time to complete a task?
Article Eleven -- Nguyen & Klein

Variables:
–
–

Methods:
–
–
59
EPSS, EPSS + web-based training, and web-based
training (with no partial-task practice in web-based
training).
User performance on the task; user attitudes; use of
EPSS; time in training; and time to complete task.
–
Focus on use of tax preparation software.
Posttest-only control group design, using assessments
and interviews.
78 participants from various organizations.
Article Eleven -- Nguyen & Klein

Findings:
–
–
Training & EPSS and EPSS-only had significantly higher scores
on the task than training-only.
Time on task




HPT Implications:
–
–
60
EPSS-only = 26 minutes
Training & EPSS = 31 minutes
Training only = 1.5 hour
–
Adding performance support to training can significantly increase
performance.
EPSS may be sufficient by itself in many situations, though the
addition of training demonstrated some attitudinal benefits.
Finding the right “mix” is key.
Reflection Activity


61
Identify one “take away” from the four
research articles previously presented.
Discuss your “take away” with the person
sitting next to you.
Survey Was Sent To…
Richard Clark (USC)
• Richard Swanson (Univ. Texas, Tyler)
• Darlene Russ-Eft (Oregon State)
• Tim Hatcher (NCSU)
• Rob Branch (UGA)
• Jim Klein (ASU)
• Steven Condly (UCF)
• Jan Visser
• Jeroen Van Merrienboer
• Gordan Rowland (Ithica)
•
62
•
•
•
•
•
•
•
•
•
•
Jim Altshuld (OSU)
Harold Stolovich
Ingrid Guerra-Lopez (Wayne)
John Wedman (Missouri)
Scott Schaffer (Purdue)
Maria Cseh (GWU)
Julia Storberg-Walker (NCSU)
Hillary Leigh
Tony Marker (BSU)
Yonnie Chyung (BSU)
Best of 2008: Research findings
you may have missed.
Ryan Watkins, Ph.D.
George Washington University
www.gwu.edu/~etl
www.ryanrwatkins.com
63