Technology and the
Practice of I/O Psychology
Nancy T. Tippins
Personnel Research Associates, Inc.
April 10,2003
Goals
•
Develop a framework for evaluating
technological interventions in the practice
of I/O psychology by
– Exploring how technology affects the practice
of I/O psychology today and in the future
– Applying systematically a process for
evaluation of a technological intervention
Agenda
•
Review examples of the use of technology in
the practice of I/O psychology (25 minutes)
– Example
– Problems
•
•
•
Offer an approach for evaluating technology
(10 minutes)
Discuss approach to specific content areas (15
minutes)
Share feedback on the approach (10 minutes)
Why Use Technology in
I-O Psychology?
• Speed
– Volume
• Distance/Geography
• Availability
– Just-in-time delivery
• Costs
• Integration with other systems
• Innovative approaches
Areas of I-O Psychology
•
•
•
•
•
•
•
•
Training
Employee Development
Surveys
Multi-rater Feedback
Performance Management
Succession Planning
Team Building
Selection
Training Examples
•
•
•
•
•
•
Communications
Course Development
Training Delivery
Simulations and Work Samples
Assessment
Data Collection
Training Issues
• Adapting computer-based training to
different learning styles may be difficult.
• Computer-based training may eliminate or
reduce the opportunities for social
interaction.
• Computer-based training often requires
personal discipline to complete training
without frequent human interactions.
Employee Development
Examples
• Communications
• Assessment
• Data Collection and Record Keeping
Employee Development
Issues
• Some development programs are difficult to complete
independently because they require
–
–
–
–
Discipline to complete activities independently
Introspective capability to assess own behavior
Acceptance of negative or critical feedback
Accurate and honest assessment
• Technology cannot replace some activities, i.e.,
observation and feedback.
• Some employees are afraid to be accurate.
• Effect on relationships is not known.
Survey Examples
• Communications
• Administration
• Analysis and Reporting
Survey Issues
• The level of confidentiality that can be
maintained is often unclear.
– Responses
– Reports
• Ease of computer administration increases
the risk of overuse of survey methodology.
Multi-Rater Feedback
Examples
• Communications
• Administration
• Analysis and Reporting
Multi-Rater Feedback
Issues
• Computer cannot anticipate reactions to
information and assist the rated individual
in working through strong emotional
responses.
• The level of confidentiality is unclear.
• Overuse is a potential problem.
Performance
Management Examples
•
•
•
•
Communications
Data Collection
Reports/Databases
Feedback and Coaching
Performance
Management Issues
• The level of confidentiality that can be
maintained is often unclear.
– Responses
– Reports
• Open, two-way communication may be
difficult when participants are not face-toface.
Succession Planning
Examples
•
•
•
•
Communications
Data Collection
Reporting
Access to Data
Succession Planning
Issues
• Controlling access to succession planning
data must be controlled in most
organizations.
• The structure of the organization must be
known and maintained.
Team Building
• Communications
– Information
– Relationship building
– Problem solving
Team Building Issues
• The effect of computer-based interactions
on relationship building is unknown.
• Opportunities for informal interactions may
be lacking.
• Confidentiality and privacy may be a
concern of participants.
Selection Examples
• Communications
–
–
–
–
–
–
Test preparation
Test administration session logistics
Interpretation guidelines
Administration instructions and rules
Feedback
Reports
Selection Examples
• Test Administration
–
–
–
–
–
Previews
Instructions
Presentation
Scoring
Storage
• Job Analysis Questionnaires
• Validation Questionnaires
• Novel Item Types
Selection Issues
• Computer administration may change the construct
measured or alter the appropriateness of the normative
data.
• Computer administration changes test taking strategies.
• Test takers may not feel comfortable using computer
despite assurances that computer ability required is
minimal.
• Reliability of timing and data recording are requirements
that may difficult to achieve.
• Security may be a problem particularly with
internet/intranet distribution.
Selection Issues
• ADA accommodations may be more difficult to
provide.
• Applicants may have potentially negative reaction
to “low touch” selection processes.
• Not all test takers have access to computer-based
information about the testing process.
Advantages
•
•
•
•
•
•
•
Communications
Delivery
Timing
Costs
Speed
Integration with other systems
New ways of doing things
Concerns
• Hardware and Software
–
–
–
–
–
Availability of equipment
Costs
Privacy
Appropriate software
Support/Maintenance
• Skill requirements for I/O psychologists
• Training required for user
• Effects of technology on interpersonal interactions
• Limitations on computer-based training
Concerns
•
•
•
•
•
•
•
User friendliness
User reluctance
User appropriateness
Need or desire for human sounding board
Anonymity, confidentiality, and privacy
Security
Ethics
Approach to Evaluating
Technology
• See Handouts
Example: Evaluating
Technology
• See Handout
• 15 minutes
• Discuss
– Questions that should be added to the
evaluation approach
– Application for content area