Assessing Predictions
1
Assessing Predictions of Domestic Violence:
The Pre-employment Psychological Evaluation
Stephen F. Curran
Abstract: Domestic violence emerges as one of the characteristics that can be isolated
by a statistical technique known as receiver-operating characteristic (ROC) analysis.
This paper also describes the psychometric problems associated with pre-employment
predictions as well as the recently revised Pre-employment Screening Standards
developed by the Police Psychological Services Section of the International Association
of Chiefs of Police.
KEY WORDS: screening, pre-employment evaluation, psychological evaluation,
guidelines, violence prediction.
Address correspondence concerning this article to Stephen F. Curran, Ph.D., Atlantic
OccuPsych, Post Office Box 4071, Timorium, Maryland 21094-4071.
Assessing Predictions
2
ASSESSING PREDICTIONS OF DOMESTIC VIOLENCE:
THE PRE-EMPLOYMENT PSYCHOLOGICAL EVALUATION
The selection of law enforcement personnel who exhibit stable emotional and
behavioral functioning has become critical to policing agencies. Excessive force
complaints, community policing initiatives, police suicides, violence against intimates
and allegations of negligent hiring have demonstrated that hiring decisions must take
into account the psychological characteristics of an applicant.
During the past 20 years, the field of police psychological screening has expanded
dramatically (Curran, 1996). In 1986, the Police Psychological Services Section of the
International Association of Chiefs of Police (IACP) adopted its first set of guidelines to
assist police chiefs and psychologists conducting programs to evaluate police
applicants (Solomon, 1992). These guidelines provided a generic approach to defining
elements of an evaluation program facilitating the valid and reliable selection of
psychologically suitable police officers. Since then, the majority of police agencies now
conduct pre-employment psychological evaluation of applicants. A number of surveys
have assessed the methodologies employed by psychologists. Scrivner’s (1994) report
on the use of excessive force emerged as the most notable effort.
Perhaps the most significant development since the 1986 IACP guidelines involved
the enactment on July 26, 1990, of the Americans With Disabilities Act (ADA, 1990). As
of 1992 the ADA applied to most law enforcement agencies. This federal law, designed
to protect individuals with disabilities, posed many challenges to administrators of police
agencies and psychologists conducting pre-employment evaluations. While the initial
Assessing Predictions
3
fears about active drug users and psychotics succeeding in gaining employment have
proven unfounded, the law has profoundly affected pre-employment procedures. The
increased tendency to view pre-employment psychological screening as a Apass or fail”
evaluation has emerged as an issue. This approach alters the threshold of suitability
and, consequently prediction accuracy.
The Police Psychological Services Section of the IACP came into existence in 1984.
In his history of the Section, Shaw (1992) noted that the formation resulted from police
psychologists’ search for an organization within which direct, rapid, two-way
consultation between police chiefs and psychologists could occur. The IACP
commenced the process of revising its pre-employment psychological guidelines in
1994. This partially addressed aspects of evaluations affected by the ADA and also
incorporated generally accepted standards of practice. The revised guidelines received
approval from Section members during the spring of 1998. The psychologist engaging
in pre-employment psychological evaluations must know the issues inherent to police
screening.
PRE-EMPLOYMENT GUIDELINES
A brief review of three of the twenty-two guidelines (See IACP paper at end for
complete guidelines) relating to the discussion of predicting domestic violence follows.
The introductory section of the guidelines defines public safety positions as the type
of employment to which they apply. While the guidelines prominently address
applicants for positions with the authority to make arrests, they apply to related law
enforcement positions as well (e.g., correctional officers, police aides, and emergency
dispatchers). Further, these guidelines constitute recommended rather than mandatory
Assessing Predictions
4
policy.
Guideline item numbers 16 and 17 cover two important considerations when
evaluating an applicant; they concern the need to focus on essential job functions
rather than diagnostic issues.
16. While a clinical assessment of overall psychological suitability may be made,
clinical diagnoses or psychiatric labeling of candidates should be avoided when
the goal of the assessment is to identify candidates whose psychological traits
may adversely affect specific job performance. In all cases, the screening should
be focused on an individual candidate’s ability to perform the essential functions
of the position under consideration.
17. Specific cut-off scores should be avoided, unless there is clear statistical
evidence that such scores are valid and have been cross-validated in research
studies by the test developer and/or in the agency where they will be used. If cutoff scores are utilized, the report should acknowledge their use and the basis for
using the specific cut-off level. Conclusions concerning a candidate’s
qualifications should be based on consistencies across data sources rather than
on a single source.
The guidelines recommend a description of psychological traits identified from more
than one data source. Linking these traits to potential deficits in behavioral functioning
(e.g., impulse control, motor vehicle violations, and job absences), emotional stability
(e.g., excessive sick leave, sleep and appetite changes) and social judgement attributes
make up the primary factors for evaluation.
Guideline number 14 states, interviews should be scheduled to allow for sufficient
Assessing Predictions
5
time to cover appropriate background and test results verification. Evaluators who have
conducted five to ten minute interviews of applicants have precipitated a number of
incidents including the initiation of criminal investigations. Because of such concerns,
many federal, state, and local agencies require documentation of 45-minute interviews.
This guideline recommends interviews of at least 30 minutes. Some interviews may
require 90 minutes to adequately assess the behavioral history of an applicant in
conjunction with psychological testing.
Havassy (1996) recommends a semi-structured interview. Interviews should have a
standard format to reduce interviewer bias and allow for probing in areas related to
essential job functions. Psychological testing results comprise an integral component of
the interview.
Emotional, behavioral, and social psychological characteristics remain essential to
successful performance of law enforcement tasks and form the main focus of an
interview. Emotional factors include stability in impulse and anger control, flexibility,
maturity, self-esteem, and management of stress. Behavioral stability elements include
an applicant’s capacity to follow rules and regulations, response to trauma, prior
substance use patterns, and job adjustment history. Social factors relate to work and
family relationships, interpersonal communication skills, assertiveness, empathy, and
cultural sensitivity. These domains comprise the breadth of interview elements
designed to elicit, in conjunction with testing data, an adequate representation of an
applicant’s capacity to perform the varied duties of a law enforcement officer.
PREDICTIONS OF FUTURE BEHAVIOR: MOVING BEYOND WIZARDRY
Mental health professionals have liability concerns. To maintain evasiveness, they
Assessing Predictions
6
often choose to hide behind the protective cloak of the law, which cannot predict future
behavior. This stance contradicts available data. We can predict future behavior. The
question really concerns the confidence levels we want to apply to those predictions.
This paper cannot include a complete discussion of quantifying prediction accuracy.
Interested readers can refer to Mossman (1994) for a detailed description of issues in
predictions of behavior. The central point of this paper focuses on testing and interview
data as potent predictors of subsequent performance. The pre-employment evaluators
using this data for employment recommendations need to consider the tradeoffs
(sensitivity and specificity).
Some relevant examples of successes serve to illuminate behavior predictions
follow. Jones, Beidleman, and Fowler (1981) used multivariate procedures to correctly
classify 72.9% of inmates who would commit violent acts and 80.6% of inmates who
would not commit violent acts while incarcerated based on scales of the Minnesota
Multiphasic Personality Inventory (MMPI) alone. Lidz, Mulvey, and Gardner (1993), in a
prospective study of emergency room psychiatric patients, found mental health
clinicians accurately predicted future violence in 63% of male patients. However, their
predictions about women and subsequent violence failed to do better than chance.
Sandberg, McNiel, and Binder (1998) reported on data of particular interest to the
discussion of domestic violence. These researchers examined psychiatric inpatients
who stalk, harass or threaten hospital staff after discharge. They found these patients to
have the diagnosis of personality disorder or paranoid disorder; and a pre-admission
history of fear-inducing or physically assaultive behavior. Inwald, Resko, and Trainer
(1997), using the Inwald Personality Inventory, conducted a study of 21 known violent
Assessing Predictions
7
police officers for which they had access to pre-employment testing data. A discriminant
analysis correctly identified 85.5% as either violent or nonviolent.
Specificity and sensitivity continue to comprise the primary issue related to
predicting domestic violence in police officer applicants. Specificity relates to the
number of true negatives (those applicants not violent) divided by the sum of true
negatives and false positives. Sensitivity involves the number of true positives
(predicted violent applicants who actually committed violence) divided by the sum of the
true positives and the false negatives. Predictions less influenced by base rates,
especially the low estimated occurrence rates for domestic violence hold a great deal of
interest for us. Of course, 100% predictive accuracy lies beyond the reach of
psychologists now and in the reasonably foreseeable future. Human behavior eludes
complete quantification.
A statistical procedure, known as receiver operating characteristic (ROC) analysis
accommodates calculating sensitivity-specificity pairs independent of base rates. These
pairings establish a threshold for accuracy prediction from which measuring prediction
above chance becomes possible. This procedure establishes the threshold for an
evaluator (receiver) and serves as the detector (an alarm) in decision making.
Evaluators using a minimal Apass or fail rating of psychological fitness may favorably
recommend more applicants prone to violence. This results from a lower threshold
(sensitivity-specificity ratio). More stringent evaluation ratings, such as multi-tiered
suitability ratings, may provide above average or superior recommendation ratings to
fewer violence prone applicants, but rates low or rejects otherwise suitable applicants
who never subsequently engage in violent behaviors. These constitute critical issues for
Assessing Predictions
8
evaluators and future research.
CONCLUSION
This paper introduces the statistical methodology of ROC analysis into the
discussion of pre-employment psychological screening of law enforcement applicants.
Recognition of this methodology for future research and current practice appears
relevant when considering the base rates of domestic violence among law enforcement
officers. Mental health professionals have demonstrated in research on other clinical
groups a better than random chance of predicting violence. Sound reasoning to support
accuracy of prediction by pre-employment evaluators exists. Adhering to current preemployment guidelines generally accepted as the standard of practice will enhance the
accuracy of behavioral predictions about future job performance.
Assessing Predictions
9
IACP POLICE PSYCHOLGICAL SERVICES SECTION GUIDELINES FOR PREEMPLOYMENT PSYCHOLGICAL EVALUATION SERVICES (1998)
OVERVIEW
The following statements are guidelines for professional practice in the area of
pre-employment psychological evaluations of candidates for public safety positions.
These positions include but are not limited to positions where incumbents have arrest
authority or the legal authority to detain and/or confine individuals. These guidelines are
presented as a recommended professional policy for public safety agencies and
individuals who are charged with the responsibility of conducting defensible preemployment psychological screening programs.
DEVELOPMENT
1. Pre-employment psychological assessments should be used as one component of
the overall selection process.
2. Before conducting their own clinical assessments of candidates, practitioners should
be familiar with the research literature available on psychological testing for public
safety positions, as well as the state and federal laws relevant to this area of practice,
including the Americans with Disabilities Act (ADA).
3. Only licensed or certified psychologists trained and experienced in psychological test
interpretation and law enforcement psychological assessment techniques should
conduct psychological screening for public safety agencies.
4. Data on attributes considered most important for effective performance in a particular
Assessing Predictions
10
position should be obtained from job analysis, interview, surveys, or other appropriate
sources.
5. Efforts should be made to provide agency administrators with information regarding
the benefits and limitations of psychological assessment procedures so that realistic
goals may be set.
6. Provisions should be made for the security of all testing materials (e.g., test
booklets). Provisions should also be made for the security of, access to, and retention
of the psychological report and raw data.
7. Prior to the administration of any psychological instruments and psychologist
interview, the candidate should sign an informed consent to the conditions of the
evaluation.
TESTING
8. A test battery including objective, job-related, validated psychological instruments
should be administered to the applicant. It is preferable that test results be available to
the evaluator before screening interviews are conducted.
9. Written tests selected should be validated for use with public safety candidates.
10. If mail order or computerized tests are employed, the licensed or certified
psychologist conducting the follow-up interview should verify and interpret individual
results.
11. The pre-employment psychological evaluation must be conducted in accordance
with the Americans with Disabilities Act (ADA) guidelines after a conditional offer of
employment has been tendered. Although some personality tests, and other methods of
inquiry, may be conducted at the pre-offer stage, they do not in and of themselves
Assessing Predictions
11
constitute the pre-employment psychological evaluation.
INTERVIEW
12. Individual, face-to-face, interviews with candidates should be conducted before a
final psychological report is submitted.
13. A semi-structured, job-related interview format should be employed with all
candidates.
14. Interviews should be scheduled to allow for sufficient time to cover appropriate
background and test results verification.
EVALUATION
15. Public safety agency administrators directly involved in making employment
decisions should be provided with written reports. These reports should evaluate the
suitability of the candidate for the position based upon an analysis of all psychological
material including test data and interview results. Reports to the agency should contain
a rating and/or recommendation for employment based upon the results of the
screening, justification for the recommendation and/or rating, and any reservations that
the psychologist might have regarding the validity or reliability of the results.
16. While a clinical assessment of overall psychological suitability may be made, clinical
diagnoses or psychiatric labeling of candidates should be avoided when the goal of the
assessment is to identify candidates whose psychological traits may adversely affect
specific job performance. In all cases, the screening should be focused on an individual
candidate’s ability to perform the essential functions of the position under consideration.
17. Specific cut-off scores should be avoided, unless there is clear statistical evidence
Assessing Predictions
12
that such scores are valid and have been cross-validated in research studies by the test
developer and/or in the agency where they will be used. If cut-off scores are utilized, the
report should acknowledge their use and the basis for using the specific cut-off level.
Conclusions concerning a candidate’s qualifications should be based on consistencies
across data sources rather than on a single source.
18. Clear disclaimers should be made so that reports evaluating current emotional and
behavioral traits or suitability for a public safety position will not be deemed valid after a
specific period of time.
FOLLOW-UP
19. Pre-employment test results should not be used for purposes other than making
pre-employment decisions and for monitoring the candidate during the probationary
period. Follow-up research may be conducted with agency approval and where
individual identities are protected. Pre-employment reports should not be used for
positions or agencies not expressly considered by the psychologist at the time of the
evaluation, nor should they be used for subsequent disciplinary or forensic matters.
20. Continuing collaborative efforts by the hiring agency and evaluating psychologist
should be made to validate final Asuitability@ ratings using behavioral criteria
measures.
21. Each agency should maintain adverse impact analyses in order to detect any
discriminatory patterns of the psychological screening program.
22. Psychologists should be prepared to defend their procedures, conclusions, and
recommendations if a decision based, even in part, on psychological results is
challenged.
Assessing Predictions
13
Assessing Predictions
14
REFERENCES
Americans with Disabilities Act of 1990 (1991). Pub. L. No. 101-336, sec. 104 Stat.
328.
Curran, S. F. (1996). Positioning psychological services for the 21st century. In J. T.
Reese & R. M. Solomon (Eds.), Organizational issues in law enforcement. (pp. 375382). Washington, DC: Federal Bureau of Investigation.
Havassy, V. (1996, October) Psychological preemployment practices survey results
of IACP PSS members. Paper presented at the meeting of the Police Psychological
Services Section of the International Association of Chiefs of Police, Phoenix, AZ.
International Association of Chiefs of Police. (1986, revised 1998). Guidelines for
providers of pre-employment psychological evaluation services to law enforcement
agencies. (Police Psychological Services Section). Alexandria, VA: Author.
Inwald, R. E., Resko, J. A., & Trainer, B. (1997). Assessing violent/aggressive
characteristics: current studies with public safety officers/violent offenders. Kew
Gardens, NY: Hilson Research.
Jones, T. Beidleman, W. B. & Fowler, R. D. (1981). Differentiating violent and
nonviolent prison inmates by use of selected MMPI scales. Journal of Clinical
Psychology, 37 (3), 673-678.
Lidz, C. W., Mulvey, E. P., & Gardner, W. (1993). The accuracy of predictions of
violence to others. Journal of the American Medical Association, 269 (8), 1007-1011.
Mossman, D. (1994). Assessing predictions of violence: being accurate about
accuracy. Journal of Consulting and Clinical Psychology, 62 (4), 783-792.
Sandberg, D. A., McNiel, D. E., & Binder, R. L. (1998). Characteristics of psychiatric
Assessing Predictions
15
inpatients who stalk, threaten, or harass hospital staff after discharge. American Journal
of Psychiatry, 155 (8), 1102-1105.
Scrivner, E. (1994). The role of police psychology in controlling excessive force.
(Research Report NCJ 146206). Washington, DC: U.S. Department of Justice.
Shaw, J. H. (1992). Psychological services section history. Police Chief, May, 54.
Solomon, R. M. (1992). Developing professional guidelines. Police Chief, May, 5356.