Professional Psychology: Research a 2000, Vol. 31, No. 2, 141-154 Copyright 2000 by the American Psychological Association, Inc. 0735-7028/00/55.00 DOI: I0.1037//0735-7028.31.2.I41 Psychological Test Usage: Implications in Professional Psychology Wayne J. Camara, Julie S. Nathan, and Antonio E. Puente American Psychological Association Do psychological assessments require more time than third parties and managed care are willing to reimburse? A survey of clinical psychologists and neuropsychologists was conducted to evaluate the current uses of psychological assessment instruments. Respondents reported their use of tests for 8 different areas of assessment, the average time spent in performing various assessment services and other assessment practices. Results suggested that a majority of neuropsychologists devote a substantial portion of their time to assessment, but only 12% of clinical psychologists reported spending more than 10 hr in assessment-related practice each week. The authors describe the typical time required to administer, score, and interpret various tests and assessments; factors that affect the time required to conduct assessments; and provide a current ranking of the most frequently used assessments in clinical and neuropsychology. How long does it take to conduct comprehensive psychological sional psychologists. With the advent of managed care, psycho- assessments? Does the time required for assessment activities logical services such as assessment services are increasingly com- depend on the nature of the assessment and the presenting prob- pensated at standard approved rates that may or may not reflect the lems? What assessments are most commonly used and do they level of effort and time required to perform these services. In differ by the nature and function of the assessment? What are the today's managed care environment, research-based knowledge of implications of these issues for psychologists conducting assess- the time it actually takes clinicians to administer, score, interpret, ment services in today's managed care environment? What strat- and write reports could "curb abuse of testing benefits . . . we egies may psychologists invoke to overcome the multiple obstacles know has gone on" (R. DeLapp, personal communication, Novem- imposed by managed care and demonstrate the efficacy and ex- ber 5, 1991). The present study was designed to provide accurate tensiveness of comprehensive assessment practices? We examined information on current psychological test usage. these and other related questions through a survey of a sample Louttit and Brown (1947) first documented psychological test of clinical and neuropsychologists who conducted assessment usage in 1935 and 1946. Subsequent national surveys to estimate activities. psychological test usage were reported in 1961 (Sundberg, 1961) Psychological assessment has been a defining practice of pro- and 1969 (Lubin, Wallis, & Paine, 1971). Lubin et al. (1971) found fessional psychology since the field's inception. Over the past that the top 10 most often used tests included 4 projective tech- several decades, national surveys of psychological test usage have niques, the Rorschach Psychodiagnostic Test (often called the enriched our knowledge of the assessment practices of profes- Inkblot Test), Thematic Apperception Test (TAT), the Draw-A- WAYNE J. CAMARA received his PhD in organizational behavior from the and the survey content were provided by Thomas Boll, Gordon Chelune, Lee Anna Clark, Munro Cullum, Elena Eisman, Alan Entin, Larry Fried- University of Illinois at Urbana-Champaign. He is executive director of research and development at the College Board in New York City. He is also president-elect of the American Psychological Association's (APA's) man, Douglas Jackson, A. John McSweeney, John Mendoza, Robert Division of Evaluation, Measurement, and Statistics. This research was Thompson, Michael Westerveld, Nancy Wilcockson, and members of the Resnick, Gayle Rettig, Cecil Reynolds, Charles Spielberger, Laetitia initiated when Dr. Camara was assistant executive director of scientific executive committees of Divisions 12 (Clinical Psychology). 40 (Clinical affairs at AP^. JULIE S. NATHAN received her MSEd in 1998 and will be completing her Neuropsychology), and 42 (Psychologists in Independent Practice), as well PhD in school psychology from Fordham University in May 2000. She is William C. Howell allocated staff and provided financial support for the currently a clinical psychology intern at Montefiore Medical Center/Albert Einstein College of Medicine. survey and study. Jessica Kohout and Marlene Wicheski coordinated the data collection and mailing. Peter Pfordresher managed the data entry and ANTONIO E. PUENTE received his PhD from the University of Georgia. coding of all survey responses. Georgia Sargeant copyedited the mono- as members from the Board of Professional Affairs. Russ Newman and He is professor of psychology at the University of North Carolina at graph. Finally, Geoffrey Reed, Heather Roberts-Fox, Dianne Maranto, and Wilmington and maintains an independent practice limited to clinical neuropsychology. He is a past president of the National Academy of Amy Rabinove assisted in coordinating review and comments from APA governance on the design of this study and the draft and final reports. Neuropsychology. A MUCH MORE COMPREHENSIVE REPORT of the study, with additional data THIS RESEARCH WAS CONDUCTED with the support of the APA Practice and and analyses, is available as an unpublished manuscript from the APA Science Directorates. We thank the many psychologists and neuropsychologists who assisted in reviewing and commenting on the design of the Practice Directorate (Camara, Nathan, & Puente, 1998). CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Wayne survey, the study methods, and the definitions and assessment areas used in J. Camara, College Board, 45 Columbus Avenue, New York, New York the study. Specifically, recommendations and reviews of the list of tests 10023. Electronic mail may be sent to wcamara@collegeboard.org. 142 CAMARA, NATHAN, AND PUENTE Person Test (DAP), and the House-Tree-Person (H-T-P) Projective Technique, as well as 3 intelligence tests, the Wechsler Adult Intelligence Scale (WAIS), the Wechsler Intelligence Scale for Children (WISC), and the Stanford-Binet Intelligence Scale. The others in the top 10 were the Bender Visual Motor Gestalt Test (hereinafter the Bender Gestalt), the Minnesota Multiphasic Personality Inventory (MMPI), and Sentence Completion tests. In the 10-year interval between Sundberg's (1961) and Lubin et al.'s (1971) studies, intellectual measures such as the WAIS and WISC began to displace some of the traditional projective personality assessment instruments, but these were still preferred over more empirical measures of personality. The MMPI, in fact, was the only objective personality test found in the top-10-ranked instruments of both studies. These earlier studies examined the rank ordering of tests but did not provide any information regarding specific test usage. In their 1975 survey of psychological test usage, Brown and McGuire (1976) asked professionals from community mental health agencies and hospitals not only which tests were used most often but also which were used most for intellectual versus personality assessments, and which were used most for different age groups. Brown and McGuire's (1976) study found little overall change in the top-ranked tests from 1969 to 1975. The most notable changes from 1971 to 1975 were the addition of the Slosson Intelligence Test for Children and Adults and the Kinetic Drawing System for Family and Schools to the list of most frequently used tests, and the increase in popularity of the WISC. Another national survey of psychological test usage was performed in 1982 (Lubin, Larsen, & Matarazzo, 1984). Of the 6 projective techniques ranked in the top 10 in 1969, 5 were still in the top 10 in 1982, and the MMPI was ranked second in 1982, demonstrating its steadily increasing use. For the first time, the Stanford-Binet was not among the top 10 tests used in the 1982 survey, ranking 15th. Before 1984, surveys on testing practices had been conducted with members of APA Division 12; however, now surveys of assessment practice included members of the Association for Advancement of Behavior Therapy (AABT) and the Society for Personality Assessment (SPA; O'Roark & Exner, 1989). Results revealed that projective techniques, specifically the Rorschach, TAT, Sentence Completion, and H-T-P tests, were in the SPA's top 10 most frequently used testing instruments. The objective personality measures in the top 10 were the MMPI (ranked third), the Children's Apperception Test (CAT), and the Sixteen Personality Factor Questionnaire (16PFQ). For cognitive assessment, the Wechsler scales were ranked first, indicating their growing popularity. The overwhelming majority of SPA members (90% of those who responded) indicated that they primarily used assessments for diagnostic purposes, and 53% also used testing as an indicator of what type of therapy would be most effective. Addressing a void in the survey research to date, Ball, Archer, and Imhof (1994) surveyed practitioners for their perceptions of the time required to administer, score, and interpret psychological test instruments. They mailed surveys to small samples of clinical psychologists, neuropsychologists, and SPA members. The instruments listed as typical for test batteries corresponded closely with the most highly ranked tests in previous surveys (Ball et al., 1994). Results showed that clinicians in private practice were administering longer test batteries than clinicians in primary employment settings. These data raised questions "as to whether clinical, financial, or some other consideration may underlie test selection" (Ball et al., p. 247). Overall, data suggest surprisingly little substantive change in the ranking of the most often used (i.e., popular) instruments over the last several decades. For example, the Stanford-Binet and the Rorschach have been in the top 20 since 1935. According to Lubin et al. (1984), the rank-order correlation between ranks in 1969 and 1982 was .89 (p < .001). The use of projective techniques has persisted in popularity since 1969. Specifically, the Rorschach, TAT, H-T-P, and DAP tests have been among the top 10 test instruments used in each decade (Lubin et al., 1984). The instruments that clinical psychologists use the most seem to cluster consistently into a core battery that includes the WAIS or WISC, the MMPI, and several of the more popular projective tests, including the Rorschach and TAT (Watkins, 1991). This has not changed much since the 1960s. Watkins noted that though the data suggest a steady decline in the percentage of time that practicing psychologists devote to assessment (from 44% in 1959, down to 22% in 1982 over all five settings) most of them currently spend a fair portion of their time conducting assessment services. The Test Usage Survey The present study was designed to gather information on the current use of psychological assessment instruments by clinical psychologists and neuropsychologists. This study diverged from previous studies of psychological test usage in several ways. First, the sample was limited to clinical psychologists and neuropsychologists who reported that assessment services accounted for a substantial part of their practice. Second, this survey asked more specific questions, such as (a) the amount of time spent per week conducting assessments and (b) whether assessments were intellectual, developmental, adaptive-functional, or neuropsychological in nature. Participants included 1,002 members of the National Association of Neuropsychology (NAN) and 1,500 clinical psychologists from the American Psychological Association (APA), randomly selected from the respective databases of each institution. The neuropsychologists were randomly selected from among approximately 2,300 NAN members. The clinical psychologist sample was randomly selected from a population of approximately 35,000 doctoral members of APA in independent practice who specialize in providing mental health services as their primary or secondary positions. Instrumentation We mailed all participants the six-page "Survey of Test and Assessment Use in Professional Psychology." The initial draft of this survey was reviewed by more than 20 psychologists who were members of APA Division 40 (Clinical Neuropsychology), Division 42 (Psychologists in Independent Practice), and APA gover- SPECIAL SECTION: PSYCHOLOGICAL TEST USAGE 143 nance groups with expertise in assessment.1 In addition to evalu- appropriate intervention strategies. Assessments may entail obtaining ating the proposed questions, reviewers were asked to study the an overall index of development or securing a detailed assessment of the child's level of functioning across different areas (e.g., motor draft list of tests and identify additional instruments that should be development, language development, social development, etc.). The added to the list. We pilot tested a revised survey with an addi- focus may be on documenting changes over time through repeated tional 20 practitioners, and they also identified a number of addi- assessments (Johnson & Goldman, 1990). tional tests to be included in the list of tests for the final survey. 5. Intellectual or achievement. Intelligence tests assess learning that Their comments and suggestions were incorporated, raising the occurs in a wide variety of life experiences. Achievement tests are number of tests for the final survey from 75 to 120. heavily dependent on formal learning acquired at school or home. The survey included items inquiring about (a) the amount of With children, the main goal(s) of intellectual or achievement assess- time devoted to assessment services in a typical week, (b) use of ments include determining the nature of the child's learning or be- tests for eight different areas of assessment and the use of specific havior problems (i.e., the child's strengths and weaknesses in abilities assessments, and (c) participant's credentials and experience. Par- related to learning). With adults, these assessments are used to deter- ticipants devoting 4 hr a week or less to assessment services were mine the level of intellectual functioning or knowledge in one or more asked to stop and return the survey, because the survey designers specific domains (e.g., math, science). Evaluations are usually based felt that individuals who engaged minimally in assessment services on norms for similar-aged individuals (e.g., young children, older might not have enough recent experience to estimate the time adults). A comprehensive battery should include a measure of global required for completing different services. Those engaged in as- intelligence and measures of the information-processing skills involved in comprehension, visualization, memory, reasoning, and sessment services for more than 4 hr per week completed the judgment (Sattler, 1992). remaining survey items. 6. Neurobehavioral clinical examinations. These instruments are used to evaluate the extent that individual's social and emotional function- We asked participants about the use of assessments for eight different areas of assessment. We decided on these practice areas ing are affected by the brain and potential brain-impaired processes. for assessment services through a consensus process involving 45 The participants are most often adults and young adults who may have members of APA divisions and governance groups with expertise encountered a head injury or are suspected of some degenerative in assessment. The areas and brief definitions adapted from the illness. These assessments are used to examine daily problem-solving survey are strategies (e.g., daily living tasks, independence) and reasoning abilities as they may be affected by the brain and any brain impairment. Assessments in this area are similar to those used in more generalized 1. Adaptive-functional behavior. The assessment of adaptive behavior is concerned with the degree to which individuals function indepen- neuropsychological assessment in that they involve a variety of in- dently and meet satisfactorily the demands of personal and social formal and formal measures about a person's level of neurological responsibility imposed by the culture. Assessments are designed to functioning. determine a person's competence in meeting the independence-related 7. Neuropsychological assessments. Most participants of these assess- needs and social demands of the environment (e.g., communication, ments are suspected of having had some type of physical brain injury, daily living skills, socialization, and motor skills). Instruments such as such as a blow to the head, a stroke, or carbon monoxide poisoning. behavior scales, behavioral checklists, and direct observation are used These assessments rely on many of the same techniques, assumptions, and theories as do other psychological assessments. The distinction to assess independent functioning skills, physical development, language development, and academic competencies (Sattler, 1992). lies in the focus on brain function as the point of departure (Lezak, 2. Aphasia. Aphasia is the loss or impairment of language due to some type of brain injury. The purpose of this type of evaluation is to 1995). The goal is to be able to evaluate the full range of basic physical and mental abilities that are controlled directly by the brain. determine the language areas affected and to provide a starting point Reducing the symptomatology of the brain-impaired individual to its basic processes requires not only a general understanding of the for language retraining. Most aphasia tests describe what the patient can do, so that remaining functions (not defects) are tested. A typical functional aspects of behavior and cognition but also a specific aphasia evaluation may include assessments of (a) perception and understanding of how these functions relate to the brain and to brain recognition of language (auditory and visual); (b) performance of dysfunction in particular (Golden, Zillmer, & Spiers, 1992). motor functions pertaining to language (speaking and writing); (c) 8. Personality-psychopathology. These assessments are usually conducted when a person's behavior problems, emotional difficulties, ability to use language symbols in reading, handwriting, and mathematics; and (d) formulation and comprehension of prepositional lan- social interactions, or ability to function independently become so guage (oral and written; Agranowitz, McKeown, & Nielsen, 1964). significantly disruptive or disrupted that mental health intervention 3. Behavioral medicine or rehabilitation. Behavioral medicine is an appears warranted (Knoff, 1986). Personality assessment helps iden- "interdisciplinary field concerned with the integration of behavioral tify and characterize an individual's social-emotional status and atti- and biomedical science knowledge and techniques relevant to health tudes, behaviors, and reactions to specific and recent or general and and illness and the application of this knowledge and these techniques long-existing situations or environments. Personality assessments in- to prevention, diagnosis, treatment, and rehabilitation. Assessments clude formal or informal observation, interviews, and evaluation within this domain seek to appraise the medical patient's present status within the context of the past and within his or her larger social processes addressing an individual's behavior, social-emotional development or progress, or self-concept formation. framework or environment, including current physical and psychosocial stressors. Objective instruments are used to assess the patient's overt behaviors and evaluate self-reported feelings and biophysical Respondents indicated the approximate number of times per year they administer a "full battery" of tests in each of the eight processes (Schneiderman & Tapp, 1985). 4. Developmental. Developmental assessment can be characterized as a process for obtaining clinical information about a child in order to provide answers to development-related questions and to generate 1 The Board of Professional Affairs, the Board of Scientific Affairs, and the Committee on Psychological Tests and Assessment. 144 CAMARA, NATHAN, AND PUENTE designated areas of assessment, and individual tests used. For example, a participant using tests for intellectual assessment would indicate the number of times he or she had conducted such assessments in a year, as well as the average time spent for each of the Table 1 Hours Spent Individually Administering, Scoring, and Interpreting Psychological Tests During a Typical Week Clinical psychologists three assessment services: administration, scoring, and interpretation. In addition, the questionnaire asked participants to indicate which tests they had used from a list of 120 individual tests, to identify which of the eight practice areas the test-was used for, and to identify the average time spent for each of the three assessment services. Participants provided information about the mode of testing (computer or paper and pencil) and were also asked to write in the names of any additional tests they had used that were not on the list. Hours 0-4 5-9 10-14 15-20 More than 20 No response Total n Neuropsychologists % n 37 4 116 62 92 105 188 4 933 100 4 567 755 81 62 7 4 39 36 4 % 21 11 16 19 33 100 Procedure We first mailed the "APA Survey" questionnaires in late 1994, accompanied by a cover letter from the APA president at the time, excess of 20 hr per week on assessment activities, with another third of them spending 10—20 hr per week on assessment. The final Ronald E. Fox, explaining that the purpose of the study was to 31 % devote 10 hr per week or less to assessment. estimate the frequency of use of psychological and neuropsycho- We conducted the remaining analyses for only those 447 neuropsychologists and 179 clinical psychologists who reported engaging in assessment activities for 5 hr or more in a typical logical assessment and the approximate amount of time required for assessment services (administration, scoring, and interpretation). We sent two mailings. We received responses from 1,499 (56%) of the total of 2,700 individuals selected, 933 clinical psychologists (62% of the sample of 1,500), and 566 neuropsychologists (47% of the sample of 1,200). Of the 1,499 respondents, 754 of the clinicians and 119 of the neuropsychologists reported conducting assessment services for under 5 hr in a typical week; these questionnaires were set aside. We conducted analyses only on questionnaires from the participants who engaged in 5 hr or more of assessment-related services in a typical week. Therefore, 179 clinicians and 447 neuropsy- week. Table 2 shows that neuropsychologists most often conduct assessments for purposes of neuropsychological assessment (95%), intellectual-achievement assessment (79%), personality-psychopathology (79%), and neurobehavioral clinical assessment (51%). Even so, more than a quarter of all neuropsychologists report conducting assessments for each of the eight purposes listed in the survey. Table 2 reports the mean and median number of full test or assessment batteries conducted by neuropsychologists who practice in a given assessment area. The median is by far the better indicator of central tendency for assessment chologists qualified for the remaining analyses. Nine additional tests "written-in" by 5 or more respondents were added to the initial list of tests, resulting hi a final list of 129 tests used in the analyses. services provided by neuropsychologists. In seven of the eight practice areas, approximately 20% of neuropsychologists account for greater than 50% of assessments being conducted. In these areas, the mean number of assessments per respondent exceeds the We conducted all analyses separately for the clinical psychologist and neuropsychologist samples. The types and uses of assess- median by 50% or more. ments and the assessment services provided by these two groups of psychologists differ greatly, so responses from these groups are not Among neuropsychologists, there was substantial variation in the number of times they annually administer a full battery of assessments in each of the practice areas. For example, for neu- combined for any analysis. However, comparisons of the types of assessment services provided, time required for completing ser- ropsychological assessment batteries and intellectual-achievement batteries, some respondents had conducted only a few assessments vices, and frequency and types of assessments used by these groups are reported later. annually, whereas one neuropsychologist conducted more than 400 full assessment batteries in the past year. Most of the variation was due to 10% of neuropsychologists who reported conducting Frequency in Conducting Assessments well more than 200 neuropsychological assessment batteries, intellectual-achievement batteries, and neurobehavioral clinical Table 1 illustrates the number of hours clinical psychologists and neuropsychologists devote to assessment during a typical examination batteries annually. Clinical psychologists involved in assessment services for 5 or week. As shown in Table 1, more than 80% of clinical psychologists reported spending less than 5 hr during a typical week in more hr per week are most often involved in personality- administering, scoring, reporting, and interpreting psychological tests, whereas more than 80% of neuropsychologists spend an average of 5 or more hr per week in providing these assessment services. Of the clinical psychologists, approximately 4% of respondents engage in assessment for 10-14 hr, 15-20 hr, or more than 20 hr during a typical week, with 7% spending 5-9 hr providing assessment services. More than a third of neuropsychologists spend hi psychopathology testing (93%) and intellectual-achievement testing (88%), followed by neuropsychological assessment (47%), adaptive-functional behavioral assessment (40%), and developmental assessment (30%). There was substantial variation in the number of full psychological batteries they administered in some areas, mostly because of the number of assessments performed by the most productive 10% to 20% of clinicians in each area. Clinical psychologists differed most in how often they conducted neuropsychological assessments, intellectual-achievement assessments, 145 SPECIAL SECTION: PSYCHOLOGICAL TEST USAGE Table 2 Number and Percentage of Full Assessment Batteries Administered Annually Clinical psychological Practice area Adaptive-functional behavior assessment Aphasia Behavioral medicine-rehabilitation Developmental Intellectual-achievement Neurobehavioral clinical examination Neuropsychologic al Personality-psychopathology Total Neuropsychological «(%) Mdn M SO n(%) Mdn M SD 72(40) 30(17) 20.0 13.5 13,5 20.0 50.0 30.0 30.0 50.0 36.3 28.4 28.3 35.9 87.3 59.1 63.6 80.4 47.8 46.4 30.8 47.0 110.0 73.5 116.0 101.0 194 (43) 205 (46) 127 (28) 115(27) 354 (79) 228(51) 427 (95) 353 (79) 20.0 25.0 30.0 20.0 60.0 50.0 70.0 50.0 36.9 53.6 70.3 46.2 90.4 88.9 96.8 87.5 54.4 70.4 92.2 91.7 96.7 87.9 91.3 115.0 26(15) 53 (30) 158 (88) 30(17) 84 (47) 166(93) 447 179 and personality-psychopathology assessments. Figures 1 and 2 illustrate total activity within each assessment area. Assessments for intellectual-achievement, personality-psychopathology, and neuropsychological purposes account for the great majority of testing for both clinical psychologists and neuropsychologists. Earlier research reporting on reasons for referral suggested that most referrals were requests for a complete assessment, personality assessment, or intellectual assessment {Lubin et al., 1984). Requests in 1982 for a neuropsychological assessment, across the five settings, yielded a composite overall low of 3%. In the current study, however, clinical psychologists involved in assessment services were testing most often for personality-psychopathology (93%) and intellectual-achievement (88%), followed by neuropsychological assessment (47%). Moreover, neuropsychologists were most often involved in neuropsychological assessment (95%), followed by intellectual-achievement and personality-psychopathology assessment (both 79%). These data suggest that neuropsychological assessments have become increasingly popular within the last decade. Given the wide variation in the frequencies and patterns of assessment services provided by clinicians and neuropsychologists, it is extremely difficult to characterize the "typical amount of assessment activity" of these practitioners. The median provides a more accurate picture of the amount of assessment activity in these areas for most practitioners in both groups, yet there is a small Beh. Med/Rehabilitation 2% Aphasia Neurobehavioral 2% 4% Developmental 5% Intellectual/Achievement 34% Adaptive/Functional Neuropsychological 13% Personality/Psychopathology 32% Figure 1. Proportion of assessment services, by area, conducted by clinical psychologists. Beh. Med. behavioral medicine. 146 CAMARA, NATHAN, AND PUENTE Beh. Med/Rehabilitation 6% Intellectual/Achievement 20% Neurobehavioral 13% Developmental 3% Personality/Psychopathology 20% Adapt! w/Functional Neurops ychological 26% Figure 2. Proportion of assessment services, by area, conducted by neuropsychologists. fleh, Med. behavioral medicine. percentage (10%-20%) of practitioners who conduct twice as many assessments as the typical practitioner in this study. It is also important to remember that practitioners providing less than 5 hr of assessment services in a typical week, 80% of clinicians, and 20% of neuropsychologists already have been excluded from these analyses. Time Required to Administer, Score, and Interpret a Full Assessment Battery The amount of time required for completing a full psychological or neuropsychological assessment battery may vary widely, for a number of obvious reasons; (a) the number and types of tests used in an assessment; (b) the mode of administration (e.g., paper and pencil, computer based); (c) the intended use(s) of the assessment (e.g., a quick neurological screening vs. a full neuropsychological examination, a full psychoeducational evaluation vs. a reevaluation); (d) the characteristics, symptoms, and abilities of the client or patient being tested; (e) the setting; and (f) the level of reporting and interpretation required (e.g., written reports, oral briefings, court reports), to name just a few. However, though the time requirements for specific tests and assessments also were examined, it is also important to provide some estimates of both the number of full assessments completed, by area, and the time requirements for administration, scoring, and interpretation, if we are to understand the general demands of assessment services. It is rare that clinical psychologists or neuropsychologists use only one or two tests ia completing an assessment. They are more likely to use several tests in conducting brief reevaluations or screenings, or to use an extensive array of tests in completing a full assessment battery for diagnostic or evaluative purposes. Table 3 provides (a) the number of clinicians who administer, score, and interpret assessments in the eight areas; (b) the average amount of time (in min) that is required to provide these services for a full battery of assessments; and (c) the standard deviation of the time required. Table 4 provides an identical breakdown for the neuropsychologists. First, nearly all clinical psychologists administer, score, and interpret assessments when conducting a full psychological assessment battery. Some have suggested that an increasing proportion of clinicians may be only interpreting assessment results, while giving the duties of administration and scoring to another provider (someone with less training who might operate at a lower cost, or another vendor who provides computehzed administrative services). Our evidence suggests that most clinicians are directly responsible for administering, scoring, and interpreting assessments in each assessment area. Second, test administration required the greatest amount of time in each assessment area, followed closely by the time required for interpretation and the time required for scoring. We used the following definitions for these services in the instructions included in the questionnaire: Administration: time required in the preparation for testing (selecting tests, preparing testing materials and test site) and actual administration of a test or assessment. Do not include time required for the client to complete self-administered tests. SPECIAL SECTION: PSYCHOLOGICAL TEST USAOE 147 Table 3 Minutes Required by Clinical Psychologists to Administer, Score, and Interpret a Full Psychological Assessment Battery Administer Practice area Adaptive-functional behavior assessment Aphasia Behavioral medicine-rehabilitation Developmental Intellectual-achievement Neurobehavioral clinical examination Neuropsychological Personality—psychopathology Total n 59 30 24 52 156 30 84 162 607 Score Interpret M Mdn SD n M Mdn SD 72.8 77.3 151.0 99.8 115.0 85.1 208.0 104.0 97.4 60.0 37.5 120.0 72.5 100.0 67.5 180.0 90.0 82.8 113.0 145.0 98.1 68.3 60.5 144.0 76.4 70 30 24 53 156 25 83 163 602 34.2 36.2 45.7 35.1 31.3 36.2 59.4 49.9 42.0 20.0 17.5 30.0 30.0 30.0 20.0 45.0 45.0 42.5 65.8 58.2 41.3 32.6 57.8 60.5 37.8 n 71 30 24 53 156 28 85 166 613 M Mdn SD 47.4 63.2 107.0 69.6 65.0 67.5 99.1 87.8 75.9 30.0 20.0 60.0 30.0 45.0 40.0 60.0 60.0 58.1 147.0 159.0 113.0 78.6 77.4 111.0 93.6 Scoring: time required to score individual responses and derived the average clinical assessment did. Interpretation and repotting of scores (including subscale scores, standardization scores, normreferenced scores, or comparisons) and to collaborate with test administrator or psychometrician when necessary. When scoring services (computer-based scoring or machine scoring) are used, scoring results from assessments conducted for behavioral medicine and rehabilitation purposes, neuropsychological assessment, and personality or psychopathology assessments consumed substantially more time than assessments in other areas (an average of 93 min includes the time required to forward raw test data to the service, review and verify score reports, and any follow-up interactions required with the service. across these areas compared with an average of 62 min for the remaining five assessment areas). Most of the variation is due to the number and choice of tests selected by clinicians for inclusion Interpretation and reporting: time required to review raw test data (quantitative and qualitative) and scoring reports and time required to in a full psychological battery. For the neuropsychological sample (see Table 4), the time synthesize all relevant data (e.g., medical, historical), complete written and oral reports, and provide interpretation to client, family, or referral source. When services provide interpretative reports, interpretation time also includes the review, verification, and elaboration of required for administering full assessment batteries (55% of the total time spent providing assessment services) far exceeded the average time required for scoring (16%) and interpretation- the report. reporting results (28%). The time required for assessments conducted for neuropsychological purposes or behavioral medicine- Across assessment areas, administration accounted for 45% of rehabilitation purposes far exceeded the time required for other the overall time required in conducting assessment services. Scoring and interpretation-reporting accounted for 36% and 19%, uses of assessment. The times required by neuropsychologists for respectively. Most notably, there was substantial variation among providing assessment services were more uniform than the times in practitioners in the time required to administer and interpret full the clinical sample, perhaps indicating more uniformity in the psychological assessment batteries but less variation in scoring numbers and types of assessment instruments used. The times required for providing assessment services are sub- time. The time required for clinical psychologists to administer and stantially similar across areas for both the clinical and neuropsy- interpret-report assessment results differed across assessment ar- chological samples but vary widely according to the intended eas, whereas the time required for scoring assessments remained purpose of the assessment (Camara, Nathan, & Puente, 1998). The fairly consistent, across all areas of assessment. Neuropsycholog- time required to complete a full neuropsychological assessment ical assessment consumed the most time, on average requiring battery requires substantially more time than assessments in other 50% more time to administer (M = 208 min, Mdn =180 min) than areas. Assessments conducted for personality-psychopathology, Table 4 Minutes Required by Neuropsychologists to Administer, Score, and Interpret a Full Psychological Assessment Battery Practice area Adaptive-functional behavior assessment Aphasia Behavioral medicine-rehabilitation Developmental Intellectual-achievement Neurobehavioral clinical examination Neuropsychological Personality-psychopathology Total n 88 202 123 114 350 227 422 335 1,961 Interpret Score Administer M Mdn SD 73.6 60.7 110.0 113.0 122.0 80.0 304.0 103.0 140.83 60.0 45.0 90.0 90.0 120.0 60.0 300.0 90.0 87.6 52.1 77.0 80.8 76.3 67.1 136.0 97.6 n 87 203 123 111 348 200 420 48 1,940 M Mdn SD 32.2 23.6 35.3 35.8 33.4 25.8 78.5 46.0 41.74 20.0 15.0 30.0 30.0 30.0 20.0 60.0 30.0 74.2 19.0 24.9 27.7 24.8 22.5 51.6 38.7 n 88 202 124 114 350 223 425 350 1,976 M Mdn SD 48.0 39.1 58.2 59.4 61.3 46.8 135.0 74.9 71.66 30.0 30.0 45.0 30.0 40.0 30.0 120.0 60.0 82.4 40.0 44.6 55.7 73.9 38.5 115.0 78.2 148 CAMARA, NATHAN, AND PUENTE behavioral medicine-rehabilitation, intellectual-achievement, and clinicians using the Rorschach Psychodiagnostic Test ("inkblot developmental evaluation purposes required more time on average test") and 107 using the TAT, followed by the H-T-P Protective than assessments Technique (60 users), Human Figures Drawing Test (49 users), conducted for adaptive-functional behavior. aphasia, and neurobehavioral clinical purposes. Assessments con- Rotter Incomplete Sentences Blank (45 users), Sentence Comple- ducted in the latter three areas can often be characterized as briefer (ion Test (40 users), and the CAT (38 users). The Bender Visual screenings or examinations used to assess behavioral functioning Motor Gestalt Test was the third most frequently used test, with or specific diagnoses, requiring less time than the more exhaustive 112 users, but this use may be divided among neuropsychological diagnostic assessments used in the former applications. screening, projective assessment, and intellectual-achievement as- Frequency of Use of Individual Tests and Assessments sessment. Several intelligence and achievement tests were among the most frequently used assessments: the WAIS-R, with 151 Table 5 provides a rank-ordered list of the top 20 tests used by clinical psychologists or neuropsychologists who conduct assessment services for 5 or more hr in a typical week. Only 161 of the users; the W^C-II, with 135 users; ^ *« Wide Rmie Achieve(WRAT), with 86 users. Finally, a few inventories and screening tests were also often cited for use by the clinical sample. ment Test Aese were me 179 respondents completed this section of the survey. Clinical Among psychologists indicated they used an average of 13.4 (Mdn = 13.0) separate tests. The MMPI was the most frequently used test used user Wechsler Memory Scale—Revised (58 by 138 clinical psychologists (more than 86%). The Beck Depression Inventory (53 users), the Millon Clinical Multiaxial Inventory Behavior Scales (37 users). Respondents were asked to indicate the number of times they (53 users), and the Millon Adolescent Personality Inventory (38 use (i.e., administer, score, and interpret-report) each test annu- users) were the next most often cited personality tests. Projective assessments were also used by a majority of respondents, with 124 ally. This question attempts to determine the highest utilization rate among psychologists using the various tests. Among tests used s>- && Trailmaking Test A&B (52 users), the Conners' Parent and Teacher Rating Scales (37 users), and the Vineland Adaptive Table 5 Frequency and Rank Order of Tests Used by Clinical Psychologists and Neuropsychologists Clinical psychologists Neuropsychologists Test Rank n Rank Aphasia Screening Test0 Beck Depression Inventory Bender Visual Motor Gestalt Test Boston Naming Test California Verbal Learning Test Category Test Children's Apperception Test (CAT-A) Conners' Parent and Teacher Rating Scales FAS Word Fluency Test Finger Tapping Test" Grooved Pegboard Test" Halstead-Reitan Neuropsychological Test Battery Hand Dynamometer (Dynamic Hand Grip Strength Test) Hooper Visual Organization Test House-Tree-Person (H-T-P) Projective Technique Human Figures Drawing Test Millon Adolescent Clinical Inventory Millon Clinical Multiaxial Inventory Minnesota Multiphasic Personality Inventory (MMPI) I and II Peabody Picture Vocabulary Test—Revised Rey Complex Figure Test Rorschach Inkblot Test Rotter Incomplete Sentences Blank Sentence Completion Test Thematic Apperception Test (TAT) Trail Making Test A&B° Vineland Adaptive Behavior Scales Wechsler Adult Intelligence Scale—Revised (WAIS-R) Wechsler Intelligence Scale for Children—Revised (WlSC-R-ni) Wechsler Memory Scale—Revised Wide Range Achievement Test—Revised and III Wisconsin Card Sorting Test 23 10 18 37 27 53 112 13 18 20 38 37 17 29 22 17 11 25 8 14 9 60 39 5 6 15 7 20 19 31 41 56 24 1 28 12 18 51 5 42 36 31 16 44 23 44 59 8 13 16 10 2 20 25 4 14 15 6 12 18 1 3 9 7 33 12 27 12 8 60 49 38 53 138 34 25 124 45 40 107 52 37 151 135 58 86 19 n Total n 156 186 200 253 208 222 207 223 67 94 258 250 192 241 148 153 138 104 73 153 96 209 189 203 29 57 241 228 180 214 136 145 78 55 35 100 359 89 196 153 41 497 123 221 177 86 42 54 94 26 4 44 91 246 51 2 16 3 331 178 257 9 203 12 196 198 298 8$ 482 313 315 289 215 Note. Only tests ranked in the top 20 by either the clinical or neuropsychology sample are listed in the table. * A subtest of the Halstead-Reitan and the Reitan-Indiana Neuropsychological Batteries. b A subtest of the Halstead-Russell Neuropsychological Evaluation System. c A subtest of the HalsteadReitan Neuropsychological Battery, SPECIAL SECTION: PSYCHOLOGICAL TEST USAGE by a substantial portion of the clinical sample, the Bender Qestalt, the Human Figures Drawing Test, the Rotter Incomplete Sentences Blank, the MMPI, the Rorschach, and the H-T-P Projective Technique were used an average of over 65 times across users. Finally, regarding frequency of test use, the current study suggests very little change since the 1960s. The Rorschach, TAT, and H-T-P were among the top 10, indicating the sustained popularity of projectives. Also unchanged from previous research were the inclusions of the WAIS and WISC, Bender Gestalt, MMPI, and WRAT in the top 10. One change was the inclusion of the Millon Clinical Multiaxial Inventory in the top 10. For further details, Table 5 provides the top rank ordering of tests ranked in the top 20 by either clinical psychologists or neuropsychologists. Neuropsychologists use an average of 17.6 (Mdn = 15.0) different tests in their assessment practice, with 10% of respondents using over 30 different tests. Neuropsychologists were much more likely than clinical psychologists to write in additional tests that were not on the original list provided in the survey; over 100 respondents wrote in five or more tests. A total of 18 respondents did not complete this section. Table 5 also provides a rank ordering of the top 20 tests used by the remaining 430 neuropsychologists. Overall, a large number of neuropsychological batteries and individual neuropsychological assessments used for screening specific functions were used by a large percentage of these respondents: the Wechsler Memory Scale—Revised (257 users), Trail Making Test A&B (Halstead-Reitan; 246 users), FAS Word Fluency Test (also known as the Controlled Word Association Test, Spreen & Strauss, 1991; 237 users), Finger Tapping Test2 (228 users), Halstead—Reitan Neuropsychological Test Battery (214 users), Boston Naming Test (20& users), Category Test (209 users), Wisconsin Card Sorting Test (196 users), Rey Complex Figures Test (196 users), California Verbal Learning Test (189 users), Grooved Pegboard Test3 (180 users), Aphasia Screening Test4 (159 users), Hooper Visual Organization Test (145 users), and the Hand Dynamometer (Dynamic Grip Strength Test; 136 users). As with the clinical sample, the MMPI was the most frequently used test (359 users). Additional personality tests often used by this sample were the Beck Depression Inventory (200 users) and the Millon Clinical Multiaxial Inventory (100 users). The Rorschach Inkblot Test and the TAT were the most frequently used projective instruments, with 153 and 91 users, respectively. The WAIS-R, WRAT, and WISC-H were the most commonly used intelligence and aptitude tests (with 331, 203, and 178 users, respectively). The following several instruments had the highest utilization rates among all tests used by a substantial percentage (25% or more) of the sample: Trail Making A&B (Halstead-Reitan), WAIS-R, MMPI, Wechsler Memory Scale—Revised, Rey Complex Figures Test, FAS Word Fluency Test, and the WRAT. Each of these was used an average of 90 or more times annually by test users. It should be noted that test use varied markedly by assessment area. The frequency and rank order of tests used within each of the eight assessment areas show some substantial disparities from the overall list. For example, the Bender Gestalt, which ranked fifth among clinical psychologists, was the most frequently used assessment for intellectual and achievement assessments and neurobehavioral clinical examinations. Similarly, the Aphasia Screening Test and the Vineland Adaptive Behavior Scales were the tests 149 most often used by neuropsychologists when assessment was conducted for aphasia screening and adaptive-functional behavior. However, these tests ranked 17th and 44th, respectively, in total use by neuropsychologists (Camara et al., 1998). Unfortunately, when the data are sorted by assessment area, the number of responses in each area is so low that the reliability of the rankings within many assessment areas is also extremely low. Fewer than 50 respondents provided specific rankings and frequency-of-use data for assessments in five of the eight areas, and between zero and seven tests were used by three or more respondents for each assessment area. Therefore, results of test use frequencies and rank ordering lack sufficient reliability to indicate relative use within these five areas: adaptive-functional behavioral assessment, aphasia assessment, behavioral medicine and rehabilitation, developmental assessment, and neurobehavioral clinical examinations. Use of Computer-Based Testing Both clinical psychologists and neuropsychologists indicated low rates for utilization of computer-based testing. The most common application of computers in testing is computer-based scoring of tests and assessments. More than 10% of tests are scored using computer-based services (e.g., in-house computer scoring, machine scanning of responses, and electronic or digital transmission of scores) for both clinical and neuropsychological practitioners. However, the administration and interpretation of tests (e.g., generating interpretive reports) are conducted much less often by computer, as illustrated in Figure 3. Computer-based scoring services were most often used with testing conducted for purposes of personality-psychopathology assessment. Relatively few practitioners indicated using computer- based services for applications other man personality-psychopathological assessment, so comparisons of mean differences in time required for administration, scoring, and interpretation are primarily restricted to this area. Although computer-based services required slightly less time (mean differences ranged from approximately 1 to 4 min less time), in nearly all comparisons these differences were not significant. Time Required to Administer, Score, and Interpret-Report Individual Assessments The actual times required for particular psychological tests were also collected from both samples. Tables 6 and 7 provide the mean and median times required for administration, scoring, and interpretation-reporting for the 50 most frequently used tests. Generally, there was substantial consistency between clinical psychologists and neuropsychologists on self-reported time required to provide assessment services. Instruments such as the Halstead— Reitan Neuropsychological Test Battery and the Luria-Nebraska Neuropsychological Battery, used primarily for neuropsychologi- 2 A subtest that appears in both the Halstead-Reitan Neuropsychological Battery and the Reitan-Indiana Neuropsychological Battery. 3 A subtest of the Halstead-Russell Neuropsychological Evaluation Sys- tem. 4 A subtest that appears in both the Halstead-Reitan Nenropsychological Battery and the Reitan-Indiana Neuropsychological Battery. 150 CAMARA, NATHAN, AND PUENTE 12.0% -f 10.4% 0 Clinical Psychologists D Neuropsychologists 10.3% 10.0%- 8.0%- 6.0%3.9% 3.6% 3.1% 4.0%2.4% 2.0%- 1.J 0.0% Administration Figure 3. Scoring Interpretation Percentage of testing services conducted with computers. cal assessments, required the most time for administration, scoring, and interpretation (an average of 6.5 hr and over 4 hr, respectively). Next in consumption of time were intelligence tests such as the Kaufman Assessment Battery for Children, the Stanford-Binet Intelligence Scale, and the various Wechsler scales. The Rorschach Inkblot Test and the Woodcock-Johnson Psycho-Educational Battery—Revised required, on average, over 2 hr for completion. Conclusions and Implications for Professional Practice This study provides a broad survey of the assessment practices for clinical psychologists and neuropsychologists today. Nearly 80% of neuropsychologists reported providing assessment services for more than 4 hr weekly, with one third of them spending over 20 hr per week in this area. Assessment services account for a relatively minor aspect of practice for the over 80% of clinical psychologists who spend 4 hr or less each week conducting assessment services. However, assessment services are a substantial component of practice for the more than 12% of clinicians who spend 10 or more hr in this practice area. Patterns of assessment practice reveal that evaluations of personality-psychopathology and intellectual-achievement account for over two thirds of assessments conducted by clinical psychologists. Not only do most psychologists conduct assessments in these two areas, but the ratio of assessments per psychologist practicing in these areas (87.3 and 80.4 assessments per practitioner, respectively) far exceeds the ratios in the remaining six assessment areas. Neuropsychological assessments and adaptive-functional behavior assessments are the next most popular use of psychological testing, with approximately 50% of clinical psychologists practicing in these areas. Assessment practice by neuropsychologists is somewhat more varied as to area, with neuropsychological, personality-psychopathology, and intellectual-achievement evaluations each accounting for 20% or more of all assessments. Neurobehavioral clinical examinations were the next most frequent use of assessment, accounting for an additional 13% of all assessment practice in the field. More than half of all neuropsychologists practice in all of these four assessment areas. The ratio of testing among neuropsychologists practicing in assessment areas was consistent across these four areas, ranging from 87.5 to 96.8 assessments per practitioner. In addition, neuropsychologists report taking substantially more time to interpret-report results from assessments than do clinical psychologists, whereas time required for administration and scoring are more comparable. Results from this project have implications for the reimbursement of psychological assessment services by third parties and reimbursement-authorization of such services by managed care. This study empirically determined that the mean time required to administer, score, and interpret a full psychological or neuropsychological battery was more than 3.5 hr by experienced clinical psychologists and 4.25 hr for neuropsychologists, with additional time required for assessments conducted in some practice areas. Psychological assessment faces enormous obstacles in the current health care delivery system, ranging from outright refusal to reimburse assessment, difficulties in gaining preauthorization for testing, or requirements that practitioners use medication for differential diagnosis (Eisman et al., 1998). Current guidelines issued by most managed care organizations do not provide separate reimbursement of assessment services, either requiring the psychologist to reduce treatment time if assessment services are 151 SPECIAL SECTION: PSYCHOLOGICAL TEST USAGE Table 6 Minutes Required to Administer, Score, and Interpret Individual Tests: Clinical Psychologists No. of uses per year Test Aphasia Screening Test" Beck Depression Inventory Bender Visual Motor Gestalt Test Boston Naming Test California Verbal Learning Test Category Test Child Behavior Checklist Children's Apperception Test (CAT- A) Children's Depression Inventory Conners' Parent and Teacher Rating Scales Developmental Test of Visual-Motor Integration FAS Word Fluency Test Finger Tapping Test" Grooved Pegboard Test6 Halstead-Reitan Neuropsychological Battery Hand Dynamometer (Dynamic Hand Grip Strength Test) House-Tree-Person (H-T-P) Projective Technique Human Figures Drawing Test Kaufman Assessment Battery for Children Kinetic Drawing System for Family and School: A Handbook Luria-Nebraska Neuropsychological Battery Memory Assessment Scales Millon Adolescent Clinical Inventory Millon Clinical Multiaxial Inventory Minnesota Multiphasic Personality Inventory (MMPI) I and n Myers-Briggs Type Indicator Peabody Picture Vocabulary Test—Revised Personality Inventory for Children Rey Complex Figures Test Reynolds Adolescent Depression Scale Roberts Apperception Test for Children Rorschach Inkblot Test Rotter Incomplete Sentences Blank Symptom Checklist-90—Revised (SCL-90-R) Sentence Completion Test Shipley Institute of Living Scale Sixteen Personality Factor Questionnaire Stanford-Binet Intelligence Scale Strong Interest Inventory (4th Ed.) Stroop Neuropsychological Screening Test Test of Visual-Motor Integration (TVMI) Thematic Apperception Test (TAT) Trail Making Test A&B° Vineland Adaptive Behavior Scales Vineland Social Maturity Scale Wechsler Adult Intelligence Scale—Revised (WAIS-R) Wechsler Intelligence Scale for Children— Revised and m (WISC-R and m) Wechsler Memory Scale—Revised Wide Range Achievement Test—Revised Wisconsin Card Sorting Test Woodcock-Johnson Psycho-Educational Battery —Revised Min to score Min to administer No. of psychologists M SD Mdn M SD 27 53 112 13 18 20 29 38 14 37 30.3 48.2 79.1 41.5 39.8 35.7 54.8 23.4 45.7 57.4 43.0 49.0 95.1 52.8 36.9 36.7 43.8 39.0 80.2 75.6 20.0 25.0 40.0 20.0 27.5 17.5 36.0 13.5 20.0 30.0 24.8 11.1 15.7 18.8 30.6 43.0 13.1 36.2 11.6 11.2 11.9 10.0 20 17 22 12 27 32.6 69.1 58.5 71.8 38.4 34.5 61.5 48.0 51.0 44.8 24.0 50.0 50.0 55.0 20.0 19.2 15.1 12.5 24.2 27.5 15.0 13.0 7.0 8.5 9.6 4.1 237.8 135.5 10.0 10.0 240.0 8.3 6.2 4.7 12 56.4 62.4 50.0 6.3 3.1 60 49 19 65.5 73.8 20.4 70.6 81.0 43.8 40.0 40.0 10.0 17.3 14.7 70.3 7.7 23.9 25 14 12 38 53 66.4 15.7 29.0 27.2 41.8 63.7 21.0 19.2 35.9 49.2 50.0 14.0 126.8 53.8 14.8 19.4 66.8 93.4 16.1 28.2 23.7 35.9 29.8 30.9 42.8 40.6 28.4 14.7 29.2 25.8 66.1 103.9 70.8 122.6 36.0 10.0 20.0 40.0 25.0 20.0 32.5 35.0 67.1 89.6 54.0 57.8 80.8 147.8 19.6 16.5 20.6 24.9 40.0 30.0 40.0 20.0 10.0 138 10 34 10 25 11 25 124 45 12 8.0 35.0 15.0 24.5 5.0 Mdn M SD 10.0 14.0 10.0 14.1 12.5 12.3 5.0 9.6 7.4 15.0 10.0 20.0 15.0 10.0 10.0 34.6 20.3 23.3 29.7 10.4 14.7 69.8 16.9 16.1 29.0 10.0 12.3 3.2 2.4 7.6 7.8 4.9 9.0 7.5 6.1 2.0 10.0 5.0 5.0 5.0 57.3 36.5 60.0 81.9 74.7 60.0 5.0 4.9 2.7 5.0 5.3 2.6 5.0 15.0 15.0 60.0 11.0 12.3 27.1 10.6 13.8 24.5 10.0 10.0 20.0 15.0 14.1 8.5 9.6 37.1 30.2 15.0 10.0 30.0 15.0 120.0 50.0 10.0 10.0 8.1 62.1 19.9 17.3 21.6 34.3 17.5 16.0 18.8 11.3 23.6 30.0 15.0 15.0 15.0 13.3 53.9 21.3 32.4 29.3 10.0 33.8 17.8 17.3 10.0 52.5 20.0 30.0 25.0 29.8 14.0 22.9 37.0 12.6 10.5 10.0 10.0 20.0 5.5 7.2 8.8 2.5 20.0 17.5 10.0 20.0 10.0 10.0 20.0 37.5 10.0 36.1 46.0 15.4 20.5 24.2 17.3 33.8 50.6 17.8 32.9 47.7 10.9 13.4 60.0 18.1 22.8 45.7 12.1 30.0 27.5 15.0 15.0 10.0 10.0 30.0 30.0 15.0 10.0 10.0 10.0 15.0 20.0 18.5 17.6 18.7 17.9 21.5 19.9 11.4 29.4 25.7 16.9 10.1 15.0 15.0 15.0 20.0 30.0 20.0 10.0 10.0 30.0 10.0 30.0 10.0 19.9 16.8 39.8 45.3 16.6 7.9 11.8 11.9 13.5 17.9 18.5 8.0 15.6 8.1 6.3 16.5 17.4 20.6 12.2 11.3 8.8 16.7 13.1 20.0 19.2 9.6 12.2 21.9 18.0 12.4 21.5 14.9 12.3 21.6 44.8 12.1 7.3 7.2 7.5 7.4 8.6 8.7 11.2 21.1 5.0 7.3 12.5 14.1 5.0 13.8 16.2 16.5 12.1 6.5 15.6 8.3 7.2 15.9 31.7 12.2 8.8 5.0 5.0 9.9 8.9 6.5 11.2 8.6 10.0 10,0 10.0 10.0 17.5 15.0 20.0 27.5 10.0 10.0 5.0 5.0 5.0 75.0 22.9 15.7 20.0 35.3 40.1 30.0 25.7 25.3 20.2 12.6 80.0 45.0 30.0 30.0 24.7 19.6 15.7 20.4 18.5 11.2 11.4 16.0 20.0 15.0 15.0 15.0 39.2 23.5 17.7 19.3 44.1 14.5 14.5 14.1 30.0 20.0 15.0 15.0 68.6 72.5 33.4 22.9 30.0 36.9 28.2 30.0 5.0 52.0 22.3 47.4 46.6 27.4 10.8 39.9 22.4 86.6 54.9 36.1 9.6 50.0 15.0 20.0 26.0 19.0 10.0 151 53.1 60.4 25.0 78.6 22.1 135 58 86 19 56.9 69.0 45.4 40.7 68.4 105.5 49.9 68.3 30.0 20.0 30.0 30.0 82.4 48.7 31.9 31.3 55.3 30.0 90.5 Note. Only the top 50 most popular tests were included in this table. • A subtest of the Halstead-Reitan and the Reitan-Indiana Neuropsychological Batteries. Evaluation System. " A subtest of the Halstead-Reitan Neuropsychological Battery. 8.2 30.0 10.0 5.1 76.0 20.0 10.0 35.0 45.0 15.0 11.4 Mdn 25.7 15.4 7.1 30 20.0 10.0 15.0 15.0 30.0 45.0 10.0 30.0 10.0 10.0 SD 19.9 10.3 12.4 14.2 15 22 15 19 25 107 52 37 13 11 Mdn 22.5 21.8 18.4 25.0 31.3 30.0 13.9 12.9 34.2 13.8 26.6 13.2 15.2 15.3 15.9 36.1 68.8 25.0 15.3 15.5 38.4 14.3 51.1 24.6 40 M Min to interpret 16.4 10.5 13.0 36.7 31.1 23.5 10.3 12.0 10.5 16.3 22.6 137.7 10.5 11.6 19.5 19.7 10.7 12.5 15.0 10.0 30.0 60.0 15.0 15.0 15.0 30.0 12.5 45.0 20.0 6.1 4.3 19.2 8.7 b 9.5 10.2 7.6 8.3 11.0 349.3 6.9 4.7 5.0 10.0 10.0 15.0 5.0 8.4 A subtest of the Halstead-Russell Neuropsychological 152 CAMARA, NATHAN, AND PUENTE Table 7 Minutes Required to Administer, Score, and Interpret Individual Tests: Neuropsychologists No. of uses per Min to administer year Test Aphasia Screening Test" Beck Depression Inventory Bender Visual Motor Gestalt Test Benton Judgment of Line Orientation Test Benton Revised Visual Retention Test Boston Diagnostic Aphasia Examination Boston Naming Test California Verbal Learning Test Category Test Child Behavior Checklist Conners' Parent and Teacher Rating Scales Dementia Rating Scale Developmental Test of Visual-Motor Integration Finger Tapping Test Grooved Pegboard Testb Halstead-Reitan Neuropsychological Battery Hand Dynamometer (Dynamic Hand Grip Strength Test) Hooper Visual Organization Test House-Tree-Person (H-T-P) Projective Technique Human Figures Drawing Test Luria-Nebraska Neuropsychological Battery Memory Assessment Scales MUlon Clinical Multiaxial Inventory Minnesota Multiphasic Personality Inventory (MMPI) I and 11 Neuropsychological Questionnaire (Adult Form) Paced Auditory Serial Addition Test Peabody Picture Vocabulary Test— Revised Peabody Individual Achievement Test—Revised Rey Complex Figures Test Rhythm Test" Rorschach Inkblot Test Rotter Incomplete Sentences Blank Symptom Checklist-90— Revised (SCL-90-R) Sensory Perceptual Examination" Sentence Completion Test Shipley Institute of Living Scale Speech Sounds Perception Testc Stroop Neuropsychological Screening Test Tactile Finger Localization Test Tactile Performance Test Thematic Apperception Test (TAT) Test of Visual-Motor Integration Trail Making Test A&B° Vineland Adaptive Behavior Scales Wechsler Adult Intelligence Scale— Revised (WAIS-R) Wechsler Memory Scale — Revised Wechsler Intelligence Scale for Children— Revised and HI (WISC-R and HI) Wide Range Achievement Test— Revised Wisconsin Card Sorting Test Woodcock-Johnson Psycho-Educational Battery— Revised Min to score No. of psychologists M SD Mdn M SD 159 200 96 90 80 66 209 189 203 52 57 120 69.0 69.2 68.6 50.3 54.1 34.1 81.3 71.9 59.8 83.3 59.4 46.4 88.2 84.9 85.5 64.0 80.0 41.5 85.8 65.6 69.4 114.0 89.5 51.8 35.0 42.5 40.0 30.0 30.0 20.0 50.0 50.0 35.0 50.0 40.0 25.0 21.5 10.4 16.5 15.7 18.2 46.8 16.8 32.4 38.1 10.0 19.8 8.3 8.0 5.0 5.0 38.3 18.4 35.0 13.8 43 66.8 81.8 74.5 76.1 37.5 50.0 50.0 40.0 14.7 12.6 5.9 8.8 5.0 250.0 136.5 15.0 10.0 10.0 240.0 12.2 18.7 214 62.6 78.3 77.8 67.1 136 145 66.0 63.5 67.1 70.1 40.0 50.0 7.3 13.7 6.7 7.6 10.0 78 55 64 50 100 55.9 60.3 24.0 43.6 52.3 53.7 57.6 41.7 51.1 62.0 35.0 42.5 12.0 22.5 40.0 19.7 14.8 160.2 51.5 22.3 22.5 359 94.3 121.8 50.0 49 88.6 60.2 47.6 97.3 68.5 66.5 43.6 92.7 70.7 54.4 76.9 56.7 88.2 75.1 66.3 66.6 60.9 70.5 61.5 40.2 43.9 99.2 28.9 228 180 9.2 14.3 6.4 15.3 36.7 15.3 15.7 18.9 16.3 Mdn 15.0 10.0 15.0 15.0 15.0 40.0 15.0 30.0 35.0 5.0 M SO 9.5 5.9 11.4 8.8 4.5 7.9 Min to interpret M SD Mdn 5.0 5.0 13.2 11.9 10.0 9.5 9.2 5.0 10.0 14.6 30.8 10.0 5.6 9.1 Mdn 5.9 4.1 5.0 7.9 12.6 19.8 8.7 15.5 10.0 15.0 10.9 22.7 7.6 6.1 5.0 9.3 7.7 8.0 17.2 10.2 13.8 11.0 15.0 10.0 12.5 10.0 15.0 17.0 13.1 13.1 12.1 19.1 15.8 15.2 15.0 10.0 10.0 10.0 15.0 9.9 6.5 8.2 5.2 7.3 22.7 8.9 10.3 9.4 5.0 10.0 15.0 8.3 3.1 4.6 10.0 10.3 11.1 10.0 5.0 4.8 5.0 5.0 7.0 6.2 5.4 4.0 5.0 5.0 54.4 37.6 60.0 95.3 94.4 60.0 4.3 6.5 2.9 6.2 5.0 5.0 5.8 8.8 4.5 7.1 5.0 5.0 12.2 10.9 16.9 12.0 62.1 20.8 27.2 11.0 56.7 12.3 23.7 15.0 10.0 45.0 20.0 20.0 9.8 8.1 94.9 22.1 21.8 15.0 15.0 157.5 50.0 15.0 46.5 19.4 19.2 42.2 10.0 21.5 10.0 10.0 30.0 20.0 15.0 38.5 44.5 15.0 24.7 54.9 15.0 30.4 21.4 30.0 50.0 40.0 20.0 27.2 22.7 22.1 27.9 11.9 20.0 20.0 20.0 10.4 12.7 6.4 10.3 6.1 10.0 10.0 10.0 23.3 10.3 11.6 23.3 9.7 15.0 10.0 10.0 54.2 91.2 77.7 71.3 103.9 53.3 97.3 119.5 97.2 79.4 57.0 93.3 73.6 45.2 52.1 92.6 33.7 20.0 70.0 40.0 35.0 37.5 50.0 50.0 37.5 27.5 35.0 50.0 30.0 35.0 25.0 20.0 70.0 15.0 42.7 17.7 12.1 47.4 12.2 12.7 23.1 13.4 18.3 18.7 11.7 24.0 10.0 40.0 15.0 10.0 45.0 16.3 14.4 11.3 15.0 15.0 16.7 12.2 12.1 9.8 6.8 34.6 37.3 16.9 11.5 45.0 94.5 93.1 92.0 85.9 70.0 60.0 203 196 59.7 89.1 77.5 63.6 88.4 69.3 58 51.2 47.9 110 89 50 196 79 153 41 43 87 54 57 75 118 48 76 91 73 246 51 331 257 178 8.7 9.9 7.3 8.5 9.4 12.5 10.0 5.7 3.8 5.0 7.6 5.3 5.0 47.7 35.9 40.0 41.9 5.0 8.0 15.6 6.3 8.4 4.0 9.4 7.9 9.9 5.6 9.3 6.2 4.4 8.2 3.3 5.1 30.0 15.0 10.0 10.0 15.0 10.0 29.9 19.7 16.3 15.1 24.7 10.0 20.0 10.0 20.0 15.0 10.0 10.0 30.0 30.0 15.0 10.0 40.0 45.0 12.9 12.4 10.5 17.5 11.6 20.5 11.6 83.4 53.3 35.1 23.0 30.0 51.0 50.0 81.6 30.8 30.3 30.0 80.2 Note. Only the top 50 most popular tests were included in this table. a A subtest of the Halstead-Reitan and the Reitan-Indiana Neuropsychological Batteries. Evaluation System. c A subtest of the Halstead-Reitan Neuropsychological Battery. 6.1 8.6 8.4 20.6 12.8 12.1 30.2 12.8 10.0 22.5 10.2 b 6.8 6.6 11.4 5.0 15.0 5.0 10.0 10.0 5.0 5.0 7.9 10.6 7.3 7.6 7.6 12.2 6.8 6.4 8.7 5.1 8.0 5.0 10.0 5.0 6.3 22.3 10.3 10.0 15.0 10.0 12.6 27.8 10.5 24.5 16.4 6.1 7.8 5.0 8.0 6.5 5.0 5.0 19.8 11.9 15.0 33.0 43.5 15.0 90.0 60.0 24.6 21.4 29.2 11.0 20.0 20.0 29.3 21.6 23.9 14.5 25.0 20.0 30.1 19.3 15.0 75.0 30.0 30.0 21.8 13.1 16.8 11.3 29.9 14.6 14.3 24.6 11.2 11.7 20.0 10.0 15.0 9.5 25.0 10.0 10.0 58.6 60.0 22.5 11.2 20.0 31.4 30.2 20.0 7.4 5.0 10.0 25.0 A subtest of the Halstead-Russell Neuropsychological 153 SPECIAL SECTION: PSYCHOLOGICAL TEST USAGE provided, to "eat these costs," or to simply pass on costs to the client. Other managed care organizations stipulate that behavioral interviews are the only necessary diagnostic assessment because the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) makes no reference to other psychological or neuropsychological assessments (Eisman et al., 1998). When assessment services are reimbursed limits of less than 2 hr are most typical. Such guidelines from managed care are clearly inconsistent with the empirical research from this study demonstrating that comprehensive assessment services require approximately 4 or more hr of time by a trained professional. Because practitioners can be reimbursed for the actual time required to conduct comprehensive assessment services under existing managed care guidelines there is a danger that assessment services will be dramatically restrained or eliminated from intake and treatment planning. This study showed that less than 12% of clinical psychologists spend 5 or more hr on assessment services. In the future, fewer practitioners may engage in such services for less time than currently reported. Clinical psychologists may also reduce the number of assessments used and possibly reduce the time expended in interpretation and reporting of results if reimbursement is not provided. The economic barriers created under managed care may continue to reduce the quality of assessment services and the extent that such services are provided in some areas (e.g., intake, diagnosis, treatment planning). There are several general strategies that the profession of psychology and individual practitioners can consider to more directly affect these economic and health care policy issues. First, practitioners must be creative in incorporating assessment as a central component within their interventions and treatments so that is not considered an option or supplemental service that must be justified and added onto reimbursement for treatment. Assessment services are often medical necessities and not an option. Practitioners should view and portray assessments as an integral component of effective treatment planning and mental health interventions. Practitioners should explore ways of incorporating and describing such services as a component of treatment plans. Second, evidence of the efficacy of psychological and neuropsychological assessment in the treatment of patients and the disposition of treatment is needed. Barlow (1994) and Broskowski (1995) observed that in the present managed care era, mental health services that do not have strong empirical support are not likely to be reimbursed. Practitioners must be able to justify the benefits of comprehensive assessment services in terms of treatment focus, treatment duration, and cost-to-benefit ratio. Has a comprehensive assessment aided in the initial diagnosis and effectiveness of treatment planning in ways that both focus the subsequent psychological interventions and reduce the overall length of treatment for a client? Has assessment been effectively used to inform treatment and make the necessary adjustments throughout the intervention, or to provide summative evidence of the efficacy of treatment? Practitioners can advance such arguments with managed care by submitting case records and other data that support the effectiveness of assessment practices. Evidence demonstrating that assessment services in diagnosis or treatment have been effective in reducing the duration of treatment or reducing the recidivism of mental health problems will be viewed as compelling by managed care because of its economic relevance. An APA task force has recently summarized evidence of the benefits of psychological assessment in assessing current functioning, in confirming or disconfirming clinical impressions, in differential diagnosis, in identifying appropriate treatments, in monitoring treatment, in risk management, and as a therapeutic intervention (Meyer et al., 1998). More constructive interactions with mental health care systems are needed to reduce the misunderstandings and biases against assessment and to help define criteria for medical necessity of assessment services. Just as practitioners must advance such evidence when arguing for the reimbursement of assessment services, the profession must advocate more forcefully with the use of such evidence of the effectiveness and utility of assessment services in addressing mental health problems. Finally, assessment services must be viewed as a more integrated component of professional practice rather than an independent service. Practice guidelines, and discussion of such guidelines, issued by the Agency for Health Care Policy and Research (AHCPR), the American Psychiatric Association, and a Task Force of APA's Division 12 generally put little if any emphasis on the use of assessment services in treatment planning and evaluation. Whereas such guidelines have been highly controversial within the mental health profession, the absence of assessment services has not been a primary concern of mental health professionals at this time. Such guidelines strive to reflect best practices, advocate empirically supported treatment, and improve the standard of care (Nathan, 1998). Objective and comprehensive assessments are essential in aiding the profession and professionals in guiding treatment and determining the efficacy of treatment. Preand postassessment strategies can help practitioners objectively demonstrate the effect of treatment. The lack of attention to assessment services in existing practice guidelines is troubling and must be addressed if the influence of such guidelines increase. Eisman et al. (1998) proposed a number of additional strategies to reduce obstacles for reimbursement and precertification of assessment services. These recommendations include enhanced legislation and advocacy, public education to make customers and patients aware of the benefits of psychological assessment, expanded and continuing training of psychologists to ensure they are skilled in assessment and aware of current practice standards and ethical requirements, and criteria to help decision makers recognize under what circumstances assessment is most helpful and appropriate. Critics have argued that assessment is time consuming, expensive, and not useful (Griffith, 1997), and it is clear that until practitioners and the profession develop more effective empirically based methods of advocacy the current misconceptions about the utility of psychological and neuropsychological assessments will remain. References Agranowhz, A., McKeown, M. R., & Nielsen, J. M. (1964). Aphasia handbook for adults and children. Springfield, IL: Charles C Thomas. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Ball. J. D., Archer, R. P., & Imhof, E. A. (1994). Time requirements of psychological testing: A survey of practitioners. Journal of Personality Assessment, 63(2), 239-249. 154 CAMARA, NATHAN, AND PUENTE Barlow, D. H. (1994), Psychological intervention in the era of managed competition. Clinical Psychology: Science and Practice, I, 109-122. Broskowski, A. T. (1995). The evolution of health care: Implications for the training and careers of psychologists. Professional Psychology: Research and Practice, 26, 156-162. Brown, W. R., & McGuire, J. M. (1976). Current psychological assessment practices. Professional Psychology, 7, 475-484. Camara, W. J., Nathan, J., & Puente, A. (1998). Psychological test usage in professional psychology: Report to the APA Practice and Science Directorates. Washington, DC: American Psychological Association. Eisman, E. J., Dies, R. R., Finn, S. E., Eyde, L. D., Kay, G. G., Kubiszyn, T. W., Meyer, G. L, ftMoreland, K. L. (1998). Problems and limitations in the use of psychological assessment in contemporary healthcare delivery: Report of Board of Professional Affairs Psychological Assessment Working Group, Pan 11. Washington, DC: American Psychological Association. Golden, C. J., Zillmer, E., & Spiers, M. (1992). Neuropsychological assessment and intervention. Springfield, IL: Charles C Thomas. Griffith, L. (1997). Surviving no-frills mental health care: The future of psychological assessment. Journal of Practical Psychiatry and Behavioral Health, 3, 255-258. Johnson, J. H., & Goldman, J. (1990). Developmental assessment in clinical child psychology: A handbook. New York: Pergamon Press. Knoff, H. M. (1986). Identifying and classifying children and adolescents referred for personality assessment: Theories, systems, and issues. In H. M. Knoff (Ed.), The assessment of child and adolescent personality (pp. 3-31). New York: Guilford Press. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press. Louttit, C. M., & Browne, C. G. (1947). Psychometric instruments in psychological clinics. Journal of Consulting Psychology, I I , 49—54. Liibin, B., Larsen, R. M., & Matarazzo, J. D. (1984). Patterns of psycho- logical test usage in the United States: 1935-1982. American Psychologist, 39, 451-454. Lubin, B., Wallis, R. R., & Paine, C. (1971). Patterns of psychological test usage in the United States: 1935-1969. Professional Psychology, 2, 70-74. Meyer, G. I., Finn, S. E., Eyde, L. D., Kay, G. G., Kubiszyn, T., Moreland, K., Eisman, E., & Dies, R. (1998). Benefits and costs of psychological assessment in healthcare delivery: Report of Board of Professional Affairs Psychological Assessment Working Group, Part II. Washington, DC: American Psychological Association. Nathan, P. E. (1998). Practice guidelines: Not yet ideal. American Psychologist, 53, 290-299. O'Roark, A. M., & Exner, J. E. (Eds.). (1989). History and directory: Society for personality assessment fiftieth anniversary. Hillsdale, NJ: Erlbaum. Saltier, J. M. (1992). Assessment of children: Revised and updated (3rd ed.). San Diego, CA: Jerome M. Sattler. Schneiderman, N., & Tapp, J. T. (1985). Behavioral medicine: The biopsychosocial approach. New York: Erlbaum. Spreen, E., & Strauss, C. (1991). Compendium of neuropsychological tests. New York: Oxford University Press, Sundberg, N. D. (1961). The practice of psychological testing in clinical services in the United States. American Psychologist, 16, 79-83. Watkins, C. E., Jr. (1991). What have surveys taught us about the teaching and practice of psychological assessment? Journal of Personality Assessment, 56(3), 426-437. ORDER FORM Start my 2000 subscription to Professional Psychology: Research and Practice! ISSN: 0735-7028 Received April 6, 1999 Revision received December 9, 1999 Accepted December 10, 1999 Send me a Free Sample Issue Q Q Check Enclosed (make payable to APA) Chargemy: Q VISA Q MasterCard Q American Express Cardholder Name Card No. Exp. date $46.00, APA Member/Affiliate $92.00, Individual Nonmember $198.00, Institution In DC add 5.75% sales tax TOTAL AMOUNT ENCLOSED $ Signature (Required for Charge) Subscription orders must be prepaid. (Subscriptions are on a calendar basis only.) Allow 4-6 weeks for delivery of the first issue. Call for international subscription rates. SEND THIS ORDER FORM TO: American Psychological Association Subscriptions 750 First Street, ME Washington, DC 20002-4242 AMERICAN PSYCHOLOGICAL ASSOCIATION Or call (800) 374-2721, fax (202) 336-5568. TDD/TTY (202)336-6123. Email: subscriptions@apa.org Credit Card Billing Address City Daytime Phone SHIP TO: Name City. APA Customer # PLEASE DO NOT REMOVE - A PHOTOCOPY MAY BE USED State State Zip. .Zip GADOO