JOURNAL OF CLINICAL AND EXPERIMENTAL NEUROPSYCHOLOGY
2012, 34 (4), 435–442
Authors’ reply to “Response to Mayers and Redick:
‘Clinical utility of ImPACT assessment for
postconcussion return-to-play counseling:
Psychometric issues’”
Lester B. Mayers1 and Thomas S. Redick2
1
Downloaded by [Indiana Universities] at 07:40 17 May 2012
2
Division of Sports Medicine, Pace University, Pleasantville, NY, USA
Division of Science, Indiana University-Purdue University Columbus, Columbus, IN, USA
Keywords: Rejoinder; Clinical utility; Immediate Postconcussion Assessment and Cognitive Testing;
Postconcussion; Psychometric issues.
We appreciate the opportunity afforded by the
Editor of the Journal to reply to the comments
by Schatz, Kontos, and Elbin (2012, pp. 428–434
of this issue, hereafter abbreviated as SKE) about
our analysis of the concussion assessment program Immediate Postconcussion Assessment and
Cognitive Testing 2.0 (ImPACTTM ). We consider
them to be significant contributors to the study
of concussion and have carefully considered their
critique.
We believe that the basic issue separating our
position and that of ImPACT’s advocates is illustrated by the following quotations:
From the ImPACT Applications, Inc. website:
“The best way to prevent difficulties with concussion is proper management of the injury using the
ImPACT concussion management program. [. . .]
ImPACT’s approach to managing concussion has
been found to be reliable, valid and extremely sensitive in determining when an athlete has recovered
sufficiently from a concussion [. . .] This information
can be used to help determine recovery from injury
and safe return to participation and overall clinical management issues” (Impact Applications, Inc.,
2012a, pp. 2, 4, emphasis added).
From our paper:
The proper time interval between the occurrence of
a brain injury (concussion) and the time of safe
return-to-play is a medical decision and should ideally
be evidence based. Tests utilized to achieve this purpose should be reliable (attaining acceptable test–retest
repeatability), sensitive (reliably detecting an affected
patient), specific (not falsely identifying normal subjects
as being affected), and have acceptable validity (sensitive
to impairment when true impairment is present).
(Mayers & Redick, 2012, p. 236, emphasis added).
GENERAL CONSIDERATIONS
1. SKE refer to our “seemingly random quotations at the start” (p. 428) of our manuscript.
These quotations were cited in order to illustrate the general acceptance of ImPACT among
researchers and practitioners involved with concussion. We believe that we then proceeded
to “deconstruct” this degree of acceptance.
Whether we accomplished a valid analysis of
the pertinent literature to prove our viewpoint
or “selectively review limited observations”
(p. 428) is the object of debate here.
Address correspondence to Lester Mayers, MD, Director of Sports Medicine, Pace University, 861 Bedford Rd – Goldstein Fitness
Center, Pleasantville, NY 10570, USA (E-mail: lmayers@pace.edu).
© 2012 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business
http://www.psypress.com/jcen
http://dx.doi.org/10.1080/13803395.2012.667790
Downloaded by [Indiana Universities] at 07:40 17 May 2012
436
MAYERS AND REDICK
2. SKE dismiss a previous critical neuropsychological test review (Randolph, McCrea, &
Barr, 2005) stating that “As computer-based
concussion assessment and management tests
became commercially available in 2000, there
was an extremely narrow time frame to conceptualize and complete prospective research
as required by the inclusion criteria put forth
by Randolph et al.” (p. 428). However, despite
this “narrow time frame,” the developers and
purveyors of ImPACT were able to publish no
fewer than 33 articles promoting the efficacy of
their product in neuropsychological and sports
medical journals during that interval (Impact
Applications, Inc., 2012c). We do not believe
that the premise for our article was “built on a
weak scientific foundation” (p. 429) as the publications by Randolph and others (Randolph,
2011; Randolph & Kirkwood, 2009) were peer
reviewed, appeared in respected journals, and
are widely accepted. Moreover, the discussion
of Randolph and colleagues’ work has no bearing on the information that we provide in our
review.
3. Regarding the general criticism by SKE that our
publication is not a “review paper” but more
of an “opinion” they are correct. We did not
present our analysis as a comprehensive review
and certainly could not perform a meta-analysis
because of the small number of studies available
for each component of review in our original
paper. For example, at the time our manuscript
was written, there were only three publications
(Broglio, Ferrara, Macciocchi, Baumgartner, &
Elliott, 2007; Iverson, Lovell, & Collins, 2003;
Schatz, 2010) that reported ImPACT test–retest
reliability in healthy subjects. Across these studies, Iverson, Lovell, and Collins (2003) reported
Pearson’s r only, Broglio et al. (2007) reported
intraclass coefficients (ICCs) only, and Schatz
(2010) reported both Pearson’s r and ICCs.
4. We should have provided more detailed explanation of our “framework for identifying
published articles.” We reviewed published
studies of ImPACT’s psychometric properties
from its website (Impact Applications, Inc.,
2012c), searches of library databases for pertinent peer-reviewed articles, and references from
articles publishing ImPACT data. Although we
did not provide this information in our original
article, researchers are able to read the studies
that we referenced in order to “validate the
conclusions” in our review. ImPACT is a computerized neuropsychological assessment tool
that is promoted by its purveyors as “reliable
and valid” for determining the time of recovery
after a brain injury. Our analysis has led us to
conclude that it is not useful for this purpose.
SPECIFIC CONSIDERATIONS
Test–retest reliability
1. We note that despite the repeated criticism by
SKE of the “selective” and “biased” nature
of our review, they indicated only two articles relevant to our discussion of ImPACT.
Elbin, Schatz, and Covassin (2011) published
a study in November 2011 after our article
was accepted, providing additional test–retest
reliability. Lau, Collins, and Lovell (2011), published in June 2011 while our article was under
review, examined the utility of ImPACT to distinguish between short and protracted recovery
after concussion, a topic we did not address in
our analysis.
2. SKE’s criticism of Broglio et al. (2007) as
“methodologically flawed” (p. 429) is unsubstantiated. For example, although the number
of subjects these researchers excluded because
of invalid baseline ImPACT data (29 out of 118,
25%, not 34% as stated by SKE) is higher than
that typically observed, the remaining 73 college students performed extremely well. The
baseline ImPACT scores of the subjects studied
by Broglio et al. (2007) was numerically better or the same as those of the college students
reported by Schatz (2010). In addition, their
suggestion of potential proactive and reactive
interference effects from performing multiple
similar cognitive tests within a 1-hour session
is unfounded. If this claim was true, than virtually every large-scale factor-analytic study on
the relationship among working memory, shortterm memory, and fluid intelligence that has
been conducted would also be “methodologically flawed.” In the absence of direct evidence
for why the study by Broglio et al. (2007) should
be ignored, other than the fact that the ICCs
they measured are lower than those found by
Schatz (2010) and Elbin et al. (2011), we feel
that it is necessary to include Broglio et al.
(2007) in our discussion of ImPACT’s test–
retest reliability (Table 1).
3. We agree that multiple measures of test–retest
reliability are desirable for ImPACT, and some
metrics such as reliable change indices (RCIs)
and regression-based methods (RBM) are more
relevant when attempting to make return-toplay (RTP) decisions about individual athletes.
However, SKE fail to acknowledge the point
AUTHORS’ REPLY TO SCHATZ, KONTOS, & ELBIN (2012)
437
TABLE 1
Test–retest reliability of ImPACT using Pearson r , intraclass correlation coefficients, and reliable change index confidence
intervals
Study
N
Iverson, Lovell, and Collins (2003)
56
Broglio et al. (2007)
Schatz (2010)
Downloaded by [Indiana Universities] at 07:40 17 May 2012
Elbin et al. (2011)
VeM
ViM
PS
RT
Type
1–13 days
.70
11.21
.67
17.37
.86
6.38
.79
0.08
Pearson
90% RCI CIs
73
∼ 45 days
.24
.23
.35
.32
.40
.38
.41
.39
95
∼ 2 years
.30
.46
16.08
.49
.65
20.19
.60
.74
8.71
.52
.68
0.10
Pearson
ICCs
95% RCI CIs
0.5–2.35 years
.45
.62
15.61
.55
.70
19.86
.74
.85
7.84
.62
.76
0.11
Pearson
ICCs
95% RCI CIs
369
Interval
Pearson
ICCs
Note. Broglio et al. (2007) Pearson r correlations were provided via personal communication (S. P. Broglio, personal communication,
February 13, 2012). Schatz (2010) and Elbin et al. (2011) RCI CIs (reliable change index confidence intervals) were obtained using
Jacobson and Truax (1991) formula. ICCs = Intraclass correlation coefficients; VeM = Verbal memory; ViM = Visual memory; PS =
Processing speed; RT = Reaction time.
in our article (see also Randolph, 2011) that
Pearson’s r is included in the formula for RCIs
(cf. Schatz, 2010, Table 4). Thus, even if SKE
prefer ICCs, an index more commonly used
for interrater reliability (Tinsley & Weiss, 1975),
low Pearson’s r values of test–retest reliability
will necessarily inflate the RCI values, meaning a recently concussed athlete will need to
show substantially larger true score change
from Time 1 to Time 2 in order to be detected as
impaired on the ImPACT composites. We also
note that in Schatz (2010) and Elbin et al.
(2011), only two of the eight ICCs meet what
Schatz (2010) provided as an indicator of good
reliability (ICCs > .75).
In addition, the discrepancy between the ICC
and Pearson r correlations observed in Schatz
(2010) and Elbin et al. (2011) may reflect a
problem with the test, not with Pearson r.
As SKE state, the ICC can produce an index of
agreement between Time 1 and Time 2. If the
true score relating to Verbal Memory really
is constant in an individual from Time 1 to
Time 2, then the higher ICC value would indicate a high degree of agreement. However, if
the score on the test shows range restriction,
then one may artificially obtain higher ICC
values without measuring the consistency of
the true ability being measured. For example,
according to the technical manual available on
ImPACT’s website (Impact Applications, Inc.,
2012e), Verbal Memory appears to be a negatively skewed score, with a ceiling effect shown
in the normative data for five different samples
(Normative Tables 5, 8, 11, 14, and 17). In the
most extreme case, in Normative Table 17, the
score corresponding to the 86th through the
100th percentile rank is the maximum score of
100.0. The 50th percentile score is 91.67, meaning that the upper half of the sample all scored
within 9 points of each other. In contrast,
the 1st percentile score is 55.14, meaning that
the lower half of the sample all scored within
46 points of each other, a much wider range of
scores.
As these are normative data, the implication is
that in a random sample, approximately 14%
of female college students would, at baseline,
achieve a perfect Verbal Memory score, and
50% would score within 9 points of the maximum score. If these same students served as
controls in a reliability study and were tested
again at a later interval, it is likely they would
score near or at the maximum possible score.
This does not indicate that the composite is reliable, even though the scores are similar at Time
1 and Time 2. It means instead that the Verbal
Memory composite was constructed in such a
way that the scores were almost guaranteed to
be identical for a sizeable portion of the sample.
4. SKE proceed to assert that in our consideration of ImPACT’s visual memory domain,
we stated that “significant practice effects have
been observed for processing speed and visual
memory” and then comment “but it is unclear
what specific data this assertion is based on.”
However, we included the direct references
within the same sentence: “significant practice
effects in healthy controls have been observed
for the processing speed (Iverson, Lovell, et al.,
2003) and visual memory (Schatz, 2010) composites” (Mayers & Redick, 2012, p. 237).
Downloaded by [Indiana Universities] at 07:40 17 May 2012
438
MAYERS AND REDICK
5. SKE stated that “one might expect variation
in human performance over a 2-year period.”
(p. 430). We agree with this statement, especially
when considering the still-maturing adolescent
period of life. However, the variation described
by SKE would seem problematic for using
ImPACT with the baseline approach, given
the guidelines on the ImPACT website: “We
suggest that athletes are tested every two years
from 6th grade to senior year of high school.
In college, athletes should only be tested once.
At the professional level, each athlete should
be tested once” (Impact Applications, Inc.,
2012b). Cognitive development is still occurring
and does not peak until the mid-20s and then
declines throughout the 30s for abilities such
as Processing Speed (Salthouse, 2009). The
current protocol of baseline testing would seem
to provide an inaccurate comparison to be used
against postconcussion performance multiple
years later.
6. We were not clear enough in our statement
about significant practice effects observed in
ImPACT. We were specifically referring to studies of nonconcussed subjects that found a significant improvement in an ImPACT composite
score from Time 1 to Time 2. In our article, we
stated that significant practice effects had been
observed for Processing Speed (Iverson, Lovell,
& Collins, 2003) and Visual Memory (Schatz,
2010). In addition, Elbin et al. (2011) observed
significant practice effects for Visual Memory,
Processing Speed, and Reaction Time (all ps =
.001). In contrast to SKE’s statement, Miller,
Adamson, Pink, and Sweet (2007) is another
example of a study in which practice effects
on ImPACT were observed. Specifically, Miller
et al. tested football players at pre-, mid-, and
postseason. The effects of practice were seen for
Verbal Memory (p = .06), Visual Memory (p =
.04), Processing Speed (p = .05), and Reaction
Time (p = .04). As we stated in the original
article, practice effects complicate the goal
of baseline testing, because the comparison
with the baseline is affected by the increase in
performance due to practice and the decrease
in performance assumed to be caused by the
concussion.
7. In a later comment, SKE state that “Further,
they selectively utilized only one of five statistical techniques presented in the Schatz (2009
[sic]) paper.” (p. 430). In fact, Schatz (2010)
reported four techniques: Pearson’s r, ICCs,
RCIs, and RBM. In our paper, we discussed
Pearson’s r and ICCs saying “Schatz (2010)
reported test–retest intraclass correlations of
.45 to .74 and Pearson r test–retest correlations
of .30 to .60” (Mayers & Redick, 2012, p. 237).
Sensitivity and specificity
Schatz, Pardini, Lovell, Collins, and Podell (2006)
administered ImPACT to 72 concussed highschool athletes and 66 nonconcussed controls. They
reported sensitivity to be 82% (nearly 1 in 5 concussed patients incorrectly classified as “normal”)
and specificity 89% (1 in 10 nonconcussed athletes incorrectly classified as concussed). Similarly,
Van Kampen, Lovell, Pardini, Collins, and Fu
(2006) administered ImPACT to 122 concussed
high-school and college athletes and 70 nonconcussed controls. Although an abnormal result in
a single ImPACT test domain had a sensitivity of 83%, specificity was only 70% (almost
1 in 3 nonconcussed athletes incorrectly classified as concussed). It seems reasonable to conclude that a test to guide “overall clinical management issues” for concussed athletes and their
“safe return to participation” should demonstrate more robust sensitivity–specificity characteristics than those shown by these studies, especially when used to guide medical decisions concerning brain-injured patients and their safety
after RTP.
Validity issues
1. SKE state that “Finally, it is unclear why the
divergent validity reported by Iverson and colleagues (Iverson, Gaetz, Lovell, & Collins, 2005;
Iverson et al., 2003) should be ‘interpreted with
caution’, or why the authors did not discuss the
specific findings from these papers.” (p. 431).
We believe that these two citations are in error.
We stated “taken with caution” and within
the same sentence pointed out that the athletes were still suffering from concussive effects
at the time the two cited Iverson studies were
conducted. We believe that we described the
pertinent findings from the following papers:
Iverson, Franzen, Lovell, and Collins (2003);
Iverson, Lovell, and Collins (2005). They have
addressed the divergent validity issue, but only
within the context of athletes still suffering
from the acute effects of concussion. We conclude that “given this limitation, the following
results should be taken with caution” (Mayers
& Redick, 2012, p. 238). We then spent an entire
paragraph addressing the results from these two
Iverson et al. studies.
Downloaded by [Indiana Universities] at 07:40 17 May 2012
AUTHORS’ REPLY TO SCHATZ, KONTOS, & ELBIN (2012)
2. Later in their critique, SKE say “Mayers and
Redick comment on methodological differences
[. . .] which they then use to make broad generalizations regarding ImPACT. [. . .] However
these authors fail to consider that Iverson et al.
included a 7–21-day retest interval in their
analyses.” (p. 432).
We really don’t know what SKE are trying to
say here. We thoroughly discussed this study
in our paper, and our “broad generalizations”
concerning ImPACT are not simply based upon
“methodological differences” but an entire consideration of the literature that we reviewed.
3. SKE assert that we “seemed to lack an understanding of the validity types.” (p. 431). Our discussion of divergent/discriminant validity was
not at the level of the entire test but, as we
stated, “among the four composites assessed”
(p. 238) in ImPACT. They then state that
our “discussion of the lack of divergent validity of ImPACT was primarily a discussion of
ImPACT’s internal structure and interrelationships, which is reflective of construct validity.” (p. 431). We are sure they know that
divergent/discriminant validity is part of construct validity, not separate from it. Later, SKE
state that “orthogonality refers to low interrelationships rather than ‘uncorrelated’.” (p. 431).
This is incorrect as orthogonal variables are
variables that are not correlated. In factor analysis, orthogonal factors have a factor correlation of zero.
More germane to the discussion of ImPACT,
and as we discussed in our article, the overlap
among the four ImPACT composites is a serious problem for clinical RTP decisions that are
based on the number of ImPACT scores that
have yet to return to baseline. Van Kampen
et al. (2006) explicitly indicated that their work
provided preliminary support for the use of the
number of abnormal ImPACT scores as part of
the diagnosis, “with a higher number of abnormal composite scores suggesting a more severe
concussion” (p. 1634). Again, as we stated in
the original article, if the ImPACT composites
are correlated, even moderately, than attempting to make decisions based on the number of
abnormal composites is misguided.
RTP studies
The time required for complete healing of concussive brain injury is currently unknown. For several
decades, athletes were allowed to RTP post concussion when they felt able to participate. Utilizing this
439
management paradigm, it was reported that 15% of
National Football League players returned to play
“immediately” after a diagnosed concussion, 32%
returned during the same game, while more than
90% returned to compete within 7 days (Pellman,
Viano, Casson, Arfken, & Feuer, 2005). This
model apparently reflected the athletes’ tolerance
for unpleasant symptoms and the pressures to
RTP, both self-imposed and externally imposed by
coaches, peers, and the culture of football. During
this period, it also became customary for highschool and college athletes to RTP within 7 days.
Not surprisingly, the 7-day interval coincided with
the next scheduled football game. Currently, many
coaches, athletes, and administrators still expect
that a concussed athlete can and should RTP in
one week.
Although this practice lacks a scientific basis,
accumulated experience has shown that, for many
concussed athletes, symptoms resolve by 7 days
post injury, and when they RTP they are able to
perform their sport adequately. Since these athletes are able to “function” on the field one week
following injury, it follows that functional performance tests (i.e., balance, exercise stress, and
neuropsychological) probably normalize within a
similar interval, and generally they do (McCrea
et al., 2003). This is not to say that administering these tests, especially the neuropsychological
ones, cannot inform us about the clinical consequences of concussive injury but only to emphasize
that a normal result on any performance test (the
most commonly utilized one being ImPACT) does
not equate with complete healing of a brain injury.
In addition, recalling that acutely concussed athletes commonly exhibit attention–concentration–
memory deficits and photophobia (sensitivity to
light), administering computerized tests of intellectual functions shortly after injury to detect a
decrease from “baseline results” violates consensus
treatment guidelines that emphasize complete physical and cognitive rest (McCrory et al., 2009) and
may in fact delay recovery.
DISCUSSION
Unfortunately, no simple, affordable, and accessible tests to answer the critical question concerning the interval required for healing after
a concussive brain injury are yet available.
However, complex and expensive tests obtainable
mainly in research medical center settings have
been developed and utilized to measure sportrelated brain injury and to estimate its duration.
These techniques include electronic recordings
Downloaded by [Indiana Universities] at 07:40 17 May 2012
440
MAYERS AND REDICK
of evoked brain wave activity (De Beaumont,
Brisson, Lassonde, & Jolicoeur, 2007; Gosselin,
Theriault, Leclerc, Montplaisir, & Lassonde, 2006),
multitask effects on balance (Halterman et al.,
2006; Parker, Osternig, van Donkelaar, & Chou,
2006; van Donkelaar, Osternig, & Chou, 2006), balance testing with destabilized posture (Slobounov,
Slobounov, Sebastianelli, Cao, & Newell, 2007),
transcranial magnetic spectroscopic measurements
of brain metabolites (Henry et al., 2011; Vagnozzi
et al., 2008), positron emission tomographic measurements of energy metabolism (Bergsneider et al.,
2001), diffusion tensor imaging of injured brain
regions (Cubon, Putakian, Boyer, & Dettwiler,
2011), and a recent functional magnetic resonance imaging (fMRI) study (Talavage et al., 2012).
Without a lengthy consideration of the complexities of these procedures, their results, summarized in
Table 2, indicate that healing of brain damage is not
complete within 4 weeks following concussive injury.
Although we and SKE have herein debated interpretations of technical issues related to neuropsychological testing in general and to ImPACT particularly, the major consideration has not been
addressed. In scientific inquiry, it is necessary
to postulate axioms (fundamental principles) and
then to demonstrate how the findings follow logically from these postulates. The ImPACT test’s
axioms may be expressed as: uninjured state =
baseline score; brain-injured state = significantly
reduced score; healed brain injury = return to
baseline score. As cited above, ImPACT and other
functional tests do not measure recovery from
a brain injury, only resolution of some of the
manifestations of the injury. Therefore, they cannot
accomplish ImPACT’s stated claim that “Our
approach to managing concussion has been found
to be a reliable and valid method for determining when an athlete has recovered sufficiently from
a concussion in order to return to athletic competition” since no one advocates the exposure of
an incompletely recovered brain-injured patient to
repeated head injury.
The divide between the RTP time interval of
one week resulting from typical application of
“usual and customary” consensus guidelines that
are largely based on symptoms resolution and functional tests (McCrory et al., 2009) and the evidence
from the various measurements of brain healing
cited above that injury persists for 4 or more weeks
is disturbing. Both clinical and experimental evidence have shown that the risk of experiencing
a second concussion and the consequences of a
second episode are greatest within the two weeks
following a first episode (Guskiewicz et al., 2003).
These consequences may include rapid uncontrolled brain swelling, resulting in death (McCrory,
Davis, & Makdissi, 2012). Although occurring very
rarely, the result is catastrophic.
SKE state that we “essentially offer no alternative
solution” (p. 430), presumably referring to ImPACT
testing, to determine time of RTP. We believe that
functional tests do not provide the “solution” for
the reasons cited above and that the RTP decision is a strictly clinical one to be decided by a
competent, qualified, and experienced practitioner,
taking into account the athlete’s age, gender (Dick,
2009), previous head injuries, severity and duration
of symptoms and signs, and associated learning disability (Collins et al., 1999). We note that these
TABLE 2
Summary of studies indicating cerebral dysfunction for longer than 2 weeks in concussed athletes and in nonathletes with
traumatic brain injury
Measurement
Abnormal response duration
Patient group
Reference
EEG ERP response
5 weeks
30 months
CA, S and A
CA, multiconcussions
Gosselin et al. (2006)
DeBeaumont et al. (2007)
TMS
Multitask-challenged gait stability
9 months
28–30 days
CA, multiconcussions
CA
DeBeaumont et al. (2007)
van Donkelaar et al. (2006)
Halterman et al. (2006)
Parker et al. (2006)
Destabilized posture balance
testing
14 days
CA, first concussion
Slobounov et al. (2007)
28 days
CA, multiconcussions
Slobounov et al. (2007)
PET scan glucose uptake
2–4 weeks
Nonathletes with TBI
Bergsneider et al. (2001)
DTI MRI
fMRI
30 days
1–3 months
CA
CA, S and A
Cubon et al. (2011)
Talavage et al. (2012)
Note. CA = concussed athletes; S = symptomatic; A = asymptomatic; EEG = electroencephalography; ERP = event-related potential;
TMS = transcranial magnetic stimulation; PET = positron emission tomography; DTI = diffusion tensor imaging; MRI = magnetic
resonance imaging; fMRI = functional magnetic resonance imaging; TBI = traumatic brain injury.
Downloaded by [Indiana Universities] at 07:40 17 May 2012
AUTHORS’ REPLY TO SCHATZ, KONTOS, & ELBIN (2012)
significant variables and confounders cannot be factored into the results obtained with any of the
functional tests.
We reiterate our conclusion that “for evaluating and advising concussed athletes when to return
to play, ImPACT test results should not be the
determining factor. Clearly, more research and consideration by neuropsychologists will be necessary
to weigh the validity of this conclusion” (Mayers
& Redick, 2012, p. 240). Supporting our position
is a more recent posting from the ImPACT website
stating that “ImPACT assists doctors in making
RTP decisions and should never be used as a standalone tool” (Impact Applications, Inc., 2012d).
Since ImPACT test results do not relate to time of
healing of concussive brain injuries, we fail to see
how they assist doctors in making RTP decisions as
claimed on their website.
Our suggested “solution” to the current RTP
issue is that, in the absence of available technology
to precisely determine the time of postconcussion
healing of brain damage in individual athletes, all of
the methods cited above for determining persistence
of brain injury should be utilized to study different injured cohorts (by age, gender, sport, injury
history, etc.) to determine mean and standard deviation intervals for healing (resolution of metabolic,
physiological, and anatomic abnormalities) of postconcussive brain injury. These results could then
be used to provide evidence-based RTP advice to
the vastly larger number of the athletes that we
serve who are without access to these studies. Until
this information becomes available, we advocate for
adoption of a more conservative approach to the
RTP issue allowing for longer recovery intervals
closer to 4–6 weeks than 1 week (Mayers, 2008).
In the future, with new information, it is possible
that this interval will need to be extended.
Original manuscript received 14 February 2012
Revised manuscript accepted 14 February 2012
REFERENCES
Bergsneider, M., Hovda, D. A., McArthur, D. L.,
Etchepare, M., Huang, S. C., Sehati, N., et al. (2001).
Metabolic recovery following human traumatic brain
injury based on FDG-PET: Time course and relationship to neurological disability. Journal of Head
Trauma Rehabilitation, 16(2), 135–148.
Broglio, S. P., Ferrara, M. S., Macciocchi, S. N.,
Baumgartner, T. A., & Elliott, R. (2007). Test–retest
reliability of computerized concussion assessment
programs. Journal of Athletic Training, 42(4), 509–514.
Collins, M. W., Grindel, S. H., Lovell, M. R., Dede,
D. E., Moser, D. J., Phalin, B. R., et al. (1999).
Relationship between concussion and neuropsychological performance in college football players.
441
Journal of the American Medical Association, 282(10),
964–970.
Cubon, V. A., Putakian, M., Boyer, C., & Dettwiler, A.
(2011). A diffusion tensor imaging study on the white
matter skeleton in individuals with sports-related concussion. Journal of Neurotrauma, 28(2), 189–201.
De Beaumont, L., Brisson, B., Lassonde, M., &
Jolicoeur, P. (2007). Long-term electrophysiological
changes in athletes with a history of multiple concussions. Brain Injury, 21(6), 631–644.
Dick, R. W. (2009). Is there a gender difference in
concussion incidence and outcomes? British Journal of
Sports Medicine, 43(Suppl. 1), i46–i50.
Elbin, R., Schatz, P., & Covassin, T. (2011). One-year
test-retest reliability of the online version of ImPACT
in high school athletes. The American Journal of Sports
Medicine, 39, 2319–2324.
Gosselin, N., Theriault, M., Leclerc, S., Montplaisir, J., &
Lassonde, M. (2006). Neurophysiological anomalies
in symptomatic and asymptomatic concussed athletes.
Neurosurgery, 58(6), 1151–1161.
Guskiewicz, K. M., McCrea, M., Marshall, S. W.,
Cantu, R. C., Randolph, C., Barr, W., et al. (2003).
Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA
Concussion Study. Journal of the American Medical
Association, 290(19), 2549–2555.
Halterman, C. I., Langan, J., Drew, A., Rodriguez,
E., Osternig, L. R., Chou, L. S., et al. (2006).
Tracking the recovery of visuospatial attention
deficits in mild traumatic brain injury. Brain, 129(3),
747–753.
Henry, L. C., Tremblay, S., Leclerc, S., Khiat, A.,
Boulanger, Y., Ellemberg, D., et al. (2011). Metabolic
changes in concussed American football players during the acute and chronic post-injury phases. BMC
Neurology, 11, 105–114.
Impact Applications, Inc. (2012a). Executive summary.
Retrieved from http://apps.uslacrosse.org/safety/pdf/
ImpactExecSum.pdf (accessed March 8, 2012).
Impact Applications, Inc. (2012b). FAQ. Retrieved from
http://www.impacttest.com/faq (accessed March 8,
2012).
Impact Applications, Inc. (2012c). ImPACT reliability
and validity. Retrieved from http://www.impacttest.
com/publications/page/reliability (accessed March 8,
2012).
Impact Applications, Inc. (2012d). Overview and features of the InPACT Test. Retrieved from http://
www.impacttest.com/about/background
(accessed
March 8, 2012).
Impact Applications, Inc. (2012e). ImPACT Test interpretation. Retrieved from http://www.impacttest.com/
about/test_interpretation (accessed March 8, 2012).
Iverson, G. L., Franzen, M. D., Lovell, M. R., & Collins,
M. W. (2003, October). Construct validity of ImPACT
in athletes with concussions. Paper presented at the
23rd Annual Conference of the National Academy of
Neuropsychology, Dallas, TX.
Iverson, G., Lovell, M., & Collins, C. L. (2003).
Interpreting change on ImPACT following sport
concussion. The Clinical Neuropsychologist, 17(4),
460–467.
Iverson, G. L., Lovell, M. R., & Collins, M. W.
(2005). Validity of ImPACT for measuring processing speed following sports-related concussion. Journal
of Clinical and Experimental Neuropsychology, 27(6),
683–689.
Downloaded by [Indiana Universities] at 07:40 17 May 2012
442
MAYERS AND REDICK
Jacobson, N. S., & Truax, P. (1991). Clinical significance:
A statistical approach to defining meaningful change
in psychotherapy research. Journal of Consulting and
Clinical Psychology, 9, 12–19.
Lau, B. C., Collins, M. W., & Lovell, M. R. (2011).
Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sport-related concussion. American Journal of Sports Medicine, 39,
1209–1216.
Mayers, L. (2008). Return-to-play criteria after athletic concussion: A need for revision. Archives of
Neurology, 65(9), 1158–1161.
Mayers, L. B., & Redick, T. S. (2012). Clinical utility
of ImPACT assessment for postconcussion returnto-play counseling: Psychometric issues. Journal of
Clinical and Experimental Neuropsychology, 34(3),
235–242.
McCrea, M., Guskiewicz, K. M., Marshall, S. W., Barr,
W., Randolph, C., Cantu, R. C., et al. (2003). Acute
effects and recovery time following concussion in
collegiate football players: The NCAA Concussion
Study. Journal of the American Medical Association,
290(19), 2556–2563.
McCrory, P., Davis, G., & Makdissi, M. (2012). Second
impact syndrome or cerebral swelling after sporting
head injury. Current Sports Medicine Reports, 11(1),
21–23.
McCrory, P., Meeuwisse, W., Johnston, K., Dvorak, J.,
Aubry, M., Molloy, M., et al. (2009). Consensus
Statement on Concussion in Sport: The 3rd
International Conference on Concussion in Sport
held in Zurich, November 2008. British Journal of
Sports Medicine, 43(Suppl. 1), i76–i90.
Miller, J. R., Adamson, G. J., Pink, M. M., & Sweet, J.
C. (2007). Comparison of preseason, midseason and
postseason neurocognitive scores in uninjured collegiate football players. American Journal of Sports
Medicine, 35(8), 1284–1288.
Parker, T. M., Osternig, L. R., van Donkelaar, P., &
Chou, L. S. (2006). Gait stability following concussion. Medicine & Science in Sports & Exercise, 38(6),
1032–1040.
Pellman, E. J., Viano, D. C., Casson, I. R., Arfken, C., &
Feuer, H. (2005). Concussion in professional football:
Players returning to the same game. Neurosurgery,
56(1), 79–92.
Randolph, C. (2011). Baseline neuropsychological testing in managing sport-related concussion: Does it
modify risk? Current Sports Medicine Reports, 10(1),
21–26.
Randolph, C., & Kirkwood, M. W. (2009). What
are the real risks of sport-related concussion, and
are they modifiable? Journal of the International
Neuropsychological Society, 15(4), 512–520.
Randolph, C., McCrea, M., & Barr, W. B. (2005). Is neuropsychological testing useful in the management of
sport-related concussion? Journal of Athletic Training,
40(3), 139–152.
Salthouse, T. A. (2009). When does age-related cognitive decline begin? Neurobiology of Aging, 30,
507–514.
Schatz, P. S. (2010). Long-term test–retest reliability
of baseline cognitive assessments using ImPACT.
American Journal of Sports Medicine, 38(1), 47–53.
Schatz, P., Kontos, A., & Elbin, R. (2012). Response to
Mayers and Redick: “Clinical utility of ImPACT
assessment for postconcussion return-to-play
counseling: Psychometric issues”. Journal of Clinical
and Experimental Neuropsychology, 34(4), 428–434.
Schatz, P., Pardini, J. E., Lovell, M. R., Collins, M.
W., & Podell, K. (2006). Sensitivity and specificity of
the ImPACT Test Battery for concussion in athletes.
Archives of Clinical Neuropsychology, 21(1), 91–99.
Slobounov, S., Slobounov, E., Sebastianelli, W., Cao, C.,
& Newell, K. (2007). Differential rate of recovery in
athletes after first and second concussion episodes.
Neurosurgery, 61(2), 338–344; discussion 344.
Talavage, T. M., Nauman, E. A., Breedlove, E. L.,
Yoruk, U., Dye, A. E., Morigaki, K., et al. (2012).
Functionally-detected cognitive impairment in high
school football players without clinically diagnosed
concussion. Journal of Neurotrauma. Advance online
publication. doi: 10.1089/neu.2010.1512
Tinsley, H. E., & Weiss, D. J. (1975). Interrater reliability and agreement of subjective judgements. Journal of
Counseling Psychology, 22(4), 358–376.
Vagnozzi, R., Signoretti, S., Tavazzi, B., Floris, R.,
Ludovici, A., Marziali, S., et al. (2008). Temporal window of metabolic brain vulnerability to concussion:
A pilot 1H-magnetic resonance spectroscopic study
in concussed athletes—Part III. Neurosurgery, 62(6),
1286–1295; discussion 1295–1296.
van Donkelaar, P., Osternig, L., & Chou, L. S. (2006).
Attentional and biomechanical deficits interact after
mild traumatic brain injury. Exercise and Sport
Science Reviews, 34(2), 77–82.
Van Kampen, D. A., Lovell, M. R., Pardini, J. E.,
Collins, M. W., & Fu, F. H. (2006). The “value added”
of neurocognitive testing after sports-related concussion. American Journal of Sports Medicine, 34(10),
1630–1635.