The Stroop Test on the iPad
Neurocognitive Testing Made Easier
VANDERBILT UNIVERSITY
April 26, 2011
Sarah E. Waring and Emily S. Whitaker, Department of Biomedical Engineering
Advisor: James L. Blair, DO: Assistant Professor, Clinical Anesthesiology, Department of
Anesthesiology, Vanderbilt University Medical Center
Abstract
Recently, concerns have risen over the safety of some anesthetic agents in pediatric and geriatric
patients due to recent evidence that indicates neurotoxic effects are leading to postoperative cognitive
dysfunction (POCD) of the central nervous system (CNS). This is most commonly seen in elderly
patients, who may end up with their ability to perform everyday tasks impaired. Although the risk of
anesthesia to almost every other organ is assessed pre-operatively, the brain is neglected. This is due, at
least partially, to our lack of understanding of exactly how anesthesia affects the brain. Currently,
neurocognitive tests (NCT) are performed post-surgery to check for POCD; this assessment is
subjective since there is not usually a pre-operative measurement to compare to, only the patient’s own
idea of their pre-surgery function. In order to better understand the functional and structural changes
caused by anesthesia, Dr. Blair has proposed a study that will compare pre- and post-operative results
for both MR imaging and NCT. The study’s aim is to determine if changes in CNS structure are seen by
MR imaging and how NCT results might correlate to these structural changes. Our project consisted of
creating an iPad program that can be used in this study to easily administer the Stroop Test, (a NCT) in
a user friendly, touchscreen format.
Introduction
Cognitive dysfunction is defined as the impairment of the mental processes required for everyday living
– perception, memory, and information processing (Hanning; Price, Garvan and Monk). When this type
of cognitive decline occurs following a surgery (Newman, Stygall and Hirani; de Lau, van Meurs and
Uitterlinden; Gao, Taha and Gauvin; Price, Garvan and Monk), we call it postoperative cognitive
dysfunction (POCD). The etiology of POCD is unknown. Manifestations of POCD can include memory
lapses, lack of problem solving abilities, difficulty retaining information, and the sudden inability to
perform tasks that were previously easy. Elderly patients are at the highest risk of developing POCD
(Moller, Cluitmans and Rasmussen; Perouansky and Hemmings). For 2010, it was estimated that over
one million elderly patients would develop POCD that would have long-term, debilitating effects on daily
living that result in a drastic (almost doubled) increase in their risk of death within one year post-op
(Monk, Weldon and Garvan; Moller, Cluitmans and Rasmussen; Monk, Saini and Weldon; Lindholm,
Traff and Granath).
Neurocognitive tests (NCT) can be administered to patients to assess their current cognitive function.
The NCT used for this study focus on checking memory and executive function (Moller, Cluitmans and
Rasmussen). Currently, various neurocognitive tests (NCT) for POCD are administered one week and
three months post-surgery, but not before surgery. To evaluate the amount of cognitive decline that
actually occurs, preoperative baseline assessments are needed (Moller, Cluitmans and Rasmussen;
Rasmussen, Larsen and Houx).
Since the etiology of POCD is unknown, our advisor has proposed a study that will look at functional
and anatomic neuroimaging (fMRI and MRI), as well as neurocognitive tests to evaluate changes in
neurocognitive function associated with POCD. All tests will be performed pre- and post-operatively,
and the results will be analyzed to determine if there is a correlation between NCT and MR imaging
changes. It is hypothesized that the neurocognitive tests will reflect physiological and functional changes
in the CNS). Hence, the development of a pre-operative battery of neurocognitive tests to screen for
patients at a high risk of POCD would be of great use. We want to develop a computerized program
that can be easily used to administer these neurocognitive tests to elderly patients via a touch screen
format.
Our focus in developing our iPad program was on the Stroop Word and Color Test, which is a measure
of selective attention and cognitive flexibility that was originally developed by Stroop. Traditionally, this
test has been administered on paper, by a trained proctor, but recently this test has been adapted for
electronic use (Stroop; Strauss and Spreen; Golden). The original test consisted of four parts, but ours
consists of just two parts. In our Part 1, the subject is shown color words printed in colored ink that
matches the word and must choose the color of the ink. In Part 2, the subject still chooses the color of
the ink, but the ink color no longer matches the word (e.g. the word “blue” printed in green ink should
be answered “green”). Of major interest is the subject’s behavior when presented with color words
printed in nonmatching colored inks. Stroop reported that normal people can read color words printed
in colored ink as fast as fast as when the words are printed in black ink. However, the time to complete
the task increases significantly when the subject is asked to name the color of the ink rather than the
word. This decrease in color-naming speed is called the “color-word interference effect.” (Stroop;
Strauss and Spreen; Golden).
Methodology
The ultimate goal of this project was to create a working program to be used on an iPad for
administration. Programming for this device allowed for an easily-administrable test due to its unique
qualities; the iPad is both portable and lightweight, has a long battery life and exhibits instant start-up,
and utilizes a touch screen, which is easy for patients to navigate. Because the program is specific to this
device, to formulate the test we used Apple’s iOS application software – Xcode and SDK. The program
was written in Objective-C language; therefore, our first step was to learn how to code in that specific
language. Initial program development arose from the idea of a simple “quiz” game, which we modified
for our purposes. To form our specific test, the Stroop test, we used current paper- and computerbased tests as resources. Questions were chosen from a written Stroop test example and answer
choices were selected using a random number generator. All questions are presented on a simple, white
background to avoid skewing the user’s perception of the words displayed. Each question prompt was
coded to contain a color word written in a specific ink color, with four answer choices exhibited on the
touch screen. Other features that were added to the program include the ability to enter a patient ID
number and store and export scores. Our scoring method was based on both response time and
accuracy of answers; we implemented this in the program by setting a timer for each question and
scoring one point per correct answer (zero points for missed question). Final processing of the program
required enrollment as an Apple Developer, allowing us to run the application on an iPad for testing.
Results
Our Xcode programming and thorough troubleshooting resulted in a working Stroop test program that
runs on the iPad. The program has many features, each of which corresponds to important aspects of
the Stroop test and data management. Each screen plays an important role in the test as a whole. The
first is a start-up screen where the physician can enter a patient ID number that will be stored with the
score. It also includes patient instructions and a “Start” button. This screen is shown in Figure 1.
Figure 1. Screenshot of the
Stroop test start screen. The
physician can enter a patient
ID number to be stored along
with the scores.
The subsequent screens display 32 quiz questions that progress from easy (the word matches the ink
color) to more challenging (the word and ink color do not match). To answer the question, the user
must tap the button corresponding to the correct color of the ink. Each question worth one point and
is timed for five seconds; if the patient cannot answer in time, he or she misses the question and
receives zero points. The score is stored for later use. Both types of question can be seen in Figure 2.
Figure 2. Screenshots of 2 example Stroop test questions. The
user must identify the color of the ink.
The last screen is a results page which displays the patient’s score (this feature can be disabled for actual
neurocognitive testing, but remains visible for program demonstration) and a button which allows the
physician to export the score along with the patient ID number via email. Thus, the patient’s identity is
not compromised and no privacy laws are violated. This view is shown in Figure 3.
Figure 3. Screenshot of the
Stroop test start screen. The
physician can enter a patient
ID number to be stored along
with the scores.
Our final program is thus successful in fulfilling our stated requirements for a functioning Stroop test.
The number and order of questions can be altered as well as the screen view; however, we felt that it
was better to keep the program looking clean and simple to avoid unnecessary distractions to the
patient (which could skew results). The program and test were found to be simple and easy to use when
tested on university students; thus, we hope that Dr. Blair will eventually be able to use the program on
patients in his study without problems.
The cost of developing this program was quite low, and the benefits are great. Two students worked
around 100 hours coding and debugging, and costs included Apple Developer program enrollment (for
application testing and future distribution) at $99 for the year and the purchase of an iPad at $500.
There are little to no safety risks associated with program development and, since it is being created for
use as a research tool, does not require any sort of federal approval. Future benefits include an increase
in knowledge of anesthesia’s effect on the brain and an eventual decrease in number of cases of postoperative cognitive dysfunction. Once tested by Dr. Blair in his study and approved by the App Store,
the program can be shared with other hospitals for pre-operative screening. This could ultimately result
in reduced lawsuits and legal fees.
Conclusions
The final iPad program will be used in conjunction with physiological imaging (performed by Dr. Blair) to
detect changes in cognitive function and brain structures, and will provide easier methods of test
administration to facilitate his study. The creation of a successful program will allow for the combination
of multiple tests and easy data transfer for analysis, thus cutting costs. We hope our program will help
continue research into POCD and help researchers gain more knowledge of how surgery and anesthesia
affect the brain. Ultimately, we met our design specifications and produced a working Stroop test
program for the iPad.
Recommendations
In the future, we hope this study will ultimately lead to a development of better anesthetic practices to
decrease cases of POCD in the elderly. Finally, the program could eventually appear in Apple’s App
Store (after getting approval) for distribution so that other hospitals can use it for screening. To
continue to improve the application for Dr. Blair’s study, more neurocognitive tests could be
programmed and inserted into the existing program to collect multiple scores (and cognitive baseline
readings) per patient. Future versions of the test could also include a more detailed score report,
displaying the time it took to answer each question as well as the exact questions missed. We hope that
Dr. Blair will be able to test and use this application in his study to help better determine baseline
cognitive function.
Ideation Process
Innovation Situation Questionnaire
1. Brief description of the situation
Neurocognitive tests are used to diagnose post-operative cognitive dysfunction. Currently, tests
are administered either via the traditional pencil-and-paper format, or an electronic format on a
computer. Development of a single application for the Apple iPad that can administer several
tests quickly has been proposed. The problems to be solved are that the current methods of
administering these tests are time-consuming and there is no touchscreen-formatted battery of
tests available.
2. Detailed description of the situation
2.1. Supersystem - System - Subsystems
2.1.1. System name
Application for the Apple iPad
2.1.2. System structure
The application will consist of C++ coding.
2.1.3. Supersystems and environment
The Apple iPad is used to administer tests. Application must be formatted for touchscreen.
2.1.4. Systems with similar problems
Electronic versions of some of these tests are available, so coding and scoring issues have been
worked out in some cases. Randomization of answer choices has also been resolved in some
electronic versions, but the specific coding is unavailable to us.
2.2. Input - Process - Output
2.2.1. Functioning of the system
The primary function is to administer neurocognitive tests to assess the cognitive function of a
patient.
2.2.2. System inputs
The inputs to the application are the answers submitted by the user to the questions.
2.2.3. System outputs
The output, at the end of all the tests, consists of the patient's scores for each test. These scores
will indicate their cognitive function.
2.3. Cause - Problem - Effect
2.3.1. Problem to be resolved
Shorten length of time taken to administer a battery of neurocognitive tests.
2.3.2. Mechanism causing the problem
The nature of the tests themselves.
2.3.3. Undesirable consequences if the problem is not resolved
None
2.3.4. Other problems to be solved
Formatting the tests for a touchscreen presentation
2.4. Past - Present - Future
2.4.1. History of the problem
Paper-and-pencil versions of some neurocognitive tests have been supplemented by electronic
versions. Some games are also available that utilize sections of some tests.
2.4.2. Pre-process time
Time before the tests are administered.
2.4.3. Post-process time
Time after the tests are administered and scored
3. Resources, constraints and limitations
3.1. Available resources
Apple iPad
Knowledge of coding
Neurocognitive tests in paper and in electronic forms
Scoring information
3.2. Allowable changes to the system
Only minimal changes are to the neurocognitive tests themselves
Small changes to presentation of test material
Small/minimal changes to the scoring methods
3.3. Constraints and limitations
Validity of neurocognitive tests must be preserved
Test content cannot be altered
Touchscreen format required
Take about 10 minutes to administer tests
3.4. Criteria for selecting solution concepts
Program runs as desired, administering neurocognitive pre-operative tests to patients
Scoring data is statistically valid, i.e. matches data we would obtain using paper-and-pencil
versions of the same tests.
Problem Formulation and Brainstorming
Coding of
application is
completed
Application is on
the iPad
iPad battery dies
Current tests are
time-consuming
iPad application
administers NCTs
Patient cannot
read English
Patient is color
blind
Neurocognitive
assessment of
patient
Diagram 1
12/7/2010 10:14:41 PM.
8. Find a way to eliminate, reduce, or prevent Current tests are time-consuming.
Develop Concepts
1. Categorize preliminary ideas
Coding of application
Determine what language to use
Research/learn how to code for necessary functions
Accept touchscreen input
Multiple sets of test questions for repeated use
Electronic test presentation
Use currently available electronic versions: pay licensing fee
Write our own coding for each test
Try to mimic presentation of test material in existing electronic version
Randomized answer choices
Method of input of answer choice on the touchscreen
Present directions
Start with sample questions
Validation of results
Compare to existing normative data
Build our own set of data
Develop scoring based on response time and accuracy
Compare to results from paper-and-pencil version
Evaluate Results
1. Meet criteria for evaluating Concepts
Created working iPad program version of the Stroop Word and Color Test
Developed scoring method based on response time and accuracy
Tested on fellow students
Touchscreen administration of test
2. Reveal and prevent potential failures
Compatibility with different versions of iPad software
3. Apply Patterns/Lines of Evolution
Continue to work on adding other neurocognitive tests to expand battery in this program
Compare to results from paper-and-pencil version and existing normative data
4. Plan the implementation
Use in Dr. Blair’s proposed study.
Possibly sell in Apple’s App Store to expand market.
Bibliography
Baranov, D, et al. "Consensus statement: First International Workshop on Anesthetics and Alzheimer's
disease." Anesth Analg (2009): 1627-1630.
Bokeriia, L A, et al. "Asymmetric cerebral embolic load and postoperative cognitive dysfunction in
cardiac surgery." Cerebrovasc Dis (2007): 50-56.
de Lau, L M, et al. "Genetic variation in homocysteine metabolism, cognition, and white matter lesions."
Neurobiol Aging (2008).
Gabel, R A. "Counting deaths due to medical errors." JAMA (2002): 2404, 2405.
Gao, L, et al. "Postoperative cognitive dysfunction after cardiac surgery." Chest (2005): 3664-3670.
Golden, C J. "Identification of brain disorders by the Stroop Color and Word Test." J Clin Psychol
(1976): 654-658.
Hanning, C D. "Postoperative Cognitive Dysfunction." Br J Anaesth. (2005): 82-87.
Johnson, T, et al. "Postoperative cognitive dysfunction in middle-aged patients." Anesthesiology (2002):
1351-1357.
Lindholm, M L, et al. "Mortality within 2 years after surgery in relation to low intraoperative bispectral
index values and preexisting malignant disease." Anesth Analg (2009): 508-512.
Mahanna, E P, et al. "Defining neuropsychological dysfunction after coronary artery bypass grafting." Ann
Thorac Surg (1996): 1342-1347.
Moller, J T, et al. "Long-term postoperative cognitive dysfunction in the elderly ISPOCD1 study." Lancet
(1998): 857-861.
Monk, T G, et al. "Anesthetic management and one-year mortality after noncardiac surgery." Anesth
Analg. (2005): 4-10.
—. "Predictors of cognitive dysfunction after major noncardiac surgery." Anesthesiology 108.1 (2008):
18-30.
Newman, S, et al. "Postoperative cognitive dysfunction after noncardiac surgery: a systematic review."
Anesthesiology (2007): 572-590.
Pantoni, L, A Poggesi and D. Inzitari. "The relation between white-matter lesions and cognition." Curr
Opin Neurol. (2007): 390-397.
Perouansky, M and H C Jr Hemmings. "Neurotoxicity of general anesthetics: cause for concern?"
Anesthesiology (2009): 1365-1371.
Price, C C, C W Garvan and T G Monk. "Type and severity of cognitive decline in older adults after
noncardiac surgery." Anesthesiology (2008): 8-17.
Rasmussen, L S. "Postoperative cognitive dysfunction: incidence and prevention." Best Pract Res Clin
Anaesthesiol (2006): 315-330.
Rasmussen, L S, et al. "The assessment of postoperative cognitive function." Acta Anaesthesiol Scand
(2001): 275-289.
Silverstein, J H, et al. "Postoperative cognitive dysfunction in patients with preoperative cognitive
impairment: which domains are most vulnerable?" Anesthesiology (2007): 431-435.
Steinmetz, J, et al. "Long-term consequences of postoperative cognitive dysfunction." Anesthesiology
(2009): 548-555.
Strauss, E S E and Otfried Spreen. A Compendium of Neuropsychological Tests - Test Administration Commentary. New York: Oxford University Press, 2006.
Stroop, J R. "Studies of the interference in serial verbal reaction." Journal of Experimental Psychology
(1935): 643-662.
Warner, M A and T G Monk. "The impact of lack of standardized definitions on the specialty."
Anesthesiology (2007): 198-199.