Running Head: THE RETINOPATHY OF PREMATURITY SCREENING AND RISK ASSESSMENT TOOL
The Retinopathy
Of Prematurity
Screening and Risk
Assessment Tool
(ROPSRAT)
Clinical Decision
Support System
406-DL CDSS
Northwestern
University
Submitted to:
Professor Robert Cundick
Submitted by
Christopher Kelly and
Maria Almacen
March 2014
The Retinopathy of Prematurity Risk Assessment Tool
1
Abstract
Retinopathy of prematurity (ROP) is a potentially blinding disease affecting premature infants. It
is critical that premature babies be screened and treated in an appropriate fashion. The
Retinopathy of Prematurity Screening and Risk Assessment Tool (ROPSRAT) is a clinical
decision support tool (CDSS) designed for use with the Epic electronic medical record (EMR)
program with a two-fold purpose: It implements the latest clinical recommendations for
screening and provides a quantitative risk estimate to facilitate communication between the
ophthalmologist and Neonatal Intensive Care Unit (NICU) staff, as well as between Neonatal
Intensive Care Unit (NICU) staff and the families of premature infants. Data gathered by
ROPSRAT can be used to evaluate ophthalmologists and NICUs to improve outcomes.
ROPSRAT uses the terminology of the International Classification of Retinopathy of
Prematurity (ICROP) to apply the latest clinical guidelines. This paper discusses the goals and
design of ROPSRAT, along with stakeholders and its effect on workflow. It then presents a
change management plan to facilitate its introduction into the NICU along with a discussion of
content specification, user interface, and method of evaluation.
Keywords: retinopathy of prematurity, screening guidelines, clinical decision-support system.
The Retinopathy of Prematurity Risk Assessment Tool
2
Table of Contents
Introduction ..................................................................................................................................... 4
Goals ............................................................................................................................................... 5
Intervention Selection for ROPSRAT ............................................................................................ 7
Objective 1: Develop a recommendation tool on the appropriate timing for follow-up and
treatment of ROP that is accurate and timely integrated in clinician’s workflow based on the
most current guidelines set by the AAP. ..................................................................................... 7
Objective 2: To increase family satisfaction by use of a developed tool that quantifies the risk
of an infant’s ROP progression to be utilized by NICU staff or ophthalmologists for
communication with families. ..................................................................................................... 7
Objective 3: To decrease incidence of critical ROP or incidence of blindness among
premature infants by using a tool that allows quantitative risk comparison across NICUs. ....... 9
Objective 4: To increase rate of consistencies in among ophthalmologists by the use of a tool
that allows comparison of ophthalmologist’s performance or practice. ................................... 12
Information System Inventory ...................................................................................................... 14
Information Sources ...................................................................................................................... 15
User Input...................................................................................................................................... 16
Output ........................................................................................................................................... 16
Stakeholders .................................................................................................................................. 17
Change Management Plan ............................................................................................................ 20
Treatment and Follow-up Recommendations ........................................................................... 21
Risk Score Calculation .............................................................................................................. 22
Planning and Development. ................................................................................................... 24
Post Implementation. ............................................................................................................. 26
Evaluation ..................................................................................................................................... 26
System Response Time ............................................................................................................. 27
Structure .................................................................................................................................... 27
Process ....................................................................................................................................... 27
Outcome .................................................................................................................................... 29
Discussion ..................................................................................................................................... 30
References ..................................................................................................................................... 32
Table ............................................................................................................................................. 34
The Retinopathy of Prematurity Risk Assessment Tool
3
Figures........................................................................................................................................... 35
Figure 1 ..................................................................................................................................... 35
Goals and Objectives ................................................................................................................ 35
Figure 2 ..................................................................................................................................... 36
Pop-up screen showing recommended follow up. ..................................................................... 36
Figure 3 ..................................................................................................................................... 37
Sample letter to families of infants with stage 2/zone 2/no plus ROP and 25% calculated risk
of progression to needing laser surgery. ................................................................................... 37
Figure 4 ..................................................................................................................................... 38
Stakeholder Workflow for the Follow-up Recommendation Tool and Order set ...................... 38
The Retinopathy of Prematurity Risk Assessment Tool
4
Introduction
ROP is a potentially blinding disease that affects the eyes of premature infants.
Normally, an infant’s retina does not become fully vascularized until shortly before birth. When
infants are born prematurely, the normal vascularization process can be altered. Instead of blood
vessels growing out from the back of the eye, along its inside and towards the front, blood vessel
growth stops. As the infant matures, vessel growth resumes but can do so in a pathologic way.
The vessels may grow inwards, towards the center of the eye. When this happens, they can end
up tearing the retina off the eye, bringing about total retinal detachment and complete blindness.
Fortunately, a treatment exists. Most blindness can be prevented if the disease is
identified and treated early. A major step in developing a treatment was the creation of a
consistent terminology for the condition. The ICROP was initially published in 1984 and most
recently revised in 2005. It represents the collaborative efforts of specialists across the world
(International Committee for the Classification of Retinopathy of Prematurity, 2005). The
ICROP divided the infant’s retinas into three zones. The disease was then labeled according to
zone involvement. Zone 1 disease, closest to the back of the eye, occurs when blood vessel
development is most severely altered. Zone 3 disease, closest to the front, is typically mild in
form. In addition to zones, there are six stages, ranging from Stage 0, where the vessels are
immature and gradually taper off, to Stage 5, where the retina has completely detached and the
child is permanently blind. Finally, there are terms used to describe the appearance of the retinal
blood vessels. “Plus disease” describes a significant amount of retinal blood vessel dilation and
tortuosity and is a poor prognostic indicator. Less severe is “pre-plus disease.” “No plus” is used
when the blood vessels appear normal.
The Retinopathy of Prematurity Risk Assessment Tool
5
Several large-scale clinical trials have shown that treatment can be effective in preventing
the progression to Stage 5 if performed at the appropriate time, so it becomes critical that these
infants have retinal examinations before they get to this point. Clinical guidelines have been
published recommending appropriate times for screening and follow up based on the infant’s
post conceptual age at birth, birth weight, current zone, and current stage. The most recent
clinical guideline was published in 2013 (American Academy of Pediatrics, Section on
Ophthalmology, et al., 2013). Infants with stage 1, 2, or 3 may need to have laser surgery
depending on the zone and blood vessel appearance to prevent the progression to stage 4, a
partial retinal detachment, or stage 5, complete blindness.
Goals
Although the incidence of blindness is about ten times lower in children compared to
adults, retinopathy of prematurity is a disease that afflicts many premature infants due to
advances in neonatal care (“Facts about retinopathy,” n.d.). As more premature infants are being
saved the incidence of ROP increases. The more premature the age of the infant the higher risk
for ROP. While approximately 16,000 infants develop ROP by some degree, almost 1500 infants
in the United States progress with ROP severe enough to require medical or surgical treatment
for the disease, and about 600 of these infants become legally blind each year (“Facts about
retinopathy,” n.d.). The financial cost of managing a blind or visually impaired child is
enormous and the disruption of the child’s and family’s life is incalculable. This financial
burden is also shared by society and healthcare payer in providing for the medical or surgical
need and home care support of a disabled child. In contrast to the slowing economy, health care
expenditure grew faster from $1106 per individual to $6280 in 24 years from 1980 (Stanton,
2006). With the accelerating pace of medical innovation and aging population combined, health
The Retinopathy of Prematurity Risk Assessment Tool
6
care costs will continue to rise from 9 percent to more than 16 percent (Stanton, 2006).
Although complete blindness from ROP fortunately is a relatively rare condition, the cost to
society of a blind infant is enormous. Malpractice judgments reflect this. Plaintiffs have been
awarded over $10 million in court cases (Day, Menke, & Abbott, 2009). ROP is thus an
important condition from a societal standpoint, as well as for healthcare organizations that take
care of premature infants. Improving the care, as well as communication with families and
family satisfaction, has the potential to mitigate significant financial risk. It is, therefore,
imperative for health care providers to work together in providing better health care quality at a
reduced cost.
Our organization aims to improve the visual health among premature infants and
optimize cost-effectiveness of visual care (see Figure 1). Clinically, our goal is to improve the
quality of ROP care and enhance communication of the condition among clinicians and families
For these reasons, our institution intends to develop the Retinopathy of Prematurity Screening
and Risk Assessment Tool (ROPSRAT). This ROPSRAT project aims to achieve the following
objectives (also see Figure 1):
1.
Develop a recommendation tool on the appropriate timing for follow-up and
treatment of ROP that is accurate and timely integrated in clinician’s workflow based
on the most current guidelines set by the AAP.
2. To increase family satisfaction by use of a developed tool that quantifies the risk of
an infant’s ROP progression to be utilized by NICU staff or ophthalmologists for
communication with families.
3. To decrease incidence of critical ROP or incidence of blindness among premature
infants by using a tool that allows quantitative risk comparison across NICUs.
The Retinopathy of Prematurity Risk Assessment Tool
7
4. To increase rate of consistencies among ophthalmologists by the use of a tool that
allows comparison of ophthalmologist’s performance or practice.
Intervention Selection for ROPSRAT
Objective 1: Develop a recommendation tool on the appropriate timing for follow-up and
treatment of ROP that is accurate and timely integrated in clinician’s workflow based on
the most current guidelines set by the AAP.
As a mechanism for improving the quality of visual care of premature infants, the
Retinopathy of Prematurity Screening and Risk Assessment Tool (ROPSRAT) will function as a
decision support tool in two ways: as a clinical decision support tool and risk assessment tool.
As a decision support tool, it will implement the most recent ROP screening guidelines. The tool
will be updated every few years to take into account advances in the diagnosis and treatment of
ROP. Screening for ROP is typically done by pediatric ophthalmologists in larger NICUs.
However, in smaller NICUs, screening may be done by general ophthalmologists who may not
be aware of the latest recommendations. Treatment is often done by retinal specialists and
typically involves laser surgery. Although the indications are very different, the operation is
technically similar to the laser surgery done for proliferative diabetic retinopathy. While ROP
screening may involve five percent or more of a pediatric ophthalmologist’s practice, it is
typically less than one percent of the retinal specialist’s, who also may not be as aware of the
latest recommendations.
The screening component of ROPSRAT provides an alert to ophthalmologists
recommending appropriate follow up based on the latest eye examination findings. For instance,
once the ophthalmologist enters the exam results of “Stage 1/Zone 2/No Plus”, an alert box will
The Retinopathy of Prematurity Risk Assessment Tool
8
pop up recommending a follow-up eye examination in two weeks, as per guidelines published by
the American Academy of Pediatrics (2013). Figure 2 shows a mock-up of how this warning
box will appear. Clicking the “Follow up in 2 weeks” button will trigger an Epic “SmartSet”
order entering the infant’s name into a list of infants who will be seen in two weeks. The
warning box also allows an override for when the examining ophthalmologist feels that an
examination should be sooner or later than two weeks. In this case, a calendar will pop up and
the ophthalmologist will schedule follow up by clicking the appropriate date. There will be
several alternative screens, saying “Surgery is recommended,” “Follow up in 1 week,” “Follow
up in 2 weeks,” or “Follow up in 6 months.” Scheduled follow up visits will be stored in a
system-wide electronic database.
The most recent guidelines will be incorporated into the alerts, making the right thing to
do as the easy thing to do. The alert message will also show if surgery is indicated. Once the risk
of blindness no longer exists (the retina is “mature” or the ROP has regressed), an alert will pop
up saying follow-up is recommended in six months as these infants are at a higher risk of
amblyopia and strabismus. This intervention has the advantage of taking place immediately after
the exam, while the data is being entered into the computer by the ophthalmologist, and so will
have minimal effect on current workflow.
The alerts can be configured into an Epic “SmartForm” using a series of “If…Then”
statements. There are three zones and four stages that occur prior to the retinal detachment
stages. Additionally, there are three terms to describe the appearance of the retinal vessels.
Counting “regressed” and “mature” descriptions, there are 38 possible combinations. It may be
difficult even for an experienced pediatric ophthalmologist to keep track all these, especially in
light of changing guidelines. The Decision Logic table shows all these possibilities and the
The Retinopathy of Prematurity Risk Assessment Tool
9
corresponding recommendations according to the 2013 American Academy of Pediatrics
guidelines. As guidelines change in the future, it should be a fairly straightforward process to
modify the system’s responses once they are configured in the system. If additional descriptive
terms are added (for example, if a term is added for “posterior zone 2”; see discussion below), it
will be possible to add those terms to the decision logic as well.
Objective 2: To increase family satisfaction by use of a developed tool that quantifies the
risk of an infant’s ROP progression to be utilized by NICU staff or ophthalmologists for
communication with families.
ROPSRAT Risk Assessment, the second portion of the clinical decision support tool, is
intended to improve communication between screening ophthalmologists and NICU staff, along
with the parents of the infants at risk. It will also provide a mechanism for measuring and
comparing outcomes between NICUs and the evaluation of ophthalmologists. Several risk
factors have been identified that influence the risk of an infant progressing to the point where
treatment is indicated, the most important being post-conceptual age at birth and birth weight.
There are other influences as well, including infection and other comorbid conditions, but these
are not consistently identified in retrospective analyses. By incorporating post conceptual age at
birth, birth weight, current age, stage, zone and a measure of blood vessel appearance into a
model, an estimate can be made of the risk of progression to needing laser surgery. This
estimate can then be conveyed to neonatologists, neonatal nurses, and the infant’s parents.
Developing a risk assessment model may be more problematic. There are a number of
risk factors that determine the probability of progression to the point where treatment is needed.
Well known risk factors include birth weight, post conceptual age of birth, current age, current
zone, stage and blood vessel appearance. Other risk factors have been identified, but they may
The Retinopathy of Prematurity Risk Assessment Tool
10
be more dependent on local factors since they do not seem to consistently show up in
retrospective analyses.
Multivariate regression analysis will be used to create a model that predicts each child’s
risk. Each of the variables that are known to influence progression can be given a weight
according to historical data. Once sufficient data has accumulated, values for each baby’s risk
factors can be used to calculate the risk of progression after each exam. However, even a
multivariate regression analysis would not necessarily be simple. Infants born at 23, 24, and 25
weeks are at significantly higher risk than infants born at 28 weeks, and risk may decline more
quickly than a linear model might predict. Additionally, current age influences the likelihood of
progression to needing laser surgery. However, the most common age for needing laser surgery
is 35 to 36 weeks post conceptual age. After that, even for infants with the same stage, zone, and
blood vessel appearance, the risk begins to decline. Any meaningful multivariate regression
analysis will need to include nonlinear parameters in its calculation of risk score. Alternatively,
statistical techniques using a survival curve may be beneficial. This may need to be explored in
more detail.
Once the score is calculated after each exam, it could then be made part of the patient’s
record so NICU staff can follow and monitor from week to week whether the score, and
consequently the infant’s risk, is increasing or decreasing. It would serve to alert neonatologists
as to which neonates face the highest risks of progression. Such alerts may influence decisions
about discharge or transfer. One important consideration is that eye disease does not occur in
isolation; these babies often have other co-morbid conditions, and a probability score could be
helpful in scheduling other procedures. For example, it might be better to delay a hernia repair
on a child if eye surgery is imminent.
The Retinopathy of Prematurity Risk Assessment Tool
11
Having a premature, ill, and hospitalized infant can be a terrifying experience for parents.
So many systems can be affected and so many critical health issues can develop, that parents
easily become overwhelmed. The idea that an infant could end up blind may be a low priority
when the infant is unable to breathe. On the other hand, being told about a risk of blindness
could be terrifying for parents, even in cases where the risk is low. One difficulty exists in
healthcare is that qualitative terms expressing risk like “unlikely” or “possible” may be
interpreted differently by different people. Parents who are told that their child may go blind
may over-interpret the risk of such a severe event. As discussed in Hunink, Glasziou & Siegel,
et al. (2001, p. 34-35), there are advantages to being able to express uncertainty and risk
quantitatively. A quantified risk score can help focus parental concern on salient risks. In
addition to the NICU staff, the score will also be given to parents each week to help them
understand their preemie’s risk. It is likely that interpreting such a score would not be an easy
task for most parents, so it will be critical to have neonatal nurses understand the risk and be able
to translate this score for parents. Preparing parents for possible laser surgery several weeks in
advance could be a major step in improving family satisfaction.
A pre-exam risk score will be based on birth weight and post conceptual age at birth.
After each exam, a calculation will be performed by the Epic “Smart Form” using results from a
recent multivariate regression analysis. This score will then be incorporated into the medical
record for neonatologists. The score will also be included in an information sheet for the family.
Infants at high risk will get different information than infants at low risk. A sample parental
information sheet for a baby with stage 2/zone 2/no plus disease and a calculated 25% chance of
progression is shown in Figure 3. This letter will be auto-generated by Epic with links showing
the infant’s name, gender, and risk, as well the name of the ophthalmologist. Different letters
The Retinopathy of Prematurity Risk Assessment Tool
12
will be generated for infants at LOW, MODERATE, and HIGH risk, and for infants requiring
surgery. When an infant’s eyes are no longer at risk for blindness, a different letter is generated
discussing that infants born prematurely have a higher risk of developing amblyopia, strabismus,
and corrective glasses at a young age, and recommending a follow-up in six months. The letter
may also be translated into Spanish, and other languages as needed.
After each exam, parents will see if their child’s disease is progressing based on whether
the score increases or decreases. As the score increases, the neonatologist and NICU staff will
discuss the risks with an increasing sense of urgency utilizing the information generated by the
ROPSRAT CDSS.
Objective 3: To decrease incidence of critical ROP or incidence of blindness among
premature infants by using a tool that allows quantitative risk comparison across NICUs.
Differences in populations being compared can make assessing outcomes in different
geographic locations challenging. Providing a quantitative risk analysis will allow the
ROPSRAT compare outcomes in different NICUs in consideration of risk adjustment.
A multivariate non-linear regression model will be used by ROPSRAT to evaluate
quality. While it is possible to determine the number of infants at each NICU who progress to
the point of needing surgery, an absolute proportion without risk adjustment is largely
meaningless. However, with ROPSRAT, it will be possible to determine if an infant born at one
NICU has a higher risk for progression than a baby with similar characteristics born in another.
Does a baby born at 25 weeks and 650 grams, now at 33 weeks and having stage 2/zone 2/no
plus, for example, have a higher chance of progressing in one NICU than another? It will likely
take years of data to develop sufficient statistical data analysis to answer this question, but if
results are not quantified, improvements cannot be made. Such information will allow data
The Retinopathy of Prematurity Risk Assessment Tool
13
mining techniques to potentially determine other risk factors. It may turn out that local practices
in care or monitoring could have an influence on outcomes that could not be predicted in
advance.
Objective 4: To increase rate of consistencies in among ophthalmologists by the use of a
tool that allows comparison of ophthalmologist’s performance or practice.
Similarly, evaluating physician’s eye examination techniques in varied settings can be
particularly difficult, since the examination is only performed once each visit. ROPSRAT will
allow risk-adjusted comparisons between ophthalmologists to see if there are any outliers in the
application of specific definitions. With the ultimate goal of preventing vision impairment
among infants, the ROPSRAT aims to manage this condition by allowing risk-adjusted
comparisons between retina specialists that may identify those with worse outcomes.
The ROPSRAT multivariate regression analysis model will also allow comparison among
ophthalmologists as well. Three ophthalmologists employed by Connecticut Children’s Medical
Center together perform approximately 1000 screening examinations each year. Since each
exam is often performed by only one ophthalmologist, limited opportunities exist to make sure
all ophthalmologists apply the standards in the same way. Zone, in particular, is challenging to
ascertain since it can be difficult to determine locations within the retina. Quantified data could
help confirm if an ophthalmologist is an outlier in his or her application of terms. For example,
does one ophthalmologist significantly identify more zone 3 disease than would be predicted
based on the characteristics of the infants examined? Additionally, one ophthalmologist may be
an outlier in terms of his or her application of the current clinical guidelines, scheduling follow
ups more or less often than recommended. This could certainly be illuminating, although it must
The Retinopathy of Prematurity Risk Assessment Tool
14
be emphasized to ophthalmologists that the tool is not intended to “grade” them, rather to
provide feedback that does not currently exist.
What really matters is the long-term visual outcome of these children. Even if complete
blindness does not result, delay in treatment can cause damage to the retina that prevents the
child from reaching full visual potential. Laser surgery on these infants is technically
challenging and some retinal specialists may be more proficient than others. Data collected by
the ROPSRAT will allow risk-adjusted outcome comparisons. If outcomes are not measured,
they cannot be improved.
Information System Inventory
The ROPSRAT will require a robust technology infrastructure for its success. It will be
developed for use with the Epic EMR, taking advantage of the clinical decision support tools
available in its ecosystem. However, since Epic EMR is already “live” in production prior to the
initiation of the CDS intervention, separate hardware and software resources will not be
necessary. As a result, performance, availability, security, and other technical as well as
operational requirements for the ROPSRAT will be based on the requirements of the currently
deployed Epic EMR.
The ophthalmologist will enter data at bedside using a laptop or mobile device equipped
with an Epic EMR client and connected to the network through a wireless access point. There
are multiple workstations throughout the NICU for neonatologists and nurses to access the
system, although there is no individual Epic client device at each bedside.
Standard vocabularies have an important role in building ROPSRAT as it is essential for
the attainment of semantic interoperability with other systems, such as for other provider’s
The Retinopathy of Prematurity Risk Assessment Tool
15
adoption of the ROPSRAT CDSS or for government reporting purposes as recommended by
Osheroff et al. (2012). The appropriate ontology for ROPSRAT may prove to be challenging. In
ICD-9, there are codes that describe the stage of the ROP disease and a general term for
“Retinopathy of prematurity, unspecified.” ICD-10 adds some specificity and includes codes for
each stage, specifying right eye, left eye, both eyes, and unspecified eye. But even ICD-10 codes
do not consider zone or blood vessel appearance, key elements in determining whether treatment
is needed (ICD-10 Data.com, 2014). Fortunately, more detail is available within the database
commonly used by Epic, so that the creation of additional diagnostic elements is not imperative.
Even the Epic database may prove limiting however. In some situations it might be ideal
to subdivide the categories further. Zones described by the ICROP scheme were based on
landmarks within the retina to provide observer agreement. Not enough was known about the
disease in 1984 to know if they reflected clinical reality. There is some evidence that disease
occurring in posterior zone 2 may behave more like disease in zone 1 than disease in mid-or
anterior zone 2 (Early Treatment for Retinopathy of Prematurity Cooperative Group, 2003). A
category for “posterior zone 2” could be developed and applied locally. It might be valuable in
retrospective evaluations and it is even possible that screening recommendations might vary for
disease in this location. However, it would likely be mapped to the same zone 2 term in the Epic
database.
Information Sources
The American Academy of Pediatrics updates clinical guidelines every few years, most
recently in 2013 (American Academy of Pediatrics Section on Ophthalmology, et al., 2013).
These are widely available.
The Retinopathy of Prematurity Risk Assessment Tool
16
After ROPSRAT has collected sufficient data, data mining techniques can help determine
other risk factors. A number of risk factors have been identified that potentially influence the
risk of progression besides post conceptual age at birth and birth weight included in the
ROPSRAT model. These factors range from ambient light exposure, to gender, to whether the
infant is fed breast milk. However, these do not seem consistent across NICUs. Searching the
literature will provide potential risk factors that can be used in a supervised data mining
approach to determine which, if any, risk factors influence outcomes locally. Alternatively, there
may be risk factors that have not been considered, demography for example, which a supervised
data mining approach could use. The approach should be supervised, rather than unsupervised,
since the progression of retinopathy is a predefined dependent variable (Hardin & Chhieng,
2007, p. 47).
User Input
The ophthalmologist will enter data on stage, zone, and blood vessel appearance into an
Epic EMR SmartForm with decision logic built in. Other data, including birth data, gestational
age at birth, and other health issues will be entered as part of the EMR.
Output
ROPSRAT will provide two types of output. First is an alert reminder for the
ophthalmologist that will provide a recommendation for follow up or treatment. The second is a
quantitative risk score for those infants that do not yet require treatment, ranging from near zero
when infants with similar risk factors have never gone on to need surgery, to near 100% if
progression is imminent. These risk scores would need to be expressed as a range, reflecting
statistical uncertainty.
The Retinopathy of Prematurity Risk Assessment Tool
17
Stakeholders
ROPSRAT will need the support of, and will benefit from, individuals at multiple levels
within the organization. The following are the stakeholders of this CDSS project:
1. Lead Clinician
The lead clinician will be a pediatric ophthalmologist. The pediatric ophthalmologist is
the expert in following and implementing the 2013 ROP guidelines that will be used in
developing this CDSS. The lead clinician will be the main user of the CDSS, entering the results
of the eye exams in the system and receiving the alert messages. This strategy will make
ROPSRAT user-friendly and will ensure that it functions as useful and relevant as possible.
The lead clinician, who is also the spokesperson for other pediatric ophthalmologists
using the system, will work with the technical team and will communicate with staff about the
interventions before, during, and after the launch.
2. Physician champion
The physician champion will be a neonatologist. Neonatologists bear primary
responsibility for the overall care of the neonate. They order the initial exam and often serve to
communicate the ophthalmologist’s findings to the parents, since the parents may not be
available when the eye exams are performed. Often, neonates are discharged or transferred when
still at risk for eye disease progression. This makes neonatologists indispensable in ensuring
appropriate follow up. The physician champion will provide essential input on how ROPSRAT
will be incorporated into the EMR, and how the data will be accessed. The physician champion
will also encourage discussions among other staff to address any concerns or issues. It will be the
physician champion who is responsible for making the user interface as straight-forward for the
The Retinopathy of Prematurity Risk Assessment Tool
18
NICU staff as possible. Ultimately, it is the physician champion who will manage and address
the staff’s expectations.
3. Nurse Champion
Nurses are on the front lines in the relationship between the NICU and the parents.
Having a baby can be stressful for any family; having a sick preemie can be overwhelming.
Parents who have discussions with ophthalmologists or neonatologists may not fully understand
the implications of clinical information or may not come up with questions until later. This may
be particularly true for ROP, since a problem that takes place inside the eyes may seem much
less urgent than respiratory or infectious complications. The nurse champion must be a good
communicator with years of experience as a neonatal nurse working with the neonatal
population. It is critical that the nurse champion understands the needs of parents. What
questions do parents typically ask? Where do misunderstandings commonly occur? The nurse
champion must also be aware of the needs of fellow nurses so that information about ROP, as
well as risks and follow up, is presented in a clear and unambiguous way. The nurse champion
shall act as the spokesperson for the nurse end-users and handle the management of revisions and
updates that will be necessary during and after the launch
4. Technical Specialist
A technical specialist shall be tasked to ensure that the ROPSRAT implementation is
properly deployed and correctly functions as intended. Since the EMR system will be Epic, the
technical specialist must be an Epic clinical analyst.
5. Super Users
The super users must be identified and be actively involved from inception. They must
represent the main users of ROPSRAT, namely: a neonatologist, a pediatric ophthalmologist, and
The Retinopathy of Prematurity Risk Assessment Tool
19
a neonatal nurse. It is critical for those who were not directly involved in the project
development to have someone they can turn to for questions.
6. Chief Financial Officer and Chief Executive Officer
Involvement of the higher levels of management will be required to finance and develop
ROPSRAT and to align the project with the organization's overall goals.
7. Chief Medical Information Officer
The Chief Medical Information Officer (CMIO), who reports to the Chief Information
Officer and the Chief Medical Officer, shall be responsible for the first level approval of
ROPSRAT prior to its development and shall also work to facilitate the approval and allocation
of development funding. The CMIO must also see to it that the guidelines implemented represent
the latest recommendations of American Association of Pediatrics. The CMIO shall present and
recommend the CDSS project to the CIO, CMO, CFO, and CEO for further discussions and final
approval for development.
8. Quality Officer
The Quality Officer shall ensure that the development and implementation of the CDSS
will uphold data quality improvement measures. The Quality Assurance Officer may help
recognize potential opportunities and shall identify risks associated with the CDSS during
development, as early identification of potential risks may help reduce actual risks.
9. Board of trustees
Discussion with the members of the Board of Trustees will help clarify and highlight the
role of the CDSS in addressing quality issues and in keeping the project in line with the
organization’s overall goals and objectives.
10. EMR committee representative
The Retinopathy of Prematurity Risk Assessment Tool
20
The EMR vendor (Epic) shall designate a committee representative who will provide
guidance in integrating ROPSRAT into the EMR. This committee member shall bring relevant
experience and invaluable input to CDSS development and deployment.
11. Risk Management Officer or Legal Counsel
The Risk Management Officer or Legal Counsel shall determine the regulatory or legal
risks involved in the development and deployment of ROPSRAT. One particular concern is that
the risk of an infant’s disease progression may vary among institutions. If infants in a particular
NICU are at a higher risk, is the organization exposed to any additional liability? Malpractice
judgments for ROP can be in the tens of millions of dollars, so this condition is particularly
fraught with financial and legal risks. In addition, communication with families must convey the
risks of the condition in an appropriate way.
12. Patient Representatives
Patient representatives are usually parents of NICU graduates. They can provide essential
insights as to what families go through in this difficult setting. Their input could be invaluable in
determining how relevant information is provided to families in an appropriate but not
overwhelming manner. Their voice will provide critical input and will have a significant impact
on the successful development and deployment of ROPSRAT.
Change Management Plan
The ROPSRAT CDSS is a two part project, providing a recommendation for follow up or
treatment of infants for retinopathy of prematurity based on the latest clinical guidelines and
serving as a communication mechanism between consultant ophthalmologists and NICU staff,
and between NICU staff and patient families. Change management of each component will be
handled separately.
The Retinopathy of Prematurity Risk Assessment Tool
21
Treatment and Follow-up Recommendations
Introducing ROPSRAT into the organization is the concern and responsibility of the
project team leader who must understand the needs and concerns of the treating ophthalmologist
stakeholders. The system is designed to improve their workflow and to make it easier to
implement the latest recommendations. Some potential reasons for resistance to change include
the fact that ROPSRAT may be perceived as threatening because it takes away physician
autonomy. It is critical to make certain that ophthalmologists understand that these are
recommendations, and that they have the ability to override them as they see clinically necessary.
These recommendations must appear in a timely fashion. If there is a significant delay after the
ophthalmologist enters his examination notes into the EMR, the ophthalmologist may become
frustrated by the tool and may just proceed without the recommendation.
To mitigate the risk of non-acceptance, the project team leader must act as a facilitator.
ROPSRAT should not be imposed upon the users and the team leader must step back and allow
the stakeholders to take part in the building process of the CDSS. The ophthalmology
stakeholders must understand that this is being developed for them. Without doubt, there will be
modifications after the roll out; this needs to be discussed in advance so stakeholder’s initial
expectation is not perfection. The lead clinician will take the role of super-user and act as a
liaison between users and the project team to help work out difficulties in user acceptance testing
before it is offered more widely. This ophthalmologist’s role is vital in identifying errors in the
offered recommendations. If the tool can be presented in such a way that ophthalmologists
understand its use and purpose, and recommendations are made in a timely fashion, acceptance
hopefully will not be a major concern.
The Retinopathy of Prematurity Risk Assessment Tool
22
Risk Score Calculation
ROP is a potentially blinding disease for premature infants, but only a small portion, less
than 5%, of those infants who are screened for the disease require laser surgery for the condition,
and only a small percentage of this population develops blindness despite laser surgery. The real
success of ROPSRAT will be in how well it is accepted by the NICU staff and families.
Generating a risk score will be meaningless if it is not used for management decisions or family
discussions.
Change management can be approached by the creation of a change management team.
The change management team will be composed of a subsection of the project team, including an
ophthalmologist, a neonatologist, and a NICU nurse. The impression of ROPSRAT’s designers
is that ROP is often not well understood by the NICU staff, and that when a patient needs
surgery, families are often taken unawares by a problem they did not realize was a major
concern. Part of this stems from the fact that the ophthalmologists often round early in the
morning when families are not present and so there is little opportunity for discussion. This has
the potential to lead to misunderstandings, anger and ultimately blame when there is a bad
outcome. ROP is a major malpractice concern (Day, Menke & Abbott, 2009). Judgments can be
in the tens of millions of dollars. Improving communications can help reduce the risk of
malpractice lawsuits. While it may be difficult to calculate a return on investment for this
project based on financial considerations for very rare events, decreasing the risk of severe
malpractice lawsuits will prove sufficient financial motivation.
At the outset, the project leader must identify this issue and present it to the NICU staff,
stressing the significance of communication to obtain parental satisfaction in reducing
malpractice risk. However, prior to discussing it with the NICU staff, it might be worthwhile to
The Retinopathy of Prematurity Risk Assessment Tool
23
gather two types of data. First, it would be useful to assess the understanding of ROP among the
NICU staff. It could be presented as a survey or questionnaire. There have been a lot of changes
in the understanding and management of this condition in the last two decades and their current
knowledge may not reflect this. Likewise, some structured interviews with NICU staff could be
performed. It may be informative to have the lead nurse on the project team observe a discussion
with a nurse and a willing family about ROP risk. In addition to assessing staff understanding,
there needs to be a means of assessing parental appreciation for the disease’s potential. It might
be possible to take a survey of families who have “graduated” from the NICU, asking them about
their experiences with retinopathy. Retrospective surveys may be difficult to interpret, so an
alternative brief questionnaire could be administered by a discharge coordinator to randomly
selected families. This information could be enlightening when presented to the NICU staff. It
may be that parents have an adequate understanding of the condition and its consequences, but
again, it is the impression of the developers of ROPSRAT that this is not the case.
ROPSRAT will be initially developed for use at Connecticut Children’s Medical Center
in Hartford, Connecticut. The structure of this institution presents some interesting
opportunities. Connecticut Children’s manages two NICU’s, one at Hartford Hospital and one
several towns away at the University of Connecticut. ROPSRAT could initially be rolled out at
one institution and parental understanding and satisfaction with their child’s treatment could be
assessed, using the other institution as a control group.
Since the risk score generated by the tool will be used primarily by neonatologists and
NICU nurses, it becomes critical to have them express how they would like such a tool to be
presented. Neonatologists, responsible for the management of the overall health of these infants,
including other possible surgical interventions and their discharge or transfer, might have
The Retinopathy of Prematurity Risk Assessment Tool
24
different needs than NICU nurses, who are often on the front lines with family communication.
Additional education of NICU staff about the condition could be helpful at this point. This
might be something as simple as a lecture. ROP is only one of many conditions that the NICU
staff needs to consider with these children and their families, so information needs to be
presented in a concise, yet informative way. The project leader, as the main advocate, will then
introduce and discuss the ROPSRAT CDSS, discussing what it has to offer and its technical
limitations.
The project team must be prepared for resistance during the first meeting, and listen to
the stakeholder’s concerns. They must fully understand the reasons behind the resistance as they
can reveal obstacles better dealt with at the beginning stage. It may be that NICU nurses have a
very different perspective on family communication than ophthalmologists and neonatologists,
and such a perspective must be given due consideration.
As a facilitator, it is important that the project leader, who is the change manager, build
the bridge of communication as open as possible by encouraging users and other stakeholders to
fully participate and contribute from project inception through conferences, meetings,
consultations, and other collaborative events for decision-making and project milestones review.
After the first forum, the change team must continually involve and communicate with other
stakeholders in the project’s design, development, implementation, deployment, and evaluation.
Planning and Development. Since this is a single score representing risk, it will likely
not involve major changes in workflow. However it must be introduced with sensitivity to the
existing burdens of a busy staff. Workshops can be done to review priorities with the
stakeholders, reach agreement regarding short term and long term action plans, discuss
implementation methods, and examine accountabilities. One potential technique that could be
The Retinopathy of Prematurity Risk Assessment Tool
25
developed for use in a workshop would be to present nursing staff with three different risk
scores, one for a child at high risk, one for a child at moderate risk, and one for a child at low
risk. The approach to families with children at different risk levels should be different.
Obviously, the parents of children at high risk need to understand the severity of the problem, but
it is just important that the parents of children at low risk not be burdened with undue worry
about a problem unlikely to affect their infant. By incorporating these stakeholders into the
change process at this point, accepting change in their workflow will not be as difficult. There
will be excitement rather than resistance as they know what and how change is coming. At this
point, it will be possible to identify nurses and neonatologists to serve as clinical champions to
help those less involved in the process understand how to use the risk score clinically.
Once the CDSS is developed, the system must be tested prior to implementation, to
determine where improvements or changes need to be made. Among other methods of
evaluation, a successful user acceptance test will be critical to the successful deployment and
adoption of the CDSS. Any errors or failures of the CDSS must be identified during the testing
period, triaged and fully documented, and eventually remediated. Otherwise, end-user adoption
may not follow which will invariably result in an overall project failure.
Implementation. One advantage of involving stakeholders and end-users in the creation
of the CDSS is providing them with a sense of ownership. Clinical champions and super users
will be critical at this phase, who will serve as ambassadors of the new CDSS. They should be
available and ready to assist end-users when needed, particularly at the time when the CDSS will
be utilized the most. Staff surveys initially may not be helpful but after implementation could
provide crucial feedback about issues and needs for improvement. During the first few weeks
after rollout, the project team members must be available to answer questions and help
The Retinopathy of Prematurity Risk Assessment Tool
26
interpretation of the results. During the initial deployment stage, allowing end-users and other
stakeholders to provide input, through face-to-face meetings or surveys, regarding needed
changes or improvements will also expedite the successful adoption of the CDSS.
Post Implementation. A few weeks after initial deployment, it is important to
communicate success indicators by reporting the results of measurable objectives. Initial
accomplishments and continuing achievements must be noted, acknowledged, and celebrated.
Part of the ongoing and continuous improvement of this change is to give individual and group
credits to fortify the change in the organization. Another part of this post-implementation period
is to evaluate successes and failures to allow for self-reflection on where individuals can improve
for another possible project (“Change management, n.d.). Clearly, ROP is only one of many
threats that a premature infant faces. A similar approach could be taken for a number of other
common conditions.
Evaluation
Evaluation of the ROPSRAT will be an ongoing process that begins from the early stage
of development and continuously during its implementation. Usability of the ROPSRAT CDSS
is integral to its adoption by clinicians and the achievement of its goals and objectives. As
recommended by Horsky et al. (2010) response to clinician emails, online surveys, observations,
and interviews will play a great role in evaluating the ROPSRAT design at different stages. The
assessment will cover the following measures as suggested by Osheroff et al. (2012): system
response time, structure, process, and outcome.
The Retinopathy of Prematurity Risk Assessment Tool
27
System Response Time
As proposed, system response time of the ROPSRAT CDSS should have the least amount
of response time as possible to promote user acceptance. If the EMR does not deliver the
recommendations in a timely way, the ophthalmologists may become frustrated and totally
disregard the recommendations. Therefore, the ROPSRAT should be evaluated based on rapid
response and availability at the point of care when the physicians enter their notes and whether it
adversely affects the efficiency of physician’s workflow; otherwise, the ROPSRAT
recommendations may not be well received by system users.
Structure
It is vital to measure how the ROPSRAT CDSS will be presented and incorporated in the
workflow of the users. The structure will be measured in terms of the nature and number of
alerts. The evaluation of the alerts will be based on whether they are configured and triggered
appropriately in critical stages of the workflow and whether the buttons are friendly and easy to
use. The order set will also be evaluated whether it is relevant and deployed at the right time.
To be specific, the order set needs to come right after or alongside the recommendation alert (see
Figure 3) for the next eye examination schedule. This way, the ophthalmologist will look at the
recommendation first then move on to ordering the next eye examination. It is also imperative to
evaluate that the buttons and the presentation of the tool so as to ensure that the order sets are
relevant and simple to use.
Process
The clinical recommendations table derived from the recent recommendations for the
management of retinopathy of prematurity will be reviewed by the ophthalmologists and the
neonatologists on the project team. Although these guidelines are intended to be straightforward,
The Retinopathy of Prematurity Risk Assessment Tool
28
there is some room for interpretation. It is important that uncertainty and disagreements are
acknowledged. It may be that in certain circumstances there are no clear recommendations. The
decision-support tool must clearly communicate this uncertainty to the user.
Perhaps more challenging will be validation of the implementation. Rules for follow up
and treatment must be entered into the Epic system in order to provide appropriate responses.
Perhaps the best way to validate this is to have the lead clinician on the project team use the tool
for a period of time prior to releasing it to other users. Any discrepancies or concerns must be
noted for review later. An effective way to do this may be by email to a central development
location. If the ophthalmologist does not have the immediate ability to comment on any
potential discrepancies, it may be difficult to remember them later.
Validation of the quantitative estimate of risk may be more problematic. When data had
been collected on only a small number of patients, it is possible and perhaps likely that a few
outliers might throw off the risk calculation dramatically. For example, if a set of triplets with
the same birth age and similar birth weights all go on to need laser surgery, other infants with
similar statistics might have an artificially elevated calculated risk. By periodically performing
the multivariate regression analysis and updating coefficients that will be used to calculate the
risk for each infant, as the data pool increases, the estimates will become increasingly accurate.
It must be stressed that users of the system should be made aware of this difficulty, especially in
the first few years after deployment.
Moreover, the ROPSRAT process should be evaluated in terms of how often the
intervention is used, how the alert is utilized, how users respond to the alert, the adoption rate of
the users, the characteristics of the users in relation to adoption rates, effects of the intervention
in the workflow, and whether the users find the intervention helpful, satisfying, or distracting.
The Retinopathy of Prematurity Risk Assessment Tool
29
Human-technology interface issues should also be identified if encountered as this will affect
workflow efficiency and user acceptance.
Outcome
Another metric that should be measured is the effect of ROPSRAT
implementation in the decreased incidence of blindness and critical ROP or decreased length of
stay among premature infants. With ROPSRAT providing follow-up recommendations in a
timely manner, there will be more consistency and accuracy in terms of follow-up eye exam
recommendations and treatment. This in turn should create a positive visual health outcome
among premature infants affected by ROP.
The rate of consistency among ophthalmologists in terms of following clinical guidelines
should be measured as stated in the objective.
Increased family satisfaction from improved communication of ROP status should be
measured as well. The risk management score will not be helpful in communicating with
families if it is ignored. Its use will depend on how nurses perceive their relevance and clinical
value. It is important to evaluate the nursing staff’s opinion of the tool and to take corrective
actions to address common issues and concerns through adequate training and education.
Employing surveys to help nurses improve their interactions with families would be ideal.
Parental satisfaction with the process may be difficult to quantify. However, surveys or
structured interviews may be done before deployment to establish a baseline. Families may be
satisfied with their babies ROP treatment due to lack of education about their child’s risk.
Connecticut Children’s dual NICU structure may allow one institution to serve as a control group
in assessing satisfaction after the introduction.
The Retinopathy of Prematurity Risk Assessment Tool
30
Adoption rate among users should also be evaluated. Since decision making for
treatment and follow-up recommendations is repeated for each evaluation, the ROPSRAT tool
will help physicians decrease the time spent on decision-making. It is important to evaluate
whether the system provides alerts to intended users at the time of decision making process;
whether it is performing its role as an assistive tool rather than an irritation.
Discussion
The portion of ROPSRAT that provides recommendations for treatment or follow-up can
be implemented as soon as it is sufficiently evaluated. Clearly it will take longer for the
probability estimate to be ready. Several years of data will be needed before quantitative
estimates can be provided, although hopefully qualitative estimates of “low,” “moderate,” and
“high” risk can be generated within six months to a year. It is essential that users of the system
understand its limitations. If not, it is possible that after a few experiences with clearly
meaningless scores, the NICU staff may develop the habit of ignoring them, even as they
become more meaningful.
The limitation of the risk assessment model, at least as it stands now, is that it has not
incorporated all potential risk factors into the calculation of the risk score. A multivariate
regression analysis will show how much of the variability has been explained with the factors
that have been taken into account. If only a relatively small amount of variability is incorporated,
say 40%, then the remaining 60% of the variability has not been incorporated into the model.
Data mining techniques can assist in trying to determine what other risk factors should be
incorporated.
Our society is in the process of transitioning to a new model of healthcare financing.
Insurers are very interested in improving quality outcomes and this tool could provide a dramatic
The Retinopathy of Prematurity Risk Assessment Tool
31
demonstration of efforts to improve quality. Additionally, ophthalmologists, and the hospitals
that employ them, are paid by insurers for every exam they do on premature infants. There is
currently no incentive to limit the number of exams performed, other than that the exams
themselves can be traumatic for the infant. As we move towards more accountable care, the
model and the data it generates could be used to perhaps modify the clinical guidelines. If for
example infants born between 1100 and 1200 g at 28 weeks have never gone on to develop the
need for surgery after several years of data accumulation, and the calculated risk is zero, perhaps
fewer exams could be done. This of course must be a step taken with caution since a mistake
could result in blindness, but the approach should be considered to reduce unnecessary
examinations.
The Retinopathy of Prematurity Risk Assessment Tool
32
References
American Academy of Pediatrics Section on Ophthalmology, American Academy of
Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus &
American Association of Certified Orthoptists, 2013. Screening examination for
premature infants for retinopathy of prematurity. Pediatrics 131:189-195.
patient satisfaction.
Day, S., Menke, A.M., & Abbott, R.L. (2009). Retinopathy of Prematurity malpractice claims.
Archives of Ophthalmology 127(6):794-798.
Early Treatment for Retinopathy of Prematurity Cooperative Group, 2003. Revised indications
for the treatment of retinopathy of prematurity: results of the early treatment for
retinopathy of prematurity randomized trial. Archives of Ophthalmology. 121:1684 –
1696
Facts about retinopathy of prematurity (rop). (n.d.). Retrieved from
http://www.nei.nih.gov/health/rop/rop.asp
Hardin, J.M., Chhieng, D.C. (2007). Data mining and clinical decision support systems. In:
Clinical decision support systems: Theory and practice, Berner, E.S., ed., New York,
NY, USA: Springer, Health Informatics Series.
Hunink, M., Glasziou, P., Siegel, J., Weeks, J., Pliskin, J., Elstein, A., & Weinstein, M. (2001).
Decision-making in health and medicine: Integrating evidence and values. Cambridge,
UK: Cambridge University Press
ICD-10 Data.com, 2014. Retrieved from http://www.icd10data.com/ICD10CM/Codes/H00H59/H30-H36/H35-.
The Retinopathy of Prematurity Risk Assessment Tool
33
International Committee for the Classification of Retinopathy of Prematurity, 2005. The
international classification of retinopathy of prematurity revisited. Archives of
Ophthalmology 123:991-999.
Osheroff, J., Teich, J., Levick, D., Saldana, L., Velasco, F., Sittig, D., Rogers, K., & Jenders, R.
(2012).Improving outcomes with clinical decision support. An implementer's guide. (2nd
ed.).
Stanton, M. (2006, June). The high concentration of U.S. health care expenditures. Retrieved
from http://www.ahrq.gov/research/findings/factsheets/costs/expriach/index.html
WHO. (2007). Vision 2020 the right to sight. Retrieved from
http://www.who.int/blindness/Vision2020_report.pdf
The Retinopathy of Prematurity Risk Assessment Tool
34
Table
Decision Logic
ZONE
1
1
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
2
2
3
3
STAGE
0
0
0
1
1
1
2
2
2
3
3
3
0
0
0
1
1
1
2
2
2
3
3
3
0
0
VESSELS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
PLUS
NO PLUS
PRE-PLUS
RESPONSE
FU 1 WEEK
FU 1 WEEK
TREATMENT
FU 1 WEEK
FU 1 WEEK
TREATMENT
FU 1 WEEK
FU 1 WEEK
TREATMENT
TREATMENT
TREATMENT
TREATMENT
FU 2 WEEKS
FU 1 WEEK
TREATMENT
FU 2 WEEKS
FU 1 WEEK
TREATMENT
FU 1 WEEK
FU 1 WEEK
TREATMENT
FU 1 WEEK
FU 1 WEEK
TREATMENT
FU 2 WEEKS
FU 1 WEEK
3
0
PLUS
TREATMENT
NOTES
CLINICALLY
UNLIKELY
CLINICALLY
UNLIKELY
3
1
NO PLUS
FU 2 WEEKS
3
1
PRE-PLUS
FU 1 WEEK
3
1
PLUS
TREATMENT
3
2
NO PLUS
FU 1 WEEK
3
2
PRE-PLUS
FU 1 WEEK
3
2
PLUS
TREATMENT
3
3
NO PLUS
FU 1 WEEK
3
3
PRE-PLUS
FU 1 WEEK
3
3
PLUS
TREATMENT
RETINA MATURE
FU 6-9 MONTHS
ROP REGRESSED
FU 6-9 MONTHS
Adapted from American Academy of Pediatrics Section on Ophthalmology et al. (2013).
The Retinopathy of Prematurity Risk Assessment Tool
35
Figures
Figure 1
Goals and Objectives
Organizational Goals:
Improve the visual health among
premature infants, and optimize
cost-effectiveness of visual care.
Clinical Goals: 1. Improve the
quality of ROP care and enhance
communcation among
ophthalmologist, NICU staff, and
families.
Develop an accurate
recommendation tool for followup and treatment of ROP that is
timely integrated in clinician's
workflow utilizing the most
current guidelines set by AAP.
Develop a tool to quantify the risk
of ROP progression for utilization
by ophthalmologist and NICU
staff.
Develop a tool that allows
quantitative risk comparison
across NICUs .
Develop a tool that allows
quantitative comparison of
ophthalmologist’s performance or
practice .
The Retinopathy of Prematurity Risk Assessment Tool
Figure 2
Pop-up screen showing recommended follow up.
36
The Retinopathy of Prematurity Risk Assessment Tool
37
Figure 3
Sample letter to families of infants with stage 2/zone 2/no plus ROP and 25% calculated risk
of progression to needing laser surgery.
To the parents of <INFANT NAME>:
<INFANT NAME>’s eyes were examined today for a condition called retinopathy of
prematurity. This is a disease that affects the eyes of premature infants and can cause complete
and permanent blindness. Fortunately, a good treatment is available for this condition that can
prevent blindness in most cases. But it is critical that the disease is identified in a timely way.
Your baby has a <MODERATE> risk of progression to needing laser surgery.
The eye is a very complex organ. It involves multiple parts that need to work together to achieve
optimal vision. The tissue inside the eye that senses light is called the retina. It requires a good
blood vessel supply in order to function. When your premature infant was born, <HIS (HER)>
retinal blood vessels were not fully complete. The premature birth has caused the blood vessels
to develop inappropriately. Instead of growing along the retina, they are growing into the center
of the eye. If this becomes severe enough, the blood vessels can tear the retina off the back of the
eye causing a complete and permanent retinal detachment and complete blindness.
Right now, your baby has a form of the disease that is not severe enough to cause blindness, but
it could progress to that point. Based on our experience at Connecticut Children’s Medical
Center, we estimate that your baby has a <25%> chance of progression to that point.
<HIS (HER)> doctors and nurses are aware of this risk and are taking precautions to make
certain that this risk is minimized. However, it is critical that <HE (SHE)> be examined again in
one week. Dr. <OPHTHALMOLOGIST> will round next week and will provide you with an
update after that exam. If <INFANT NAME> is discharged prior to next week’s scheduled
rounds, it is critical that you keep the follow-up visit arranged in the outpatient retinopathy of
prematurity follow-up clinic. Missing even one visit could result in permanent blindness.
Your ophthalmologist, Dr. <OPHTHALMOLOGIST>, is available if you would have additional
questions. Please call at 860 – XXX – XXXX should you wish to discuss this further.
The Retinopathy of Prematurity Risk Assessment Tool
38
Figure 4
Stakeholder Workflow for the Follow-up Recommendation Tool and Order set
NICU Nurse
Inputs patient
data in EHR
Neonatologist
Receives alert for eye
examination
recommendation.
Or
Receives risk score
calculation
Ophthalmologist
Receives list of patients
to be seen for eye
examination
Examines patient
Receives and carries out
orders for eye exam
schedule
Receives
order on
scheduled
eye
examination
from
ophthalmol
ogist and
fluid and
nutrition
order from
neonatologi
st
Order Set
(for first eye
examination and
fluid and nutrition
orders)
Enters medical
notes
Receives alert on eye
examination schedule and/or
treatment recommendation
Receives alert on
scheduled eye
examination made
by ophthalmologist
Order Set
(for next eye
examination)
Order Set for
fluid and
nutrition