Translating Research Into
Policy & Practice
Call for Papers
New Approaches to Translating Research into
Policy & Practice
Chair: Linda Bilheimer, The Robert Wood Johnson
Foundation
Monday, June 27 • 9:00 am – 10:30 am
●A Review of Knowledge Transfer Models, Frameworks
and Theories
Ian Graham, Ph.D., Jo Logan, RN, Ph.D., Jacqueline Tetroe,
MA, Nicole Robinson, The KT Theories Research Group
Presented By: Ian Graham, Ph.D., Senior Social Scientist,
Associate Director, Clinical Epidemiology Program, Ottawa
Health Research Institute, 1053 Carling Avenue, ASB 2-008,
Ottawa, NY K1Y 4E9; Tel: (613)798-5555 x18273; Fax: (613)7615402; Email: igraham@ohri.ca
Research Objective: 1) To conduct a focused search for
conceptual models, frameworks, or grand theories of
knowledge translation, 2) To undertake a theory analysis of the
identified models to determine their strengths and limitations
and to determine similarities and differences among them,
and 3) To determine the extent to which each model has been
used and/or tested.
Study Design: 1) A focused literature search of the social
science, education, management, and health sciences
literature, the internet, and hand searching of the journals
Science Communication (formerly the journal Knowledge:
Creation, Diffusion, Utilization), and Knowledge, Technology
and Policy (formerly Knowledge and Policy: the International
Journal of Knowledge Transfer and Utilization). 2) A theory
analysis of the identified models/frameworks to determine
their strengths, limitations, similarities and differences. 3) A
bibliometric analysis using the science and social science
citation journals to identify additional literature which might
report on the use and/or testing of the models/theories.
Population Studied: Conceptual models, frameworks, or
grand theories of knowledge translation.
Principal Findings: To date a total of 175 articles have been
retrieved. They were classified as being either a theory, a
framework or a model (the authors’ terminology was used for
the classification – i.e. if the title used the term “framework”, it
was classified that way). There were 41 theories, 63 models
and 58 frameworks. In addition, there were 13 syntheses. The
articles ranged in date of publication from 1903 to 2004. They
were written by researchers representing a range of
disciplines, including Nursing, Behavioural Sciences, Clinical
Epidemiology, Geography & Geology, Education, Business,
Law and Computer Sciences. Relatively few of the
models/theories/frameworks have been evaluated. A set of
common and unique components of the various
conceptualizations of KT will be presented.
Conclusions: The session will inform participants about
conceptual models, frameworks and grand theories in the field
of knowledge translation.
Implications for Policy, Delivery, or Practice: The
information provided will have the potential to increase
understanding of research utilization and may be useful to
guide best practice implementation endeavors. The findings
may also provide foundational information upon which new
theories or frameworks of knowledge translation may be
derived.
Primary Funding Source: Canadian Institutes of Health
Research
●Towards Systematic Reviews that Inform Healthcare
Management and Policymaking
John N. Lavis, M.D., Ph.D.
Presented By: John N. Lavis, M.D., Ph.D., Associate Professor
and Canada Research Chair in Knowledge Transfer and
Uptake, Clinical Epidemiology and Biostatistics, McMaster
University, 1200 Main Street West, Hamilton, L8N 3Z5; Tel:
(905) 525-9140 x22907; Fax: (905) 529-5742; Email:
lavisj@mcmaster.ca
Research Objective: To inform discussions about how to
improve the usefulness of systematic reviews for healthcare
managers and policymakers, we asked five questions: 1) what
is the nature of decision-making and approach to research
evidence in healthcare management and policymaking? 2)
what types of questions are asked in healthcare management
and policymaking that could be informed by systematic
reviews? 3) how is research evidence assessed before it is
used to inform healthcare management and policymaking? 4)
how much value is placed on researchers providing
recommendations about a preferred course of action for
healthcare managers and policymakers and on using language
that is locally applicable? and 5) what is the optimal way to
present research evidence for use in healthcare management
and policymaking?
Study Design: We reviewed studies of decision-making by
healthcare managers and policymakers (phase 1), conducted
our own interviews with healthcare managers and
policymakers (phase 2), and reviewed websites that include
healthcare managers and policymakers among their target
audiences (phase 3). We built our systematic review on a
completed review (thereby preserving its rigour in the
identification and appraisal of studies) but we refined the
selection criteria and the approach to data extraction and we
analyzed the findings separately for healthcare managers and
healthcare policymakers. We conducted semi-structured
interviews with 10 healthcare managers and 19 healthcare
policymakers in Canada and the United Kingdom to explore
their experiences with acquiring information to inform their
decisions and their perspectives on how researchers can
better produce and adapt systematic reviews to inform
healthcare management and policymaking. We reviewed the
websites of 14 research funders, 14 producers/purveyors of
research, and 17 journals to explore their approaches to
reporting systematic reviews and whether and how their
approaches matched with the suggestions arising from our
systematic review and interviews.
Population Studied: We conducted semi-structured
interviews with a purposive sample of healthcare managers
(or the senior staff of associations that seek to inform
managers) in Ontario and England and healthcare
policymakers in the Canadian federal and Ontario provincial
governments and the United Kingdom government to explore
their experiences with acquiring information to inform their
decisions and their perspectives on how researchers can
better produce and adapt systematic reviews to inform
healthcare management and policymaking. Our sample frame
was defined both by jurisdiction and role and we identified
potential study participants through key informants and
government websites. We interviewed 10 healthcare managers
and 19 healthcare policymakers in Canada and the United
Kingdom. The number of study participants counted in each
role category is an underestimate, however, because many
individuals had experience with and spoke about more than
one role. Study participants were almost always drawn from
the top ranks of their respective organizations (in the case of
healthcare managers), department (in the case of civil
servants) or office (in the case of political advisors).
Principal Findings: Our systematic review demonstrated that
the research evidence about decision-making by healthcare
managers and policymakers is not that plentiful, rigorous (in
the sense of using more than one data-collection method and
adequately describing the sampling and measurement
methods that were used) or consistent (in the sense of similar
factors emerging across a number of contexts). The most
rigorously demonstrated and consistent factors to emerge
from the studies all pertained to healthcare policymakers: 1)
individual-level interactions between researchers and
healthcare policymakers increased the prospects for research
use; 2) timing and timeliness increased (and poor timing or
lack of timeliness decreased) the prospects for research use;
3) individuals’ negative attitudes towards research evidence
decreased the prospects for research; and 4) individuals’ lack
of skills and expertise decreased the prospects for research
use. Our interviews with healthcare managers and
policymakers suggest that: 1) most do not highly value
systematic reviews as an information source; 2) nevertheless
many have used systematic reviews to address many different
types of questions; 3) some identified that they would benefit
from having the attributes of the context in which the research
was conducted highlighted in order to inform assessments of
a review’s local applicability; 4) all would value information
about the benefits, harms (or risks), and costs of
interventions, the uncertainty associated with estimates, and
variation in estimates by subgroup in order to inform
programming decisions; 5) they disagree about whether
researchers should provide recommendations; 6) almost all
would value reports presented using something like a 1:3:25
format; and 7 ) some identified that they would value
systematic reviews being made more readily available for
retrieval when they are needed. Our analysis of websites found
that: 1) attributes of the context in which the research was
conducted were rarely provided; 2) recommendations were
often provided; and 3) reports using a 1:3:25 format were rare.
Conclusions: The exploratory nature of our research, and the
general lack of research evidence against which we can
compare our findings, mean that our interpretations and the
guidance that we offer are provisional. Our provisional
answers to question 1 lead us to argue for thinking broadly
about healthcare managers and policymakers as target
audiences, demonstrating to them the value of systematic
reviews, engaging them in the production and adaptation of
systematic reviews, and building their capacity to identify
quality-appraised sources of systematic reviews and to
appraise their local applicability. Our provisional answers to
question 2 lead us to argue for the production of systematic
reviews that address a broad array of questions. Our
provisional answers to question 3 lead us to argue for making
available an online source of all types of quality-appraised
systematic reviews, identifying the benefits, harms (or risks)
and costs of interventions (not just benefits), highlighting the
uncertainty associated with estimates, describing any
differential effects by sub-group (e.g., ethnocultural group),
and identifying attributes of the context in which the research
included in a systematic review was conducted to inform
assessments of the applicability of the review in other
contexts. Our provisional answers to question 4 lead us to
argue for not providing recommendations in systematic
reviews and avoiding the use of jargon in reports about
systematic reviews. Our provisional answers to question 5 lead
us to argue that more user-friendly “front ends” should be
developed for potentially relevant systematic reviews (e.g., one
page of take-home messages and a three-page executive
summary) to facilitate rapid assessments of the relevance of a
review and, when the review is deemed highly relevant, more
graded entry into the full details of the review.
Implications for Policy, Delivery, or Practice: Researchers
could help to ensure that the future flow of systematic reviews
will better inform healthcare management and policymaking
by making three changes to how they produce and update
systematic reviews: augment the stock of investigator-driven
systematic reviews with reviews that involve healthcare
managers and policymakers in posing questions, reviewing
the proposed approach, and interpreting the results; for
systematic reviews about “what works,” identify the benefits
and harms (or risks) of interventions (not just benefits),
highlight the uncertainty associated with estimates, and
describe any differential effects by sub-group (e.g.,
ethnocultural group); and identify attributes of the context in
which the research included in a systematic review was
conducted to inform assessments of the applicability of the
review in other contexts; Research funders could help to
ensure that the global stock of systematic reviews will better
inform healthcare management and policymaking by
supporting three types of local adaptation processes led jointly
by researchers and healthcare managers and policymakers:
develop a more user-friendly “front end” for potentially
relevant systematic reviews (e.g., one page of take-home
messages and a three-page executive summary) to facilitate
rapid assessments of the relevance of a review and, when the
review is deemed highly relevant, more graded entry into the
full details of the review; add additional local value to
systematic reviews about “what works” by describing the
benefits, harms (or risks) and costs that can be reasonably
expected locally and to any type of systematic review by using
language that is locally applicable; and make user-friendly
“front ends” of systematic reviews available through an online
database that can be linked to the full reviews through other
sources, such as The Cochrane Library.
Primary Funding Source: Canadian Health Services Research
Foundation / National Health Service (NHS) Service and
Delivery Organization R&D Programme
●Synthesizing Quality Improvement Research:
Methodological and Empirical Challenges and Solutions
Brian Mittman, Ph.D., Marjorie Pearson, Ph.D., Patricia
Parkerton, Ph.D., Nancy Takahashi, MPH, Nina Smith, MPH
Presented By: Brian Mittman, Ph.D., Senior Social Scientist,
VA Center for the Study of Healthcare Provider Behavior, VA
Greater Los Angeles Healthcare System, 16111 Plummer
Street, mailcode 152, Sepulveda, CA 91343; Tel: (818)895-9544;
Fax: (818)895-5838; Email: Brian.Mittman@med.va.gov
Research Objective: Research synthesis methods such as
meta-analysis and meta-regression have produced valuable
evidence in the clinical sciences, leading to the development
of evidence-based practice recommendations to facilitate
improvements in clinical practices and outcomes. Metaanalysis represents an important tool for combining and
reconciling seemingly inconsistent findings from individual
clinical trials, as well as an effective mechanism to obtain
value from small studies that offer little statistical power and
thus limited value individually. The application of research
synthesis methods to empirical studies of programs to
improve healthcare quality is challenging, however, due to
heterogeneity in intervention designs, research settings, and
methods; and lack consistency and comparability in study
measures and data. To address these challenges, variants and
extensions of prevailing research synthesis tools have been
developed, capturing qualitative data and accommodating the
heterogeneity and other unique features of social and
behavioral research. This presentation reviews the major
challenges in effective application of research synthesis to QI
evaluations, describes the newer synthesis methods applicable
to QI studies and findings, and illustrates the application and
benefits of these methods.
Study Design: Literature review of research synthesis
methods and application of innovative synthesis tools.
Population Studied: The synthesis sample includes 54
studies of programs to improve healthcare quality conducted
within the Veterans Health Administration (VHA) during
2001-2004. Each study addressed at least one of the eight
chronic diseases targeted by VHA’s Quality Enhancement
Research Initiative (QUERI) centers.
Principal Findings: The application of prevailing synthesis
methods to the VHA sample results in incomplete evidence
tables containing only a subset of the factors hypothesized to
influence QI program effectiveness, and few of the critical
details of the improvement program interventions and
settings. Expanding the synthesis effort to include a broader
range of data sources and synthesis methods generates a
more complete set of evidence tables and findings. For
example, published data must be supplemented by reviews of
key internal study documents (protocols, timelines, memos,
progress reports) and in-depth interviews with study staff.
The toolbox of synthesis methods is expanded from metaanalysis and meta-regression to include meta-synthesis of
qualitative data and case study-based approaches. The latter
involve generation and testing of study-specific hypotheses
and development of new conclusions for each study
individually, followed by quantitative and qualitative synthesis
of these conclusions across the entire sample.
Conclusions: The nature of quality improvement problems
and solutions requires a diverse research approach combining
“gold standard” clinical trial methods, program evaluation and
qualitative research ranging from interpretive, inductive efforts
to more conventional deductive strategies. The synthesis of
findings and insights from these studies requires equivalent
breadth and flexibility in synthesis methods and in their ability
to handle diverse types of data and evidence.
Implications for Policy, Delivery, or Practice: Limitations in
the strength and utility of healthcare QI research findings
represent an important barrier to the achievement of critical
quality improvement goals. Enhanced methods for
synthesizing QI research findings offer the possibility of
accelerated development of a valid, useful QI evidence base
and more effective applications of QI research results to
achieve required improvements.
Primary Funding Source: VA
●Realist Synthesis: An Approach to Synthesising Research
Evidence on Complex Social Interventions for
Policymakers and Managers
Kieran Walshe, BSc(Hons), DipHSM, Ph.D., Dr Ray Pawson,
Professor Trish Greenhalgh, Dr Gill Harvey
Presented By: Kieran Walshe, BSc(Hons), DipHSM, Ph.D.,
Professor of Health Policy and Management, Centre for Public
Policy and Management, Manchester Business School,
University of Manchester, Booth Street West, Manchester,
M15 6PB; Tel: 0(161)275 3852; Fax: 0(161) 273 5245; Email:
kieran.walshe@man.ac.uk
Research Objective: This paper describes a new model of
research synthesis which is designed to work with complex
social interventions or programmes, and which is based on
the emerging “realist” approach to evaluation. Realist review
is a relatively new strategy for synthesising research, which has
an explanatory rather than judgemental focus. It seeks to
‘unpack the mechanism’ of how complex programmes work
(or why they fail) in particular contexts and settings.
Compared to clinical treatments, which are conceptually
simple and have generally been evaluated in randomised
controlled trials, the literature on healthcare management and
policy-level interventions is epistemologically complex and
methodologically diverse, which we argue makes it highly
suited to realist review.
Study Design: This paper does not report an empirical study,
but the theoretical development of the realist review
methodology.
Principal Findings: The quest to understand ‘what works?’ in
social interventions involves trying to establish causal
relationships. The hallmark of realist inquiry is its distinctive
generative (rather than successionist) understanding of
causality. Under realism, the basic evaluative question – ‘what
works?’ – changes to ‘what is it about this programme that
works for whom in what circumstances?’. Because realist
review is especially appropriate for highly complex
interventions, we deliberate use complex examples in our
paper such as a published review on the public disclosure of
performance data (Marshall et al, 2000). Because of the
predominant position of the conventional Cochrane style
systematic review in health services research, we offer a
comparison of the two approaches in terms of both
theoretical assumptions and practical methods.
Conclusions: Realist reviews do not provide simple answers
to complex questions. They will not tell policymakers or
managers whether something “works” or not, but they will
provide the policy and practice community with the kind of
rich, detailed and highly practical understanding of complex
social interventions which is likely to be of much more use to
them when planning and implementing programmes at a
national, regional or organisational level.
Implications for Policy, Delivery, or Practice:
Commissioners of research reviews to inform policy and
management decisionmaking should be increasingly involved
in the production of the research synthesis, and reviewers
should increasingly bring their technical expertise closer to the
policy question by ensuring that their research takes account
of the needs of a range of stakeholders. This healthy two-way
dialogue is what Lomas has called ‘linkage’ (Lomas, 2000).
Realist review raises the status of linkage from a
recommendation to a methodological requirement. We argue
that the tasks of identifying the review question and
articulating key theories to be explored cannot meaningfully
occur in the absence of input from practitioners and
policymakers, because it is their questions and their
assumptions about how the world works that form the focus
of analysis. Furthermore, the ‘findings’ of a realist review
must be expressed not as universal scientific truths but in the
contextualised grammar of policy discourse. Key references:
Pawson R, Greenhalgh T, Harvey G, Walshe K (2005). Using
realist methods to produce syntheses of evidence for use by
managers and policy makers. Journal of Health Services
Research and Policy, submitted. Pawson R, Greenhalgh T,
Harvey G, Walshe K. Realist synthesis: an introduction.
Manchester: ESRC Research Methods Programme, 2004.
Walshe K, Rundall T. Evidence based management: from
theory to practice in healthcare. Milbank Quarterly, 2001;
79(3):429-457.
Primary Funding Source: Canadian Health Services Research
Foundation and NHS Service Delivery and Organisation
research programme
●Applying Knowledge Transfer and Exchange Strategies to
Promote Integrated Stroke Care
Grace Warner, Ph.D., Stephen Phillips, M.D., Renee Lyons,
Ph.D.
Presented By: Grace Warner, Ph.D., Adjunct Faculty, Atlantic
Health Promotion Research Centre, Dalhousie University,
Suite 209, 1535 Dresden Row, Halifax, NS, B3J 3T1; Tel:
(902)494-2604; Fax: (902)494-3594; Email:
grace.warner@dal.ca
Research Objective: The Atlantic Canada Integrated Stroke
Strategy (ACISS) research project is a partnership between
researchers, non-governmental agencies, and health care
decision-makers funded through the Canadian Stroke
Network. The ACISS project is evaluating how knowledge
transfer and exchange (KT) strategies, such as using a
knowledge broker, improve the exchange of best-practices
research and policy implementation criteria between
researchers and users of research in the Atlantic Provinces.
This is one of the first research projects to evaluate the efficacy
of the knowledge broker position.
Study Design: This longitudinal study evaluates the efficacy of
KT strategies over two years. The project assesses strategies
using theoretical models from KT, diffusion of innovations,
and quality improvement initiatives with both qualitative and
quantitative methods. Key informant interviews assess: the
needs and expectations of the partners, the process of
creating acceptance for the knowledge broker position, the
ability of the knowledge broker to meet partner expectations,
and ways to improve the process of KT. In addition, key
environmental drivers for health system change around stroke
care are assessed using both qualitative and quantitative
methods.
Population Studied: Representatives from non-governmental
agencies and health care decision-makers in the Atlantic
Provinces
Principal Findings: Initial partnering was challenging.
Frequent personal communication, bi-annual forums, and
salary support were used to facilitate the hiring of a knowledge
broker, which took from 9-14 months in the different
provinces. Key issues which affected this hiring process were
political and in-kind support, confidentiality, prior biases, and
outcome expectations. After the knowledge broker was hired
different partner expectations were identified that the
knowledge broker had to address when communicating
between partners. For example, the drive to move forward
from both the non-governmental and research partners
contrasted with the cautionary speed of the government which
made communications contentious. Although the knowledge
broker was appreciated by all the partners an acceptable level
of trust needed to be established before communications
could be effective. Knowledge translation strategies, and
especially knowledge brokers, have improved the
communications and interactions between partners. Key KT
deficiencies have been identified in the implementation of a
complex integrated strategy, such as education of best
practices research to administrative and health care
practitioners who are affected by the strategy, the need to
create a health care system that allows for best practices to
evolve as research progresses, and improvements in
communication channels between different sectors of
government.
Conclusions: Limited resources, the changing political
climate, boundaries between health care decision makers and
non-governmental advocates, and accountability to
bureaucratic procedures pose strong barriers to acquiring and
adapting new research evidence. Communication forums
combined with a knowledge broker facilitates the
implementation of evidence-based integrated care systems.
Implications for Policy, Delivery, or Practice: The
knowledge broker can be a good mediator for the push for
system change and the reality of implementing those changes.
It remains to be seen how a knowledge broker can be used
and financed through out the health care system so we can
have an evidence-based health care system.
Primary Funding Source: Canadian Stroke Network
Call for Papers
Funders’ & Researchers’ Efforts to Translate Research into
Policy & Practice
Chair: John Lavis, McMaster University
Tuesday, June 28 • 10:30 am – 12:00 pm
●Translating Research to Practice: Insights from
Experience with AHRQ's IDSRN Initiative
Marsha Gold, Sc.D., Erin Fries Taylor, Ph.D.
Presented By: Marsha Gold, Sc.D., Senior Fellow, ,
Mathematica Policy Research Inc., 600 Maryland Avenue SW,
Suite 550, Washington, DC 20024; Tel: (202)484-4227; Fax:
(202)863-1763; Email: MGold@Mathematica-MPR.com
Research Objective: To promote better understanding of
what it means to "translate research to practice" by exploring
lessons from the Agency for Healthcare Research and
Quality's Integrated Delivery Systems Research Network
(IDSRN) program. Begun in 1999, the IDSRN is one of
AHRQ's earliest attempts to foster public-private collaboration
between health services researchers and operational delivery
systems. IDSRN's goals have increasingly focused on projects
designed to support the translation of research to practice.
Study Design: Independent program evaluation with analysis
based on largely qualitative methods including: initial program
review, descriptive analysis of characteristics of funded
projects, structured interviews with AHRQ leadership and with
staff associated with the IDSRN partners and their
collaborators, and case studies of four successful projects.
Population Studied: Experience in 58 projects funded for
$14.2 million over 4 years (FY2000-FY2003) to 9 teams of
researchers embedded in (5) or collaborating with (4) one or
more collaborating managed care plans and delivery systems.
Included interviews with 26 AHRQ staff and 65 individuals in
IDSRN teams and their affiliated systems.
Principal Findings: IDSRN supported a diverse variety of
types of work that was subsumed under the rubric of
"translating research to practice", some challenging traditional
boundaries of health services research. Projects viewed as
successful share a tendency to be responsive to user needs
and operational interests, to benefit from ongoing support
over multiple projects, and to involve work that is
generalizable across multiple settings.
Conclusions: Tying research to systems can help make that
work more relevant and enhance AHRQ's links with those
outside the research community. But effective translation also
is challenging and even those actively engaged in translation
view it differently and disagree on how it best can be carried
out. Localized adoption does not translate to other settings
automatically and research in practice is not the same as
linking research to practices and taking full advantage of
accumulated findings. Successful translation requires moving
beyond funding individual projects to develop translation
strategies and priorities for defined types of end users.
Generating opportunities for translation also often requires an
up front investment of resources.
Implications for Policy, Delivery, or Practice: Though the
value of health services research may be judged on its capacity
for practical use in a way that has not occurred previously,
there is little consensus even among researchers involved in
IDSRN on the concept and variable buy-in. With resources
scarce an open debate on what translation should mean, who
it seeks to reach, and how it should be supported (including
what share of research dollars should go to it) is crucial. For
that, we need to learn as much as we can from existing
experience with translation in all its forms.
Primary Funding Source: AHRQ
●How and to Whom are Canadian Health Researchers
Promoting the Uptake of their Findings? Part 1: The
Quantitative Piece
Ian Graham, Ph.D., Jacqueline Tetroe, MA, Jeremy Grimshaw,
M.D., Ph.D., Nicole Robinson, Paul Hébert, M.D., MSc
Presented By: Ian Graham, Ph.D., Senior Social Scientist,
Associate Director, Clinical Epidemiology Program, Ottawa
Health Research Institute, 1053 Carling Avenue, ASB 2-008,
Ottawa, K1Y 4E9; Tel: (613)798-5555 x18273; Fax: (613)7615402; Email: igraham@ohri.ca
Research Objective: 1) Determine what knowledge
translation activities Canadian health researchers use to
promote their findings and the extent to which researchers
engage in these activities. 2) Identify gaps in target audiences
whom researchers report need to know about their findings
but do not.
Study Design: A cross-sectional self-administered mail survey
of 368 Canadian health researchers
Population Studied: Applied health researchers who were
funded by the Canadian Institutes of Health Research, the
former Medical Research Council of Canada, the former
National Health Research and Development Program of
Health Canada, and the Canadian Health Services Research
Foundation between 1995 and 2001.
Principal Findings: There was a 73% (229/312) response rate.
75% of the respondents reported that the study findings were
quite or very useful to decision makers. The top four diffusion,
dissemination, implementation activities included: conference
presentation (86%), publication (80%), final report to funder
(53%), non-peer reviewed publication (36%); summary to
policy makers (25%), summary to practitioners (21%), tools
creation (21%), press releases (18%); educational sessions
with practitioners (43%), stakeholder involvement (40%),
media involvement (23%), educational sessions with policy
makers (19%). The largest gaps between needing to know
about and actually being aware of study findings were for
federal/provincial policy makers (gap=38%),
patients/consumers (gap=35%), health care professional
bodies (gap=35%), and health care managers (gap=29%). The
smallest gap was for study stakeholders (7%). Researchers’
confidence in their ability to engage in KT activities varied:
43% (very or quite confident) with community/municipal
organizations, media (45%), federal/provincial policy makers
(48%), managers (53%), patients/consumers (54%), to health
care practitioners (79%), study stakeholders (83%),
researchers/academics (94%). Findings often differed
depending on the specific agency funding the study.
Conclusions: The researchers in this study engaged in KT
activities to varying degrees and the majority tended not to
focus their activities on policy makers. They acknowledged
that their findings were relevant to policy makers, health care
managers, and health care professional bodies, yet for over a
third of the studies, these audiences were not aware of the
findings. Furthermore, researchers were also least confident in
their abilities to engage in KT activities with these audiences.
Implications for Policy, Delivery, or Practice: Researchers
need help to push findings to policy makers, or alternatively,
to create a pull for these findings. They need the skills,
experience and confidence to be able to interact productively
with these audiences. The nature of the
expectations/requirements that a given funding agency has of
their funded researchers would appear to influence the type
and successfulness of the KT activities engaged in by the
researchers.
Primary Funding Source: The Canadian Institutes of Health
Research
●How and to whom are Canadian Health Researchers
Promoting the Uptake of their Findings? Part 2: the
Qualitative Piece
Ian Graham, Ph.D., Jacqueline Tetroe, MA, Jeremy Grimshaw,
M.D., Ph.D., Nicole Robinson, Paul Hébert, M.D., MSc
Presented By: Ian Graham, Ph.D., Senior Social Scientist,
Associate Director, Clinical Epidemiology Program, Ottawa
Health Research Institute, 1053 Carling Avenue, ASB 2-008,
Ottawa, K1Y 4E9; Tel: (613) 98-5555 x18273; Fax: (613)7615402; Email: igraham@ohri.ca
Research Objective: To identify barriers and facilitators to
researchers’ engagement in knowledge translation activities.
Study Design: A cross-sectional self-administered mail survey
of 368 Canadian health researchers. Semi structured
telephone interviews were conducted with a purposive sample
of 13 respondents.
Population Studied: Applied health researchers who were
funded by the Canadian Institutes of Health Research, the
former Medical Research Council of Canada, the former
National Health Research and Development Program of
Health Canada, and the Canadian Health Services Research
Foundation between 1995 and 2001.
Principal Findings: Respondents identified a number of
barriers to engaging in appropriate KT activities, including:
lack of resources – such as time, funding, training, staff and
expertise in KT; lack of ability and confidence – KT is a new
requirement, it is time consuming and challenging; no reward
or support system within universities, nor with policy makers
– evidence is not perceived to be important to them; political
and/or media backlash – policy makers may have to be
strategic with their use of evidence and media quite often
distort it; there is so much turn over in the decision making
world and the process of research is so long that research
results can lose their relevance before they can be
implemented; and finally, some study findings are not “ready”
for KT – for a variety of reasons. Many respondents also
identified facilitators to engaging in KT activities: having the
financial or administrative support of a funding agency that
values KT and has a KT infrastructure; previous experience
with KT and a having a team member who is committed to
this activity; the current research climate puts KT in a positive
light – hence it is valued more; having stakeholders involved
in the research project from its inception results in a ready
audience and relevant results; researching a “sexy” topic that
the media/politicians want to exploit; and finally, having an
established relationship with decision makers.
Conclusions: Many of the respondents recognize the
importance of actively trying to disseminate the results of their
research and many are successfully doing so. However, many
feel a lack of support for and recognition of these efforts from
academic institutions – which largely reward publication in
peer-reviewed journals and not the time spent in relationship
building necessary for relevant applied research. More junior
researchers could benefit from mentoring to build the skill set
and contact list necessary for effective dissemination through
partnerships.
Implications for Policy, Delivery, or Practice: Funding
agencies could consider providing capacity enhancing
opportunities for researchers to hone their dissemination
skills and to link them with relevant stakeholders. Academic
institutions could consider acknowledging the contribution of
research dissemination activities when making promotion and
tenure decisions.
Primary Funding Source: The Canadian Institutes of Health
Research
●An International Study of Health Research Funding
Agencies’ Support and Promotion of Knowledge
Translation. Part 1: The Study
Ian Graham, Ph.D., Jacqueline Tetroe, MA, Nicole Robinson,
Jeremy Grimshaw, M.D., Ph.D.
Presented By: Jacqueline Tetroe, MA, Research Program
Manager, Clinical Epidemiology Program, Ottawa Health
Research Institute, 1053 Carling Avenue, ASB 2-007, Ottawa,
K1Y 4E9; Tel: (613) 798-5555 x19288; Fax: (613) 761-5402; Email:
jtetroe@ohri.ca
Research Objective: To increase our understanding of the
knowledge transfer policies and activities of applied health
research funding bodies within Canada and internationally by
examining the perceived role of funding agencies in
promoting the results of the research they fund.
Study Design: This was a qualitative study in which semistructured interviews were conducted with a sample of key
informants from applied health funding agencies identified by
the collaborators. These were supplemented with information
from the agency websites.
Population Studied: Up to three representatives were
interviewed in each of 30 applied health funding agencies in
Canada, the United States, Scandinavia, the United Kingdom,
France, the Netherlands and Australia.
Principal Findings: Two thirds of the agencies had explicit or
implicit reference to KT in their mission/mandate. All
agencies engaged in some sort of KT activity. The majority of
agencies consider KT as a shared responsibility between the
agency and the researchers. Within the 30 agencies studied,
27 different terms used for KT were identified in the
transcripts, many of which were not defined. The operational
definitions of KT varied between agencies. The transcripts
were analyzed in terms of two major categories: agency
requirements of the researcher and agency initiatives that were
broader than an individual funded study. We examined agency
requirements of the researcher at the time of application, at
the end of the study and in terms of allowable expenses for
dissemination-related activities. Overall, there appeared to be
a more systematic approach to the expectations of researchers
than in the other broad category: initiatives taken by the
agency. No single activity was practised by all of the agencies.
We divided the agency initiated activities into the following
broad categories: tools/techniques, services, linkage and
funding. The most common activities/strategies were the
creation of audience tailored web pages, consulting
stakeholders in setting the research agenda, funding
workshops, creating linkage and exchange opportunities with
various parties (such as decision makers, the public, health
managers, etc) and helping researchers with plain language
writing and communicating with the media.
Conclusions: This would appear to be a time of rapid
evolution for agencies in terms of their approaches to and
means of increasing the uptake and dissemination of the
results of the research they fund. A wide range of knowledge
transfer activities were reported across the 30 agencies, with
large variation in terms of creating a pull for research findings;
engaging in linkage and exchange between agencies,
researchers and decision makers; and pushing results to
various audiences. Evaluation of these activities is challenging
and most agencies are either not ready to embark on a formal
evaluation or are doing so on a very preliminary and/or
exploratory basis.
Implications for Policy, Delivery, or Practice: Little is known
about the effectiveness of the range of KT strategies reported
by the agencies in our study. This is an area that is seen as
being very important to funding agencies because of the gap
between research and practice and because of pressures they
feel to be accountable to their funding sources. Agencies may
benefit from an opportunity to examine what other agencies
are doing in this important area.
Primary Funding Source: The Canadian Institutes of Health
Research
●An International Study of Health Research Funding
Agencies’ Support and Promotion of Knowledge
Translation. Part 2: The Workshop
Ian Graham, Ph.D., Jacqueline Tetroe, MA, Nicole Robinson,
Jeremy Grimshaw, M.D., Ph.D.
Presented By: Jacqueline Tetroe, MA, Research Program
Manager, Clinical Epidemiology Program, Ottawa Health
Research Institute, 1053 Carling Avenue, ASB 2-007, Ottawa,
K1Y 4E9; Tel: (613)798-5555 x19288; Fax: (613)761-5402; Email:
jtetroe@ohri.ca
Research Objective: We undertook a study of international
health research funding agencies’ support and promotion of
knowledge translation (KT) (abstract for this submitted
separately). After compiling the results of the study we held a
workshop with selected representatives of these agencies to
elicit their interpretation of our findings and to discuss the
various mechanisms that can be put into play to facilitate KT.
Study Design: Agency representatives were invited to attend a
one day workshop in which the investigator team presented
the results of our qualitative study, based on semi-structured
interviews and information from agency documents and
websites, of 30 health research funding agencies in Canada,
the United States, the United Kingdom, France, the
Netherlands, Australia and Scandinavia.
Population Studied: Representatives from the following
agencies attended the workshop: CHSRF, CIHR, AHFMR,
FRSQ, NCIC, AHRQ, NIH-NCI, RWJF, California HealthCare
Foundation, Center for Evidence Based Policy, and CDC. Also
in attendance were members of the investigator team and
Réjean Landry, a Canadian expert in knowledge utilization.
Principal Findings: Funding agencies discussed the need to
set minimum expectations of researchers to encourage KT
and to ensure that there is a scientific record of a project.
Some of the suggestions included preparation of a structured
final report; identification of target audiences for KT activities;
and stakeholder involvement where appropriate. There was
less agreement on the specific roles of funding agencies, but
suggestions included linking with policy makers, in-house
synthesis of funded research and creation of a research
register. Agencies agreed that a systematic approach to KT is
required. The specific activities may vary depending on the
goal of the agency, the range of programs, the desired
outcomes, and how the outcomes might be met. The
workshop participants agreed that funders need to agree on a
common set of terms and operational definitions; that it is
important to fund the science of KT; that there is a need for a
set of criteria for deciding what to disseminate, how and to
whom and that these would be linked to the knowledge
synthesis capability of the agency; that agencies can take on a
training role through various mechanisms; that agencies can
have a pivotal linking role between researchers and research
users; and that evaluation of their KT programs is a new area
that many agencies find difficult.
Conclusions: Several key issues formed an overlay for the
discussions: funding agencies need to decide what part of the
KT agenda it can address internally and which through outside
agencies; the understanding that work in this area depends on
the “K” you are trying to “T” to whom; different agencies may
see their role differently, depending on their mandate/mission
or background in KT; and the amount of dedicated funding for
KT in many agencies is minimal.
Implications for Policy, Delivery, or Practice: Funders need
to develop coordinated strategies and processes to effectively
manage the issues raised in the workshop. One suggestion
echoed by many of the participants is that there would be
value in forming a funders’ association to provide a forum for
sharing ideas and problems as well as to pool resources to
leverage change.
Primary Funding Source: The Canadian Institutes of Health
Research
Related Posters
Poster Session B
Monday, June 27 • 6:15 pm – 7:30 pm
●Logic Models as a Translational Research Tool: Lessons
From the Los Angeles Medication Management Project
Gretchen Alkema, MSW, Kathleen H. Wilber, Ph.D.
Presented By: Gretchen Alkema, MSW, Doctoral Candidate,
Leonard Davis School of Gerontology, University of Southern
California, 3715 McClintock Avenue, Los Angeles, CA 900890191; Tel: (213)740-9685; Email: alkema@usc.edu
Research Objective: To inform the translation of evidencebased research to practice by creating and applying a logic
model to the implementation and evaluation process.
Study Design: We used an iterative process of logic model
development and application with an AoA-funded EvidenceBased Prevention Program site, the Los Angeles Medication
Management Project, to create a logic model that informs the
translational of evidence-based research into practice for all
members of the project team. Logic model development
included 1) literature review of logic models; 2) group
facilitation with key team members to gain content for model;
3) draft of model; 4) consultation with representatives from
original study and providers applying the model; 5) model
revision; and 6) application of model in program
implementation and translational research evaluation.
Population Studied: Two Medicaid-waiver care management
programs serving frail, community-dwelling, low-income older
adults (65+) currently implementing evidence-based research.
Principal Findings: Logic model development was valuable in
1) maintaining adherence to program fidelity and
acknowledging where adaptations were needed due to agency
context; 2) creating a shared vision of translating evidencebased research into the agency therefore building partnerships
among stakeholders; and 3) creating a useable product to
inform external parties of project’s core elements and
intended outcomes.
Conclusions: The logic model offers a useful tool for the Los
Angeles Medication Management Project team to
communicate to internal and external stakeholders the
mechanics of implementing evidence based research into
practice and its relationship to intended outcomes in new
sites.
Implications for Policy, Delivery, or Practice: As a roadmap
for successful implementation in translational research
projects, logic model development and application are useful
for helping practitioners, program administrators, evaluators,
funders, and other stakeholders understand their own roles
and communicate more effectively with each other.
Primary Funding Source: Administration on Aging
●State Mandated Benefits Review Laws: The Growth of
Evidence-Based Health Insurance Policy
Nicole Bellows, MHSA, Helen Halpin, Ph.D., Sara
McMenamin, Ph.D.
Presented By: Nicole Bellows, MHSA, Graduate Student
Researcher, Center for Health and Public Policy Studies, UC
Berkeley, 140 Warren Hall #7360, Berkeley, CA 94720-7360;
Tel: (510) 643-1675; Fax: (510) 643-2340; Email:
nstark@berkeley.edu
Research Objective: The growth of state mandated health
insurance benefits has given rise to concerns regarding the
effects of these mandates on rising premium costs and access
to health insurance. This paper examines the laws states have
enacted to review mandated health insurance benefits and
analyzes variations of the laws with regard to who has the
responsibility and authority to conduct the reviews and the
criteria they must use to evaluate the mandated benefits.
Study Design: The state statutes for all fifty states and
Washington D.C. were reviewed to determine which states
have enacted mandated benefit review (MBR) laws as of
September 2004. For those states where no statutory
reference to a MBR process was found, the state legislative
librarian or similar state official was contacted to confirm that
such a law did not currently exist. Additionally, current state
legislative agendas and activity were reviewed to confirm the
list of states that had recently passed MBR legislation.
Population Studied: This study analyzed the state laws for all
50 states and Washington D.C. as of September 2004.
Principal Findings: A total of 26 states were found to have a
MBR in place. The oldest laws date back to the mid-1980s;
however, almost half of the MBR laws have been enacted
since 2000. These findings demonstrate that there is
substantial variation in the way the MBR laws are constructed
in terms of who conducts the review and the criteria that are
used in conducting the review. Four types of reviewers are
used in MBR laws: proponents of the legislation (5 laws),
administrative or legislative personnel (12 laws), legislatively
established commission or task force (8 laws), and a state
university system (1 law). Five types of criteria used in
conducting the reviews were identified: cost considerations,
utilization and demand considerations, medical effectiveness,
public health considerations, and other considerations.
Conclusions: The strategies employed in conducting state
mandated benefit reviews vary considerably from state to
state. While each approach has strengths and weaknesses, we
conclude that the unique aspects of the state university
system used in the California MBR law can serve as a potential
model for other states.
Implications for Policy, Delivery, or Practice: The growth of
MBR laws in recent years point to an increased reliance on
systematic reviews of published research by health policy
makers in determining whether health insurance mandates
are beneficial to the state populations. The types of reviewers
and criteria used in the review are important to understand
because these differences have both strengths and
weaknesses, particularly with respect to potential biases.
Understanding these strengths and weaknesses can lead to
improved state MBR laws.
Primary Funding Source: No Funding Source
●Enrollment Barriers in the Medicare Lifestyle
Modification Program Demonstration (LMPD) for
Medicare Beneficiaries
Sarita Bhalotra, M.D., Ph.D., Gail K. Strickler, MS, Donald S.
Shepard, Ph.D., Grant Ritter, Ph.D., A. James Lee, Ph.D.,
William B. Stason, M.D., MSc
Presented By: Sarita Bhalotra, M.D., Ph.D., Assistant
Professor, Heller School, Brandeis University, 415 South
Street, Waltham, MA 02454; Tel: (781)736-3960; Email:
bhalotra@brandeis.edu
Research Objective: In response to a White House directive,
Medicare initiated a payment demonstration to test the
feasibility and cost effectiveness of cardiovascular lifestyle
modification in 1999. Brandeis University was awarded the
contract to evaluate implementation, feasibility, and costeffectiveness. Evaluation design was predicated on enrolling
1,800 Medicare beneficiaries with heart disease into each of
two programs, the Dr. Dean Ornish Program for Reversing
Heart Disease (Ornish) and the Cardiac Wellness Extended
Program of the Mind/Body Medical Institute (MBMI).
Enrollment was capped anticipating that provision of lifestyle
modification programs would attract many enrollees. At the
end of 55 months (96% of the enrollment period), only 401
Medicare beneficiaries (11% of the target) had enrolled at the
39 sites nationwide. This study analyzes reasons for low
enrollment into the LMPD.
Study Design: Implementation was studied from both clinical
and managerial perspectives, using a qualitative, case study
design incorporating on-site and participant observation,
semi-structured interviews, reviews of written materials, with
quantitative evaluation of marketing, recruiting, enrollment,
and retention. Data were triangulated and analyzed using
principles of grounded theory.
Population Studied: The units of analysis were
demonstration sites. Implementation was studied in person
or via telephone interviews and document review at many of
the 28 Ornish and 11 MBMI sites nationwide over a four-year
period.
Principal Findings: Enrollment was low in both programs.
Less restrictive eligibility criteria, adopted in 2001, increased
enrollment only slightly. Ornish made several mid-course
enhancements to its marketing and recruitment strategies,
including engaging some nurse recruiters, yet its enrollment
(114 participants, or 26 per year) was less than half that
achieved by the MBMI (237 participants; 88 per year). MBMI
provided less central support and oversight for both clinical
and managerial activities than Ornish. The Ornish program
was substantially more demanding to program sites (more
specialized staff) and to participants (e.g. more hours per
week of required activities). Barriers to enrollment were
systematically identified at patient, provider and
organizational levels for both programs: key were senior
leadership support for the program, physician buy-in and
referral, other insurance support, program personnel belief
and practice of Program, collaborative relationship with
related programs (e.g. cardiac rehabilitation), time demands
on participants, ease of eligibility determination.
Conclusions: Enrollment was restricted due to a complex
interplay of barriers involving patients, providers, and
organizations. These impediments occurred at multiple
levels, and needed to be addressed through multi-pronged
and interconnected strategies and tactics. In addition to
clinical leadership, it was vital that managerial and political
entities within organizations engage in implementation.
Patient resistance to participation might benefit from social
marketing.
Implications for Policy, Delivery, or Practice: To improve
health status and control costs of treating chronic illnesses, it
is imperative that the health care system promote lifestyle
modification. Unless health care policymakers and program
planners learn more about the challenges, barriers and
incentives for engaging in multi-component lifestyle
modification programs and ways to overcome them, such
efforts are likely to incur huge costs while providing little
benefit to the overall population in need. Benefits that are
complex in design and delivery will be difficult for
organizations to implement, providers to employ, and patients
to utilize.
Primary Funding Source: CMS
●Is Acupuncture Complementary or Alternative Medicine?
Machaon Bonafede, MPH
Presented By: Machaon Bonafede, MPH, Doctoral Candidate,
Community and Preventive Medicine, University of Rochester,
School of Medicine and Dentistry, 601 Elmwood Avenue, Box
644, Rochester, NY 14642; Tel: (585) 766-8622; Email:
machaon_bonafede@urmc.rochester.edu
Research Objective: The objective of this study was to
determine whether or not acupuncture is a complement to or
substitute for conventional medicine. A secondary objective
was to determine what factors influenced or were associated
with acupuncture use, with a separate model predicting
acupuncture expenditures.
Study Design: Since causal estimates were desired,
instrumental variables models were employed in this crosssectional study. The influence of acupuncture expenditures
was estimated using OLS and instrumental variables
regression (IVREG), while utilization models were estimated
using PROBIT and Bivariate Probit models. In each case,
acupuncture was included as an independent variable in a
model and the conventional service of interest was the
dependent variable. Distance to the nearest acupuncturist
was the instrument used in these models.
Population Studied: This study utilized claims data from a
midsize metropolitan insurance company. After inclusion
criteria were satisfied, the total sample included every eligible
acupuncture user (n=1688) and every 18th eligible nonacupuncture user (n=17,790).
Principal Findings: From the model predicting acupuncture
use, several characteristics were associated with increased
acupuncture use, namely, being female, being age 41 to 50,
and suffering from musculoskeletal disorders or headaches.
These same factors were also most strongly associated with
increased acupuncture expenditures. Through instrumental
variables models, causal estimates were generated to
determine the nature of the relationship between acupuncture
and the various conventional services of interest. Utilization
models indicated that acupuncture was a statistically
significant substitute for primary care (odds ratio = .51), all
outpatient services (.78), pathology services (.63), all surgery
(.63), and laxatives (.60). One expenditure model showed
that an increase of one dollar spent on acupuncture resulted
in a $6.32 reduction in primary care expenditures. Similar
results were found in the seven different diagnostic
subpopulations, those being: headaches, pain symptoms,
digestive disorders, circulatory disorders, respiratory
disorders, infectious diseases, and musculoskeletal disorders.
In all, the study outlines results from over 200 models
predicting acupuncture’s influence on utilization of or
expenditures on various conventional services.
Conclusions: In many situations, acupuncture was a
substitute for conventional services in terms of both
utilization and expenditures. In general, acupuncture
substituted outpatient primary care services, all outpatient
services, musculoskeletal surgery, total pharmacy, steroids,
and pain medications. On the other hand, acupuncture
tended to complement chiropractic, physical therapy,
preventive services, and allergy-related services. Models
predicting the effect of acupuncture on total expenditures
yielded strongly negative, non-statistically significant results.
These results suggest that acupuncture expenditures trend
towards reducing total medical expenditures, or in the very
least acupuncture expenditures balance themselves out by
substituting for other conventional services.
Implications for Policy, Delivery, or Practice: These findings
are of particular importance to practitioners and policy-makers
as they indicate what services are substituted by acupuncture.
More importantly, these results show what services are not
substituted by acupuncture combating the fear that people
forego necessary medical treatment when using alternative
medicines such as acupuncture. For managed and policymakers looking to contain costs, acupuncture coverage may
be a valid cost-containment strategy as it replaces more
expensive conventional therapeutic options.
Primary Funding Source: AHRQ Training Grant
●The Cost-Effectiveness of Advanced Implant Options in
Orthopaedic Surgery
Kevin Bozic, M.D., MBA, Saam Morshed, M.D., Marc
Silverstein, M.D., James Kahn, M.D., MPH, Harry E. Rubash,
MD
Presented By: Kevin Bozic, M.D., MBA, Assistant Professor in
Residence, Institute for Health Policy Studies, University of
California, San Francisco, 500 Parnassus, MU 320W, San
Francisco, CA 94143-0728; Tel: (415) 476-3900; Fax: (415) 4761304; Email: bozick@orthosurg.ucsf.edu
Research Objective: Total joint replacement (TJR) is one of
the most common and clinically successful operations in
surgery, with over 95% of patients experiencing significant
improvements in quality of life for up to 20 years following
surgery. However, costs associated with TJR implants
continue to rise, and TJR now accounts for a higher proportion
of the Medicare budget than any other inpatient diagnosis or
procedure. The purpose of this study was to compare the
impact of advanced implant technologies on the clinical and
cost-effectiveness of total joint replacement.
Study Design: A Markov decision analysis model was
constructed to estimate the incremental lifetime costs,
effectiveness, and net health benefits gained with four
advanced implant options for use in total joint replacement.
All new technologies were compared with the conventional or
gold standard implants that have been in widespread use for
over 20 years. Model inputs, including costs, clinical outcome
probabilities, and health utility values, were derived from a
hospital cost accounting system, the published literature, and
expert opinion. Sensitivity analyses were performed to identify
the key determinants of costs saved, quality adjusted life years
(QALYs) gained, net health benefits, and the threshold values
for the critical variables of interest.
Population Studied: Adults over the age of 50 years old with
advanced degenerative arthritis of the hip and knee. Currently,
over 32 million Americans suffer from these conditions, with
an estimated economic impact of over $80 billion per year in
direct and indirect medical expenses.
Principal Findings: Newer implant materials offer the
possibility of improved implant longevity, therefore reducing
the need for further surgery in the future. Based on the results
of our model, this would result in both higher lifetime QALYs
and lower lifetime costs per patient. However, the use of
these newer technologies also leads to higher costs associated
with the initial surgery, and as a result, many hospitals and
payors have been reluctant to approve their use.
Conclusions: Advanced implant technologies could be both
more effective and cost saving over the lifetime of the patient.
However, current reimbursement policies result in strong
financial disincentives for hospitals and payors to approve the
use of these technologies in TJR. Hospitals, payors, clinicians,
and policy makers should work together to develop
reimbursement policies that take into consideration the
immediate and downstream costs, risks, and benefits of new
medical technologies before accepting them into widespread
use.
Implications for Policy, Delivery, or Practice: Total joint
replacement is one of the most common operations
performed in the United States, with over 750,000 procedures
per year. Currently, Medicare spends over $3.8 billion per year
on total joint replacement procedures, more than any other
inpatient procedure or diagnosis, and implants costs account
for over 50% of the total expenditures. Documenting the
clinical and cost-effectiveness of new implant technologies
prior to widespread adoption and use is important in
controlling costs and improving outcomes in surgery
Primary Funding Source: Orthopaedic Research & Education
Foundation Health Services Grant
●Patient Loyalty, Trust, and Satisfaction: Data and
Observations from a Medical Group Practice
Srinivas Emani, Ph.D., Craig Samitt, M.D., Robert Yood, M.D.,
Elizabeth Dugan, Ph.D.
Presented By: Srinivas Emani, Ph.D., Senior Research
Associate, Research Department, Fallon Clinic, 640 Lincoln
Street, Worcester, MA 01605; Tel: (508)595-2206; Fax: (508)
595-2225; Email: srinivas.emani@fallon-clinic.com
Research Objective: Along with patient satisfaction and trust,
loyalty to healthcare providers is emerging as a key concept of
interest to researchers and practitioners. The concept of
loyalty has attained a prominent place in the business and
marketing literature. Yet the literature on loyalty in healthcare
is not as extensive and includes varied definitions and
measures. The objective of this paper is two-fold: (1) To report
the results of a systematic literature search and review on
patient loyalty, and (2) To present the results of empirical
studies relating trust, loyalty, and satisfaction. One such study
is described in this abstract.
Study Design: The study was conducted at Fallon Clinic, a
multi-specialty medical group practice, located in Central
Massachusetts. Between September and November 2004,
patients seen by a physician or other healthcare professional
at the Clinic were mailed a patient satisfaction survey a few
days after their visit. The surveys are routinely conducted by
the Clinic as part of its quality assessment and physician
compensation process. One item on the survey asks patients
whether they would recommend the physician/healthcare
professional they saw to their family and friends (choices
include Definitely No, Probably No, Maybe, Probably Yes, and
Definitely Yes). We defined patients who responded Definitely
Yes to this item as loyal patients.
Population Studied: All patients who saw a Fallon Clinic
provider or healthcare professional between the months of
September and November 2004 received a survey. Patients
were excluded if: (1) they had received the survey in the
previous six months, and (2) the provider’s quota of 150
patients was reached.
Principal Findings: Of 26,098 surveys that were mailed, 8,681
were returned (33% response rate). 60% of the patients were
female and 40% male. The mean age of the patients was 63
years. With regard to loyalty, 77% responded Definitely Yes to
recommending the physician/healthcare professional they saw
to their family and friends. There were no differences in loyalty
by gender: 77% of females and 78% of males reported that
they were loyal to the provider they saw. Similarly, there was
no association between loyalty and age (t=-1.56, p=0.117). With
respect to the satisfaction measures, attributes of the
providers (personal manner, technical skills, explanation of
what was done, and time spent with the patient) rather than
attributes of access to healthcare (such as getting through to
the office by phone, length of time waiting in office, and length
of time to get an appointment) were associated with loyalty.
For example, 75% of the patients who responded that the
technical skills of the provider they saw were Excellent also
reported that they were loyal patients. Similarly, 77% of
patients who responded that the personal manner of the
provider they saw was Excellent reported themselves as loyal
patients. In contrast, 36% who responded that the length of
time waiting in the office was Excellent and 32% who
responded that getting through to the office by phone was
Excellent reported themselves as loyal patients. Controlling for
overall satisfaction with visit, technical skills of the provider
had the highest correlation with willingness to recommend the
provider (r=0.33) followed by explanation of what was done
(r=0.279) and personal skills of the provider (r=0.276).
Satisfaction with attributes of access to healthcare was not
related to willingness to recommend provider.
Conclusions: More than three-fourths of the patients
surveyed at a multi-specialty medical group practice reported
that they were loyal to the provider they saw. Loyalty was
associated with satisfaction with provider characteristics
rather than satisfaction with access. The provider
characteristics, including technical competence, personal
manner, and explanation of what was done to the patient,
have been examined as dimensions of trust in studies on trust
in providers. Not surprisingly, there appears to be a close
relationship between trust and loyalty. Additional studies are
underway to explore these relationships.
Implications for Policy, Delivery, or Practice: Patient loyalty
and the factors driving loyalty are important characteristics of
the practice delivery system. Data on such characteristics are
also valuable to decision makers involved in efforts to improve
the delivery of care and strengthen patient-provider
relationships. However, further attention is required on
transferring conceptual measures of loyalty to useful and
practical guidelines for improving care.
Primary Funding Source: No Funding Source
●The Role of Personal Experience in Health Management
and Policy
Mark Exworthy, Ph.D., BSc, Tim Scott, Ph.D., John E.
McDonough, Ph.D.
Presented By: Mark Exworthy, Ph.D., BSc, Senior Lecturer,
School of Management, Royal Holloway - University of
London, Egham, Surrey, TW20 0EX; Tel: 44-1784-414186; Fax:
44-1784-439854; Email: M.Exworthy@rhul.ac.uk
Research Objective: To investigate the conceptual and
practical implications of personal experience of ill-health and
health-care upon decision-making by policy-makers and health
service executives.
Study Design: Two stage design. a. Conceptual stage:
Knowledge from literatures of various disciplines was
evaluated to understand and explain the interaction between
evidence, experience and decision-making. b. Empirical stage:
Qualitative analysis of written examples of how policy-makers
and health-care executives have drawn on their personal
experience of (ill-)health and health-care to shape and
influence their decision-making.
Population Studied: The written accounts about personal
experience (of ill-health and health-care) by policy-makers and
health service executives in the US and UK.
Principal Findings: a. Conceptual findings: A conceptual
`double movement’ was identified in decision-making which
involved `experience-as-evidence’ and `evidence-asexperience.’ Conventional distinctions between evidence and
experience became blurred but this allowed a more thorough
examination of evidential norms and procedures and
prompted analysis of the weight given to each of the terms:
evidence-based and movement. b. Empirical findings: The
influence of personal experience ranged from the subtle to the
catastrophic and was used in various ways to justify decisions,
to empathize with patients and staff, to inform co-workers etc.
The mechanisms and degrees of change in personal and
professional lives varied according to the nature of the
experience and the purpose of the written account. These
changes related to perception, ideas, attitudes, values,
assumptions and/or actions. This generated different
interpretations of the transition to the newly ‘awakened’ state.
Policy-makers and executives relied more on first or secondhand accounts of experience than on a scientifically derived
evidence base.
Conclusions: Policy-makers and executives were
heterogeneous groups, their organizations equally varied. A
coherent body of specialist knowledge was lacking. Decisionmaking was often fragmented between and within
organizations. Dynamic executives shaped the tone and
character of an organization’s culture and decision-making.
Policy-makers and executives have been urged to incorporate
research evidence into their decisions but incomplete or
contradictory research evidence hardly altered the character of
such decision-making.
Evidence-based management and policy has largely ignored
the role that personal experience plays in shaping the
decisions of health system executives and policy-makers. This
is unfortunate. Personal experience is the common
denominator of knowledge: the body, its thoughts, actions
and the ability to generate, define, search, sift and synthesise
evidence, consciously and unconsciously. By combining and
applying empirical, intuitive and cognitive knowledge to
practical problems, experience is capable of providing an
intensity of insight incorporating, but unmatched by, all other
methods. This paper concludes by questioning whether the
concept of evidence-based policy and management is
appropriate given qualitative differences between policy,
management and clinical sciences.
Implications for Policy, Delivery, or Practice: Changes in
professional conduct were initiated by movement in other
compartments of their personal lives. Policy-makers and
health service executives should expect such changes in their
professional lives, embrace them and account for them.
Furthermore, given a lack of evidence, the paper calls for an
acknowledgement of and investigation into the uncertain
influence of personal experience on health policy and
executive decision-making processes.
Primary Funding Source: No Funding Source
●Establishing an Interdisciplinary Research Center: A
Component of the NIH Roadmap
Kristine Gebbie, Ph.D., Beth Fatato, BA, Frank Lowy, M.D.
Presented By: Kristine Gebbie, Ph.D., Associate Dean for
Research, Columbia University School of Nursing, 630 West
168th Street, New York, NY 10032; Tel: (212) 305-0723; Fax:
(212) 305-0722; Email: ell23@columbia.edu
Research Objective: As identified by the Institute of
Medicine's Clinical Research Roundtable, two major gaps that
hinder the translation of research to improved health
outcomes are the translation from basic science to human
studies, and from clinical research to practice and policy.
These translational blocks “impede efforts to apply science to
better human health in a expeditious fashion.” Translational
research collaboratives have a high failure rate. The purposes
of this presentation are to (1) describe the establishment of an
NIH-funded Interdisciplinary Research Center which is based
on a translational model of research to policy and practice,
(2)discuss the major challenges and barriers associated with
the translation of research across disciplines and into policy
and practice,and (3)summarize the views of core Center
members on what constitutes success for such a translational
center.
Study Design: Systematic review of research on outcomes of
collaborative research and interviews with the 14 core
members of the Center.
Population Studied: 14 core members of the translational
research Center from diverse disciplines including medicine,
public health, epidemiology, nursing, microbiology, dentistry,
and education.
Principal Findings: Few data are published regarding the
success of collaborative approaches and assessment of
comprehensive, step-by-step pathways for collaboration.
Predictors of success identified in the literature include strong
leadership, contributions and early 'buy-in' by all members,
efficiency, identifiable work products, and formal training in
interdisciplinary research. In one-on-one interviews, Center
team members identified 27 measures of success of a
translational research center. 74.1% described publications,
fundable grants, and new networks, connections and ideas as
the primary measures of success. 11% (3 individuals) listed
the establishment of interdisciplinary training programs, but
only one person (4%) indicated that policy change would be
an indicator of success.
Conclusions: Most members of this interdisciplinary research
Center focused on specific work products related to their own
research rather than policy implications of the Center.
Implications for Policy, Delivery, or Practice: For individual
researchers and clinicians to have an impact on the health
care policy and practice, their awareness and interest in the
broader policy implications of their work will need to be
enhanced.
Primary Funding Source: National Center for Research
Resources, NIH
●Assessing the Financial Impact of State Health Benefit
Mandates in California: Findings from the California
Health Benefits Project
Kominski Gerald, Ph.D., Miriam Laugesen, Ph.D., Nadereh
Pourat, Ph.D., Jay Ripps, FSA, MAAA, Robert Cosway, FSA,
MAAA
Presented By: Kominski Gerald, Ph.D., Associate Director,
Center for Health Policy Research, UCLA, 10911 Weyburn
Avenue, Suite 300, Los Angeles, CA 90024; Tel: (310) 7942681; Fax: (310) 794-2686; Email: kominski@ucla.edu
Research Objective: To develop a model for estimating
financial impacts of proposed health insurance benefit
mandates introduced in the California legislature.
Study Design: Using three primary data sets (The California
Health Interview Survey, the Health Research and Education
Trust’s California Employer Health Benefits Survey, health care
cost guidelines and claims data compiled by Milliman), we
developed a cost and coverage model to produce baseline
estimates of California’s insured population, grouped by
insurer and type of coverage (HMO, PPO, POS, FFS), and the
average expenditures for each group, including insurance
premiums and out-of-pocket expenditures. We then estimate
changes in the covered population affected by a mandate and
changes in utilization resulting from the mandate to produce
estimates of the overall change in total expenditures, including
both insurance premiums and out-of-pocket expenditures.
Population Studied: California's privately insured population
(n equals 16,261,000). The specific population studied varies
by mandate based on the treatment mandated and the
covered populations affected by the mandate.
Principal Findings: (1) To date, the estimated impact of the
seven mandates examined as part of this project on permember per-month (PMPM) expenditures among insured
Californians has been very small, usually less than 0.1 percent.
(2) The impact is low in most cases because coverage of
mandated benefits is already high. (3) The impact varies by
market segment, with low impact for most large employer
groups and a greater impact on small employer groups. (4)
Mandates have a greater impact on the individual market, and
particularly affects some subgroups within the individual
market who may face substantial premium increases resulting
from proposed mandates, however. For example, a proposed
mandate requiring maternity benefits would have produced an
estimated average increase in premiums within the individual
market of 13 percent for those currently holding policies that
exclude maternity benefits.
Conclusions: Our model is the first comprehensive effort to
develop an “open source” model by actuaries and health
services researchers to estimate the effects of health insurance
benefit mandates for different types of insurers and for
different employer firm sizes and individuals. Benefit
mandates are not necessarily costly, but their impact is much
greater in the individual market.
Implications for Policy, Delivery, or Practice: When making
policy on mandates, policymakers should consider the
differential impact among individuals and groups, rather than
‘averaging’ costs. Analysis of financial impacts of proposed
state health benefit mandates, combined with analyses of
evidence of medical effectiveness and public health impacts
produced as part of this project, provides state legislators with
evidence-based information for public decision making. Our
ability to develop a California-specific model to estimate the
financial impacts of proposed mandates in a timely fashion
provided California legislators during the 2004 legislative
session with more-detailed, specific coverage and cost
information than is generally available to legislative bodies.
Our work represents an important contribution in translating
research findings directly into the legislative process.
Primary Funding Source: CHBRP
●Improving Otitis Media Care through the
Implementation of National Guidelines in an Integrated
Delivery System
Margaret J. Gunter, Ph.D., Sarah J. Beaton, PhD, Kathryn A.
Paez, MBA, MSN, Lance Chilton, M.D., Denise Dougherty,
Ph.D.
Presented By: Margaret J. Gunter, Ph.D., President and
Executive Director, Lovelace Clinic Foundation, 2309 Renard
Place SE, Suite 103, Albuquerque, NM 87106; Tel: (505) 2623152; Fax: (505) 262-7598; Email: maggie@LCFresearch.org
Research Objective: The purpose of this study was to
contribute to the scientific and practical understanding of the
most effective methods of changing physician behavior in
accordance with evidence-based guidelines. The specific aims
of the study were to: (1) implement an evidence-based practice
guideline for otitis media, and (2) evaluate the effectiveness of
the implementation methods used to change physician
behavior in accordance with the otitis media guideline.
Study Design: The study used a quasi-experimental design,
with the guideline intervention (formal education program,
sequential reminder mailings to providers, and repair of
otoscopes) implemented in half the primary care, pediatric,
and urgent care clinics in a large southwestern integrated
delivery system, and the remaining clinics serving as controls
(mailed guidelines only).
Population Studied: The study population was limited to
health plan members, aged six months through five years,
who were seen by a staff-model physician. The clinical
outcome measures obtained from claims data were: (1)
percentage of otitis media episodes with a primary diagnosis
of either acute otitis media (AOM) or otitis media with
effusion or otalgia (combined into OME/Otalgia); (2) rate of
antibiotic fills for each of the two diagnosis categories; and (3)
rate of analgesic drop fills for pain relief. A post-intervention
provider survey provided self-reported outcome measures. In
addition, all measures were examined with respect to provider
specialty type (urgent care, family practice, and pediatrics).
Principal Findings: Claims data analysis showed significant
improvement for intervention clinic providers over control
group providers in decreased diagnosis of AOM (94.0% prevs. 91.2% post-intervention, p=.0231) and increased analgesic
drop fills (7.6% pre- vs. 10.4% post-intervention for AOM;
2.4% to 11.5% for OME/otalgia, p=.0250), but no statistical
significance for reduction in antibiotic fills. The provider
survey showed statistically significant self-reported differences
between the intervention and control clinic providers in the
hypothesized direction for all three outcome measures.
Significant differences in outcomes were seen by provider type
(family practice, urgent care, or pediatrician). Of the ten
intervention methods used, those ranked by the providers as
highest in value were: repair of the otoscopes, the formal
training, and the guidelines themselves.
Conclusions: The analyses indicated that the interventions
were effective in changing diagnosis and treatment patterns in
compliance with guidelines, especially in the reduced
diagnosis of AOM and in the increased prescribing of
analgesic drops for pain. Improvement was more marked in
the provider self-reports than in the claims data. Outcome
differences were mediated by provider specialty.
Implications for Policy, Delivery, or Practice: This study was
designed to be practical yet rigorous in its design in order to
create and validate guideline implementation tools and
approaches that would be both feasible and effective in a
variety of health care settings. Effective implementation of
guidelines requires multiple, reinforcing interventions over
time, tailored to the specific needs and barriers of the practice
setting.
Primary Funding Source: AHRQ
●Assessing the Public Health Impact of State Health
Benefit Mandates
Helen Halpin, Ph.D., MSc, Sara McMenamin, Ph.D., MPH,
MSc, Ted Ganaits, M.D.
Presented By: Helen Halpin, Ph.D., MSc, Professor of Health
Policy, School of Public Health, University of California,
Berkeley, 140 Warren Hall MC#7360, Berkeley, CA 947207360; Tel: (510) 642-2862; Fax: (510) 643-2340; Email:
helenhs@berkeley.edu
Research Objective: To develop a methodology for assessing
the public health impacts of proposed state health insurance
benefit mandate legislation in California.
Study Design: Four primary date sets are used (The California
Health Interview Survey, the California Behavioral Risk Factor
Survey, Centers for Disease Control and Prevention WONDER
database, and Milliman USA claims data) to estimate the
covered population affected by a mandate, changes in
utilization resulting from the mandate, and to estimate
changes in health outcomes using estimates of effectiveness
of utilization from the literature and projecting to the affected
population.
Population Studied: California's insured population. The
specific population studied varies by mandate based on the
treatment mandated and the covered populations affected by
the mandate.
Principal Findings: Baseline data on important health
outcomes and information on differences in health services
utilization by race/ethnicity are often not available at the state.
Using estimates of medical efficacy from the scientific
literature to estimate health outcomes in a population is likely
to overstate the effects of a mandate. However, limitations in
available data are likely to underestimate the scope of reported
public health impacts.
Conclusions: The approach California has developed for
assessing the public health impacts of benefit mandate
legislation, while challenging, represents a leap forward
towards a more comprehensive assessment of proposed state
health benefit mandates. The approach represents the first
attempt to translate health insurance benefits policy into
population health outcomes.
Implications for Policy, Delivery, or Practice: Analysis of
public health impacts of proposed health benefit mandates,
combined with analysis of evidence of medical efficacy and
costs, enables state legislators to consider the "value" of
health care services in policy making. Value represents the
improvement in health status realized for an investment of
health care dollars. Such analysis has the potential to lead to
the design of more cost-effective health insurance policy.
Primary Funding Source: State of California
●Pharmaceuticals, Prescription Plans, and Promoting
Progress
James Henderson, Ph.D., Earl Grinols, Ph.D.
Presented By: James Henderson, Ph.D., Ben Williams
Professor in Economics, Economics, Baylor University, One
Bear Place #98003, Waco, TX 76798; Tel: (254)710-4139; Fax:
(254)710-6142; Email: Jim_Henderson@baylor.edu
Research Objective: In this paper we discuss a socially
preferable alternative to the patent system as a mechanism
protecting intellectual property rights.
Study Design: Three important issues are addressed. 1) How
can we deal effectively with the above-mentioned monopoly
pricing response to a prescription drug co-pay? 2) What is the
best way to reward innovators to insure that the benefits of
innovation are spread quickly and widely (at marginal cost
prices)? 3) How do we go about calculating the reward for
innovation?
Population Studied: NA
Principal Findings: The only way out of the unacceptable
quandary is to sever the link between monopoly power created
by the patent system and drug distribution. In fact, patent
protection for new inventions is an anachronistic holdover
compromise solution from an earlier era. Modern theory and
tools imply an entirely different approach. This paper reviews
historical policies toward research and development in light of
the intervention principles of public finance.
Conclusions: The paper concludes that an intertemporal
bounty satisfying certain conditions is more efficient than
current law and also dominates other structures including
patent buyouts, except for a full-economy Ramsey pricing
solution. Since Ramsey pricing is unlikely ever to be seen in
practice, the investigation of efficient replacements for
pharmaceutical patenting is timely and significant. The
investigation identifies a mechanism for marginal cost drug
pricing consistent with inducement to innovation as strong as
provided by patents, consistent with increased social efficiency
and practical implementation. The latter is argued from
positive consideration of the fact that a similar system is
successfully in place and has been used for many years in
another sector of high innovation, the music industry.
Implications for Policy, Delivery, or Practice:
Pharmaceutical patent protection places Americans on the
horns of an untenable dilemma. On one hand, monopoly
protection limits the treatment options for individuals without
prescription drug coverage and causes American consumers –
many of whom are sick and not working – to pay too much for
their medications. On the other hand, providing insurance for
prescription drugs brings into play the monopolist's response
which is to raise the drug's price in proportion to the inverse
of the plan co-pay. A 20% co-pay, for example, creates an
overwhelming incentive that leads the supplier to raise price
to 5 times the initial level and reap the profit windfall. The
firm adjusts its price-output pair from the (5p0, q0) point only
if profits are thereby raised further.
The present research raises the possibility of change in the
pharmaceutical drug industry as part of the larger movement
in the United States to find a more desirable social ethic that
can drive and sustain our health care system.
Primary Funding Source: No Funding Source
●Prevalence, Treatment, and Control of Chest Pain
Syndromes, Blood Pressure, and LDL Cholesterol in
Hypertensive Patients
Katharine Hendrix, Ph.D., MS, Brent M Egan, M.D., Susan
Mayhan, MS, Daniel T Lackland, DrPH
Presented By: Katharine Hendrix, Ph.D., MS, Assistant
Professor, General Internal Medicine, Medical University of
South Carolina, 96 Jonathan Lucas Street, 826 CSB,
Charleston, SC 29425; Tel: (843)792-6340; Fax: (843)792-0816;
Email: hendrikh@musc.edu
Research Objective: The purpose of this study was to define
the prevalence and medical management of chest pain,
angina pectoris, chronic angina, and pre-infarction
angina/intermediate coronary syndrome (ICS) among
hypertensive patients in primary care.
Study Design: The Hypertension Initiative database
(N=72,508 hypertensive patients’ primary care records) was
analyzed to characterize prevalence and management of chest
pain syndromes (CPS) and control of blood pressure
(<140/90 mm/Hg) and LDL-cholesterol (<100 mg/dL).
Patients with more than one CPS were categorized by the
most severe diagnosis.
Principal Findings: Eleven percent of patients (n=7996) had a
CPS diagnosis. Of these, 66% (5284) were diagnosed with
chest pain only, 15% (1204) with angina, and 19% (1508) with
ICS. More men than women were diagnosed with angina (18%
vs. 4%) and ICS (21% vs. 10%). More women than men were
diagnosed with chest pain only (86% vs. 61%). (Figure 1)
African Americans received more chest pain diagnoses (71%
vs. 62%), similar angina diagnoses (14% vs. 16%), and fewer
ICS diagnoses (15% vs. 22%) than Caucasians. Most striking,
women and African-Americans received fewer medications
than men and Caucasians within each diagnostic category. In
addition, prescription rates differed more by gender
(male/female) than by racial group (Caucasian/AfricanAmerican) for angiotensin converting enzyme inhibitors,
diuretics, aspirin, statins, and nitrates.
Conclusions: We conclude that Hypertensives with CPS
received more medications and achieved better cardiovascular
risk factor control (blood pressure <140/90 mm/Hg and LDLcholesterol <100 mg/dL) than hypertensives without a CPS
diagnosis but the majority in both groups remained above
recommended levels. Primary care physicians treat
cardiovascular risk factors relatively aggressively in these highrisk hypertensive patients. However, concerning numbers of
these patients do not reach BP or LDL-c goals and substantial
differences in treatment and outcomes, especially between
gender groups, represent opportunities to reduce disparities.
Implications for Policy, Delivery, or Practice: Accurate,
confidential, and regular feedback to primary care physicians
such as is provided by the Hypertension Initiative database
could assist health care providers track their adherence to
treatment guidelines and their patients progress to reduce or
eliminate some of the observed disparities.
Primary Funding Source: Duke Endowment
●Impact of Nonresponse on Medicare Current Beneficiary
Survey Estimates
Galina Khatutsky, MS, Gregory C. Pope, MS, James R.
Chromy, Ph.D., Gerald S. Adler, MPhil
Presented By: Galina Khatutsky, MS, , Aging, Disability, and
Long Term Care Program, RTI International, 411 Waverley
Oaks Road, Suite 330, Waltham, MA 02452; Tel: (781) 7888100 x 125; Fax: (781) 788-8101; Email: gkhatutsky@rti.org
Research Objective: The Medicare Current Beneficiary Survey
(MCBS) is used by policymakers and research analysts to
provide information on a wide array of topics about the
Medicare program. This study had several objectives in
evaluating the impact of nonresponse on MCBS estimates: 1)
to examine unit nonresponse for beneficiaries in their initial
interview round; 2) to evaluate panel attrition; 4) and to
measure item nonresponse. The study also sought to
recommend possible improvements to the current procedures
for handling missing data in the MCBS.
Study Design: Initial round unit nonresponse was
decomposed into the response rate and the difference in
respondent and nonrespondent means, and estimated for
variables available for both respondents and nonrespondents
(demographic, enrollment, health status, and service
utilization measures). In addition to analyzing the
components of nonresponse bias, we analyzed nonresponse
bias directly by comparing means (proportions) for
respondents and eligibles. We estimated bias before and after
applying MCBS nonresponse adjustment weights. The
analysis of second, third, and fourth year panel attrition was
performed using a similar methodology but employing the
conditional response rates. Item nonresponse rates were
calculated using the ratio of item nonrespondents to item
eligibles.
Population Studied: The sample for analyzing initial round
unit nonresponse was confined to MCBS eligibles in their
initial round of the survey with beneficiaries pooled across
three years of MCBS data (1997-1999) to maximize sample
size. For panel attrition, two pairs of MCBS file years, i.e.,
1997–98 and 1998–99, were pooled to construct three
separate panel attrition analysis samples. The analysis sample
for item nonresponse was confined to 1999 MCBS Access to
Care sample respondents.
Principal Findings: For initial round nonresponse, although
statistically significant differences occurred between
respondents and nonrespondents on such demographic
characteristics as gender, race and geographic distribution,
the magnitude of the differences between eligibles and
respondents was relatively small and unlikely to cause a major
potential for bias. However, current nonresponse adjustments
were not as effective for health status, expenditure, and service
utilization characteristics. Although initial nonresponse bias
was small and further reduced by MCBS nonresponse
weights, it was not entirely eliminated. Beneficiary
characteristics affecting the response propensity varied for
each attrition sample, but because of the high MCBS
conditional response rates for each attrition sample, the
magnitude of the differences between eligibles and
respondents was relatively small, and thus unlikely to create
bias. Cumulative response rates were found to be comparable
to other large national surveys. While conditional response
rates increased over the MCBS interview cycle, cumulative
response rates decreased. Item nonresponse was generally
low in the MCBS, with the exception of several items
pertaining to recall of past events and knowledge of certain
health insurance information.
Conclusions: Our findings indicated that for most of the
analysis variables studied, the bias caused by differences
between nonrespondents and respondents was substantially
reduced or eliminated by the nonresponse procedures
currently employed in the MCBS.
Implications for Policy, Delivery, or Practice: Implications
for future MCBS administration are grouped into several
domains and include recommendations on new or improved
weighting adjustment procedures.
Primary Funding Source: CMS
●Assessing the Financial Impact of State Health Benefit
Mandates in California: Findings from the California
Health Benefits Project
Gerald Kominski, Ph.D., Jay Ripps, FSA, MAAA, Miriam
Laugesen, Ph.D., Robert Cosway, FSA, MAAA, Nadereh
Pourat, Ph.D.
Presented By: Gerald Kominski, Ph.D., Professor, Health
Services, UCLA School of Public Health, 650 Charles E. Young
Drive South, Los Angeles, CA 90095-1772; Tel: (310) 794-1238;
Email: kominski@ucla.edu
Research Objective: To develop a model for estimating
financial impacts of proposed health insurance benefit
mandates introduced in the California legislature.
Study Design: Using three primary data sets (The California
Health Interview Survey, the Health Research and Education
Trust’s California Employer Health Benefits Survey, health care
cost guidelines and claims data compiled by Milliman), we
developed a cost and coverage model to produce baseline
estimates of California’s insured population, grouped by
insurer and type of coverage (HMO, PPO, POS, FFS), and the
average expenditures for each group, including insurance
premiums and out-of-pocket expenditures. We then estimate
changes in the covered population affected by a mandate and
changes in utilization resulting from the mandate to produce
estimates of the overall change in total expenditures, including
both insurance premiums and out-of-pocket expenditures.
Population Studied: California's privately insured population
(n=16,261,000). The specific population studied varies by
mandate based on the treatment mandated and the covered
populations affected by the mandate.
Principal Findings: (1) To date, the estimated impact of the
seven mandates examined as part of this project on permember per-month (PMPM) expenditures among insured
Californians has been very small, usually less than 0.1%. (2)
The impact is low in most cases because coverage of
mandated benefits is already high. (3) The impact varies by
market segment, with low impact for most large employer
groups and a greater impact on small employer groups. (4)
Mandates have a greater impact on the individual market, and
particularly affects some subgroups within the individual
market who may face substantial premium increases resulting
from proposed mandates, however. For example, a proposed
mandate requiring maternity benefits would have produced an
estimated average increase in premiums within the individual
market of 13% for those currently holding policies that exclude
maternity benefits.
Conclusions: Our model is the first comprehensive effort to
develop an “open source” model by actuaries and health
services researchers to estimate the effects of health insurance
benefit mandates for different types of insurers and for
different employer firm sizes and individuals. Benefit
mandates are not necessarily costly, but their impact is much
greater in the individual market.
Implications for Policy, Delivery, or Practice: When making
policy on mandates, policymakers should consider the
differential impact among individuals and groups, rather than
‘averaging’ costs. Analysis of financial impacts of proposed
state health benefit mandates, combined with analyses of
evidence of medical effectiveness and public health impacts
produced as part of this project, provides state legislators with
evidence-based information for public decision making. Our
ability to develop a California-specific model to estimate the
financial impacts of proposed mandates in a timely fashion
provided California legislators during the 2004 legislative
session with more-detailed, specific coverage and cost
information than is generally available to legislative bodies.
Our work represents an important contribution in translating
research findings directly into the legislative process.
Primary Funding Source: California State Legislature
●An Investigation of the First-Mover Advantage of
Pharmaceutical Advertising and its Implications for
Regulations
W. Jacqueline Kwong, PharM.D., Ph.D., Edward C. Norton,
Ph.D., Marisa Domino, Ph.D.
Presented By: W. Jacqueline Kwong, PharM.D., Ph.D., Health
Outcomes Manager, Global Health Outcomes,
GlaxoSmithKline, MAI C4674.4B, P.O. Box 13398, Reserach
Triangle Park, NC 27709-3398; Tel: (919)483-3945; Fax:
(919)483-3096; Email: jackie.w.kwong@gsk.com
Research Objective: The objective of this study was to
determine if there was an asymmetry in the effectiveness of
advertising between earlier and later entrants that would
support the entry deterrence theory of advertising in the
pharmaceutical market. The effect of advertising is often
assumed to be durable, because consumers are likely to retain
memory of the information they heard and that memory
depreciates over time. As such, products that come to the
market earlier may have a competitive advantage over later
entrants. As a result, advertising can create potential barriers
of entry and regulation may be needed.
Study Design: The effectiveness of advertising was evaluated
using estimates of advertising elasticity of demand (i.e., the
percent change in a product’s sales per each percent change
in its advertising expenditures). Advertising elasticity of
demand of existing products and new entrants in the periods
following new entry was estimated using time and disease
fixed effects estimation. Because the effect of advertising on
product sales is not immediate, one-period and two-period lag
models were estimated. For instance, if a new product
entered the market during period t, the advertising
expenditures for existing products and the new product in
period t+1 or t+2 were analyzed. The logarithm of a product’s
sales was regressed on its own advertising expenditure, the
total advertising expenditure of competing products in the
same market, the number of competing products, number of
years the product has been on market, a dummy variable
indicating whether the product is a new entrant, and an
interaction term between the dummy variable and the
product’s own advertising expenditure.
Population Studied: Advertising expenditure and product
sales data were obtained from Scott-Levin Market Research
Audit data. New product entry data were from USPDI, Orange
Book and the Scott-Levin Market Research Audit data. Data
from January 1995 to December 2001 were used. The analysis
was conducted on a quarterly basis. Products from eight
therapy markets were examined: asthma, migraine, obesity,
Parkinson’s disease, seizure, depression, lipid disorder, and
gastric and duodenal ulcer.
Principal Findings: In both lag models, product sales
significantly increased with a product’s own advertising
expenditure, and significantly decreased with the total
advertising expenditure of competing products, and the
number of competing products. For existing products,
advertising elasticity of demand was estimated to be 0.068%
in the one-period lag model and 0.062% in the two-period lag
model. The elasticity estimates for new entrants was lower at
0.046% and 0.044%, respectively. The elasticity estimates
between new and existing products were not significantly
different. Comparing results of the two lag models,
advertising elasticity of demand depreciated at 8.7% per
quarter.
Conclusions: The analysis did not find any significant firstmover advantage in the effectiveness of advertising between
earlier and later entrants that would support the potential
entry deterrence effect of advertising in the pharmaceutical
market.
Implications for Policy, Delivery, or Practice: These results
did not support regulation of pharmaceutical advertising
based on anti-competitive grounds.
Primary Funding Source: American Foundation for
Pharmaceutical Education
●All Improvement is Local: Evaluating the Use of an
Innovative, Multi-faceted Intervention by a National
Professional Organization to Translate its Guidelines into
Practice
Suzanne Lazorick, M.D., MPH, Virginia L.H. Crowe, RN, MS,
Judith C. Dolins, MPH, Carole M. Lannon, M.D., MPH
Presented By: Suzanne Lazorick, M.D., MPH, Primary Care
Research Fellow, Center for Children's Healthcare
Improvement, UNC Chapel Hill, 200 Timberhill Place, Suite
201, Chapel Hill, NC 27599-7226; Tel: (919)966-0045; Fax:
(919)966-9203; Email: lazorick@email.unc.edu
Research Objective: Professional organizations have not
engaged in partnership at local levels to translate guidelines
into practice. The objective of this study was to understand
facilitators, barriers, and outcomes of using American
Academy of Pediatrics (AAP) chapters to implement the
evidence-based guidelines for diagnosis and treatment of
Attention Deficit Hyperactivity Disorder (ADHD). With
support from a quality improvement (QI) organization, this
intervention targeted: 1) leadership of AAP state chapters and
2) primary care practices (including participation in online
education modules, attendance at a workshop, completion of
chart reviews, and ongoing coaching in QI).
Study Design: Qualitative analysis of responses from semistructured interviews and chapter monthly progress reports.
Two researchers coded responses for factors related to
program implementation. Outcomes assessed were
development of infrastructure to support participating
practices, or future plans for expansion. Practices audited
charts to identify gaps and track improvements in care based
on the guidelines
Population Studied: Five state chapters were selected from
22 applications. We interviewed project leaders from the four
chapters that completed the intervention phase: the physician
champion, executive director, and project director, where
applicable. One state deferred participation due to competing
state priorities.
Principal Findings: All nine project leaders, representing 60
practice teams, were interviewed and all 22 reports were
reviewed. Facilitators to involving practices, sustaining
interest, and stimulating changes included: personal contact
from local opinion leaders, a face-to-face gathering of teams,
involvement of office staff in addition to a physician, regular
conference calls with teams and chapter staff, involving
experts on calls, frequent contact from project administrators,
and use of measurement to identify gaps in care. Baseline
chart audits demonstrated that no practice was providing care
consistent with the seven components of the guidelines, but
few teams continued ongoing measurement.
Barriers included time commitment, lack of reimbursement
for children’s mental health services, competing clinical
priorities (e.g., winter “flu” season), time required for
clinicians to complete online modules, and lack of availability
of data about progress for tracking changes in care.
Three chapters plan ongoing activities with current practices
in improving ADHD care. Two have secured funding to
expand the program and engage additional practices. Three
have specific plans to use the QI infrastructure developed to
address additional clinical topics.
Conclusions: Chapter leaders believe this intervention helped
them develop the nascent infrastructure needed to support
local practice teams in quality improvement activities. Direct
contact from state opinion leaders and a face-to-face meeting
engaged participants and facilitated collaboration. Use of
baseline measurement identified gaps in care and motivated
participation; however, lack of ongoing measurement inhibited
the use of data to drive further improvements. Support from
the national AAP and a QI organization was important.
Implications for Policy, Delivery, or Practice: These findings
will be applied in the next project year involving six additional
chapters. Professional societies at the national and state level
are interested in and can develop local infrastructure for
improvement and translation of guidelines into practice.
Coaching, tools, and support from the national organization
and quality improvement experts are helpful in facilitating
these efforts.
Primary Funding Source: AHRQ
●Knowledge Dissemination in a System of Care: Moving
from policy to the front line
Louise Lemieux-Charles, Ph.D., Larry Chambers, Ph.D., Ken
LeClair, M.D., Carole Cohen, M.D., Barbara Schulman,
Rhonda Cockerill, Ph.D.
Presented By: Louise Lemieux-Charles, Ph.D., Chair, Health
Policy, Management and Evaluation, University of Toronto, 12
Queen's Park Crescent West, Toronto, M4E 3R1; Tel: (416)
978-4210; Fax: (416)978-8350; Email:
l.lemieux.charles@utoronto.ca
Research Objective: To analyze, at local and system levels,
the use of knowledge transfer processes by studying the
effectiveness of four community-based dementia service
delivery networks in Ontario, Canada. The study was funded
under a collaborative researcher-partner program to maximize
information exchange among researchers, policy makers and
clinical decision makers. Partners included the Ontario
Ministry of Health and Long Term Care and the Ontario
Alzheimer’s Association. Funding coincided with the
province’s adoption of a strategy to implement networks
throughout the province. Researchers participated at the
policy level in assisting decision makers to develop guidelines
for network development; while simultaneously studying the
four networks which were in existence.
Study Design: A case study design was used. Using
Nonaka’s model of knowledge creation processes, knowledge
translation elements employed throughout the three year
study are described. The model assumes a continuous
learning process as understanding deepens and changes. It
represents the processes that take place through the sustained
interaction of receptor functions involving researchers and
decision makers. We describe how multiple organizations,
professionals, managers and policy makers developed a
shared understanding of what was entailed in creating a
dementia network. The researcher’s role and the complex
interactions involved in translating knowledge are analyzed.
Population Studied: Key informant interviews and document
analyses were carried out. Key informants were drawn from
three groups associated directly or indirectly with the study: 1)
the Research Steering Committee, involving researchers and
clinical decision-makers from the four networks ; 2) the
Provincial Working Group, comprised of researchers and
policy decision-makers associated with the provincial
government’s Alzheimer Strategy; and 3) clinical decision-
makers from developing dementia care networks who
participated in either local workshops delivered by researchers
involved in the study or in a centralized train-the-trainer
workshop for individuals associated with dementia care
networks. A knowledge transfer timeline was developed.
Principal Findings: The knowledge transfer timeline reveals
that researchers were involved in the development of
collaborations prior to the funding of the study, and have
remained involved with the ongoing provincial Alzheimer’s
strategy. The receptor functions facilitated the linkage among
people, ideas and resources and the translation of policy
between the provincial and local levels. A fine balance existed
between carrying out of research and sharing expert
knowledge about network development. There were pressures
to share research findings prior to their finalization. However,
the input of an experienced research team legitimized the
guidelines developed for other networks.
Conclusions: Sustained interactivity is considered to be
essential to the dissemination of knowledge. However, there
has been little consideration of the complexity of these
interactions when system change is implemented. What are
the implications for academics of this interactivity over time?
Does the commitment assume a life long work?
Implications for Policy, Delivery, or Practice: It became
evident that processes and interactions alone are not
sufficient to sustain change, though we found that these
interactions led to individuals developing new relationships
and exploring other areas for improvement. The use of
learning forums to codify knowledge and its
institutionalization are critical, as are models such as “trainthe-trainers” to build system capacity.
Primary Funding Source: Other Foundation
●Coding Algorithms for Defining Comorbidities in ICD-9CM and ICD-10 Administrative Data
Hude Quan, M.D., Ph.D., Vijaya Sundararajan,Patricia Halfon,
M.D., MPH, FACP, Andrew Fong,Bernard Burnand, BComm;
M.D., MPH, Jean-Christophe Luthi, Duncan Saunders, M.D.,
Ph.D., MBBCh, Cynthia Beck, Thomas E Feasby, M.D., MASc;
M.D., William A Ghali, M.D., MPH
Presented By: Hude Quan, M.D., Ph.D., Assistant Professor,
Community Health Science, University of Calgary, 911 - South
Tower, FMC 1402-29th Street NW, Calgary, T2N 2T9; Tel:
(403) 944-8912; Fax: (403)944-8950; Email:
hquan@ucalgary.ca
Research Objective: Implementation of the International
Statistical Classification of Disease and Related Health
Problems, 10th Revision (ICD-10) coding system has
presented challenges for using administrative data. To
develop ICD-10 coding algorithms to define Charlson and
Elixhauser comorbidities in administrative data and to
compare the consistency between ICD-9-CM and ICD-10
coding algorithms in defining these comorbidities.
Study Design: ICD-10 coding algorithms were developed by
"translation" of the ICD-9-CM codes constituting Deyo's (for
Charlson comorbidities) and Elixhauser's coding algorithms
and by physicians' assessment of the face-validity of selected
ICD-10 codes. The process of carefully developing ICD-10
algorithms also produced modified and enhanced ICD-9-CM
coding algorithms for the Charlson and Elixhauser
comorbidities. We then used in-patients aged 18 years and
over in ICD-9-CM and ICD-10 administrative hospital
discharge data from a Canadian health region to assess the
comorbidity frequencies and mortality prediction achieved by
the original ICD-9-CM algorithms, the enhanced ICD-9-CM
algorithms, and the new ICD-10 coding algorithms (these
coding algorithms will be presented in detail)
Population Studied: Inpatients of the Calgary Health Region,
2001/2002 to 2002/03 fiscal year.
Calgary, Alberta. Canada.
Principal Findings: Among 56585 patients in the ICD-9-CM
data and 58805 patients in the ICD-10 data, frequencies of the
17 Charlson comorbidities and the 30 Elixhauser comorbidities
remained generally similar across algorithms. The new ICD-10
and enhanced ICD-9-CM coding algorithms either matched or
outperformed the original Deyo and Elixhauser ICD-9-CM
coding algorithms in predicting in-hospital mortality. The Cstatistic was 0.842 for Deyo's ICD-9-CM coding algorithm,
0.859 for the ICD-10 coding algorithm, and 0.859 for the
enhanced ICD-9-CM coding algorithm. The Elixhauser ICD-9CM coding algorithm yielded a C-statistic of 0.867, versus
0.871 for the ICD-10 coding algorithm and 0.880 for the
enhanced ICD-9-CM coding algorithm.
Conclusions: These newly developed ICD-10 and ICD-9-CM
comorbidity coding algorithms detect similar estimates of
comorbidity prevalence in administrative data, and may
outperform existing ICD-9-CM coding algorithms.
Implications for Policy, Delivery, or Practice:
Primary Funding Source: Canadian Institute of Health
Research
●Combination of Hospital Discharge and Physician Claims
Data Improves Detection of Procedures
Hude Quan, M.D., Ph.D., Corrine Truman, Ph.D., Yan Jin, MA,
William A Ghali, M.D., MPH
Presented By: Hude Quan, M.D., Ph.D., Assistant Professor,
Community HealthScience, University of Calgary, 911 - South
Tower, Foothills Medical Centre 1403 29th Street NW, Calgary,
T2N 2T8; Tel: (403)944-8912; Fax: (403)944-8950; Email:
hquan@ucalgary.ca
Research Objective: We have previously reported that
administrative hospital discharge data had low validity in
coding some minor procedures (Medical Care 2004; 42:8019). Physician claims data could be a potential source of
information to improve validity of detecting procedures.
However, little is known about validity of physician claims in
coding procedure. Therefore, we assessed agreement between
physician claims data and chart review data for
surgical/diagnostic procedures and whether a combination of
hospital discharge and physician claims data increased
detection of procedures compared with either hospital
discharge or physician claims data alone
Study Design: Procedure information was extracted from
hospital discharge data for a sample of 1200 (600 general
surgery and 600 general internal medicine) randomly selected
1996/97 separation records from three hospitals in a
Canadian health region. Unique personal identifier of personal
health numbers was used in conjunction with admission and
discharge dates to retrieve corresponding physician's claims
information. A nursing trained professional coder examined
the corresponding medical charts for evidence of 5 major
procedures (i. e. appendectomy, cholecystectomy, colectomy,
mastectomy and repair of hernia) and 12 minor surgical or
diagnostic procedures (i.e. arterial catheterization,
arteriogram, bone and marrow aspirate, cardiac
catheterization/coronary angiography, chest tube,
esophagogastroduodenoscopy, hemo/peritoneal dialysis,
lumbar puncture, percutaneous abdominal drainage,
pulmangiography, sigmoidoscopy and thoracentesis). To
interpret the extent of agreement with chart review data above
chance, we calculated kappa for hospital discharge, physician
claims data and both of the databases and categorized values
into five groups: <0.2 (poor agreement), 0.21-0.40 (fair
agreement), 0.41-0.60 (moderate agreement), 0.61-0.80
(substantial agreement), and 0.81-1.00 (near perfect agreemen
Population Studied: Inpatients from three hospitals in
Calgary Health Region, Alberta, Canada.
Principal Findings: Physician claims data had near perfect
agreement for 4 major procedures and substantial agreement
for 1 major procedure relative to chart data. Hospital
discharge data also had near perfect agreement with chart
data for 4 major procedures and substantial agreement for 1
major procedure relative to chart data. For the 12 minor
surgical or diagnostic procedures assessed, physician claims
data had near perfect agreement with chart data for 4,
substantial agreement for 1, moderate agreement for 4 and
fair agreement for 1 and hospital discharge data had near
perfect agreement for 5, substantial agreement for 6 and fair
agreement for 1 relative to chart data. The agreement was
improved for 13 procedrues when both of the hospital
discharge and physician claims data were used as opposed to
the hospital discharge or physician claims data alone,
reaching near perfect agreement for 5 major procedures and 8
minor procedures, substantial agreement for 2 minor
procedures and moderate agreement for 1 minor procedure
relative to chart data
Conclusions: Physician claims data appear to have a high
level of agreement with medical charts for major procedures
but varing, sometimes low, levels of agreement for minor
procedures.
Primary Funding Source: Calgary Health Region
●Genetic Influences on Health Services Use and Healh
Status
James Romeis, Ph.D, Hong Xian, Ph.D, Nancy Pedersen,
Ph.D, Andrew Heath, DPHil
Presented By: James Romeis, Ph.D, Professor of Health
Services Research, Health Management & Policy, School of
Public Health, 3545 Lafayette Avenue, St. Louis, MO 63104;
Tel: (314)977-8148; Fax: (314)977-8150; Email:
romeisjc@slu.edu
Research Objective: The overall goal of the study is to assess
genetic and environmental influences on condtion, health
services use and health status as outcome.
Study Design: Classical twin design using telephone survey
and tri-variate analytical model fitting methods.
Population Studied: The Vietnam Era Twin Registry - a nonclinical, community sample of white, middle age, middle class,
male-male twins [n=2936 pairs].
Principal Findings: The analysis indicates that there are
moderate unique and common genetic effects for conditon
[alcohol dependence], all cause health services use and health
status as reflected in self-reports. Variance component
analysis indicates that specific additive genetic influences are
found for condition [55%], health services use [41%] and health
status [29%]. Small, but significant effects were found across
all variables. The majority of variance is explained by nonshared environmental factors and error.
Conclusions: The data reinforce previous research designed
to assess the the relative importance of genes and
environmental influence on help-seeking and use of services.
Genetic effects are not trivial and should be incorportated in
our models if our theories are to advance.
Implications for Policy, Delivery, or Practice: Health
services research has mostly ignored behavioral genetics in
explaining individual differences in health behavior. As the
magnitude of genetic influences increase, serious
consideration needs to be given to altering policies and
interventions to account for this important source of variation.
Primary Funding Source: NIA, NIAAA, VAHSRD
●Building Community Capacity for Evidence-Based
Practice
Shiloh Turner, MPA, Eric Rademacher, Ph.D., Mark Carrozza,
MA
Presented By: Shiloh Turner, MPA, Director, Health Data
Improvement, Program, The Health Foundation of Greater
Cincinnati, 3805 Edwards Road, Suite 500, Cincinnati, OH
45209; Tel: (513) 458-6608; Fax: (513) 458-6610; Email:
sturner@healthfoundation.org
Research Objective: The objective of this session is to
describe a variety of community and state level research
approaches for gathering, analyzing and disseminating health
information for the purposes of planning, evaluation, and
policy work.
Study Design: The Health Foundation of Greater Cincinnati,
in partnership with the University of Cincinnati’s Institute for
Policy Research, has developed a strategic research agenda
aimed at providing community health data for use by state
and local healthcare agencies. Organizations use this
information for needs assessments, program design and
evaluation, and policy analysis.
Research approaches employed by the Foundation include
community-based participatory research of the emerging
Hispanic and Latino population, a broad-based computeraided telephone survey of over 2,000 adults in Greater
Cincinnati assessing community health status, and a
statewide telephone survey of 800 respondents assessing
public opinions regarding a variety of health policy issues.
Each approach provides data that are not otherwise available
at the state or local level. The surveys are conducted semiregularly, utilize national indicators whenever possible, and
build on past work to allow for trending capability.
The strategic research agenda does not end with the
production of the various health data resources. It also
provides technical assistance in communication and
dissemination efforts. For example, some organizations are
quite intimidated by data. They are not staffed with experts in
program evaluation or health policy. However, this does not
mean that their program work is unsophisticated. It simply
means that their ability to evaluate and communicate their
success sometimes falls short. Good technical assistance can
get them over this barrier, and can lead to some of the best
success stories in policy change. Other organizations are
quite sophisticated in working with health data. The capacity
exists, but they need access to the information in a timely
manner. For this group, it is ideal to offer some sort of online
data dissemination system.
Population Studied: Greater Cincinnati residents living in
Southwest Ohio, Northern Kentucky, and Southeast Indiana.
Principal Findings: Based on the application of this model,
we have found that certain strategies for data gathering and
dissemination are a more natural fit to some organizations,
while other strategies work better for others. A couple of case
studies that illustrate this point include: 1) a rural mental
health board was able to achieve policy change for their jail
diversion program through collecting a few key data elements
and benchmarking them with the region; 2) a local health
collaborative was able to analyze community obesity data
using an innovative web application called the Online Analysis
and Statistical Information System (OASIS). Once the need
for programming was identified, the collaborative convened a
large group of stakeholders to address the problem. Based on
this effort, national funding was obtained to create the Ohio
affiliate to the America on the Move program. In both cases,
the organization achieved success, but the approach used was
quite different.
Conclusions: Having a pro-active strategic research agenda is
an efficient way of filling data gaps in a community. However,
it is important to surround those data collection efforts with
technical assistance and training. It is also important to offer
a variety of methods for accessing and disseminating data, as
the same approach does not work for all. A complete strategy
of data collection, analysis, and dissemination encourages
widespread community participation in the health policy
arena.
Implications for Policy, Delivery, or Practice: Communities
given access to quality state and local health data, surrounded
by technical assistance on how to use the data for their own
areas of interest, are empowered to design programs that
better address service gaps in the region. They are better able
to evaluate the services offered and make adjustments to
improve service delivery. And finally, communities are able to
take evaluation data and contextualize it within the larger
community. This builds a much stronger case for influencing
policymaker decisions about funding and other means of
support for programs.
Primary Funding Source: The Health Foundation of Greater
Cincinnati
●Transitional Research on Nursing Home Practice: an
Optimization System for Health Care Management
Ning Zhang, M.D., Ph.D., Thomas T.H. Wan, Ph.D., MS, Lynn
Unruh, Ph.D., RN, Shriram Marathe, MBA, M.D.
Presented By: Ning Zhang, M.D., Ph.D., Assistant Professor,
Department of Health Administration/College of Health and
Public Affairs, University of Central Florida, 3280 Progress
Drive, Orlando, FL 32826; Tel: (407)823-3344; Fax: (407)8234895; Email: nizhang@mail.ucf.edu
Research Objective: To develop a decision support system
for optimizing nursing home performance based on a
theoretically informed framework and to evaluate the
effectiveness of the system through multilevel structural
modeling of the quality improvement on resident outcomes.
Study Design: This is a longitudinal study using the 19992002 Minimum Data Sets (MDS) and Online Survey
Certification and Reporting System (OSCAR) which include all
the resident care and facility management information for
Medicare and Medicaid certified nursing homes. We linked
these clinical and administrative data to Area Resource File
(ARF) and state reimbursement data. A data warehouse was
established to structure the multilevel data. A managerial
decision support system was formulated in three steps. First,
our analysis established multi-factor measurement of
organizational efficiency and quality using confirmatory factor
analysis. Second, a benchmarking tool was developed to
determine the best practice of performance in the market.
Finally, organizational, market and financial factors, and
resident characteristics that affect organizational performance,
such as geographic area, nurse staffing, market competition,
operating margins, acuity levels were identified by using
multilevel modeling of panel data. With a user-friendly,
graphics-user interface (GUI), nursing home administrators
and practitioners are able to compare their practice with their
competitors in the similar metropolitan area and find best
practice models to optimize the performance or output of
facilities under the internal and external environmental
constraints. To assess the applicability of the decision support
system for performance improvement, this study used
simulation techniques to examine its stability in varying
homogeneous subgroups of nursing homes.
Population Studied: This study included approximately
15,000 nursing homes and 13 million residents each year for
four consecutive years.
Principal Findings: Multilevel modeling of the quality
improvement on resident outcomes identified higher
performance facilities that differed in many aspects of
organizational structure and operating processes from lower
performance facilities. A strong evidence shows that this
modeling approach yields stable results to profile the best
practice in nursing home quality (r=0.93). A small root-meansquare-error estimate (<0.05) was generated by the model.
The decision support system, based on the theoretically
specific model coupled with the constraint-oriented
methodology, has been formulated to provide decision
makers to optimize their operational management.
Conclusions: This graphic-user interface based expert system
can simulate the optimal performance of management
practices in nursing homes. The study demonstrates that this
decision making system is reliable and cost-effective.
Implications for Policy, Delivery, or Practice: This system
allows health care executives and decision makers rely on
theoretically based and empirically validated evidence in
management. Such evidence-based knowledge can be
integrated with practical and experiential knowledge to shed
light on the cause-and-effect relationships between the
problems and solution sets. The study demonstrates the
feasibility and strength of translating health services research
for improving nursing home practice and performance.
Primary Funding Source: NIA, NINR