Re: Public Comment on Draft Methodology Report: “Our Questions, Our Decisions:
Standards for Patient-centered Outcomes Research”
On behalf of Siemens Healthcare USA, an integrated manufacturer of imaging and lab
diagnostics, we would like to offer the following comments in response to PCORI’s
request for comments on its ground-breaking draft methodology report. In particular we
would like to offer one suggestion on how to make PCORI-funded research more
transparent in the focus on patient outcomes and two thoughts on making the analysis of
diagnostic tests more rigorous.
Enhanced Transparency
Transparency in understanding the type of analysis embedded within each PCORIfunded research study is important. Today, comparative effectiveness research is
typically done from the perspective of the payer rather than from a patient or a societal
perspective. At times the payer perspective is implicit in the analysis but not explicitly
identified as such. While the payer perspective in comparative effectiveness is certainly
valuable, it does not include an analysis of concerns important to patients like freedom
from pain, ability to work or to pursue daily activities, nor life expectancy beyond the
study evaluation period. Similarly the payer perspective would typically not consider or
fully evaluate important societal issues such as workforce productivity, long-term
healthcare costs and long-term non-healthcare costs which are the central issues with
chronic illnesses such as diabetes, coronary artery disease, congestive heart failure and
COPD.
One simple way to address this issue and ensure that the PCORI-funded research
methods are in essence branded as such, would be to require research journal articles
based on PCORI-funded comparative effectiveness research to state in the abstract of
each publication whether the analysis was done from a “patient perspective”, a “payer
perspective” or a “societal perspective”. This would likely have the effect of shifting
PCORI research to analyses that are patient-centered rather than payer-centered.
Importantly, requiring PCORI-funded CER to state the research perspective in the
abstract of each article would provide an important measure of transparency for the
research methods and assumptions. While savvy researchers can evaluate the
perspective from reading the methods section of a paper, having this information in the
abstract will, with the information load of hundreds of thousands of papers published
annually, make this information available to patients, journalists and decision-makers
who rarely have the time or skills to delve into analytic bias.
Diagnostic Test Evaluation
The PCORI research committee is to be commended for identifying that diagnostic test
evaluation entails additional challenges. In particular, many modern tests are done
because they provide extraordinary amounts of clinical information. Modern crosssectional imaging such as CT or MRI scans have benefited from the same information
explosion as other computer hardware and software driven services; next-generation
sequencing and similar high-throughput diagnostics are impossible without vast
information flows. It seems that diagnostic test evaluation for these high information
density tests should measure the full value of this additional information spectrum.
Current diagnostic test evaluation methodologies typically evaluate a test for one-time
impact on a single disease or treatment. However, these cross-sectional tests are often
ordered to evaluate a wide set of potential diagnoses. With EMR’s and information
exchanges providing more access to historical data, tests results also have increasing
durability over time – in particular for studies looking at disease burden in the vascular
tree.
PCORI diagnostic test evaluation research should be structured to look at the impact of
the test over the entire potential differential diagnosis not just one particular disease.
For example cardiac CT angiography done to look for coronary artery disease can
potentially also provide information on other illnesses such as valvular heart disease,
lung cancer, aortic dissection, pulmonary embolus, liver disease, and bone density.
While many of the items in the differential diagnosis are individually very low probability
cumulatively all of these low probability possibilities add up. This impact of individually
rare but collectively common instances is inherent with deep and broad information
sources. In particular, this phenomenon, which has been described as a “long tail” is
central to the efficiency of information in deeply customer-focused businesses such as
Amazon, Google and eBay.
Importantly, the “long tail” phenomenon also describes the distribution of illnesses in the
Medicare patient population.1 Test evaluation methodologies which do not incorporate
explicit evaluation of the entire differential significantly underestimate the value of the
test to the patient. One possible explanation for the disconnect between some health
services researchers’ low valuation of cross-sectional imaging compared to that of
ordering physicians (who often have no or even a negative incentive to order these
tests) is that the ordering physicians are responsible for evaluating the entire differential
diagnosis while researchers can focus on a single disease process or component.
PCORI diagnostic test evaluation research should also be structured to look at the
impact of the test over an adequately long time period. The imperative of clinical trials
often leads to evaluation of tests or treatments over six months or a year or two. While
research over short time periods is important to private payers with rapid turnover in their
insured populations, most patients and Medicare can statistically expect much longer life
expectancies than the length of typical study durations so PCORI studies should model
the impact of the information provided by diagnostic tests over the entire clinically
relevant duration of the information provided by the test. For example, one recent study
looking at cardiac CT angiography for an increased risk population stopped at 18 months
post-test and concluded that there was no benefit to this test.2 Interestingly, patients in
this study with a positive CT result had roughly triple the compliance with statin and
aspirin therapy than did controls or patients with a negative result. Unless one
concludes that this study shows statins and aspirin do not impact heart disease, one
could presume that this study came to an erroneous conclusion from a patient-centered
perspective simply because of the short study duration.
Appropriate consideration of the value over time is of particular note as the volume of
information rises because tests such as gene-sequencing, protein expression, and
cross-sectional imaging provide potential information with applicability for the duration of
the patient’s life. Since these tests generate storable data sets, the underlying
sequence or structure data even offers the opportunity to be reanalyzed later based on
new biologic insights. With recent laws, patients themselves are increasingly gaining
actionable access to their own clinical data so one can expect the impact of test
information on patient lifestyle and clinical care choices to grow.
Evaluation of tests over the entire differential diagnosis will require changes in evaluating
outcomes and evaluation of tests over longer periods of time may require modeling
future disease states as well as more use of discount rates.
Thank you for the opportunity to comment on increasing patient-centered research
transparency through explicit “abstract-level” identification of analytic perspective.
Thank you also for the opportunity to suggest evaluation of high information content
diagnostic tests based on their patient impact across the entire differential diagnostic
range and for the entire length of clinically relevant time.
Donald W. Rucker, MD
Vice President and Chief Medical Officer
Siemens Healthcare USA
August 31, 2012
1.
Sorace J, Wong HH, Worrall C, Kelman J, Saneinejad S, MaCurdy T. The
complexity of disease combinations in the Medicare population. Popul Health Manag.
Aug;14(4):161-166.
2.
McEvoy JW, Blaha MJ, Nasir K, et al. Impact of coronary computed tomographic
angiography results on patient and physician behavior in a low-risk population. Arch
Intern Med. Jul 25;171(14):1260-1268.