Cancer Registries and Medical Records Rich Data Resources Carol Lowenstein, MBA, CTR Assistant Director Survey and Data Management Core Dana-Farber Cancer Institute Co-Coordinator, Survey & Statistical Methods Core Objectives • Describe cancer registries • Review the history of cancer registries – why we collect data • Review the types of registries and data available • Understand the varied uses of registry data – where to go, what to use, and the limitations • Explore the value added through linkages of registry data with other data sources • Discuss medical record review vs. cancer registry data • Describe medical record review What Is a Cancer Registry? • Information system designed to: – Collect – Manage – Analyze data on persons with the diagnosis of a malignant or neoplastic disease (cancer). • Cancer registries can be classified into three general types: – Hospital based registries: maintain data on all patients diagnosed and/or treated for cancer at their facility and report cancer cases to the central or state cancer registry as required by law. – Population-based central registries: maintain data on all cancer patients within certain geographical areas. – Special purpose registries: maintain data on a particular type of cancer, such as brain tumors. Source: National Cancer Registrars Association What Is a Cancer Registry? • Registry data is: – Used to make public health decisions – A valuable research tool for those interested in the etiology, diagnosis and treatment of cancer – Used in fundamental research on the epidemiology of cancer Source: National Cancer Registrars Association What Is a Cancer Registry? • Information maintained in a cancer registry: – Demographic Information: Age, gender, race/ethnicity, birthplace and residence. – Medical History: Physical findings, screening information, occupation and any history of a previous cancer. – Diagnostic Findings: Types, dates and results of procedures used to diagnose cancer. – Cancer information: Primary site, cell type and extent of disease. – Cancer Therapy: Surgery, radiation therapy, chemotherapy, hormone or immunotherapy. – Follow-up: Annual information concerning treatment, recurrence, and patient status is updated to maintain accurate surveillance information. Source: National Cancer Registrars Association What Is a Cancer Registry? How are these data used? • Evaluate patient outcome, quality of life, and satisfaction issues and implement procedures for improvement • Provide follow-up information for cancer surveillance • Calculate survival rates by various data items • Provide information for cancer program activities • Analyze referral patterns • Allocate resources at the health care facility, the community, region or state level • Develop educational programs for health care providers, patients and the general public • Report cancer incidence as required under state law • Evaluate efficacy of treatment modalities Source: National Cancer Registrars Association History of Cancer Registries Key Events 2500 B.C. • Earliest known description of "cancer": the "Edwin Smith" and "George Ebers" papyri which describe surgery, pharmacology, and mechanical and magical treatments 400 B.C. • Hippocrates described a breast "cancer" as "karkinoma" (known now as carcinoma) during surgical removal of a tumor 1629 A.D. • Cancer is first mentioned as a cause of death in the Bills of Mortality in England 1728 • London's "General Census of Cancer" - the first known systematic collection of information on cancer is generated Source: NCI SEER Program History of Cancer Registries Key Events 1839 • Implementation of death registration (what we now know as "death certification") in the United States 1901 • Earliest known population-based systematic collection of data on people with leprosy in Norway (a population-based leprosy registry) 1926 • A bone sarcoma registry established by Dr. Ernest Codman at Massachusetts General Hospital, one of the earliest registries established for a specific type of cancer • The first hospital-based cancer registry at Yale-New Haven Hospital was organized in New Haven, Connecticut Source: NCI SEER Program History of Cancer Registries Key Events Ernest Amory Codman, M.D • Kept track of his patients with End Results Cards • Tracked the outcomes of patient treatments • Identified areas of improvement in patient care • Believed that this information should be made public and that this, not seniority, be the basis for physician promotion • Help found the American College of Surgeons and the Joint Commission on Accreditation of Healthcare Organizations Source: Countway Medical Library History of Cancer Registries Key Events 1935 • First population-based cancer registry in the United States established in Connecticut 1956 • The American College of Surgeons requires a cancer registry as a component of an approved cancer program 1971 • The U.S. National Cancer Act budgets monies to the National Cancer Institute for research, detection, and treatment of cancer 1973 • The Surveillance, Epidemiology and End Results (SEER) Program of NCI establishes the first national cancer registry program Source: NCI SEER Program History of Cancer Registries Key Events 1992 • U.S. Public Law 102-515 establishes the National Program of Cancer Registries (NPCR) and is administered by the US Centers for Disease Control and Prevention (CDC) 1993 • Many state laws make cancer a reportable disease Source: NCI SEER Program History of Cancer Registries Cancer Registration • Late 16th Century Prompted by the seemingly random geographic ravages of the plague, the English Crown appointed elderly, epidemic-scarred women to prowl the countryside in search of the dead and dying. These 'Ancient Matrons' published weekly 'Bills of Mortality' for each parish, tabulating deaths by causes such as 'the purples' (probably leukemia), 'riting of the lights', 'consumption' (often an effect of cancer), and of course, the plague. Just how this information was used is not recorded. Perhaps the royalty found it helpful to determine where the plague was active so they could be somewhere else Source: NCI SEER Program History of Cancer Registries Cancer Registration • Mid 17th Century Around 1665 a London businessman, John Graunt, created medical history by subjecting decades of mortality data to critical and mathematical analysis. He literally invented the science of medical epidemiology and statistics, publishing a pamphlet with 108 conclusions. The list included such revolutionary observations as the facts that women saw physicians twice as often as men yet lived longer, and plague epidemics moved outward from swampy areas. Graunt also was the first person to use mortality statistics to project population survival, probably by crudely fitting data samples to a logarithmic curve. Source: NCI SEER Program History of Cancer Registries Cancer Registration •Early 1900’s –Bone sarcoma registry at MGH –Yale-New Haven Hospital Cancer Registry –Other hospitals began to develop registries as physicians saw the benefits of comprehensive data collection –Connecticut population-based cancer registry History of Cancer Registries Cancer Registration • 1937-39 – First National Cancer Survey • • • • • • • • Initiative of the newly formed NCI Directed by Harold F. Dorn Included only MD dx’d cases, histolgic confirmation 3 regions of the US – north, south, west 10 registries in metropolitan areas, MD abstractors Included 10% of the overall population, Resulted in several papers Sparked questions about genetic factors, racial, gender and economic disparities Harold F. Dorn Source: Lilienfeld, Am J Public Health, Dec 2008 History of Cancer Registries Cancer Registration • 1947-48 – Second National Cancer Survey • 1956 – American College of Surgeons Commission on Cancer (CoC) requires hospital cancer registries for Commission approved cancer programs • 1967-71 – Third National Cancer Survey History of Cancer Registries Cancer Registration • 1973 – SEER Registry • 1992 – NPCR funding for state cancer registries History of Cancer Registries Cancer Registration • What were they trying to achieve? – A standard classification of disease – A systematic form of data collection – Analysis and use of the data Sources of Cancer Registry Data • Levels of registries –Hospital –Central –National –International Sources of Cancer Registry Data • Data collected using ICD-O – Based on ICD-9 codes – Separate codes for site (T – topography codes) and histology (M – morphology codes) – Coding manual • Topography numerical • Morphology numerical • Alphabetic index Sources of Cancer Registry Data • Reliance on data standards – Each data item collected has a very specific set of rules • Edits – Developed by national organizations – Applied at hospital and central levels Sources of Cancer Registry Data • Hospital Cancer Registries • Massachusetts Cancer Registry (MCR) • North American Association of Central Cancer Registries (NAACCR) • Surveillance Epidemiology and End Results (SEER) • International Agency for Research o Cancer (IARC) Hospital Cancer Registries • Cancer Reporting Law in MA • Variability in Size • Collect a large data set established by national organizations • Sophisticated edits process Hospital Cancer Registries • Demographic Data - Address, age, gender, race, ethnicity, insurance, census tract, birthplace, occ/ind, tobacco/alcohol hx, family hx • Tumor Data - Primary site, histology, grade, stage, laterality • Treatment Data - Surgery, chemo, XRT, hormone, BRM • Vital Status- Follow up Data • Report to the state/central registry • Within 6 months to 1 year after diagnosis – capture first course of treatment Hospital Cancer Registries • Quality of data – Varies • • • • ACoS vs non-ACoS approved programs Size of hospital Teaching/research vs. community hospital Involvement and interest of MD’s and administration – Standardized data collection software – Standard edits developed by national organizations Hospital Cancer Registries Data Produced • ACos required studies – – – – Requirement of ACoS approval Site specific Quality and outcomes focused Patient care improvements • Annual report • Administrative reports • Ad-hoc reports/requests for data Questions Answered With Hospital Cancer Registry Data • • • • • • • • Who are we serving? Where are patients coming from? How were they referred to our facility? Was access a problem? How sick are they when they’re diagnosed? How are they treated? How well do they fare compared to others? Do we have adequate numbers to support various clinical trials? American College of Surgeons – National Cancer Database (NCDB) • Joint program of the ACoS Commission on Cancer (CoC) and the American Cancer Society • Nationwide oncology outcomes database for more than 1,500 Commission-accredited cancer programs in the United States and Puerto Rico • 70 percent of all newly diagnosed cases of cancer in the United States are captured at the institutional level and reported to the NCDB • Primarily used by CoC-accredited cancer programs as a means by which to evaluate and compare the cancer care delivered to patients diagnosed and/or treated at their facility with that provided at state, regional, and national cancer facilities Massachusetts Cancer Registry • Population based state cancer registry • Data from 1982 • Massachusetts Population – 2010 - Total: 6,547,629 White: 80.4% Black: 6.6% American Indian/Alaskan Native: 0.3% Asian: 5.3% Native Hawaiian-Other Pacific Islander: 0.0% Reporting two or more races: 2.6% Latino or Hispanic Origin: 9.6% White, not Hispanic: 76.1% Massachusetts Cancer Registry Race/Ethnicity Mass US White 80.4% 72.4% Black 6.6 12.6 American Indian and Alaska Native 0.3 0.9 Asian 5.3 4.8 Native Hawaiian and Other Pacific Islander 0.0 0.2 Persons reporting two or more races 2.6 2.9 Hispanic or Latino origin 9.6 16.3 White not Hispanic 76.1 63.7 Massachusetts Cancer Registry • Data collected – same data items as hospital data set • Provides the Abstracting and Coding Manual for Hospitals • Consolidation of information from all reporting sources – Incidence data – each tumor (occurrence of cancer) counted once – Primarily reported from hospitals – Some outpatient/path lab reporting • Data stored at multiple levels – Patient level data – Tumor level data – Treatment level data Massachusetts Cancer Registry Patient A Name Address Age etc. Tumor 1 Site Histology Stage etc. Treatment A CTX Treatment B Surgery Tumor 2 Site Histology Stage, etc. Treatment A CTX Treatment B XRT Massachusetts Cancer Registry Consolidation Process • Algorithms to dictate – Match – Non-match – Review • Consolidation rules • Multiple primary rules – Lymphomas and leukemias – Laterality – Benign vs. malignant Massachusetts Cancer Registry Consolidation Process Patient A Hospital 1 Patient A Hospital 2 Name: Jones Name: Jonnes Primary Site: LLL Lung Histology: Small cell Primary Site: LUL Lung Histology: Adenoca Massachusetts Cancer Registry Consolidation Process Hospital A Name: Jones Primary site: LLL Lung Histology: Small Cell Consolidated Patient Record Name: Jones Consolidated Tumor Record Site: LLL Lung Histology: Adneoca Hospital B Name: Jonnes Primary site: LUL Lung Histology Adenoca Massachusetts Cancer Registry Consolidation Process • Very time consuming! • Always looking for ways to further automate • Also involves real-time edit resolution – Males with endometrial ca – Impossible/rare site-histology combinations – Age flags Massachusetts Cancer Registry Data - Reports • Two main reports: – Cancer Incidence and Mortality in Massachusetts – The City/Town Supplement • Special reports – Cervical and Uterine Cancers in MA – Cancer Incidence and Mortality in Boston Neighborhoods – Childhood Cancers in MA – Cancer in MA by Race and Ethnicity – … and more on their website Massachusetts Cancer Registry Data - Reports • MassCHIP - Massachusetts Community Health Information Profile – Online access to health and social indicators – http://www.mass.gov/dph/masschip • Linkages and data requests – Linkages with data sets provided to them – Approval for all data requests by the MDPH Human Research Review Committee Massachusetts Cancer Registry Data - Reports • MCR website: www.mass.gov/dph/mcr Questions Answered With Massachusetts Cancer Registry Data • What is the race, gender, age breakdown for cancers in the state? • What do certain areas look like in terms of cancer type and demographic variables? • How does our hospital population compare to the state in terms of stage at diagnosis? • Are there sufficient numbers of patients to justify a new service or to open a trial? North American Association of Central Cancer Registries • Promotes uniform data standards • Provides education and training; certifies population-based registries • Gold and Silver status • Aggregates and publishes data from central cancer registries • Promotes the use of cancer surveillance data North American Association of Central Cancer Registries • All central cancer registries in the United States and Canada are members • Cancer In North America (CINA) • Published annually • Contains: general demographic information about each state, data quality indicators, site specific incidence rates • Individual states • US Combined Report North American Association of Central Cancer Registries - Reports • • • • • Cancer Incidence in North America (CINA) Monographs - four published volumes of cancer statistics for a five-year period CINA+ Online - an on-line query system utilizing CINA data CINA Monograph Data in SAS Data Set - published data in CINA Monograph (all four volumes) available for use in SAS CINA Plus in SEER*Stat - data from CINA+ Online that are available in SEER*Stat software, allowing more flexible queries and statistical testing. Only available to NAACCR members or collaborators with NAACCR members. CINA Deluxe - a multi-registry NAACCR members analytical data file for cancer researchers that includes more detailed data variables and groupings. Only available to NAACCR members or collaborators with NAACCR members. CDC National Program of Cancer Registries (NPCR) • Established by Congress through the Cancer Registries Amendment Act in 1992 • Provided funding, supplemental and start up, to state cancer registries • Contracted with NAACCR to provide guidance in the areas of data standards and quality control • In collaboration with the NAACCR, produce the United States Cancer Statistics: Incidence and Mortality report NPCR - Program Contacts by Funding Status NCI SEER • Data from 18 population based central cancer registries • Covers approximately 26% of the US population • 23.4% of Whites • 22.7% of African Americans • 40.4% of Hispanics • 42.2% of American Indians and Alaska Natives • 53.3% of Asians • 69.8% Hawaiian/Pacific Islanders NCI SEER NCI SEER NCI SEER Goals of the SEER Program • Collect data • Conduct quality control and quality improvement programs • Report on the cancer burden in selected segments of the population • Identify unusual changes and differences in patterns of occurrence • Describe temporal changes in the extent of disease at diagnosis, trends in therapy and changes in survival • Monitor the occurrence of possible cancers that are caused by cancer therapy NCI SEER Goals of the SEER Program • Collaborate with other organizations on cancer surveillance activities, data standards and training • Serve as a research resource to the NCI regarding studies dealing with cancer prevention and control, program and registry operations • Provide research resources to the general research community • Provide training materials and web-based training resources to the cancer registry community. NCI SEER Data Available • SEER data are updated annually • Provided in print and electronic formats • SEER web site (http://seer.cancer.gov) • Reports on cancer statistics • Instructions on how to access the SEER public use file free of charge on a CD-ROM or access provided through a server at NCI. • Software provided free of charge by NCI that aids in the analysis of the SEER database (SEER*Stat) and can be used to analyze data from other cancer registries (SEER*Prep and SEER*Stat). NCI SEER Data Available Three major categories of data: 1. Cancer Statistics Web-based systems provide access to statistical tables, graphs, and maps from various data sources. 2. Datasets and Software Use SEER data to address multiple issues, such as looking at ne stage at diagnosis by race/ethnicity, calculating survival by stage at diagnosis, age at diagnosis, and grade or size of tumor, or determining trends and incidence rates of cancers at various sites over time. 3. Publications The statistical publications provided were produced by the SEER program or through collaborative efforts between SEER and other organizations. NCI SEER Data Available – Cancer Statistics • Cancer Stat Fact Sheets Printable summaries of the latest cancer statistics for major cancer types. • Cancer Statistics Review (CSR), 1975-2008 Detailed report published annually to present a broad profile of cancer statistics to the public. • Fast Stats Build your own tables and graphs of key SEER and US cancer statistics. • Cancer Query Systems Provides more flexibility and a larger set of cancer statistics than Fast Stats but requires more input from the user. • State Cancer Profiles Dynamic maps and graphs enabling the investigation of cancer trends at the county, state, and national levels NCI SEER Data Available – Datasets and Software • SEER*Stat version 7.0.5 (09/26/2011) – Statistical software that provides a convenient, intuitive mechanism for the analysis of SEER and other cancer-related databases. It is a powerful personal computer tool to produce statistics for studying the impact of cancer on a population. • SEER*Prep software version 2.4.5 (2/7/2011) – Software that converts user-supplied ASCII text data files to the SEER*Stat database format, allowing analysis of populationbased registry data using SEER*Stat. • Health Disparities Calculator (HD*Calc) version 1.1.0 (01/13/2010) – An extension of SEER*Stat designed to generate multiple summary measures to evaluate and monitor health disparities. NCI SEER Data Available – Publications • Statistical Reports – SEER Cancer Statistics Review (CSR) – Annual Report to the Nation on the Status of Cancer – Cancer Incidence and Mortality Patterns among Specific Asian and Pacific Islander Populations in the US – United States Cancer Statistics: Cancer Incidence and Mortality Data • Monographs (recent) – An Update on Cancer in American Indians and Alaska Natives, 19992004 – Selected Comparisons of Measures of Health Disparities: A Review Using Databases Relevant to Healthy People 2010 Cancer-Related Objectives – SEER Survival Monograph: Cancer Survival Among Adults: US SEER Program, 1988-2001, Patient and Tumor Characteristics NCI SEER Linked Databases • SEER-Medicare Linked Database Links SEER and Medicare data. • SEER-Medicare Health Outcomes Survey Linked Database Links SEER and the Medicare Health Outcomes Survey (MHOS) that provides information about the health-related quality of life (HRQOL) of Medicare Advantage Organization (MAO) enrollees. • National Longitudinal Mortality Study (NLMS) and Linked SEER-NLMS Databases SEER registry cancer patient records for 1973-2003 are linked with the NLMS database, which was developed to examine the relationships between self-reported demographic and socioeconomic characteristics and mortality among a representative sample of the US non-institutionalized population. NCI SEER Linked Databases – SEER-Medicare SEER data Clinical, demographic and cause of death information for persons with cancer Medicare claims data Covered health care services from the time of a person's Medicare eligibility until death Unique population-based source of information Epidemiological and health services research Why link the SEER-Medicare Data? The linked data can be used for a number of analyses that span the course of cancer control activities Diagnosis/ Tx Survivorship Second Occurrence Terminal Care Patterns of care Late effects of treatment Rates of second primaries Use of hospice services Post-diagnostic surveillance Relationship of second events to initial treatment and ongoing surveillance Patterns of care during the last year of life Peri-operative complications Volume outcomes studies Extent of staging Treatment of prevalent cancers Comorbidities Survival Source: Warren, NAACCR,NCI SEER-Medicare Data • SEER Data – Incidence, site, stage, initial tx, demographics and vital status • Medicare claims – – – – Short stay hospitals Physician and lab services Hospital outpatient claims Home health and hospice bills Source: Warren, NAACCR,NCI SEER-Medicare Data • Limitations – Non-covered Medicare services are not included – Reasons for ordering tests and test results are not included – Primarily over 65 population NCI SEER SEER – RTR Program • Residual Tissue Repository (RTR) Program Established in 2003, the RTR maintains bio specimens obtained from three of SEER’s population-based cancer registries: Iowa, Hawaii, and Los Angeles. Investigators at government, academic, and nonprofit institutions may apply to the program to obtain specimens to study biomarkers, etiology, and other aspects with a population-based sample of cancer cases. Questions Answered With NAACCR and SEER data • How is Massachusetts similar or different than other parts of the country? • How does Massachusetts compare in terms of age at diagnosis, stage and outcomes? • Where else might there be special populations or rare cancers of interest? • Where might there be the potential for collaboration regarding these populations? International Agency for Research on Cancer - IARC • WHO • Data held and managed by the Section on Cancer Information (CIN) of IARC • Cancer in Five Continents – Published every year since 1966 • CIN Website – CANCERMondial http://www-dep.iarc.fr/ IARC • CIN Databases – GLOBCAN provides access to the most recent estimates (for 2008) of the incidence of, and mortality from 27 major cancers worldwide. – CI5-Cancer Incidence in Five Continents provides access to detailed information on the incidence of cancer recorded by cancer registries (regional or national) worldwide. – ACCIS (Automated Childhood Cancer Information System) provides access to data on cancer incidence and survival of children collected by European cancer registries. IARC • CIN Databases (cont.) – ECO (European Cancer Observatory) provides access to the estimates (for 2008) of the incidence of, and mortality from 25 major cancers in the countries of the European Union (EU-27), together with a quick overview of on-going programmes of screening for breast, cervical and colorectal cancers implemented in the 27 European countries. – NORDCAN presents up-to-date long time series of cancer incidence, mortality, prevalence and survival from 40 cancers recorded by the Nordic countries, together with advanced prediction facilities. – SurvCan presents cancer survival data from cancer registries in low and middle income regions of the world. (Africa, Asia, the Caribbean and Central America) Questions Answered With IARC data • How is the US, MA, Boston similar or different than other parts of the country? • Where else might there be special populations or rare cancers of interest? • Where might there be the potential for collaboration regarding these populations? Rapid Case Ascertainment • Provides the ability to identify cancer cases shortly after diagnosis • Central registry editing and consolidation can be time consuming • Located within central cancer registries and/or at academic medical centers Rapid Case Ascertainment Value to researchers: provides immediate access to patients • Interview patients prior to surgery • Obtain blood/DNA etc. • Initiate a series of questionnaires over time (i.e. 3, 6 and 9 months after diagnosis) • Interview patients about lifestyle elements that might tend to change after diagnosis (alcohol consumption, smoking history) SEER Rapid Response Surveillance Studies • Began as Patterns of Care Studies • Evolved into a mechanism that allows studies to move from the initial concept through completion in a relatively short interval, usually within two years • Studies address new and emerging issues related to cancer prevention and control • Conducted through he SEER Registries • Largely pathology based active case-finding (e-path or circuit riding) vs. waiting for reports to be sent to the central registry Limitations of Using Cancer Registry Data • Limited data set – extensive, but limited • Strict rules for data collection – good, but restrictive • Time delays – data can be 1 to 2 years old • Data collected and defined by the priorities of others • Dataset designed by committee Limitations of Using Cancer Registry Data • Reliance on primarily pathology based case-finding – image and lab based dx’s may not be as readily picked up (pancreas, brain, multiple myeloma) • Majority of cases reported by hospitals – some lab and outpatient reporting • Data can be limited by information captured only through the hospital system • Outpatient information is often missing – XRT at freestanding centers, CTX in MD offices Limitations of Using Cancer Registry Data • Lack of information about events leading up to dx – screenings, health behaviors • Limited information about co-morbid conditions • Follow-up information often limited to vital status – no detailed information on side effects to tx, tx compliance • Very little information about recurrence of disease Medical Record Review • Advantages – Flexibility – Timeliness – Customized data set – Investigator control Medical Record Review – Recent Examples • Helping Ourselves Helping Others PI: Ann Partridge, MD Breast cancer in women 40 and under Captured within 16 weeks of dx Cases identified and reviewed at DFCI, MGH, BWH, NWH, Faulkner, BIDMC, North Shore Cancer Center, Lowell General Medical Record Review – Recent Examples • The Environment, Genes and Testicular Cancer PI: Russ Hauser, MD Newly diagnosed cases of testicular cancer Identified shortly after dx at DFCI, MGH, BWH, Faulkner, North Shore, BIDMC, Boston Medical Center, Lowell General, Tufts-NEMC Medical Record Review – Process • Plan data set with investigator/study team – Define data elements and data definitions (coded 1, 2 responses vs. yes, no) – Think about the future • Will this data be linked or compared to other data sets – should race and ethnicity be collected using cancer registry/census rules • What is being measured – can’t evaluate data that hasn’t been collected Medical Record Review – Process • Design/program the data collection tool – Access, Excel, other programs – Consider output – where is the data goingother databases, are the variables the same Medical Record Review – Process • Define rules for: – Missing data – Incomplete data – Illegible data • Keep in mind – Medical records are not designed for research, there are limits to what they contain Medical Record Review – Process • Develop case identification procedures – Review eligibility criteria – Identify facilities – Case sources • • • • Pathology Radiology New patient admission/visits Clinic logs – Access to case sources • Who has access • How does one obtain access • Is training required Medical Record Review – Process • Consider the frequency of data collection – Is time to dx important – Are there other patient or patient flow events to consider • How often do patients come in for treatment • Do they stay in the hospital • Is there a planned schedule of testing – Many cases will need to be reviewed to find eligible cases – this can be time consuming Medical Record Review – Process • Work with investigator/study team on IRB issues – Provide background information on personnel – Provide information about methods for identifying cases • Build in quality control measures – Computerized edits – Double abstraction (5-10%, inter rater reliability > 95%) Abstraction Process Research Hypothesis Define Variables Guidelines Staff Revisions Exploration Instrument Development Logic checks Default values Pilot testing Plans for Case Identification Record review Quality Mgmt Adapted from Keyzer, UC Davis What Would They Think? Progress • Amount of work in Outcomes Research • Focus on End Results Reporting, particularly Comparative Effectiveness Research • Accountability • Evidence based practices • Research based interventions Questions?