APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY MAIN BIBLIOGRAPHY rd i 3 Annual APA Conference Advancing Quality Improvement Science for Children’s Healthcare Research FINAL 04/14/2013 Australian Institute of Health Innovation http://www.aihi.unsw.edu.au/ SECTION II: Journals that publish QI research Implementation Science May 3, 2013 Washington, DC International Journal of Healthcare Quality and Safety Note: Since the First Annual APA Conference, the literature in healthcare quality improvement has grown dramatically. This bibliography is necessarily selective and highlights recent additions to the literature, as well as very selected classics with general guidance, and required and suggested reading for the breakout sessions. We have tried to emphasize pediatric examples. Previous bibliographies are still available, both for the conference breakout sessions and the ii supplementary bibliographies. Suggestions for additions to the bibliography can be shared with members of the Conference Planning Committee. BMJ Quality and Safety Health Care: The Journal of Delivery Science and Innovation www.elsevier.com/locate/healthcare eGEMS. http://repository.academyhealth.org/ege ms. A pragmatic approach to building a learning health care system, eGEMS aims to disseminate innovative ideas about how electronic clinical data can be leveraged in comparative effectiveness research, PCOR and quality improvement. SECTION I: SOME USEFUL WEBSITES AND LISTSERVS 1 Implementation Network e-Newsletter – contact Wynne E. Norton, PhD Assistant Professor University of Alabama at Birmingham wynne.norton@gmail.com UK Health Foundation – Inspiring Improvement – evidence reviews, job announcements, and more http://www.health.org.uk/ Improvement Science Research Network. www.isrn.net Training in Dissemination and Implementation Research in Health (TIDIRH) online training site sponsored by NIH. http://tidirh2011.ning.com/ Global Implementation Initiative. http://globalimplementation.org/collaboration SECTION III: Books Brownson R, Colditz G, Proctor E, eds. Dissemination and Implementation Research in Health. New York: Oxford; 2012. o Moen, R., Nolan, T., Provost, L.P. Quality improvement through planned experimentation. nd 2 edition 1999; McGraw-Hill, NY. o Provost, L., Murray, S. The data guide: learning from data to improve health care. Associates in Process Improvement, 2010. APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY SECTION IV: SPECIAL ISSUES OF JOURNALS, 1 SUMMARIES OF QI PROJECTS 1 Forthcoming special issue from Academic Pediatrics on Pediatric QI – commentaries, research methods, QI education, site-specific QI. (expected Nov-Dec 2013). Kleinman, L., & Runyan, D. (2013). Moving the discourse on quality in pediatrics: Recent contributions of Robert Wood Johnson Foundation Clinical Scholars [Introduction to esupplement]. Pediatrics, 131. Kleinman LC (Editor), Chung PJ, Fieldston ES, Fiks AG, Rosenthal MS, Tarini BA (Associate Editors). “Visioning, Measuring, and Improving the Quality of Health Care for Children: Insights from the Robert Wood Johnson Foundation Clinical Scholars Program. Supplement to Pediatrics 2013 13: Supplement S1-S119. Holve, E. (2013). Ensuring support for research and quality improvement (QI) networks: Four pillars of sustainability--An emerging framework. eGEMs (Generating Evidence & Methods to improve patient outcomes), 1(1). Papers from the epistemology of improvement research meeting, UK, April 12-16. BMJ Quality and Safety. 2011; 12:(suppl 1). http://qualitysafety.bmj.com/content/20/Suppl_1 The Health Foundation. (2011). Evidence: Helping people help themselves. In Author (Ed.). London: Author. SECTION V: THEORY AND LOGIC MODELS IN HEALTHCARE QUALITY IMPROVEMENT Kleinman LC, Dougherty D. Assessing Quality Improvement in Health Care: Theory for Practice. Pediatrics 2013; 131:S110-S119. Kitto, S., & et al. (2013). Positioning continuing education: boundaries and intersections between the domains continuing education, knowledge translation, patient safety and quality improvement. [Reflections]. Adv in Health Sci Educ, 18, 141-156 May, C. (2013). Towards a general theory of implementation. Implementation Science, 8. Gibb, H. (2013). An environmental scan of an aged care workplace using the PARIHS model: assessing preparedness for change. J Nurs Manag, 21(2), 293303 Robert Wood Johnson Foundation. Improving the Science of Continuous Quality Improvement Program and Evaluation: An RWJF National Program. http://www.rwjf.org/content/dam/farm/reports/pro gram_results_reports/2012/rwjf73230. This report summarizes the process and outcomes of an array of projects aimed at improving evaluation frameworks, quality improvement measures, and data collection and methodology. The projects were part of an RWJF program costing $1.5 million over 48 months, from August 2007-August 2011. Nickel, N., Taylor, E., Labbok, M., Weiner, B., & Williamson, N. (2013). Applying organisation theory to understand barriers and facilitators to the implementation of baby-friendly: A multisite qualitative study. Midwifery English, M. (2013). Designing a theory-informed, contextually appropriate intervention strategy to improve delivery of paediatric services in Kenyan hospitals. Implementation Science, 8(39). Goeschel, C., Weiss, W., & Pronovost, P. (2012). Using a logic model to design and evaluate quality and patient safety improvement programs. Int J Qual Health Care, 24(4), 330-337. Dixon-Woods M, Bosk C, Aveling E, Goeschel C, Pronovost P. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Quarterly. 2011;89(2):167-205. See below for systematic evidence reviews 2 APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. Kaplan HC, Provost LP, Froehle CM, Margolis PA. BMJ Qual Saf. 2012 Jan;21(1):13-20. doi: 10.1136/bmjqs-2011-000010. Epub 2011 Aug 10. Damschroder, L., Aron, D., Keith R., et.al. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science. 2009; 4(50). Grol, R.P., Bosch, M., Hulscher, M., et al. Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Q. 2007; 85(1):93-138. Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJWhat Does It Mean to ''Employ'' the RE-AIM Model? Eval Health Prof. 2012 May 21. . Bakken, S., Ruland, C.M. Translating clinical informatics interventions into routine clinical care: How can the RE-AIM framework help? Journal of the American Medical Informatics Association. 2009; 16(6). SECTION V. CONTEXT AND IMPLEMENTATION PROCESSES IN HEALTHCARE QUALITY IMPROVEMENT INTERVENTIONS 3 healthcare professional practice. Implementation Science, 8. Mittman B, Ovretveit J, Plsek P, Salem-Schatz S. How To Think About Evidence When Deciding Whether To Adopt an Innovation Health Care Innovations Exchange 2013; http://www.innovations.ahrq.gov/content.aspx?id=38 37&utm_source=issueanc&utm_medium=email&utm _campaign=20130227. Accessed Feb. 27, 2013. Flottorp, S., & et al. (2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies that prevent or improvements in Brennan, S., & et al. (2013). Measuring team factors thought to influence the success of quality improvement in primary care: a systematic review of instruments. Implementation Science, 8(20). Rycroft-Malone, J., & et al. (2013). The role of evidence, context, and facilitation in an implementation trial: Implications for the development of the PARiHS framework. Implementation Science, 8(28). Beehler, G., & et al. (2013). Developing a measure of provider adherence to improve the implementation of behavioral health services in primary care: a Delphi study. Implementation Science, 8(19). The Health Foundation. Safety Culture: What Is It and How Do We Monitor and Measure It? http://www.health.org.uk/public/cms/75/76/313/4156 /Safety%20culture%20%20What%20is%20it%20and%20how%20do%20w e%20monitor%20and%20measure%20it.pdf?realN ame=xn5DLO.pdf Lee, S.-Y. D., Weiner, B., Harrison, M., & Belden, C. (2013). Organizational transformation: A systematic review of empirical research in health care and other industries. Med Care Res Rev, 70. Chaudoir S, Dugan A, Barr C. Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science 2013;8(22). Leu M, et al. Systematic Update of Computerized Physician Order Entry Sets. Pediatrics 2013. Rabin, B., Purcell, P., Naveed, S., Moser, R., Henton, M., Proctor, E., . . . Glasgow, R. (2012). Advancing the application, quality and harmonization of implementation science measures. Implementation Science, 11(7). APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY Kaplan HC, Froehle CM, Cassedy A, Provost LP, Margolis PA. An exploratory analysis of the Model for Understanding Success in Quality. Health Care Manage Rev. 2012 Aug 20. [Epub ahead of print] PMID: 22914176 Duan, N. Description of ideal evaluation methods: quantitative approaches to context heterogeneity in assessing the evidence for context-sensitive effectiveness and safety of patient safety practices: Developing criteria. Contract Final Report. 2011. http://www.ahrq.gov/qual/contextsensitive/context12.ht m Kolehmainen N, Francis JJ. Specifying content and mechanisms of change in interventions to change professionals' practice: an illustration from the Good Goals study in occupational therapy. Implement Sci. 2012 Oct 18;7:100. doi: 10.1186/1748-5908-7-100. Eccles M et al. Explaining clinical behaviors using multiple theoretical models. Implem Sci 2012 7:99. Chan, K., Hsu, Y. J., Lubomski, L., & Marsteller, J. (2011). Validity and usefulness of members’ reports of implementation progress in a quality improvement initiative: findings from the Team Check-up Tool (TCT). Implementation Science, 3. Michie S, Fixsen D, Grimshaw, Eccles MP. Specifying and reporting complex behavior change interventions: the need for a scientific method. Implement Sci 2009 4(40). Carroll C, et al., A conceptual framework for implementation fidelity. Implement Sci 2007, 2:40. Green, L.W., Glasgow, R.E. Evaluating the relevance, generalization, and applicability of research: Issues in external validation and translation methodology. Eval Health Prof. 2006; 29:126. http://ehp.sagepub.com/content/29/1/126 Green, L. W., Glasgow, R. E., et al. Making evidence from research more relevant, useful, and actionable in policy, 4 program planning, and practice slips "Twixt Cup and Lip." American Journal of Preventive Medicine. 2009; 37(6):S187-S191. Hawe, P., Shiell, A., Riley T., et al. Methods for exploring implementation variation and local context within a cluster randomized community intervention trial. J Epidemiol Community Health. 2004; 58:788– 793. Shekelle, P.G., Pronovost, P.J., Wachter, R.M., et al. Assessing the evidence for context-sensitive effectiveness and safety of patient safety practices: developing criteria. AHRQ Publication No. 11-0006EF Contract No. HHSA-290200910001C.December 2010. Also see references included in Appendix E (Chapter 9) from the Shekelle et al AHRQ publication. http://www.ahrq.gov/qual/contextsensitive/context.p df SECTION VI. SPECIFIC RESEARCH DESIGNS AND METHODS FOR EVALUATING QUALITY IMPROVEMENT INTERVENTIONS-- FOCUS ON iii INTERNAL VALIDITY Statistical Process Control (SPC) o Parker MW et al. Rapid Adoption of Lactobacillus rahmnosus GG for Acute Gastroenteritis. Pediatrics 2013: 131:S96-102.’ o Mazzocato P, Holden RJ, Brommels M, Aronsson H, Bäckman U, Elg M, Thor J How does lean work in emergency care? A case study of a lean-inspired intervention at the Astrid Lindgren Children's hospital, Stockholm, Sweden.BMC Health Serv Res. 2012 Feb 1;12:28. doi: 10.1186/1472-696312-28. Used a mixed methods single case study design including ANOVA and SPC. APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY o o Mdege ND, Mana M, Taylor CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate intervention during routine implementation. Journal of Clinical Epidemiology. 2011; 64:936-948. o Glynn RJ, Brookhart AM, Stedman M, Avorn J, Solomon DH. Design of clusterrandomized trials of quality improvement interventions aimed at medical care. Med Care. 2007; 45:S38–S43. Rinke ML, Chen AR, Bundy DG, Colantuoni E, Fratino L, Drucis KM, Panton SY, Kokoszka M, Budd AP, Milstone AM, Miller MR Implementation of a central line maintenance care bundle in hospitalized pediatric oncology patients. Pediatrics. 2012 Oct;130(4):e996-e1004. doi: 10.1542/peds.2012-0295. Epub 2012 Sep 3. o Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. 2012 Contemporary Clinical Trials 28:182-191. o Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Medical Research Methodology 2006, 6:54. Biomedcentral. Kaplan HC, Lannon C, Walsh MC, Donovan EF; Ohio Perinatal Quality Collaborative Ohio statewide qualityimprovement collaborative to reduce late-onset sepsis in preterm infants. Pediatrics. 2011 Mar;127(3):427-35. doi: 10.1542/peds.2010-2141. Epub 2011 Feb 21. Castello FV, Maher A, Cable G Pediatrics. Reducing bloodstream infections in pediatric rehabilitation patients receiving parenteral nutrition 2011 Nov;128(5):e1273-8. doi: 10.1542/peds.2010-3617. Epub 2011 Oct 24. Realist Evaluation Kotz et al., Use of the stepped wedge design cannot be recommended: a critical appraisal and comparison with the classic cluster randomized controlled trial design. 2012 Journal of Clinical Epidemiology 65:1249-1252. Best A., Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-system transformation in health care: a realist review. Milbank Q 2012 Sep; 90(3):42156. Cluster Randomized Controlled Trials o Rycroft-Malone J, Seers K, Crichton N, Chandler J, Hawkes CA, Allen C, Bullock I, Strunin L. A pragmatic cluster randomised trial evaluating three implementation interventions. Implement Sci. 2012 Aug 30;7:80. o James W. Stout, MD, MPH; Karen Smith, MD; Chuan Zhou, PhD; Cam Solomon, PhD; Allen J. Dozor, MD; Michelle M. Garrison, PhD; Rita Mangione-Smith, MD, MPH. Learning Stepped Wedge o 5 o Interrupted Time Series o Morris AC, Hay AW, Swann DG, Everingham K, McCulloch C, McNulty J, Brooks O, Laurenson IF, Cook B, Walsh TS. Reducing ventilator-associated pneumonia in intensive care: impact of implementing a care bundle. Crit Care Med. 2011 Oct;39(10):2218-24. doi: 10.1097/CCM.0b013e3182227d52. Mozza@doctors.org.uk APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY Bickman L, Kelley SD, Breda C, de Andrade AR, from a Distance: Effectiveness of Online Spirometry Training in Improving Asthma Care. ACADEMIC PEDIATRICS 2012;12:88–95 Riemer M Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial Psychiatr Serv. 2011 Dec;62(12):1423-9. doi: 10.1176/appi.ps.002052011. Berwanger B, Guimaraes HP, Laranjeira LN, Cavalcanti AB et al. Effect of a multifaceted intervention on use of evidence-based therapies in patients with acute coronary syndromes in Brazil. The BRIDGE-ACS Randomized Trial. JAMA. 2012; 307(19)E1E9. Walleser S, Hill SR, Bero LA Characteristics and quality of reporting of cluster randomized trials in children: reporting needs improvement J Clin Epidemiol. 2011 Dec;64(12):1331-40. doi: 10.1016/j.jclinepi.2011.04.006. Epub 2011 Jul 20. Selby, J., & et al. (2012). Improving treatment intensification to reduce cardiovascular disease risk: a cluster randomized trial. BMC Health Serv Res., 12(183). Dreischulte T, Grant A, Donnan P, McCowan C, Davey P, Petrie D, Treweek S, Guthrie B. A cluster randomised stepped wedge trial to evaluate the effectiveness of a multifaceted information technology-based intervention in reducing high-risk prescribing of non-steroidal anti-inflammatory drugs and antiplatelets in primary medical care: The DQIP study protocol. Implementation Science. 2012, 7:24 Glynn, R.J., Brookhart, M.A., Stedman, M. et al. Design of cluster-randomized trials of quality improvement interventions aimed at medical care providers. Med Care. 2007; 45: S38–S43. http://www.ncbi.nlm.nih.gov/pubmed/17909381 o Acolet D, Allen E, Houston R, Wilkinson AR, Costeloe K, Elbourne D Improvement in neonatal intensive care unit care: a cluster randomised controlled trial of active dissemination of information Arch Dis Child Fetal Neonatal Ed. 2011 Nov;96(6):F434-9. doi: 10.1136/adc.2010.207522. Epub 2011 Mar 10. Kahan, B., & Morris, T. (2012). Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. [Research]. BMJ Qual Saf, 345, e5840. Scales,D.C., Dainty,K., Hales,B., et al. Multifaceted Intervention for Quality Improvement in a Network of Intensive Care Units A Cluster Randomized Trial. JAMA, 2011; 305(4):363-372. http://jama.amaassn.org/content/305/4/363.full Guthrie, L., Oken, E., Sterne, J., Gillman, M., Patel, R., Vilchuck, K., . . . Martin, R. (2012). Ongoing monitoring of data clustering in multicenter studies. BMC Medical Research Methodology, 12, 29. Epstein JN, Langberg JM, Lichtenstein PK, Kolb R, Altaye M, Simon JO Use of an Internet portal to improve community-based pediatric ADHD care: a cluster randomized trial Pediatrics. 2011 Nov;128(5):e1201-8. doi: 10.1542/peds.2011-0872. Epub 2011 Oct 17. 6 Multiple case study approach o Roberge, P., & et al. (2013). Implementing a knowledge application program for anxiety and depression in community-based primary mental health care: a multiple case study research protocol. Implementation Science, 8(26). APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY Shepard P, Bubp C, Azar J, Nazir A, Adams N, Campbell NL, Chehresa A, Dexter P. Connecting research discovery with care delivery in dementia: the development of the Indianapolis Discovery Network for Dementia. Clin Interv Aging. 2012;7:509-16. doi: 10.2147/CIA.S36078. Epub 2012 Nov 16. Randomized controlled trials with health care professionals as unit of intervention o Tucker AL. Fostering organizational learning: the impact of work design on workarounds, errors, and speaking up about internal supply chain problems. 2012. Working paper 13-044. Harvard Business School. o MacDermid, J., & et al. (2012). Push versus Pull for mobilizing pain evidence into practice across different health professions: a protocol for a randomized trial. [Protocol]. Implementation Science, 7(115) Cluster, stepped non-randomised study o The Matching Michigan Writing Committee. (2013). "Matching Michigan": a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England Quality and Safety in Health Care, 22, 110-123. SECTION VIII. MEASURING OUTCOMES OF QI RESEARCH Patrick, S., Davis, M., Sedman, A., Meddings, J., Hieber, S., Lee, G., . . . Schumacher, R. (2013). Accuracy of hospital administrative data in reporting central line-associated bloodstream infections in newborns Pediatrics, 131 The Health Foundation. Lining up: How is harm measured? London, England: The Health Foundation; February 2013. Dixon-Woods M, Leslie M, Bion J, Tarrant C. What counts? An ethnographic study of infection data reported to a patient safety program. Milbank Quarterly. 2012;90(3):548-591. Chamberlain, J., Shaw, K., Lillis, K., & et al. (2013). Creating an infrastructure for safety event reporting and analysis in a multicenter pediatric emergency department network. Pediatr Emerg Care, 29, 125-130. SECTION VII. FORMATIVE RESEARCH AND DISCOVERY CAPACITY o o o 7 Perl H. Addicted to discovery: Does the quest for new knowledge hinder practice improvement? Addict Behav. 2011 Jun;36(6):590-6. doi: 10.1016/j.addbeh.2011.01.027. Epub 2011 Feb 1. Leviton LC, Schuh RG. The importance of a discovery capacity in community-based health and human service program evaluation, in Telfair J, Leviton LC, and Merchant JS (editors). Evaluating Health and Human Service Programs in Community Settings. Jossey-Bass, San Francisco, 1999. Essen A, Lindblad S. Innovation as emergence in healthcare: Unpacking change from within. Soc Sci Med 2012 Sep 12. Boustani MA, Frame A, Munger S, Healey P, Westlund J, Farlow M, Hake A, Austrom MG, SECTION IX. STATISTICAL ISSUES OF NOTE Oakes M. Effect identification in comparative effectiveness research. eGEMS 1(1). http//repository.academyhealth.org/egems. Makes points appliable to observational research beyond CER. Thomas L, Peterson ED. The value of statistical analysis plans in observational research. JAMA 2012 308(8):773-4. o Beachell, E., Monda, M. A primer for enumerative vs. analytic studies: Using caution APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY in statistical inferences. ASQ Statistics Division Newsletter. 1996; 16 (3):6-10. SECTION X. REVIEWS OF QI INTERVENTION STUDIES Coker, T., Windon, A., Moreno, C., Schuster, M., & Chung, P. (2013). Well-child care clinical practice redesign for young children: A systematic review of strategies and tools. Pediatrics, 131 8 SECTION XI. OTHER Shojania, K., & Thomas, E. (2013). Trends in adverse events over time: why are we not improving? Quality and Safety in Health Care, 22, 273-277 Marshall M. Sounds good but is it science? 2012 http://islondon.org/2012/18/what-haveacademics-ever-done-for-us/. (blog). Defense of good improvement science as science. Vincent C. Science and patient safety. CMAJ 2012 DOI:10.1503/cmaj.120792. Argues for a coordinated research program on patient safety, including measurement development, rigorous observational studies, and monitoring of harm. Benbassat J, Taragin MI. The effect of clinical interventions on hospital readmissions: a metareview of published meta-analyses. Isr J Health Policy Res 2013 Jan 23;2(1). Margolis, P., Provost, L.P., Schoettker, P.J., et al. Quality improvement, clinical research, and quality improvement research—opportunities for integration. Pediatric Clinics North America. 2009; 56(4):831-41 Legare F, et al. Patients’ perceiptions of sharing in decisions: a systematic review of interventins to enhance shared decision making in routine clinical practice. Patient 2012;5(1):1-19. Wensing M, Grimshaw JM, Eccles MP. Does the world need a scientific society for research on how to improve quality? Implement Sci 2012 Feb 29;7:10. Pronovost PJ, Cardo DM, Goeschel CA, Berenholtz SM, Saint S, Jernigan JA. A research framework for reducing preventable patient harm. CID 2011:52:507-513 (15 February). Importance of the full spectrum of clinical and translational research. Through the Quality Kaleidoscope: Reflections on the Science and Practice of Improving Health Care Quality: Closing the Quality Gap: Revisiting the State of the Science [Internet]. (2013). In K. McDonald, C. Chang & E. Schultz (Eds.), AHRQ Methods for Effective Health Care. Rockville, MD: AHRQ Nanji, K., Ferris, T., Torchiana, D., & Meyer, G. (2012). Overarching goals: a strategy for improving healthcare quality and safety? BMJ Quality & Safety. Lindegren et al., Integration of HIV/AIDS services with maternal, neonatal and child health, nutrition, and family planning services. Cochrane Database Syst Rev. 2012 Sep 12;9. Scott A, et al., The effect of financial incentives ont eh quality of health care provided by primary care physicians. Cochrane Database Syst Rev. 2011 Sep 7;(9). (included RCT, controlled before and after studies [CBA], and ITS study designs. in acute hospital settings - limited evidence of effectiveness. BMC Health Serv Res., 11. Ko, H., & et al. (2011). Systematic review of safety checklists for use by medical care teams i Contact denise.dougherty@ahrq.hhs.gov ii http://www.ambpeds.org/specialInterestGroups/sig_quality_ improvement.cfm APA QI RESEARCH METHODS CONFERENCE MAIN [2013 BIBLIOGRAPHY iii Note: Inclusion of a publication under a specific study design does not mean that we evaluated whether the study met all desirable characteristics of the study design. We relied on the description in the study abstract. 9