Quality Management Workshop August 2012 This project has been funded in whole or in part with Federal funds from the Division of AIDS (DAIDS), National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health, Department of Health and Human Services, under contract No. HHSN272201200009C, entitled NIAID HIV and Other Infectious Diseases Clinical Research Support Services (CRSS). Reminders and Helpful Hints Review of presentation materials Please turn off or silence mobile phones Exits Restrooms Assistance 2 Audience Response System (ARS) Respond to Questions Change an Answer Responses are Anonymous Question cue is a on preceding slide Ensure remote is on by pressing and holding the “On/Off” button Please leave remotes on the tables 3 Choose your answer Send or change your answer Audience Response System (cont’d) Good documentation practices include following the principles of: A. Electronic Record Management B. “ALCOA” Choose your answer C. Electronic Signatures Send or change your answer 4 Audience Response System (cont’d) Name one of the words that is applied within the acronym “ALCOA”. Choose your answer Send or change your answer 5 Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review 6 Objectives Differentiate between Quality Assurance (QA) and Quality Control (QC) Discuss the roles and responsibilities associated with maintaining a Quality Program at the site Apply the DAIDS Clinical Quality Management Plan (CQMP) Policy for the development, implementation, evaluation, and adaptability of a site CQMP Determine corrective and preventive actions (CAPA) based on scenarios of common findings Discuss clinical site monitoring and how monitoring and data management queries can influence quality at the site Discuss actions a site can take to implement a quality management culture at the site 7 Additional QM Resources DAIDS Learning Management System (DLMS): https://daidslms.plateau.com/learning/user/login.do Orientation to Quality Management eLearning - Course 676 Quality Management Roles and Responsibilities eLearning Course 991 CAPA for Clinical Staff eLearning - Course 950 CAPA for Laboratory Staff eLearning - Course 990 CAPA for Pharmacy Staff - Course 970 CQMP Plan Policy, Session 1 - Webinar Clinical Quality Management Plans - Policy eLearning Clinical Quality Management Webinars - Courses 267, 287, and 288 Clinical Quality Quick Reference Card DAIDS Clinical Research Policies CQMP Annual Summary Report 8 Pre-Assessment Question #1 The clinical site has had three site monitoring visits since the first subject enrolled in the study. Each subsequent site monitoring visit report has reflected a 50% decrease in site issues. The site diligently reviews its site monitoring report (SMR) to clearly identify the issues listed and adjust its CQMP accordingly to ensure the amount of queries generated during monitoring visits and by the data management center has significantly decreased. Based on the findings in the SMR what step has the site taken to decrease the number of monitoring findings and data management queries? A. Developed corrective action plans to address the monitoring findings B. Submitted the SMR to the Institutional Review Board/Ethics Committee (IRB/EC) C. Filed the SMR in the regulatory files 9 Pre-Assessment Question #2 Once a CQMP is developed, how often should the Plan be reviewed? A. Every two years B. Annually C. Per policy 10 Pre-Assessment Question #3 Which of the following individuals is ultimately responsible for the quality management activities at a research site? A. B. C. D. 11 The Investigator The Study Coordinator The Pharmacist The Research Participant Pre-Assessment Question #4 Which quality management activity is conducted in real-time? A. Quality control B. Quality assurance 12 Pre-Assessment Question #5 The corrective action component of Corrective and Preventive Action (CAPA) is defined as the action taken to remove or improve a process to prevent potential future occurrences of a nonconformance. A. Yes B. No 13 Pre-Assessment Question #6 What actions can site staff take to implement a quality management culture at their site? A. Identify a Quality Manager who is responsible for monitoring and addressing trends identified in site monitoring reports B. Create a site CAPA plan C. Engage stakeholders in implementing the site’s CAPA D. All of the above 14 Your Mission… 15 work Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review 16 Quality Management Activities Quality Management Quality Control 17 Quality Assurance Quality Control Quality Control is the real time, ongoing (day-to-day) operational techniques and activities that are undertaken to verify requirements for quality trial-related activities. Example: Verification that all headers, required fields, and dates are completed correctly on case report forms (CRFs) prior to data entry. 18 Quality Assurance Quality Assurance is a retrospective, objective, systematic, and periodic review of trial-related activities to ensure that the trial is performed and the data are generated, documented and reported in compliance with Good Clinical Practice and any applicable regulatory requirements. Example: Staff may evaluate key elements of source documentation and compare them to completed CRFs for agreement on a weekly basis. 19 Clinical Quality Management Plan (CQMP) According to the DAIDS Policy on “Requirements for Clinical Quality Management Plans (CQMP)”, all clinical research sites conducting or participating in NIAID DAIDS supported and/or sponsored clinical research must have a CQMP. CQMP Clinical Site or Screen Shot 20 Why is implementation of a CQMP important? 21 VOICE Enrolled versus Monitored Quality Roles and Responsibilities 22 Develop and Implement a CQMP Collaborate with team members to: Identify areas of performance to monitor Identify measurable components to record Communicate plan Measure performance at established intervals Document all CQMP activities Report performance measurements to appropriate members of the team 23 CQMP Documentation Name of the reviewer Date of the review Research participant identification number Specific items that were reviewed Time period covered by the review Findings/results of the review 24 Key Indicators What are Key Indicators? Established measures to determine how well an organization is meeting its regulatory and operational expectations 25 Key Indicators (cont’d) Informed Consent Form and process Concomitant/prohibited medications Eligibility criteria Study product administration/dosing Scheduled tests and procedures Missed visits, tests, or procedures 26 Clinical endpoint identification Identification and reporting of SAEs, EAEs, and AEs Key Indicators Activity 1 27 Key Indicators (cont’d) Scenario A large Phase IIB randomized trial of a vaccine to prevent HIV-1 acquisition is being conducted at three sites in South Africa. Sites were selected, in part, for estimated enrollment of 500-750 volunteers per site. There is a strong interest in this trial within the communities. A large number of volunteers are illiterate. Many volunteers may have health issues related to poor nutrition and crowded living conditions. 28 Group Discussion Define Quality Indicators Name quality performance goals for documenting eligibility. Measure Quality Indicators Name some measurable indicators for eligibility verification. Process Evaluation and Improvement Analyze results Improve processes Control processes 29 Define Key Indicators Choose one or more quality performance goals for documenting informed consent. 100% of enrolled research participants will have: A. Signed/dated informed consent on file B. Source documentation in the clinic/medical record C. Informed consent obtained prior to collection of any research-specific data D. All of the above 30 Measure Key Indicators Choose the measurable indicator(s) for the informed consent process: A. The percent of research participants who have signed informed consent B. The percent of research participants who have documentation in the clinic/medical record of informed consent process C. The percent of research participants who signed informed consent prior to collection of any researchspecific data D. All of the above 31 Group Discussion: Key Indicators Let’s discuss ideas for evaluating informed consent performance for a site enrolling 500-750 research participants in a Phase IIB/III trial. What would you consider as you develop this aspect of your CQMP? 32 Performance Metrics: Use Available Data Use available data and reports to monitor and measure trends in performance: Site monitoring reports Data quality management reports 33 Create Your Own Tools Tracking timelines Set reminders for EC/IRB due dates: Protocol continuing review Record submission dates: Safety reports to EC/IRB Protocol-specific checklists Source documentation tools to ensure compliance with: Informed consent process Confirming eligibility criteria 34 Tools The CQMP includes tools or checklists to be used in the QA and QC processes. Examples include: 35 Visit reminder checklist Data entry reports Query reports Error reports Site monitoring reports Chart review tools Sponsor Quality Oversight Sponsor Clinical Site Oversight Clinical Site Monitoring Regulatory Affairs Protocol Registration Safety Reporting 36 Statistical and Data Management Edit checks, queries, delinquency reports Pharmaceutical Affairs Clinical Research Product Management Your Mission… 37 work Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review Quality Control and Assurance Activity 2 39 Quality Control and Assurance Activity 1. Review the scenario 2. Review the documents included in the scenario packet for issues 3. Complete the Issue/Trend Worksheet for each issue/trend identified 4. Worksheets are labeled for Quality Control and Quality Assurance 40 Documents You Need For This Activity Clinical Quality Management Plan (CQMP) Training Protocol and Site Information – Reagan’s Research Clinic Protocol Summary Letter of Amendment March 2010 Clinical Site Monitoring Group Site Monitoring Report Protocol MTN-003 Line Check Error Log 41 Documents You Need For This Activity (cont’d) Clinical Laboratory Report PID 555100101 Clinical Laboratory Report PID 555100095 Queries Report Quality Management Chart Review Site Investigational Product Accountability Record - Tablets Site Investigational Product Accountability Record – Tubes 42 Documents You Need For This Activity (cont’d) Subject I.P Accountability Record – Subject 101 Subject I.P. Accountability Record – Subject 95 Subject I.P Accountability Record – Subject 96 Delegation of Responsibility Log/Signature List Document to Note Information: Quality Control (QC) Issue and Trend Worksheet 43 Quality Control and Assurance Reagan’s Research Clinic is located at: 63847 Capitol Hill Boulevard Durban, KwaZulu-Natal Scenario The Research Team 44 Investigator Dr. Dominick Reagan Sub-Investigator Dr. Phillip Dole Study Coordinator Cecilia Cooper Pharmacist Olivia Seagull Counselor Kevin Jones Quality Control and Assurance (cont’d) Scenario The site has had numerous audit findings identified during monitoring visits and during internal quality management review activities over the course of the year. The Investigator is holding a staff meeting to review audit findings from quality management activities over the past year. 45 Quality Control and Assurance (cont’d) Scenario During the meeting Dr. Reagan has asked all the research team members to review the following documents to identify trends: QC Log Letter of Amendment (LOA) QM Chart Review Tool Queries Report Site Monitoring Report (SMR) Pharmacy Logs CQMP Annual Report 46 Quality Control and Assurance Activity FEEDBACK 47 Quality Control and Assurance Activity 3 48 Document You Need For This Activity Clinical Site Monitoring Group Site Monitoring Report (28-Sep-2010) 49 What Can You Do? Next Monitoring Visit: October Work with the participants at your table to review the Site Monitoring Visit Report (SMR) Rank issues/trends according to number of times the issue is found and priority Identify the trends 50 Next Monitoring Visit: October (cont’d) Why is it important to note issues/trends? What can happen if these issues are left unresolved? What do these issues mean at the site? What are your practices on site when you identify an issue? 51 Quality Control and Assurance FEEDBACK 52 PID(s) Obse Visit Observation rvatio n Code Date Code 555100095 M04 30/Sep/2010 A23 Missed Visit Adequate Source Documentation: The study participant did not return to the research clinic for their scheduled M04 study visit. The subject took a holiday and was away during the visit window. 555100095 M05 30/Sep/2010 A17 Transcription Error: The following laboratory values were not transcribed from the source to the CRF: Sodium, Total Bilirubin, ALT (SGPT) and AST (SGOT). 555100095 M05 30/Sep/2010 A72 Procedural Inadequate Source Documentation: The fax copy of the chemistry laboratory results was not certified per the DAIDS Source Documentation requirements by the research clinician who received the document. *555100096 M04 01/Oct/2010 A17 *555100096 M05 01/Oct/2010 A23 555100101 M00 28/Sep/2010 A1A Observation Description A1A: Inadequate Missed Visit A72: Inadequate A17:Adequate Inadequate Source Missed Visit Adequate Source Documentation: Source The study participant did not return to the research clinic for her scheduled M05Documentation study visit. The study coordinator contacted the Source Documentation participant on the following dates without success: 16/Sep/2010, 21/Sep/2010, Documentation 23/Sep/2010, 27/Sep/2010, 28/Sep/2010 29/Sep/2010. The participant’s visit was IsIs ititaand atrend? trend? Documentation scheduled to occur 13/Sep/2010. Messages were left on the participant’s voicemail to Is it a trend? Transcription Error: The participant’s systolic blood pressure reading was incorrectly transcribed from the source to the CRF. contact the clinician to reschedule the study visit before the protocol window closed. The IsWhat it a can trend? What happen if this finding is could happen if this window closed and the site still had not heard from the participant. What could happen if this unresolved? left unresolved? Informed Consent Violation Level left 1:finding Pages 5 and 6 were missing from the Enrollment What could happen if this finding left unresolved? Informed Consent Form, V1.0, dated 02/Nov/2009, signed on 07/Sep/2010. Action was corrected during the monitoring visit. What finding left this unresolved? Whatdoes does thisfinding findingmean meanfor for What does this finding mean for the site? 555100101 555100101 M00 28/Sep/2010 A17 M01 28/Sep/2010 A72 the Transcription Error: The participant’s yearsite? of birth isthis incorrectly transcribed from the What does finding mean for the site? Demographic source document to the Demographic CRF. Action was corrected by the What are the site? What areyour yourpractices practiceson onsite site research clinician during the monitoring visit. What are practices on site when you identify an whenare youyour identify anissue? issue? What your practices on when you identify an issue?site Procedural Inadequate Source Documentation: to verify source documents the when youUnable identify anin issue? pregnancy test results entered on the CRF. 53 Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review Corrective and Preventive Action (CAPA) PA CA CAPA Corrective Action 55 Preventive Action Corrective and Preventive Action (CAPA) Process of identifying the root cause of a problem, nonconformity, or undesirable situation. The root cause is the most basic reason for the effect, which if eliminated or corrected, can prevent the event from existing or reoccurring. 56 Corrective and Preventive Action Plans Develop CAPA Plans Perform root cause analysis Identify and implement corrective actions Propose and implement preventive actions Continue measuring performance at established intervals Compile and report performance measurements within the team 57 Corrective and Preventive Action (CAPA) Root Cause Analysis Activity 4 58 Document You Need For This Activity Root Cause Analysis Worksheet 59 Corrective and Preventive Action (CAPA) Root Cause Analysis Using the trends identified, complete the root cause analysis worksheet. 60 60 Root Cause Analysis Feedback Session 1. Why is this issue occurring? 2. Why is this issue occurring? 3. Why is this issue occurring? 4. Why is this issue occurring? 5. Why is this issue occurring? 61 Corrective and Preventive Action (CAPA) Developing a CAPA Plan Activity 5 62 Corrective and Preventive Action (CAPA) Develop a CAPA Plan Using the CAPA Worksheet, develop a CAPA Plan using your noted trends Be prepared to discuss this as a group 63 Root Cause Analysis Feedback Session Proposed Corrective Action: Who is involved in this Corrective Action? What resources are needed to complete this Corrective Action? What is the timeline for completing this Corrective Action? How will the effectiveness of this Corrective Action be measured? What other areas could this Corrective Action impact? What are the risks and benefits for implementing the proposed Corrective Action to the affected areas? What strategies could you employ to minimize risks to the affected areas? 64 Root Cause Analysis Feedback Session Proposed Preventive Action: Who is involved in this Preventive Action? What resources are needed to complete this Preventive Action? What is the timeline for completing this Preventive Action? What are the risks and benefits for implementing the proposed Preventive Action to the affected areas? What strategies could you employ to minimize risks to the affected areas? 65 Corrective and Preventive Action (CAPA) Implementation Activity 6 66 Document You Need For This Activity CAPA Worksheet 67 Corrective and Preventive Action (CAPA) Implementation Plan Worksheet Using the Implementation Plan Worksheet, develop an Implementation Plan for your CAPA Outline roles and responsibilities at the site 68 Corrective and Preventive Action (CAPA) Implementation Plan Feedback Session CAPA 69 How you will accomplish this? Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review Reminder: CQMP Review Proactively review your CQMP to assess the effectiveness of your current plan Make sure it adequately reflects all necessary QM activities Review your CQMP after a monitoring visit to determine if changes need to be made to the CQMP to address recent findings 71 What Can You Do? Become a change agent Motivate site staff to implement the change Communicate the change effectively Ensure you have stakeholder buy-in Allow for adequate time to implement the change Recognize site/staff improvements Maintain realistic expectations 72 CQMP Review Activity 7 73 Documents You Need For This Activity Clinical Quality Management Plan (CQMP) Clinical Quality Management Plan Summary Report 74 Activity: Update CQMP Review the completed sample CQMP Update the plan to reflect corrective actions determined in the CAPA Plan 75 CQMP Evaluation and Reporting – Be Ready! Required annual review and submission of Annual Summary Report Submit CQMP to Program Officer on an annual basis, if there have been any changes Program Officers may ask to review plan based on findings at the site CQMP Annual Summary Report 76 Implement a Quality Management Culture Basic and ongoing training: Provide and reinforce rationale for quality Protect the safety, welfare, privacy and confidentiality of research participants Generate complete, accurate and timely data to promote the safety of research participants and future consumers Establish efficient and streamlined processes Document and train on work processes Evaluate and revise processes as needed 77 Engage Team Members in CQMP Processes Build QC steps into processes Promote rapid, selfidentification and correction of errors Perform root cause analysis activities Applicable team members to gather input Share perspectives in an open (non-blaming) manner Share results of QA activities Acknowledge and celebrate performance improvements 78 ALCOA ttributable: Is it obvious who wrote it? egible: Can it be read? ontemporaneous: Is the information current? Is it in the correct time frame? riginal: Is it a copy? Has it been altered? ccurate: Is conflicting data recorded elsewhere? From DAIDS Policy on “Requirements for Source Documentation in DAIDS Funded and/or Sponsored Clinical Trials” 79 Enforce Excellence in Source Documentation “If it isn’t written down, it didn’t happen” Good documentation practices: Use pens. No pencils! No white out! Sign, credential and date Follow ALCOA standards Train staff on DAIDS source documentation requirements Build internal review and feedback of source documentation practices into CQMP 80 Activity – ALCOA Practices Discuss source documentation at your CRS Identify methods used to ensure clinic notes are: Attributable Legible Contemporaneous Original Accurate Share your best practices and lessons learned! 81 Communication Pathways Staff has clear understanding of roles and responsibilities Creates a positive quality culture Benefits Increases teamwork, establishes common goals, promotes staff unity 82 Promotes an integrated Quality Management Program Increases efficiencies and decreases redundancy Remember your mission… work 83 Quality Management Workshop Quality Management Quality Assurance CQMP Policy Quality Control Activity 1: Key Indicators QA and QC Activities Activity 2: Detecting and assessing trends/issues Activity 3: SMR Review Corrective and Preventive Action Activity 4: Root cause analysis Activity 6: Implementation Plan Activity 5: Develop a CAPA Plan Review, evaluate and create a culture of Quality Management Activity 7 : CQMP Review Summary Differentiate between QA and QC Discuss the roles and responsibilities associated with maintaining a quality program at the site Apply the DAIDS Clinical Quality Management Plan Policy for the development, implementation, evaluation, and adaptability of a site CQMP Determine corrective and preventive actions based on scenarios of common findings Discuss clinical site monitoring and how monitoring and data management queries can influence quality at the site Discuss methods to implement a Quality Management culture at the site 85 Remember Your Additional QM Resources DAIDS Learning Management System (DLMS): https://daidslms.plateau.com/learning/user/login.do Orientation to Quality Management eLearning - Course 676 Quality Management Roles and Responsibilities eLearning Course 991 CAPA for Clinical Staff eLearning - Course 950 CAPA for Laboratory Staff eLearning - Course 990 CAPA for Pharmacy Staff - Course 970 CQMP Plan Policy, Session 1 - Webinar Clinical Quality Management Plans - Policy eLearning Clinical Quality Management Webinars - Courses 267, 287, and 288 Clinical Quality Quick Reference Card DAIDS Clinical Research Policies CQMP Annual Summary Report 86 Post-Assessment Question #1 The clinical site has had three site monitoring visits since the first subject has enrolled in the study. Each subsequent site monitoring visit report has reflected a 50% decrease in site issues and trends. The site diligently reviews their site monitoring report (SMR)to clearly identify the issues listed and adjust their CQMP accordingly to ensure the amount of queries generated during monitoring visits and by the data management center has significantly decreased. Based on the findings in the SMR what step has the site taken to decrease the number of audit findings and data management queries? A. Developed corrective action plans to address the audit findings B. Submitted the site monitoring report to the Institutional Review Board/Ethics Committee (IRB/EC) C. Filed the site monitoring report in the regulatory files 87 Post-Assessment Question #2 Once a CQMP is developed, how often should the Plan be reviewed? A. Every two years B. Annually C. Per policy 88 Post-Assessment Question #3 Which of the following individuals is ultimately responsible for the quality management activities at a research site? A. B. C. D. 89 The Investigator The Study Coordinator The Pharmacist The Research Participant Post-Assessment Question #4 Which quality management activity is conducted in realtime? A. Quality control B. Quality assurance 90 Post-Assessment Question #5 The corrective action component of Corrective and Preventive Action (CAPA) is defined as the action taken to remove or improve a process to prevent potential future occurrences of a nonconformance. A. Yes B. No 91 Post - Assessment Question #6 What actions can site staff take to implement a quality management culture at their site? A. Identify a Quality Manager who is responsible for monitoring and addressing trends identified in site monitoring reports B. Create a site CAPA plan C. Engage stakeholders in implementing the site’s CAPA D. All of the above 92 Quality Management Workshop Culture of Quality Management Quality Management Corrective and Preventive Action QA and QC Activities THANK YOU 94 Return to Slide 95 Return to Slide Return 96