ارزشيابي صالحيت باليني Workshop for assessing clinical competence دكتر محمدرضايوسفي MD-MPH- MS.c Medical education عضوهیئت علمی مرکز مطالعات و توسعه آموزش پزشکی دانشگاه دفترتوسعه آموزش باليني اهداف آموزش ی محتوای برنامه برنامه شیوه های تدریس و اجرای برنامه 3 ارزشیابی Evaluation is for Improvement Not Provement Professional Competence • The habitual and judicious use of communication, knowledge, technical skills, evidence based decision making, emotions, values, and reflection to improve the health of the individual patient and the community. “بههک رههری یی عردت ونههک و مدبیهنههک دهن ه مهریتهههر هیتبرطههرتی بهههههروینب عوهطههههه هی ههههههر و،مهریتههههههري عملهههههب هنهههههتدف دی هههت مهدمت بهک فهید،بر هندی ب دی فعرویت حیفکهي یو مهی "و هرمعک Epstein and Hundert, Designing and assessing professional competence. JAMA 287:226, 2002 5 What is Clinical Competency? The ability to handle a complex professional task by integrating knowledge, skills and attitudes. نگرش مهارت عملی شناخت ()دانش مهارت عملی و،تلفیق دانش نگرش های حرفه ای در ضمن انجام یک وظیفه بالینی پیچیده Reference Van der Vleuten, C. The assessment of professional competence: developments,research and practical implications Advances in Health Science Education 1996, Vol 1: 41-67 Rice et al., Competency-based objectives for clinical training. Can. J. Med. Technol. 57:136, 1995 What is the best way to evaluate Clinical Competency? Behavior Skills & attitudes Performance assessment: Masked SP Competency assessment: OSCE Clinical based tests: MCQ Knowledge Knowledge tests: MCQs Miller’s Pyramid of Competenc Miller GE. The assessment of clinical skills, competence, performance Acad Med, 1990; 65(Suppl.):S63-S67. Professional authenticity A simple model of competence Does Performance or hands on assessment Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. Written, Oral or Computer based assessment Professional authenticity Model of competence Behaviour~ skills/attitudes Cognition~ knowledge Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67. climbing the pyramid Does Does Shows how Knows how Knows Performance assessment in vivo: Masked SPs, Video, Portfolio….. Competence assessment in vitro: OSCE, ….. (Clinical) Context based tests: MCQ, essay type, oral….. Factual tests: MCQ, essay type, oral….. Validity of testing formats Professional practice assessment Performance assessment: OSCEs, long/short cases, etc Problem-solving assessment: PMP MCQs,.. Knowledge assessment: MCQs.. Miller’s Pyramid for Assessing Clinical Competence Work Action Does OSCE Performance Shows How PMPs Competence Knows How MCQs Knowledge Knows Expanding our toolbox….. Does Shows how Knows how how Knows Knows Knows Established technology of efficient written or computer-based high fidelity simulations (MCQ, Key Feature, Script Concordance Test, MEQs….) Expanding our toolbox….. Does Shows how Knows Knows how how Knows Established technology of structured high fidelity in vitro simulations requiring behavioural performance (OSCE, SPbased testing, OSPE….) Expanding our toolbox….. Does Does Shows how Shows how Knows how Knows Emerging technology of appraising in vivo performance (Work-based assessment: Clinical worksampling, Mini-CEX, Portfolio, practice visits, case orals….) Expanding our toolbox….. Does Shows how Emerging technology of appraising in vivo performance (self-, peer, co-assessment, portfolio, multisource feedback, learning process evaluations……) Knows how Knows “Domain specific” skills “Domain independent” skills ACGME Core Competencies (The 6 Competencies) 1. 2. 3. 4. 5. 6. Patient Care Medical Knowledge Professionalism Interpersonal & Communication Skills Practice-Based Learning and Improvement Systems-Based Practice 17 انواع آزمونها -1آ مونهر تکوینی (Formative): • ارزيابي هاي محل خدمت ( )Workplaceبا پس خوراند ( )feedbackمناسب درطول دوره آموزشی • پيشرفت تحصيلی وتجربيات يادگيری دانشجو • تمام عملكرد و رفتار يك دانشجو مورد ارزيابي قرار مي گيرد. -2آ مونهري تهمعی )Summative( : • نشانگر قبولي يا ردي از امتحان آخر ترم يا ارتقائ ساالنه دستياري مي باشد . • ارزيابي فقط در حيطه دانش مي باشد. ?What are assessment tools كتبي كوتاه پاسخ گسترده پاسخ extended بسته جور كردني چندگزينهاي انجام تكاليف Assignments Cued questions صحيح-غلط Uncued questions باز تشريحي محدود پاسخ restricted آ مونهر رتبی کاربرد: آزمونهای کتبی What are assessment tools? Lab-based practical PBA(Procedure – based Assessment) DOPS آزمونهاي عملي Practical What are assessment tools? (Chart Stimulated Recall ) CSR یاCBD GRF OSCEs Mini-CEX آزمونهاي باليني Clinical What are assessment tools? Structured شفاهي Unstructured ارزيابي هاي در محل خدمت ) Workplace Based Assessment) • Mini-CEX )Clinical Evaluation Exercise) • • • • • • CBD ياCSR (Chart Stimulated Recall) DOPS ) Direct Observation Of procedural Skills) PBA (Procedure – based Assessment) 360° ) هی یربیMini-PAT( OSCE (objective structured clinical examination) GRF(Global Rating Form ) Climbing the Pyramid Does Does Showshow how Shows Knowshow how Knows Knows Knows Performance Assessment : Video, Real patient( Mini-CEX – DOPS-…) Competence Assessment :OSCE (Clinical) Context Based Tests: essay type, oral Knowledge Tests: MCQ ارزشیابی صالحیت بالینی به وسیله آزمون بالینی ساختارمند عینی )(OSCE )(Objective Structured Clinical Examination What’s in a name? O S C E bjective tructured linical xamination OSCE OSCE Objective Structured Clinical Examination عینی ساختاردار بالینی آزمون ن آسووکی يووک آزمووون ارزيووابی صووالحيت بووالينی • آزمووو اسووت کووه ب ووورت عينووی در ايسووتگاههای م تلووف سووازماندهی شووده و تهري ووا هوور چهووار ضووابطه ي وک آزمون عادالنه را داراست. • همک دهوطلبین بر یك مقیرس یکنرن موید ننه معیار ارزشیابی می شوند. • همه دانشجویان با یک قیهی مب ییند. OSCEs - Structured تدهبیی و بینرمک هی هینتگر ،نظرم دهی هنت. • هب هیهنده ،یی دیهیهینتگر ،ویژ ،همرن هینتگر ،تنظیم د ،هنت. • دیهیهینتگر ،یک مهریت بروینی ویژ ،مویدهی یربی قیهیمی یید. مهارت هاي ويژه اي كه در هر ايستگاه مورد آزمون قرار مي گيرند شامل موارد زير است: o History taking o Interpersonal communication o Clinical examination o Procedures • .آزمون توانایی انجام مهارتهای بالینی استOSCE Students have to demonstrate their skills, not just describe the theory Clinical Examination هی یربی یفتری حیفک ه و مهریتهر هیتبرطی • هینتگر ،هریی بر حضوی بیمرینمر • فیهیند یه هی یربی رنید نک محتوه یه مثال دهن هو چگونک صحبت رید نک هینکک چک فت. هی یربی مهریتهر بروینی • توهنمند دهن هو و فیهیند هنهرم مهریت یه هی یربی رنید نک نتیهک رری یه. Educational Outcomes Service Group Educational Outcomes Service Group Competencies: Six and Improvement: TheRespond HOW YOU GET BETTER Competencies: The Six Doma Sample Behaviors Within Each Domain Sample Behaviors Within Each Domain Writes leg Interpersonal and Consider Sample Behaviors Within Each Domain Skills:about patients Gathers essential andCommunication accurate information Gathers essential of and accurate informationDocumen about pa Prioritizes the purpose(s) the visit HOW YOU INTERACT Uses effe Interpersonal and of the visit Prioritizes the purpose(s) Makes accurate diagnoses WITH OTHERS Possesse Makes accurate diagnoses Gathers essential and accurate information about patients Communication Skills: Prescribes appropriate therapeutic interventions Creates a Prescribes appropriate therapeutic Prioritizes the purpose(s) of the visit Counsels and educates patient and familiesinterventions HOW YOU INTERACT Interpersonal Counsels and educates patient and familiesand colle Makes accurate diagnoses Incorporates prevention into care planand WITH OTHERS Incorporates prevention care plan examCreates a Prescribes appropriate therapeutic interventions Communication Skills: Conducts an appropriate historyinto and physical families, Conducts an appropriate history and physical exam a Counsels and educates patient andINTERACT families HOW YOU Performs procedures well Works eff Performs procedures well Incorporates prevention into care plan Orders appropriate diagnostic tests WITH OTHERS appropriate diagnostic tests exam Presents ConductsOrders an appropriate history and physical Performs procedures well Orders appropriate diagnostic tests Applies knowledge to patient care Professionalism: Appliesinvestigatory knowledge toand patient carethinking Respond Demonstrates analytic HOW YOU ACT Demonstrates investigatory Reliably c Knows and applies basic sciences and analytic thinking Knows and applies basic sciences Applies knowledge to patient Sees pat Understands a)Professionalism: anatomy b) care physiology c) biochemistry Understands a) anatomy b) physiology c) biochemis Demonstrates investigatory and analytic thinking Demonst d) microbiology e) pathology f) pharmacology g) neurosc HOW YOU ACT microbiology e) pathology f) pharmacology g) n Knows and d) applies basic sciences Demonst Understands psychology/behavioral medicine Understands psychology/behavioral medicine Understands a) anatomy b) physiology c) biochemistry gender, d Understands functionfunction of Professionalism: surgicalsurgical instruments Understands instruments d) microbiology e) pathology of f) pharmacology g) neuroscie Serves ef HOW YOUmedicine ACT Understands psychology/behavioral Understands function of surgical instruments Reads Reads Uses information technology to support own education ho Uses information technology to support ownKnows educatio Locates, evaluates and utilizes evidence-based resources Knows ho Systems-Based Practice: Reads Locates, evaluates and utilizes evidence-based reso Teaches students and other health care professionals (facili Understa HOW Teachestechnology students and otherWORK health care professiona Uses information to YOU support own education learning) Bills appr learning) andWITHIN Locates, evaluates utilizes evidence-based resources THE SYSTEM Systems-Based Practice: Engages in continuing medical education Engages in continuing medical education Understa Teaches students and other health care professionals (facilita Responds to feedback Able to u HOW YOU WORK learning) Responds to feedback Understa WITHINSystems-Based THE SYSTEM Engages in continuing medical education Practice: Practices Responds to feedback HOW YOU WORK Ensures a Handout prepared by Writes legibly Writes Lynne legibly Effectivel Tomasa, PhDTHE SYSTEM WITHIN Considers literacy level of level others A more extens Considers literacy of others 520-626-7823 Uses app Competencies: The SIX Domains Patient Care: Patient Care: WHAT YOU DO WHAT YOU DO Patient Care: WHAT YOU DO MedicalMedical Knowledge: Knowledge: WHAT YOU KNOW WHAT YOU KNOW Medical Knowledge: WHAT YOU KNOW PracticePractice Based Learning Based Learning and Improvement: and Improvement: HOW YOU GET BETTER Practice Based Learning HOW YOU GET BETTER and Improvement: HOW YOU GET BETTER Objective Structured Clinical Examination Medical authorities & faculties started to implement the OSCE exams for clinical and communication skills assessment of medical students, clerks, interns, residents, and graduates since the 1980s. It is a standardized method of clinical evaluation that eliminates bias. www.OSCEhome.com Objective Structured Clinical Examination OSCE is a form of performancebased testing used to measure candidates’ clinical competency. During an OSCE, candidates are observed and evaluated as they go through a series of stations in which they interview, examine and treat standardized patients (SP) who present with some type of a medical problem. www.OSCEhome.com Objective Structured Clinical Examination “The OSCE is an approach to the assessment of clinical competence in which the components of competence are assessed in a planned or structured way with attention being paid to the objectivity of the examination”. Harden, 1988 www.OSCEhome.com Objective Structured Clinical Examination The curriculum tells the staff what to teach... The OSCEs tells the students what to learn ! And that is the point…. What OSCE candidates should learn? What candidates can do rather than what they know. www.OSCEhome.com Objective Structured Clinical Examination The application of their knowledge rather than the recall of knowledge. The ‘WHAT’ to ask and do. The ‘HOW’ they ask and do. The ‘WHEN’ they ask and do. How do they organize themselves. How do they communicate with patients. www.OSCEhome.com Objective Structured Clinical Examination • The OSCE has offered a strikingly new and exiting way of making valid assessment of clinical performance of medical student ,resident, and fellows . What is the purpose of the OSCE? • Provide feedback on performance • Evaluate basic clinical skill • Measure minimal competency What is OSCE? • It is some sort of student evaluation that examinee will go through several stations to be evaluated in clinical fields by special rating scales and checklists. M. Mohammadi MD OSCE is Useful to Evaluate: • Clinical skill • Practical attitudes • Practical knowledge OSCE =Performance-Base assessment (Skill Based Assessment) : Actual Performance of clinical skills is observed in order to adjudicate the competence of Health professionals What is the format of OSCE? What is the format of OSCE? • It consists of several Stations • (12-20 each lasting 5 to 15 minutes not more than 30 seconds rest between each). OSCE • نکاتی که بايد هنگام طراحی اين آزمون رعايت نمود: – تهیه :blueprint • تعیين اهداف آموزش ی مهمی که باید مورد ارزیابی قرار گيرند. – تعداد کافی ایستگاه: • تعداد ایستگاه بیشتر از زمان هر ایستگاه اهمیت دارد. – برنامه ریزی دقیق و منظم – پیش بینی وقایع احتمالی OSCE نحوه برگزاری آزمون: ایستگاههای پشت سر هم و در هر ایستگاه یک taskمشخص 12 تا 20ایستگاه و هر ایستگاه 5تا 15دقیقه نقشه طرحريزي شده از 9ايستگاه OSCE 4 3 2 1 Start 5 6 7 8 9 End تعيين پروسه امتحان 1 2 3 4 5 6 7 8 10 9 10 9 8 7 6 5 4 3 2 1 ايستگاهها: یو :معموف ً Observerف م هنت. نؤهفت و تفرنیی Observe :ف م نینت. معموف ً What is the stations’ contents? What is the stations’ contents? • • • • • Case history Physical exam Real patient Standardize pt. Simulated pt. • • • • • History taking Clinical procedure Radiographs Slide presentation … OSCE محتوای ایستگاه ها: اخذ شرح حال ،معاينه فيزيکی ،مشاوره و آموزش بيمار ،تفسير اطالعات بيمار، پروسيجرمانندکشيدن دندانهاي قدامی فک باال و پايين وبی حسی موضعی وتنه ای روی مانکن و... ایستگاه های manned ایستگاه های unmanned محتوای ایستگاه ها: اختالالت مفصل گيجگاهی – فکی گرافی OPG محتوای ایستگاه ها: • تزريق بی حسی موضعی فک باال روي مانکن • تزريق بی حسی تنه ايی فک پايين روي مانکن مشاوره و آموزش بيمار مال اکلوژن-بيماری های لثه مال اکلوژن گرفتن تاريخچه بيمار How do you evaluate the examinee? How do you evaluate the examinee? • Checklist • Rating scale • Short answered questions •… چگونگی ارزيابی : OSCE – معیارهای ارزیابی از قبل و بر اساس اهداف آموزش ی تعیين می شوند. – از یک چک لیست استاندارد شده برای ارزیابی استفاده شود. می Who are the Raters? Who are the Raters? • • • • • Simulated pt. Standardized pt. Clinical staff Non clinical staff Non expert raters OSCE • ارزيابی کنندگان: – ناظر متخصص – ناظر غيرمتخصص – بیمار(نما) OSCE is suggested the first best method for evaluation of: • Interviewing • • • • • • • Counseling & educating pt’s & families Performing procedures Physical examination Preventive health services Creation of therapeutic relationship with patients Listening skills Respectful & altruistic behaviors • Sensitive to cultural, age, gender & disability issues چند مثال براي فعاليتهايي كه مي توانيم براي ايستگاهها در نظر بگيريم : معاينه يك بيمار واقعي معاينه بيمار استاندارد شده آموزش بيمار واقعي انجام عمل مشاوره درباره يك موضوع با يك بيمار واقعي يا تهليدي (تلفني يا حضوري ) شناسايي عوامل بيماريزا زير ميكروسكوپ EEG, EMG , ECG راديوگرام ،اسپيروگرام و ... تشخيص بيماري از روي اساليد و ت وير يا فيلم چند مثال براي فعاليتهايي كه مي توانيم براي ايستگاهها در نظر بگيريم : ديدن شرح حال و آزمايشات بيمار و تشخيص بيماري بررس ي و نهد پرونده بيماران مرخص شده گوش كردن نوار صداهاي قلبي و تشخيص انواع صداهاي ط يعي و غير ط يعي انجام يك كار عملي بر روي موالژ كار با دستگاهها و تجهيزات پزشكي انجام يك آزمايش پاراكلينيكي افتراق داروها و مواد شيميايي تشخيص ماكروسكوپي نمونه اتوپس ي شده 4/13/2015 80 • آزمووون آسووکی يووک آزمووون ارزيووابی صووالحيت بووالينی اسووت کووه ب ووورت عينووی در ايسووتگاههای م تلووف سووازماندهی شووده و تهري ووا هوور چهووار ضووابطه ي وک آزمون عادالنه را داراست. مزايای استفاد از : OSCE .1 موثر است. اين روش براي سنجش صالحيتهاي كلينيك و حل مسئله .2 روش ارزيابي من فانه ايست چون سواالت آن عيني است. .3 براي ارزيابي مهارتهاي رواني – حركتي روش مناسبي است. .4 كار واقعيش شرايط ط يعي برقر ار مي كند. اين روش بين امتحان شونده و محيط .5 بازخورد آني است. .6 در آن دخالتي ندارد. شانس مزايای استفاد از : OSCE -7طیف وسیعی از مهارت های تعداد زیادی از فراگيران در ً مدت نسبتا کوتاهی ارزیابی می شوند. -8هم بیمار و هم ارزیابی کننده برای تمام آزمون شوندگان یکسان هستند. -9شرایط آزمون به شرایط واقعی نزدیک است. -10از روایی و پایایی خوبی برخوردار است. محدوديتهای استفاد از : OSCE .1اين امتحان بودجه ،تداركات غني و هماهنگي هاي كامل درون ب ش ي را طلب مي كند. ي اين روش براي تعداد زياد دانشج و مشكل است. .2بكارگير ميسر نيست. بيمار واقعي همواره .3استفاده از گير است. .4طراحي و اجرا آن مشكل و وقت محدوديتهای استفاد از : OSCE -5مهارت ها به صورت مجزا و در ایستگاه های متفاوت ارزیابی می شوند. -6برای حفظ پایایی آزمون باید تعداد ایستگاه ها زیاد باشد. ً • در مورد برخی مهارت ها از جمله برقراری ارتباط عمال بسیار مشکل است. -7سازماندهی و اجرای آن نیازمند نيروی انسانی زیادی است. -8از لحاظ منابع مادی و زمان هزینه بر است. GRF, 360-degree DOPS, Mini-CEX OSCE MCQ