Data-Informed Faculty Development Planning Howard Community College August 17, 2012 Shelley A. Chapman, PhD Plan for this Session Using IDEA to help O Individual Faculty O Groups of Faculty Reflective Practice using Individual Reports Suggest new ideas to try Collect Feedback Improve Invite Reflect & Conversations Discuss Help them use the FIF well Interpret Results Help them interpret Individual reports Read & Learn Help them use IDEA resources that are keyed to reports Recommended: Diagnostic Form O More data O Specific suggested action steps based on research O IDEA resources (no cost) are keyed to reports I. Individual Faculty 1. Individual Consultations O Faculty Information Form O How to select objectives O Adjusted and Converted scores O Suggested Actions 2. Help them Learn How to Log their Results on the Faculty Worksheet Reflective Practice using Individual Reports Suggest new ideas to try Collect Feedback Improve Invite Reflect & Conversations Discuss Help them use the FIF well Interpret Results Help them interpret Individual reports Read & Learn Help them use IDEA resources that are keyed to reports Suggested Action Steps from Page 3 3. Help Faculty Make Connections to Free Resources that may help them Improve #16 #18 #19 IDEA Website POD-IDEA Notes O Background O Helpful Hints O Assessment O References and Resources Reflective Practice using Individual Reports Suggest new ideas to try Collect Feedback Improve Invite Reflect & Conversations Discuss Help them use the FIF well Interpret Results Help them interpret Individual reports Read & Learn Help them use IDEA resources that are keyed to reports 4. Encourage “Conversations” about IDEA O Coffee and Conversations O Paired Discussions O Classroom “Visitations” O Brown Bag Lunch/Supper Conversations O Testimonials at End-of-year Faculty Meetings Encourage the Use of the IDEA Help Community Search for information Ask specific questions Talk to other clients who use IDEA Share ideas about how to use IDEA II. Groups of Faculty 1. Workshops (Sample Topics) O Incorporating IDEA into a Syllabus O How to Select “Important” or “Essential” O O O O Objectives Encouraging High Response Rates Interpreting Reports (Adjusted & Converted Scores, Contextual data, etc.) How to get better ratings: Focus on learning How to use POD IDEA Center Notes 2. Use IDEA for feedback on Faculty Development “Courses” O Example: Faculty Summer Institute at PLNU Using IDEA for Faculty Development at Point Loma Nazarene University tile Using IDEA for Faculty Development at Point Loma Nazarene University Integrating IDEA with the TILE Program tile Model the Good Use of IDEA O Consult IDEA research for planning TILE activities O Correlation Studies O Relationship between Methods and Learning Objectives O Relationship between Methods and Global Items (Excellence of Course and Excellence of Teacher) tile Learning Objectives O How to select “important” or “essential” IDEA objectives related to the course O Use the Teaching Goals Inventory to help select objectives: Use the Three Guiding Questions O Is this a significant part of the course? O Do you do something specific to help students accomplish this objective? O Is progress on it part of their grade? tile Incorporating IDEA into Syllabus O Show examples of how faculty put IDEA Objectives in their syllabi with their specific course objectives. Example: My three objectives are for you to 1. learn to analyze and critically evaluate ideas, arguments, and points of view. O • To do this, you will be reading and critiquing scholarly articles and participating in class discussions. 2. develop skill in expressing yourself in writing. O • To do this, you will propose a written research project in APA format. 3. learn how to find and use resources. O • To do this, you will use scholarly books and articles for your research proposal. tile Bloom’s Taxonomy and IDEA IDEA OBJECTIVES BLOOM’S TAXONOMY 1. Gaining factual knowledge 2. Principles, generalizations, theories 3. Application for improved thinking 6 4. This profession’s points of view, skills 5. Team skills 10, 11 6. Creative capacities 10, 11 7. Broad , liberal education 3, 8, 9 8. Expressiveness (written and oral) 2, 7 1 9. Find & use resources 10. Personal values 11. Analyze and critically evaluate ideas 12. Learn more by asking questions Average Ratings for Objectives Objective # Average Rating IDEA OBJECTIVES 1 4.0 1. Gaining factual knowledge 2 3.9 2. Principles, generalizations, theories 3 4.0 3. Application for improved thinking 4 4.0 4. This profession’s points of view, skills 5 3.9 5. Team skills 6 3.9 6. Creative capacities 7 3.7 7. Broad , liberal education 8 3.8 8. Expressiveness (written and oral) 9 3.7 9. Find & use resources 10 3.8 10. Personal values 11 3.8 11. Analyze and critically evaluate ideas 12 3.8 12. Learn more by asking questions Week 3: CATs and Midterm Feedback O Think about your IDEA objectives and create a midterm evaluation just for yourself. O Ask questions that get at whether they feel they are making progress on the types of objectives you said are relevant. tile IDEA Response Rates O Talk about ways to boost response rates: O Can’t “set it and forget it” O Create a culture of reflective practice O Show sample report O Put objectives is syllabus O Talk about how you use student feedback O Explain its reciprocal value tile End-of-TILE Assessment O Administer the Diagnostic Form to participants O Instructors use the FIF in a way to assess team teaching tile 3. Using IDEA in a Curriculum Caucus General Education Howard Community College General Educatino Possible IDEA Learning Objectives1 English 121: Composition 8, 9 Biology 101: General Biology 1, 2 Math 108: 1, 2, 5 Business Math Art History 104: Art History 1 6, 4, 1, 2 History 121: The Ancient World 1, 11, 8 Psychology 101: General Psychology 1, 2, 3, Anthropology 110: Global Archeology 7, 10,12 1Based partly on study of percent of classes selecting objectives by discipline What learning are we targeting? IDEA Learning Objectives Name of Course 1 2 3 4 5 6 7 8 9 English 121: Composition X X X Biology 101: General Biology X X Math 108: Business Math X X Art History 104: Art History 1 X History 121: The Ancient World X X 11 X X X X X 12 X X Psychology 101: General Psychology Anthropology 110: Global Archeology X 10 X X X X X What learning are we targeting? IDEA Learning Objectives Name of Course 1 2 3 4 5 6 7 8 9 English 121: Composition X X X Biology 101: General Biology X X Math 108: Business Math X X Art History 104: Art History 1 X History 121: The Ancient World X X 11 X X X X X 12 X X Psychology 101: General Psychology Anthropology 110: Global Archeology X 10 X X X X X Using GSRs to Analyze Targeted Learning Page 2 Percent of Classes Selecting Obj. as Important or Essential This Group Institution IDEA System Objective 1 16% 70% 78% Objective 2 13% 59% 75% Objective 3 41% 58% 75% Objective 4 32% 35% 55% Objective 5 23% 19% 32% Objective 6 32% 14% 25% Objective 7 22% 27% 27% Objective 8 78% 43% 47% Objective 9 19% 23% 41% Objective 10 7% 11% 23% Objective 11 68% 42% 49% Objective 12 20% 23% 41% Average # of Obj. Selected 3.7 4.2 5.7 How do students rate their learning? GSR page 3 Do Students’ report of learning meet our expectations? Objective 1: Gaining factual knowledge (terminology, classifications, methods, trends ) Raw Average Adj. Average # of Classes This Report 3.9 3.9 11 Institution 4.2 4.2 3,963 IDEA System 4.0 4.0 31,991 GSR, Pages 5 and 6 This report At least 4.0 Institution IDEA System 0 20 40 60 80 100 Which Teaching Methods Should We Target for Improvement? % of Classes Where Method was “Infrequently” Used Teaching Method 6 Frequently Infrequently Teaching Method 10 0 20 40 60 4. Use Data from IDEA to Target Specific Needs Collect Feedback Improve Reflect & Discuss Interpret Results Read & Learn HCC General Education Goals and IDEA Howard Community IDEA Learning Goals College General Education Goals IDEA Teaching Methods Associated with Progress on Learning Goal Written Communication 8 Developing skill in expressing myself orally or in writing 5, 4, 14, 15, 18 Oral Communication 7 Gaining a broader understanding and appreciation of intellectual/cultural activity 7, 8, 13, 19 Scientific and Quantitative Reasoning 2 Learning fundamental principles, generalizations, or theories 11 Learning to analyze and critically evaluate ideas, arguments, and points of view 2, 7, 8, 13, 15, 16, 18, 19 9 Learning how to find and use resources for answering questions or solving problems 2, 7, 8, 9, 13, 14, 15, 18, 19 7. Gaining a broader understanding of intellectual/cultural activity Extra Question 7, 8, 13,19 Critical Thinking Global Competency Example: Pre-Pharmacy Pre-Pharmacy Goals & Outcomes IDEA data Mastery of fundamental biological, chemical, and physical principles underlying the living systems Learning Objective 2, Teaching Methods 2, 4, 6, 8, 10, 12, 15, 16 Apply knowledge of scientific principles and concepts to clinical/scientific problems Learning Objective 3 Teaching Methods 1, 2, 3, 4, 6, 8, 10, 11, 15 Prepare effective written reports and oral presentations to communicate science to professionals and lay persons Learning Objective 8 Teaching Methods 7, 15, 16, 18, 19 Use mathematical concepts to analyze data Learning Objectives 2 & 3 Teaching Methods 1, 2, 3, 4, 6, 8, 10, 15 Example: Pre-Pharmacy Pre-Pharmacy Goals & Outcomes IDEA data Mastery of fundamental biological, chemical, and physical principles underlying the living systems Learning Objective 2, Teaching Methods 2, 4, 6, 8, 10, 12, 15, 16 Apply knowledge of scientific principles and concepts to clinical/scientific problems Learning Objective 3 Teaching Methods 1, 2, 3, 4, 6, 8, 10, 11, 15 Prepare effective written reports and oral presentations to communicate science to professionals and lay persons Learning Objective 8 Teaching Methods 7, 15, 16, 18, 19 Use mathematical concepts to analyze data Learning Objectives 2 & 3 Teaching Methods 1, 2, 3, 4, 6, 8, 10, 15 Overall Progress Ratings (Courses) Page 3 Percent of Classes at or Above the IDEA database Average 90 80 70 60 50 Raw Adjusted 40 30 20 10 0 PRO Excellent Excellent Summary Teacher Course Correlation of Teaching Method Items to Summary Items IDEA Teaching Method 6. Make it clear how each topic fits into the course 10. Explain course material clearly and concisely 13. Introduce stimulating ideas about the subject 4. Demonstrate the importance and significance of the subject matter Excellent Teacher Excellent Course .84 .80 .90 .79 .83 .82 .83 .80 Other Considerations O Consider sending faculty to the IDEA User Group Meeting to do presentations O Consider becoming a member of POD and joining these conversations on the international level O Design a Faculty Development Day around some of these ideas Questions? shelley@theideacenter.org