Aviation Assessment Report, December 8, 2011 Central Washington University Assessment of Student Learning – Aviation Academic Year of Report: Spring 2010 to Spring 2011* College: _____CEPS____ Department ____Aviation______ Program: Bachelor of Science in Flight Technology Specializations: Flight Officer (FO) & Commercial Pilot (CP) *Due to the difficulty of obtaining thorough and complete data in time for an annual June 15 deadline— especially flight lab data—the department had adjusted the report year to a spring-winter time frame. However, since the due dates for annual Assessment Reports have now been pushed back to December, this report spans a 15-month period from the beginning of spring 2010 quarter to the end of the spring 2011 quarter. Next year’s report will include data from fall 2011, winter 2012 and spring 2012 quarters. Check here if your assessment report covers all undergraduate degree programs: [ X ] Check here if your assessment report covers all graduate degree programs: [ ] 1. What student learning outcomes were assessed this year, and why? The four learner outcomes listed below were assessed this year because data were collected for all areas (knowledge, skills, and attitudes) pertaining to those criteria and programs. The department will assess other specializations in following years after a more comprehensive data set is collected so as to make the assessment meaningful and useful. 1. Apply the foundational knowledge, skills, and attitudes necessary to fly commercial transport aircraft in a safe and efficient manner. This outcome relates to Department Goal 1 - Provide for an outstanding academic, professional growth experience for students at all CWU locations; College Goal 2 - Prepare students to participate in an increasingly diverse economy and environment; University Goals 1 - Maintain and strengthen an outstanding academic and student life on the Ellensburg Campus and 6 - Build inclusive and diverse campus communities that promote intellectual inquiry and encourage civility, mutual respect, and cooperation. Assessments were conducted to meet department and Federal Aviation Administration Title 14 Code of Federal Regulations (CFR) Part 141 regulatory and quality of instruction standards. 2. Interpret, calculate, and analyze meteorological, aerodynamic, performance, and loading factors to conduct safe and efficient commercial flights. This outcome relates to Department Goal 1 (see above); College Goal 2 (see above) and University Goals 1 and 6 (see above). Assessments were conducted to meet department and Federal Aviation Administration Title 14 CFR Part 141 regulatory and quality of instruction standards. 3. Demonstrate knowledge, skills, and attitudes to effectively operate aircraft airframe, powerplant, navigation, and communication systems in a multi-crew environment, including Crew Resource Management (CRM), Hazardous Attitudes Assessment, and Aeronautical Decision Making (ADM). This outcome relates to Department Goal 1 (see above); College Goal 2 (see above) and University Goals 1 and 6 (see above). Assessments were conducted to meet department and Federal Aviation Administration Title 14 CFR Part 141 regulatory and quality of instruction standards. 4. Describe the physiological and psychological limitations to flight crew performance (including personal limitations, hazardous attitudes and antidotes, and pilot decision making) and effectively cope with these limitations in the flight environment. This outcome relates to Department Goal 1 (see above); College Goal 2 (see above) and University Goals 1 and 6 (see above). Assessments were conducted to meet 1 Aviation Assessment Report, December 8, 2011 department and Federal Aviation Administration 14 CFR Part 141 regulatory and quality of instruction standards. 2. How were they assessed? A) Methods of Assessment and B) Who was assessed: Outcomes 1, 2, 3, and 4 were assessed at the Ellensburg campus through FAA written examinations (29 students enrolled in FLT 142, 13 students enrolled in FLT 242, 40 students enrolled in two sections of FLT 354, and 37 students enrolled in two sections of FLT 358), EOC written examinations (30 students enrolled in FLT 142, 16 students enrolled irrfn FLT 242, 17 students enrolled in FLT 352, 41 students enrolled in two sections of FLT 354, and 38 students enrolled in two sections of FLT 358), and FAA EOC practical flight examinations (27 students enrolled in lab FLT 103, 38 students enrolled in FLT 203, 14 students enrolled in FLT 301, 16 students enrolled in FLT 306, and 18 students enrolled in FLT 401 who completed these labs by the end of Spring 2011 quarter). Note: these courses and examinations are taught at the CWU campus—they are not taught or administered by CWU at the Moses Lake Center (MLC); students transfer in to CWU with equivalents completed at Big Bend Community College. All examinations were graded based on FAA computer test standards or FAA Practical Test Standards pertinent to the certificate or rating for which the student was enrolled. Skills assessed include the following (non-comprehensive) list of content knowledge, operational skills, and appropriate attitudes as applied to each lab course, EOC exam, or FAA certificate as listed in documents FAA-S-8081-14A, FAA-S-8081-4D, FAA-S-8081-12B, or FAA-S-8081-6C with Changes 1 & 2). See www.faa.gov for full text of those criteria – those documents are far too extensive to include in this report. Aircraft control (all flight maneuvers required for FAA certifications) Flight planning and decision making Navigational skills Aircraft systems knowledge Airport operations Air Traffic control procedures (VFR and IFR) and radar services Meteorological considerations Interpretation and application of weather reports and forecasts Theory and application of instrument navigation Assessment of hazardous attitudes Application of pilot decision making skills Situational awareness Workload management Aeromedical factors Aerodynamics Weight and balance Night operations High altitude operations Federal Aviation regulations and publications The National Airspace System Airworthiness requirements Fundamentals of instruction 2 Aviation Assessment Report, December 8, 2011 Human behavior Commercial pilot operations Instrument flight rules Pilot certification and logbook endorsements Flight Instructor responsibilities In addition, Outcomes 1 through 4 were partially assessed through surveys given to 21 of students at the Ellensburg campus who completed FLT 142, 15 students at the Ellensburg campus who completed FLT 242, and 9 students at the Ellensburg campus who completed FLT 354. The Likert-scale survey is designed to measure students’ perceptions of their experience in these courses. Outcome 2 was also partially assessed through pass rates of 41 students who completed FLT 211 Meteorology for Pilots, and 29 students who completed FLT 312 Aviation Weather Services. These two courses are not taught by CWU at the MLC, so only Ellensburg students were assessed. Crew Resource Management (CRM) skills in Student Learner Outcome 3 were also partially assessed through pass rates of 26 students who completed FLT 489 Turbojet FTD, CRJ-200 spring 10, winter 11 and spring 11 quarters. This course is not offered at the MLC, so only Ellensburg students were assessed. Outcome 4 was also partially assessed through surveys given to seventeen (17) students at the Ellensburg campus who completed FLT 337 Aviation Physiology & Survival (one section) and forty-three (43) students at the Moses Lake Center who completed FLT 337 Aviation Physiology & Survival (three sections). It was also partially assessed through surveys given to forty-three (43) students at the Ellensburg campus who completed FLT 340 Human Factors in Flight (two sections) and ten (10) students at the Moses Lake Center who completed FLT 340 Human Factors in Flight (one section). The Likert-scale survey is designed to measure self-perception of improvement of knowledge of the physiological and psychological limitations of pilot performance and improvement in their ability to effectively cope with these limitations in the flight environment as a result of taking these courses. Outcome 1, along with measures of general satisfaction with the CWU and Midstate Aviation flight training program, was partially assessed through a survey given to 15 and 12 graduating senior Ellensburg campus Flight Officer (FO) & Commercial Pilot (CP) students who participated in the “Flight Technology Senior Exit Survey” at the end of the spring 2010 and 2011 quarters respectively. C) When it was assessed: Students were assessed each quarter depending on which FAA course they were enrolled in: Spring quarter 2010: FLT 103, FLT 203, FLT 301, FLT 306, FLT 354, FLT 358, FLT 401, FLT 358 Fall quarter 2010: FLT 103, FLT 203, FLT 301, FLT 306, FLT 401 Winter quarter 2011: FLT 103, FLT 142, FLT 203, FLT 242, FLT 301, FLT 306, FLT 352, FLT 401 Spring quarter 2011: FLT 103, FLT 203, FLT 301, FLT 306, FLT 354, FLT 358, FLT 401 Students enrolled in FLT 142 and FLT 242 were assessed through a survey administered at the end of winter quarter 2011. Students enrolled in FLT 354 were assessed through a survey administered at the end of spring quarter 2010 (a survey was not administered in spring 2011). Students enrolled in FLT 211 Meteorology for Pilots and FLT 312 Aviation Weather Services, were assessed at the end of winter 2011 quarter. 3 Aviation Assessment Report, December 8, 2011 Students enrolled in FLT 489 Turbojet FTD, CRJ-200 were assessed in the quarter they took the course (spring 2010, winter 2011, spring 2011). Students who completed FLT 337 Aviation Physiology & Survival and FLT 340 Human Factors in Flight were assessed through surveys administered at the end of their respective quarters (spring 2010, fall 2011, winter 2011, spring 2011). 3. What was learned? The following tables show stated criteria and goals for each criterion. Criteria 1, 2, 3, 4, 5 and 6 below are applicable to all four student learning outcomes: Students will: 1. Apply the foundational knowledge, skills, and attitudes necessary to fly commercial transport aircraft in a safe and efficient manner. 2. Interpret, calculate, and analyze meteorological, aerodynamic, performance, and loading factors to conduct safe and efficient commercial flights. 3. Demonstrate knowledge, skills, and attitudes to effectively operate aircraft airframe, powerplant, navigation, and communication systems in a multi-crew environment, including Crew Resource Management (CRM), Hazardous Attitudes Assessment, and Aeronautical Decision Making (ADM). 4. Describe the physiological and psychological limitations to flight crew performance (including personal limitations, hazardous attitudes and antidotes, and pilot decision making) and effectively cope with these limitations in the flight environment. Criterion 1: Ellensburg campus students enrolled in FAA ground schools Passed FAA written exam on the first attempt with a minimum score of 80% Quarter Class n Spr 10 FLT 354 FLT 358 FLT 142 FLT 242 FLT 354 FLT 358 23 Spr 10 Win11 Win11 Spr11 Spr11 20 Criterion 1 Passed FAA written exam on the first attempt with a minimum score of 80%. 82.6% Goals for Criterion 1: 80% of students in lower division classes and 90% of students in upper division classes met criterion 1 Not Met FAA National Average Test Scores 84.7% 86.6% FOI FIA 92% 85.1% 86.5% FOI FIA 88.3% 83.8% 84.2% 29 FOI FIA 100% 80% 79.3% 13 77% Not Met 82.9% 80.2% 17 88.2% Not Met 88.5 86.6 17 FOI 35.3% FIA 58.8% FOI FIA Exceeded Not Met Not Met CWU Average Test Score, FAA written FOI Not Met 4 FIA Not Met FOI 76.8 FIA 81.8 FOI n/a FIA n/a Aviation Assessment Report, December 8, 2011 Criterion 2: Ellensburg campus students enrolled in FAA ground schools Passed FAA EOC written exam on the first attempt with a minimum score of 80% Quarter Class n Goals for Criterion 2 Passed EOC written exam on the first attempt with a minimum score of 80%. Spr 10 Spr 10 Win11 Win11 Win11 Spr11 Spr11 FLT 354 FLT 358 FLT 142 FLT 242 FLT 352 FLT 354 FLT 358 24 21 30 16 17 17 17 100% 90.5% 90% 56.3% 94.4% 100% 100% Goals for Criterion 2: 80% of students in lower division classes and 90% of students in upper division classes met criterion 2 Exceeded Exceeded Exceeded Not Met Exceeded Exceeded Exceeded Criterion 3: Ellensburg campus students enrolled in flight labs Passed FAA EOC practical flight exam on the first attempt. Quarter Class *n Goals for Criterion 3 Goals for Criterion 3: % Passed EOC flight 80% of students in lower division classes and exam on the first 90% of students in upper division classes met attempt criterion 3 Spr10 FLT103 21/24 88% Exceeded FLT203 20/23 87% Exceeded FLT301 4/4 100% Exceeded FLT306 0/0 n/a -FLT401 4/6 67% Not met Fall 10 FLT103 1/1 100% Exceeded FLT203 3/3 100% Exceeded FLT301 7/8 88% Not met FLT306 1/2 50% Not met FLT401 6/9 67% Not met Win 11 FLT103 1/1 100% Exceeded FLT203 0/0 n/a -FLT301 2/2 100% Exceeded FLT306 9/11 82% Not met FLT401 1/1 100% Exceeded Spr11 FLT103 1/1 100% Exceeded FLT203 10/12 83% Exceeded FLT301 0/0 0% -FLT306 3/3 100% Exceeded FLT401 2/2 100% Exceeded *n = number of students who passed EOC flight exam on the first attempt / number of students who completed EOC flight exam 5 Aviation Assessment Report, December 8, 2011 Criterion 4: Survey results from 21 students at the Ellensburg campus who completed FLT 142. Question: Private Pilot Ground School 1. The Flt 141/142 Private Pilot Ground School has met the requirements required by the Federal Aviation Regulations. 2 The Private Pilot Ground School training met my expectations. 3. I feel that the Private Pilot Ground School has prepared me to pass the FAA Written Test. 4. I was able to understand aircraft performance information and how to go about calculating it. 5. The Private pilot ground school gave me an overall good understanding of airplane aerodynamics and aircraft systems. 6. After the ground school, I am now able to distinguish between the different classes of airspace and identify the rules for each. 7. The Federal Aviation Regulations (FAR’s) were presented in a way that allowed me the understand them. 8. When going through the basic meteorological process, I was able to understand the information being presented. 9. If I had difficulty on certain subject areas, I was able to obtain help outside the classroom. 10. I am overall satisfied with the Aviation classes that I have taken so far. 11. I feel that the Private Pilot Courses (Flt 141, 142, 221, 211) has prepared me to be a safe and competent private pilot. 12. I feel that the training received in the FAA 141 ground school classes and the FAA 141 flight training received at Midstate were complementary in terms of timing and instruction. Total Quarter Winter 10 Students Surveyed n FLT 142 21 Strongly Disagree 1 5% Disagree Neutral 1 5% 1 5% 13 62% 13 62% 7 33% 5 24% 11 52% 15 71% 1 5% 1 5% 9 42% 10 48% 3 14% 8 38% 10 48% 2 10% 8 38% 11 52% 1 5% 4 19% 5 24% 3 15% 15 71% 15 71% 17 85% 1 5% 6 30% 13 65% 6 (2%) Goals for Criterion 4 6 7 33% 7 33% 1 5% 80% percent agree or strongly agree 93% 3 14% Strongly Agree 17 81% 2 10% 1 5% 1 5% 1 5% 3 (1%) Agree 9 (4%) 72 (29%) 160 (64%) Goals for Criterion 4: 80% of students surveyed met criterion 4 Exceeded Aviation Assessment Report, December 8, 2011 Criterion 5: Survey results from 15 students at the Ellensburg campus who completed FLT 242. Question: Instrument Pilot Ground School 1. The Flt 242/242 Instrument Pilot Ground School has met the requirements required by the Federal Aviation Regulations. 2 The Instrument Pilot Ground School training met my expectations. 3. I feel that the Instrument Pilot Ground School has prepared me to pass the FAA Written Test. 4. I was able to understand the flight instruments and understand how they operate. 5. Holding was presented to me in a way that allowed me to understand and conduct holding. 6. Through the instrument ground school, I am now able to interpret and understand instrument approach charts. 7. If I had difficulty on certain subject areas, I was able to obtain help outside the classroom. 8. I have seen program changes occur as a result of the feedback that I have provided. 10. I am overall satisfied with the Aviation classes that I have taken so far. 11. I feel that the Instrument Pilot Courses (Flt 241, 242, 312) has prepared me to be a safe and competent instrument pilot. 12. I feel that the training received in the FAA 141 ground school classes and the FAA 141 flight training received at Midstate were complementary in terms of timing and instruction. Total Strongly Disagree Disagree Neutral 1 (6% 3 (20% 1 (6% 1 (6% 2 (13%) 1 (6% 1 (6% 2 (13% 2 (13%) 4 (27% 1 (6%) 2 (13%) Agree Strongly Agree 4 (27%) 11 (73%) 2 (14%) 6 (40%) 9 (60%) 3 (20%) 2 (13%) 4 (27%) 10 (67%) 7 (47%) 4 (27%) 6 (40%) 4 (27%) 8 (53%) 4 (20%) 6 (52%) 1 (6%) 3 (20%) 1 (6%) 2 (1%) 8 (5%) 16 (10%) 46 (28%) 7 (48%) 10 (68%) 11 (73%) 4 (27%) 8 (54%) 9 (60%) 93 (56%) Results from question 9, which is structured differently than the rest of the questions above, are included below: 9. The quality of instruction I have received this year compared to last year has: Quarter Winter 11 Students Surveyed n FLT 242 15 N/A 1 (7%) Decreased 3 (20%) Goals for Criterion 5 80% percent agree or strongly agree 84% 7 Same 5 (33%) Improved 6 (40%) Goals for Criterion 5: 80% of students surveyed met criterion 5 Exceeded Aviation Assessment Report, December 8, 2011 Criterion 6: Survey results from 9 students at the Ellensburg campus who completed FLT 354. Question: Commercial Pilot Ground School 1. The Flt 354 Commercial Pilot Ground School has met the requirements required by the Federal Aviation Regulations. 2 The Commercial Pilot Ground School training met my expectations. 3. I feel that the Commercial Pilot Ground School has prepared me to pass the FAA Written Test. 4. The advanced systems were presented to me in a way that allowed me to understand them. 5. I was able to understand advanced aircraft performance information and how to go about calculating it. 6. The weight and balance and weight shift formula were covered enough to give an understanding of their operations. 7. If I had difficulty on certain subject areas, I was able to obtain help outside the classroom. 8. I have seen program changes occur as a result of the feedback that I have provided. 10. I am overall satisfied with the Aviation classes that I have taken so far. 11. I feel that the Commercial Pilot Courses (Flt 354, 322, 340) has prepared me to be a safe and competent instrument pilot. 12. I feel that the training received in the FAA 141 ground school classes and the FAA 141 flight training received at Midstate were complementary in terms of timing and instruction. Total Strongly Disagree Disagree 2 Strongly Agree 7 1 4 4 1 3 5 1 3 4 1 4 4 1 4 4 1 1 4 3 2 4 2 1 1 2 2 4 2 3 4 5 3 1 Neutral 1 0 (0%) 16 (16%) 14 (14%) Agree 36 (36%) 43 (44%) Results from question 9, which is structured differently than the rest of the questions above, are included below: 9. The quality of instruction I have received this year compared to last year has: Quarter Spring 10 Students Surveyed n FLT 354 9 Decreased: 3 (33%) Goals for Criterion 6 80% percent agree or strongly agree 80% 8 Same: 4 (44%) Improved: 2 (23%) Goals for Criterion 6: 80% of students surveyed met criterion 6 Met Aviation Assessment Report, December 8, 2011 Criterion 7 results below apply to Student Learner Outcome 2: Interpret, calculate, and analyze meteorological, aerodynamic, performance, and loading factors to conduct safe and efficient commercial flights. Ellensburg campus students enrolled in FLT 211 Meteorology for Pilots and FLT 312 Aviation Weather Services who passed the course with a minimum of 80% mastery level. Quarter Winter 11 Class FLT 211 FLT 312 n Goals for Criterion 7 41 29 Passed course with a minimum score of 80%. 90% 95% Goals for Criterion 7: 80% of students in each course met criterion 7 Exceeded Exceeded Criterion 8 results below apply to Student Learner Outcome 3: Demonstrate knowledge, skills, and attitudes to effectively operate aircraft airframe, powerplant, navigation, and communication systems in a multi-crew environment, including Crew Resource Management (CRM), Hazardous Attitudes Assessment, and Aeronautical Decision Making (ADM). Ellensburg campus students enrolled in FLT 489 Turbojet FTD, CRJ-200 who passed the course with a minimum of 80% mastery level. Quarter Spring 10 Fall 10 Winter 11 Spring 11 Class FLT 489 FLT 489 FLT 489 FLT 489 n Goals for Criterion 8 10 0 6 10 Passed course with a minimum score of 80%. 100% n/a 100% 100% 9 Goals for Criterion 8: 80% of students in course met criterion 8 Exceeded n/a Exceeded Exceeded Aviation Assessment Report, December 8, 2011 Criterion 9 results below apply to Student Learner Outcome 4: Describe the physiological and psychological limitations to flight crew performance (including personal limitations, hazardous attitudes and antidotes, and pilot decision making) and effectively cope with these limitations in the flight environment. Results for 20 students who completed FLT 337 Aviation Physiology & Survival and participated in the survey below at the Moses Lake Center Spring 2010. Question: “As a result of taking FLT 337 … 1. …my understanding of the physiological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the physiological limitations of flight” 3. … I’m more confident in my ability to successfully avoid the hazards associated with these physiological limitations” 4. …I believe my ability to make safe aeronautical decisions has significantly improved” Total Quarter Spring 10 Strongly Disagree 0 (0%) Disagree 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 337 20 80% percent agree or strongly agree 96% 10 Neutral 15 (75%) Strongly Agree 5 (25%) 14 (70%) 6 (30%) 1 (5%) 14 (70%) 5 (25%) 2 (10%) 9 (45%) 9 (43%) 3 (4%) Agree 52 (65%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 25 (31%) Aviation Assessment Report, December 8, 2011 Results for 17 students who completed FLT 337 Aviation Physiology & Survival and participated in the survey below at the Ellensburg campus Fall 2010. Question: “As a result of taking FLT 337 … 1. …my understanding of the physiological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the physiological limitations of flight” 3. … I’m more confident in my ability to successfully avoid the hazards associated with these physiological limitations” 4. …I believe my ability to make safe aeronautical decisions has significantly improved” Total Quarter Fall 10 Strongly Disagree 1(6%) Disagree Neutral Agree 7 (41%) Strongly Agree 9 (53%) 1(6%) 1(6%) 5 (29%) 10 (59%) 1(6%) 2(12%) 3 (18%) 11 (65%) 1(6%) 1(6%) 6 (35%) 9 (53%) 4 (6%) 21 (31%) 4 (6%) 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 337 17 80% percent agree or strongly agree 88% 11 Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 39 (57%) Aviation Assessment Report, December 8, 2011 Results for 10 students who completed FLT 337 Aviation Physiology & Survival and participated in the survey below at the Moses Lake Center Fall 2010. Question: “As a result of taking FLT 337 … 1. …my understanding of the physiological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the physiological limitations of flight” 3. … I’m more confident in my ability to successfully avoid the hazards associated with these physiological limitations” 4. …I believe my ability to make safe aeronautical decisions has significantly improved” Total Quarter Fall 10 Strongly Disagree 0 (0%) Disagree 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 337 10 80% percent agree or strongly agree 90% 12 Neutral Agree 1(10%) 4 (40%) Strongly Agree 5 (50%) 1(10%) 2 (20%) 7 (70%) 1(10%) 3(30%) 6 (60%) 1(10%) 3 (30%) 6 (60%) 4 (10%) 12 (30%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 24 (60%) Aviation Assessment Report, December 8, 2011 Results for 13 students who completed FLT 337 Aviation Physiology & Survival and participated in the survey below at the Moses Lake Center Spring 2011. Question: “As a result of taking FLT 337 … 1. …my understanding of the physiological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the physiological limitations of flight” 3. … I’m more confident in my ability to successfully avoid the hazards associated with these physiological limitations” 4. …I believe my ability to make safe aeronautical decisions has significantly improved” Total Quarter Spring 2011 Strongly Disagree Disagree Neutral 1(8%) 0 (0%) 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 337 13 80% percent agree or strongly agree 98% 13 1 (2%) Agree 6 (46%) Strongly Agree 6 (46%) 5 (42%) 7 (58%) 6(46%) 7 (54%) 5 (42%) 7 (58%) 22 (44%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 27 (54%) Aviation Assessment Report, December 8, 2011 Results for 17 students who completed FLT 340 Human Factors in Flight and participated in the survey below at the Ellensburg campus Spring 2010. Question: “As a result of taking FLT 340 … 1. …my understanding of the psychological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the psychological limitations to flight and avoid them” 3. …I believe my ability to make safe aeronautical decisions has significantly improved” 4. …I better understand the concepts of crew resource management and as a result I feel I can effectively use all available resources to achieve safe and efficient flight.” Total Quarter Spring 10 Strongly Disagree Disagree 1 (6%) 0 (0%) 1 (1%) Students Surveyed n Goals for Criterion 9 FLT 340 17 80% percent agree or strongly agree 88% 14 Neutral Agree 3(18%) 7 (41%) Strongly Agree 7 (41%) 8 (47%) 9 (53%) 2 (11%) 10 (59%) 4 (24%) 2 (11%) 10 (59%) 5 (30%) 7 (10%) 35 (51%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 25 (37%) Aviation Assessment Report, December 8, 2011 Results for 26 students who completed FLT 340 Human Factors in Flight and participated in the survey below at the Ellensburg campus Fall 2010. Question: “As a result of taking FLT 340 … 1. …my understanding of the psychological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the psychological limitations to flight and avoid them” 3. …I believe my ability to make safe aeronautical decisions has significantly improved” 4. …I better understand the concepts of crew resource management and as a result I feel I can effectively use all available resources to achieve safe and efficient flight.” Total Quarter Fall 10 Strongly Disagree 0 (0%) Disagree 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 340 26 80% percent agree or strongly agree 94% 15 Neutral Agree 2(8%) 17 (65%) Strongly Agree 7 (27%) 2(8%) 14 (54%) 10 (38%) 2(8%) 13(50%) 11 (42%) 16 (62%) 10 (38%) 6 (6%) 60 (57%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 38 (37%) Aviation Assessment Report, December 8, 2011 Results for 10 students who completed FLT 340 Human Factors in Flight and participated in the survey below at the Moses Lake Center Fall 2010. Question: “As a result of taking FLT 340 … 1. …my understanding of the psychological aspects of flight has grown considerably.” 2. …I’m more confident in my ability to recognize the psychological limitations to flight and avoid them” 3. …I believe my ability to make safe aeronautical decisions has significantly improved” 4. …I better understand the concepts of crew resource management and as a result I feel I can effectively use all available resources to achieve safe and efficient flight.” Total Quarter Fall 10 Strongly Disagree 0 (0%) Disagree 0 (0%) Students Surveyed n Goals for Criterion 9 FLT 340 10 80% percent agree or strongly agree 90% 16 Neutral Agree 1(10%) 6 (60%) Strongly Agree 3 (30%) 1(10%) 7 (70%) 2 (20%) 1(10%) 7 (70%) 2 (20%) 1(10%) 5 (50%) 4 (40%) 4 (10%) 25 (62%) Goals for Criterion 9: 80% of students surveyed met criterion 9 Exceeded 11 (28%) Aviation Assessment Report, December 8, 2011 Criterion 10 results below partially assess Student Learner Outcome 1: Apply the foundational knowledge, skills, and attitudes necessary to fly commercial transport aircraft in a safe and efficient manner. It is also included to indicate overall general strengths and weaknesses, from the graduate’s perspective, of the CWU and Midstate Aviation flight training program. Results for 15 graduating senior Flight Officer (FO) & Commercial Pilot (CP) students who participated in the “Flight Technology Senior Exit Survey” at the Ellensburg campus Spring 2010. N/A Strongly Disagree Disagree 1. Central Washington University’s Flight Technology Program met my expectations. 2. This program prepared me for entry into the aviation industry. 6 Strongly Agree 9 5 10 3. I felt confident in Flight Technology faculty knowledge. 2 13 4. Flight Technology faculty were approachable. 5 10 5. Flight Technology faculty effectively taught industry related material. 6 9 12 2 5 10 3 12 1 3 11 2 3 8 3 4 8 1 9 3 3 4 7 1 1 8 5 2 3 6 4 3 6. Flight Technology faculty varied their delivery methods of course material. 7. Flight Technology faculty conducted themselves professionally. 8. The Flight Technology faculty treated me with respect. 9. The academic advising provided by the faculty every quarter was useful in completing my degree. 10. I felt knowledgeable about the activities taking place in the program. 2 11. I felt confident in the Flight Technology part-time lecturer’s knowledge. 1 12. Flight Technology part time lecturers effectively taught industry related material. 13. Text books required for Flight Technology courses were relevant to course material. 14. Flight Technology curricula appears relevant to the aviation industry. Neutral 15. I have had an opportunity to 17 Agree Aviation Assessment Report, December 8, 2011 provide feedback to the Flight Technology program. 16. Changes occurred as a result of feedback I have provided. 3 7 2 3 17. I understand why changes may not have occurred as a result of feedback provided. 2 5 5 2 18. Flight Technology curricula and Midstate Aviation curricula are complementary. 6 6 3 19. Effective communication existed between Midstate Aviation and Flight Technology. 6 1 6 2 20. Expectations listed in the Standard Operating Procedures in relation to the completion of flight training were reasonable. 5 2 3 5 21. I was scheduled enough on the Master schedule to complete my certificates/ratings on time. 5 1 1 7 22. Midstate Aviation flight instructors were approachable. 5 2 8 23. Midstate Aviation flight instructors treated me with respect. 5 1 9 24. Midstate Aviation flight instructors returned my inquiries. 5 2 8 25. Midstate Aviation flight instructors used scheduled flight time effectively. 4 2 8 26. Midstate Aviation flight instructors used scheduled ground time effectively. 4 4 6 27. I felt confident in Midstate Aviation flight instructors’ knowledge. 4 1 10 28. Midstate Aviation flight instructors conducted standardized instruction. 4 2 9 29. Midstate Aviation flight instructors were prepared for lessons. 4 1 10 30. Midstate Aviation flight instructors were available to meet my scheduling needs. 4 3 5 31. Midstate Aviation flight instructors conducted themselves professionally. 4 1 10 32. The Chief Flight Instructor and Assistant Chief Flight Instructor were approachable. 4 4 7 1 1 1 1 3 18 Aviation Assessment Report, December 8, 2011 33. The Chief Flight Instructor and Assistant Chief Flight Instructor treated me with respect. 4 1 10 34. Midstate Aviation office personnel were approachable. 4 3 7 35. Midstate Aviation office personnel treated me with respect. 4 3 8 36. Midstate Aviation office personnel considerately communicated with me regarding scheduling issues. 4 1 8 37. Midstate Aviation office personnel considerately communicated with me regarding charges to my account. 4 3 8 38. Aircraft rental rates are comparable to other flight schools with equivalent aircraft. 4 4 2 3 39. The quality of Midstate Aviation aircraft are above average. 6 1 2 6 40. The maintenance of Midstate Aviation aircraft is excellent. 6 41. The flight training I received from Midstate Aviation prepared me well for a career as a professional pilot. 6 5 4 42. How would you rate the overall quality of flight instruction received from your flight instructors at Midstate Aviation, Inc.? 4 4 7 43. How would you rate the overall quality of instruction you received from Flight Tech Faculty? 1 4 10 44. How would you rate the overall quality of the CWU Flight Technology program? Total Total with “N/As” dropped 1 4 10 1 1 1 2 9 Please rate the following questions Quarter Students Surveyed 125(19%) 0 (0%) 9 (1%) 44 (7%) 159 (24%) -- (0%) 0(0%) 9 (2%) 44 (8%) 159 (30%) 323 (60%) n Goals for Criterion 10 80% percent agree/strongly agree and good/very good Spring 10 Graduating Seniors 15 *90% *”N/As” dropped from total to compute percentage for Criterion 10 19 323 (49%) Goals for Criterion 10: 80% of students surveyed met criterion 10 Exceeded Aviation Assessment Report, December 8, 2011 Results for 12 graduating senior Flight Officer (FO) & Commercial Pilot (CP) students who participated in the “Flight Technology Senior Exit Survey” at the Ellensburg campus Spring 2011. N/A Strongly Disagree Disagree Neutral Agree 1. Central Washington University’s Flight Technology Program met my expectations. 2. This program prepared me for entry into the aviation industry. 1 5 Strongly Agree 6 8 4 3. I felt confident in Flight Technology faculty knowledge. 1 2 9 1 11 4. Flight Technology faculty were approachable. 12 5. Flight Technology faculty effectively taught industry related material. 6. Flight Technology faculty varied their delivery methods of course material. 4 8 7. Flight Technology faculty conducted themselves professionally. 8. The Flight Technology faculty treated me with respect. 9. The academic advising provided by the faculty every quarter was useful in completing my degree. 10. I felt knowledgeable about the activities taking place in the program. 1 11 12 2 9 8 4 3 8 12. Flight Technology part time lecturers effectively taught industry related material. 5 7 13. Text books required for Flight Technology courses were relevant to course material. 2 10 14. Flight Technology curricula appears relevant to the aviation industry. 3 9 1 2 9 5 1 5 11. I felt confident in the Flight Technology part-time lecturer’s knowledge. 1 1 15. I have had an opportunity to provide feedback to the Flight Technology program. 16. Changes occurred as a result of feedback I have provided. 1 20 Aviation Assessment Report, December 8, 2011 1 17. I understand why changes may not have occurred as a result of feedback provided. 3 5 3 1 2 4 18. Flight Technology curricula and Midstate Aviation curricula are complementary. 5 19. Effective communication existed between Midstate Aviation and Flight Technology. 5 1 3 3 20. Expectations listed in the Standard Operating Procedures in relation to the completion of flight training were reasonable. 5 1 3 3 21. I was scheduled enough on the Master schedule to complete my certificates/ratings on time. 5 1 2 4 22. Midstate Aviation flight instructors were approachable. 5 1 1 4 23. Midstate Aviation flight instructors treated me with respect. 5 1 2 4 24. Midstate Aviation flight instructors returned my inquiries. 5 2 2 3 25. Midstate Aviation flight instructors used scheduled flight time effectively. 5 1 1 5 26. Midstate Aviation flight instructors used scheduled ground time effectively. 5 2 4 27. I felt confident in Midstate Aviation flight instructors’ knowledge. 5 1 5 28. Midstate Aviation flight instructors conducted standardized instruction. 5 1 3 3 29. Midstate Aviation flight instructors were prepared for lessons. 5 1 1 5 30. Midstate Aviation flight instructors were available to meet my scheduling needs. 5 1 1 5 31. Midstate Aviation flight instructors conducted themselves professionally. 5 1 32. The Chief Flight Instructor and Assistant Chief Flight Instructor were approachable. 5 2 1 4 33. The Chief Flight Instructor and Assistant Chief Flight Instructor treated me with respect. 5 1 1 5 34. Midstate Aviation office 5 1 1 1 1 21 6 6 Aviation Assessment Report, December 8, 2011 personnel were approachable. 35. Midstate Aviation office personnel treated me with respect. 5 1 36. Midstate Aviation office personnel considerately communicated with me regarding scheduling issues. 5 1 1 5 37. Midstate Aviation office personnel considerately communicated with me regarding charges to my account. 5 1 1 5 38. Aircraft rental rates are comparable to other flight schools with equivalent aircraft. 5 3 1 39. The quality of Midstate Aviation aircraft are above average. 5 40. The maintenance of Midstate Aviation aircraft is excellent. 5 41. The flight training I received from Midstate Aviation prepared me well for a career as a professional pilot. 5 1 6 42. How would you rate the overall quality of flight instruction received from your flight instructors at Midstate Aviation, Inc.? 5 1 6 43. How would you rate the overall quality of instruction you received from Flight Tech Faculty? 1 2 9 6 6 94(18%) 94(23%) 264(50%) 264(66%) 3 6 3 4 7 Please rate the following questions 44. How would you rate the overall quality of the CWU Flight Technology program? Total 126(24%) 1(0%) Total with “N/As” dropped -- (0%) 1(0%) Quarter Students Surveyed n 12(2%) 12(3%) Goals for Criterion 10 80% percent agree/strongly agree and good/very good Spring 10 Graduating Seniors 15 *89% *”N/As” dropped from total to compute percentage for Criterion 10 22 31(6%) 31(8%) Goals for Criterion 10: 80% of students surveyed met criterion 10 Exceeded Aviation Assessment Report, December 8, 2011 Summary: Mean FAA written exam scores were above national means for 5 of the 6 certification exams (where national means are known) associated with FLT 142, 242, 354 and 358 (Criterion 1). EOC test performance of students in FLT 352 winter 2011 improved from 83 percent the previous year to over 94 percent (Criterion 2). The percentage of students who passed the aviation weather courses with a minimum score of 80 percent increased to 90% of 41 students enrolled in FLT 211 and 95% of students enrolled in FLT 312 respectively. During the previous reporting period 72% of 36 students enrolled in FLT 211 and 90% of students enrolled in FLT 312 passed the course with a minimum score of 80% (Criterion 7). Survey results continue to be very positive with 80 percent to 98 percent of responses indicating “Agree” or “Strongly Agree” for all surveys administered (Criteria 4, 5, 6, 9 & 10) compared to a range of 75% to 95% for the previous reporting period. o Survey results for FLT 142: 93% of responses were “agree” or “strongly agree;” last year it was 91%. o Survey results for FLT 242: 84% of responses were “agree” or “strongly agree;” last year it was 84%. o Survey results for FLT 354: 80% of responses were “agree” or “strongly agree;” last year it was 75%. o Average survey results for FLT 337 (four sections): 93% of responses were “agree” or “strongly agree;” last year it was 95%. o Senior Exit survey results Spring 2010: 90% of responses were “agree” or “strongly agree;” last year it was 82%. o Senior Exit survey results Spring 2011: 89% of responses were “agree” or “strongly agree;” last year it was 90%. The above data shows that goals were met or exceeded with exceptions in three areas: Criterion 1 o The goals for Criterion 1 were met in only one of the eight FAA written exams taken in six courses. During the previous reporting period the goals were met in six out of seven FAA written examinations taken in five courses. Failure to meet the goal of 90% of students passing the FAA written exam on the first attempt with a minimum score of 80% in FLT 358 Spring 2011 can be explained in part by an unexpected and significantly changed FOI exam. National first-time FOI pass rates (minimum score of 70% needed to pass) dropped to an estimated 48 percent. [It should be noted that the FOI first-time pass rates for CWU students was significantly higher at 82.3%.] Unexpected changes to the FAA instrument exam also explain performance in FLT 242. However, no such explanation can be given for FLT 142 and 354. Criterion 2 o As was the case during the previous assessment period, the goals for Criterion 2 were met with the exception of only one course. However, during this period, results for FLT 242 did not meet the goals while FLT 352 did not meet the goals in the previous period. Criterion 3 o To make comparisons between assessment reporting periods more meaningful, average EOC practical test pass rates were calculated across quarters for the current and previous periods (see tables below). Results indicate the goals for Criterion 3 were exceeded for four of the five flight labs and three of the five flight labs during the previous and current assessment reporting periods, respectively. However, during the present reporting period, results for 23 Aviation Assessment Report, December 8, 2011 FLT 306 and 401 did not meet the goals while FLT 301 did not meet the goals in the previous period. Total number of students who completed EOC practical flight exams and overall percentage who passed on the first attempt for previous reporting period (Spring 2009-Winter 2010). Class # of Goals for Criterion 3 Goals for Criterion 3: Passed EOC flight 80% of students in lower division classes and students exam on the first 90% of students in upper division classes met completed attempt Criterion 3 FLT103 29 93% Exceeded FLT203 25 80% Exceeded FLT301 6 66% Not met FLT306 11 100% Exceeded FLT401 9 100% Exceeded Total number of students who completed EOC practical flight exams and overall percentage who passed on the first attempt for this reporting period (Spring 2010-Spring 2011). Class *n Goals for Criterion 3 Goals for Criterion 3: Passed EOC flight 80% of students in lower division classes and exam on the first 90% of students in upper division classes met attempt Criterion 3 FLT103 24/27 89% Exceeded FLT203 33/38 87% Exceeded FLT301 13/14 93% Exceeded FLT306 13/16 81% Not met FLT401 13/18 72% Not met *n = number of students who passed EOC flight exam on the first attempt / number of students who completed EOC flight exam 4. What will the department or program do as a result of that information? The department feels that only minor changes are needed to improve the program. Criterion 1 Beginning Fall 2011, ground school instructors created additional homework assignments designed to assist students to master the material; they also created more comprehensive and challenging midterm, stage, and final examinations, relying on fewer published FAA test questions. This issue has been raised before (see previous reports) and faculty now agree that the 90% pass rate with a minimum score of 80% for FAA certification examinations associated with upper division ground school courses (FLT 354, 352, 358) is unrealistic and should be changed to 80%. This is consistent with other lower division FAA ground courses and will be reflected in next year’s Assessment Report. Criterion 2 Beginning Fall 2011, ground school instructors for Part 141 courses—including the instructor for AVP 242 (previously FLT 242) created additional homework assignments designed to assist students to master the material; they also created more comprehensive and challenging midterm, stage, and final examinations, relying on fewer published FAA text questions. 24 Aviation Assessment Report, December 8, 2011 Criterion 3 The issue of standards of mastery for pass rates in upper division FAA-related courses has been raised before (see previous reports); faculty now agree that the 90% criterion goal for FAA EOC practical examinations associated with upper division flight lab courses (FLT 301, 306, 401) is unrealistic and should be changed to 80%. This is consistent with other lower division FAA flight labs and will be reflected in next year’s Assessment Report. Lower performance in FLT 401 may reflect the recent change in FAA-designated check airmen examiners. Since the department now has a new course solely devoted to Crew Resource Management (AVP 410), next year’s assessment should include results from surveys for that class. 5. What did the department or program do in response to the feedback from last year's assessment report? Note: previous assessment reports used performance measures from FLT 445 Turboprop Simulator FMS for Criterion 8. FLT 489 has since become our “capstone course” for Flight Officer students expanding in size (now three credits) and scope to include training in CRM, EFIS, EICASA, FD, and automation management through simulated flights in a turbojet-powered CRJ-200 aircraft. A question was posed in last year’s “2009-2010 Assessment of Student Learning Report: Feedback for the Department of Aviation” about whether the established standards of mastery, especially for the surveys, were set too high or too low? As noted above, the department has decided that the standard of mastery for pass rates in upper division FAA-related courses —90% pass rate with a minimum score of 80%—is unrealistic and will be changed to 80% to be consistent with lower division courses. This will be reflected in next year’s Assessment Report. 6. Questions or suggestions concerning Assessment of Student Learning at Central Washington University: -none 25