Summary of IME Senior Project Presentation Assessment Results Spring 2005 Prepared by Dr. Phil Rosenkrantz Ten individual and team senior project presentations were given and evaluated on May 20, 2005 as part of the Engineering Projects Symposium Day. A scoring rubric was used that evaluated nine IME outcomes. A final assessment question asked whether or not the evaluator would hire the student or team based on what they observed. An overall average for each presentation was also computed. Six industry representatives performed the evaluations, although not every student was scored by all six evaluators. The Box Plots below summarize the results of the ten presentations by outcome. The lower line on each box represents the first quartile value. The middle line represents the median, and the top of the box represents the third quartile. Outliers are shown as “*”. Following the boxplot graphic is a graphical summary of overall performance by student/team followed by a general summary of written comments made by the industry reviewers on the evaluation sheets. Boxplot of Senior Project Presentation Average Scores - Spr 2005 Industry Evaluations Only (4 = Professional; 3 = Proficient, 2 = Marginal; 1 = Unacceptable) 4 Data 3 2 1 0 v Ad l at M ID ob Pr g En pr Ap h at M od M 10 presentations; 6 industry reviewers al su i V al Or W S/ al m ci te n s na Sy Fi re Hi ? g Av The bar chart below shows the percentage of scores each student/team received in the “Professional (4)” or “Proficient (3)” categories. Percentages ranged from a low of 33% to a high of 100%. The average was 68%. The interpretation of this chart is that any amount less than 100% is a proficiency gap. Percent Professional + Proficient Percentage of Professional + Proficient Scores 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 Student/Team Summary of Comments – Overall the comments reflect the scores above. Thirty-two percent of all scores were “Marginal (2)” or “Unacceptable (1)”. The reasons given include: 12a Financial – Lack of financial analysis given to justify project or recommendations. Very few presentations included ROI or Net Present Worth calculations. Many costs not mentioned or ignored. This was clearly the most common critical comment. 1b Advanced Material, 4a Mathematical Models – Lack of evidence that advance material was used to analyze and solve problems. Several presentations appeared to be handled by “common sense” rather than engineering methodology. 4b Oral Presentation – Several presenters talked way too fast. Some presentations did not flow well with items being repeated or points being poorly made. 4b Visual Presentation – Some presentations were praised and others were criticized. Biggest complaints were graphs with font too small to see or follow. Some students passed around samples that only served to distract. Preliminary Interpretation of Findings – There are many issues at play here. The lack of financial analysis is probably the result of three compounding problems: 1. Relaxation of emphasis on cost and ROI for all projects as other topics have been elevated 2. Students running out of gas and not including the financial analysis 3. Misalignment in the curriculum – EGR 403 taken very late in the curriculum by many students. Students not expected to use economic analysis as often as they should. 4. Faculty not doing a thorough review and critique prior to the final presentation. Many of the other items are also the result of relaxed emphasis or misalignment. Expectations for visual presentations could be better communicated.