METHODOLOGY FOR THE CALCULATION OF THE MEDIAN IN THE SETU REPORTS Dr Clare Hourigan 11 March 2011 The SETU (Student Evaluation of Teaching and Units) is a university wide online survey of students designed to collect feedback to evaluate quality of teaching and units. It contains a series of five point Likert scale statements. Most of these use an agree/disagree scale where 1 = strongly disagree and 5 – strongly agree. The ‘overall’ rating for the teaching evaluation component uses an outstanding/very poor scale where 1 =very poor and outstanding = 5. There are a number of ways of reporting on Likert scale data but the key measure used for the SETU is the median. This measure is used in the teaching evaluation reports provided to teachers through the my.monash portal, in the unit evaluation static reports published on the Office of Planning and Quality website1, and in the Business Intelligence System2 reporting of unit evaluation data. The median result is also used in the reporting of the unit evaluation overall satisfaction question to identify units that are: • Needing critical attention (median ൑ 3), • Needing improvement (median 3.01‐3.59), • Meeting aspirations (median 3.6‐4.69), or • Outstanding (median ൒ 4.7). The median is effectively the middle value. It’s the score that divides the respondents 50:50 so that half of the responses are above the median and half are below. It is used in reporting because it is argued that it is less likely to be skewed by outliers than a mean (i.e. average). It is usually calculated by ranking the data in ascending order and selecting the value that is at the half way mark. In the following example, the median is 3. There are 11 responses in total. It is the value in the middle, with five values above it and five values below it. 1, 1, 2, 2, 2, 3, 3,4, 5, 5, 5 If there is an even number of values, the average of the two middle values is used as the median. However, there are two downfalls to using this median calculation for the SETU: 1. There will be little variation in results. All questions would receive a median of either 1, 2, 3, 4, 5 or 1.5, 2.5, 3.5 or 4.5 2. This calculation assumes that everyone giving a particular answer (eg a 4) has exactly the same level of satisfaction when in fact there is probably variation between them. 1
https://emuapps.monash.edu.au/unitevaluations/index.jsp 2
https://bis.monash.edu.au 1
Office of Planning and Quality – Statistics Unit
To account for these issues, the median is calculated in a way that assumes that the 5 point scale represents a continuous random variable rather than five discrete categories. So for example, all those who select a 4 (i.e. agree) are assumed to be evenly distributed between 3.5 and 4.49. Similarly all those who select a 5 (i.e. strongly agree) are assumed to be evenly distributed between 4.5 and 5.49. The median is then calculated from these transformed responses. The following example shows the steps to calculating the median. Category No. of respondents Cumulative no. of respondents 1 – Strongly Disagree 0 0 2 – Disagree 0 0 3 – Neutral 1 1 4 – Agree 22 23 5 – Strongly Agree 26 49 Total 49 Step 1 – The middle point is calculated by dividing the total number of responses by 2. So 49/2 = 24.5. Step 2 – The category in which this middle point falls is identified. In this case 24.5 falls in the ‘5‐Strongly Agree’ category as there are 23 respondents up to category 4 (see the Cumulative no. of respondents column). Step 3 – As there are 23 respondents up to category 4, the difference between the middle point (i.e. 24.5) and 23 is calculated. So 24.5‐23 = 1.5. This tells us how many ‘places’ away from the bottom of category 5 is the median. Step 4 – The 26 respondents who answered with a ‘5 – strongly agree’ are then assumed to be evenly distributed between 4.5 and 5.49. We need to identify where ‘1.5 places’ into that range sits. As there are 26 respondents the fraction 1.5/26 is calculated (0.06). This is then added to 4.5 (the bottom of the range) to give a median of 4.56. 2