Which would you choose?
Many on-line sites offer customers an option to review
a product or service to give other potential customers
an idea of what to expect.
Most use a star rating system as a simple way of
giving an overview together with an optional
comment.
5 stars usually represents the best and 1 star
represents the worst.
What sites and products have you seen which use
this system for feedback?
Which would you choose?
The feedback others leave can help when making a
decision about buying an item or booking a hotel or
planning a visit to an attraction.
On the next slide there are brief details about two
similar hotels. Which would you choose?
Which would you choose?
Hotel’s own website details
Surf and Sand
Sandy Shoes
Surf and Sand is a
renowned 3 rosette hotel
with a top reputation for
hospitality. Guests will find
everything they need in their
rooms, many of them have
stunning sea views.
Breakfast is served in the
sunny ‘Wave Watcher’ room.
User reviews
Sandy Shoes is a beach
front hotel offering a warm
welcome to guests.
Rooms are comfortable and
breakfasts are plentiful.
Based on 12 reviews
Based on 12 reviews
Official 3 rosette rating.
User reviews
Which would you choose?
It doesn’t seem as if there’s much to choose between
the two.
Have a closer look at the summary of the reviews on
the next slide and then decide.
The numbers indicate how many reviewers have
given each rating.
Which would you choose?
Customer Reviews
Surf and Sand





   
  
 

Sandy Shoes
5
    
4
3
   
  
 

6
1
2
1
2
0
0
Which would you choose?
Does the more detailed information help?
Both hotels seem to have a customer rating of 4.
Do you think that is correct?
What summary rating would you give each hotel?
How do you think the customer rating is calculated?
How the star rating is calculated
On many sites, the star rating shown is simply the
mean of all reviews posted; the summary rating is
often visually rounded to the nearest half star.
Using this method, should the hotels both have 4
stars?
Stars
Number of ratings
Stars
Number of ratings
5
5
5
4
4
3
4
6
3
1
3
2
2
2
2
0
1
1
1
0
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
5
5
4
3
3
1
2
2
1
1
Totals
Calculation
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
5
5
4
3
3
1
2
2
1
1
Totals 12
Calculation
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
Calculation
5
5
5 x 5 = 25
4
3
3
1
2
2
1
1
Totals 12
What does the
calculation
represent?
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
Calculation
5
5
5 x 5 = 25
4
3
4 x 3 = 12
3
1
3x1= 3
2
2
1
1
Totals 12
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
Calculation
5
5
5 x 5 = 25
4
3
4 x 3 = 12
3
1
3x1= 3
2
2
2x2= 4
1
1
1x1= 1
Totals 12
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
Calculation
5
5
5 x 5 = 25
4
3
4 x 3 = 12
3
1
3x1= 3
2
2
2x2= 4
1
1
1x1= 1
Totals 12
45
How the star rating is calculated
Surf and Sand
Stars
Number of
ratings
Calculation
5
5
5 x 5 = 25
4
3
4 x 3 = 12
3
1
3x1= 3
2
2
2x2= 4
1
1
1x1= 1
Totals 12
45÷ 12 = 3.75
45
How the star rating is calculated
Sandy Shoes
Stars
Number of
ratings
Calculation
5
4
?
4
7
?
3
1
?
2
0
?
1
0
?
Totals 12
?? ÷ ?? = ??
?
How the star rating is calculated
Sandy Shoes
Stars
Number of
ratings
Calculation
5
4
5 x 4 = 20
4
7
4 x 6 = 24
3
1
3x2=6
2
0
2x0=0
1
0
1x0=0
Totals 12
50 ÷ 12 = 4.17
50
How the star rating is calculated
The Lighthouse
A third hotel in the
area also has an average
customer rating of 4.
What might the breakdown
of ratings be?
Can you find 3
alternatives?
Stars
Number of
ratings
5
?
4
?
3
?
2
?
1
?
Totals 12
How much difference does
each review make?
Details of another two hotels are given on the
following slide.
Which one would you choose and why?
Which would you choose?
Hotel’s own website details
Mountain Vista
Lakeside House
Mountain Vista provides
guests with a relaxing
retreat set amongst the
lakes and forests in the
foothills of the mountains.
Experience our superb
cuisine or relax in our
luxurious spa facilities.
User reviews
Lakeside house provides
the comfort of your own
home in a five star setting.
The spa is available for all
guests to use throughout
your stay. Our team of world
class chefs will tantalise
your taste buds.
User reviews
Based on 120 reviews
Based on 6 reviews
How much difference does
each review make?
The breakdown of reviews is given on the next slide.
Do these present a different picture?
Would you change your decision or not?
Why?
Which would you choose?
Customer Reviews
Mountain Vista





   
  
 

Lakeside House
61
    
2
18
   
  
 

2
3
27
11
1
0
1
How much difference does
each review make?
If each hotel were to receive one more 5 review (as
shown below) what happens to their average ratings?
Mountain Vista
Lakeside House
Stars
Number of ratings
Stars
Number of ratings
5
62
5
3
4
18
4
2
3
3
3
1
2
27
2
0
1
11
1
1
How much difference does
each review make?
Sometimes hotels and business go through a ‘bad
patch’ when they might have issues with staff or the
building etc. Reviews at these times can be lower
than normal.
When the hotel returns to normal, it can be difficult to
counteract the bad reviews with newer good reviews.
How much difference does each
review make
How many more 5
reviews would it take for
the average customer
rating to reach the next
level of 4.5?
Mountain Vista





   
  
 

61
18
3
27
11
Is this the best way?
Many websites do use this approach to calculate
average ratings and then present the items in the
order of their average rating.
Consider this extreme example for two similar
televisions:
Stars Number
Stars Number of
of ratings
ratings
Which one has the
highest average rating?
5
99
5
4
1
4
3
3
Which would you be
more likely to buy?
2
2
1
1
1
What would your advice be?
When looking at ratings on review sites, what should
people be looking at and why?
What’s
important?
Teacher notes: Star Rating
This activity is designed to encourage students to look more closely at
common statistics they are presented with in everyday life.
Students are required to calculate the mean of data presented in a
frequency table and to consider the impact of a single value within large
and small data sets.
The realistic context makes the activity accessible to a wide range of
students.
» Students should have the opportunity to discuss this
with a partner or in a small group
» Students should calculate
Teacher notes: Star Rating
Many sites use customer feedback to present a rating for a product or
service.
These ratings are subject to many of the issues that arise when using
the mean in statistics, namely:
• The same mean value can be obtained through a set of data with
little variability and a set of data with wide variability.
• One value in a small data set has a bigger impact on the mean than
a similar value in a larger data set.
Teacher notes: Which would you choose?
In this part of the activity, two hotels with seemingly similar profiles and
feedback are presented.
The detailed breakdown of review grades draws attention to the fact
that the same rating can be obtained in different ways.
Slide 8: checking the rating. Students may already have met this
content or may be able to work out for themselves how to find the mean
of data in a frequency table. If not, slides 9 to 17 give an overview.
Teacher notes: How the star rating
is calculated
Slide 18: The Lighthouse.
To obtain a rating of 4, the value of the average rating must be:
3.75 ≤ rating <4.25.
This means that for 12 reviewers, the sum of their ratings should be:
45 ≤ sum < 51
There are many possibilities, including:
5
1
5
4
5
4
10
4
4
4
3
3
1
3
4
3
2
2
2
2
1
1
1
1
5
4
12
9
3
This one has
the maximum
number of 5s
and minimum
number of 1s
Teacher notes: How much difference
does one review make?
Slides 19 - 22
In the next part of the activity, two hotels with slightly different feedback are
presented, but the averages are obtained from 120 reviews and 6 reviews
respectively.
Discussion could be about the relative ‘reliability’ of the values and about
whether using a percentage rather than a number for each value in the
breakdown would be helpful or not.
Mountain Vista
Lakeside House
Number
of ratings
% of
reviews
5
2
33
15
4
2
33
3
2.5
3
1
17
2
27
22.5
2
0
1
11
9
1
1
Stars
Number
of ratings
% of
reviews
5
61
51
4
18
3
Stars
17
Teacher notes: How much difference
does one review make?
Slide 23: what difference does one 5 rating make?
Mountain Vista
Lakeside House
Star
Freq
Calculation
Star
Freq
5
62
5 x 62 = 310
5
3
5 x 3 = 15
4
18
4 x 18 = 72
4
2
4x2=8
3
3
3x3=9
3
1
3x1=3
2
27
2 x 27 = 54
2
0
2x0=0
1
11
1 x 11 = 11
1
1
1x1=1
Total
121
Total
7
456
Calculation
27
Average rating = 456÷121 = 3.77
Average rating = 27÷7 = 3.86
(still rounds to 4)
(now also rounds to 4)
This should help to emphasise that adding the same value to both has a
bigger impact in a smaller data set.
Teacher notes: How much difference does
one review make?
Slide 24 -25 How many more 5 reviews would it take for the average
customer rating to reach the next level of 4.5?
This part of the activity demonstrates how much positive feedback is
needed to balance out negative feedback.
This is much more challenging to consider as it requires students to set
up and solve an algebraic equation, but it does demonstrate that they
have fully understood work on averages from frequency tables.
If this content is beyond students, a simple trial and error approach to
find an approximate (or exact) number, by adding 10 or 20 extra 5
reviews each time, will work.
Whichever approach is taken, students are likely to find the number
required surprising.
Teacher notes: How much difference does
one review make?
Slide 24 How many more 5 reviews would it take for the average
customer rating to reach the next level of 4.5?
Average = sum of reviews ÷ number of reviews
Average needs to be ≥4.25
in order to round to 4.5
Rating
Freq
Calculation
5
61
5 x 61 = 305
4
18
4 x 18 = 72
4.25 = (451+5n) ÷ (120+n)
3
3
3x3=9
4.25 x (120+n) = (451+5n)
2
27
2 x 27 = 54
510+4.25n = 451+5n
1
11
1 x 11 = 11
59 = 0.75n
Totals 120
451
n = 78.666
79 more 5reviews are required (with no additional lower reviews).
Teacher notes: Is this the best way?
Slide 26
Two extreme cases are given such that a single 5 rating will be rated
more highly than anything with many top ratings and a single other
rating.
Examples of this can be found on many sites – though usually less
extreme cases. Visually a ‘half star’ rating is shown, but the reality is
that the ratings are ordered according to their mean values.
A real example is Televisions on Amazon - scrolling to the bottom of
the first page, a television with a single 5 rating is before one with 681
ratings which average at 4.58.
For interest, teachers might like to look at Evan Miller’s website for an
alternative approach.
Teacher notes: What would your advice
be?
Slide 27
To sum up the lesson, students can be asked to work in pairs or groups
to consider what advice they would give to people reading customer
feedback and reviews. Responses can be invited to be shared with the
class.
Their responses might include some of the following:
• Look at the summary statistic, but be also look at the breakdown of
values
• Take into account the number of reviews that contribute to the
average – a higher number of reviews is likely to be a more reliable
value
• Read the comments and look at the dates to see what reviews have
been like more recently.
Acknowledgements
• http://www.amazon.co.uk/s/ref=sr_nr_p_n_size_browsebin_3?rh=n%3A560798%2Cn%3A560858%2Cn%3A560864%2Ck
%3Atelevisions%2Cp_n_size_browsebin%3A161401031&sort=reviewrank&keywords=televisions&ie=UTF8&qid=1401724746&rnid=1613
98031
• http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
Accessed 2/6/14