Introduction
This document provides the technical details about the calculations used for the sections of the
Quality Indicators for Learning and Teaching (QILT) website relating to data derived from the
University Experience Survey (UES). It is intended for an audience with some technical and data
background who wish to understand the statistical details of the calculations.
Data sources, variables and coverage
Data sources
University Experience Survey (UES)
The UES is Australia’s first national survey of undergraduate experience. Students in their first and
later years of their course are surveyed in the second semester and asked about their experience
at their university. The following UES data are used:

Quality of educational experience (item): the proportion of undergraduates who were satisfied
with the quality of their overall educational experience – it is calculated from two years of
pooled data;

Teaching quality (scale): the proportion of undergraduates who were satisfied with the quality of
teaching they had experienced as measured by the items in the teaching quality scale – it is
calculated from two years of pooled data;

Learner engagement (scale): the proportion of undergraduates who were satisfied with their
engagement with learning as measured by the items in the learner engagement scale – it is
calculated from two years of pooled data;

Learning resources (scale): the proportion of undergraduates who were satisfied with the
learning resources provided by their university as measured by the items in the learning
resources scale – it is calculated from two years of pooled data;

Student support (scale): the proportion of undergraduates who were satisfied with the support
they received at their university as measured by the items in the student support scale – it is
calculated from one year of data due to a change made in the scale; and

Skills development (scale): the proportion of undergraduates who were satisfied with the skills
development they experienced through their university studies as measured by the items in the
skills development scale – it is calculated from two years of pooled data.
Table 1 below shows the data sources and years that were used in the calculation of each
indicator.
Table 1: Data sources and years used for the UES based indicators
Indicators
Data source
University Experience Survey (UES)
Survey data file
Frame file
Quality of
educational
experience
Teaching
quality
Learner
engagement
Learning
resources
Student
Support
Skills
Development
2013,2014
2013,2014
2013,2014
2013,2014
2013,2014
2013,2014
2013,2014
2013,2014
2014
2014
2013,2014
2013,2014
The variables that were used to filter the data can be found in Table 2 below. The coverage for
each variable is applied before the calculation of the indicators and the SAS code used is provided
in brackets after each variable in the table. The full SAS code to create the indicators is available
from the Social Research Centre (SRC) on request.
QILT methodology – UES based indicators – June 2015
1
Table 2: Data coverage for the UES based indicators
Indicators
Variables (coverage)
Quality of
educational
experience
Teaching
quality
Learner
engagement
Learning
resources
Student
Support
Skills
Development
X
X
X
X
X
X
X
X
X
X
X
X
UES Survey data file:
Undergraduates only
(if course_type in
(8,9,10,13,20,21,22))
In scope including different study
areas for double degree students
(if myuniflag in (1,2))
Question regarding quality of
educational experience answered
(qlovledu in (1,2,3,4))
Valid Teaching Quality scale score
(if teaching_sat in (0,1))
Valid Learner Engagement scale score
(if engagement_sat in (0,1))
Valid Learning Resources scale score
(if resources_sat in (0,1))
Valid Student Support scale score
(if support_sat in (0,1))
Valid Skills Development scale score
(if development_sat in (0,1))
Total minimum sample size of 25
across the 2 years of pooled data or 1
year for Student Support
(if n ≥ 25)
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
UES Frame file:
Enrolled at time of frame checking
in 2014: (if enrolmentstatus = 0)
in 2013: (if enrolmentstatus = 1)
In scope for the UES – including those
randomly excluded from large strata
(if exclusions in (0,6))
Not contained in extra quota
(if finalframe ≠ 2)
Not middle year students
(if stage ≠ 2)
X
Indicates that the restriction is applied to the data before a particular indicator is calculated.
Data variability
Standard errors, and therefore the 90% confidence intervals, were calculated using the Finite
Population Correction (FPC) to account for the relatively large samples when compared with the
population. The FPC is generally used in cases where the sampling fraction, the proportion of the
population sampled, exceeds 5%. In addition, in order to calculate the standard errors for the
survey estimates, no non-response bias was assumed and, therefore, the use of simple random
sample survey errors was used.
The general formula used for the proportions was:
𝑝̂ (1 − 𝑝̂ )
𝑆𝐸(𝑝̂ ) = √
𝑛
𝑁 − 𝑛 𝑝̂ (1 − 𝑝̂ )
̂ = 𝑡∝
√
90% 𝐶𝐼 𝑏𝑜𝑢𝑛𝑑(𝑝)
× 𝐹𝑃𝐶 × 𝑆𝐸(𝑝̂ ) = 𝑡0.05(𝑛−1) × √
(𝑛−1)
𝑁−1
𝑛
2
Where:
p̂ is the estimated proportion
n is the number in the sample in the relevant strata over the relevant two years combined
QILT methodology – UES based indicators – June 2015
2
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
N is the estimated population in the relevant strata over the relevant two years combined
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
As the FPC was used to calculate the 90% confidence intervals and the standard errors, an
estimate of the population was required. The survey frame was used to determine the population,
see Table 2 for the restrictions applied.
When the estimated proportion is either 0% or 100% it does not allow for the calculation of standard
errors but, unless the entire population has been sampled, there would be some error in the
estimate. Therefore, for the calculation of standard errors to be used in the confidence intervals, the
estimate was perturbed slightly, 0% was changed to 0.04% and 100% was changed to 99.96%.
This was only done for the calculation of standard errors and not for the reporting of the estimate.
Calculation of indicators and confidence intervals
Quality of educational experience
The quality of educational experience indicator is defined as the proportion of undergraduates who
indicated that the quality of their entire educational experience was good or excellent. The indicator
can be expressed as ‘the proportion of undergraduates who were satisfied with the quality of their
educational experience’.
The quality of educational experience indicator is calculated as follows:
QEE pooled 
Number of undergradu ates satisfied with the quality of their educationa l experience 2013-2014
Number of undergradu ates with a valid response 2013-2014
Where:
Number of undergraduates satisfied with the quality of their educational experience2013-2014 is
the total number of undergraduates who responded with a 3 or 4 (good or excellent) to the
quality of entire educational experience item ‘Thinking of this year, overall at your university,
how would you rate the quality of your entire educational experience?’, over the two years
combined, after filters are applied. It should be noted that this item is reported on a four point
scale
Number of undergraduates with a valid response2013-2014 is the total number of undergraduates
who responded to the quality of entire educational experience item, over the two years
combined, after filters are applied
The 90% confidence interval for the quality of educational experience indicator is calculated as
follows:
90%CI QEE  QEE pooled  t 
2
 QEE pooled  t 0.05(n-1) 
(n -1)
 FPC  SEQEE
QEE pooled  1  QEE pooled 
N n

N 1
n
Where:
N is the population in the institution subject area, or at institution level as relevant, over the
two years combined
n is the number of undergraduates with a valid response in the institution subject area, or at
institution level as relevant, over the two years combined
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
QILT methodology – UES based indicators – June 2015
3
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
Teaching quality
The teaching quality indicator is defined as the proportion of undergraduates who indicated they
were satisfied with the quality of teaching they had experienced. The indicator can be expressed as
‘the proportion of undergraduates who were satisfied with the quality of teaching they had
experienced’.
The teaching quality indicator is calculated as follows:
TQ pooled 
Number of undergradu ates satisfied with teac hing quality 2013-2014
Number of undergradu ates with a valid response 2013-2014
Where:
Number of undergraduates satisfied with teaching quality2013-2014 is the total number of
undergraduates whose teaching quality scale score was at least 55 out of 100
(teaching_sat=1), over the two years combined, after filters are applied. Details of the
derivation of the scale score can be found in Appendix B of the 2014 University Experience
Survey National Report
Number of undergraduates with a valid response2013-2014 is the total number of
undergraduates who had a valid response (teaching_sat in (0,1)), i.e. responded to at least
eight of the eleven teaching quality items, over the two years combined, after filters are
applied
The 90% confidence interval for the teaching quality indicator is calculated as follows:
90%CI TQ  TQ pooled  t 
2
 TQ pooled  t 0.05(n-1) 
(n -1)
 FPC  SETQ
TQ pooled  1  TQ pooled 
N n

N 1
n
Where:
N is the population in the institution study area, or at institution level as relevant, over the two
years combined
n is the number of undergraduates with a valid response in the institution study area, or at
institution level as relevant, over the two years combined
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
Learner engagement
The learner engagement indicator is defined as the proportion of undergraduates who indicated
they were satisfied with their engagement with learning. The indicator can be expressed as ‘the
proportion of undergraduates who were satisfied with their engagement with learning’.
The learner engagement indicator is calculated as follows:
LE pooled 
Number of undergradu ates satisfied with learner engagement 2013-2014
Number of undergradu ates with a valid response 2013-2014
Where:
QILT methodology – UES based indicators – June 2015
4
Number of undergraduates satisfied with learner engagement2013-2014 is the total number of
undergraduates whose learner engagement scale score was at least 55 out of 100
(engagement_sat=1), over the two years combined, after filters are applied. Details of the
derivation of the scale score can be found in Appendix B of the 2014 University Experience
Survey National Report
Number of undergraduates with a valid response2013-2014 is the total number of
undergraduates who had a valid response (engagement_sat in (0,1)), i.e. responded to at least
five of the seven learner engagement items, over the two years combined, after filters are
applied
The 90% confidence interval for the learner engagement indicator is calculated as follows:
90%CI LE  LE pooled  t 
2
 LE pooled  t 0.05(n-1) 
(n -1)
 FPC  SE LE
LE pooled  1  LE pooled 
N n

N 1
n
Where:
N is the population in the institution study area, or at institution level as relevant, over the two
years combined
n is the number of undergraduates with a valid response in the institution study area, or at
institution level as relevant, over the two years combined
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
Learning resources
The learning resources indicator is defined as the proportion of undergraduates who indicated they
were satisfied with the learning resources provided by their university. The indicator can be
expressed as ‘the proportion of undergraduates who were satisfied with the learning resources
provided by their university’.
The learning resources indicator is calculated as follows:
LR pooled 
Number of undergradu ates satisfied with the learning resources 2013-2014
Number of undergradu ates with a valid response 2013-2014
Where:
Number of undergraduates satisfied with the learning resources2013-2014 is the total number of
undergraduates whose learning resources scale score was at least 55 out of 100
(resources_sat = 1), over the two years combined, after filters are applied. Details of the
derivation of the scale score can be found in Appendix B of the 2014 University Experience
Survey National Report
Number of undergraduates with a valid response2013-2014 is the total number of
undergraduates who had a valid response (resources_sat in (0,1)), i.e. responded to at least
five of the seven learning resources items, over the two years combined, after filters are
applied
The 90% confidence interval for the learning resources indicator is calculated as follows:
90%CI LR  LR pooled  t 
2
QILT methodology – UES based indicators – June 2015
(n -1)
 FPC  SE LR
5
 LR pooled  t 0.05(n-1) 
LR pooled  1  LR pooled 
N n

N 1
n
Where:
N is the population in the institution study area, or at institution level as relevant, over the two
years combined
n is the number of undergraduates with a valid response in the institution study area, or at
institution level as relevant, over the two years combined
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
Student support
The student support indicator is defined as the proportion of undergraduates who indicated they
were satisfied with the support they received at their university. The indicator can be expressed as
‘the proportion of undergraduates who were satisfied with the support they received at their
university’. As there was a change in the calculation of this indicator in 2014, only 1 year is used to
calculate the estimate.
The student support indicator is calculated as follows:
SS 
Number of undergradu ates satisfied with student support 2014
Number of undergradu ates with a valid response 2014
Where:
Number of undergraduates satisfied with student support2014 is the total number of
undergraduates whose student support scale score was at least 55 out of 100 (support_sat =
1), in 2014, after filters are applied. Details of the derivation of the scale score can be found in
Appendix B of the 2014 University Experience Survey National Report
Number of undergraduates with a valid response2014 is the total number of undergraduates
who had a valid response (support in (0,1)), i.e. responded to at least six of the thirteen
student support items, in 2014, after filters are applied
The 90% confidence interval for the student support indicator is calculated as follows:
90%CI SS  SS  t 
2
 SS  t 0.05(n-1) 
(n -1)
 FPC  SE SS
N n
SS  1  SS 

N 1
n
Where:
N is the population in the institution study area, or at institution level as relevant, in 2014
n is the number of undergraduates with a valid response in the institution study area, or at
institution level as relevant, in 2014
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
Skills development
QILT methodology – UES based indicators – June 2015
6
The skills development indicator is defined as the proportion of undergraduates who indicated they
were satisfied with the skills development they experienced through university studies. The
indicator can be expressed as ‘the proportion of undergraduates who were satisfied with the skills
developed through their university studies’.
The skills development indicator is calculated as follows:
SD pooled 
Number of undergradu ates satisfied with skills developmen t 2013-2014
Number of undergradu ates with a valid response 2013-2014
Where:
Number of undergraduates satisfied with skills development2013-2014 is the total number of
undergraduates whose skills development scale score was at least 55 out of 100
(development_sat = 1), over the two years combined, after filters are applied. Details of the
derivation of the scale score can be found in Appendix B of the 2014 University Experience
Survey National Report
Number of undergraduates with a valid response2013-2104 is the total number of
undergraduates who had a valid response (development_sat in (0,1)), i.e. responded to at
least six of the eight skills development items, over the two years combined, after filters are
applied
The 90% confidence interval for the skills development indicator is calculated as follows:
90%CI SD  SD pooled  t 
2
 SD pooled  t 0.05(n-1) 
(n -1)
 FPC  SE SD
SD pooled  1  SD pooled 
N n

N 1
n
Where:
N is the population in the institution study area, or at institution level as relevant, over the two
years combined
n is the number of undergraduates with a valid response in the institution study area, or at
institution level as relevant, over the two years combined
𝑡0.05(𝑛−1) is the 95th quantile from the Student’s t distribution with (n-1) degrees of freedom
FPC is the finite population correction which adjusts the estimated survey standard error
based on how large the sample was compared with the population
The data sources and restrictions for this indicator can be found in Tables 1 and 2.
QILT methodology – UES based indicators – June 2015
7