ASSET 2016 sampling strategy
ASSET 2016 aims to capture a representative sample of STEMM academics working in eligible
institutions. Eligible institutions are defined as publicly funded higher education institutions (HEIs)
operating in the UK which employ at least 100 STEMM academics (N=116).
A central obstacle for any sampling strategy deployed by ASSET is the lack of individual contact
details in the HESA staff record, which makes the record unusable as a sampling frame (although it
can still be used to calculate non-response weights).
In the absence of a sampling frame, ASSET 2010 took a convenience sampling approach: the
survey was distributed to contacts in each eligible institution, who were asked to send it on to all
of their STEMM academics. This process produced variable institutional responses. Some
institutions engaged well with the survey and produced a high number of responses, while some
opted not to participate at that point. Among those institutions who did participate, the
academics to whom were sent the survey link varied in composition, for example including or
excluding post-docs from their individual samples.
Convenience sampling has well known drawbacks in terms of sample representativeness, since
non-respondents often have features in common, introducing sample skew. This can bias survey
responses in favour of particular groups, and lead to conclusions that would not reflect the
national picture.
ASSET 2016’s sampling strategy
ASSET 2016 will use a cluster sampling method to obtain a workable, representative sample in as
an efficient manner as possible. Cluster sampling exploits the fact that elements often appear in
‘naturally occurring’ groups known as ‘clusters’. The total number of clusters in which all elements
appear becomes the study population.
In ASSET 2016, the clusters are represented by the eligible HEIs. Therefore, the 116 eligible HEIs,
rather than the total number of UK STEMM academics, represent the study population.
Extracting a sample of institutions from this population will allow ECU to focus attention on a
manageable number of HEIs in order to monitor and boost response rate. Furthermore, stratifying
the population of eligible universities can counteract non-response bias, by ensuring that clusters
differ from each other as much as possible (see below). By restricting the number of institutions
we will be able to focus our attention efficiently in order to increase survey response rate, and
avoid survey fatigue in what is, after all, a frequently surveyed topic.
Once selected, named contacts will be approached and advised on the aims and procedures for
ASSET 2016. ECU will support these contacts to circulate the survey link to all eligible STEMM
academics in their institution and encourage them to participate.
Dealing with respondent bias
Since the process of data gathering within each cluster is essentially a process of convenience
sampling at that level, it is still possible that sample heterogeneity will be reduced to the extent
that the dataset becomes unrepresentative. That is, that since non-respondents tend to have
features in common, similar people may fail to return data across clusters that are in turn similar.
This can result in a sample being skewed towards a particular ‘type’ of cluster and, therefore,
person.
In order to overcome this, we have opted to select clusters non-randomly, an approach that will
ensure that there is variation between elements within the final sample.
In order to illustrate this point, as a purely hypothetical example, it may be the case that male
academics working within institutions in the South of England are unlikely to respond to ASSET. A
sample made up of a majority of HEIs located in the South England region (a potential
consequence of random sampling) would therefore be skewed towards female respondents.
Ensuring that the sample include HEIs from other regions, where response bias may be different,
would counterbalance this effect.
We cannot, of course, predict non-response with this level of accuracy, but non-random cluster
selection adds a level of protection against sample skew. The population of N=116 eligible HEIs will
therefore be stratified prior to sampling.
Stratification variables
HEIs will be stratified by the 12 geographic regions and mission group affiliation (or nonaffiliation)1. These variables have been selected on the basis that they may impact on staff’s
experience of and attitudes towards gender equality. Of note, both annual income and the
number of STEMM academics employed were also considered as potential strata; however,
because they were found to co-vary with mission group2, it was decided that their inclusion was
not necessary.
There are, of course, a variety of other variables on which UK HEIs could be stratified. ASSET 2016
may be useful in informing future approaches to this issue.
1
Since comparatively few Universities belong to the Cathedrals or GuildHE mission groups, these two groups have
been combined with the unaffiliated institutions to produce an ‘Other’ group, alongside Russell, 1994, Million + and
University Alliance groups. The 1994 group is no longer extant, but still remains a useful grouping category for this
exercise.
2
Chi square tests, p<0.05