PhD thesis quality: The views of examiners
Keynote address: Sid Bourke, University of Newcastle. Australia 1
Stellenbosch University, South Africa
Abstract
The PhD is clearly the pinnacle of formal education qualifications internationally, representing
excellence and attracting both resources and prestige to universities. The expectation is that
each PhD project will provide an original and significant research outcome, presented in the form
of a thesis. In Australia, normally the only assessment for the award of a PhD is through the
medium of written reports on the thesis by either two or three external examiners. The
examiners effectively set PhD standards. The major focus of our work has been on the content
of the examiner reports in providing indicators of thesis quality, and the bases for their
recommendations on the fate of the thesis. These reports were also linked with candidature and
examiner information to provide background information thought to be relevant to the thesis
examination process.
The study discussed here adds considerable empirical information to the literature in the
important and neglected area of doctoral assessment. On the basis of 2121 examiner reports for
804 theses across all discipline areas at eight Australian universities, discipline differences,
examiner consistency, and thesis quality are discussed. The incidence and proportion of
examiner comments devoted to (1) themselves and the context of their examination of the
thesis, (2) to assessable areas covered, (3) dialogic elements of the reports, and (4) to the
evaluative elements of their reports are described. Information about what examiners comment
on in their reports, and therefore presumably value, should be of considerable use to candidates
and their supervisors in assisting them to meet examiner expectations of theses more
effectively.
In general, the nature of examiner comments do not differ by discipline, although some
candidature characteristics do differ in this way, eg, nature of enrolment and time taken to
completion. Examiners of the same thesis are usually quite consistent in the nature of the
comments they make, although they more often diverge in their final recommendations. It would
seem that the same issues in a thesis are identified by examiners, although the extent of change
they recommend or demand may vary rather more. Because examiner recommendations are
intended for another specific purpose, there are problems in identifying thesis quality from the
recommendations, except in distinguishing between the best and the marginal theses. However,
more fine-grained distinctions between theses would seem to be possible from a detailed
examination of the written comments. Our work is proceeding along these lines.
1
This paper is based on results of an Australian Research Council Discovery Grant awarded to Allyson
Holbrook, Sid Bourke and Terence Lovat, and subsequent work undertaken by Sid Bourke.
INTRODUCTORY REMARKS
Despite differences in the nature of PhD programs and examination internationally, the
doctorate is clearly recognised as the pinnacle of our formal educational systems and
has been referred to as the ‘gold standard’ (Scott et al, 2004, p.149). And the quality of
the PhD remained largely unquestioned until recently when there has been a large
increase in the numbers undertaking doctorates in what has been termed the
‘massification’ of doctoral qualifications.
As there is no PhD curriculum as such, it is the thesis examiners who set the standard of
what is acceptable as a thesis (or dissertation), and consequently set the standard for
the award of a PhD. To a large extent the examiners also influence what style of
reporting may be used and what should be included. What examiners do in examining is
crucial. A few years back it could be said, with authority, that little research had been
undertaken into the written assessments of PhD theses (Morley, Leonard & David, 2002;
Mullins & Kiley, 2002; Tinkler & Jackson, 2000).
In the UK, the major focus in doctoral assessment has been on the oral examination or
viva (see, for example, the work by Trafford, 2003 and by Tinkler & Jackson, 2004). With
respect to assessment of the written thesis itself, there is a recent and developing
literature on examination criteria and process based on interviews with either
experienced examiners (Mullins & Kiley, 2002), inexperienced examiners (Kiley &
Mullins, 2004) or on questionnaires completed by examiners (Winter et al, 2000).
Interest in the USA concerning dissertation quality (Lovitts, 2007) and doctoral
assessment (Maki & Borkowski, 2006) is even more recent.
But there were a few prior studies of thesis assessment based on examiner reports
(Ballard, 1996; Hansford & Maxwell, 1993; Johnston, 1997; Nightingale, 1984; Pitkethly
& Prosser, 1995). In the main these were single-institution studies. Only Hansford &
Jackson (1993), which examined reports on Masters level theses, identified the literature
review and conceptual framework as substantial themes. In this study 34 per cent of
examiners made critical comment on the literature review. The two ‘most common
criticisms’ were the ‘failure to use recent literature and the inability to critically assess the
existing literature’ (p.179). The study drew on 125 students in Education in one
institution over a four year period. The authors noted that if their study had ‘endeavoured
to identify the nature of examiner comment’ relating to high quality theses, their ‘belief’
was that a number of such comments would refer to aspects of ‘theoretical or conceptual
frameworks’ (p.181).
Despite these few studies of various aspects of thesis examination, there remains a
dearth of direct information about what examiners value as indicators of thesis quality.
The most direct source of such information is the content of the written examiner report
itself – these reports can be analysed in detail to determine how important each
component is in the assessment regime. The study reported below involved direct and
detailed examination of examiners’ reports on PhD theses.
OUR STUDY
A total of 2121 examiner reports on a total of 804 PhD theses across all discipline areas
(Broad Fields of Study) at eight Australian universities (with varying research
intensiveness) were collected. The associated examiner recommendation on the thesis
and information about the candidates, their candidature and their supervision were also
collected. The content of the reports were coded across specific categories (described
below), and the report length was also recorded. I will return to the examiner reports
shortly, but first I will describe the samples of candidates and examiners and relate
these characteristics to the examiner recommendations made. The distributions of
candidates and examiners across disciplines are shown in Table 1.
INSERT TABLE 1 ABOUT HERE
There were initially 10 Broad Fields of Study (BFOS) recorded. However, as the
numbers of theses in some areas, notably Architecture, Law and Veterinary Science,
were small, theses in these areas were included with the fields with which they had the
most affinity, Built Environment, Business and Science respectively. It will be noted that
the broad field of Science had the largest number of theses included, followed by the
Arts, Humanities and Social Sciences grouping.
The majority of Australian universities use three external examiners for PhD theses,
others use two. In the main, reports are provided to the university independently by each
examiner, without prior consultation between examiners. This is a strength of the thesis
examination regime – the university has multiple and independent assessments
available to it when making a final decision on the fate of the thesis. Each examiner
recommendation on the thesis and the University decision were recorded for this study.
In most cases the decision is made by a university-wide committee, in other cases
where there is little difference between examiner recommendations, the decision may be
made by the Dean of Graduate Studies (or equivalent). Other information collected
included candidate individual characteristics and some information on the candidate’s
supervisors, the pattern of supervision and the examination panel.
Although there are some minor variations across Australian universities, essentially the
range of recommendations available to examiners and decisions of the university
committee can be consistently aggregated to the five categories shown below:
•
•
•
•
•
Accept the thesis as submitted
Accept the thesis following minor (or invited) corrections
Accept the thesis subject to more major (required) corrections
Require the thesis to be revised and resubmitted for further examination
Fail the thesis without the opportunity for revision
The distribution of examiner recommendations across these five categories is shown in
Figure 1 by BFOS. Overall, the proportions of recommendations are: Accept (31%),
Invite (minor) correction (40%), Require (more major) correction (22%), Revise and
resubmit (6%), and Fail (1%).
INSERT FIGURES 1 AND 2 ABOUT HERE
The distribution of university decisions by BFOS on the same five categories is shown in
Figure 2. The proportions are: Accept (13%), Invite (minor) correction (47%), Require
(more major) correction (38%), Revise and resubmit (3%), and Fail (one thesis only). In
considering the very high ‘pass’ rate for PhD theses it should be noted that the vast
majority of theses has already been scrutinized by at least one supervisor and declared
to be ready for examination before being actually submitted for external examination.
It will be noted that the distribution for the university decision differs in two ways from
that for the individual examiners. First, differences between BFOS are reduced and
secondly, there is a shift from the extremes to two middle categories Invite or require
(minor) change (from 40% by the examiners to 47% by the committee), and Require
(more major) change (from 22 to 38%).
THESIS QUALITY
It has been argued elsewhere (Bourke, Holbrook & Lovat, 2005) that examiner
recommendations on a thesis, although in the main serving a different purpose, provide
at least a general guide to thesis quality. The consistency of recommendations is
particularly marked for the best and the worst theses. Using examiner recommendation
as a measure of thesis quality, thesis quality was not related to any of the following:
•
•
•
•
•
•
•
•
BFOS. Despite the differences observed in Figure 1, the distributions of
recommendation by discipline area are not statistically significant.
Research intensiveness of the university attended was not related to thesis
quality. The universities were grouped into three levels of research intensiveness:
high (consisting of the traditional ‘sandstone universities’), medium (consisting of
newer universities with a noted research focus), and low (consisting of new
generation universities still in the process of developing a comprehensive
research record).
Gender. There were no differences in examiner recommendation related to any of
candidate, supervisor or examiner gender. It should be noted that almost half
(47%) of the candidates were female.
Whether the candidate was a native English speaker (78% of all candidates were
native English speakers).
Qualifications on entry to candidature. The most common qualifications for PhD
entry were having a bachelor degree with honours (46%), and a master level
qualification, either a coursework degree (27%) or a research degree (17%).
Nature of enrolment. Whether candidates had been enrolled full-time throughout
their candidature (53%), enrolled part-time throughout (15%) or had a mix of full
and part-time enrolment (32%) was not related to examiner recommendation on
the thesis.
Leave taken. Approximately 28% of candidates took leave during candidature.
Supervision. None of the following was related to examiner recommendation –
number of supervisors (which ranged from one (21%) to four (4%) with 49%
having two supervisors), supervisor experience, having had a change in
supervision arrangements during candidature (30% did have such a change).
By contrast the following were related to examiner recommendation on the thesis:
•
Candidate age. The relationship here was complex – on average, candidates
aged between 25 and 30 years on entry received the best recommendations and
candidates aged more than 45 years received the worst.
•
•
•
Whether a scholarship was held during candidature. Candidates who had held a
scholarship (59%) received more favourable recommendations.
A problem reported during candidature. On average, these candidates (34%)
received less favourable recommendations.
Examiner location. Almost half the examiners of these theses were located
outside Australia (47%), being scattered over the rest of the world with the next
largest groups being located in the USA (19%) and the UK (9%). The Australian
examiners (and the small proportion of New Zealand examiners, ie, 3%) made
less favourable recommendations on the theses than examiners in all other
countries and regions.
EXAMINER REPORT TEXT
The complete texts of the examiner reports on the PhD theses submitted by the 804
candidates at the eight universities were standardised to a given line length before being
coded, with lines as coding units, in a framework developed across five parent-level
categories using QSR N6 software. Multiple coding of a line (or text unit) was possible
and, in fact, it was common. For example, a positive judgement might be made (and
coded) about the literature coverage (also coded) in the thesis to give two individual
codes for each line of the judgement.
In all there were 36 sub-categories identified in the coding of report content, 30 of which
were suitable for possible inclusion in the text constructs to be developed. Categories
labelled ‘Other’ were considered too general and were excluded from any further
consideration in the present context.
The correlation coefficients were calculated for each sub-category between the
proportion of text and the examiner recommendation made. The data for each text code
and report length are shown in Table 2.
INSERT TABLE 2 ABOUT HERE
The first category, Examiner and process, is not directly related to the thesis
assessment but records occurrences where the examiners are writing about themselves,
their capacity to examine the thesis, and about what they consider relevant to the
process of examining. Although each of the sub-categories in this area constitutes a
small proportion of the overall report, two of the sub-categories are represented in at
least half the reports. All sub-categories are significantly related to the examiner
recommendation, mostly positively. However, where examiners write more about their
application of the anticipated criteria for a PhD thesis, they tend to have a less
favourable view of the thesis, in some cases expressing disappointment that the criteria
were not well met.
The second category, Assessable areas covered, clearly does relate to the thesis
assessments made by focussing on comment on the different areas – the scope,
significance of the thesis, the literature review, the approach taken and methods used,
and issues of communication. By far the most comment was made on the analyses and
reporting of findings which appeared in 95% of the reports and, on average, constituted
37% of reports. This sub-category was significantly negatively related to the examiner
recommendation, indicating that such comment tended to be a criticism of what was
done. The next most common area commented on was the approach taken in the
thesis. However, this sub-category was not significantly related to examiner
recommendation, because there was as much positive as negative comment on this
aspect. The third most common area was the significance and contribution of the thesis,
and this sub-category had the strongest relationship with examiner recommendation.
The relationship was positive indicating that there was much more positive than negative
comment for this sub-category. Two of the sub-categories concerning the literature
review (breadth/depth/recency and use/application) require further explication, and
these have been investigated in detail by Holbrook et al (2007).
The Dialogic elements category does not directly relate to assessments made of the
thesis, but is an indicator of tone in the examination report. It relates to the extent to
which the examiner becomes personally involved in the thesis and its examination. The
most common sub-category is use of the first person by the examiner, and this is
positively related to examiner recommendation. When a thesis is found wanting,
examiners are more likely to stand back from it and comment in the third person.
Similarly, although this is not as prevalent, examiners tend to engage intellectually with
what they see as better theses.
The Evaluative elements category identifies all those instances where examiners are
making some form of evaluative statement about the thesis, whether positive, negative
or indeterminate (called neutral here). One of the most prevalent comments is a
summative positive assessment of the thesis as a whole or a significant part of the
thesis. As would be expected, this sub-category has a very strong relationship with
examiner recommendation. The next most extensive comment in terms of the proportion
of text is formative instruction, coded when the examiner was attempting to teach the
candidate something in order to improve the thesis or the candidate’s subsequent
research or publication. As such, the use of formative instruction indicates that
something could or should have been done better and, as a consequence, this subcategory is negatively related to the examiner recommendation. The sub-category
prescription was found in most examiner reports and also accounted for more than 10%
of report text. Prescription was coded when an examiner did not attempt to teach the
candidate, but simply told the candidate what needed ‘fixing’ and how it should be
‘fixed’. In some cases prescriptive comment was related to minor shortcomings, but not
always. Overall, prescription was negative correlated with examiner recommendation.
The final row of Table 2 indicates that, on average, examiners wrote reports of 132 lines
(the range was from one line to 1272 lines) and longer reports were associated with
theses that received less favourable examiner recommendations. Clearly it takes more
text to point out and explain why something was in error than simply to state something
was well done. In any case, examiners would feel a need to justify negative comment
more fully than positive comment.
PRACTICAL MEASURES OF THESIS QUALITY
We now need to make a leap in this presentation from simple percentages of subcategories and correlations with examiner recommendations to further analyses
undertaken involving the intersection of assessment areas with evaluative elements, the
latter focussing on clearly positive and negative evaluations. As described by Bourke et
al (2005), there was a total of ten such cross-classifications of assessment areas and
evaluations that could be distinguished as being strongly related to examiner
recommendations, particularly for theses at the extremes. These cross-classifications
are referred to henceforth as indicators of thesis quality.
The ten indicators could be grouped into four main assessment areas – namely
contribution, literature, methodology and presentation, as shown in Figure 3. This figure
also includes examples of typical text for each indicator found in examiner comments on
PhD theses at two disparate levels – those of notably higher and lower quality.
INSERT FIGURE 3 ABOUT HERE
Thesis contribution was assessed by examiners with reference to originality, making a
substantive contribution and impact on either the discipline or practice or both. When
broken down into positive and negative comments, all of these indicators distinguished
between theses of higher and lower quality.
For the literature review, positive examiner comment on literature use and application
was the most closely linked with higher quality theses while comment on inaccuracy in
the review was most closely linked with lower quality theses. Literature coverage,
perhaps because it was simply expected, had weaker links with quality, and requires
further explication (see Holbrook et al, 2007).
With respect to methodology, positive examiner comments on both appropriateness
and application were particularly strongly related to theses being identified as high
quality. Finally, examiner comment on the substantive issue of communication and logic
in presentation was related to thesis quality, for both higher and lower quality theses.
The issue of grammar and expression basically was raised by examiners only when
there were problems in this area, and so it was strongly related to theses identified as
being of lower quality.
Our work to date suggests that the 10 indicators of thesis quality in the four areas
mentioned above, and illustrated in Figure 3, would be useful in any endeavour to
distinguish reliably between PhD theses of different quality. However, it should be noted
that these indicators were based on their effectiveness in making distinctions between
theses at the extremes of high and low (although passable) quality. The ability of these
indicators to make distinctions between theses across the spectrum of quality is yet to
be established.
IN CONCLUSION
This is where the story ends to date. Next we intend to trial the ten indicators described
by asking examiners to rate a thesis they recently examined on each of the ten
indicators as well as an overall assessment of thesis quality. We anticipate that the
indicator list may require further selection, refinement, addition and subtraction to enable
us to more closely differentiate between all theses, not only those at the extremes. Once
the final list of indicators was agreed, they could become a standard component of the
thesis examination process.
The centrality of examiners to PhD thesis assessment is unquestioned. Some of our
earlier work investigated the reasons given for the selection of 300 PhD thesis
examiners across all discipline areas. Not surprisingly, this study found that examiners
were chosen for multiple reasons, the most common criteria being in descending order,
expertise (77%), reputation (59%), publications (40%) and experience (36%) (see
Bourke, Scevak & Cantwell, 2001). Perusal of other university guidelines for examiner
selection suggests these criteria are in general use.
Finally, if such an assessment system were adopted for PhD thesis examination, it
would be a practicable one. That is, it would be possible to implement as simply a
refinement of the current examination system used in Australia, but we would have a
justifiable method of distinguishing more reliably between theses of different quality. As
the quality indicators that examiners were asked to respond to would have been
developed from what examiners wrote in their reports, the assessment would still be
based on examiner judgement. The examiners would remain the arbiters of PhD thesis
quality and, given the criteria and international basis on which examiners of Australian
PhD theses were selected, I maintain that is as it should be.
REFERENCES
Ballard (1996). Contexts of judgement: an analysis of some assumptions identified in
examiners’ reports on 62 successful PhD theses. Paper presented to the Conference on
Quality in Postgraduate Research, Adelaide.
Bourke, S., Scevak, J. & Cantwell, R. (2001). PhD examination and examiner
characteristics. Paper presented at the AARE Annual Conference, Fremantle,
December. Available at http://www.aare.edu.au/01pap/bou01588.htm
Bourke, S., Holbrook, A., & Lovat, T. (2005). Using examiner reports to identify quality in
PhD theses. Paper presented at the AARE Special National Focus Conference: Quality
in Educational Research, July 4 -5, 2005, Cairns, Published at:
http://www.aare.edu.au/05papc/bo05011y.pdf
Hansford, B.C. & Maxwell, T.W. (1993). A masters degree program: Structural
components and examiners’ comments. Higher Education Research and Development,
12, 2, 171-187.
Holbrook, A., Bourke, S., Fairbairn, H. & Lovat, T. (2007). Examiner comment on the
literature review in PhD theses. Studies in Higher Education, 32, 3. 337-356.
Holbrook, A., Bourke, S., Lovat, T., & Dally, K. (2004). Investigating PhD thesis
examination reports. International Journal of Educational Research, 41, 2, 98-120.
Johnston, S. (1997). Examining the examiners: an analysis of examiners' report on
doctoral theses. Studies in Higher Education, 22, 3, 333-347.
Kiley, M. & Mullins, G. (2004). Examining the examiners: How inexperienced examiners
approach the assessment of research theses. International Journal of Educational
Research, 41, 2, 121-135.
Morley, L., Leonard, D. & David, M. (2002). Variations in Vivas: quality and equality in
British PhD assessments. Studies in Higher Education, 27, 3, 263-272.
Mullins, G. & Kiley, M. (2002). 'It's a PhD, not a Nobel Prize': how experienced
examiners assess research theses. Studies in Higher Education, 27, 4, 369-386.
Nightingale, P. (1984). Examination of research theses. Higher Education Research and
Development, 3, 2, 137-150.
Pitkethly, A. & Prosser, M. (1995). Examiners’ comments on the international context of
PhD theses. In C. McNaught & K. Beattie (Eds), Research into Higher Education:
Dilemmas, Directions and Diversion. (pp. 129-136). Melbourne: HERDSA.
Scott, D., Brown, A., Lunt, I. & Thorne, L. (2004). Professional Doctorates: Integrating
Professional and Academic Knowledge, Berkshire England: Society for Research into
Higher Education and 0pen University Press.
Tinkler, P. & Jackson, C. (2000). Examining the doctorate: institutional policy and the
Ph.D. examination process in Britain. Studies in Higher Education, 25, 2, 67-180.
Tinkler, P. & Jackson, C. (2004). The Doctoral Examination Process: A Handbook for
Students, Examiners and Supervisors. Berkshire: Society for Research into Higher
Education and 0pen University Press.
Trafford, V. (2003), Questions in doctoral vivas: views from the Inside. Quality
Assurance in Education, 11, 2, 113-21.
Winter, R., Griffiths, M. & Green, K. (2000). The 'academic' qualities of practice: what
are the criteria for a practice-based Ph. D.? Studies in Higher Education, 25, 1, 25-37.
TABLE 1: The samples of candidates and examiners
By discipline across 8 universities
DISCIPLINE AREA
Agriculture
No. of
candidates
No. of
examiners
41
111
183
487
Business & Law
78
221
Education
84
241
Engineering & Built Environment
71
180
Health
131
320
Science, IT, Veterinary Science
216
561
OVERALL
804
2121
Arts, Humanities, Social Sciences
TABLE 2: The distributions of examiner report text codes and correlation with examiner
recommendation
EXAMINER REPORT TEXT CODE
INCLUDED
IN TEXT %
PROPORTION
OF TEXT %
CORREL. WITH
RECOMMEND.
Examiner and process
21: Personal & professional context
50
5
.174**
22: Application of anticipated criteria
61
5
-.053*
23: The model PhD
14
1
.120**
24: Mention of supervisor(s)
22
1
.052*
Assessable areas covered
311: Scope of thesis
73
8
.113**
312: Significance and contribution of thesis
82
10
.314**
313: Publications arising
44
3
.243**
314: Existing publications
13
1
.083**
321: Lit. review: Breadth/depth/recency
70
6
-.051*
322: Literature review: Inaccuracies
30
2
-.125**
323: Literature review: Use/application
38
3
NS
33: Approach: method/design/execution
81
13
NS
341: Analysis/reporting of findings
95
37
-.109**
342: Topic related issues
28
5
NS
351: Communication: Substantive issues
79
6
NS
352: Communication: Editorial issues
57
8
-.121**
Dialogic elements
41: Intellectual engagement
21
3
.064**
42: Conversation with the reader
30
3
NS
43: Use of first person
89
17
.119**
Evaluative elements
511: Summative assessment: Positive
87
19
.463**
512: Summative assessment: Neutral
47
4
-.085**
513: summative assessment: Negative
33
2
-.368**
52: Formative instruction
63
21
-.369**
531: Other instruction: Commentary
64
10
NS
532: Other instruction: Prescription
84
11
-.361**
541: Other judgment: Positive
72
10
.368**
542: Other judgment: Neutral
30
2
.094**
543: Other judgment: Negative
32
2
-.198**
.
132
-.382**
Total report length (lines)
* = correlation significant at the 0.05 level; ** = correlation significant at the 0.01 level; NS =
correlation not statistically significant
FIGURE 1. Examiner recommendation by Broad Field of
Study
BFOS Recoded
Agriculture
Arts, Hum, SS
Business & Law
Education
Engin & Archit
Health
Science & VetSc
50.0%
Percent
40.0%
30.0%
20.0%
10.0%
0.0%
Accept
Invite (minor)
Require
(major)
Revise &
resubmit
Examiner Recommendation
Fail
Figure 2: University decision by Broad Field of
Study
BFOS Recoded
60.0%
Agriculture
Arts, Hum, SS
Business & Law
Education
Engineering &
Architecture
Health
Science & Vet.
Science
50.0%
Percent
40.0%
30.0%
20.0%
10.0%
0.0%
Accept
Minor change Major change
(Invite)
(Require)
Revise &
resubmit
University Committee Decision
Fail
EXAMINER
COMMENT
HIGHER QUALITY
LOWER QUALITY
CONTRIBUTION
1. Originality
… highly original and thought-provoking.
It is hard to see anything new or original about this
thesis.
The reader is eventually left with the feeling of ‘so
what?’
Although sound methodologically, there is little of
substance in this thesis.
… conclusions, potentially misleading to
practitioners.
… not likely to have practical outcomes
2. Substance
… impressive scholarly contribution, rich in
insight …
… a major contribution to the discipline of …
3. Impact
Provides a helpful heuristic for advancement
of knowledge in the field.
Practitioners will welcome the guidance
LITERATURE
4. Coverage
The literature review is most comprehensive
and probably definitive.
5. Use &
application
There is clear evidence that the literature has
been fully consulted and understood.
… thoughtfully and carefully analysed the
literature, and integrated it where appropriate
6. Accuracy
… scholarship that is carefully and accurately
summarised
METHODOLOGY
7.
Methodologically the thesis is impressive,
Appropriateness
evidencing insightfulness, originality and
sensitivity.
The methodology is well-justified, sound and
clearly described.
8. Application
… high level of perceptiveness and acumen in
using the materials to draw out appropriate
and interesting insights and conclusions
… always an appropriate method, fully
justified …
PRESENTATION
9.
Cogent and straightforward … easy to read …
Communication
occasionally eloquent
& logic
All the necessary steps in the argument
supporting this essential point were made
crystal clear
10. Grammar &
Almost flawless expression and presentation
expression
Perhaps the most error-free thesis I have ever
read.
The now quite rich literature on the topic of [X] to
enhance biological control is represented by only
two articles.
… fails to recognise useful and reject non-useful
research …
It is rare to find a reference in this thesis that has
been critically appraised.
… has not engaged adequately with a body of
relevant knowledge
This is not what the author concluded – in fact the
opposite.
… an extremely misleading interpretation which
needs to be corrected
The method chosen is quite inappropriate as a
means of approaching the research question.
… there clearly are better ways of addressing the
issues …
… offers no critical analysis of the information
presented
… understanding of the methodology used seems
to be rudimentary, at best
It was not at all clear what the aim of the study was
…
There was no logical flow in several places leading
to poor communication of ideas.
… so careless and error-ridden as to almost defy
interpretation
I gave up on correcting expression after the first 20
pages.
FIGURE 3: Examples of typical examiner text by category for higher and lower quality
theses.