The first three years of UKLO: A review of
the UK Linguistics Olympiad1
Laura Tomlinson
Graeme Trousdale
1
The research reported in this work was funded by the British Academy as part of its Language and
Quantitative Skills (L&QS) Programme. The authors would like to acknowledge the guidance and comments
from the project’s steering committee: Richard Hudson, Louis Blois, Billy Clark, Jeanine Treffers-Daller, and Neil
Sheldon.
1
Contents
Introduction
Section 1: A description of the competition’s current organisation
1.1 Competition structure
1.2 Question selection
1.3 Administrative procedure
1.4 Recruitment
1.5 Liaison
Section 2: A summary of findings from research into Linguistics Olympiads
from around the world
Section 3: Studies investigating male and female students’ attitudes to subjects
and academic competition
3.1 Female underrepresentation in Science and Mathematics
3.2 Male and female attitudes to language learning
3.3 Gender and UKLO
Section 4: Information gathered from students, teachers and UKLO records
4.1 A description of the data gathered
4.2 Results and discussion of the data
4.2.1 How effective were questions?
4.2.2 Benefits for school subjects
4.2.3 How effective was the structure of the competition?
4.2.4 How effective was administration?
4.2.5 School support
4.2.6 School recruitment
4.2.7 Liaison
Summary of recommendations
Works Cited
3
4
4
5
5
7
7
9
14
14
15
17
19
19
20
20
26
27
29
31
33
34
35
36
2
Introduction
This report analyses data gathered from research into the structure and organisation of the UK
Linguistics Olympiad (UKLO), and aims to provide a full description of current procedures in place, as
well as make suggestions for possible future developments for the competition. A relatively recently
established Olympiad in the UK, UKLO has been running as a national competition since 2010, and
each year of the competition has seen an increase in participating schools, as well a broader age
range of students taking part. This report takes the opportunity to analyse the competition’s current
organisation, to investigate student performance in the Olympiad to date, and to make
recommendations for future developments.
The report is divided into four sections:
- Section 1 provides a description of UKLO’s current organisation, giving an overview of the
competition’s structure, as well as information on administrative procedures, and a
description of UKLO’s current links to supporters, universities and other organisations.
- Section 2 describes some of the features of comparable competitions, both in the UK and
internationally. This section investigates the organisation of other Linguistic Olympiads
worldwide, as well as looking at well-established Olympiads in the UK. The data gathered in
this section is discussed in relation to UKLO, to investigate plans for future development of
the competition, and to make recommendations for areas of improvement.
- One of the key issues raised in literature regarding language learning and academic
Olympiad performance is gender inequality: female students are underrepresented in
Mathematics and Science Olympiads worldwide, while in the UK fewer male students
continue to study languages to higher levels. Section 3 presents some of the recent
literature on these topics, and investigates male and female students’ participation and
performance in UKLO.
- As part of the research undertaken for this report, quantitative data was gathered to assess
student participation and performance over the three years of the competition, and
feedback was gathered from students and teachers involved with UKLO 2012. Further
details of the methods used are presented in section 4.1. Section 4.2 presents the results of
this research, and provides a series of recommendations based on its findings.
3
Section 1: A description of the competition’s current organisation
A large part of the data for this section was gathered from members of the UKLO committee. A
questionnaire was compiled, designed to elicit information from committee members about the
competition’s current organisation, including the current administrative procedures involved in
UKLO, what relationships exist between UKLO and other organisations, and whether there have
been any recent changes to the structure or organisation of the competition. After evaluation of a
draft questionnaire by a sample of the UKLO committee, an adjusted questionnaire was emailed to
the entire committee at the time. Ten responses were returned: three committee members
commented that they were relatively new to the committee, and felt unable to comment in any
detail. The seven remaining committee responses (representing one third of the group) are used
alongside data found on the UKLO website to describe as fully as possible the current procedures
involved with UKLO. The responses also allowed for investigation into planned changes and
improvements to the competition.
1.1: Competition structure
The current structure of the competition is an in-school first round, followed by a national second
round, where the International Olympiad team is selected.
Round One
- An in-school paper, usually made up of
five questions
- Three levels available: Foundation,
Intermediate and Advanced
- Usually takes place in early February
Round Two
- Residential round, held at a UK
university
- Competitors are highest scoring entrants
from the Advanced Round One test
- Individual paper, usually consisting of
five questions
- Top performers are selected for the IOL
team(s)
- Usually takes place in late March
Foundation/Intermediate
- Can be taken at any point during a
specified week
- Marked in school
- Students may compete in teams
- Two-hour paper, which may be divided
into several shorter test sessions
International Linguistics Olympiad (IOL)
- Held at a different international location
each year
- Organised by the IOL committee and
committees from the host country
- IOL 2013 due to be held in the UK
- Individual paper, consisting of five
questions
- Separate team question
- Medals and commendations awarded to
top performers
Advanced
- Schools must notify UKLO in advance
how many candidates are expected and
when the test will be taken
- Marked externally
- Students must participate individually
- Run as a single, two and a half hour
exam
- Certificates awarded for participation
- Bronze, Silver and Gold awards for top
performers
- Highest scorers nationally are invited to
Round Two
4
Most questions used in the UKLO papers are sourced from ELCLO (the English Language
Computational Linguistics Olympiad). In 2012, nine questions were selected for use in the Round
One paper, with questions 1-5 appearing in the Foundation paper, questions 3-7 in the Intermediate,
and questions 5-9 in the Advanced paper. The questions and test material that come from ELCLO
are usually available from early February, and questions are emailed to schools prior to the week
allocated for the Round One test.
As far as timing for each round is concerned, enough time needs to be left after Round One to allow
for marking, announcement of results, and for competitors to organise travel to the Round Two
location. The residential nature of the second round, and the involvement of the host institution
means that university term-times need to be considered. As many students will be preparing for and
sitting exams throughout the spring term, this also needs to be considered when deciding when the
two rounds should be held. Generally, Round One takes place in early February, and Round Two in
late March.
The 2012 competition saw the introduction of the Intermediate paper. This was introduced to allow
the Foundation paper to involve some lower level questions, enabling younger students to take part.
This new structure links well with the National Curriculum Key Stages, with guidance for teachers
suggesting that the Foundation paper is ideal for Key Stage 3 (years 7-9), Intermediate for Key Stage
4 (years 10 and 11) and Advanced for Key Stage 5 (years 12 and 13). There are no restrictions to
who can take part in which stage, with the only consideration being what students feel able to do.
UKLO is keen to attract younger participants, and recognises the benefits of starting at a young age:
some of the younger participants from 2012 will be able to compete multiple times before they
leave school. With Foundation and Intermediate scripts being marked in school, an increase in
entries for these levels would mean no extra considerations for UKLO in terms of sourcing markers
for the first round papers.
1.2: Question selection
Although UKLO writes some of its own questions, these have mainly been used in the Foundation
papers, and most questions are written by members of ELCLO, based in the USA. Other members of
ELCLO (including the North American and Australian Olympiads) have used questions from UKLO in
their own competitions. The UKLO group aims to write easier questions, suitable for the lower level
papers, and using languages that competitors might be familiar with: for instance, Welsh appeared
in the 2012 papers. For students who are speakers of these languages, alternative questions are
available. There is no specific guidance provided by ELCLO to question-writers, but composers in the
ELCLO team aim at a level similar to that found in the UKLO Advanced papers. Questions for the
UKLO papers have so far been moderated mainly by the committee chair, Richard Hudson. From
2013, other committee members will be involved in question moderation.
1.3: Administrative Procedure
Registration
Teachers only need register once for the competition, by supplying their name, email address and
other details about themselves and their school. Registration is available online (via the UKLO
website: www.uklo.org) through a questionnaire, the responses to which are posted into a database,
where schools are primarily identified by postcode. Separate to the one-off registration is the
notification of Advanced entries. The deadline for this is one week before Round One, giving the
committee time to source markers. There is no pre-test registration for Foundation or Intermediate
entries, but schools are asked to send their marks in to UKLO after the Round One competition.
5
Publicity
UKLO has developed a number of contacts with the following organisations:
- The Association for Language Learning (ALL) (www.all-languages.org.uk)
- The British Academy (www.britac.ac.uk)
- The British Association for Applied Linguistics (BAAL) (www.baal.org.uk)
- The English Subject Centre (www.english.heacademy.ac.uk)
- The Independent Schools’ Modern Languages Association (ISMLA) (www.ismla.co.uk)
- The Linguistics Association of Great Britain (LAGB) (www.lagb.org.uk)
- Links into Languages (www.linksintolanguages.ac.uk)
- National Association for the Teaching of English (NATE) (www.nate.org.uk)
- The National Centre for Languages (CILT) (www.cilt.org.uk)
- The Philological Society (www.philsoc.org.uk)
- Routes into Languages (www.routesintolanguages.ac.uk)
These organisations advertise UKLO online and to mailing lists, and teachers on committee forward
advertisements on to local networks. Committee members also presented at the Language World
Conference in 2012, discussing UKLO as one of a range of language activities designed to promote
language learning in schools (Hawkes et al. 2012). In summer 2012, the competition also expanded
its online presence, with pages on social media websites used to further develop online networks
and publicise the competition.
School Support and Training Materials
UKLO provides online material for schools to use in training, and for teachers to use to develop their
own skills in training students. These include practice papers, answer sheets and training
presentations. As linguistics is not currently offered as a curriculum subject, training in schools, for
both students and teachers, is low. UKLO has offered various types of support to teachers, including
demonstrations from committee members, but little has been requested from schools up to now.
Since the 2012 competition, a group of students from the University of Edinburgh have been
working to develop a series of training materials, designed to help school students prepare for the
UKLO papers. These materials have also been designed with a view to promoting UKLO in schools,
by providing students with a sample of the competition problems. Three of these worksheets are
included in the attached body of training material, along with existing powerpoint training materials
currently available via the UKLO website. These materials are designed with students in mind, to
take them through the process of working on a linguistics puzzle. The worksheets were designed by
a group of linguistics undergraduates at the University of Edinburgh, in collaboration with members
of the academic staff. With further funding, it would be possible to produce more material, targeted
at participants at different levels in the UKLO competition (i.e. those working at the Foundation,
Intermediate and Advanced levels).
Marking
Foundation and Intermediate papers are marked by teachers in-school, using a mark scheme
provided by UKLO. In 2012, UKLO had around 1000 Advanced scripts to mark, and marking was
done using a combination of individual markers and markathons (see below). In the autumn term,
markers are recruited using a variety of online lists, and markathon markers are recruited. Marking
supervisors work alongside the school support team to match markers with schools. They also
ensure that markers and schools are put into contact with each other, that markers receive the
scripts they have been allocated, that marking is done on time, and that any necessary re-allocation
is done. Once the marking is completed, supervisors collect Advanced marks from markers and
Foundation/ Intermediate marks from schools to produce statistics for Round One.
6
The 2012 competition was the first to use volunteer markathons to help with script marking: earlier
competitions were marked by the committee or by volunteer individual markers. Markathons were
organised by a number of staff members from UK universities, who enrolled undergraduate student
volunteers to spend an afternoon marking papers. Another change to the marking system involved
schools sending scripts directly to their allotted marker, which worked to speed up the marking
operation. With an expected increase of entries for 2013, UKLO is looking to make more changes to
ensure the marking system copes.
Round Two Organisation
The three kinds of organisation required for Round Two are logistic, academic, and competitive.
Three Round Two tutors manage the academic organisation of the round, including training. The
competitive organisation (providing the test papers etc.) is managed by the problem-setting team
and one of the markers. The logistic organisation includes finding a host university, booking rooms,
and making travel and meal arrangements. This is undertaken by both committee members and
local organisers. In terms of finding a host university, UKLO advertise online and via mailing lists
such as the Linguistics Association of Great Britain (LAGB). Once a host is decided, a date is
confirmed through coordination between the host and the Round Two tutors. Other considerations
at this stage are making sure question papers reach the host site, and organisation of Round Two
marking.
Coordination with IOL
Organisation of the annual IOL trip is often problematic, due to issues with finding a local venue and
funding for the team. Regular IOL chaperone and UKLO vice-chair Neil Sheldon, and UKLO chair
Richard Hudson, currently represent UKLO on the IOL board. This board decides where the IOL
should be held, and what the competition rules should be. The board works closely with the
Problem Committee, who organise the IOL questions, marking and rewards. UKLO communicates
with the IOL board via email, and also makes use of an IOL Google Group and website.
1.4: School Recruitment
The main focus in terms of recruitment is through developing links with Foreign Language and
English Language professional associations and mailing lists, some of which are detailed in section
1.3.2, above. This approach, as well as advertisements online, has proved successful, and there has
been an annual increase since the beginning of the competition, both in the number of schools
registering and in the number of schools taking part. UKLO has been very successful in recruiting
independent schools, and has been moderately successful in recruiting maintained schools. There
are many more competitors from independent schools, which might be explained by a general
orientation in independent schools towards competitions such as UKLO and other academic
Olympiads, and might be connected to stronger Modern and Classical Language departments. The
problem is not that maintained schools don’t register: more maintained schools than independent
are currently registered for UKLO. Maintained schools enter far fewer students into the final
competition than independent schools. A current focus of the UKLO committee is to encourage
maintained schools to enter more participants.
1.5: Liaison
Supporters
UKLO has a good relationship with its supporters, and most supporters have been enthusiastic to get
involved. The main involvement in supporting UKLO is financial, although two supporters have
provided books (used as Round Two prizes), and the Round Two host department is also considered
a supporter. At the end of each year’s competition, UKLO provides a brief report to its supporters,
and information on organisations supporting UKLO has been incorporated into the competition
website. UKLO is not currently looking to expand its list of supporters.
7
Universities
The Round Two hosts to date have been the Universities of Sheffield, Edinburgh and York, with St
Mary’s University College set to host in 2013, and Liverpool Hope University in 2014. The list of
universities involved in the 2012 markathon is available on the UKLO website, with involvement
from eleven institutions. The relationships held between UKLO and these universities range from
very strong to much less so, with some institutions developing links with UKLO over the three years
of the competition, and others becoming involved more recently. Although strengthening its
connections to UK universities is not an immediate priority for UKLO, these relationships are
valuable. As well as the contribution universities make to the running of the competition (in terms
of financial support, hosting Round Two, marking, and possibly training for IOL), an important
consideration is showing universities how UKLO can be effective in attracting students to linguistics
and language study at higher levels.
Government, exam boards and the media
There is no relationship at the moment between UKLO and the government, and there is no
evidence that any government department is aware of UKLO. Due to financial cutbacks, it seems
unlikely that the Department for Education would be able to assist with funding. UKLO is interested
in developing relationships with the Department, however, especially with a view to recognising
linguistics as part of the curriculum.
There has been limited contact so far between UKLO and exam boards. The Welsh Joint Education
Committee (WJEC) has been approached about the possibility of an A-Level in linguistics, but this
was before the setup of UKLO. The issue of an A-Level in linguistics has now been raised again with
another examination board, since there is potentially a much stronger case, now that the
competition has proved so popular. Some of the key points raised in the development and
proposals for the linguistics A-Level include:
-
-
Strong links to other subjects (particularly English, foreign languages, mathematics and
psychology)
‘Knowledge About Language2’ (KAL) and its place in the curriculum for language learning and
English: linguistics can help students develop KAL, helping them better understand language
data, both in English and in learning foreign languages.
Development of analytical skills through analysis of language data (including data
presentation and use of analytical frameworks)
Development of key transferable skills of communication and critical thinking
Appeal to both male and female students (discussed in more detail in relation to UKLO in
section three, below)
Similar points can be raised in discussion of UKLO and its benefits for learning: if UKLO is able to
demonstrate a positive effect on students’ attitudes and performance in other subjects, through
development of KAL, analytical skills, or communication skills, this may add weight to the proposal to
introduce linguistics as a stand-alone A-Level subject, and to incorporate linguistics into English and
foreign language classrooms.
There have been articles in the local press about Round One winners of UKLO, as well as articles in
professional journals for teachers and linguists. Ideally, UKLO is looking to raise its profile in the
national educational press, which would recognise UKLO as a new element in the education system.
As discussed in section two, below, raising the profile of UKLO will prove valuable in terms of
2
‘Knowledge About Language’ is one of the key objectives of the curriculum for language teaching, which aims
to develop students’ understanding of language structure.
8
ensuring continued support for future years, and will also support proposals for future
developments in incorporating linguistics into pre-university study.
9
Section 2: A summary of findings from research into Linguistic Olympiads from around the world.
A list of national Linguistic Olympiads was compiled using the list of participating countries on the
IOL website. From this list, information from nine national Linguistic Olympiads was gathered from
websites, news articles, blog entries and email correspondence concerning the IOL and national
Olympiads. A similar list of subject Olympiads run in the UK was compiled, using data from online
searches and from international Olympiad websites. In total, four subject Olympiads were
investigated, with information on their structure and organisation gathered from individual
websites. Some of the key points raised in the data gathered are presented below, and discussed in
relation to UKLO.
Involvement from previous Olympians
The Russian Olympiad website describes how previous olympians (especially those at university
level) are enrolled to train new students in the competition, although no information is given to
clarify the stage at which this is offered. A similar scheme is in place as part of the training for the
British Mathematical Olympiad, where previous Olympians are enrolled to mentor candidates for
the International Mathematical Olympiad squad, and to provide mentoring for younger students
who are interested in taking part in the Olympiad. In 2012, UKLO enrolled university students to
mentor the IOL team, and is looking to expand this scheme in future years. Although the
competition is only in its early years, there may be potential to recruit former IOL students
(especially those continuing on to related degree courses) in the training of the UK IOL team.
Committee structure
The Brazilian Olympiad website describes how the committee for the competition is divided into two
teams: the Organising Committee, responsible for managing test material, logistics and coordination
with the IOL, and the Academic Committee, responsible for question composition and providing
support material for teachers. A team structure was proposed in summer 2012 for UKLO, and the
current committee has been divided into a number of sub-groups, each responsible for different
aspects of organising the competition. The intention was to focus committee activities onto a small
team of members, making it easier to know who to contact regarding different issues involved in
competition organisation, and ensuring that no one committee member is left with sole
responsibility for more than one operation.
Liaison with language education in schools
There is some indication online of links between Linguistics Olympiads and language learning in
schools. The Brazilian Olympiad website, for example, offers suggestions to help teachers
incorporate more Linguistics related material into language lessons. In an article about the Estonian
Olympiad, Pajasallu (2005: 12) states that linguistics is not taught in Estonian schools, but that the
Olympiad is seen as an ideal opportunity to engage students with ideas about language study. The
competition is one of a number of linguistics teaching events held in the country: courses in the
‘Fundamentals of Language’ are run online, and Linguistics workshops are involved as part of the
training process for the Olympiad itself (Pajasallu 2005: 12). Similarly, an article on German
linguistics blog Der Linguistische SprAACHENblog (Keus and Corr 2012) suggests that, although
linguistics is not offered as a subject in German schools, there is a growing movement to incorporate
more linguistics into German classrooms, with courses offered to prospective German teachers, to
help incorporate linguistics and linguistic discussion into the German language classroom.
Given the interest that other Olympiads have shown in creating stronger links to linguistics in
schools, it could prove useful for UKLO to develop links with some of these initiatives.
Communication with similar projects would allow for international support in developing plans to
incorporate linguistics into schools: what works, and what has proved difficult? Since UKLO aims to
10
promote linguistics in the classroom, liaison and communication with other groups aiming to do the
same could prove extremely valuable.
Links between nations
The British Mathematical Olympiad website describes links between the British and Hungarian
teams, with annual training sessions held where the two nations’ teams prepare for the
international competition together. As UKLO has established links with AILO (the All Ireland
Linguistics Olympiad), through involvement with ELCLO, and from entering alongside AILO in IOL
2009, it would be worth considering developing training with AILO as part of the two teams’
preparations for the IOL. The issues to bear in mind with this are: the cost of arranging a meet-up,
the logistics of organising a training session, and the time students would be able to commit to
training/competing on top of commitments to the IOL. As well as developing links with other
countries running the Olympiad, such a scheme would allow students to train and compete
alongside other national teams, helping to improve students’ performance and confidence.
Other non-Olympiad events
A number of the Olympiad websites for competitions in other countries mention similar activities or
competitions linked to the Linguistics Olympiad. Many countries offer a language competition
designed to test students’ knowledge of particular languages. In Estonia, a competition of this sort
has been run since the 1960s (Pajasallu 2005), while a similar competition in Russian (the ‘Russian
Bear – Linguistics for All’ project) is proving popular with students learning the language, both as
native speakers and as second language learners.
Linguistics competitions are also available for younger students. In Poland, the ‘Tower of Babel’
competition is designed to help younger students prepare for future participation in the Olympiad.
The Russian University Linguistics Tournament is described as being one of the most popular extracurricular activities among students, despite linguistics not being offered as a school subject. The
Tournament is aimed at newcomers to linguistics competitions: participating students are informed
about the Olympiad, which is organised about six weeks later, and are invited to participate.
The existence of these competitions aimed at younger students or at those who have yet to
participate in the Olympiad is similar to the Challenges offered in the UK for Mathematics, Physics
and Biology:
-
-
-
The Mathematical Challenges on offer in the UK are aimed at students at different ages, and
are used as ‘first steps’ into participation in Mathematical Olympiads. As well as the British
Mathematical Olympiad itself, there are also Junior and Intermediate Olympiads,
participants for which are selected on the basis of performance in Mathematical Challenges.
Those who perform well in the Olympiads are invited to take part in the first round of the
British Mathematical Olympiad.
The British Physics Olympiad is aimed at students in year 13. Physics Challenges are on offer
to students from year 11 and above, and a practical challenge (the Experimental Project) is
available for students in year 10 and above. The Challenges are in-school papers marked by
teachers, while the Experimental Project is a group project, with the opportunity to tackle a
practical physics problem.
The British Biology Olympiad is aimed at students aged 16 and over, with the Biology
Challenge available for students in years 9 and 10. The Challenge is an in-school paper, and
is not subject to the same restrictions as the formal Olympiad paper.
In many ways, this is comparable to the distinction between Advanced and Foundation/Intermediate
papers in UKLO: the Advanced paper is aimed at older students, is marked centrally, and is used in
11
selecting potential competitors for the IOL. In contrast, the Foundation/Intermediate papers are
aimed at younger students, are marked in-school, and are not used in the IOL selection process. Top
performers in these papers may, however, be encouraged to take a higher level paper in future
competitions.
It might be worth pursuing the idea of introducing ‘Challenge’ equivalents, or presenting the
Foundation/Intermediate papers as ideal for younger students or ‘first-timers’, as seen in other
Olympiads. Some of the evidence from feedback surveys suggests this would prove popular:
-
-
-
A lot of teachers felt that students were daunted by the difficulty of the questions, and were
put off from continuing with the paper. A less formal ‘introduction’ challenge might
encourage students to continue onto the Olympiad papers themselves.
Non-participating schools feedback suggests that students used the material and enjoyed
the activities, but preferred not to compete, or take the test as a formal ‘exam’, with some
schools reporting that they would continue to use the material in this way. By either
introducing a less competitive round, or by promoting the Foundation/Intermediate levels as
less formal, schools might be encouraged to enter officially. This would be beneficial for
UKLO’s records, as a way of recording where and how material is being used. It might also
encourage high performing students to consider entering a higher level in future years.
The 2012 competition saw younger students taking part than in previous years, and
feedback from teachers suggests that younger students are enthusiastic to get involved in
the competition next year. By promoting the Foundation/Intermediate rounds as suitable
for first-time competitors, UKLO might encourage younger students to participate. The
‘forward-looking’ approach of viewing the Foundation/Intermediate papers as steps towards
participation in the Advanced paper could encourage students to continue participation year
on year, allowing them to benefit from training and participation year after year, and to
further develop their skills.
Funding issues
Online material discussing Linguistics Olympiads describes how certain countries have encountered
difficulties in securing funding for the competition. Germany has taken part in the IOL since 2008,
but was unable to participate in 2011 due to a lack of funding available for competitors. An article
appearing on Der Linguistische SprAACHENblog on Germany’s participation in the competition
(Meinunger 2012) suggests that few organisations have been willing to support the German team.
The main concern here is to raise the profile of the Olympiad, in order to encourage more
organisations to see the competition as worthy of backing.
Von Schantz et al. (2012) discuss the reasons why language competitions struggle to find funding in
Sweden. This article raises the point that competitions in mathematics, science and technology are
supported by the Swedish government, while equivalent competitions in linguistics receive no such
support. Funding for Sweden’s Ling Olympiad in 2012 was provided by Young Scientists Stockholm
and the Royal Academy of Letters, with voluntary work undertaken to allow the competition to run.
When Sweden hosted the IOL in 2010, the Olympiad received no government funding, and was
required to charge an entry fee. The article argues that it is in the nation’s interest to develop
international links within linguistics research: the authors describe a decline in language learning and
language skills, which poses a threat to Sweden’s position in a competitive global market. The Ling
Olympiad website suggests that the lack of funding might simply be a lack of awareness of the
existence of the Swedish Olympiad, or, more worryingly, might be a conscious decision to prioritise
funding for science programmes over humanities.
Although UKLO has a strong network of supporters and associated universities in the UK, it is
important to consider some of these issues when planning for the future of the competition. One of
12
UKLO’s primary aims is to promote an analytical approach to language study, both in English and in
foreign language teaching/learning. As described in section three, below, an approach combining
the analytical skills associated with mathematics and science with the linguistic skills of language
use, may prove beneficial in encouraging students to develop skills in both subject areas. Feedback
from teachers also suggests that, even at this early stage, participation in UKLO has had a positive
effect on students’ attitudes to language learning. With such potential benefits for students in the
long-term, it is important that UKLO continues to find support.
If the issues with funding experienced in other countries are caused by a lack of awareness of such
competitions, it is important for UKLO to be aware of the need to raise its profile with organisations
and government agencies. As mentioned in section 1.5.3, above, there is no evidence of any links
between UKLO and the government at the moment. It would be valuable for UKLO to explore where
these links could be made, and to investigate how best to present UKLO to the Department for
Education as a valuable and popular extra-curricular activity. An increased liaison with the
government would raise the profile of UKLO, so ensuring continued support in future years, and
would help raise the profile of linguistics activities in schools, providing further support for proposals
of incorporating linguistics into the school curriculum.
Online resources
The Science Olympiads offered in the UK have a strong online presence, offering additional
resources to students in a variety of forms. The Chemistry Olympiad provides online questions for
students to tackle, with the opportunity for students to test their knowledge and see how well they
score. The Biology Olympiad website also offers students the opportunity to subscribe to BioNet, an
online subscription that includes access to e-magazines, newsletters, and emailed quarterly articles,
aimed at school and college students.
The UKLO website has recently been redesigned, to ensure it remains accessible as more content is
added. As well as access to past papers, the website links to a variety of training and promotional
materials for use in schools. There are plans to develop an online portal (described in more detail in
section four, below), which could include space to provide resources for schools. Here, test papers,
answer sheets, candidate recording sheets, certificates and promotional material could be made
available for schools to download.
It might be worth investigating whether students would benefit from training with more interactive
practice questions – perhaps developing an online test using past material. Although the online
Chemistry Olympiad was evaluated as ineffective in training for the international competition, it
remains useful and popular with students training for the national competition. Past Olympiad
questions might have to be modified to allow for easy marking online, but students may benefit
from being able to revise online, and get instant feedback on their performance, or tips on how to
improve. This approach could help support students training for the competition, without putting
additional pressure on teachers to mark more practice material. It might also be used to support
teachers when marking both practice material and the test itself.
13
Key Issues
Involvement of previous Olympians
Recommendations
Continued contact with IOL students, with a view
to developing the IOL training scheme to involve
input from previous Olympians.
Links to linguistics in the curriculum
Increased communication with similar schemes
to incorporate Linguistics into the curriculum in
other nations.
Use of IOL contacts to develop links with other
nations, and to potentially create a network of
groups looking to promote Linguistics in schools.
Links to other nations
Consider developing links with AILO, via existing
contacts and committee members, with a view
to discussing the potential for joint training
schemes or workshops.
Challenges and introductory competitions
Investigation into the potential of introducing a
lower level introductory paper, aimed
specifically at younger students or at first-time
competitors: a Linguistics Challenge.
Consideration of promoting the F/I papers as
ideal for first-time competitors.
Invitations to high performing students to enter
the next level paper the following year.
Liaison with government departments
Online resources
Further investigation into current links between
existing UK Olympiads and the government.
Development (or creation) of links with
government departments, especially the
Department for Education.
Possible development of interactive online
training, with (modified) past paper activities
available for online completion and feedback.
14
Section 3: Studies investigating male and female students’ attitudes to subjects and academic
competition
Part of the educational rationale of UKLO is to allow students to develop their academic abilities, not
only in language study, but also in formal disciplines of analysis and logical thinking. A large part of
the test paper itself requires a formal analysis of unfamiliar language data. Although originally
aimed at students at (the equivalent of) Key Stage 5 in England and Wales, UKLO now has a much
broader range of students getting involved, and can be seen as a means of introducing younger
students to formal analysis. This is a skill valuable not only in language study, but in any subject
where critical analysis of data is required. Activities such as UKLO may help to build links between
language learning and formal analysis.
A key issue involved in both language learning and in the analytical fields of Mathematics and
Science is gender imbalance. Research has found that, although there has been a general decline in
popularity of foreign language classes, this is especially the case for male students, who tend to
choose not to study languages at higher levels. In mathematics and science, however, higher level
(i.e. non-compulsory) classes tend to be male-dominated, with girls typically underrepresented in
extra-curricular mathematics and science disciplines. This section examines some of the research
that has been done to investigate these differences.
3.1: Female underrepresentation in science and mathematics
Tirri (2001) looks at some of the major life events reported by past competitors in the Finnish
Mathematical Olympiad to have influenced them in their career choices. The article looks at
interview data with previous Olympians, discussing both their personal and professional lives. Tirri
discovers that Olympiad participation played an important role in the academic development of all
the Olympians interviewed. Other important factors included encouragement from teachers, and
support from peers. Female Olympians reported experiencing less positive peer support during their
time in school, suggesting that it is important for female students to experience support to help
them develop their academic talents. The author recommends that female students “should be
guided to pursue intellectual hobbies that support their talent development during their school
years.” (7)
Feng et al. (2002) conducted a similar study in the USA, interviewing 15 American Olympians. They
found that the three main reasons for underrepresentation of females in mathematics and science
competitions were:
-
Bias towards male students in scientific fields
Attitudes towards competitive work environments
Social expectations of male and female students.
In terms of differences in approaching scientific problems, female Olympians reported a preference
for group-work activities, while male Olympians preferred to work independently. Many
Mathematical and Science Olympiads focus on individual, competitive work, although some offer
opportunities to work in teams. Female Olympians reported confidence in their academic abilities,
but a lack of confidence in the face of the pressures of intensive Olympiad training programmes,
showing a tendency to be more affected by scores than male Olympians are. The study suggests
that Olympiad structures and training schemes favour male attitudes to training and competing. Not
only does this risk discouraging female students from participating, but it might also prevent them
from reaching their potential if they do participate. Researchers suggest de-emphasising the
competitive nature of Olympiad projects, and emphasising their role in developing talent, and
incorporating more collaborative aspects to the schemes.
15
Although UKLO is a language-based competition, it requires students to spot patterns in datasets,
and encourages logical thinking. Practice in these skills can be applied to mathematics and science
subjects, and can develop students’ abilities in these subject areas. The above studies suggest that
female students are discouraged from taking part in Mathematics and Science competitions, due in
part to biases that favour male participation. This can have an adverse effect on their development
in these subject areas, and reduce their confidence in mathematics and science classrooms. If UKLO
can be seen as a means of helping students develop analytical thinking skills in an environment that
is not explicitly mathematics- or science-based, this could have benefits for female students, who
might otherwise be put off from getting involved in such activities.
3.2: Male and female attitudes to language learning
Clark and Trafford (1995) conducted interviews with staff and students from maintained schools in
England to investigate the differences between male and female language students. The
researchers asked students about their perceptions of learning languages. Their aim was to discover
potential reasons for male students performing less well in modern languages than their female
classmates. They found that all students struggled to find language learning relevant or beneficial,
with responses indicating that the material learned “did not correspond with what they felt they
would say if in conversation with native speakers.” (320) The researchers state that the most
effective way to improve students’ attitude towards foreign language learning is to improve their
confidence in handling language data.
Stables and Wikeley (1999) find similar issues in students’ attitudes to language learning. Using a
longitudinal study to investigate students’ attitudes over time, they found that French and German
had dropped in popularity, and were ranked bottom in terms of subject preference (in 1996).
Students did not seem to fully understand the benefits of learning foreign languages, mentioning
that they did not want to work or live in France or Germany as being reasons for their belief that
foreign language study was not useful. The research also found that students find languages
difficult, and that a lack of confidence in lessons discourages them from continuing language study
beyond compulsory levels. One of the study’s conclusions is that a more general approach to
language learning might help:
Curriculum planners might consider options relating to generic language awareness in an
attempt to increase motivation before concentrating on long-term study of a particular
language, on the assumption that this might have the potential to 'grab' more students'
interest in a greater number of ways than the specialist teaching of one language in the early
years. (30)
Fisher (2001) conducted a similar study, interviewing students from maintained schools in the UK to
investigate attitudes to foreign languages. The motivation behind this was that, despite an increase
in the overall numbers of students choosing to study post-16, foreign languages had witnessed a
decline in numbers. The responses at interview suggested that students lacked confidence in
dealing with Foreign Languages, and were especially discouraged when tackling unfamiliar terms and
making mistakes. The more popular tasks in class were translation exercises, where students
reported enjoying working collaboratively to translate passages. Although students reported finding
grammar exercises challenging, they indicated that awareness of grammatical concepts helped them
feel more at ease with learning new concepts in different languages. The researchers’
recommendations include introducing grammatical exercises and translation activities to younger
students, and including more collaborative work in class.
16
More recently, Davies (2004) compared attitudes towards modern languages in secondary school
students, gathering data from students and their teachers. Davies notes that female students
outperform male students in modern languages at GCSE level, and at a higher rate than in other
subjects (53). When asked about their experiences of language learning, male students indicated
that they felt languages were irrelevant and too difficult to engage with (53). The research found
that there was not only a difference in performance in foreign languages, but also in attitude, with
male students tending to find French less useful than female students. The study suggests that
these attitudes are found even in younger students, and increase over time.
The findings from these studies suggest that students respond well to collaborative work, translation
exercises, and learning about grammatical structures. These aspects of language learning give
students more confidence in handling unfamiliar language data. Students report a lack of
confidence as one of the reasons why they do not enjoy learning foreign languages, and why they
are more likely to drop languages at a younger age. Stables and Wikeley’s suggestion that a more
general approach to language learning may be beneficial, especially for younger students, fits in with
the focus of UKLO. By getting students to think about some of the general properties of language,
and by enabling them to explore grammatical features in language, UKLO may give students more
confidence in the classroom environment, and encourage more students to study languages to a
higher level.
3.3: Gender and UKLO
Quantitative data from the past three years of UKLO shows that participation from both male and
female students has been largely equal (see Table 1):
Year
2010
2011
2012
Proportion female
Proportion male
Proportion
undisclosed
39%
50%
11%
50%
47%
3%
52%
48%
0%
Table 1. Proportions of male and female students participating in UKLO from 2010-12
Even when paper levels are taken into consideration, there is little variation in this pattern, with
roughly equal numbers of male and female students being entered at each level (see Table 2).
Female
Male
Foundation
2010 (33.3% undisclosed)
33.3%
33.3%
2011 (9% undisclosed)
42%
48%
2012
54%
56%
Intermediate
2012
49%
51%
Advanced
2010
43%
57%
2011
54%
46%
2012
50%
50%
Table 2. Proportions of male and female students entered at each level from 2010-12
Tables 1 and 2 show that, over the three years of UKLO, there has been fairly equal participation of
male and female students. Equal numbers of male and female students have also been entered at
each level, suggesting that teachers assessed male and female students as equally able to approach
17
each competition level, and that equal numbers of male and female students felt confident
approaching each level of the competition.
Figure 1 shows the distribution of marks according to gender, taking into account data from 201012. Female students tend to score in the middle mark bands, peaking at band five, while scores for
male students are more evenly spread across bands three to six. A logistic regression analysis of
data from all three years of the competition shows that gender is not a strong predicting factor in
students’ scores, suggesting that, despite the apparent differences found in Figure 1, overall
performance from male and female students has been similar over the three years UKLO has run.
Figure 1. Mark bands for male and female students 2010-12
As well as having roughly equal numbers of male and female entries, UKLO also sees a fairly equal
distribution of scores for both male and female students. There is no strong indication, after three
years of the competition, that either gender group is out-performing the other. The top mark band
in this data (representing students scoring 87.5% and over at Round One) shows the sharpest
difference between male and female students – in 2012, twice as many male students as female
students scored in this top band. However, the top competitors at Advanced level (i.e. those
students invited to Round Two) tend to be reasonably evenly split, in line with total proportions of
male and female students taking part at Advanced level. Students who progress from Round Two to
the IOL have so far tended to be male. It is early in the competition’s development to suggest
whether this is a significant trend – this is an area worth monitoring over future years, to ensure the
competition continues to reach out to both male and female students, and to attempt to address
any issues, should they arise.
The outcome from analysing UKLO’s impact for male and female students is generally positive: it
suggests that UKLO is not only attracting both male and female students, but that it is well-suited to
the academic interests and abilities of both male and female students. As discussed above, this
could have a positive effect for students in either gender group. Female students, who may be
underrepresented in extra-curricular mathematics and science activities, are given a chance to
develop their analytical skills in a less explicitly ‘mathematics and science’ environment, while male
18
students, who may be underrepresented in language classrooms at higher levels, are given the
opportunity to build their confidence in dealing with language data.
Key Issue
How this applies to UKLO
The importance of extra-curricular activities in UKLO provides an intellectual activity for high
supporting academic development
school students, which allows them to develop
their linguistic and analytical skills, and supports
their academic development. This is seen as
especially valuable for female students, who
typically report lower levels of support in their
interests in analytical subjects.
Differing attitudes to competition
UKLO has seen fairly equal numbers of male and
female participation (see section 3.3, below),
suggesting that the competition structure is
suitable for a range of students. UKLO allows for
collaborative training and for team competition
in the F/I papers. UKLO emphasises personal
skills development as well as achievement at an
international level, which may help it appeal to
both competitive and less competitive students.
Lack of confidence in handling unfamiliar UKLO provides an opportunity for students to
language data
approach entirely unfamiliar language data and
decode it.
By approaching language data
logically, students are able to understand
examples of languages they have never
encountered before.
This can build their
confidence in handling unfamiliar language data.
A general approach to language learning
Rather than developing students’ skills in one
particular language, UKLO allows for the
development of a more general knowledge of
language structure and use, in line with
suggestions that general linguistic awareness
could provide a strong foundation for later,
language-specific, learning.
Value of grammatical knowledge
UKLO allows students to develop their
understanding of the grammatical concepts at
work in languages of the world. This can be used
comparatively to help them understand
grammatical concepts in particular languages.
An increased understanding of grammar may
help increase student confidence in the language
classroom.
19
Popularity of collaborative work
As discussed in section 3.1, above, UKLO offers a
balance of individual and collaborative work,
appealing both to students who prefer team
approaches to work, and to those who prefer to
work independently.
20
Section 4: Information gathered from students, teachers and UKLO records
4.1: A description of the data gathered
Quantitative data
In order to fully describe and evaluate UKLO’s structure and organisation, this report also looks at
quantitative data from UKLO over the past three years. This section deals mainly with the numbers
of students who have taken part in UKLO since 2010, and looks at:
-
How many students have taken part
How many schools have entered students
How many independent/maintained schools have taken part
Which age groups students have fallen into
How many students have been entered for each paper level
The results from this section also assess which factors are most influential in students’ performance.
By looking at these figures, the research aims to show who UKLO is currently including, and how
performance patterns demographically.
The data for this section of the research has come from documents held by the UKLO committee,
including lists of registered schools, marks lists for individual competitors, and total numbers of
papers entered per school. A problem with using these sources is that they might not represent the
total numbers of students taking part in UKLO: once a school registers for UKLO, they are sent the
papers and are able to sit them in school whenever they wish (at least for Foundation/Intermediate
papers). There is no obligation to send completed scripts back to UKLO, or to inform UKLO of
participation or scores. A number of schools may have used UKLO materials and sat the round one
paper without entering students into the national competition itself. The data presented in this
section will therefore be limited to looking at the schools that have registered with UKLO and have
returned completed round one papers/marks.
Alongside this data, feedback has been gathered from teachers and students involved in UKLO 2012,
to allow for evaluation of the competition’s current structure.
The teacher survey
Data was gathered from teachers who had registered between June 2011 and August 2012. This
sample includes teachers who have registered for UKLO but not yet participated. A 10-question
online survey was constructed to gather feedback from teachers on the competition structure, plus
indications of how popular the test was with students, and whether there have been any positive
effects of participation on learning or attitude in class. The link to the survey was distributed via
email to a pilot sample of four teachers, who were asked to respond to the survey questions and
give feedback on the survey structure. The final survey was distributed to the 312 teachers on the
current UKLO mailing list. After six weeks, 35 (=11% of those contacted) teachers in total responded
to the survey (see Table 3 for further details).
Teachers from independent schools
20 (=57% of total responses)
Teachers from maintained schools
15 (43%)
Table 3. Number of teachers responding to the online teacher survey
Fifteen teachers specified that they were foreign language teachers, although some responses
provided no information on subject area.
21
The teacher survey asked for feedback on how the competition was run and organised. The results
provide information on how teachers ran the competition in their own schools, and on whether
schools are thinking of participating in 2013.
Detailed feedback was gathered from some of these responses: 31 out of the 35 responses (89%)
indicated that they would be willing to make further contact with UKLO to discuss their answers.
Three responses were from teachers who were also members of the UKLO committee – these
teachers were excluded from the group emailed for further contact. A further three participants
expressed that they would be willing to talk further about their responses, but provided no email
address. Excluding these six participants left 25 contacts. These 25 teachers were emailed, asking if
they would be willing to talk in more detail about their feedback. In-depth comments were gathered
from six teachers, either via email or telephone contact.
Data from non-participating schools
A large number of schools registered for UKLO 2012 did not submit scores. To investigate why
schools chose not to participate, a survey was designed and distributed to schools. All schools that
had registered for UKLO after the date of Round One 2012 were excluded, as were all schools who
had submitted marks or papers. This left 183 schools, who were sent a survey link via email. In
total, 17 teachers responded (9%).
The student survey
A survey was designed for distribution to students who participated in UKLO 2012. A short written
questionnaire was designed and distributed to the IOL team for 2012, asking for feedback on the
competition, including the suitability of questions, fairness of marks, and general attitudes towards
the Olympiad. All four members of the IOL team responded, and the questionnaire was re-designed
as an online survey. This was sent out via email directly to the remaining 12 Round Two participants
from 2012, six of whom responded online. The survey was then distributed to the 312 addresses on
the UKLO mailing list (as of 4.9.12), and teachers were asked to distribute the survey link to students
who had participated in Round One 2012. The survey was designed to be completed individually,
and comprised eight questions. After six weeks, a total of 36 students had responded (see Table 4
for details).
Age range of student responses
Gender
School type
Paper level
13-18 years
Female: 13
Male: 22
Undisclosed: 1
Independent: 31 (from 10 different schools)
Maintained: 5 (from 1 school)
Foundation: 1
Intermediate: 10
Advanced: 25
Table 4. Number of students responding to the student survey
4.2: Results and discussion of the data
4.2.1 How effective were questions?
Of the ten questions included in Round One 2012, all appeared in at least one of the North American
(NACLO) or Australian (OzCLO) Olympiad papers from the same year, with four of the first round
questions appearing on round two papers for NACLO or OzCLO. Three of these questions (question
2: Danish, question 3d: Dutch and question 5: Esperanto) appeared on the UKLO Foundation paper,
and one (question 9: Waanyi) on the Advanced paper. Although question 3d (Dutch) scores highly
on average, questions 5 (Esperanto) and 9 (Waanyi) proved difficult for students, and were the
22
lowest scoring questions of the competition (see Figure 2). These low scores, combined with
teacher feedback indicating that students found these questions particularly difficult, and their
appearance on the second round papers in other competitions, suggests that certain aspects of the
UKLO paper were more difficult than expected at a first round level.
Figure 2. Highest and lowest scoring questions from the 2012 paper
Factors influencing scores
Despite the difficult questions on this year’s paper, marks have improved year on year, with more
students scoring in higher bands in 2012 than in previous years3 (Figure 3, over page).
Scores from 2010 are, on average, lower than scores in 2011 and 2012, with the latter two years
patterning similarly. Students taking the test in 2012 tended to score slightly higher than in 2011.
There is a broader distribution of marks from 2011 onwards, suggesting that most students fall
within a wide score range. In 2010, this distribution is tighter, with a large proportion of entries
falling into one or two bands.
3
For the purposes of analysis, all marks were converted into percentages of total marks available, and were
grouped into eight percentage bands. Each band represents a range of 12.5%.
23
Figure 3. Distribution of marks per year
A logistic regression analysis of scores from all three years of the competition shows that the most
significant factors affecting scores were student age and prior experience of the competition. As
discussed in section 3.3, above, gender was not a strong predictor of mark band. Neither school
type (independent vs. maintained) nor paper level (Advanced vs. Foundation/Intermediate) were
shown to be strong predicting factors in mark band achieved.
Looking at the top performers from the Advanced paper (i.e. those invited to Round Two), students
from independent schools dominate, with students from maintained schools generally being
underrepresented at Round Two. This could be as a result of the generally small numbers of
students from maintained schools taking part in general. One way for UKLO to address this issue
would be to encourage more maintained schools to enter students into the competition formally,
and to aim to reduce the dramatic drop-off rate of maintained schools between registration and
actual participation. This might improve the representation of maintained schools in the
competition, at all levels. Another consideration would be to extend the feedback material used in
this survey to specifically approach teachers from maintained schools, and to provide additional
support if necessary.
Figure 4, below, shows the relationship between age and score over the three years. There appears
to be a positive relationship between NC year and score, with more year 12-13s occupying the
higher mark bands. While there is a reasonably sharp drop in the number of younger students (year
9 and below) in the higher bands, marks for students in years 10-12 are fairly evenly distributed
across the range.
24
Figure 4. Distribution of marks per NC year (2010-2012)
Figure 5, below, shows the distribution of scores according to whether students were entering the
competition for the first, second or third time. Students with prior experience of the competition
score on average higher than students participating for the first time, with those students competing
for the third time scoring slightly higher than students competing for the second time. There are no
students participating for the third time in either the top or bottom band. One explanation for this is
that only 70 students fell into this category. This result shows, however, that student scores
improve over time. This may be linked into the association between age and performance. Older
students, and especially those who have entered the competition before, are more likely to score
highly than younger students with less experience.
Figure 5. Distribution of marks according to prior experience of the competition
25
Student feedback
When asked to describe their experiences of UKLO, the majority of feedback from students was
positive, with comments on the competition’s difficulty.
-
29 out of the 36 students surveyed (=81%) described the competition as ‘challenging’
21 students rated the competition as ‘fun’ (=58%)
27 students rated the competition as ‘interesting’ (=75%)
One third of students found the questions ‘abstract’ (12 students)
One third of students also rated the questions as ‘frustrating’ (12 students)
Very few students indicated that they found the paper boring (2), dull (1), or too difficult (1)
Most students reported that they felt their marks for Round One were fair, with only one
student disagreeing.
A total of 12 students felt unable to comment on the fairness of the Round One marks, with
half of these reporting that they did not receive their marks from Round One
Teacher feedback
Of the 35 teachers who responded, 31 agreed that the level of difficulty in
the papers was appropriate, with only four stating that they felt the level
was not appropriate.
Although the majority of teachers felt the difficulty level was appropriate, many expressed that
some students, especially those in lower years, found the test daunting. From the more in-depth
feedback gained from teachers, responses indicated that students largely enjoyed the challenge, and
were encouraged when they were able to successfully find the correct solution. A point that was
raised by more than one teacher was that it might encourage students to have an easier question at
the beginning of the foundation paper: students were easily put off by a difficult question early on in
the paper, but enjoyed the challenge once they continued with the test.
In terms of variety, 34 teachers agreed that the 2012 paper offered a wide
variety of questions, with only one response disagreeing. Students
responded very positively to the material, and enjoyed tackling the various
questions.
A similar pattern was found with responses in terms of how engaging the
material was, with 34 teachers indicating that they felt the material was
engaging for students.
Specific comments suggested that students were not immediately enthusiastic to take part, but
;elfgefk
engaged with
the material after spending time doing an example problem. Although students found
some of the material challenging, teachers emphasised that many students enjoyed the challenge,
with particular enthusiasm for tackling difficult logical problems.
Feedback from students:
- 22 out of the 35 responses from teachers indicated that comments
made by students were mainly positive
- 11 teachers indicated that they received both positive and negative
comments
- Only one teacher reported mainly negative feedback
26
More detailed comments suggest that students had found the exercise difficult but enjoyable. One
teacher reported that students had commented on how enjoyable UKLO was because it was
different from anything they did as part of their A-Level study.
Key issues
Errors in the test paper
The 2012 paper contained errors, which were
known to UKLO before this project began.
Teachers reported that these errors confused
students, and led to difficulties in giving
guidance. With many teachers and students
commenting on the test’s difficulty, unnoticed
errors on the paper may further discourage
students, and cause difficulties in an exam
situation.
Recommendations
Using more moderators for the Round One
paper.
With more than one moderator, the chances of
errors going into the final paper will be reduced.
More moderators will also mean that no one
committee member has sole responsibility for
question moderation, putting less pressure on
individuals.
This is in line with UKLO’s current plans to
establish a moderating team on the committee.
Alternative questions not equivalent in terms of
difficulty
One teacher commented in feedback that the
Welsh question seemed more challenging and
time consuming than the Dutch alternative,
stating that most students were unable to finish
the Welsh question. The scores from 2012
suggest that students found the Dutch question
easier than the Welsh question, with question 3d
scoring on average two marks higher than
question 3w.
Pre-testing of questions, especially where
alternatives are provided, to ensure estimates of
difficulty and/or time required for completion
are accurate.
Difficult opening questions
Opening questions were more difficult than
expected, and students were discouraged by
this.
Inclusion of an easier ‘starter’ question,
especially in lower level papers. This need not
mean a lowering of the overall paper difficulty.
A more accessible opening question might
increase student confidence, and completion of
the paper.
Increasing interest from younger students
There was no clear call for introducing an easier
paper level, although some teachers commented
that younger students were getting more
interested in the competition.
Continued monitoring of performance and
suitability of questions to ensure suitable
material and support is provided for a
broadening age range.
Underrepresentation of students from
maintained school in top mark bands
Although too early to tell whether this is a
significant trend, this is a tendency worth
monitoring.
Continued monitoring of performance, and use
of teacher and student feedback to ensure
necessary support is provided to maintained
schools. Furthermore, a greater focus on
encouraging maintained schools to enter the
competition after registration is possible.
27
4.2.2 Benefits for school subjects
Feedback from teachers
Fifteen teachers reported feeling no effects of student participation in UKLO. Of these, five
commented that it was too soon to tell, and one commented that too few students had taken part
to tell. Eight of the 15 ‘no’ responses commented that they felt sure benefits had been felt by
individuals, and that, in time, positive effects would be felt on a larger scale.
Of the 20 teachers who reported seeing effects of participation, 15 were foreign language teachers.
The positive effects described included:
-
Improved student interest and enjoyment
Improved student attitude and motivation
Improved student confidence and enthusiasm
Help in letting students see the value in learning languages
Help in developing language skills
The remaining five ‘yes’ responses came from English Language or Classics teachers, who
commented on:
-
Increased student confidence
Positive effects on raising the profile of the subject area
Increased student interest
Improves teacher-student relationships
Feedback from students
When asked whether they had experienced any benefits from participating in UKLO, 23 of the
students indicated that they felt participation had helped them in their academic work. Where
specified, most of these students had felt benefits in languages (both classical and modern).
Comments indicated that participation particularly helped with:
-
Approaching language logically: helping students to break down unfamiliar words and
phrases, which has helped with analysing and translating passages
Approach language problems more confidently: helping students to understand grammatical
features, and to tackle texts with unknown vocabulary
Exploring linguistics as a possible subject for further study
Skills required in subjects such as mathematics, computer logic and physics: helping students
to develop analytical skills in approaching data and finding patterns
This feedback suggests that, in the three years it has been run, UKLO has already begun to benefit
students and teachers, both in English and foreign language departments, and in other subjects,
such as mathematics and science. As discussed in section 3, UKLO’s appeal to students with abilities
in languages and in mathematics makes it a valuable tool in bringing together the two disciplines,
and encouraging development in both subject areas. As considered in section 1.5.3, evidence of
UKLO’s benefits over a wide range of subjects will prove advantageous, not only in helping to secure
continued support for the competition, but also in supporting plans for incorporating linguistic study
into the classroom. Continued support will also ensure that long-term benefits of participation in
UKLO can be felt, and will allow the competition to further develop to meet the needs of growing
interest from schools.
28
4.2.3 How effective was the structure of the competition?
The 2012 competition was the first to use volunteer markathons to help with script marking: earlier
competitions had been marked by the committee or by volunteer individual markers. Markathons
were organised by a number of academics, who enrolled student volunteers to spend an afternoon
marking papers. Another change to the marking system involved schools sending scripts directly to
their allotted marker, which worked to speed up the marking operation. With an expected increase
of entries for 2013, UKLO is looking to make more changes to ensure the marking system copes.
UKLO 2012 also saw a pilot scheme of mentoring for the IOL team. For six weeks between selection
and the international competition, the four students were each teamed with two undergraduate
Linguistics students from the University of Edinburgh. Using regular Skype-based training sessions,
mentors assisted the team in approaching IOL questions. The aim was to build students’ awareness
of linguistic structures, to allow them to provide full explanations for their answers in the IOL test.
The IOL team responded positively to the training, commenting that it improved their confidence,
and was an enjoyable way of training. The team also performed strongly in Slovenia 2012, with all
four students awarded medals.
Feedback from teachers
Responses from teachers indicated that some students found it difficult to commit time to training
for UKLO. For students in years 10-13, preparations for UKLO often clash with exams or mock
exams, with feedback suggesting that this discouraged some student from continuing. Other
academic and extra-curricular commitments, such as music examinations and participation in other
Olympiads also made it difficult for some students to commit. Teachers expressed concern that
competing demands might put too much pressure on students, and might discourage them from
continuing.
The following are areas which teachers specified as potentially problematic:
-
The layout of the paper was not very user-friendly
There were problems knowing what was to be sent where
Details for each round weren’t known well enough in advance
It was difficult to know what the scores meant, and to explain this to students
The certificates contained incorrect information
It took a long time for marks to be returned
There were also problems with registration, paper structure and timing
Further comments indicated that students found the paper difficult to navigate, with specific
comments that there was not always enough room for answers. There was some indication that
schools would prefer the competition process to take less time. This was mainly in reference to
receiving feedback and marks. Two schools reported issues with registration and notification of
Advanced entries, but gave no further details. Most schools indicated that they felt there were no
changes that needed to be made to the competition.
Feedback from non-participating schools
Reasons given for not participating in UKLO 2012 were varied.
Five schools reported sitting the test, or using the test
materials, but not submitting scores to UKLO.
29
Further comments indicated that staff felt that sitting the formal test was not suitable for students
at that time of year, commenting that academic and time pressures involved in running the test as a
formal exam would put too much pressure on students. Of these schools, one reported that scores
would probably not be submitted in future, and another commented that they were unsure whether
the school would participate in 2013.
-
-
Timetabling and staffing were also primary reasons for not participating: some schools
found they did not have enough time between registering and round one to prepare, while
others commented that they could not find time, staff or resources to run training sessions
or the test itself.
One school reported that, amongst other issues, there was not enough interest from
students to run the competition.
Two schools encountered problems with administration, with two schools reporting no
further contact from UKLO after registration, and one school reporting that they were
unclear on the dates and logistics of round one.
Eight of the schools indicated that they wanted to participate in UKLO 2013, with
three unsure and three stating that they do not intend to participate. Of these three,
one indicated that the school had used the materials, but that students had found it
difficult and there had not been much departmental support.
Key Issues
Timetabling
Teacher feedback suggests that timetabling was
an issue, with some schools unable to participate
due to being unable to find the time for
preparation or for the test itself
Structure of the Advanced paper
This has presented difficulties, for schools in
terms of finding time and staff to sit the test, and
for students in terms of sitting a two and a half
hour exam.
Student commitment
Teacher feedback indicates that, while most
students enjoy UKLO, some find it difficult to
commit to alongside other academic and extracurricular commitments.
Post-competition
teachers
feedback
Training for the IOL team
from
Recommendations
These issues are all difficult to overcome. See
section 1.1 for considerations in terms of timing
for Round One. There is little room to adjust the
Advanced paper’s length or structure while
ensuring it remains a reliable means of selection
for Round Two.
With so many schools
participating, it would be impossible to make
adjustments to accommodate to every
timetable.
Continued
post-competition
feedback from teachers may help highlight
particular areas for improvement.
students/ Establishment of post-competition evaluation
form. This could be a separate form, or could be
incorporated into test papers/material sent to
schools. This could be incorporated into the
UKLO portal, accessible via student/teacher
profiles.
Development of the pilot scheme from 2012, to
involve more UK universities.
Continued
contact
with
previous
IOL
competitors, and consideration of using previous
Olympians to help train future IOL teams.
30
4.2.4 How effective was administration?
Registration:
o One teacher in the teacher survey indicated that changes to registration were
needed, but gave no further information.
o Two of the non-participating schools indicated that they had had no further contact
from UKLO after registration.
o More in-depth interviews with teachers revealed that, although many preferred to
get fewer emails, some also found email contact valuable in terms of reminding
them about the competition.
Advertising:
o Most of the teachers who responded reported hearing about UKLO for the first time
via a recommendation, either from a fellow teacher (at the same school or at a
different establishment), or from a senior member of staff.
o Links between UKLO and the ISMLA also provided teachers with information about
the competition.
o Many teachers reported hearing about UKLO through publications or emails from
various organisations, including:
 Attain Magazine
 British Academy
 JACT (Joint Association of Classical Teachers)
 Routes into Languages
 ALL (Association for Language Learning)
o Five teachers reported that they had actively searched for a UKLO-type project, to
run in parallel with similar competitions held in Mathematics and Science subjects in
their school.
Data from non-participating schools
How schools first heard about UKLO
Reasons for registering
- Recommendations from teachers/senior
- It was free
staff members (5/16)
- There was interest from pupils
- Email or web link (4/16)
- It sounded like fun
- Previous involvement within school
- It was a good way of promoting language
(1/16)
study in school
- Courses aimed at teachers (1/16)
- ISMLA (1/16)
School support: hand-outs, print-outs, past papers etc.
- 15 of the teachers who responded to the teacher survey reported using past papers as part
of their training for students, with others reporting the use of example questions.
- At least one teacher mentioned using the PowerPoint available on the UKLO website, to
introduce students to the Olympiad.
- Teachers also mentioned using online resources including videos on ‘how to approach an
Olympiad question’, as well as the promotional posters available for download from UKLO.
None of the feedback suggested that teachers had struggled to find the resources online, although
in-depth interviews with teachers indicated that additional training resources would be very useful.
Teachers were particularly keen to see online material, as they felt it would be the most accessible,
both for staff and students preparing for the competition.
31
Marking and feedback
The main area where teachers reported a desire for a change in arrangements was in marking and
feedback, with many teachers commenting on the complicated structure of the mark scheme, and
the time taken for marks to be returned to schools. More in-depth comments indicated that some
teachers would prefer a more detailed explanation of how students should be approaching
questions. Comments also indicated that schools would prefer to receive marks from Round One
sooner, as students were keen to know how they performed. Further feedback indicated that
students would benefit from more in-depth feedback, with suggestions of implementing a medal
scheme for non-Advanced entries. Teachers felt this might help students understand how they
performed in context, and know where they did well and where they could improve.
Key Issues
Registration
No further contact from UKLO after registration
Recommendations
The recent implementation of a new mailing
system allows teachers to be contacted directly
from the database. This is more reliable than
having email addresses transferred from the
database to a mailing list, and does not risk
addresses being inadvertently lost.
Marking
Complicated mark scheme
Revision of mark scheme to ensure accessibility
for all teachers (supported by possible additional
training for teachers, see section 4.2.4, below).
Marks took a long time to be returned
Plans to develop the online portal for scripts and
marks. This could be used for registration of
markers, assigning markers to schools, uploading
mark schemes, script transfer and creation of
marks lists. This may speed up the marking
process, and ensure marks are returned to
schools as soon as possible, preferably before
the end of the summer term.
Students were keen to know their marks in
context
Implementation of a Bronze, Silver, Gold scheme
for Foundation/Intermediate papers. These
results were reported as having more meaning
for students than scores and percentages.
Advanced Notification
It is difficult to predict numbers of scripts that
will need to be marked, with a short time
between receiving final figures from schools and
the competition itself.
One solution could be to ask schools for
preliminary figures earlier in the year: this might
put teachers under unnecessary pressure, and
could result in problems with over- or
underestimation.
The proposed portal could provide a facility for
teachers to report numbers of Advanced
candidates and finalise with UKLO the dates of
the test in each school.
32
Logistical problems
Knowing what should be sent and where,
knowing details for each round
Incorrect information on certificates
Clearer outlines in preliminary emails, to ensure
all schools are clear on the logistics of the
competition.
The proposed portal and improvements to
registration systems should help reduce the risk
of incorrect information being printed on
certificate and rewards.
4.2.5 School support
Teacher feedback
- 23 of the 35 schools provided some training for UKLO 2012
- 16 teachers describing regular practice sessions, either at lunch-time or after school.
- The more detailed feedback showed that around half of the schools that provided training
used regular (or at least more than one) training sessions.
- Around half reported using past papers or example questions
- 5 teachers indicated that students worked together in groups, either to work through
problems or to discuss their solutions. (NB: this refers to teachers reporting preparation
work in groups, with no specific reference to regular training sessions).
- 2 responses described the use of UKLO materials in-class, with one teacher reporting
homework assignments.
- 4 teachers indicated that they had distributed material via email/school intranet services.
For some schools, this was in conjunction with more structured training sessions, while
others used only online distribution.
Of the schools that reported little or no training, reasons cited included:
-
Not having enough time to prepare
Not being able to find time for training
Not feeling confident in delivering training
Feedback suggested that these issues may be easier to overcome in future years, with more time to
plan training around staff and student commitments. Some teachers commented that they
themselves had had no training. Although some teachers mentioned that they enjoyed working
collaboratively with students, one teacher mentioned that a lack of knowledge was a reason for not
providing in-depth training, as the students seemed better able to tackle the problems than the
teachers themselves.
-
All but two schools reported that they intended to enter UKLO 2013, with one indicating that
they were not sure whether their school would re-enter.
Most schools also reported that they intended to provide training for 2013, with three
schools reporting an intention to provide training where none had been provided for the
2012 competition. Responses indicated a wish to provide more structured training sessions,
and to use more resources than in 2012, with plans including:
o Running ‘Introduction to Linguistics’ sessions
o Using more past paper resources
o Providing training for younger students interested in taking part
One response indicated a lack of certainty about what the best kind of training would be, stating that
the teacher was unsure whether any training would be provided. No schools that had provided
training for 2012 indicated that they intended to provide no training in 2013.
33
Student feedback
When asked about the training they had done for Round One:
-
11 students reported not having prepared for the test.
The rest had done some form of preparation:
o 11 students taking part in organised school sessions (i.e. 11 responses made specific
mention of lunchtime/after school sessions or clubs, or worksheets in-class)
o 11 going through at least one past paper (either in school or at home). Three
students stated that previous participation had helped them with their training.
When asked to evaluate their training, students gave a wide variety of responses. Many felt that
they had done enough training (looking at past papers, worksheets in class, organised sessions in
school), and said that preparation was interesting and engaging. Others reported that they would
have liked to do more, with additional comments indicating that some students found it difficult to
find time in the midst of other academic commitments. A small number of students indicated that
they felt the training they did was not useful, with comments such as ‘It’s not something you can
really train for’ and ‘I wasn’t cut out for the Olympiad’.
How can UKLO help schools?
A suggestion for the development of school support is for UKLO to hold recruitment talks, designed
to encourage students to take part. Another idea that has been raised to try and tackle the problem
of schools registering but not participating is the introduction of training days for teachers and/or
pupils. A training day aimed at teachers is currently being considered by the University of
Huddersfield. Given the success of the IOL training scheme, a similar scheme made available for
teachers might be valuable.
The issues to consider here are
whether teachers would have time to
commit to training schemes, and who
would be recruited as mentors.
A one-off training session might
require less long-term commitment,
but responses to interviews with
teachers suggest that it is difficult to
find time and support for full-day
training events.
A popular suggestion from teachers was
accessible, online resources, such as training
videos or worksheets.
These could be
accessed by both students and teachers, and
would put fewer demands on schools’ time.
Central training sessions are still worth considering,
however, as they will provide opportunities for
communication between teachers and the committee on a
larger scale, as well as the opportunity for more detailed
input from teachers, which may help UKLO tailor its
support to staff from different schools and subject areas.
34
Following from the success of the IOL training in 2012, a number of students and graduates from the
University of Edinburgh have started a project to develop training materials for use by teachers and
students in preparation for Round One, and by students wishing to participate in the training
scheme in 2013. These resources would fit in well with requests from teachers, which indicate that
accessible, online material in the form of training videos or worksheets would be the most effective
forms of material, for training both students and teachers in how to approach some of the
competition questions.
4.2.6 School recruitment
Between June 2011 and Round One 2012, 278 schools registered online.
Of these, 127 were independent schools and 151 were maintained (with one school not
disclosed).
In total, 97 schools entered students into the 2012 competition, with 63 independent schools
and 33 maintained schools entering.
Although more maintained than independent schools registered for participation, fewer go
on to enter students: overall, only around 35% of registered schools entered scores.
While just under half the registered independent schools went on to participate, only a
quarter of registered maintained schools entered round one officially.
These results suggest that UKLO’s recruitment is successful at encouraging both maintained and
independent schools to register, but may need to work more closely with maintained schools to
encourage them to enter more students into the final competition.
-
In total, 1848 students participated in UKLO 2012, with 1311 students from independent
schools and 537 students from maintained schools.
A large number of younger students took part in 2012, with 495 students in year 9 or below.
This represents just over a quarter of all participants, which compares to 20% in 2011 and
16% in 2010.
The competition has increased the proportions of younger students taking part, suggesting that
UKLO’s recruitment has been successful in appealing to younger students. If this growth continues,
UKLO may have to make changes to arrangements in order to provide full support for students in a
wider range of age groups.
Most students reported hearing about UKLO through a teacher at their school, with four students
indicating in the survey that they were told about UKLO by a fellow pupil. When asked why they first
came to participate, the primary reasons were:
-
It sounded interesting
A teacher had recommended participation
Students also reported that it sounded fun, and that they were entered into the competition
as part of a class or larger group of students.
Five students said that they had taken part in 2011 and wanted to participate again
One indicated that a teacher had entered them into the competition on their behalf
One student reported that incentives to attend practice sessions had encouraged him to
take part.
35
4.2.7 Liaison
- Liaison with schools:
o Other nations have noticed the popularity of the Olympiad, despite linguistics not
being offered in schools, and have taken the opportunity to use the competition as a
way of helping integrate linguistics into the classroom, and of supporting arguments
for incorporating linguistics knowledge into language learning. Feedback from
schools suggests that teachers are keen to integrate UKLO-type activities into their
regular teaching, and use the material not only for extra-curricular activities, but to
benefit all students in their learning.
o Feedback also suggests that additional support and resources in training teachers to
tackle the material would be valuable.
As well as the current contact between UKLO and schools, in terms of promoting the competition,
and communicating during the competition itself, it is worth considering what liaison UKLO could
develop with schools by means of supporting schools in their use of UKLO material, and in training
teachers to prepare students for the competition. Finally, continued liaison with schools in the form
of post-competition feedback, both from students and teachers, would help UKLO keep track of key
issues that need resolving.
Other key areas for liaison
Other nations
Details from other Olympiads suggest that links
between nations, both in training and in
competition, have proved successful.
Recommendations
As UKLO has existing links to the other
Olympiads within ELCLO, it might be possible to
develop stronger ties between countries, and
provide mutual support in training and
development of competitions (see further
discussion in section 2.4, above).
Universities in the UK
Maintain links to UK institutions, especially
through markathons and continued contact with
previous competitors progressing to higher
education.
Development of the IOL training scheme to
involve undergraduates from more UK
institutions. See section 1.5.2 for further
discussion.
Supporters and the government
Investigation into links with the Department for
Education.
Maintenance of links to current supporters, and
continued contact, to ensure continued support.
See section 1.5.3, above, for further discussion.
36
Summary of recommendations made in this report
Below is a summary list of the key recommendations made in this report, with references to relevant
sections of the report where each issue is discussed in more detail.
-
-
-
-
-
-
-
-
Development of the IOL training scheme, as used in the 2012 competition, to extend the
scheme to more UK universities, and to involve input from previous Olympians. This will
involve continued contact with IOL students after the competition (see section 1.5.2, section
2, and sections 4.2.3 and 4.2.7).
Increased communication with other nations looking to incorporate linguistics into the
classroom, and use of IOL contacts to develop a potential network of groups aiming to
promote linguistics in schools (see section 2).
Development of links with AILO (the All Ireland Linguistics Olympiad), and possible
development of joint IOL training schemes and workshops (see sections 2 and 4.2.7).
Investigation of either introducing a lower-level introductory paper, aimed at younger
students and first-time competitors, or promoting the Foundation/Intermediate papers as
ideal for first-time competitors, and inviting high-performing students to enter a higher level
paper the following year (see section 2).
Investigation into the current links between academic Olympiads in the UK and their links to
the government, and development of links between UKLO and the Department for
Education and other relevant government departments (see sections 2 and 4.2.7).
Development of online resources to include online training resources, incorporating past
paper materials for students to use in preparation for the competition (see section 2).
Increasing the number of moderators for the Round One, to reduce the risk of errors in the
paper, and to ensure no one committee member is left with sole responsibility for
moderation (see section 4.2.1).
Pre-testing of questions prior to inclusion in the paper, and especially where alternatives are
provided, to ensure difficulty level and time required for completion are accurate.
Inclusion of an easier ‘starter’ question on the Round One paper (see section 4.2.1).
Monitoring of student performance over future years to ensure continued suitability for a
broader age range of entrants (see section 4.2.1).
Continued monitoring of performance, and use of teacher and student feedback to ensure
necessary support is provided to maintained schools.
Where issues with timetabling, paper length/structure and competing academic demands
arise, these problems are difficult to overcome. Establishing a post-competition feedback
form for teachers and students may highlight particular areas where improvements can be
made (see section 1.1 for considerations when planning the timing of Round One. See
section 4.2.3 for further discussion of addressing issues of timetabling, paper structure, and
feedback forms).
Revision of the Round One mark scheme, in order to ensure it is accessible for all teachers.
Use of the planned UKLO portal to speed up the marking process and ensure marks are
returned to schools, ideally before the end of the summer term (see section 4.2.4 and 4.2.5).
Introduction of a medal scheme for the Foundation and Intermediate papers, in line with
that used for the Advanced paper (see section 4.2.4)
Use of the planned UKLO portal to allow teachers to amend estimated numbers of Advanced
entries (see section 4.2.4).
Sending a clear outline of the competition structure in preliminary emails, to address issues
where some schools were unclear on the competition’s structure (see section 4.2.4).
Maintain links to UK institutions through Markathons and contact with previous competitors
progressing to higher education.
Maintain links to current supporters to ensure continued support (see sections 1.5.3 and
4.2.7).
37
Works Cited
Clark, Ann and John Trafford. 1995. Boys into modern languages: An investigation of the discrepancy
in attitudes and performance between boys and girls in modern languages. Gender and Education 7:
15-26
Davies, Beatrice. 2004. ‘The gender-gap in modern languages: a comparison of attitude and
performance in year 7 and year 10’. The Language Learning Journal 29: 53-8
Feng, Annie X., James R. Campbell and Marilyn A. Verna. 2002. ‘Understanding gender inequity in
America: Interviews with academic Olympians.’ Journal of Research in Education 12: 93-100
Fisher, Linda. 2001. ‘Modern foreign languages recruitment post-16: the pupils’ perspective’. The
Language Learning Journal 23: 33-40
Hawkes, R., Rogers, V., Evans, J., Driver, J. & Schechter, S. 2012. ‘Giving language the X-factor:
Competitions and creative initiatives to support language learning’. Paper presented at Languages
for All – Defining Today, Transforming Tomorrow! Language World Conference 2012. University of
Manchester, March 2012
Keus, A. & Corr, A. 2012. ‘Sprachwissenschaft macht Schule! Denn wer den Lehrer nicht ehrt’, Der
Linguistische SprAACHENblog [blog] Available at:
http://spraachenblog.wordpress.com/2012/07/16/sprachwissenschaft-macht-schule/ [Accessed:
August 2012]
Lengfelder, A. and K. Heller. 2001. ‘German Olympiad studies: Findings from in-depth
interviews’. Paper presented at the Annual Meeting of the American Educational Research
Association
Meinunger, A. 2012. ‘Die Internationale Linguistik-Olympiade und was sie (in) Deutschland wert ist’,
Der Linguistische SprAACHENblog [blog] Available at:
http://spraachenblog.wordpress.com/2012/07/05/die-internationale-linguistik-olympiade-und-wassie-in-deutschland-wert-ist/ [Accessed: August 2012]
Pajasallu, R. 2005. ‘Keeleteadus Olumpia-Alana’. Oma Keel 11: 5-12
von Schantz, H., Roos, J. P., Skirgård, H., Engwall, G., Bergman, B., Barting, I., Hartgvigsson, T., Wirén,
M., Wardini, E., Linder, K. & Rask, E. 2012. ‘Varför inget stöd till internationella språktävlingar?’
Svenska Dagbladet [online] Available: http://www.svd.se/opinion/brannpunkt/varfor-inget-stod-tillinternationella-spraktavlingar_7339724.svd#article-comments [Accessed: August 2012]
Stables, Andrew and Felicity Wikeley. 1999. ‘From bad to worse? Pupils’ attitudes to modern foreign
languages at ages 14 and 15’. The Language Learning Journal 20: 27-31
Tirri, Kirsi. 2001. ‘Actualizing talent in science: Case studies of Finnish Olympians’. Paper presented at
Annual Meeting of the American Educational Research Association. Seattle, WA, April 2001.
38