Are lecturers' and students'
needs different? A needs
analysis for reading tasks in
Flemish higher education
Elke Peters & Tine Van Houtven
Lessius University College, Antwerp
elke.peters@lessius.eu & tine.vanhoutven@lessius.eu
Outline
•
•
•
•
•
•
Project description
Background
Aim and research questions
Methodology
Results and interpretation
Conclusion
TBLT 2009
2
Introduction
• Language plays a key role in education.
• Mastery of academic language is crucial.
– But research has shown that many students,
non-native as well as native speakers of Dutch,
struggle with academic language upon entering
Flemish university colleges.
– poor command of Dutch and of academic Dutch
in particular
 Projects centering around the theme of (L1)
language support 
 Project focusing on text competence/reading skills
TBLT 2009
3
Project
• Aim of our project is to provide an answer to this
problem by
• Determining required level of text competence
• Carrying out a descriptive study into first year students’
reading skills and text competence
• Comparing students’ existent level of reading/text
competence with the required level
• developing reading materials for four courses in four
different curricula
• In order to facilitate first-year students’ chances of
achieving academic success
TBLT 2009
4
Project
• How?
– Not “one-size-fits-all-approach”
– Necessity of a large scale needs analysis in four
different curricula.
• “the language learning needs of particular groups of
learners or individuals (…) are learner- or groupspecific, (…) are tied to local contexts and may change
over time” (Van Avermaet & Gysen, 2006: 19)
• What?
– NA findings used in design and development of
task-based reading support materials.
TBLT 2009
5
Background: Long (2005)
• Long (2005)
– a number of methodological issues that need to
be considered in learner needs analysis in terms
of sources, methods, and source x method
combinations.
– The aim should be to obtain reliable, valid, and
usable data about the tasks students need to
carry out to be successful.
• Van Avermaet & Gysen (2006)
– Take into account both subjective and objective
needs.
TBLT 2009
6
Background: Long (2005)
• A task-based needs analysis
• Possible sources for a needs analysis:
– Literature, learners, teachers/applied linguists, domain
experts, and triangulation.
– Needs analysis should involve insiders/domain experts
– Use of multiple sources: add breadth/depth to the analysis
• Possible methods for a needs analysis:
– intuitions, (un)structured interviews, questionnaires,
observation, tests, diaries, role plays etc. .
– Unstructured interviews.
– Questionnaires: ascertain existing views, not creating new
views; often over-rated.
– Use of multiple methods of data collection
• A needs analysis = time-consuming
TBLT 2009
7
Aim and research questions
• Which reading tasks do we need to develop for first
year students from four different curricula?
– First-year students of four different curricula & university
colleges  clearly-defined domain = academic language
proficiency
• What is the required level of text competence?
• What is the actual of first year students’ text
competence?
• Is there a difference between the two?
– Practical RQ in order to develop the reading materials
TBLT 2009
8
Aim and research questions
• Which source(s) or method(s) or source x methodcombinations is/are the most reliable and
informative?
– As compared in four case studies (four different curricula
and university colleges)
– Methodological/evaluative RQ in order to +/- corroborate
Long’s hypotheses
TBLT 2009
9
Methodology
•
•
•
•
Four sources
Four methods
Triangulation of sources and methods
Same methodology in four case studies
TBLT 2009
10
Sources
• PTHO (= Profiel Taalvaardigheid Hoger Onderwijs
(Language Proficiency Higher Education))
– Description of tasks students need to be able to
carry out at the start of their academic career
• Determining expected level of text competence
• Students from 4 different curricula:
– First-year students
– Third-year students
– Convenient and purposive sample
• Lecturers from 4 different curricula (= domain
experts)
• Language experts  methodological advice
TBLT 2009
11
Methods
•
•
•
•
•
Literature survey
Reading test
Questionnaire
Interview
Triangulation by sources & methods
TBLT 2009
12
Method 1: Reading test PTHO
• Target group: Dutch as a foreign language
• Based on needs analysis  typical tasks a student
needs to be able to carry out
• N = 176 (L1 Dutch = 165; L2 Dutch = 9)
• Part 1: multiple choice questions
– Questions = “reading-the-lines” level (Alderson, 2000) or
descriptive level (Bogaert et al., 2008)
• Part 2: summary
– Read three texts on same topic
– Write one summary = “reading-between-the lines” level
(Alderson) or upper-textual level (Bogaert et al.)
TBLT 2009
13
Method 1: Reading test PTHO: results
• Part 1 (multiple choice questions): high scores
– Ceiling effect
• Part 2 (summary)
– 1/3 of students = problematic
• Difficulty with information processing  functional reading
• Wrong/incomplete account of information
– Large differences in terms of educational program in
secundary education/preparatory training
• General > technical > vocational secundary education
– Problem areas were identified: vocabulary, text cohesion
and synthesis
• Answer to RQ1 in terms of problem areas for each
curriculum
TBLT 2009
14
Method 2: Questionnaire
• Questionnaire tapped into
– Types of reading texts
– Strategy use
• Orientation and planning (e.g. reading title/images/…)
• Monitoring reading process (e.g. looking up unknown
words)
• Evaluating reading process (e.g. how difficult do you
find … linked to activities of different levels of
information processing)
– Possible, useful reading tasks
• Closed questions with pre-specified response
categories + 1 open question
• Questionnaire was piloted
TBLT 2009
15
Method 2: Questionnaire - example
Arrange in order of difficulty.
- Visualize the structure (e.g. highlighting,
annotating)
- Detect the topic sentence in a section
- Interpret charts and diagrams
- Attain a high level of comprehension
- Make comparisons and connections
- Represent information schematically
* setting: course or handbook
TBLT 2009
16
Method 2: Questionnaire - results
• Sources/participants:
– Students: N = 455  what do you think/do?
– Lecturers: N = 97  what do students do/think?
• Tasks with increasing text competence
• Reading tasks were perceived more difficult by
lecturers compared to the students
• Answer to RQ1 in terms of problem areas,
students’ strategy use, and useful tasks for each
curriculum.
TBLT 2009
17
Method 3: Interview
• Semi-structured interview
– Partially based on results reading test
– Partially based on data of questionnaire
• One-hour audio-taped interviews with
– Lecturers in four different curricula
– Students in four different curricula
• 1st and 3rd year students in one curriculum
• Nine interviews in total
• All interviews were transcribed
TBLT 2009
18
Method 3: Interview - results
Information obtained about …
• Target reading tasks and implementation methods
were identified
• Students modified students’ answers supplied in
questionnaires  more in line with lecturers’
opinions
• Students contribute to ‘means analysis’ (they
provide useful information on learning styles, likes
and dislikes, etc.)
TBLT 2009
19
Discussion
Four case studies
Reading test  identify problem areas
Questionnaire  identify problem areas,
students’ strategy use, useful
tasks
Interview
 indentify target reading
tasks and implementation
methods
Design support materials
TBLT 2009
20
Discussion: RQ1
• Differences in target reading tasks and
implementation methods between the four
curricula  needs vary greatly
–
–
–
–
one-size-fits-all approach doesn’t work
NA = prerequisite for effective design of support materials
taking into account specificities of each course and curriculum
beneficial for both students’ and lecturers’ motivation + gain
an insight into their attitudes (what they think and do) 
self-knowledge ; level of awareness 
• combining and balancing needs of students,
lecturers and language experts
– Students tended to overestimate themselves in the
questionnaires but counterbalanced in the interviews
TBLT 2009
21
Discussion: RQ2
• Evidence of four case studies
– Use of several methods and sources  obtain
more reliable data
– Sources: triangulation of sources
• Lecturers (domain experts)
– Methods: triangulation of methods
• Interviews  semi-structured interview
– BUT only because of the results of the reading test and
questionnaire
– Interview alone would not have sufficed
• Our results tend to corroborate Long’s findings but
with regard to the method there is an “if”.
TBLT 2009
22
Discussion: RQ2
Methods
+
-
reading test
PTHO
- Language Proficiency
Higher Education 
relevant tasks
- small sample
- unfeasible to test all
relevant tasks
questionnaires
- large sample
- normalized, quantifiable
data
issues of validity:
- overestimation of oneself
- pre-specified anwering
categories  limit variety of
responses
- interpretation differences
between lecturers and
students
semi-structured - thorough coverage of
interviews
the matter
- target reading tasks and
methodologies were
identified
TBLT 2009
-time-consuming
23
Conclusion
NA
• time-consuming undertaking, but prerequisite for design of support
materials
• multiple sources and methods  should be carefully sequenced
Sources
• insiders/domain experts: informative source
• 1st year students: can’t be the sole or principal source because they
lack experience and understanding of present/future needs
Methods
• interviews  yield important information, but only because results
of test and questionnaires could be used
• questionnaires  effective for ascertaining existing beliefs, not for
creating new views
TBLT 2009
24
References
• Alderson, J.C. (2000). Assessing Reading. Cambridge: Cambridge
University Press
• Bogaert, N., Devlieghere, J., Hacquebord, H., Rijkers, J.,
Timmermans, S. & Verhallen, M. (2008). Aan het werk! Adviezen ter
verbetering van functionele leesvaardigheid in het onderwijs.
Den Haag: Nederlandse Taalunie Den
• Long, M. (Ed.) (2005). Second Language Needs Analysis.
Cambridge: Cambridge University Press
• Profiel Taalvaardigheid Hoger Onderwijs van het Certificaat
Nederlands als Vreemde Taal. Downloaded from
http://www.cnavt.org/files/Profielbeschrijving%20Profiel%20Taalvaa
rdigheid%20Hoger%20Onderwijs.pdf on September 5 2008
• Van Avermaet, P. & Gysen, S. (2006). From needs to tasks:
Language learning needs in a task-based approach. In K. Van den
Branden (Ed.), Task-Based Language Education (pp.17-46).
Cambridge: Cambridge University Press
TBLT 2009
25
Acknowledgements
• OOF-comittee of the Association K.U.Leuven
• Projectpartners
– Katholieke Hogeschool Limburg, Katholieke Hogeschool
Kempen, Katholieke Universiteit Leuven, Katholieke
Hogeschool Leuven, GroepT, KATHO, Katholieke
Hogeschool Mechelen,Katholieke Hogeschool BruggeOostende, Katholieke Hogeschool Sint-Lieven, Hogeschool
Universiteit Brussel
• If you have any questions, you can always send us
an e-mail:
– Elke.peters@lessius.eu
– Tine.vanhoutven@lessius.eu
TBLT 2009
26