THROUGH LANGUAGE
TO LITERACY
A Report on the Literacy Gains of
Low-level and Pre-literate
Adult ESOL Learners in Literacy Classes
A Collaborative Project
National Association of ESOL Home Tutor Schemes Inc
School of English and Applied Linguistics UNITEC Institute of Technology
Centre for Refugee Education Auckland University of Technology
THROUGH LANGUAGE
TO LITERACY
A Report on the Literacy Gains of
Low-level and Pre-literate
Adult ESOL Learners in Literacy Classes
Nikhat Shameem
Keryn McDermott
Jeannie Martin Blaker
Jenny Carryer
UNITEC Institute of Technology
Auckland University of Technology
UNITEC Institute of Technology
ESOL Home Tutor Service
UNITEC Institute of Technology
March 2002
2
TABLE OF CONTENTS
TABLE OF CONTENTS __________________________________________________ 3
LIST OF TABLES ________________________________________________________ 7
LIST OF FIGURES _______________________________________________________ 8
LIST OF APPENDICES ___________________________________________________ 9
Abstract _______________________________________________________________ 11
Abbreviations used in this report___________________________________________ 13
BACKGROUND TO A STUDY OF LOW-LEVEL AND PRE-LITERATE
LEARNERS ____________________________________________________________ 15
Introduction __________________________________________________________ 15
Literacy______________________________________________________________ 16
Defining literacy _____________________________________________________ 16
Literacy and ESOL ___________________________________________________ 18
Low- level, ESOL literacy ______________________________________________ 19
Low- level, ESOL, language and literacy learners ____________________________ 19
Research methodologies ________________________________________________ 22
Cross-cultural research ________________________________________________ 22
Case studies _________________________________________________________ 23
Interviews __________________________________________________________ 24
Observations ________________________________________________________ 25
Assessment and evaluation ______________________________________________ 25
Assessment procedures and tools ________________________________________ 27
Competency-based assessment ________________________________________ 27
Self-assessment in language learning ___________________________________ 28
Self-assessment measures __________________________________________ 29
Performance assessment _____________________________________________ 30
Reliability and validity in performance assessment. ______________________ 31
Factors affecting literacy gains ___________________________________________ 34
Literacy in Aotearoa/New Zealand _______________________________________ 36
Summary: Literature review ____________________________________________ 36
BACKGROUND TO THE PROJECT ______________________________________ 38
Introduction __________________________________________________________ 39
Background and rationale of programme __________________________________ 39
Refugees in West and Central Auckland___________________________________ 40
Language learning issues _______________________________________________ 41
Bilingual tutor training project __________________________________________ 42
The current situation in Auckland________________________________________ 44
3
METHODOLOGY ______________________________________________________ 47
Introduction __________________________________________________________ 47
Research questions ____________________________________________________ 48
Part A: Personal profiles _____________________________________________ 48
Tutors____________________________________________________________ 48
Students __________________________________________________________ 48
Part B: Self-assessment ______________________________________________ 49
Part C: Performance testing ___________________________________________ 49
Hypotheses ___________________________________________________________ 50
Time span of project ___________________________________________________ 50
Research participants __________________________________________________ 51
Research participants: Case studies ______________________________________ 53
Research methods _____________________________________________________ 54
Personal profiles: Tutors (See Appendix 2) ________________________________ 54
Personal profiles: Students (See Appendix 3)_______________________________ 54
Self-assessment (See Appendix 4) _______________________________________ 55
Performance assessment (See Appendix 14)________________________________ 58
Case studies (See Appendix 11) _________________________________________ 59
Test trial _____________________________________________________________ 61
Trial of personal profiles and self-assessment _______________________________ 61
Changes made _____________________________________________________ 61
Trial of performance assessments ________________________________________ 61
Changes made _____________________________________________________ 61
Ethics concerns _______________________________________________________ 62
Information sheets (See Appendix 12) ____________________________________ 63
Consent forms (See Appendix 13) _______________________________________ 63
Issues arising that imposed limitations on the project ________________________ 64
Continuity of students _________________________________________________ 64
Availability of bilingual tutors __________________________________________ 64
Time of day constraints ________________________________________________ 64
Ramadhan __________________________________________________________ 64
Cultural and social resistance to self-assessment and testing ___________________ 65
Data coding and analysis________________________________________________ 65
Summary: Methodology ________________________________________________ 66
RESULTS ______________________________________________________________ 70
TUTOR AND STUDENT PROFILES ______________________________________ 70
Introduction __________________________________________________________ 70
Section One: Tutor Profiles: Background__________________________________ 70
4
Language use and proficiency ___________________________________________ 72
Educational background ________________________________________________ 77
Issues arising from teaching current literacy classes _________________________ 79
Summary: Tutor profiles _______________________________________________ 81
Section Two: Student profiles: Background ________________________________ 83
Countries of origin and gender __________________________________________ 83
Age, age at arrival and length of residence _________________________________ 84
Change in socio-economic status _________________________________________ 86
Educational background ________________________________________________ 88
Language use in NZ____________________________________________________ 90
Mother tongue and first language ________________________________________ 90
Home language use ____________________________________________________ 91
English study in NZ____________________________________________________ 94
Bilingual instruction ___________________________________________________ 94
Participant perception of host community attitude towards their mother tongue _ 95
Reasons for learning English in NZ_______________________________________ 95
Language proficiency __________________________________________________ 96
English Proficiency ___________________________________________________ 97
Variables affecting English literacy skills __________________________________ 97
Summary: Student profiles_____________________________________________ 101
REPORTED LITERACY________________________________________________ 104
Introduction _________________________________________________________ 104
Total sample self-assessments: Reading __________________________________ 104
Total sample self-assessments: Writing ___________________________________ 106
Student perceptions of literacy gains: Reading ____________________________ 107
Student perceptions of literacy gains: Writing _____________________________ 109
Student perceptions of literacy gains: Bilingual and English-speaking tutors ___ 110
Student perceptions of literacy gains: 2-hour and 12-hour weekly classes ______ 112
Summary: Student perceptions of literacy gains ___________________________ 115
ASSESSED LITERACY GAINS AND COMPARISON WITH SELF-ASSESSED
LITERACY ___________________________________________________________ 118
Introduction _________________________________________________________ 118
Total sample test results: Reading _______________________________________ 118
Total sample test re sults: Writing _______________________________________ 120
5
Literacy gains: Reading _______________________________________________ 121
Literacy gains: Writing ________________________________________________ 122
Literacy gains: Bilingual and English-speaking tutors ______________________ 123
Literacy gains: 2-hour and 12-hour weekly classes _________________________ 126
A comparison – reported and assessed literacy ____________________________ 128
Summary: Assessed and reported literacy gains ___________________________ 131
CASE STUDIES _______________________________________________________ 133
Introduction _________________________________________________________ 133
Specific case studies: Research findings __________________________________ 135
Student interviews ___________________________________________________ 135
Tutor interviews_____________________________________________________ 140
Classroom observations _______________________________________________ 144
Summary: Case studies________________________________________________ 146
DISCUSSION AND RECOMMENDATIONS _______________________________ 150
Recommendations derived from this report ______________________________ 153
General recommendations _____________________________________________ 155
Recommendations for further research __________________________________ 155
Conclusions _________________________________________________________ 156
REFERENCES ________________________________________________________ 158
APPENDICES _________________________________________________________ 168
6
LIST OF TABLES
Table 1
Table 2
Table 3
Table 4
Table 5
Table 6
Table 7
Table 8
Table 9
Table 10
Table 11
Table 12
Table 13
Table 14
Table 15
Table 16
Table 17
Table 18
Table 19
Table 20
Table 21
Table 22
Table 23
Table 24
Table 25
Table 26
Table 27
Table 28
Table 29
Table 30
Table 31
Table 32
Table 33
Table 34
Table 35
Table 36
Table 37
Table 38
Table 39
Table 40
Refugees in Central and West Auckland ___________________________ 40
Profile of literacy students participating in project____________________ 45
Research participants __________________________________________ 52
English ability: Reading scale ___________________________________ 56
English ability: Writing scale ____________________________________ 57
Tutor profiles ________________________________________________ 71
Tutor age and length of residence in NZ ___________________________ 73
Home language use ____________________________________________ 73
Language proficiency in languages known by tutors on literacy project ___ 75
Language proficiency __________________________________________ 76
Educational level of tutors ______________________________________ 78
Qualifications of tutors _________________________________________ 78
Teaching experience of tutors on literacy project_____________________ 79
Tutor issues in current literacy classes _____________________________ 80
Countries of origin of research participants _________________________ 84
Age of research participants _____________________________________ 84
Age at time of arrival __________________________________________ 85
Length of residence in NZ of participants __________________________ 85
Duration of schooling in home country ____________________________ 89
Duration of English study in home country _________________________ 90
Mother tongues and first language identified by participants ___________ 91
Home language use by key variables ______________________________ 92
Gender and home language use __________________________________ 92
Country of origin and home language _____________________________ 93
Mother tongue and home language use ____________________________ 94
English literacy and key variables ________________________________ 98
Differences in first language and English literacy ____________________ 99
Self-assessed proficiency: Reading _______________________________ 105
Self-assessed proficiency: Writing ________________________________ 107
Self-assessed reading ability: Time 1 and 3 _________________________ 108
Self-assessed writing ability: Time 1 and 3 _________________________ 109
Student perception of literacy gains: Bilingual and English-speaking tutors 111
Student perception of literacy gains: 2-hour and 12-hour weekly classes __ 114
Performance assessment: Reading ________________________________ 119
Performance assessment: Writing_________________________________ 121
Performance assessments 1 and 2: Reading _________________________ 122
Performance assessments 1 and 2: Writing _________________________ 123
Literacy gains in performance tests: Bilingual and English-speaking tutors 124
Literacy gains in performance tests: 2- hour and 12- hour weekly classes __ 126
Reported and assessed literacy: Pre- and post-programme _____________ 130
7
LIST OF FIGURES
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Timetable for research 2001 – 2002 _______________________________ 51
Classes established during research project _________________________ 52
Mean proficiency levels for tutor languages ________________________ 76
Tutor English proficienc y _______________________________________ 77
Change in socio-economic status _________________________________ 87
Reasons for learning English in NZ _______________________________ 96
Student English proficiency (four skills) ___________________________ 97
Student English and first language proficiency ______________________ 99
Student first language proficiency ________________________________ 100
Student perception of literacy gains: Bilingual and English-speaking tutors 112
Student perception of literacy gains: 2-hour and 12-hour weekly classes __ 113
Student performance gains: Bilingual and English-speaking tutors _______ 125
Student performance gains: 2-hour and 12- hour clasess _______________ 127
8
LIST OF APPENDICES
Appendix 1 Course Document _____________________________________________ 168
Appendix 2 Personal Information Profile for Literacy Tutors _____________________ 170
Appendix 3 Personal Information Profile for Students __________________________ 173
Appendix 4 Self-assessment Schedule _______________________________________ 173
Appendix 5 Literacy Curriculum ___________________________________________ 178
Appendix 6 Test Specifications ____________________________________________ 180
Appendix 7 English Literacy Test Descriptors and Score Sheet ___________________ 182
Appendix 8 Test Syllabus_________________________________________________ 182
Appendix 9 Impressions from an English-Speaking Tutor _______________________ 189
Appendix 10 Recording Sheet ______________________________________________ 191
Appendix 11 Case Study Instruments ________________________________________ 191
Appendix 12 Information Sheets ____________________________________________ 193
Appendix 13 Consent Form________________________________________________ 197
Appendix 14 Literacy Tests: Accompanying Document _________________________ 201
9
Acknowledgements
Mahadsanid
§
The 8 tutors and 118 students at the three different sites who participated in the
research project.
§
The Ministry of Education who funded the project as part of its Adult Literacy
Strategy, 2001 and the National Association of ESOL Home Tutor Schemes Inc. for
contracting this project.
§
Judi Altinkaya, CEO – National Association of ESOL Home Tutor Schemes Inc.
whose vision for a collaborative research approach has been realised in this report.
§
Lois Bellingham, Head of School of English and Applied Linguistics, UNITEC
for her proactive support throughout the implementation of the research.
§
Julia Castles of South Auckland Home Tutor Scheme, for help with the selfassessment schedule.
§
Sarah Hardman of UNITEC, for contribution to the section on 'Background to the
Project' and for the photographs.
§
Jainesh Sharma, research assistant, for many and odd hours of work.
§
Colleagues and friends in the School of English and Applied Linguistics at
UNITEC, for listening, support, encouragement and resources.
§
Trish Burns of UNITEC, for proof-reading and editorial assistance.
Research team
March 2002
10
Abstract
The primary aim of this project was to determine literacy gains for low- level and
pre-literate ESOL students on a one-semester literacy programme. The project was
undertaken collaboratively by the National Association of ESOL Home Tutor Schemes
Inc., the Auckland University of Technology and the School of English and Applied
Linguistics, UNITEC. A majority of the student participants were refugees and the
remainder new immigrants. The students were from seven different source countries. One
hundred and eighteen students participated in Semester 2, 2001. Sixty-two students
completed most of the tasks and tests associated with the project. The research involved the
use of tutor and student profiles, use of self-assessments, the design and administration of
parallel performance tests, the use of case studies, classroom observations and personal
interviews.
The results of the study found that the students had made significant gains. Gains
were measured through the use of test descriptors which represented eight levels of literacy
ability as defined by the programme curriculum. This scale was used for both selfassessment and performance tests. The gains were clearer on the tests than on the selfassessments, signalling that testing, even if students from this background are unused to it,
is a worthwhile exercise. Tests, however, need to allow for some student collaboratio n. It is
felt that by allowing learners to work in groups of similar ability anxiety would be
significantly reduced.
Variances in test results between the classes taught by native speakers of English
and those taught by bilingual tutors were not significant, thus indicating that both types of
instruction are of equal value at this level. Two thirds of the students at the beginning of the
programme indicated that they preferred to be taught by a bilingual tutor. Therefore it is
clearly important, especially in the early stages of a course for students to have access to a
tutor who understands their language and background.
Results also indicated the significant benefits of the 12- hour weekly classes over the
2-hour ones. However, even those students in the 2-hour weekly classes had made
11
significant progress over the 20-week programme. Clearly, any instruction is better than
none, and the more intensive a programme, the greater the potential gains.
12
Abbreviations used in this report
EAL
-
English as an additional language
ESL
-
English as a second language
ESOL -
English for speakers of other languages
L1
-
First language
L2
-
Second language
NESB -
Non-English speaking background
NZ
Aotearoa/New Zealand
-
13
14
BACKGROUND TO A STUDY OF LOW-LEVEL AND PRELITERATE LEARNERS
Introduction
This research is placed in two large and complex fields of applied linguistics. These
are the fields of Literacy and Assessment. In this study various assessment procedures
were used to determine literacy gains among low- level literacy (ESOL) students.
The first field, literacy, involves coming to terms with the value of literacy to
humans and the many difficulties related to arriving at an appropriate definition of that
word. For the purpose of this study, literacy must be placed specifically in the context of
ESOL (English for Speakers of other Languages) at low levels of proficiency, where
literacy in any language may not be present. The terms ESOL, (used in New Zealand), ESL
(English as a Second Language, used in Australia and USA) and NESB (non-English
speaking background) will all be used, at times, to mean people for whom English is an
additional language (EAL).
In the second field, assessment, it is necessary to look at what assessment aims to
achieve and for whom. Major tensions exist because of two linked yet distinct requirements
of assessment: the first requirement is that it is part of the learning process, with feedback
primarily to learners and teachers; and the second, that it supplies the means to evaluate a
programme, with feedback primarily to an institution or funding body. The first focuses on
individual learner gains and the second, on the numbers of people reaching pre-determined
levels.
We need then to focus on assessment in the area of low- level ESOL literacy and to
distinguish these issues from the broader concerns of adult literacy that currently occupy
governments of developed countries.
Finally, the research needs to be placed in the New Zealand (NZ) context of ESOL
language and literacy provision for low- level, adult learners, specifically addressing the
15
range of assessment measures already in place, as well as current needs and future
directions.
Literacy
The value placed on literacy by developed nations is very high. Literacy not only
impacts on a nation’s economy through educational achievement and employment
possibilities, but is also a valuable resource for the individual (Johnson, 2000).
Defining literacy
The precise meaning of literacy is not clear from the various definitions that have
surfaced in the last two decades. Grant (1986:1), in her attempt to define literacy, begins her
paper with a quote from Alice in Wonderland “When I use a word…it means just what I
choose it to mean, neither more nor less.”
The once simple distinction that literacy is the ability to read and write no longer
holds true. There is now a large degree of agreement that literacy is not an absolute or a
single standard of competence to be reached; it is not a collection of functional skills that
are context- free and unified; and it cannot guarantee economic benefits. While definitions
still include the basic skills of reading and writing at the core, they also include the ability
to use these skills creatively and appropriately, as a means of empowering people to fully
participate in society. The many definitions regard people as being on a life- long continuum
of proficiency.
Wickerts (1990:180) believes that “literacy is relative. The concept is socially
constructed… The question of definition and measurement must reflect this relativity and
also give some indication of performance across the population on a range of literacy tasks
at varying levels of difficulty.” Wickerts also points out that literacy is political and
findings about literacy will be put to political purposes.
Two English-speaking countries where the literacy issue has been researched and
developed are the United States of America and Australia. The 1989 Australian National
Survey of Adult Literacy in English No Single Measure, looked at three types of literacy
16
proficiency: document literacy, prose literacy and quantitative literacy. It recognised that
different types of literacy exist and that they are acquired in different ways and for different
purposes. Reports by the Organisation for Economic Co-operation and Development
(OECD) examined similar areas of literacy and revealed significant literacy problems for
the 22 industrialised nations of the western world. Importantly, they note that a low level of
literacy is not limited to migrants and refugees or any other marginalised group (Johnson,
2000).
In the US, the main approaches to literacy are humanistic and writing-process
oriented (Murray, 1998). These approaches are also discussed by Huerta Macias (1993)
who gives definitions of three key concepts in adult literacy that are complementary and
share basic philosophies. These are, that “the learner should inform literacy instruction, that
learners and their background knowledge and experiences should be respected and valued
and that learning activities should be relevant to learner’s personal situations” (Huerta
Macias, 1993:4). The approaches used in the US are:
q
Whole Language Approach involves teaching language in real and natural contexts
and building on existing knowledge and experiences that are interesting and relevant
to the learner. It is not a specific method or strategy as much as a perspective on
language learning and teaching (Edelsky, et al. 1991 in Peyton & Crandall, 1995).
q
Learner Centred Approach involves learners in both the content of the course and
the anticipated outcomes. It works closely with the whole language approach, and
extends it by emphasising that language learning is a collaborative effort between
teacher and learner.
q
The Participatory Approach is an approach that was made popular through the work
of Paulo Freire, “Education and knowledge have value only insofar as they help
people liberate themselves from the social conditions that oppress them and achieve
personal and social change” (Peyton & Crandall, 1995:1). It is closely linked to the
following theory.
q
Critical Literacy Theory goes beyond basic functions of reading and writing,
decoding, predicting and summarising, to a critical examination of text, focusing
also on its social, political and ideological elements.
This approach holds that
practices have the capability to both reflect and shape issues and power relations in
society. Literacy seeks to empower learners and is concerned with reading the
17
ideological message within texts, and consciously accepting (or reflecting) the
message (Murray, 1998).
So far, the discussion has focused on adult literacy, referring to literacy issues of
first language speakers. Although many of the issues are equally pertinent to ESOL literacy,
the range of techniques used, and the use of the word literacy to mean acquiring basic skills
in reading and writing in English as an additional language can further confuse the issue.
We now look at reasons why it is important to identify the difference and keep it to the fore
when discussing ESOL literacy and particularly when the focus is on the very first stages of
acquisition, as it is in this study.
Literacy and ESOL
Hammond and Derewianka outline the possibility that a new focus in Australia on
literacy for all could pose a threat to the special provision needed for ESL literacy. They
conclude that, “While there are important areas of overlap between ESL and literacy
education, there are also important differences…and that these diverse needs cannot be met
by the same programmes” (Hammond and Derewianka, 1999:1). They suggest that adult
literacy usually assumes an oral basis of the language. Teaching approaches are different
when oral language proficiency is being acquired at the same time as literacy.
In Australia, in the Adult Migrant English Programme (AMEP), approaches similar
to those used in the US are combined with a unified linguistic approach, based on systemicfunctional linguistics developed by Halliday.
This theory of language provided a coherent model of language that relates language to the cultural
and social contexts in which it is used. They are divided into knowledge and learning; oral; reading
and writing, with each being des cribed in terms of elements, performance criteria, range of variables
and sample texts and assessment tasks. They incorporate learning gains other than language; relate
language to the cultural and social contexts of its use; are based on whole spoken and written texts
and describe language development in terms of learners progressively accessing increasingly
demanding contexts of language use.
Hagan, 1994:36-37
18
Low-level, ESOL literacy
As previously stated, the term ‘literacy’ has, in the past, often been misunderstood
and considered to be a simpler set of skills than is the case. In addition, as Grant (1986)
writes, low- level literacy is often confused with illiteracy (nominal or non-existent reading
and writing skills) a problem that affects only a small proportion of adults in industrialised
societies, where opportunities for literacy instruction exist in the environment, even if they
are not accessed. In addition, we also have a large number of people who are pre-literate –
they either use a language which comes from an aural/oral tradition, or they have never
acquired literacy skills because of political or social upheaval.
It should be noted that at the very lowest levels of literacy, the view that literacy can
be taught as discrete, non-contextual skills has been replaced by a view that even the
teaching of the first steps of letter recognition and formation be conducted within a relevant
context such as learning to read and write personal details rather than learning the alphabet.
There is a strong emphasis on communication and meaningful context (Wrigley, 1993).
This current research focuses on the very beginning of the literacy spectrum where the line
between pre- literacy (nominal or non-existent reading and writing skills) and literacy can be
drawn.
Low-level, ESOL, language and literacy learners
Learning English presents problems for those immigrants and refugees who have
little oral proficiency in that language, who are not literate in any language and have little
experience of formal learning. Hood (1990) discusses the need to develop appropriate
approaches and reiterates that it is possible to learn English in the formal context of a
classroom.
The relationship between oral and written language is a crucial factor in approache s
to second language literacy. Hammond (1989) writes about the Australian National
Languages and Literacy Institute's literacy project 1987-88, which identified key issues in
the relationship between spoken and written language, the role of the teacher and teaching
methodology.
19
A key question for educators is whether it is more effective for learners to first
acquire literacy in their first language, before attempting to transfer those skills to a second
language. On the efficacy of teaching literacy in L1 first, the jury is still out. Studies such as
Kalantzis (1987) concluded that there was no evidence that second language literacy cannot
be acquired without the development of first language literacy, nor that prior development
of first language is a necessary or an efficient way for adults to acquire literacy in a second
language. In a previous study, Kalantzis suggested “it is not so much that students are
unable to achieve greater literacy skills in the second language than the first. It is simply
that they do not always see the need to do so and so are unmotivated to do so” (Kalantzis et
al., 1986, in Hood, 1990:5). In contrast, Roberts (1994) stresses the value of developing
first language literacy and Rivera (1999) gives a great deal of evidence tha t developing
literacy in the first language plays a positive role in literacy acquisition of a second
language. She looks at models where literacy is concurrent (bilingual teaching of both), coordinate (separate instruction of both literacies in the same time period), and sequential
(leaving ESOL until a certain level has been reached in the first language).
The use of the word ‘bilingual’ has often been used in ESOL and literacy studies to
denote proficiency in more than one language. The level of relative proficiency can range
from minimal in one of the languages, to a situation that produces a ‘balanced’ bilingual
ability in both languages so the individual may choose and use the most appropriate
language for a particular purpose and context.
In a bilingual teaching situation where English is being acquired as a second
language, it could mean teaching all subjects in the students’ first language until their
English is at a sufficient level to become the teaching medium. This definition is the focus
of current controversy in North America. In the US, the use of Spanish to teach English to
students whose first language is Spanish is being strongly questioned and opposed.
Research studies on bilingual education have a more positive view of the value of
bilingual assistance or teaching where it is available. An interpretation of bilingual teaching
is outlined in O’Grady (1987:172) referring to her research on learner use of the first
language in course provision, as the “careful, controlled use of the mother tongue for
certain purposes, such as, to contextualise a lesson, to reduce anxiety and to provide the
20
rationale for a learning activity.” O’Grady is careful to clarify that “effective bilingual
teaching uses learners’ languages as a resource, but English remains the dominant language
in the classroom.” Furthermore, she puts a strong case for enabling learners at initial stages
of learning to have the option of bilingual courses.
In the current study, bilingual teachers come from the same background s as their
students, hence the frequency and purposes for which the mother tongue is used in the
classroom varies with the level of complexity needed to explain and discuss the feature of
English to be acquired. Often the use of the mother tongue in these situations shortens time
and effort needed for language acquisition. Other important reasons for initially focusing
on first language literacy would be the relative status of languages or the likelihood of first
language loss particularly among migrants and refugees who have suffered trauma and
faced a period of ‘anomie.’ Hood (1990) provides evidence that the relative socio-cultural
status of the first language and the host language affect learning outcomes. Hood also
points out that there are advantages in gaining L1 literacy first where there is a danger of
losing that language.
Apart from language learning issues such as using appropriate and relevant
materials, providing opportunities to achieve success, practising flexibility in class
scheduling and providing separate classes for those with no literacy skills, other factors that
affect the learner’s ability to progress in a formal class have been identified. Cumming
(1992) looks at why literacy programmes designed for majority populations may not be
seen as accessible or relevant by groups most in need of literacy education. He suggests
four kinds of obstacles: institutional, situational, psychosocial and pedagogical. He also
suggests that women and refugees have the greatest difficulty in accessing education.
Brod (1995:2) suggests that personal factors such as the age of the learner, low selfesteem, pressures from work, home problems of schedule, childcare and transportation
could all affect learning. Allender (2001) identifies learning barriers such as lack of formal
education, disruptions from war and politics, first language illiteracy, use of non-roman
script in the first language, age, effects of torture and trauma, and different cultural
backgrounds as further factors mitigating against literacy in English. She suggests that
course options need to vary in length, learning pace, intensity, focus and delivery modes in
order to overcome these difficulties. Allender outlines strategies that help such as use of
21
methods that build confidence, promote success and reduce learner anxiety. These include
small classes, bilingual assistance, on-site childcare and times to suit students.
The focus of the current study is learners who have minimal oral English, little or no
literacy in their first language and who are likely to have many of the additional personal
and social blocks to learning. Information on these aspects has been gathered through our
research instruments in order to provide a comprehensive profile on the nature and
background of the learners currently undertaking the literacy programme through
community based classes organised by the ESOL Home Tutor Service. This information in
turn provides us with some indication of the factors which affect literacy gains among lowlevel literacy students from an ESOL background. In order to look at literacy gains, it is
important to identify the most appropriate research methods for the targeted group.
Research methodologies
This research project needed to explore the factors that influence language and
literacy acquisition. This aspect of the project required the investigation of the learning
situations and the nature of the involvement, the characteristics, and the experiences of
students and their tutors.
Cross-cultural research
Attempts to understand the perceptions and experiences of people from other
backgrounds raise many challenging issues. To be successful, researchers must find ways to
accommodate the implications of cultural differences. Anderson (1994:51) describes
researchers as having to develop “research practices that acknowledge and take as central
the class, race and gender relations in which researchers and research subjects are situated”.
Other significant factors raised by Eckermann (1993:55) are the varying values, languages,
experiences, beliefs and mind frames that all those involved bring to the project.
Some researchers such as Stanfield and Dennis (1993:17) query that such research
can be done. They raise issues such as the power imbalance between the researcher and the
researched and the impact of colonial attitudes, ethnocentrism and positions of privilege.
However, others believe that the barriers to collaboration can be overcome. Patton
22
(1990:57) suggests that Weber’s concept of verstehen is relevant as it emphasises “the
human capacity to know and understand others through empathetic introspection and
reflection based on direct observation and interaction with people.”
Several specialists in cross-cultural research recommend inclusion of members of
the ethnic community or communities in the research team (Loo, 1982; Eckermann, 1993;
Stanfield & Dennis, 1993; Fitzgerald, 1995). Such team members can provide a cultural and
linguistic bridge, gaining access to as well as the trust of the participants. They can ensure
the research methodology is culturally appropriate and acceptable and also advise on data
interpretation. However, as Goldman and Schwartz (1987:149) warn, factors such as
knowledge and understanding of the host population, existing power relations and the
influence of politics of the past must be given careful consideration.
Case studies
“According to John Dewey, social life, defined as a process or movement, could
only be understood if the meanings assigned to it by its own actors were incorporated
within it” (Hamel et al., 1993:18). Denzin and Lincoln agree, commenting that “qualitative
researchers study things in their natural settings, attempting to make sense of, or interpret
phenomena in terms of the meanings people bring to them” (cited in Bassey, 1999: 2).
The case study is frequently used in educational research to inform “educational
judgements and decisions in order to improve educational action” (Bassey, 1999:39). It is
also a common research design in applied linguistics and the study of language
development and acquisition. Examples are Schumann (in Nunan, 1992:79) who
investigated the hypothesis that second language development will be governed by the
extent to which the learner identifies with and wishes to acculturate with the target language
community. Spada attempted to link teaching practice in second language classrooms with
learning outcomes. An advantage of this approach is that “multiple data sources illuminate
the immediate experience of the student in the lived culture of classrooms and educational
implications” (Spada, 1990:19). Thus theory and practice can be linked, identifying the
factors which promote successful learning and the barriers which prevent it.
23
Cohen and Manion (1994:150) identify the advantages of case studies and their use
in educational research. They describe case study data as “strong in reality” and therefore
interesting and aligned with the learner’s own experience. They allow generalisations based
on the “subtlety and complexity of the case in its own right.” The “embeddedness of social
truths” is acknowledged, as are the differences in participants’ perspectives. Case studies
are “a step to action” and provide the database for other research in an accessible form.
Interviews
Patton (1990:24) comments that the qualitative researcher must provide a
framework within which people can respond in order to represent accurately and thoroughly
their points of view. Thus a method commonly used in cross-cultural research is the one-toone, open-ended interview (Pit taway, 1991; Shadbolt, 1996).
The benefits of one-to-one interviews are that they are flexible in terms of location,
timing and privacy. As observed by McSpadden (1987:800), the method encourages trust
and open communication and provides a less threatening mode of participation for
participants who may feel afraid, mistrustful and guarded in their contributions.
In such circumstances, it is essential that the interviewer has the appropriate skills,
experience and awareness of the complexities of the situation. As McDermott (1997:46)
observes, the interviewer should have credibility with and extensive experience of working
with refugee communities in order to have gained respect and established relationships of
trust. S/he should be “sincere, understanding, tolerant, adaptive and humble as well as
competent in cross-cultural communication.”
Cohen and Manion (1994:318) concur, stating that “studies have shown that colour,
religion, social class and age can, in certain contexts, be potential sources of bias.” They
recommend that questions are carefully formulated so that the likelihood of
misinterpretation is reduced and that the characteristics of the interviewer are matched with
those of the sample. Careful consideration should also be given to the format of the
questions. These should reflect the purpose of the interview, the subject matter, the
participant’s level of education, and the depth and nature of information s/he can be
expected to have (Cohen & Manion, 1994:321).
24
Observations
Observation is a very useful method when investigating the process of education. It
can capture the immediacy of the classroom and more subtle aspects such as atmosphere,
tone and sense of purpose. It can detail the classroom behaviours of the teacher and students
as well as identify the effectiveness of the learning.
Allen, Fröhlich and Spada (1985) developed a sophisticated scheme for classroom
observation entitled The Communicative Orientation of Language Teaching (COLT) to
enable researchers to compare different language classrooms. It has two sections. The first
focuses on the description of classroom activities and includes the activity type, participant
organisation, classroom materials, lesson content and student modality. The second section
isolates communicative features such as use of sustained speech in the target language (also
see Nunan, 1992; Shameem, 2000).
The use of the COLT observation scheme in the English language classroom
attempted to link teachers’ interpretations of communicative language teaching theory with
their classroom practice. It also tried to identify the impact of different teaching practices
on learning outcomes. Whilst being very difficult to implement, such studies enhance
practitioners’ awareness of how to select teaching methods which enhance student learning
and improve their achievement. By observing classrooms, the process of good language
teaching can be better understood.
Assessment and evaluation
Two methods of assessment were used in the project – self and performance
assessment (tests). Student self- assessed proficiency was compared to the test results of
student’s actual performance on a compatible scale. An earlier study in NZ among IndoFijian immigrant teenagers found that they were self-assessing their aural and oral language
proficiency in their mother tongue to a high degree of compatibility (Shameem, 1995;
1998).
This testified to the validity of self-report proficiency scales when language
proficiency is tied to real and authentic language functions in the language being assessed.
However, the respondents in the earlier Wellington study were teenage first generation
25
migrants who had high levels of proficiency in their mother tongue. Moreover, reports of
their literacy ability were not validated.
Burt and Keenan (1995) define assessment as placement; measuring progress by
demonstrating improvement to learners and clarifying for educators what has been learned
and what has not. It is vital that the assessment process in no way diminishes the learning
and teaching process. Cambourne (1992:12) writing of language assessment in primary
schools, sounds some words of caution. He points out that the validity of testing is doubtful
because “the theoretical principles that guide testing as a form of assessment are not
congruent with the theoretical principles that guide literacy teaching and learning.” He does
not discount tests but warns that they can only serve useful purposes if the purpose and
value of testing is clear to students and they are combined with classroom environments
that support students and enhance self-esteem. Tests should not be used as a basis to allot
resources to a programme, hold teachers or learners accountable, or create anxiety. This is
particularly true for programmes such as those run in the institutions we have included in
the research, as they cater for the needs of already severely disadvantaged individuals.
Assessments are also used as proof of suitability for further study or job training. The chief
stakeholders in assessment are the learner and the teacher.
Programme evaluation is a way to quantify gains to verify programme effectiveness.
It assists in decisions as to whether a course needs to be modified or altered in any way so
that objectives may be achieved more effectively. It is also used to justify funding.
Evaluation refers to a wider range of processes that may or may not include assessment
data. The stakeholders are the programme designers and funders. Tensions can arise when
assessment procedures are used in isolation, to evaluate and justify the programme to
funders.
According to Wrigley, (1992) “Learner assessment is one of the most troublesome
areas of adult English as a second language”. The general response appears to be that a
range of tools must be used.
26
Assessment procedures and tools
Language teaching developments from 1970s to 1990s cover the movement from a
structure-based curriculum to a needs-based, learner-centred one, and from normreferenced to criterion-referenced assessment in the form of competency-based tasks.
Norm-referencing compares students to levels achieved by other students whereas criterionreferencing assesses students against a pre-existing set of criteria, and allows for tools
ranging from tests to self-assessment.
Competency-based assessment
Competency-based assessment has been widely used in adult ESOL literacy
instruction since the mid 1970s. Its objectives are described in task-based terms such as
‘students can’ or ‘students are able to’ and include a verb describing a demonstrable skill,
for example ‘student can understand requirements in a simple form.' Competencies include
basic survival skills such as answering personal information questions or obtaining food
and shelter (Peyton & Crandall, 1995). An example of competency-based testing is found
in the Certificate in Spoken and Written English (CSWE) developed by the Adult Migrant
Education Service (AMES) in the 1990s.
The CSWE models given in Assessment
Guidelines (Navarra, 1993) together with sample tasks, marking grids, suggestions for
teachers and programme managers for administering and managing assessment were used
in developing the assessment tasks used in this research.
Brindley (1998) discusses the problems of using this type of testing on a large scale.
They are related to the comparability and relative difficulty of assessment tasks, the
consistency of rater judgement and the generalisability of underlying skills within and
across competencies. Among his solutions were banks of tasks, trialed and tested in a wide
range of situations, that elicit the same type of specified performance and in which
performance criteria are clearly observable.
In his opinion, the necessity of adequate
preparation for language and literacy teachers, and the ongoing opportunity for them to
develop necessary skills is a key factor, as “allocating to teachers responsibility for
designing and conducting their own assessment tasks rests on the assumption that they have
the skills required to carry out this task effectively” (Brindley, 1998: 67).
27
Self-assessment in language learning
The idea of self-assessment is not a new one. Rolfe (1990) refers to various
proponents who have used self- assessment for a range of purposes in a number of
languages. The move towards a less prescriptive, more learner-centred, negotiated
curriculum has also highlighted its importance and its contribution to the general field of
language assessment.
Despite acknowledgement of the positive contribution that self-assessment makes to
language learning, there continues to be some divergence of opinion about the validity of
self-assessment. Some of the anxieties about self-assessment have been: lack of objectivity;
that learners are not capable; and that they will rate themselves too highly or cheat. Studies
have generally shown these concerns to be unfounded. There is a need for some preparation
of students about the value and methods of self-assessment, but the tendency has been for
learners to under-rate rather than over-rate themselves. There is more evidence of cheating
in standard tests than with self-assessment.
Advantages of self-assessment include flexibility in gathering information about
learners, and measurement of what has been taught in class. Studies also note advantages
of self- assessment to both learners and teachers as including a higher sense of involvement
and responsibility for learning by the former, and ease of administration for the latter.
Lewis (in Brindley, 1990:187) concurs with these findings: “Informal self-assessment is a
natural part of language learning.
Formalising self- assessment should help focus the
learners’ attention on their strengths and weaknesses.”
While self-assessment is unlikely ever to be used as a sole means of assessment for
certification, it has proven value as a method of giving individual feedback on the language
teaching and learning process, and allowing for self-perception of knowledge and skills
gained. Rolfe (1990) compared assessment ratings using both self and peer assessment in
the AMES Speaking Proficiency Descriptions, and found a significant degree of
correlation. He also refers to other studies that have found a high degree of reliability and
correlation between both teacher- and standardised test assessment and self- assessment
(LeBlanc & Painchaud, 1985; Brindley, 1989; Paltridge, 1989).
28
In NZ studies, Shameem (1998:86), testing oral and aural Fiji Hindi proficiency
among teenagers of the Wellington Indo-Fijian community found that “the results of the
performance test correlated strongly with the self-reported data, thereby demonstrating the
validity of the self-report scale.” Subsequently, Roberts (1999) uses self-assessment in the
context of studying language maintenance in the Gujarati, Samoan and Dutch speech
communities in Wellington.
Roberts (1999), Shameem (1998) and Lewis (in Brindley, 1990) all include
extensive references (LeBlanc & Painchaud, 1985; Kalantzis, 1989; Bachman, 1990;
Hakuta & D’Andrea, 1992; Bachman & Palmer, 1996) to the positive use and usefulness of
self-assessment. Self-assessment is a quick, practical and anxiety-free way for a learner and
a teacher to reach an understanding of learner competence and, if conducted at various
points over a period of time, learner progress and achievement.
Self-assessment measures
A variety of measures have been used for self-assessment purposes in applied
linguistics, although many studies have used a numerical rating, having learners indicate the
degree of ease or difficulty they experienced in completing specific tasks (LeBlanc &
Painchaud, 1985; English Language Institute Proficiency Scale, 1992; Shameem, 1995,
1998).
In the current study with low- level literacy and English language learners, a simple
five-point scale was used, ranging from ‘ability to complete a real- life literacy task with
ease’ to ‘being unable to do so at all’ for eight reading and eight writing competencies.
These same competencies were then used to create performance ‘test tasks’, which were
administered by the classroom teacher (see Shameem 1995, 1998 for a similar procedure).
The methodology section of the report details the two measures further.
The results would show whether self-assessment is a valid, relevant and appropriate
measure of literacy progress for low-level literacy learners for whom English is an
additional language. The use of performance assessment at this level is time-consuming,
and has the added disadvantage that cultural expectations and educational backgrounds can
29
make students resistant to their use. However, while the use of self- assessment instruments
to assess literacy gain might be pedagogically successful and backed by research evidence,
it may not satisfy funding bodies that require hard data, because it is difficult to quantify
outcomes without using ‘real’ tests. Employers, for instance, might prefer to see the results
from commercially available tests such as IELTS, as scores are easy to read and interpret,
and have universal application regardless of the place of study. However, commercial tests
do not adequately test individual strengths and weaknesses, especially at low-level literacy.
Nor do they necessarily measure what has been learned in class or how it has been taught,
or give priority to learner goals.
Performance assessment
A communicative performance test, being task-based, reflects language use in
realistic situations and at the same time assesses communicative appropriateness. The key
argument for this approach is that knowing a language essentially involves the ability to use
it appropriately in a range of circumstances (Spolsky, 1978, 1985; Galang, 1982; Van
Weeren, 1984). Clark (1975) suggests that the broad aim of assessment in any language is
that the process should determine the ability of the individual to receive or transmit
information in that language for some pragmatically useful purpose, within a real- life
setting. Authentic elicitation and realistic language use are particularly important.
Several writers express their reservations about taking this 'real- life' approach. They
argue that the closer the apparent replication of reality and the higher the face or 'genuine'
validity, the harder it becomes to lay claim to any other type of validity or to attain test
reliability (see Clark, 1975; Shohamy, 1983; Bachman, 1990). Stevenson (1985) points to
the uniqueness of each sociolinguistic situation and the infinite possibilities of language
choices in each situation according to domain, context, interlocutors, region, and other
factors which are unreplicable in test situations. Bachman (1990:81) too, states that
language use is ultimately context-dependent, and is therefore unique. And similarly,
Shohamy and Reves (1985) tell us that in multilingual communities, language is used for
effective communication and is rarely judged for correctness, which is normally a primary
focus in a language test. These observations had various implications for test design.
Although the ideal was to develop a real- life 'authentic' test, we were aware of the
constraints operating on such an undertaking and that at best we could elicit what might
30
sound like authentic language behaviour but which would in reality be influenced by the
participants' awareness of their involvement in a research study, as well as other factors
specific to the test situation (also see Shameem, 1995; 1998).
A communicative literacy test that included functions representing real- life
language use had the greatest degree of validity for the research sample. A further
consideration was the notion of ‘appropriateness’ of response to fulfil particular functions,
as much as correctness. Hence this was carefully taken into account in the rating process, as
among pre- literate learners particularly, even the move from the inability to formulate a
letter of the alphabet to the ability to write a misspelt, simple, one-syllable word is an
achievement worthy of credit.
In taking this approach it was essential that the procedure be perceived by test takers
and administrators as honest, fair, sensitive to gender, religion, and the varying levels of
performance ability in both the pre- and post-programme testing procedures.
It is acknowledged that allowances also need to be made for the differences between
the students and the test administrators, as some of these were bilingual and others were
native English speaking tutors. Differences such as these have influenced results in other
language tests (Bachman, 1990:350).
Reliability and validity in performance assessment.
To be valid, a test must demonstrate that it is measuring what it is purporting to
measure; to be reliable it must show stability over time and across raters. One of the
greatest tensions in language performance assessment is the one between validity and
reliability. The closer one gets to replicating reality and eliciting real- life type language use,
the more unreliable the data may become, with the absence of structural constraints and the
lack of consistency from one test-taker to another.
Despite this tension between reliability and validity in assessing language
performance, the two concepts are closely linked. For a test to be valid at all, it must be
reliable. Although the traditional view saw reliability as the precursor to validity, it is now
31
perceived to be an integral part of it.
In language tests, a central concern is the
identification of the factors which affect language performance and therefore the reliability
and validity of the tests. If raters are making unreliable judgements of performance in rating
test results, then one cannot make any valid judgements on the basis of the test results.
Bachman (1990:241) suggests that reliability and validity should in fact be seen as part of a
broader concept of test 'usefulness' that might be a better measure of a good test. Because of
the tension between validity and reliability in communicative language tests, it is important
to look at each in turn in relation to assessing the performance of low- level literacy students
involved in this project.
First of all, validity has to be considered in relation to the purpose of the test and the
use that is to be made of the results. The purpose of this literacy test was to gather evidence
of the performance ability of pre- literate and low- level literacy students undertaking a
course of study. Thus, the results should give an accurate description of the ability of these
students to read and write English at the very basic 'emerging', 'survival' and 'social' literacy
phases at the beginning and end of their study programme.
The most easily achievable and most contentious type of validity is 'face' (also
known as 'genuine') validity - the test must ‘look’ right at a glance to an ordinary lay
person. Although face validity has traditionally not been taken seriously in testing literature,
it nevertheless has importance in terms of test takers' perceptions of the ir reasons for taking
the test. Since the test takers’ motivation was a serious consideration in a low-level literacy
test, it was important that the test should look right and show relevance to the stated
purpose of determining levels of survival and social literacy ability in a range of authentic
language tasks.
Content validity means ensuring that the tasks and topics which are included in the
test are representative of the larger domain in which the language is normally used. It is
quite difficult to attain content validity because it is simply not possible to cover all
possible areas and functions in which the language is used. Since the main concern of the
present survey was the collection of appropriate information on a respondent's performance,
thereby gaining a profile of performance ability among these students, content validity
could be improved by identifying the appropriate language tasks the test should include.
However, by its very nature, a test is very limiting in terms of the topics that can be
32
discussed, the method of elicitation, and the artificiality of it being a test situation, despite
the pains taken to make it as realistic as possible. Moreover, quantifying real- life language
use may lower content validity. Hence a major concern in establishing content validity is
the extent to which a low-level literacy test can provide us with a representative enough
sample of the test taker’s actual literacy ability in English in tasks and topics covered in the
curriculum – which is what the rater has eventually to judge.
A measure of concurrent validity for the test was obtained by comparing the two
literacy performance assessments with the two results of the self-report assessments
conducted at the beginning and end of the programme of instruction. Although the different
aspects of validity have been discussed discretely, as mentioned above, one needs to take an
overall view of validity in order to demonstrate the usefulness of a test in relation to the
group and language under study. Validity is invariably affected by reliability. In order to
have test reliability, one needs to manage and control those factors which might cause a
variation or distortion of performance scores. These might be features within test design,
implementation and rating.
Despite the reliability/validity tension in performance assessments, steps may be
taken to find an optimum balance between consistency and reality in a practical, systematic
way. High on the list of factors to address for test validity was the development of an
authentic communicative test which was representative of real- life language functions.
To build reliability into test design, a structure or framework was provided to
restrict the range of possible responses with each test task. The framework for the test
content included a description of tasks which could be achieved at each level, and set out a
procedure to be followed so that the respondents were provided with ample opportunity to
demonstrate performance ability, with several tasks at each level. Even if each test did not
follow exactly the same path, or have the same administrator or test site, at least a common
test provided uniform set tasks and a procedure to be followed to ensure some reliability.
Language performance with the same test paper covering the objectives and outcomes of a
similar curriculum provides a better basis for test reliability.
It was recognised that the test venue might also affect reliability; the test
environment differed widely for each group of respondents. The overriding concern
33
however, was for test validity and since the classroom was the place being used to acquire
the necessary literacy skills, it was therefore the best place for elicitation of performance.
Although one can improve the reliability of performance tests through test design
and implementation, a further important concern is that we are able to measure the
reliability of the results which are obtained from the performance test procedure, especially
since some subjective judgements are being made. In this test, reliability of ratings was
evaluated by means of inter-rater checks, which were obtained by having two raters
marking one or more class sets. A comparison was then made of the ratings. A checklist
was provided to ensure consistency of judgement during the rating process.
Factors affecting literacy gains
A third focus of the research was to collect information about additional factors that
might affect language and literacy gains among the learners under study. Since learner
assessment at low levels of ESOL literacy has been identified as one of the most
troublesome areas of adult literacy (Wrigley, 1992), it is crucial to determine which factors
might contribute to language learning and which mitigate against it. Some testing issues
are unique to ESOL learners, for example, problems with vocabulary leading to reading
difficulties, or with the cultural notions underlying the actual test items such as
understanding instructions or methods of testing (multi- choice, matching, cloze).
It is possible that assessing language gain is difficult because of the background
factors of the learners themselves and the minimal rate of progress that can be expected
from refugees and migrants who are suffering from post-traumatic stress disorders and
feelings of anomie. When low- level learners also have a refugee background, the impact of
past experiences has been shown to have a critical effect on the ability to learn, especially
in formal classroom situations.
The trauma of past events often leads to chronic
psychological symptoms, memory impairment, short attention span, severe anxiety and
limited concentration.
It also has an impact on confidence and self-esteem. Refugee
learners may need a longer time over which to spread their study, flexibility in attendance
and additional hours (see Abbot, 1989; Kasanji, 1994; Smith, 1994, 1997).
34
In addition, for many low-level learners, a minimal or broken education indicates a
lack of familiarity with classroom behaviours and conventions, such as test-taking, or
functional skills such as holding a pencil. For these reasons and in order to develop
effective assessment tools for a low- level ESOL language and literacy class of people from
refugee backgrounds, we must clarify what is occurring in the class and whether there are
gains other than in language and literacy that should be assessed in order to demonstrate
both student progress and effectiveness of the programme. Allender (2001) suggests that
the development of less tangible, non- linguistic skills and qualities such as confidence,
cultural awareness and learning skills are essential. It is likely that gains will be made here,
and these should be taken into account when considering programme outcomes.
It is possible to measure these gains qualitatively through teacher-student interviews
which use an initial student profile, self-assessment surveys, and needs assessment as the
basis for student-teacher discussion and goal-setting during and after the programme of
study. In the current research project, persona l profiles and case studies illuminate some of
the affective problems and perceived gains of students.
This study hopes to provide some additional data and point the way to the need for
more research in the area of low- level literacy provisions for refugees and new immigrants.
A reduced ability to keep up with the pace of a class may mean that learners need a class
that is specifically geared to their employment or other needs. Refugees are inevitably
placed in a low socio-economic group, have settlement difficulties with finance, and a high
rate of unemployment.
Family commitments, such as family reunification, children’s
adjustment to school, and childcare are priorities. Even the simple question of transport can
have an effect on learning (see for exa mple report by White, et al., 2001). Reasons for
special consideration for ESOL provision for refugees in NZ have been clearly outlined by
Altinkaya (1998).
The programme design elements that provide a suitable class for low-level learners
of refugee background include flexibility of learning pace and goals as well as continuity,
and are designed to take into account previous levels of language proficiency. Small class
size is needed, together with a safe, comfortable and supportive location close to the
learners’ community, because of the need for bilingual assistance and on-site childcare.
Times must suit learners. Content needs to integrate settlement, parenting and health
35
information with language and literacy. The classes need to be available when the learners
need them and to ensure that they are part of a process which will assist them to get families
and lives together, to heal and build up confidence in the future, to bridge cultural gaps and
to learn how to learn. A well-designed, relevant programme can do much in creating high
levels of awareness in the importance of literacy, thereby increasing student motivation and
encouraging teacher commitment.
Literacy in Aotearoa/New Zealand
The report Changing Skills for a Changing World (Johnson, 2000) that followed the
OECD surveys Literacy in the Information Age - Literacy Economy and Society and
Literacy Skills for the Knowledge Society, addresses the need to understand and define
literacy, gives an outline of the obstacles that prevent the issue of literacy being addressed,
and makes suggestions for a comprehensive strategy. The survey focuses on vocational
literacy because much of the NZ work in literacy has been workplace literacy. In the current
context however, vocational skills are only a part of the bigger picture. Low- level ESOL
literacy students are first of all concerned with survival and social literacy (the ability to
perform functions which will allow them to deal with social and health services) before
vocational skills are acquired. This may be particularly true of women in these classes, who
are the primary caregivers of children and families, while men would be concerned with
also acquiring vocational skills. It is imperative that a personal profile/needs analysis is
conducted at the start of each programme to ensure congruence between teacher, student
and programme goals.
Summary: Literature review
The scope of this project encompassed two major fields in applied linguistics:
literacy and language assessment. More specifically they concerned aspects of these fields
in relation to low- level and pre- literate learners most of whom had arrived in NZ as
refugees and dispossessed individuals with a history of persecution, and who were likely to
suffer from post-traumatic stress disorders.
36
While the original definition of literacy was rather narrow in its focus solely on
reading and writing ability, in recent times the broader definition, which includes all aspects
of communication through the written word, has become a more salient definition. It
includes numeracy ability and the ability to manipulate numeracy skills appropriately for
various purposes. Moreover, it looks at literacy in a broader context – in the ability of the
individual to use literacy skills for realistic functions and authentic purposes, with effective
and clear communication being primary objectives. It also takes into account the
background factors that students bring to the learning process and the affective gains made
in self-confidence and self-esteem.
The difficulty in measuring low- level literacy skills is acknowledged, particularly
when the students undertaking the programme have no previous experience of test-taking
situations. Ideally, a self-assessment or self-report of skills would be sufficient to indicate
performance level or progress – however in order to make this a valid system of assessment
for a group such as this, it would need to correlate with student performance. In addition,
personal factors in the background of learners such as previous education, first language
competencies and history of English learning needed to be taken into consideration in
making decisions about progress and achievement. The personal profiles, case studies and
classroom observations gave further evidence of the affective domain and environment that
learners were experiencing.
This review of literature covers the areas of pertinent concern in this current project.
The next section looks specifically at the background of the project and the issues that were
present and arose during the teaching and assessment processes.
37
38
BACKGROUND TO THE PROJECT
Introduction
Although this project involved an analysis of literacy gains in low- level literacy
classes for both refugees and first generation immigrants, it is important to look at the
refugees in particular because of their background, their language learning needs and the
contribution that the bilingual refugee tutors have made to this group.
This section is divided into three main parts. The first looks at the recent refugee
situation in Auckland in terms of literacy experience and English language level. The
second examines the language learning issues for refugees and the consequent impact on
their resettlement. Finally, a pilot training project with bilingual tutors from refugee
communities is outlined and issues which arose from the project are discussed.
Background and rationale of programme
Through an agreed quota system, NZ receives around 750 United Nations refugees
per year. In addition to this group, refugees are admitted under the Family Reunification
provisions of the Immigration Act and a large number of spontaneous refugees arrive in the
country each year. Forty percent of refugees stay in Auckland. Provision for meeting the
learning needs of pre- literate refugees in Auckland has been varied. Of refugees arriving in
NZ, 80% have less than four years of schooling. Following Davison's (1996) definition of
literacy, few refugees are able to function effectively in NZ and use literacy as part of
everyday communication. Davison’s (1996) definition of literacy is the ability to:
•
read and use information
•
write appropriately (in a range of contexts)
•
function effectively in society
Data provided to UNITEC by Centre of Refugee Education, AUT indicates that
typical English proficiency levels of UN refugee intakes to NZ over the last four years are:
40% pre-literate; 40% semi- literate; 15% elementary to intermediate level and 5% high
intermedia te. Pre- literate refugees are considered not literate in any language and come
from a society or community where literacy plays little or no part in communication. Semi-
39
literate refugees have some literacy skills in their first language but are not confident to use
their literacy skills in a new language (Gardner et al., 1996:2).
Experience has shown that pre- literate students cannot succeed in existing beginnerlevel English classes because of their use of print-based media. The initiative of training
suitable bilingual tutors to help members in their pre- literate communities achieve initial
literacy and numeracy skills in English has assisted these students to enter beginner- level
classes at tertiary institutes and in community education. This has provided a first step
towards employment for long-term unemployed refugees, as well as providing more
immediate employment opportunities for the bilingual tutors.
Starting from the premise that successful community-based training programmes for
pre-literate refugees occur when trained bilingual tutors are available, staff from UNITEC
initiated bilingual discussions with WINZ/DWI (Work and Income NZ/Department of
Work and Income, previously NZES, NZ Employment Service), along with members of the
refugee groups concerned, and advocacy organisations. These discussions resulted in an
agreement for UNITEC's School of Languages and School of English and Applied
Linguistics to deliver a pilot tutor training course as part of the WINZ 1999-2000 funding
round.
Refugees in West and Central Auckland
The statistics for Ethiopian, Eritrean, Somali and Kurdish refugees who have moved
into the Central / West Auckland area are (Table 1):
Table 1
Refugees in Central and West Auckland
Pre-literate
Somali
Ethiopian
Eritrean
Kurdish
Mean
S.d.
27%
22%
25%
36%
28%
0.06
No
English
literacy
18%
27%
16%
25%
22%
0.05
Beginner
English
Elementary
English
33%
27%
31%
25%
29%
0.04
12%
18%
10%
10%
13%
0.04
Total:
Elementary and
under
90%
87%
89%
96%
91%
0.04
(Source: Martin-Blaker and Hardman, 2001:2)
40
Since the majority of refugees and immigrants join previously established
communities in Auckland, the issues that arise consequently loom larger in Auckland than
elsewhere. Of the United Nations quota refugee intake living in West/Central Auckland,
most lack basic survival skills – they cannot read a medicine label, understand their child’s
school report, understand a power bill or get a bus to an unfamiliar destination without help,
often needing assistance from one of the few literate, English-speaking members of their
community. The English-speaking refugees usually spend many hours helping friends visit
WINZ or the doctor, and so on, since there are few paid community workers or translators.
For the pre- literate person, this is extremely disempowering.
The sense of anomie has been further heightened by the lack of employment
opportunities because the arrival of these refugees has coincided with a time in NZ’s
political history when social equity objectives were not deemed to be the responsibility of
the State.
Whereas in a different set of economic and political circumstances people with little
formal education may have been absorbed into the workforce, in the last decade they have
been considered to lack marketable job skills, including the ability to speak and understand
English. The English language, in the last decade, has been a market commodity, in that
those who lack English proficiency must pay substantial fees to acquire it. If they are
unable to do so, they may be funded by the State (through Work and Income NZ or Skill
NZ) to acquire a strictly limited degree of English proficiency. The success of fully- funded
training has been measured by the percentage of trainees who have either found jobs or who
have moved up a step on a learning pyramid of which the apex is any form of paid
employment.
Language learning issues
Several language learning issues become apparent when teaching literacy to preliterate and low literate level learners. Ramm’s (1992) NCELTR project in Australia,
involving learners with minimal formal education found that, for maximum benefits:
•
learners should be placed in classes with others of a similar educational background;
41
•
low- level literacy learners should stay in these classes until they acquire formal learning
strategies that will enable them to access other learning options;
•
all teachers should receive professional development support to assist them with
appropriate methodology and teaching materials;
•
low- level literacy learners need 400 hours tuition to progress one point on the ASLPR
scale (Ingram, 1981). As Survival English level on the ASLPR scale is between levels
1+/2, pre- literate learners would need between 800-1200 hours to reach this level
(equivalent to 18mths – 2 years full time study).
Surprisingly few researchers in this area have mentioned the important issue of
teacher-student communication when the teacher does not speak the learner’s language.
This issue is particularly pertinent for learners with little formal educational experience,
because they are not aware of the common expectations of the Western education system
and the relevant study skills needed to help them learn. Bilingual English teachers or
teachers from language minority communities present the ideal solution for learners in these
circumstances.
Bilingual tutor training project
When, in the mid 1990s, African and Middle Eastern refugees began to appear at
UNITEC Institute of Technology, funding was sought to provide appropriate ESOL and
literacy instruction for them. The West Auckland Home Tutor Scheme and the Auckland
Institute of Technology (now Auckland University of Technology) at the time also tried to
respond to the growing and urgent needs of these learners. As one of the greatest difficulties
was that of communication, it was decided to train bilingual and peer tutors to aid with
teaching members of their own community. The Auckland North region of WINZ funded a
training course for ten bilingual tutors at UNITEC, with the first intake of tutors completing
their training in mid-2000.
The training programme was originally designed mainly to train or re-train members
of refugee communities as tutors of basic English language and literacy skills. Accordingly,
the training focused on introducing theories of literacy, numeracy and language teaching,
theories of adult learning and practical tutoring skills (See Appendix 1 for aims and specific
42
outcomes of the course and course details). The trainees were recruited in consultation with
community organisations and were from the Somali, Ethiopian, Oromo and Iraqi
communities. They needed to have an appropriate standard of English and to have had at
least received a secondary education. This posed some problems during recruitment, since
there are few educated women in these communities yet the majority of the future students
would be women. Two Somali women started the course but one had to withdraw for
personal reasons. The other tutors were all male, and had a range of educational and
professional backgrounds. Two were ex-tertiary lecturers themselves, three were exteachers and four did not have a teaching background. Both those with and those without
previous teacher training were able to become effective tutors, and it is worth noting that
other projects which used previously untrained teachers have also found that they are
effective (See D’Annunzio, 1995).
The course included some computing skills and discussion of community funding
issues. This proved to be more important for the tutors than had been expected, partly
because there was no provision to fund the classes that they were hoping to teach. However,
it was quickly realised that the application for funding was best left to administrators in
community organisations, as the tutors’ community organisations had not yet developed the
infrastructure to do this. As one tutor remarked:
We know we have to knock on doors but we don’t know which doors or how to knock.
This highlighted the key role that the tutors take when they are working with their
students: they pass on knowledge and information which their pre- literate learners
otherwise have no access to, in an equitable and empowering fashion. Yet the tutors
themselves are recently arrived and often don’t know “how things work”. Understanding
how and why WINZ was funding their training course yet was not going to fund their
subsequent teaching of actual classes was one of the issues they were concerned about. The
course lecturers of the bilingual tutors found themselves positioned as sharers of knowledge
about systems, as much as being skill developers.
The tutors have a unique approach to their teaching, primarily because they share
the students’ language and culture. During teaching practice observation, it was realised
that the shared language between the trainee and students dominated classroom talk. There
43
also seemed to be a lot of teacher talk and student questioning in process. In most of the
observed classes, a large proportion of time was taken up with teacher-dominated
discussion. In one group, students and tutors who were all from the same country shared
five languages between them, so the linguistic complexities of discussions were quite
elaborate.
It was clear that students and tutors were working hard to negotiate shared meaning
and understanding. Student evaluations of bilingual tutors who began to run community
classes showed that a majority preferred tutor explanations using their own language.
Students often brought handouts from their day class to their evening class, for discussion
and further clarification with their bilingual teacher.
Murray (1998) points out that much English and literacy teaching to adults here and
overseas has taken a narrow, functional view of literacy. If the teacher is unable to
communicate with the learners at anything above a very basic level, students will continue
to function at this level. Teachers who can communicate with students in a common
language can help acquire the higher order, more cognitively demanding literacy skills
which are needed in the NZ environment.
The current situation in Auckland
Towards the end of the tutor-training programme, the issue of funding of language
teaching classes arose. The West Auckland ESOL Home Tutors donated their Community
Education hours in term two 2000, and many other Community Education providers
donated unused hours for the remainder of that year. Meanwhile further training is urgently
needed for bilingual tutors to assist other pre- literate learners living in Hamilton,
Wellington and Christchurch.
During the course of this research project in 2001, the National Association of
ESOL Home Tutor Scheme Inc. under the Adult Literacy Strategy received under the Adult
Literacy Strategy funding to fund classes for 32 weeks for low- level literacy learners in
West and South Auckland. While the project was set up in the first semester, data gathering
was conducted in the second half when West Auckland ran four 12- hour a week and one 2-
44
hour a week classes for 20 weeks, while South Auckland was running one 12-hour class
and two 2-hour a week classes with native English-speaking teachers (See Table 2).
Table 2
Location
West Auckland
UNITEC campus
West Auckland
Wesley Intermediate
South Auckland
Profile of literacy students participating in project
Tutors
Hours
12,2,12
Weeks
(Semester 2)
20 Weeks
Classes and
codes
3 (C,D,E)
Bilingual
Bilingual/English
12,12
20 Weeks
2 (A,B)
English-speaking
2,2,12
20 Weeks
3 (F,G,H)
This funding was connected to the current project, which was measuring the gains
made in literacy by these learners. Because of the low level of literacy expected of this
group at the start of the project, and the difficulties associated with measurement of
learning at this level, a clear gain was not expected. Moreover, because students began the
course at different levels of proficiency it was expected that gains would invariably differ,
depending on the starting point of the individual student. Hence a longitudinal assessment
at the beginning and end point s of the programme would provide us with some knowledge
of the existence and nature of any literacy gains made during the 20-week period of the
research project.
45
46
METHODOLOGY
Introduction
This literacy research project was funded by the Ministry of Education through a
grant to the National Association of ESOL Home Tutor Schemes Inc., as part of the NZ
government’s Adult Literacy Strategy. The first objective of the project was to develop
measures of literacy gain for adult students with very low levels of literacy and English. It
also aimed to identify other factors which impact on learners’ progress.
A more general objective of the research was to better understand the educational
needs of Non-English Speaking Background (NESB) students at emerging levels of
literacy, and determine appropriate methodologies for effective educational provision. The
research produced generalised measures of literacy gains, which will be available for use in
courses with similar, low literacy level learners. It will also indicate the relationship
between classroom testing and students’ self-assessment.
Moreover, the research encouraged further development of the curriculum and the
provision of on going professional development support of the tutors involved in the
project. As a consequence, the findings should inform future teaching practice and course
design. An additional outcome will be the establishment of an information base to inform
the judgements and decisions of the Ministry of Education regarding policy development
and the funding of programmes.
Tests consisting of competency-based tasks linked with the curriculum were
developed and administered in August and December 2001. Students also completed three
self-assessments and a personal profile. Case studies and classroom observation
supplemented the quantitative study. While it is acknowledged that literacy has a much
broader definition than just the ability to read and write, the nature of the literacy
programme and the time span made it difficult to expand on the narrower definition of
literacy.
However, other important personal variables were taken into account in the
measure of progress, as was the students' starting point in first and second language
proficiency on our own literacy measures of those who were undertaking the programme.
47
Research questions
The research questions for the project were:
Part A:
Personal profiles
Tutors
1. What are the profiles of the tutors involved in the project in terms of:
-
age, gender, ethnicity?
-
educational and linguistic background?
-
socio-economic status and change from country of origin to NZ?
-
NZ length of residence and age at time of immigration?
-
proficiency in first language, English and other languages?
2. Language and educational attitudes.
What beliefs do tutors have about:
-
literacy?
-
preferred learning styles of students?
-
learner characteristics that lead to successful language learning?
-
methods of classroom assessment being used in the project and informally?
Students
1. What are the ages, countries of origin and gender of current learners?
2. What was the socio-economic background of the learners in the study and what has
been their change in status in NZ?
3. What is their educational background?
4. What are their current proficiencies in English and their literacy levels in their first
language, compared to English?
5. What are their personal reasons and goals for learning English?
6. What has been the length of residence and what was the age at time of migration to NZ
of current learners?
7. What English language provision have current learners experienced in NZ?
8. Do learners have a literate background?
9. Language and social attitudes: how are these perceived by the learners:
§
from the host community of their culture, ethnicity and language and
§
in terms of their own usefulness in learning English?
48
Part B:
Self-assessment
1. What are the English language reading and writing abilities of low- level literacy
learners on a range of communicative, authentic literacy tasks when compared between
points:
-
at the beginning of a period of instruction
-
at the mid-point
-
at the end of a period of instruction
2. Has any comparative gain been made on the self-assessment scale across the period of
instruction?
3. What variables have contributed to the greatest degree of perceived gain in a) reading
ability and b) writing ability, for example, bilingual tutors, native speaking tutors and
intensity of tuition?
Part C:
Performance testing
1. How valid is the learners’ self-rating of their literacy assessment when actual
performance is judged by their tutors?
2. How can language proficiency at low- level and pre- literacy stages be conceptualised so
that it may apply to adult learners needing survival and social literacy skills in English
in the NZ context?
3. Can low- level English literacy skills be meaningfully assessed and measured in terms of
performance on a range of authentic and communicative tasks?
4. Can oral self- assessment of literacy skills be used as a valid alternative to actual
performance testing among low- level and pre- literate learners?
The research questions reflect the main focus of this study – the design,
development and use of measures of literacy gain among learners who are studying English
language literacy skills in order to survive in the NZ context.
It is important to
acknowledge that the learners involved come from disadvantaged backgrounds with a
history of pre- or low- level literacy in their first language. In addition they are highly likely
to suffer from anomie, post-traumatic stress syndrome and gender bias in communities
49
where men are more likely to be educated and literate than women.
The following
hypotheses were formulated from the research questions.
Hypotheses
1. There will be little measurable gain after 20 weeks of 2 hours per week. There will be
measurable gain for some learners after 20 weeks of 12 hours per week
2. Unfamiliarity with testing procedures and the low level of literacy makes classroom
testing challenging. It is hoped that self-assessment measures will show a significant
correlation with performance tests.
3. Research has identified various factors affecting language learning among immigrants
such as: previous level of education, literacy in L1, proficiency in oral English, literacy
in English, time of class, frequency of class, whether there is bilingual assistance or
teaching, size of class, gender, age, socio-economic pressures, employment status (in
prior countries and in NZ), access to classes (money, transport, childcare) and mental
health, including effects of previous trauma.
4. It is a strongly- held opinion among many involved in adult literacy that there can be no
formal measurement of progress at this level and that attempting to do so can be
counter-productive. Aspects such as learning about learning, self-confidence, enjoyment
of literacy, awareness of its value in their lives, ability to use skills for personal and
work reasons may equal in value any progress in measurable language and literacy
skills. This question is beyond the scope of the current research, but points the way to
further research that is of great importance.
Time span of project
The following table (Figure 1) sets out the time span of the literacy project. A year
was allocated to the project, with much of the design, development and trial of research
instruments taking place in the first half of 2001.
50
Figure 1
Timetable for research 2001 – 2002
March – July 2001
March - April
March – April
April – July
April – May
April – June
July – December
July
July – August
July
August
September
December
July – October
December 2001 –
January 2002
Initial Stages
Formulation of research proposal
Gathering of research team
Review of literature
Ethics approval
Development and pilot testing of assessment tools
Implementation
Revision of assessment tools
Confirmed classes for project
Ensure uniformity of administration of tests
Test 1 and Self-assessment 1
Personal and tutor profiles
Data entry and initial analysis after each testing round
Self-assessment 2
Test 2 and Self-assessment 3
Case study research
Data analysis
Report writing
Research participants
One hundred and eighteen students from refugee and immigrant backgrounds
participated in the classes. Sixty-two students completed most aspects of the research
except the final writing tests, which 55 students completed. Students were representative of
a range of language competencies. Some students were literate in their first language whilst
others were pre- literate both in their first language and in English. Thus, the outcomes in
terms of progress in literacy and English were expected to be different.
The English classes participating in the project were confirmed in 2001. The
students on the programme came from a range of ethnic backgrounds (see Table 3). They
were: Somali, Ethiopian, Afghani, Cambodian, Vietnamese, Chinese, Indian, Iraqi or of
Macao and Hong Kong origin. Tutoring in these classes was administered and delivered by
the West and South Auckland ESOL Home Tutor Schemes. The duration and timing of the
classes varied (See Figure 2). Some were 12 hours per week whilst others were two hours
weekly. The teaching delivery in three of the 12-hour classes and one 2-hour class was
bilingual. The remaining four classes were taught by native-English speakers. Two of these
51
were 12- hour classes and the other two were 2-hours each week. All classes ran for 20
weeks during the research and assessment cycle.
Table 3
Class
Research participants
A
Teachers:
Bilingual/
English
speaking
English
Student
ethnicities
Student
numbers
enrolled
in classes
27
Student
numbers
completing
project
17
B
Bilingual
C
Bilingual
Afghani,
Somali
Ethiopian,
Somali
Somali
18
7
25
9
D
Bilingual
Ethiopian
8
0*
E
Bilingual
Ethiopian
9
6
F
English
8
4
G
English
14
H
English
Cambodian
Vietnamese
Iraqi
Chinese
HongKong
Vietnamese
Cambodian
Chinese
Indian
Vietnamese
Chinese
Cambodian
12
Contact
time/ week
(hours)
Duration of
course
(weeks)
Time of
course
Mon-Thurs
9.30-12.30
Mon-Thurs
9.00-12.00
Mon-Thurs
16.00-19.00
Wed
17.30-19.30
Mon-Thurs
17.30-20.30
Wed/ Fri
13.30-14.30
20 weeks
Morning
20 weeks
Morning
20 weeks
Evening
20 weeks
Evening
20 weeks
Evening
20 weeks
Afternoon
10
Wed/ Fri
12.15-13.15
20 weeks
Afternoon
9
Mon-Fri
9.00-11.25
20 weeks
Morning
*One student completed the tasks but was excluded from sample as she was the only one in a two-hour bilingual class.
Figure 2
Classes established during research project
The classes involved in the project were:
West Auckland:
3 x 12-hour/week classes, 1 x 2- hour/week class with bilingual teachers
1 x 12-hour/week class with an English teacher
These classes had learners who spoke Amharic, Farsi, Pushto, Tigrigna and Somali as their
mother tongues.
All students in these classes had refugee backgrounds.
South Auckland
1 x 12-hour/week classes
2 x 2-hour/week classes
These classes were taught by an English speaker, and the students spoke Cantonese, Khmer,
Vietnamese, Mandarin and Hindi as their mother tongues. Students were new immigrants.
52
Research participants: Case studies
Several factors influenced the case study sample selection. The research team felt
that it would be most useful to interview students and tutors from the 12-hour weekly
classes, as the more substantial provision was expected to accelerate student progress, and it
was anticipated that availability for interviews would be enhanced. This also facilitated an
exploration of the impact of bilingual and native speaker programme delivery and student
response. In addition, the choice of two contrasting groups - a community-based, Somali
bilingual class for refugees in West Auckland, and a South Auckland group consisting of
students from a variety of language groups and backgrounds - informed the qualitative
analysis of the impact of factors such as the extent of prior education, first language
literacy, ethnic background and the similarity of first language script to that of English.
Thus a class comprising 15 Somali and Ethiopian women studying English language
and literacy in a 12- hour, bilingual day class at Wesley Intermediate School in Mt Roskill
(Class 1) was selected for inclusion in the study. The second class (Class 2), located at the
South Auckland Home Tutor Scheme in Papatoetoe, consisted of men and women from
Vietnam, Cambodia and China. This group was led by a native English-speaking tutor.
The selection of students was based on the typical sampling model devised by Burns
(1994:465). He recommended the selection of three cases representative of the range of
attainment. He argued that these cases would demonstrate the profiles of students who were
least and most likely to succeed. Thus the highest, lowest and average achievers in the first
tests were selected from both classes.
Attention was also paid to ensuring that the students interviewed matched the
balance of ethnicities as well as the gender balance of the entire sample. The major ethnic
groups were Somali and Vietnamese. Three Somalis, two Vietnamese and one Cambodian
students were interviewed. Four women and two men were interviewed.
All of the women selected in Class 1 were Somali 1 . Two were aged over 50 and one
was between 21 and 30 years old. Time spent in NZ ranged from less than one year to
1
The Ethiopian students in Class 1 were excluded from the study as they were unable to participate in the
Somali bilingual component of the programme. Tutors did, however, use Arabic to communicate with them.
53
between three and five years. The women had had no previous education and were from
refugee backgrounds.
In Class 2, two of the participants were male and one was female. Two were from
Vietnam and one was from Cambodia. All had received six to eight years of education
previously and had arrived in NZ less than one year previously. One student was aged
between 21 and 30, one was aged between 31 and 40 and one was over 50 years.
Participants had arrived under the Family Reunification programme.
Research methods
Various research methods were used in this project to enable the gathering and
processing of the comprehensive data needed to answer the research questions.
Personal profiles: Tutors (See Appendix 2)
Personal profiles of the tutors’ backgrounds provided valuable information on the
tutors who were teaching the literacy classes. These questionnaires were self-administered
by the tutors.
The bilingual tutors were bilingual in English and the predominant language of the
class they were teaching. In the tutor profiles, both bilingual and English-speaking tutors
identified the variables known to affect language proficiency among immigrants (age at
time of arrival, length of residence, home language, literacy levels in their mother tongue or
first language, educational background). In addition, they were asked about their
qualifications and teaching experience. Finally, they were asked to state the positive aspects
of the literacy programme they were participating in and the classes they were teaching, and
to identify areas in which they felt they needed further support.
Personal profiles: Students (See Appendix 3)
The use of personal profiles in the form of a questionnaire in the project determined
the background of the students involved in the survey and gave the researchers an
54
indication of some of the variables which may affect student rate of progress. These profiles
were administered verbally by tutors when necessary, with the help of the interpreters.
In the current study, the student profiles gave tutors and researchers access to
student information which helped to inform the methods and content of their teaching, and
which may prove to affect literacy gains. Independent variables included age, gender,
length of stay in NZ, place and period of stay elsewhere, educational background, and
change in occupation. Profiles also gave some indication of the current proficiencies of the
students in their own language, their language learning experiences, and the literacy level at
which they assessed their language skills in their first and subsequent languages. This data,
if it informs teaching, invariably empowers students and teachers to negotiate an
understanding of what is needed and necessary in the curriculum. It also helps researchers
to gather a maximum amount of useful data, to give structure to a study, particularly at the
beginning stages, and makes coding and data analysis easier (see Shameem, 2001).
Self-assessment (See Appendix 4)
Validation of self- report proficiency data is necessary, given the drawbacks
associated with it. In fact, caution in interpreting self-report data has been advised by
several writers who have been involved in research in this area (see for example, Nicholas,
1988; Martin-Jones, 1991; Shameem, 2001). Martin-Jones (1991:50) advises that this is
particularly so in bilingual and multilingual communities, where language use does not
necessarily fall into neat little patterns of complementary distribution across domains.
Nicholas (1988) suggests that crosschecks on self- reported data are especially
desirable in multilingual communities and gives as an example a study of the Creolespeaking community in Britain. In this study, the quality and reliability of the data had
unavoidable limitations because, as with the students involved in the current study, the
Creole-speaking community has an uncertain political and social status in the host
community.
In the self-assessment scales which were designed, the specific areas covered were
the self-reports of reading and writing abilities, which included numeracy as a component.
Self-reported literacy ability was rated on a scale according to the real- life tasks that could
55
be performed at a ‘survival’ and ‘social’ literacy level to enable new immigrants and
refugees to survive in the English-speaking host environment. The tasks were deve loped so
that they represented real- life functions. They were also developed in congruence with the
tasks and topics covered in the curriculum for the literacy programme (see Appendix 5).
Different situations make varying demands on language users, so it was therefore useful to
enquire into the ability to use reading, writing and numeracy skills across a range of
different activities.
The reading and writing scales that were subsequently developed demonstrated the
student’s ability to function in their host environment at a very basic social proficiency
level, but carried progressively greater demands in terms of what they could realistically
achieve within this level (See Tables 4 and 5).
Table 4
English ability: Reading scale
Reading Skill Level 0 – 8
Question
No.*
Skill Level
0
Level Description
No proficiency
(pre-literate)
Familiarity with formulaic
personal words
Performable tasks
-
1
1
I can recognise words on a form I have seen
before and which requires personal
information
I know the names and sounds of all the letters
in the English language.
2
2
3
3
4
4
5
5
Initial reading ability
Familiarity with reading
alphabet
Initial numeracy ability
Familiarity with known
numbers
Minimum reading ability
Familiarity with reading
numerals
Emerging reading ability
6
6
Survival reading ability
I can read and interpret timetables
7
7
Basic reading ability
8
8
Social reading ability
I can find words which are arranged in
alphabetical order e.g. in a phone book,
dictionary
I can read a short and simple story
I can recognise familiar dates which I have
seen before.
I can recognise and read any combination of
numbers in English
I can recognise and read public signs
56
Table 5
English ability: Writing scale
Writing Skill Level 0 – 8
Question
No.*
Skill Level
0
Level Description
10
1
No proficiency
(pre-literate)
Alphabet recognition
11
2
Initial writing ability
12
3
13
4
14
5
Observation of writing
conventions
Minimum writing ability
Initial numeracy ability
Emerging writing ability
15
6
Survival writing ability
16
7
Basic writing ability
Performable tasks
I can write the upper and lower case letters of
the alphabet
I can write my name, address, age and the
names of my family members.
I know when to use capital letters and full
stops
I can write dates – days of the week, months
of the year, and numerals
I can write a short letter of explanation on a
personal matter.
I can take/write down a phone message
I can write a short piece about my personal
experience and life in NZ
17
8
Social writing ability
I can write a personal journal entry and keep
a diary
* These question numbers correspond to question numbers on self-assessment forms and tests.
The self-assessment procedures were conducted orally at three points in time by the
tutors who were teaching each class. Interpreters were used when necessary. One selfassessment was at the start of the programme, the second in the middle, and the final one
was held prior to the final test.
The scale used was a five point literacy one with students circling their perceived
ability on a scale of zero (I cannot do this at all) to five (I can do this very well) in
performing a range of progressively difficult literacy functions. The scale had reading and
writing parts with eight performable language functions in each one (see Appendix 7 for
descriptors).
The three self- assessments were identical. Students did not, however, look at their
earlier assessment of perceived ability when filling out the mid-point and final assessments.
Varying self- report scales differ, depending on whether learners have access to their earlier
responses and perceptions when rating their proficiencies longitudinally. In the English
Language Institute Proficiency Scale (1992) for instance, students of English report their
perceived proficiency during a 12-week intensive English programme three times during
the course. In the mid and end-of-term self-assessments they have access to their earlier
57
reports, so they are able to make a relative judgement of perceived progress. Shameem
(1995) on the other hand, during her longitudinal study of Indo–Fijian teenagers' language
proficiencies, did not supply her respondents with their earlier self-assessments.
Performance assessment (See Appendix 14)
To validate the data that was gathered from the self-report study, a performance test
was designed and administered to all the students involved in the project. As in any kind of
assessment, one can at best make judgements only about the communicative performance
of the respondents, which may or may not be a true reflection of their language abilities
(see Bachman, 1990:37). Hakuta and D’Andrea (1992) acknowledged this problem in their
Spanish/English tests with Mexican-background high school students in the United States,
when the y wrote that the test performance of their respondents could not really indicate
their true potential.
The assessment was influenced by a number of practical and theoretical difficulties
involved in the development of an assessment procedure which would yield valid and
reliable data. The researchers were especially constrained by the literacy levels of the
students involved, since these ranged from pre- literacy, even in their first language, to
varying degrees of survival and social literacy ability in English. Moreover, the students
were unlikely to have ever participated in any form of literacy assessment, consequently,
the procedure had to appear non-judgemental and non-threatening, be closely tied to the
teaching and learning programme, take up a minimum period of time and be easy to
implement by both bilingual and English-only tutors, while being sensitive to the differing
abilities expected among the refugee and first-generation new immigrants participating in
this study. Given that the bilingual tutors may have had little experience in administering
English language literacy tests, instructions and rating procedures had to be clear and easy
to follow and use.
Several training sessions prior to and following the performance
assessments needed to be held in order to ensure test administration and rating were
consistent across all the sites involved.
The test contained the realistic survival functions enquired about in the selfassessment procedures. These were tasks that the students were likely to meet in their dayto-day lives in NZ, which they needed to understand and respond to when the situation
58
demanded. A table of specifications was drawn up at the start of the test-writing process to
ensure congruence between the two performance tests that were devised and which were to
be administered at the start and the culmination of the program of study (see Appendix 6).
The test specifications, which were confidential to the test development team, cover
all aspects of the testing process, including the test design and development, administration,
and rating. It identifies the test takers and implementers, the methods of administration, the
possible content and topics to be covered, the mode of delivery, the objectives and the
rating procedures. Further performance scale descriptors were developed for delivering
comprehensive feedback to students on their performance level and its implications in terms
of directing their future learning (see Appendix 7).
In addition, a test syllabus (see Appendix 8) gave clear criteria on the content,
timing and instructions of the test, and the information was made available to all the tutors
and participants prior to the testing process. Tests were administered by the class tutors
during class time, and it is acknowledged that differences in test environment and test
administrators might have influenced performance on these tests. Appendix 9 gives a tutor
account of the test-taking processes and reactions in a class taught by an English-speaking
tutor. Bachman (1990) refers to social and environmental characteristics which influence
test results as ‘test method facets’, and some of these pertinent variables were used to look
at differences in performance in the study. However, it is difficult to attribute differences in
performance to any one factor, as the situation for each class participating in the study
varied greatly – from the ethnicity, age and educational background of the students to the
level and language of literacy at which they began their programmes. At best we can only
make assumptions about degree of progress, which is indicated by comparing individual
results. All results were recorded on a record sheet (Appendix 10), which made later
reporting and data entry easier.
Case studies (See Appendix 11)
The researcher met with the tutors of both classes prior to the commencement of the
project. The qualitative component of the project was explained, and samples of the
interview questions and the classroom observation schedule were distributed. Discussion
and feedback was encouraged. Once the procedures were understood and agreed to, the
59
tutors were asked to speak to the proposed participants to brief them on the process.
Appropriate interpreters for the student interviews were arranged. Dates were established
for the tutor interviews and classroom observations. Interview and observation schedules
for the case studies are included in Appendix 11.
The first method of data collection was one-to-one, face-to- face interviews with
students via an interpreter. The interview schedule consisted of open-ended questions.
These focussed on the reasons for studying, perceptions of progress and the outcomes of
learning literacy and English in terms of coping with life in NZ. Three students from each
class were selected in accordance with the sampling technique detailed earlier.
The
interviews were conducted in the students’ classrooms and were of approximately 30
minutes’ duration. They were supported by an interpreter in all cases. The interpreter
translated the interviewer’s questions and the participant’s responses. Each interview
commenced with the reiteration of the informed consent process (see section on Ethics
concerns). The purpose of the interview was explained to all students, and assurances were
given regarding their anonymity and the confidentiality of the discussions. Participants
were asked if it was acceptable to tape-record the interviews and, in all but one instance,
they agreed. At the conclusion of the interviews, students were asked if they had any
questions.
Subsequently, tutor interviews were conducted at the two teaching locations. The
English language was used and the conversations were recorded. The questions explored
the tutors’ definitions of literacy, philosophies of English teaching, choice of appropriate
teaching methodologies, anticipated course outcomes and the range of assessment
techniques used. The discussions were generally less structured and more wide-ranging
than the student interviews.
To succeed in efforts to achieve triangulation of data collection, the second research
method used was classroom observation. The observations of Classes 1 and 2 were based
on an adaptation of the COLT model (see Fröhlich & Spada, 1995). This provided the basis
for the observation and analysis of the English and literacy classrooms, and activities
therein. Each class was observed once, for 1.5 hours.
60
Test trial
Trial of personal profiles and self-assessment
They were conducted with all the literacy classes in the first semester of 2001, and
were undertaken during the week 19-23 March.
Changes made
The personal profile was looked at by a statistician and changes were made to
ensure responses were more consistent and easier for coding and analysis. The 0 – 5 selfassessment needed to be reversed to fit with performance assessment scoring procedures,
and the number of functions that students were asked to rate their abilities on was reduced.
Trial of performance assessments
Also in the first semester, a trial of performance tests and procedures was conducted
with classes at UNITEC during a short, ‘English Plus’ course in Reading and Writing for
Farsi speakers, taught by a native English tutor and a Farsi-speaking tutor. The students
were Farsi speakers from Iran and Afghanistan. The course was eight weeks long and ran
for eight hours per week. It was for refugees who did not have access to mainstream
courses because of their low literacy skills in English. The aim was to prepare them to
move into elementary classes.
The reading and writing trials were conducted on 19th and 20th June, 2001. The
tutors were trained in test administration procedures prior to the testing process. On each
occasion, the bilingual tutor conducted half the test, followed by the native-speaking tutor.
Changes made
For this trial, the general instructions for all the tests were given at the training
session, with specific instructions given again before each test. The tutors, however, did
not refer back to the general instructions, which led to some inconsistencies. Therefore all
61
the instructions were included on the tutor’s page for each test. For every test, a model
answer was included and testers were instructed to refer to these.
Reading test (see Appendix 14)
Test Item 1: the instructions for this were re-done because students wrote the answers in
full, instead of writing in the number only.
Test Item 2: this was re-done to make 90% of it an exercise that could be done as a class
(rather than all of it being done individually).
Test Item 4: the scope of this was reduced to include a narrower range of questions and
expected responses.
Writing test (see Appendix 14)
Test Item 10: the instructions were expanded, as the bilingual tutors were not familiar with
this process.
Test Items 14,16,17: the marking criteria and process were clarified.
Test Item 15: the tape and exercise were redone to make it more realistic for this level.
Ethics concerns
Ethical considerations were particularly important in this survey, since the literacy
project involved learners who came from disadvantaged, minority- language backgrounds;
had suffered great personal losses in their exodus from their home countries, often via
refugee camps in other, perhaps hostile, nations; and who might continue to suffer from
post-traumatic stress syndrome. Many were pre-literate even in their first language and
lingua franca, and had very little formal education in their background. For many, the
transition to NZ culture and society, and the effort of learning a new language, even for
aural and oral communicative purposes had been recent, painful and difficult. Consequently
their time in NZ was a story of financial struggle, further social upheaval and great
adjustments. Given that the students who participated in the project might be continuing to
face feelings of powerlessness and anomie, it was important that they be treated with
openness, sensitivity and honesty. The project was given the approval of the UNITEC
Research Ethics Committee after they had studied all aspects of the design of the
instruments and the methods to be used.
62
Information sheets (See Appendix 12)
Informed consent is one of the most fundamental ethics requirements, and the
student and tutor participants in the project were first of all given sufficient information on
the background and purpose of the research to enable them to make a decision whether to
participate in the project.
The information sheet explained to participants their own roles in the project; who
else was involved; and the methods that would be used to collect information. It promised
confidentiality and gave contact numbers for each of the researchers who were involved, in
order to enable participants to contact them for further clarification if they so wished.
All the information on the sheets was explained verbally to the student participants
by interpreters and bilingual tutors prior to the commencement of the project.
Tutor
information was given in writing to the tutors. This also explained the background and
purpose of the project, but more specifically - so that tutors were able to understand their
own roles but also the function and purpose of the research instruments - the personal
profiles, the case studies and classroom observation components. Responders were advised
about how and when they would receive the results of the assessment procedures.
Consent forms (See Appendix 13)
Once the students and tutors had thoroughly understood the background and
implications of the project, they were asked to sign consent forms, to make their
participation official and to safeguard the interests of the participants and researchers.
Students and tutors were assured anonymity and were told that all names used in the
project would be fictitious or would use untraceable identity numbers. All consent forms
were returned to the researchers through the tutors and have been filed as required by the
UNITEC Research Ethics Committee for a period of at least six years.
63
Issues arising that imposed limitations on the project
Continuity of students
Students signing the consent form indicated their willingness to attend classes and
take the required assessment tests. However, the classes were drawn from people who live
with many external pressures and a lot of uncertainty in their lives. There was a high risk
of numbers falling off during the duration of the research project. On the other hand, the
target number for the research was 50 and 118 students had enrolled on the programme.
Availability of bilingual tutors
Four of the classes were taught by bilingual EAL tutors. There is a very limited
pool of such tutors and if any of the tutors had dropped out, it is unlikely that we could have
replaced them with another bilingual tutor. The class would have had to be taught by a
native speaker, and this could have affected attendance and the outcomes of the class.
Fortunately, this was not the case, as tutors remained with their original classes for the
duration of the project.
Time of day constraints
It was difficult for literacy tutors to be available for the possible hours during the
day. Literacy teaching tends by nature to be part-time and it is difficult for tutors with
families to commit to part-time in secure work as a priority over more secure employment.
The classes offered in the day had the highest number of women learners attending.
Ramadhan
The time frame for the whole project was not ideal. To meet project deadlines,
classes ran from July to December. Ramadhan in 2001 occurred between November 17th
and December 16th .
The daytime fasting and related evening festivities affected both
attendance and progress, especially for the daytime classes. Although adjustments were
made with assessment schedules, because of the 30-day Ramadhan period, this was not
always easy to do.
64
Cultural and social resistance to self-assessment and testing
There was some resistance to self- assessment because of the unfamiliarity of the
method and its need to "speak well of yourself". Students were not used to being 'tested' as
such and therefore maintaining a 'test' environment during the formal testing sessions was
extremely difficult - tutors reported a lot of 'talking' going on (despite attempts to stop
them) during the test sessions and bilingual tutors reported student attempts to 'collaborate'
to show that they were competent in the set tasks (see Appendix 9). However, there seemed
to be less collaboration in the final tests than in the initial ones showing that students had
learnt something about the test-taking process over the 20 weeks.
During the final tests too there was a death in the Somali community, and
consequently tests were held a week later for the half that had missed the final tests. There
would undoubtedly have been some cross-checking between the two groups regarding the
content of the test questions, as the class was closely-knit.
Data coding and analysis
The data was analysed for the classes involved in the study, and combined for
pertinent computations using the SPSS software. Analysis of the data was descriptive and
quantitative regarding the social and linguistic background of the students and their tutors,
student self-assessed literacy ability and test results. For the case study and observation
portions, data was analysed qualitatively. The key quantitative tests used were:
-
The Kruskal Wallis test for ordinal data, which determined the significance of
the relationship between literacy levels and the major variables believed to
affect literacy, such as age at immigration, length of NZ residence, current age
and gender.
-
The Wilcoxon Matched-Pairs Signed-Ranks test for non-parametric data, which
matched the three self- reports of literacy and then compared them to
performance in the initial and final tests for individuals. This was to establish the
validity of the self-report scales. This test also established whether high literacy
levels in the first language were matched with higher literacy achievement in
English.
65
-
The Fisher Exact Chi-square test for nominal data determined the existence and
strength of association between two variables, for example, home language use
and age or length of residence.
-
Spearman Correlation Coefficient test which measured the strength of the
relationship between two variables, for example, reported and assessed literacy
levels at the beginning and end of the programme.
-
Independent Samples T-test compared means for two groups of cases. The
subjects were assigned to two groups so that any difference in response was due
to treatment and not to other factors. For example, assessment results of a group
taught by bilingual tutors were compared to a group taught by English-speaking
tutors.
-
Paired Sample T-test compared the means of two variables of a single group. It
computed the differences between values of two variables for each case and
tested whether the average differed from zero. For example, pre-and postprogramme test results for a group of students.
A p-value of 0.05 was the standard measure for statistical significance in this study.
Summary: Methodology
The most important aspect of the methodology used in this project was the
formulation of the aims, and subsequently, the research questions. The hypotheses were
formed by evaluating the research questions in relation to the nature of the programme and
the research participants.
The research instruments included a student personal profile which also functioned
as a needs analysis of the research participants, and a tutor profile for both Englishspeaking and bilingual tutors. Next, self-assessment forms were designed so that students
could initially evaluate their literacy ability on a range of increasingly-difficult authentic
'survival' literacy tasks. Self-assessments were conducted at three points in the programme beginning, middle and end - to look at changes in perceived progress or otherwise.
Two parallel performance tests with tasks which were congruent to those on the
self-assessment forms, were designed, developed and trialled. These were administered at
66
the beginning and end of the literacy programme. In addition to the assessment measures,
case studies were cond ucted in two classrooms at the West and South Auckland sites. One
of these was led by a bilingual tutor and the other by a native English-speaking tutor.
The final component of the research methodology was the use of an observation
schedule to note differences and similarities in the two case study classrooms. Ethics
concerns were of utmost importance in this project, and information sheets were given out
to all participants, who subsequently signed the accompanying consent forms.
Data
analysis took both descriptive and analytic procedures into account.
67
68
Sunday Star Times (ellison)
69
RESULTS
TUTOR AND STUDENT PROFILES
Introduction
The results are discussed in four main sections. The first section profiles the tutors
and student participants of the survey. The second section looks at the results of the selfassessments completed by the students at the beginning, middle and end of the project
period of 20 weeks. The third section compares the results of the parallel initial and final
performance tests, which were developed by the researchers and implemented at the
beginning and end of the project period. It then compares the initial and final selfassessments with the performance test results to determine the validity of self-assessments
when used with the low- level literacy and pre- literate learners who participated in the
survey.
The final fourth section reports the results of the case study interviews and
classroom observations.
Section One: Tutor Profiles: Background
Eight tutors participated in this study; two were native speakers of English, 5
bilingual tutors who had undergone tutor training at UNITEC Institute of Technology in
1999; one had undergone TESOL training in Australia. The two native speakers identified
NZ as their home country, three bilingual tutors were from Somalia and three were from
Ethiopia. The two native speakers were born in NZ; the others had arrived at varying ages–
only one (from Somalia) having arrived as a teenager. A majority of the bilingual tutors
(four of the six) had arrived in NZ when they were between 31 and 40 years of age and only
one when he was over 41. Half the tutors were male. Table 6 (below) outlines the home
country, gender and age of arrival of the eight tutors. The names used are fictitious, in
keeping with UNITEC’s research ethics requirements.
70
Table 6
Tutor profiles
Name
Gender
Home Country
Arrival Age
Jane
F
New Zealand
Born in NZ
Joan
F
New Zealand
Born in NZ
Siffat
F
Somalia
Less than 20 years
Amira
F
Somalia
31-40 years
Hamid
M
Somalia
31-40 years
Khizer
M
Ethiopia
31-40 years
Ameen
M
Ethiopia
31-40 years
Reyhan
M
Ethiopia
41-50 years
Language proficiency data among immigrants shows that age at time of arrival is a
significant factor in contributing both to language acquisition of the host language and
language shift of the mother tongue (Clyne, 1975; Shameem, 1995). Generally, those who
arrive in the host community before 10-12 years of age tend to replace their mother tongue
skills with the host language. This was not likely to be the case with the bilingual tutors
participating in this project, as the youngest, Siffat, was older than the threshold age of 1012 years when she arrived here. Given this scenario, the concern would be that host
language acquisition was at the point where the tutors were comfortable in order to teach
English language literacy rather than be concerned about mother tongue maintenance. It
would be more likely that host language acquisition is linked to age and length of residence
in an English-speaking environment. This data is described in Table 7 (below).
The tutor who had arrived in NZ as a teenager was now in her 20s. The other tutors
were all over thirty years of age. One of the two over 50 and one of the three in their 40s
were native English speakers. This being the case, it could be expected that all the tutors
would be fluent users of their first language which, considering all are currently literacy
tutors, would include literacy ability.
The question related to their first language
proficiency in all four skills was asked later in the questionnaires.
71
Table 7
Tutor age and length of residence in NZ
Tutor
Jane
Joan
Siffat
Amira
Hamid
Khizer
Ameen
Reyhan
Age
Over 50 years
41-50 years
21-30 years
41-50 years
41-50 years
31-40 years
31-40 years
Over 50 years
Length of residence
More than 8 years
More than 8 years
6-8 years
Less than 1 year
6-8 years
3-5 years
3-5 years
3-5 years
Only one of the tutors, Amira, from Somalia, had lived in NZ for less than a year.
Three of the bilingual tutors had been residing in NZ between three and five years, and two
between six and eight years. Only the two native English-speaking tutors had lived in NZ
for a period longer than eight years. These figures indicate that a majority of the bilingual
tutors had had time to settle into the NZ environment, and had some knowledge of host
community culture and systems in order to impart more than just literacy skills to their
students. This was particularly important in a classroom environment where the majority of
learners were refugee women with limited English language ability and very little access to
knowledge about the host environment. Bilingual tutors who are comfortable in the host
environment are ideally placed to give all-round instruction, knowledge and practice, based
on their own linguistic and contextual knowledge and experience.
Language use and proficiency
Language maintenance data in NZ shows the home to be the last bastion of mother
tongue use, with its declining use in the home being indicative of language shift. In the
case of the bilingual tutors, this would indicate language shift within the first generation
and in the older age bracket–the group that is, in fact, most likely to be the vehicle for first
language maintenance for later generations, which are more susceptible to language shift.
The data from the project was particularly interesting in that it showed a shift in language
use within this first generation (see Table 8). While the data did not gather information on
the specific reasons for which this was occurring, this would be worthy of further study as
language maintenance data has not yet been systematically gathered for first-generation
refugee immigrants.
72
Table 8
Home language use
Language
Mainly English
Both English and my language
Mainly my language
Frequency
2 (NZ-born tutors)
4
2
Four of the six bilingual tutors were using both English and their own language at
home. Of the two bilingual tutors who continued to use mainly their own language, one
was in the 31-40 year age group, the other was in the 41-50 year group.
An important issue in literacy study and research is that of the writing script used in
the first language, and whether it uses Roman letters - making it easier to learn literacy
skills in English. Of the six tutors who had first languages other than English, the three of
Somali background used the English script, while the two who had Amharic and one who
had Tigrigna as first language (and listed Amharic as their second) used a different script.
Hence, the latter three who taught students from their own backgrounds would probably
have greater difficulty with teaching literacy skills as writing the alphabet would be a skill
they would specifically teach, while Somali tutors may not have to with their literacy
students. Although with pre-literate students the issues would be the same, Somali students
would probably still have a greater ability to recognise the alphabet if they have been
exposed to any written material such as public signs in their first language background.
The tutors who participated in the project had a diversity of languages in their
speech repertoire. Only one tutor was a monolingual English user. The bilingual tutors
ranked English as their second, third or fourth language (see Table 9).
Levels of proficiency in their second language differed. Of interest here was that the
two Amharic first language users listed Tigrigna as their second language with only one of
them stating their level of proficiency as 5 in all four-skill areas. The tutor who used
Tigrigna as his first language was equally proficient in Amharic, his second language. The
nature of bilingualism, and the purposes for which these two languages are used would
make worthwhile study among the Ethiopian refugee population.
73
For most of the Somali tutors, English was the second language, with one
nominating Italian as their second language. Arabic also featured as a third or fourth
language with four of the tutors, testifying to the important role of the religious language in
their speech repertoire. As NZ language data shows, for maintenance purposes and the
learning of languages other than English, immigrant parents often prefer their children to
learn the religious language (for example, Sanskrit, Urdu, Arabic) rather than their first
language (Shameem, 2001). Whether this will be the case among the NZ Islamic refugee
population has yet to be established, and has obvious implications for the maintenance and
roles of Somali, Amharic and Tigrigna.
Other languages known by the tutors at Level 1 in various skill areas included
French, German and Hindi. Table 10 outlines the means and standard deviations of the
various languages known by the tutors.
As Table 10 shows, the mean value in all four skill areas in the first language is
consistently placed at 5, with a standard deviation of 0. The mean values vary in subsequent
languages, with the lowest values recorded for the literacy skills. Figure 3 graphically
represents the differences in mean proficiency levels over the four skill areas in the first
three languages known by the tutors.
74
2
Write
1
1
Arabic
4
4
0
0
Speak
Understand
Read
Write
4
5
4
4
English
5
5
5
5
Amharic
5
5
5
5
Tigrigna
Khizer
Fourth language
1
3
Read
1
Write
3
Understand
1
1
3
Speak
French
Read
English
Second language
5
1
5
Write
5
Understand
5
Read
5
1
5
Understand
5
Speak
5
Speak
English
German
Somali
First language
Jane
Third language
Siffat
Tutor
5
5
5
5
English
Joan
3
4
4
4
English
5
5
5
5
Tigrigna
5
5
5
5
Amharic
Reyhan
Key: 5 = Very well , 4 = Well, 3 = Average, 2 = So-so, 1 = A little, 0 = Not at all
75
-
-
1
1
Hindi
5
5
5
5
Arabic
5
5
5
5
English
5
5
5
5
Somali
Amira
5
5
5
5
English
0
0
4
4
Arabic
3
3
4
4
Tigrigna
5
5
5
5
Amharic
Ameen
4
3
2
3
English
4
4
3
4
Arabic
4
4
4
4
Italian
5
5
5
5
Somali
Hamid
Table 9
Language proficiency in languages known by tutors on literacy project
Table 10
Proficiency
Speak
Understand
Read
Write
Figure 3
Language proficiency
Mean
St. dev.
Mean
St. dev.
Mean
St. dev.
Mean
St. dev.
N =
First language
5.00
0.00
5.00
0.00
5.00
0.00
5.00
0.00
8
Second language
3.86
1.46
3.86
1.46
3.71
1.50
3.57
1.62
7
Third language
3.67
1.37
3.50
1.38
3.17
2.14
2.83
1.94
6
Mean proficiency levels of tutor languages
Mean proficiency of tutor language
First language
6
Second language
Third language
5
4
3
2
1
0
Speak
Understand
Read
Write
Skills
Of particular concern in this project was the level of proficiency in English of the
tutors, as they were involved in English literacy teaching. These levels are represented by
the graph in Figure 4. Four of the tutors rated their English proficiency levels at 5 for all
four skills. Two of these tutors were native English-speakers. The remaining four bilingual
tutors reported varying levels of English language proficiency in the four skills. While the
76
graph represents reported proficiency level, it may also be indicative of the level of
confidence that tutors have in their own abilities.
Figure 4
Tutor English proficiency
English proficiency of the tutors
6
Speak
Understand
Read
5
Write
4
3
2
1
0
1
2
3
4
5
6
7
8
Tutor
Educational background
All eight tutors had attended an Institute of Technology in NZ. For the six bilingual
tutors this meant attendance at UNITEC, to participate in the bilingual tutor training
programme (see Table 11).
77
Table 11
Educational level of tutors
Educational background
Frequency
Primary school
Secondary school
Institute of Technology
University
Other
2
8
3
2
Two of the tutors indicated that they had attended secondary school in other
countries, and three had been to university. One of these was a native English-speaking
tutor.
When asked what qualifications they held, the tutors indicated a wide range (see
Table 12 below). Most prevalent was a certificate, and these were certificates in language
teaching, teaching literacy, and teaching. All the tutors had some qualification in teaching
or language teaching, with one bilingual tutor having completed an MA in TESOL.
Table 12
Qualifications of tutors
Qualifications
1,2
1,2,3,4,
2
2,3
2,4
Total
Frequency
1
1
3
1
2
8
Percent
12.5
12.5
37.5
12.5
25.0
100.0
1=Informal qualification, 2=Certificate, 3=Diploma, 4 = Degree
The tutors also stated the length of any previous teaching experience they had, and
the locations at which they had taught. This data was quite difficult to code and analyse
quantitatively, as the nature, length and location of teaching experiences varied widely even
for individuals. Hence, for the purposes of this report, the data has been summarised into
areas of teaching experience, the average years spent in each by the eight tutors, and the
location at which the experience took place (see Table 13).
78
Table 13
Teaching experience of tutors on literacy project
Teaching experience
Primary school
Secondary (Eng. Maths)
Language teaching
Teacher trainer /headteacher
Literacy teaching
Adult: individual/group
Number of
tutors
involved
4
1
3
2
8
3
Average
years per
tutor
7.5
25.0
4.0
6.75
2.5
2.0
Experience location
Somalia, Sudan, NZ
Somalia, India, UAE
NZ
Sudan, UAE
NZ (Auckland)
Sudan, NZ
Three bilingual and one native English-speaking tutor have had primary teaching
experience. The tutor with secondary experience has taught English throughout her career
and has supervised and trained teachers. Another bilingual tutor was a headmaster of a
primary school.
Both native English-speaking tutors have ESOL teaching and home
tutoring experience, and one of these has also been an adult literacy tutor. One bilingual
tutor has taught mother tongue language maintenance to children in NZ as well as adult
education in health and hygiene in the Sudan. Only one bilingual tutor has had no previous
teaching experience.
The data showed the wide range of educational backgrounds, qualifications and
teaching experience among the eight tutors involved in the project. With this range, the
tutors bring with them a strong knowledge and skill base which is at their disposal in
teaching the low- level and pre- literate students that they are currently involved with.
Issues arising from teaching current literacy classes
The tutors responded to four questions on their attitudes towards the work they were
involved with. Firstly they were asked which aspect of tutor support they found useful;
secondly, what further support they needed; third, which aspects of the course they had
enjoyed; and finally, the problems they had encountered in teaching the course.
The
qualitative data was coded quantitatively into similar responses, and then ranked in order of
importance, with majority to minority responses (see Table 14).
79
Table 14
Tutor issues in current literacy classes
Ranking
1
Useful tutor support
Meetings and
resources
2
Teaching methods and
equipment
3
Encouragement and
motivation
Level of support for tutors
Support needed Enjoyed
Increased hours Helping
and different
community
classes
Funding and
Student
further inattendance and
service training daily progress
Adequate
All aspects
support already
given
Problems
Student
forgetfulness
Curriculum and
testing
procedure
Student
attendance
Tutors found the regular meetings with fellow tutors and the course co-ordinator of
greatest value in terms of support available to them. Second, they found their knowledge of
teaching methods and the equipment that was available for them to use useful. Finally,
tutors felt that the moral support and encouragement they received from the course coordinators and each other were useful too.
In terms of support they continued to need, tutors wanted to teach an increased
number of hours in each class, as they felt current provisions to be inadequate for reaching
literacy goals.
They also wanted to teach classes other than the ones they had been
allocated in order to give themselves a wider range of experience teaching at different
levels. Next, the tutors felt they needed funding in order to offer classes of a greater
intensity and with extended hours. They also wanted to feel that the classes had more
‘performance’ opportunities, such as the production of written and spoken language in
realistic contexts for meaningful purposes, in order to ensure continuity of teaching points
and learning. Two tutors felt that the current level of support was adequate.
Tutors enjoyed several aspects of the course, but most importantly, they liked
helping and empowering the members of their community acquire literacy skills. Next, they
appreciated the response they got from their students through their regular attendance at
classes and perceived daily progress. Thirdly, tutors stated that they enjoyed all aspects of
the course. As one tutor wrote on her questionnaire:
80
Before this year, my ESOL teaching has been two times one-hour lessons per week, per class. This
course has been a wonderful opportunity for me to teach for a longer duration on a daily basis. It has
been most rewarding to see the students’ daily progress, and to have the time to consolidate their
learning and to respond to the needs of the class as they arise. It has been pleasing to see all students
develop greatly in both confidence and oral communication (as well as literacy)! The students have
been very supportive of one another, and there has been a very pleasant team spirit.
The final question on the questionnaire required tutors to list the problems they had
on the course. A majority of tutors found student forgetfulness of skills and content a
difficulty. This may be partly due to the limited number of hours on the programme,
especially for the 2-hour classes. Next, tutors raised the issue of having cle ar curriculum
guidelines across all the classes. Although the current curriculum is literacy-based, tutors
felt they needed specific links to be drawn with topic, task-types and clear outcomes. The
testing procedures were also problematic for the duration of the project, mainly because of
test-taker unfamiliarity with tests in general and therefore lack of understanding about
content, purpose, methodology, and processes of testing. Ranked third under problems was
student attendance, and this was signalled too by the rate of student attrition during the
course of the project.
Summary: Tutor profiles
Eight tutors - two native English-speakers and six bilingual speakers - participated
in this project. All bilingual tutors had arrived in NZ as adults and had lived in NZ for
varying periods of time; the shortest period was a year and the longest, six to eight years.
Language use data showed some shift in the use of language at home among the
participating tutors. While four of the tutors (two of them native English-speaking tutors)
spoke their first language at home, four other tutors (all bilinguals) said they used their first
language in combination with English. This data warrants further investigation, as it is
fairly unusual for a newly immigrant adult population to shift so rapidly in their use of the
home language.
Language proficiency data showed that while all the tutors rated their proficiency at
the highest Level 5 in all four skill areas (speaking, understanding, reading and writing) in
81
their first language, proficiency in their second languages varied considerably. English was
the first language for the two NZ-born tutors but was the second, third or fourth language of
the bilingual tutors. The bilingual tutors' proficiency in English varied, with two rating
themselves at Level 5 in all four skill areas.
The data on the tutors' educational background and experience showed they were all
well–qualified, with at least a certificate in the teaching field, and considerable experience
in teaching in different contexts. This showed that tutors were adaptable and practised, and
would be able to bring their knowledge - base to benefit those in the community who had
been less fortunate.
When asked about the main issues arising from their teaching, the tutors revealed a
high degree of enjoyment and motivation in teaching the literacy classes. A majority felt
the support they had received was adequate, and particularly so in the area of co-ordinated
meetings; quality and quantity of available resources and equipment; knowledge about
teaching methods; and the encouragement and motivation they had received from their
institutions during their teaching period. Some concern was expressed about the continuity
of funding, the transient nature of classes, the inadequate number of hours spent each week
in the classroom, and the need for further in-service training.
A majority of tutors expressed their satisfaction at being able to help less- fortunate
members of their communities to gain knowledge and literacy skills in the NZ context.
They enjoyed watching them make progress in their learning, self-confidence and self esteem. However, some concern was expressed about student ‘forgetfulness,’ which may
be partly due to the nature and duration of the classes. Lack of knowledge of learning
strategies and methods of retaining learning points are also issues with pre-literate learners.
Some tutors felt the curriculum needed further work, with clearer outlines, objectives and
outcomes linked to specific topics and task-types. The test procedures, too, were difficult
because of student unfamiliarity with them. Regular attendance of students during the
whole programme was a concern, which is reflected in the number of students who
participated in the project and those who completed all aspects of the research and
assessment process.
82
Section Two: Student profiles: Background
A total of 118 students enrolled in the classes which participated in this project. Of
these, 30 students enrolled in the 2-hours-a-week class and 88 students in the 12–hoursweekly class. Only 62 students of the total were eligible as research participants however,
being the ones who completed most aspects of the research, the personal profiles, the three
self-assessments and the two performance tests. Of these 62 students, 14 were enrolled in
the 2-hour weekly classes, and 48 in the 12-hour ones. The final writing tests were
completed by 55 students. Those students who were ineligible had either dropped out of the
programme of study, or had joined late and therefore, had not done the first test. Thus, all
data analysis included the 62 students who fulfilled a majority of the requirements of the
project. Student personal profiles were completed at the beginning of the project period.
The personal profile questionnaires indicated some of the literacy difficulties that
students at this level face, as some of the data was contradictory and in some cases not
supplied at all. This was despite the use of interpreters and bilingual tutors to ensure as full
and accurate a response as possible. Any needs analysis questionnaire to be used with low
and pre- literate students should therefore be as simple and brief as possible. The research
needs of this current project, however, necessitated the use of a more extensive profile.
Countries of origin and gender
The 62 participants listed seven different countries of origin (see Table 15). A
majority of the students were from Somalia (43.5%), followed by 11 students who were
from Vietnam. Other countries of origin were Afghanistan, Ethiopia, Cambodia, China,
and one student was from Hong Kong. Of these students, three quarters of the sample (47
students) were women, attesting to the greater demand for literacy education amongst
refugee women.
83
Table 15
Home country
Afghanistan
Cambodia
China
Ethiopia
Hong Kong
Somalia
Vietnam
Total
Countries of origin of research participants
Frequency
5
7
4
7
1
27
11
62
Percent (%)
8.06
11.29
6.45
11.29
1.61
43.55
17.74
100.00
Age, age at arrival and length of residence
Nearly a third of the students (19) were aged between 31 and 40 years, and 17
students were over 50 years. In fact, the statistics indicated a greater demand for literacy
education among the older age groups, with only two students being less than 20 years of
age (see Table 16). 80.6% of the participating students were over 30.
Table 16
Age
< 20 years
21 – 30 years
31 – 40 years
41 – 50 years
> 50 years
Total
Age of research participants
Frequency
2
10
19
14
17
62
Percent (%)
3.2
16.1
30.6
22.6
27.4
100.0
As well as age being a significant factor in language acquisition, two further factors
which may influence this are age at time of arrival and length of residence in the host
country. Both of these were indicated by the research participants (see Tables 17 and 18).
84
Table 17
Age at time of arrival
Age at arrival
< 10 years
10 – 15 years
16 – 20 years
21 – 30 years
Over 30 years
No response
Total
Frequency
3
7
8
17
26
1
62
Percent
4.8
11.3
12.9
27.4
41.9
1.6
100.0
Age at arrival in the host country is of particular importance, since generally, the
younger immigrants are quicker to adapt to the host environment. This is usually because
they have more contact with host community members, educational institutions and
systems. On the other hand, those who immigrated when they were older are better able to
maintain their own language and culture while they are acquiring the host language. Clyne
(1975) believes the cut-off age is about ten years, with those who are younger than ten at
immigration often acquiring English to a native- like level, but at the cost of productive use
of their home language. Only three of the survey participants had been less than ten years
of age when they immigrated and over two thirds of the participants had been over the age
of 21. Hence, an all-out effort to acquire English language and literacy would not be likely
to hinder language maintenance among the research participants, although the passing-on of
mother-tongue skills to the next generation may still be affected if English was being used
in the home environment.
Table 18
Length of residence in NZ of participants
Length of residence
< one year
Frequency
12
Percent (%)
19.4
1 – 2 years
25
40.3
3 – 5 years
18
29.0
6 – 8 years
3
4.8
> 8 years
3
4.8
No response
1
1.6
Total
62
100.0
85
A majority of the participants had arrived in NZ within the last five years, with only
six of them having lived here for longer than that (Table 18). This data is in line with NZ’s
policies on accepting refugees, in particularly over the last five years, and for making
educational provisions for them.
Those who have lived in NZ for a longer period of time might also have had some
access to other types of classes by now; are now more comfortable in NZ; or already have
some form of employment and no longer feel the need to enrol in literacy classes.
Change in socio-economic status
A majority of studies among immigrant groups in NZ have indicated a change in
socio-economic status between the country of origin and the host country among research
participants.
Studies looking at socio-economic status have generally dealt with new
immigrants, rather than with refugees, who have originated from countries with greater
political and social problems tha n new immigrants have, on the whole. The scale used in
this study was similar to the one used by Shameem (1995, 2001) in her studies of new
immigrants and minority groups in NZ. The scale used was:
1. Professional: Managerial or senior position in a company; ex-principal; accountant;
lawyer; primary or secondary school teacher; dentist; school administrator;
lecturer; educator; development officer; computer programmer; priest or monk;
professional sportsperson.
2. Self-employed: Working proprietors’ own business - dairy, shoe shop, taxi or florist
owner/operator.
3. White collar office/field workers: Clerk; bank teller; office assistant; cashier; bilingual
receptionist; secretary; computer operator; nurse; insurance salesperson; bilingual
worker; translator; tutor.
4. Skilled/trade: Carpenter; upholsterer; plumber; plasterer; mechanic; welder; electrician;
tailor.
5. Semi-skilled: Nurse-aid; shop assistant; check out operator.
6. Casual: Factory hand; packer; mail sorter; fruit picker; harvesting; unspecified.
7. Other: Student; unemployed; retired.
86
Figure 5 illustrates the change in socio-economic status between the country of
origin and NZ. While nearly two thirds of the participants (66.1%) had been unemployed
in their country of origin, 93.5% of them were now unemployed in NZ. This is illustrated
by level seven in Figure 5 below, which shows the sharp increase in rate of unemployment
between the two countries.
Figure 5
Change in socio-economic status
60
N
u
m
b
e
r
o
f
p
a
r
t
i
c
i
p
a
n
t
s
Occupational change for participants
50
40
30
20
Home Occupation
NZ Occupation
10
0
1
N = 62 participants
2
3
4
Occupational category
5
6
7
In fact, a recent stud y by White et al. (2001) found that a majority of their research
participants wished to further their English language acquisition in order to increase their
employment prospects. This is not surprising, given the data on this current project.
The 21 research participants on this project who had worked had done so in a
variety of capacities in their home countries. This ranged from being professionals such as
teachers or sportsperson (three respondents); being self-employed as shopkeepers or
farmers (five respondents); being employed as white collar workers (officers, office
87
assistants); and being employed in a trade or skilled position. Some of the respondents had
held semi-skilled or 'casual' positions. Unfortunately, since the respondents were not given
a list of possible occupations, housework and farmwork at home were not nominated as
possible occupations by the respondents, so that the ‘other’ category (seven) or the ‘casual’
category (six) do not include these possibilities, which earlier projects had (see Shameem,
1995, 2001). If respondents were not in paid work, they were categorised at level seven. If
they said they had worked in their home countries, but did not specify in what capacity,
they were placed at level six.
Only four participants had work in NZ. Two of these had full-time work and the
other two part-time. Of those with full- time work, one was involved in ‘harvesting’ the
other was a welder. The part-time worker was an interpreter; the other did not specify what
work they were involved in. Three of the four had been employed for less than a year; the
welder had been employed for six to eight years.
The socio-economic data showed that unemployment was high in this group in NZ
and, regardless of their hopes in their adopted country, the rate of employment was still
higher than it had been in their home countries, which they had left for a variety of political
reasons. This undoubtedly would have a further adverse effect on new immigrants who are
already suffering from low self-esteem and a sense of dislocation, as they cannot
immediately perceive a positive consequence of their immigration.
Educational background
The educational backgrounds of the participants indicated whether they had some
experience of learning, and perhaps literacy, prior to their leaving their home country. Just
over half the respondents (53.2%) had attended school in their home countries, with almost
a third of them having done so for between six and eight years (see Table 19).
88
Table 19
Duration of schooling in home country
Duration
< 1 year
1 – 2 years
3 – 5 years
6 – 8 years
> 8 years
No school
Total
Frequency
2
3
7
19
2
29
62
Percent (%)
3.2
4.8
11.3
30.6
3.2
46.8
100.0
The data shows that only two of the participants had been to school for longer than
eight years, and twelve had been for less than five. Hence, a majority of the respondents
had either not been to school at all (46.8%) or had attended only primary schools prior to
their leaving their home countries. The most common reason given by the respondents for
not attending school was that either a school did not exist, or they were stopped from going
to school in order to attend to household and farming chores at home (18 respondents).
One respondent said war had prevented him from going to school, while another said their
nomadic lifestyle had been the barrier. Only 20 gave a reason for not going to school.
When participants were asked if they had studied any English before coming to NZ
(see Table 20), a quarter of the research sample (16 respondents) said they had, but of these,
thirteen respondents had in fact learnt English for less than two years. This period may have
been sufficient to pick up some basic communicative skills, depending on intensity of the
course, but hardly enough to learn more than very rudimentary cognitive and literacy skills.
Only three students said they had spent longer than three years learning English.
One of these was from Somalia (3 – 5 years of study), who rated her proficiency as ‘three’
on a scale of zero to five (highest) across the four skill areas. The other two were from
Vietnam, but they reported minimal proficiencies (mainly Levels Two and One) in the four
skill areas.
89
Table 20
Duration of English study in home country
Duration
< 1 year
1 – 2 years
3 – 5 years
6 – 8 years
> 8 years
No English study
Total
Frequency
8
5
1
1
1
46
62
Percent (%)
12.9
8.1
1.6
1.6
1.6
74.2
100.0
Language use in NZ
Various aspects of language use are discussed in this section: the reported
differences between the mother tongue and first language; language use at home in NZ and
the variables which seem to affect this; English language classes that students have taken
previously in NZ, their venue and length; whether they have been taught by a bilingual tutor
and if they found it valuable; perceived host community attitudes to their mother tongue;
and the reasons why they wish to learn English in NZ.
Mother tongue and first language
The data clearly indicated the importance of the distinction between mother tongue
and first language among ethnic minorities. When the participants were asked to nominate
their mother tongue, six of the total 62 responses did not match up with the later question
which asked them to identify their first language and state their level of literacy in this (see
Table 21).
Two participants who identified Dari, Pushto as their mother tongue when asked
about their literacy ability in their first language, elected only Pushto in their response.
Similarly, the three students from Ethiopia who had identified Tigrigna as their mother
tongue did not mention this at all in their response to the first language literacy question, for
which they elected Amharic. One student who had nominated Mandarin as his mother
tongue, called Cantonese his first language. A clear distinction between the two concepts is
important if we are to reach a greater understanding of the perceived and actual differences
90
between mother tongue and first language use (see Shameem, 2001:9, on this). It could be
that students, when identifying their mother tongue, are among minorities literally
identifying their mother's first or tribal language as their mother's tongue.
Table 21
Mother
tongue
Farsi
Mother tongues and first language identified by participants
Frequency
First
language
Farsi
Frequency
3
Percent
(%)
4.8
3
Percent
(%)
4.8
Dari, Pushto
2
3.2
Pushto
2
3.2
Somali
27
43.5
Somali
27
43.5
Amharic
4
6.5
Amharic
7
11.3
Tigrigna
3
4.8
Vietnamese
10
16.1
Vietnamese
10
16.1
Cantonese
8
12.9
Cantonese
9
14.5
Khmer
4
6.5
Khmer
4
6.5
Mandarin
1
1.6
Total
62
100.0
Total
62
100.0
Home language use
Language studies to date in NZ have often included a question on language use at
home. Language maintenance studies, in particular, show that language shift at home is
usually the precursor for mother tongue loss in later generations. Nearly 60 % (37) of the
participants were still mainly using their mother tongue at home, but of particular interest
are the 25 students (40.3%) who were preferring to use English bilingually with their
mother tongue. This is quite a high figure for first generation immigrants who do not feel
fully competent in English, especially in comparison with their mother tongue skills. This is
an area which needs urgent further study as, clearly, for a disadvantaged community with
low self-esteem, these results augur the possibility of language loss with the next
generation. It is particularly important to look at the reasons behind the shift, and whether
91
the parents among the sample are aware of the cognitive and social implications of
language shift and loss.
A chi-square Fisher Exact two-tailed test was applied to the data on home language
use, to determine the relationship between this and various variables which may have
contributed to the shift (see Table 22).
Table 22
Home language use by key variables
Home language use and
Gender
Length of NZ residence
Age at time of arrival in NZ
Age
Home country
Mother tongue
Chi-square Fisher Exact two -tailed
0.005*
0.907
0.123
0.378
0.004*
0.010*
* These values are significant at p < .05,
N = 62 respondents
Home language use was related significantly at p<0.05 to gender, country of origin,
and participants’ mother tongue. A breakdown of this data is presented in tables 23, 24 and
25, to illustrate the distribution of participants across these variables.
Table 23
Gender
Female
Male
Total
Gender and home language use
English mainly
0
0
0
Both English and
mother tongue
14
11
25
Mother tongue
mainly
33
4
37
Total
47
15
62
None of the participants showed a preference for using mainly English at home, but
proportionately more males than females were using English bilingually at home. A
majority of the women were still showing a preference for mother tongue use at home.
92
Table 24
Country of origin and home language use
Country of
origin
English mainly
Both English and
mother tongue
Mother tongue
mainly
Total
Afghanistan
Somalia
Ethiopia
Vietnam
Cambodia
China
Hong Kong
Total
0
0
0
0
0
0
0
0
0
9
6
8
1
1
0
25
5
18
1
3
6
3
1
37
5
27
7
11
7
4
1
62
Because of the variable number of participants in each ethnic group, it is appropriate
to talk about apparent trends, rather than make any generalisations. Participants from
Afghanistan, Somalia and Cambodia were showing a preference for using mainly their
mother tongue at home. There was also some evidence that the Chinese immigrants were
doing the same. On the other hand, those from Ethiopia and Vietnam seemed to show some
preference for bilingual language use at home. Further study of these trends with larger
groups is needed in order to reach more valid suppositions for each ethnicity, as well as an
understanding of possible reasons for these phenomena, if they are duplicated in those
studies.
Finally, language use at home was also significantly related to the mother tongue of
participants (see Table 25). Two thirds of Somali speakers continued to use mainly their
mother tongue at home, as did all the Farsi and Pushto speakers, and a majority of the
Cantonese and Khmer speakers. A majority of Amharic, Tigrigna and Vietnamese speakers,
on the other hand, had begun to use their mother tongues bilingually with English. Again,
these results warrant further study into the rate of language shift among new and low- level
literacy refugees and immigrants, in order to benefit this and the next generation, both
cognitively and academically.
93
Table 25
Mother
tongue
Farsi
Dari, Pushto
Somali
Amharic
Tigrigna
Vietnamese
Cantonese
Khmer
Mandarin
Total
Mother tongue and home language use
English
mainly
0
0
0
0
0
0
0
0
0
0
Both English and
mother tongue
0
0
9
3
3
7
1
1
1
25
Mother tongue
mainly
3
2
18
1
0
3
7
3
0
37
Total
3
2
27
4
3
10
8
4
1
62
English study in NZ
More than half of the research participants (34/62) had studied English in NZ prior
to enrolling in the current classes. Of these, half (16) had studied either with home tutors or
in home tutor groups, and studied at Auckland University of Technology, and the remainder
had either attended community education classes or had studied with other small groups.
The data indicated that participants who had studied elsewhere had tried a variety of
delivery methods in different venues and for varying lengths of time. The longest lengths of
time spent in learning English were three years (one student), two years (one student), a
year and a half (one student), a year (seven students), a semester or two terms (twenty
students). Others had studied for much shorter lengths of time, ranging from two to ten
weeks. Weekly hours of study also varied a great deal, ranging from one to six hours a
week (in 42 instances,) to a full time government funded course (six instances). Most
venues and providers seem to favour an instruction time of between four to six hours a
week.
Bilingual instruction
Only 11 of the 62 participants said they had been previously taught by a bilingual
tutor, although admittedly this might have been in the first semester of their course prior to
the data gathering stage. Two thirds of the total sample said they would like to be instructed
by someone who was able to use both their mother tongue or first language, and English.
94
There was a clearly-articulated need for bilingual instruction, with only 13 students (21%)
saying they preferred to have a native English-speaking tutor. Since this part of the project
was conducted at the start of the programme, it would have been interesting and valuable to
evaluate the responses to this question at the end. Roughly half of the students had studied
with bilingual tutors over the research period.
Participant perception of host community attitude towards their mother
tongue
Participants were offered five choices in saying how they felt about speaking their
own language in front of New Zealanders. Over a third of them (23 respondents) were
comfortable and nine felt confident with it, while one said they felt both comfortable and
confident. Nearly as many respondents (nineteen) said, however, that they were not
comfortable using their mother tongue when other New Zealanders were around, while two
stated they felt uncomfortable and non-confident. Two respondents said they felt it was
impolite to use their mother tongues around other New Zealanders. Participants were
clearly ambivalent about mother tongue use in the host community, and a further interesting
area of research would be to look at a breakdown of this data by mother tongue and country
of origin, and the reasons behind the varied attitudes.
Reasons for learning English in NZ
Students were asked to nominate the reasons why the y wanted to learn English, and
were offered a range of choices. The results showed that all choices seemed equally
important to the participants (see Figure 6).
Almost equal numbers of students chose the following reasons: to get a job; to fill
out forms; to talk to and understand other New Zealanders; to understand television; and get
NZ citizenship. The option ‘to study further’ was not as popular as the other choices,
however. These results are similar to the ones that White et al. (2001:15) found in their
study of refugees and new immigrants in Auckland, Wellington and Tauranga.
95
Figure 6
Reasons for learning English in NZ
Reasons why the participants want to learn English
To get a job
To fill in forms
14%
16%
To talk to and understand NZers
8%
14%
To talk to and understand
neighbours
To understand the T.V
16%
15%
17%
To study further
To get NZ citizenship
Language proficiency
Students were asked to self- assess their specific literacy skills in English at the
beginning of the programme. These skills were related to upcoming course objectives and
outcomes. The results of this self-assessment are discussed in the next section. In their
personal profiles, however, students stated how well they felt they could perform in the four
skills in English, and those variables, which seemed to have a significant relationship with
English literacy, are presented and discussed.
This section of the results also presents the participants’ reported English
proficiency and compares their English and first language ability. It also compares the
average literacy proficiency in their first language and in English of each of the ethnic
groups which participated.
96
English Proficiency
Reported English proficiency data followed the expected distribution over the four
skills (see Figure 7). These were ranked as understanding ability scoring the highest mean
proficiency, followed by speaking, reading, then writing.
Figure 7
Student English proficiency (four skills)
English proficiency for participants
Very well
Well
45
Average
40
So-so
A little
35
Not at all
30
25
20
15
10
5
0
Speak
(Mean = 1.42)
N = 62
Understand
(Mean = 1.47)
Read
(Mean = 1.34)
Write
(Mean = 1.27)
Skills
Variables affecting English literacy skills
Various variables which may have affected the participants' literacy skills were
tested for their relationship with literacy ability. They included age, age at time of arrival,
length of NZ residence, gender, previous study of English in the home country, English
study in NZ, and the use of the Roman script for the first language (see Table 26). Finally,
the relationship between literacy skills in the first language and English was determined.
97
Table 26
English literacy and key variables
English literacy and
Age
Reading
Writing
Age at arrival Reading
in NZ
Writing
Length of NZ Reading
residence
Writing
Gender
Reading
Writing
Use of Roman Reading
script in L1
Writing
English study Reading
before NZ
Writing
English study Reading
in NZ
Writing
Chi-square
3.37
2.91
7.34
7.32
10.06*
9.89*
4.07*
4.30*
3.06
2.23
1.47
1.55
2.69
2.24
d.f.
4
4
4
4
4
4
1
1
1
1
1
1
1
1
Significance
.499
.573
.119
.120
.039
.042
.044
.038
.080
.135
.228
.213
.101
.135
* Kruskal Wallis test significant @ p<.05,
N = 62
The two variables which demonstrated a significant relationship with English
reading and writing ability were gender and length of NZ residence. A closer examination
of the data revealed that, generally, women demonstrated a lower mean proficiency in
English reading (1.21) than men (1.73). There was a similar trend with English writing
ability; women had a mean proficiency of 1.15, compared to men with 1.67.
A cross-tabulation of means between English reading and writing proficiency and
length of NZ residence showed a steady incline in proficiency with longer length of
residence, particularly in the three categories of less than one year residence (reading &
writing = 1.08), one to two years (reading = 1.4, writing = 1.24) and three to five years
(reading & writing = 1.78). This was true for 56 of the total research sample of 62. The six
participants who had lived in NZ for longer than six years, however, showed lower literacy
skills than even those who had arrived over the last year, showing that other possible factors
may have affected their literacy gains.
In comparing the participants’ English and their first language literacy skills, it was
clear that mean literacy in the first language for the whole group was at least a level higher
than in English, in both reading and writing (see Figure 8). The number of individual
participants who were pre- literate in their first language, however, was higher than the
98
number who could not read and write English at all. For example, 14 participants said they
could not read and 15 could not write in their first language at all while 11 said this about
their English reading and 12 about their English writing ability.
Figure 8
Student English and first language proficiency
English and first language (L1) proficiency for participants in reading and writing
35
30
Very well
Well
25
Average
So-so
A little
20
Not at all
15
10
5
0
Read English
(Mean = 1.34)
N = 62
Read L1
(Mean = 2.77)
Write English
(Mean = 1.27)
Write L1
(Mean = 2.66)
Skills
In order to compare these proficiencies on a case-by-case basis, and to determine the
relationship between first language and English language literacy skills, a Wilcoxon
Matched-Pairs Signed-Ranks Test was used (see Table 27).
Table 27
Differences in first language and English literacy
Wilcoxon Matched-pairs
Signed-ranks test*
Z
Significance: 2-tailed
L1 – English Reading
- 4.281*
.000019
L1 – English Writing
- 4.178*
.000029
*Based on negative ranks Significant @ p<.05
99
The result showed a significant relationship between literacy in the first language
and English literacy. A majority of the participants in both reading (37 respondents) and
writing (38 respondents) showed a positive relationship between the two languages. There
were ten ties in reading and eleven ties in writing skills.
A breakdown of literacy skills in each first language nominated by the participants
showed that Cantonese-speaking students had the highest literacy ability in their own
language, followed by the Pushto group, then the Vietnamese and then the Cambodian (see
Figure 9). Those who nominated Farsi, Somali and Amharic as their first language showed
a generally lower literacy ability. The sample size in some ethnic groups such as
Cambodian (four), Pushto (two), Farsi (three) is quite small, and therefore cannot be used to
make any generalisations. It is still of interest, however, to look at the trends in data.
Figure 9
Student first language proficiency
Mean first language proficiency for participants in reading and writing
5
NOTE: Proficiency Scale
0 = Not at all, 1 = A little, 2 = So-so,
3 = Average, 4 = Well, 5 = Very well
Read
Write
4
3
2
1
0
Farsi
(N=3)
Pushto
(N=2)
Somali
(N=27)
Amharic
(N=7)
Vietnamese
(N=10)
Cantonese
(N=9)
Khmer
(N=4)
First languages
100
Summary: Student profiles
Although 118 students were enrolled in the programmes during the course of the 20
weeks, only 62 of them were eligible as participants on the research project. They
originated from seven different countries, with the majority being refugees over 30 years of
age, who had arrived in NZ when they were over 21 years of age and within the last five
years. While nearly two thirds of the participants had been unemployed in their home
country, in NZ this was true for over 90% of them. However, the data showed that
prospective employment was not the only reason for their demand for English literacy.
Other reasons such as functional use of English to fill in forms, watch TV and adapt to NZ
society were just as important to them.
Half the research participants had attended primary school prior to arriving in NZ,
and most of these had received less than three years of English language instruction in their
country of origin. Moreover, more than half the participants had received at least some
English instruction in NZ before enrolling in the current classes. There was some
discrepancy in student responses to the questions regarding mother tongue and first
language, as in six cases these did not match up. Clearly, this distinction is important
among new immigrants and refugee groups who have more than three languages in their
speech repertoire.
Response to a question on home language use showed over 40% of students
preferring to use their mother tongue bilingually with English – a clear sign of language
shift within the first few years of immigration. The three variables which seemed to be
affecting the rapid shift were gender (men were shifting faster than women were), country
of origin, and mother tongue (Somali, Afghani and Cambodian participants were more
resistant to shift).
The results showed a demand for bilingual tuition on the part of two thirds of the
respondents at the beginning of the programme, although only eleven participants said they
had been taught by bilingual tutors to date. They might have received this bilingual tuition
in the first semester of the currnet programme.
101
Participants were ambivalent about host community attitudes towards their use of
the mother tongue in public, with half saying they were comfortable or confident doing so
but nearly as many being unsure of their reception if they did.
English language proficiency data followed the expected pattern, with respondents
being strongest in listening skills, followed by speaking, reading, then writing. First
language proficiency also showed reading ability to be stronger than writing. Overall,
participants’ first language literacy ability was higher than their English ability, and of
significance was the positive influence of first language literacy skills on English literacy.
English literacy was also affected by gender (women showed lower mean proficiencies) and
length of NZ residence (earlier arrivals within the last five-year span were more competent
than more recent arrivals). Different ethnicities displayed varying first language literacy
abilitie s.
The next section looks at the assessment data from the study and evaluates whether
literacy gains at emerging literacy level can be measured through the use of self- assessment
scales.
102
Sunday Star Times (ellison)
103
REPORTED LITERACY
Introduction
Self-assessment procedures were carried out with the research participants at three
points of their programme: at the beginning, at mid-point and at the end. The scales that
were used for this assessment were at eight different levels in ascending order of
proficiency for both reading and writing. Students used a five-point scale (5 = Very well; 4
= Well; 3 = Average; 2 = So-so; 1 = A little; 0 = Not at all) to report their own perceived
proficiency on a range of real- life tasks which they would expect to be able to perform at
the end of their period of study.
Hence, the self-assessment at the beginning of the
instructional period indicated what they perceived to be the gaps in their survival literacy
ability; the self-assessment in the middle indicated their progress; and the assessment at the
end gave some indication of their perceived achievement. All three self-assessments used
the same procedures and forms, and were administered in the classroom.
This section of the results presents both descriptive and comparative data on two of
the batches of self-assessments, taken from the beginning and from the end of the project
period, since we are mainly concerned with literacy gains over this period.
Computations have been carried out first for the total research sample, then
separately for the classes taught by bilingual and English-speaking tutors, and finally for the
2-hour and 12-hour weekly classes.
Total sample self-assessments: Reading
The trends in mean values of the reading self-assessment scales illuminated some
interesting patterns (see Table 28). The scales were developed by the testing team with
reference to the curriculum (see Appendix 5) and were meant to be in ascending order of
difficulty. Hence students were expected to have found the task at Level 1 easier to perform
than those at higher Levels 2 – 8.
104
Table 28
Level
1
2
3
4
5
6
7
8
Self-assessed proficiency: Reading
Task description
Time 1
Mean s.d.
I can read information on a 2.82 1.55
simple form
I can read names and know the 3.11 1.47
sounds of letters
I can read dates
3.63 1.41
I can read numerals
3.64 1.69
I can read public signs
2.69 1.68
I can read bus timetables
2.05 1.80
I can find words arranged in 2.16 1.90
alphabetical order
I can read a simple story
2.18 1.58
Total mean/40 22.29 9.18
Level (0 – 8)
4+
N
62
Time 2
Mean s.d.
2.90 1.30
Time 3
Mean s.d.
2.92 1.37
3.29
1.33
3.40
1.18
3.53
3.53
2.71
2.02
2.55
1.31
1.25
1.31
1.55
1.40
3.61
3.65
2.95
2.68
2.90
1.17
1.12
1.38
1.42
1.52
2.29 1.35
22.82 8.41
4+
62
2.52 1.49
24.62 9.15
4+
62
Mean range = 0 – 5
The results of all three self-assessments showed that students were in fact placing
their numeracy skills ‘I can read the date’ (Level 3) and ‘I can read numerals’(Level 4) at
Levels 1 and 2 respectively before their reading ability with letters. At Level 3 they felt
they could read names and knew the sounds of letters – a task which the testing team had
placed at Level 2. Of greatest interest was the consistent placing at Level 4 of the skill of
reading information on a simple form, when in fact, this had been perceived to be the
easiest task to perform, and was placed by the test team at Level 1 on the self- assessment
form. Also clearly misplaced was the students’ perceived ability to read a bus timetable,
which they almost consistently felt was the most difficult reading skill – more so than
finding words arranged in alphabetical order or reading a simple story.
Since self-perception of reading ability is an important part of learning, the mean
results give some indication that a change in emphasis in the curriculum is needed if
reading ability is to be gained progressively, and scaffolded so that easier skills are learned
and consolidated first. The difference in perception between the curriculum developers/test
writers and the students themselves of the order of difficulty of these real- life reading tasks
need to be taken into account in further courses. However, these perceptions need to be
compared with actual test results before any clear conclusions can be reached regarding
105
trends and levels of ability in the various survival reading tasks which are included in the
curriculum for pre- literate and lower literacy level students.
A look at the total mean values for each self-assessment procedure shows some
gains in perceived ability over time, however, when converted to a ‘Level’ score using the
8-point descriptors of possible performance ability (Appendix 7), students are moving
slightly higher with each self-assessment within the Level 4 band in reading.
Total sample self-assessments: Writing
The means of the students’ self-assessed writing ability also showed a variation in
the expected levels of difficulty of the tasks on the self-assessment form (Table 29). While
at Level 1, the 62 research participants consistently agreed that they had the greatest ability
in writing lower and upper case letters, in self-assessment Time 1, 2 and 3, they placed their
ability to write the date, at Level 2 rather than at Level 4, where the test and curriculum
writers had placed it. Also in Time 1, 2 and 3, the means showed that they found it easier to
use capital letters and fullstops than to write personal information on a simple form, thereby
inverting the order in which these had been included on the self- assessment form. Hence,
both the reading and writing results showed that students felt they had greater proficiency in
numeracy and writing conventions than in interpreting or providing information in a
familiar context, such as filling out a form requiring personal information.
Also different from the expected level of difficulty on the form was student ability
to write a short letter of explanation, which had been placed at Level 5 by the test
writers/curriculum developers, but which in Time 1 and 3 of the self-assessment, students
placed at the most difficult Level, 8, perceiving it to be more difficult than writing a
detailed telephone message, writing about familiar things, or keeping a diary. In Time 2,
this pattern differed slightly in that students put a letter of explanation at Level 7. Although
the ascending order of difficulty of the final three tasks (placed by students from Levels 5 –
7 rather than 6 – 8 generally held true, Time 3 results showed that, by the end of the
programme, the students were finding it easier to keep a diary than to write about familiar
things. It could be argued that they had had more practice in the former over the course of
106
the programme and had gained confidence, as well as greater skills, in keeping a regular
journal.
Table 29
Level
1
2
3
4
5
6
7
8
Self-assessed proficiency: Writing
Task description
Time 1
Mean s.d.
3.47 1.69
I can write the lower and upper
case letters
I can write information on a 2.87 1.66
simple form
I know when to use capital letters 2.95 1.76
and full stops
I can write the date
3.31 1.74
I can write a short letter of 1.26 1.58
explanation
I can write a detailed telephone 1.63 1.63
message
I can write about familiar things
1.65 1.67
I can keep a diary
1.48 1.79
Total mean/40 18.61 9.47
Level (0 – 8)
3+
N
62
Time 2
Mean s.d.
3.61
1.06
Time 3
Mean s.d.
3.55 1.44
2.89
1.29
3.02
1.50
3.05
1.25
3.08
1.21
2.94
1.77
1.68
1.58
3.16
1.94
1.39
1.62
1.95
1.48
2.27
1.44
1.81
1.59
1.66
1.43
19.68 8.44
3+
62
2.02 1.49
2.23 1.60
21.26 9.27
4+
62
Mean range = 0 – 5
The mean values show that there needs to be some re-thinking of the order in which
the writing skills are taught on the programme if students are to be scaffolded in their
learning and to progress from performing easier to more difficult tasks.
The total mean values on the writing self- assessment (out of a possible total of 40)
show progress being made between each self-assessment. Unlike the reading selfassessment, however, students felt they had moved up by one band, from Level 3 to Level 4
by the last self-assessment at Time 3.
Student perceptions of literacy gains: Reading
Since this study was conducted mainly in order to measure literacy gains among the
research participants, various statistical tests were run to determine if mean differences
between Time 1 and Time 3 of the self-assessment procedures were significant. The means
107
on the reading self-assessment scales generally showed consistent progress being made on
all the listed tasks, except for the numeracy skills, which remained at about the same,
perceived easiest Levels 1 and 2, with roughly equivalent mean values for all three times.
For statistical computations, each student was given a score out of a total possible
score of 40 on self-assessments at Time 1 and Time 3. Group trends are borne out by the
positive value of the Spearman correlation coefficient, which signals the strength of the
relationship between Time 1 and Time 3 results (see Table 30). Using the Wilcoxon
matched–pairs signed-ranks test, a closer look at the paired data in the two tests for
individual students showed significant gains having been made in reading by a majority of
research participants. While 22 students had placed themselves at a lower mean score in the
final self-assessment, a far greater number (36) had felt more able to cope with the listed
tasks by this time. The total paired-sample T-test, which is designed to measure the
significance gains made by the sample between Time 1 and 3, also showed a significant
result (see Table 30).
Table 30
Self-assessed reading ability: Time 1 and 3
Ranks
Self-assessment scores
Time 3 < Time 1
N
22
Time 3 > Time 1
36
Time 3 = Time 1
4
Total
62
Tests
Values
Wilcoxon matched-pairs
Z = -2.450*
signed-ranks
Sig. (2-tailed) = 0.014
Total paired-sample
t = -2.203*
T-test (pre- and post-programme)
df = 61
Sig. (2-tailed) = 0.031
Spearman correlation
coefficient = rho
Correlation coefficient = 0.597*
* These values were significant at p < .05
108
Student perceptions of literacy gains: Writing
The mean values of student self-assessment in writing had also showed a perception
of gain over Time 1, 2 and 3. Again, students were given a score out of a possible total of
40 for the statistical computations. The results of the writing assessment were similar to
those of the reading, with a positive value for the Spearman correlation coefficient. The
higher positive value for the Spearman correlation coefficient in writing reflected the
greater variance of the ratings on the writing scale than on the reading one (see Table 31).
A matched comparison of individual reports again reflected the reading reports with
36 students generally reporting a higher ability in their writing skills on each task in Time 3
and 22 reporting a lower one. The difference in reports between the two self-assessments
was significant and this was further evidenced by the significant value at p<.05 of the total
paired sample T-test.
Table 31
Self-assessed writing ability: Time 1 and 3
Ranks
Self-assessment scores
Time 3 < Time 1
N
22
Time 3 > Time 1
36
Time 3 = Time 1
4
Total
62
Tests
Values
Wilcoxon matched-pairs
z = -2.479*
signed-ranks
Sig. (2-tailed) = 0.013
Total paired-sample
t = -2.582*
T-test (pre- and post-programme)
df = 61
Sig. (2-tailed) = 0.012
Spearman correlation
coefficient = rho
Correlation coefficient = 0.619*
* These values were significant at p < .05
109
It must be remembered that, as students did not have reference to their own Time 1
self-assessments at subsequent sessions, they did not have a reference point on which to
base their comparative self-assessments. It is to be expected, therefore, that a number would
place themselves at a lower level in Time 2 and 3 as, after a period of instruction, they
became more aware of the gaps which remained in their learning. These results are,
nevertheless, compared with actual test results later in the result section of this report, to see
if these perceptions are borne out by the matched literacy tests.
Student perceptions of literacy gains: Bilingual and English-speaking
tutors
The mean results showed that both groups of students – those who had been taught
by bilingual tutors (22 students) and those taught by English-speaking tutors (40 students) had made some gains in literacy between Time 1 and Time 3 of the self-assessments (see
Table 32). This was much more apparent, however, in the mean values of the students
taught by the English-speaking tutors. Figure 10 graphically illustrates the perceptions of
the progress made in reading and writing skills for both groups. When the student responses
in the pre- and post-programme self-assessments were matched up for each student, it was
apparent that, while the differences between the two self-assessments completed by those
who had had English-speaking tutors were significant, this was not so for those who had
had bilingual tutors. This was the case for both the reading and writing proficiencies (see
Table 32). As a breakdown of students’ individual comparative responses shows, twice as
many students in the English-speaking tutors’ classes felt they were better in the selfassessment tasks than they had indicated in the first self-assessment. For students in the
bilingual tutors’ programmes, generally equal numbers felt they either had greater ability to
perform the literacy tasks or that they had less. This may well be due to those in the
bilingual tutors' groups becoming more aware of the gaps in their literacy knowledge than
they had been in Time 1.
An independent-samples T-test was used to determine whether the difference in
literacy gains between the groups of students - those who had been taught by bilingual
tutors and those taught by English-speaking tutors - were significant (see Table 32). Results
for the Levene’s test for equality of variances showed significance levels (p-values) to be
110
higher than 0.05 in both the reading and writing self-assessments during Time 1 and Time
3. Hence the variances of the two groups were not significantly different from one another,
thereby showing the effectiveness of literacy instruction, regardless of the language of
instruction.
Table 32
Student perception of literacy gains: Bilingual and English-speaking
tutors
Reading
Means
Self-assessment 1
Self-assessment 3
Range 0 – 40
N
Tests
Wilcoxon matched-pairs
signed-ranks test
Self-assessment scores:
Self-assessment 3 < Selfassessment 1
Self-assessment 3 > Selfassessment 1
Self-assessment 3 = Selfassessment 1
Z
Significance (2-tailed)
Independent-samples Ttest:
Levene’s test for equality
of variance
Self-assessment 1
Self-assessment 3
Total
Writing
Bilingual
tutors
Englishspeaking
tutors
Bilingual
tutors
Englishspeaking
tutors
Mean
S.d.
Mean
S.d.
Mean
S.d.
Mean
S.d.
25.73
25.95
8.75
9.93
20.40
23.90
8.96
8.74
20.50
21.36
17.58
21.20
9.59
9.53
22
40
9.15
9.01
22
11
11
10
12
11
25
12
24
0
4
0
4
-.130
.897
f
-3.220*
.001
Significance
-.374
.709
f
-2.964*
.003
Significance
.418
.990
.520
.324
.178
.203
.674
.654
62
40
62
* These values were significant at p<. 05.
111
Figure 10
Perceived student literacy gains: Bilingual and English-speaking tutors
Student perceptions of literacy gains: Bilingual and English-speaking tutors
30
Self-assessment 1
Self-assessment 3
25
Mean proficiency
20
15
10
5
0
Reading:
Bilingual tutors
Reading:
English tutors
Writing:
Bilingual tutors
Writing:
English tutors
Tutor type
Student perceptions of literacy gains: 2-hour and 12-hour weekly classes
In both classes (2- and 12- hour weekly), the research participants began with higher
perceived reading than writing abilities, and this pattern carried through to the final selfassessments. The results also showed that the students who had been taking the 2-hour
classes began the programme with self-assessed literacy abilities which were considerably
lower than the abilities of those enrolling in the 12- hour classes. In the 2-hour classes, 14
out of 30 students completed the programme and almost all the project tasks. Forty-eight
students out of 88 did so in the 12-hour ones (see Table 33).
112
The mean values indicated that perceived reading progress had been made in both
classes but that, while students rated themselves at a higher level of ability in the final
writing self-assessment in the 12-hour classes, in fact the mean fell slightly in the two- hour
ones. Figure 11 graphically illustrates these patterns for the two classes.
Figure 11
Student perception of literacy gains for 2-hour and 12-hour weekly
classes
Student perceptions of literacy gains: 2-hour and 12-hour weekly classes
30
Self-assessment 1
Self-assessment 3
25
20
15
10
5
0
Reading:
2-hour classes
Reading:
12-hour classes
Writing:
2-hour classes
Writing:
12-hour classes
Class type
113
When the data from individual students was compared using the Wilcoxon matchedpairs signed-ranks test, it was clear that the increase in self-assessed literacy proficiency
was greater for those in the 12-hour classes (Table 33). This increase in numbers placing
themselves at a higher level of proficiency was significant. In reading, on the other hand, in
the 2-hour classes, seven of the 14 had placed their abilities at a higher level in Time 3, and
only five had done so in the writing. These increases, however, were not significant. An
independent-samples t-test for equality of variances between the two groups of students
found that the variances between them were not significant in self-assessed reading ability
in Time 1 or in Time 3 or in writing in Time 1. However, they were significantly different
in writing in Time 3, showing that the perceived writing gains in the 12-hour classes at the
end of the programme were significantly higher than in the 2- hour ones.
Table 33
Student perception of literacy gains: 2-hour and 12-hour weekly classes
Reading
Means
Self-assessment 1
Self-assessment 3
Range 0 – 40
2-hour
classes
N
Tests
Wilcoxon matched-pairs
signed-ranks test
Self-assessment scores:
Self-assessment 3 < Selfassessment 1
Self-assessment 3 > Selfassessment 1
Self-assessment 3 = Selfassessment 1
z
Significance (2-tailed)
Independent-samples Ttest:
Levene’s test for equality
of variance
Self-assessment 1
Self-assessment 3
Total
Writing
12-hour
classes
2-hour
classes
12-hour
classes
Mean
S.d.
Mean
S.d.
Mean
S.d.
Mean
14.93
16.86
5.70
7.16
24.44
26.90
8.92
8.44
12.00
11.86
6.71
4.80
S.d.
14
48
14
20.54 9.33
24.00 8.44
48
6
16
6
16
7
29
5
31
1
3
3
1
-1.192
.233
f
-2.090*
.037
Significance
-.089
.929
f
-2.705*
.007
Significance
3.276
.255
.075
.615
1.808
4.510*
.184
.038
62
62
* These values were significant at p < .05
114
Summary: Student perceptions of literacy gains
The results of the self-assessment procedures showed that for future use, some
changes are needed in the order in which tasks and topics are covered in the curriculum,
and in the placing of functional tasks from easier to more difficult on the self- assessment
scales. Research participants reported higher abilities in numeracy skills than in reading and
writing. There was also some evidence that students were finding it easier to follow reading
and writing conventions than they were to use the skills in context, such as in reading
information on a simple form or in filling one out.
Group trends in data showed that a majority of students felt they had made gains in
literacy in both reading and writing skills, and this perception was found to be a
statistically-significant one. It was also apparent that students who came into the
programme with perceived higher levels of ability were also likely to complete it with a
similarly positive perception of their abilities.
When student perceptions of their gains were compared between those being taught
by bilingual tutors and those in classes with English-speaking tutors, it was found that
variances between the two groups were non-significant. However, the gains within each
group showed that a majority of individuals being taught by English-speaking tutors felt
they had a significantly higher literacy ability in the Time 3 reports at the end of the
programme than in the Time 1 ones at the beginning.
In comparing the literacy gains made between the 2-hour and 12-hour weekly
classes, it was found that variances between the two groups were significant only for
reported writing ability in Time 3. They were non-significant between them for reading
ability in Time 1 and 3 or in Time 1 reported writing ability. The data showed significant
gains made in reading and writing literacy within the 12-hour classes between Time 1 and
Time 3.
It is apparent, the refore, that a high standard of teaching, and longer contact hours
were conducive to perceived literacy gains among this group of research participants. The
115
results showed that both bilingual and English-speaking tutors have crucial roles to play,
depending on the beginning proficiency level of the students in their classes and their desire
for bilingual instruction. Perhaps exposure to both types of tutors would increase perceived
gains among the students, by allowing access to a variety of various effective role models.
The performance test results in the next section will enable a comparison to be
drawn between perceived and actual literacy gains among the research participants.
116
117
ASSESSED LITERACY GAINS AND COMPARISON WITH SELFASSESSED LITERACY
Introduction
The literacy abilities of research participants were assessed at two points of the
programme – at the beginning, prior to the commencement of the programme and then
again at its closure. Parallel tests were devised for both occasions so that sound appraisals
could be made regarding student achievement. The tests were also congruent with the selfassessment scales discussed in the previous section, so that judgements could be made on
the validity of the self-report scales, which may well be used in the future in place of actual
testing. This is particularly due to the fact that testing at this level is time-consuming and
difficult to administer, as learners have no previous experience with test-taking (see
Appendix 9 for tutor comments on testing).
The raw scores of the test were converted to discrete scores out of five for each of
the eight reading and writing levels. This was for the purpose of statistical computations
and for ease of comparison with the discrete five-point scores used in the self- assessment
scales. Hence, when it was necessary to use test totals, these were out of a total possible
score of 40 for both the reading and writing tests.
Test results are presented in three parts. The first part presents and analyses total test
results from the two tests. This is followed by a breakdown of results according to whether
students were taught by bilingual or English-speaking tutors. Thirdly, the number of hours
of weekly instruction (two or twelve hours) they were receiving is considered.
Total sample test results: Reading
The mean values at each level of the two assessments of performance had increased
between the two reading tests. There was also a clear distinction between two main areas of
ability - those covered by Levels 1 - 4 on the test scale and those between Levels 5 - 8. Test
takers found the tasks at the first four levels considerably easier to perform (See Table 34).
Just as with the self-assessments, the performance assessments showed that students were
118
not necessarily finding the level of difficulty of test tasks in the expected sequence.
However, the result did show that, with instruction, the perception of the sequencing from
easier to more difficult tasks changed among the participants, with their perception of the
sequence closer in the final tests to the levels chosen by the test writers than had been the
case in the initial tests. By the final tests, students had begun to find it easier to read
information on a simple form, whereas in the initial tests they had found it easiest to read
names and recognise the sounds of letters.
The results also showed that students were finding it easier to read a short simple
story in both tests than to perform the tasks listed at the three higher levels. The test
descriptors had placed this at the highest level (8) of the test. Students had greater difficulty
reading bus timetables and finding words arranged in alphabetical order than in reading a
story, although these tasks had become slightly easier after instruction.
Table 34
Level
1
2
3
4
5
6
7
8
Performance assessment: Reading
Task description
Performance Test 1 Performance Test 2
Mean
s.d.
Mean
s.d.
Is able to read information on a
3.31
1.47
4.57
1.06
simple form
Is able to read names and know
3.94
0.97
4.26
0.89
the sounds of letters
Is able to read dates
3.06
1.52
4.28
1.05
Is able to read numerals
3.19
1.35
4.00
1.05
Is able to read public signs
2.15
1.79
2.97
1.82
Is able to read bus timetables
.73
1.15
2.48
1.62
Is able to find words arranged in
1.11
1.52
2.36
1.72
alphabetical order
Is able to read a simple story
1.98
1.88
2.74
2.00
Total mean/40
18.90
8.6
27.38
8.11
Level (0 – 8)
3+
5+
N
62
61
Mean range = 0 – 5
These results indicate that the level descrip tors and task order in the tests need revisiting and fine tuning, to allow for a scale which is more representative of student levels
of performance, particularly at the culmination of the programme of study. This would need
119
to be done with reference to and in combination with the self-assessment scales and the
curriculum.
The mean values on the two reading tests (out of the possible total of 40)
demonstrate the considerable progress students had made in their reading skills. This
translated to a difference of two levels on the 8-point scale - from a 3+ level to a 5+ level of
mean performance (Table 34).
Total sample test results: Writing
Fifty- five research participants (of the 62 who did the initial tests) completed the
writing tests at the end of the programme (see Table 35). The mean results illustrate the
difficulty in predicting progress in performing survival- level writing tasks among preliterate and low- level literacy students. Instruction also influences which skills are gained
more fully than others. Generally, the mean results show students had improved in their
skills at each level of performance, although the mean value of student ability to write down
a detailed telephone message had declined considerably in the final test, thereby placing
this task at the highest level (8) of difficulty, when it had been placed at Level 4 in the
initial test. This indicates that the content of this test task or the requirements for answering
this question need to be re-addressed. It may well be that the test task or instructions for its
performance are not clear, or that the task is at a higher level of difficulty due to the
wording. The order of difficulty of the test tasks correlated with the order of test descriptors
to a much greater degree in the final tests than it had in the initial ones. This showed that
instruction had made a difference in learning, particularly in the areas where students
previously had gaps and difficulties.
120
Table 35
Performance assessment: Writing
Level
Task description
1
Is able to write lower and upper
case letters
Is able to write information on a
simple form
Is able to use capital letters and
full stops
Is able to write the date
Is able to write a short letter of
explanation
Is able to write a detailed
telephone message
Is able to write about familiar
things
Is able to keep a diary
Total mean/40
Level (0 – 8)
N
2
3
4
5
6
7
8
Performance Test 1 Performance Test 2
Mean
s.d.
Mean
s.d.
2.57
1.35 4.02
1.15
3.02
1.47
3.60
1.33
1.56
1.46
2.60
1.40
2.56
1.28
1.68
1.21
2.98
2.13
1.69
1.69
1.69
1.90
1.55
1.66
1.18
1.15
1.87
1.35
1.11
14.44
1.16
8.83
1.80
20.13
1.24
8.43
4
55
2+
61
Mean range = 0 – 5
Although the mean values out of the total possible score of 40 were not as high as
they were for the reading tests, this nonetheless also represented a gain of two levels on the
8-point scale from, a Level of 2+ to Level 4.
Literacy gains: Reading
Statistical analyses were conducted to determine whether mean differences between
Tests 1 and 2 were significant. For this analysis, all students were given a test score out of a
total possible score of 40. As the results in Table 36 show, the gains made in reading were
highly significant. In fact, when total scores were matched for each individual student
between Test 1 and Test 2, 53 of the 61 participating students had gained higher scores in the
final tests. Six students' scores had declined, and it may be that other factors had a bearing on
this; irregular student attendance and the onset of Ramadhan were possible factors (see
methodology section on constraints).
121
Table 36
Performance assessments 1 and 2: Reading
Ranks
Performance scores
Test 2 < Test 1
N
6
Test 2 > Test 1
53
Test 2 = Test 1
2
Total
61
Tests
Values
Wilcoxon matched-pairs
z = -5.871*
signed-ranks
Sig. (2-tailed) = 0.000000004
Total paired-sample
t = -8.019*
T-test (pre- and post-programme)
df = 60
Sig. (2-tailed) = ~0
Spearman correlation
coefficient = rho
Correlation coefficient = 0.503*
* These values were significant at p < .05
The gains in the reading tasks were confirmed also by the paired-sample T-test,
which found the differences between the initial and final literacy tests to be highly
significant. The correlation coefficient also shows the positive correlation between the two
tests, attesting to the reliability of the performance judgements being made on the tests.
Literacy gains: Writing
Statistical analyses on the results of the writing tests showed similar trends to those
of the reading. Forty-three of the total 55 students who took the final tests scored higher
marks then they had in the initial ones. Nine students scored lower marks, and again other
factors may have influenced this lower mark (Table 37).
122
Table 37
Performance assessments 1 and 2: Writing
Ranks
Performance scores
Test 2 < Test 1
N
9
Test 2 > Test 1
43
Test 2 = Test 1
3
Total
55
Tests
Values
Wilcoxon matched-pairs
z = -5.414*
signed-ranks
Sig. (2-tailed) = 0.0000001
Total paired-sample
t = -7.606*
T-test (pre- and post-programme)
df = 53
Sig. (2-tailed) = .0000000005
Spearman correlation
coefficient = rho
Correlation coefficient = 0.756*
* These values were significant at p < .05
When scores between the two writing tests were matched for individual students the
difference in literacy gains was found to be highly significant. The paired-sample T-test for
the whole sample, which tested for mean differences in the pre- and post-programme
assessment, also found the difference to be significant. The correlation co-efficient between
the two writing tests had a higher positive value than that of the reading tests,
demonstrating the greater variance in the allocation of marks for the writing tests.
Literacy gains: Bilingual and English-speaking tutors
As Table 38 illustrates, both groups made significant progress on the programme.
When the pre- and post-programme means are compared, the greatest gains appear to be in
reading in the classes taught by the bilingual tutors, and the smallest degree of gain also in
the classes taught by bilingual tutors, but in writing. Figure 12 graphically illustrates the
mean gains made in reading and writing for both groups.
123
The variances of the two groups taught by bilingual and English-speaking tutors
were found to be significantly different from one another in the initial Writing Test 1 and in
the final Reading Test 2 (see Independent samples T-test, Table 38). This showed that
students who were to be taught by bilingual tutors were in fact starting the programme with
significantly lower writing ability than those who were to be taught by English-speaking
tutors. By the end of the programme, those taught by bilingual tutors had made significant
gains in reading, and had also managed to reduce the variance in writing skills so that the
variances between the bilingual tutors' and English-speaking tutors' classes no longer
existed.
Table 38
Literacy gains in performance tests: Bilingual and English-speaking
tutors
Means
Performance test 1
(pre-programme)
Performance test 2
(post-programme)
Range 0 – 40
Test 1 total sample
Test 2 total sample
Tests
Wilcoxon matchedpairs signed-ranks test
Test scores:
Test 2 < Test 1
Test 2 > Test 1
Test 2 = Test 1
Totals
Z
Significance (2-tailed)
Independent-samples
T-test:
Levene’s test for
equality of variance
Test 1
Test 2
Total
Reading
Bilingual
Englishtutors
speaking
tutors
Writing
Bilingual
Englishtutors
speaking
tutors
Mean
S.d.
Mean
S.d.
Mean
S.d.
Mean
S.d.
19.91
9.81
18.35
7.98
13.91
14.74
7.15
30.14
5.40
25.93
8.95
18.14
11.3
9
8.06
21.35
8.53
22
21
40
40
22
21
39
34
2
19
0
21
-3.775*
.0002
f
4
34
2
40
-4.449*
.0001
Significance
5
15
1
21
-2.990*
.003
f
4
28
2
34
-4.572*
.000005
Significance
3.733
6.610*
.058
.013
16.025*
.011
.0002
.918
62
62
* These values were significant at p < .05
124
On a case-by-case comparison, it was clear that substantial and significant gains had
been made in both reading and writing tests by a majority of students of both groups (see
Wilcoxon matched-pairs signed-ranks test, Table 38). A comparison with self-report data
will show whether perceived and actual gains were similar. Verbal comments by the tutors
on a somewhat ‘collaborative’ approach to the tests on the part of the students may have
skewed the results (Appendix 9). However, even if collaboration contributed to the
significant gains recorded in the tests, this is a feature that future tests will need to
accommodate, since it is clear that collaboration works well with this group of learners in
literacy tests. Students collaborated in both tests but did better in the final test. This could
show an increased ability in test-taking skills, in collaborative approaches to testing or in
performing literacy tasks by the end of the programme.
Figure 12
Student performance gains: Bilingual and English-speaking tutors
Performance gains versus bilingual and English-speaking tutors
35
30
Mean proficiency
25
20
Test 1
Test 2
15
10
5
0
Reading:
Bilingual tutors
Reading:
English tutors
Writing:
Bilingual tutors
Writing:
English tutors
Tutor type
125
Literacy gains: 2-hour and 12-hour weekly classes
The mean figures in reading and writing gains in the two tests showed a greater
amount of progress made in the tests taken by participants enrolled in the classes run for 12
hours a week (see Table 39). This was especially so for the reading tests. The participants in
the 2-hours a week classes had made some gains in reading and writing, although not as
much as in the 12-hour classes. Figure 12 illustrates the differences in gains made by the
two groups of participants.
Table 39
Literacy gains in performance tests: 2-hour and 12-hour weekly classes
Means
Reading
2-hour
12-hour
classes
classes
Mean
Performance test 1
(pre-programme)
Performance test 2
(post-programme)
Range 0 – 40
Test 1 total sample
Test 2 total sample
Tests
Wilcoxon matched-pairs
signed-ranks test
Test scores:
Test 2 < Test 1
Test 2 > Test 1
Test 2 = Test 1
Totals
z
Significance (2-tailed)
Independent-samples
T-test:
Levene’s
test
for
equality of variance
Test 1
Test 2
Total
S.d.
Mean
S.d.
Writing
2-hour
classes
Mean
12-hour
classes
S.d.
Mean
S.d.
16.57
7.96
19.58
8.77
14.23
6.34
14.50
9.44
20.86
7.68
29.32
7.24
15.33
6.42
21.07
8.51
14
48
13
48
14
47
9
46
1
13
0
14
-3.088*
.002
f
5
40
2
47
-5.126*
.0000003
Significance
3
5
1
9
-.769
.442
f
6
38
2
46
-5.309*
.0000001
Significance
.488
.063
.488
.803
3.45
.744
.068
.392
61
55
* These values were significant at p < .05
126
The Wilcoxon matched-pairs test of significance matched the final test results of
individual students who had completed both tests to their initial test results (Table 39). This
test found significant progress made in both reading and writing tests by students on the 12hour weekly programmes, but only in the reading tests by the 14 students who had
completed the 2-hour weekly classes. In fact when the matched-pairs are compared for the
2-hour classes, 13 of the 14 students did better in the final reading test than in the initial
one. The number of students who did the final writing test in the 2-hour classes had fallen
considerably, as only nine of the 14 students who took the initial and final reading tests also
did the final writing one. Also, only five of the nine students of the 2-hour classes who took
the final writing test did better in it than in the initial test. The matched-pairs signed-ranks
figures illustrate the large number of students in the 12-hour classes who scored marks
which were higher in the final tests than in the initial ones.
Figure 13
Student performance gains: 2-hour and 12- hour classes
Performance gains versus 2-hour and 12-hour classes
35
30
25
20
Test 1
Test 2
15
10
5
0
Reading:
2-hour classes
Reading:
12-hour classes
Writing:
2-hour classes
Writing:
12-hour classes
Class type
127
The independent-sample T-test, which tested for equality of variances between the
two groups of students showed significance values of p>0.05, therefore the variances of the
groups were not significantly different from each other in any of the tests (Table 39).
The data shows, therefore, that while classes which constitute 12 hours a week of
instruction are more effective for literacy study than those which offer two hours, both are
of benefit to students. Thus, even two hours of instruction a week for a period of 20 weeks
can be effective, if attendance is consistent and other factors do not disrupt the study
programme. The graph in Figure 13 illustrates the gains made in literacy for the 2- hour and
12-hour classes.
A comparison – reported and assessed literacy
The results for both the self- and the performance assessments have so far shown the
gains made in the literacy programme undertaken by the research participants in the study.
They have also indicated ways in which the instruments could be revised and fine-tuned to
more accurately reflect the processes and order in which learning takes place.
A comparison of the means of the initial self- assessments and literacy test shows
that students were overestimating their abilities in both reading and writing (See Table 40).
A paired-samples T-test was used to compare and test the difference between the means of
the initial self- and performance assessments. The difference was significant in both reading
and writing assessments. When converted to a 'Level' score, based on the 8-point
descriptors of achievement for the programme, there was a difference of a whole level
between what the students felt they could do and the functions which they sho wed they
could realistically perform when tested.
By the time the programme finished however, there was a greater congruence
between reports and actual test results of writing proficiency, although students had
underestimated their reading proficiency by a level when compared to their reading test
result. This confirms the earlier premise that by the end of the programme students seemed
to have greater awareness of their shortcomings in literacy.
128
The participating students consistently showed a higher reading ability than writing
in all self- assessments and tests – a result which was expected, as it is usually easier to
acquire passive language skills than active.
In order to validate the self- report scales, the Wilcoxon matched-pairs test of
statistical significance was used to match the self-assessment and test results of each
individual student, and this is illustrated by the figures in Table 40. The result showed
significant differences between individuals' self- assessment and test results for the initial
assessments in reading and writing. They were also significantly different for the final
reading assessments. The result of the final writing self-assessment and test showed nonsignificant differences, which means students were estimating their writing proficiency with
a greater degree of accuracy. This was also borne out by the results of the paired-samples
T-test, which showed non-significant variances between the two procedures for the final
writing assessments, and significant variances for the other three assessment events (see
Table 40).
Finally, the Spearman correlation coefficient indicates that generally, there was a
positive relationship between the self-assessments and the tests. This seemed to be stronger
for the reading assessments than for the writing. However, they do provide general validity
for the self-report scales, since the majority of the respondents who were reporting their
proficiency at relatively high levels were, in fact, likely to be rated higher on the
performance scales as well.
129
Table 40
Reported and assessed literacy: Time 1 and Time 3, pre - and postprogramme
Assessment: Means
N
Level
Reading
N Level
Writing
Mean s.d.
Mean s.d.
Reported Time 1 62
4+
22.29 9.18 62
3+
18.61 9.46
Test 1 62
3+
18.90 8.62 61
2+
14.44 8.82
Reported Time 3 62
4+
24.16 9.15 62
4+
21.26 9.27
Test 2 61
5+
27.38 8.11 55
4
20.13 8.43
Spearman correlation coefficient = rho
Report Time 1 – Test 1
0.570
0.268
Report Time 3 – Test 2
0.550
0.439
Wilcoxon matched-pairs signed-ranks test
Time 1
Reported ability < Test 1
18
26
Reported ability > Test 1
36
34
Reported ability = Test 1
8
1
z
-3.141*
-2.873*
Significance
.002
.004
Time 3
Reported ability < Test 2
36
22
Reported ability > Test 2
22
30
Reported ability = Test 2
3
3
z
-2.670*
Significance
.008
Paired-samples T-test
-1.135
.256
Report Time 1 – Test 1
t
df
Sig. (2-tailed)
-3.287*
61
.002
-3.106*
60
.003
t
df
Sig. (2-tailed)
2.853*
60
.006
-1.273
54
.209
Report Time 3 – Test 2
*These values were significant at p<. 05.
130
Summary: Assessed and reported literacy gains
The results of the final tests indicated the significant gains made in literacy over this
programme of study. For the total test population, this represented a movement of two
levels on the 8-Level table of descriptors (Appendix 7) developed for the programme, in
both reading and writing. The research participants had entered the programme with higher
levels of reading ability than writing. They also finished the programme with better reading
than writing skills. Those who had entered the programme with higher levels of literacy
ability performed correspondingly well on the final assessments.
Analyses were performed between the two groups taught by bilingual and Englishspeaking tutors and found significant gains made in both groups. Significant differences
were found in the variances between the two groups on the final reading assessments,
where those being taught by bilingual tutors appeared to have made considerable gains.
Analyses were also performed between groups having 2-hour and 12-hour weekly
instruction; the mean differences indicated that 12 hours a week of instruction benefits both
reading and writing. Two hours a week of instruction seems to benefit reading more so than
writing ability. Overall, group variances were not significantly different however, which
illustrates the general benefits of literacy instruction, regardless of the contact hours
devoted to it.
When assessed and reported literacy skills were compared, significant differences
were found in the initial reading and writing assessments, and in the final reading
assessment. By the end of the programme, students were making a more reliable estimate of
their writing abilities. The results indicate that further work is needed to fine-tune the selfassessment scales and comparative test performance descriptors to allow greater
congruence between the two, and a more realistic grading of skills in an ascending order of
difficulty. There was evidence, however, that the mid-point at Level 5 of the descriptors
was appropriately placed, and student performance on tasks from Levels 1 - 4 and Levels 5
- 8 were generally falling on either side of this level.
131
132
CASE STUDIES: RESULTS
Introduction
The main objective of this component of the project was to determine the needs of
learners in low- level literacy and ESOL courses.
It was also to identify the teaching
methodologies and course design which best support their progress.
The analysis of the data demonstrated that the needs of the students in the two
classes included in this study were very different. This was made abundantly clear in the
initial test results. The range of raw marks for Class 1 was 44 – 171, with the median mark
being 107, which was approximately half of the total score. These students had recently
arrived under the family reunification criterion. All were literate in their first language, and
all had had a reasonable amount of previous education. These characteristics would be
expected to enhance the students’ capacity to make progress. Their age range was similar to
that of Class 2. Class 1 had a native English-speaking tutor.
The range of raw marks for Class 2 was 7 – 72, with the average student achieving
44.5. The profile of the Somali women in Class 2 demonstrates the factors which may have
influenced their assessment performance. All were pre- literate and none had had any
education nor, therefore, any experience of classroom learning. This profile is consistent
with that of a considerable proportion of refugees, and has had a negative impact on their
achievement. Class 2 was being taught by a bilingual tutor.
The results of this study confirmed that there was a range of levels and needs within
the sample of students selected for literacy and ESOL courses. The fact that each group has
different starting points at the commencement of the course must be reflected in both the
course design and its delivery. Prior knowledge of the students’ first language literacy and
prior schooling should inform class placement and expectations regarding student progress
and achievement. The profiles of the two different classes reflects the differing students’
learning needs.
The data indicated that the expectations of the students in terms of the outcomes of
the course also differed. While all of the interviewees stressed the importance of learning
133
literacy and English to improve their participation and independence, Class 1 stressed that
their aim was finding employment, whilst the Somali women in Class 2 hoped to achieve
competence in everyday tasks. Both students and tutors expected that the course would
have practical outcomes. As one student commented, her ultimate goal was to have “a
happy and successful life” in NZ.
Tutors felt that it was vital that the course design was responsive to student needs. If
course materials and activities are based on real- life situations, this enhances the students’
motivation to learn, as they can see the relevance and usefulness of the language.
Both tutors identified a delay in the confirmation of the course curriculum and the
content of the first assessment. This was viewed as problematic by Tutor 1, who also felt
that the assessment became the focus of the course, which was confusing for students and
demanded a certain amount of guesswork for the tutor. At the outset of the course, Tutor 2
based her curriculum content and the introduction of literacy skills on the students’ needs.
The fact that this delay was of less significance to her perhaps reflects the lower level needs
of her students and the foundation skills which they had yet to develop.
Both tutors felt that classrooms should have a supportive and respectful atmosphere
where mutual trust has been established. They identified the essential components of course
delivery as continual repetition, and the introduction of new language in a carefully
controlled and appropriately-paced manner. They agreed that realia and pictures were the
most effective teaching resources for low- level students.
In terms of the characteristics of successful learners, the tutors ranked literacy in the
students’ first language as the crucial factor. Youth was a definite advantage, as the elderly
have difficulty in retaining new learning. Having previous education and prior familiarity
with classrooms and learning were seen as being very important. Having confidence,
motivation and study skills were the other indicators of a successful student.
The students were unanimous in their support for the timing, duration and location
of the courses. Both classes expressed appreciation of their tutors and all students were
satisfied with the progress achieved in the course. All students identified the need for
134
furthe r study, and getting employment was the ultimate aim of most. All students had a
positive view of NZ and NZers.
The Asian students all professed a sense of it being their duty to undertake English
study, as the government was supporting them. One of them also felt reluctant to comment
on his tutor’s teaching methods, as this was perceived as disrespectful.
There was some frustration regarding admission to TOPs courses. This was
perceived as being a very slow process for those with employment aspirations, and there
were particular problems related to eligibility and level of English proficiency. Two
students suggested that the courses could be linked with employment or practical activities
such as sewing and cooking.
Specific case studies: Research findings
Student interviews
In each of classes 1 and 2, three students who had achieved at the upper, middle and
lower ends of the initial performance assessments were interviewed with the support of
interpreters. The interviews were of 30-40 minutes’ duration. Class 1 had a native Englishspeaking tutor and class B a bilingual tutor.
The highest achiever in Class 1 was a young Vietnamese woman who had been in
NZ for less than a year, and had previously completed between six and eight years’
education. She was unemployed.
When asked about why she had chosen to study English, she explained that she
could not speak the language on arrival. She understood that it was essential for building a
satisfying and successful life in NZ. She had chosen this particular course because it offered
a higher number of hours of study per week. She expected to learn from her fellow students,
and hoped to learn language that would be useful in her everyday life. She felt that she had
grown in confidence during the course and that she had made good progress. She identified
learning street names, how to shop and catch a bus as useful components of the course.
135
She described her enthusiasm for attending the class and the challenge of learning
something new. Understanding NZers’ ve ry fast delivery of spoken English was her
greatest challenge. In terms of which classroom activities she found most enjoyable, she
identified listening to tapes, reading and working from pictures. She described the timing of
the course as being convenient, as transport could be a difficulty in the evening. She
commented that as the government was paying a benefit, it was her duty to study in order to
find employment.
In terms of the social outcomes of her English study, her vocabulary was becoming
more extensive, and she was learning about the NZ way of life. She commented that the
lifestyle in Asia was so completely different that everything was new in NZ and had to be
re-learnt.
When asked about how well she had settled, she responded by explaining that she
had been very lonely initially, but had found people friendly and helpful. The climate suited
her and she hoped to learn English quickly and find a job. She acknowledged, however, that
she would need to enrol in another course in order to achieve this aim.
Participant 258 achieved an average mark in the assessment task. He was a
Vietnamese man aged between 31 and 40 years, who had been in NZ for less than one year,
and had been repairing electrical appliances from home. His reasons for studying English
were to improve his interaction with NZers, and to get a job in order to support his family
rather than remaining dependent on the government. He had chosen this particular course in
order to accelerate his English learning.
He described himself as under-confident at the start of the course, but he had
improved, particularly in listening, as he could now identify individual words but had yet to
understand the structure of English. He acknowledged the inaccuracy of his pronunciation.
He felt a little dissatisfied with the pace of the lessons, describing them as too slow. He
refused to comment on the teaching techniques that he enjoyed, remarking that it was the
teacher’s role to design the lessons. He did describe the NZ style of teaching as being
different from that in Vietnam.
136
He explained that he was accustomed to working all day and that he felt nervous
and insecure about having to think and learn at this stage of his life. He suggested that an
ideal combination would be study during the day and having a job in the evening. He was
finding it difficult to make friends in the class, commenting that his fellow students
“weren’t open”. He also felt that the students would have benefited from the class being
bigger. In terms of settling into life in NZ, he felt that the course was helping him
considerably in his new life. He was now able to shop independently, and was beginning to
understand English when it was spoken to him. He described himself as feeling sad on
arrival but, with increased familiarity, he was enjoying being in NZ.
He described himself as having grown in confidence since the start of the course. He
felt the course had given him a good start, and he hoped to continue his English study. His
ultimate goal was to gain sufficient proficienc y to attend a TOPs course.
Participant 261 was a Cambodian male aged over 50 years. He had completed eight
years of formal education. His was the lowest mark in the class. He described his reasons
for studying English as being to interact with NZers and to achieve a better future in this
new country.
He agreed that by being more substantial than previous courses he had taken, this
course was likely to extend his knowledge of English. He explained that it was not difficult
for him to attend class, as he had nothing else to do. He expressed thanks to the government
for providing the course.
He said that he was very glad to come to lessons, as he enjoyed interacting with
others. When asked about the learning activities he found challenging and easy, he
commented that he struggled to understand the tutor’s instructions, and that he still needed
to master the basics of English. He identified his progress as becoming more confident in
using the language and improving his listening skills.
He identified the opportunity to establish good relationships with his fellow students
as another highlight. He felt he was getting to know himself better and getting to find his
way around. He described NZ as a law-abiding country where the lifestyle is modern and
137
sophisticated, and NZers are happy. He also anticipated enrolling on another, similar course
to continue his preparation for finding work.
Participant 36, a Somali woman, achieved the highest score in Class 2. She had been
in New Zealand for less than a year and was aged between 21 and 30 years. She had no
previous education or employment.
She described her motivation for studying English as being due to the fact that
English is an international language and that she needed the language to converse with
others and to get a job. She also acknowledged that learning English for 12 hours a week
would enhance her rate of progress. She described herself as feeling happy and relaxed in
class and that the course was providing a good foundation for her English studies. Whereas
she began the course understanding no English, she was now able to fill in forms, answer
the telephone and go shopping using the new language. She could also comprehend some
English on television. She was appreciating being able to now understand others using
English, and being understood when she used it herself. She felt that with increased English
literacy and language skills, she was now better able to cope with everyday life in NZ.
She gave an enthusiastic response regarding the quality of the course, commenting
that her favourite classroom activities were multiple-choice comprehension exercises, cloze
tests, speaking and listening. She said that the timing of the class was ideal for her,
particularly as she had no children. She was appreciative of the location of the course, as
she was able to walk to class. Furthermore, she was enjoying the camaraderie of the course
and the opportunity to chat with fellow students. She said that she often received support in
solving personal problems.
She described herself as feeling happy in NZ, as it was a peaceful and beautiful
country occupied by friendly people. Her aspirations were to acquire enough English to
participate fully in life in NZ. She also hoped to continue her English study.
Participant 46 achieved in the middle of the class in the initial tests. She was a
Somali woman of over 50 years of age who had been in NZ between three and five years.
She had no previous education and had never had a job. She described her reasons for
studying English as wanting to speak the language spoken in NZ. By achieving some
138
proficiency, she hoped to gain independence in everyday tasks such as banking and
shopping.
In terms of the course, she said that longer hours were helping her learn more than
the one to two hours weekly she had previously received from a home tutor. She explained
that she had gained sufficient confidence in using the English language to answer the
telephone, fill forms, give her address and answer her children’s questions.
She felt that the bilingual delivery of the lessons provided a “short cut, has made the
study easier and has supported more rapid progress". She also commented that she would
like to learn English linked with activities such as sewing, first aid and cooking.
In terms of the classroom activities, she said that she liked taped listening exercises,
but found grammar and sentence-writing difficult. She was using television to practice her
English, and acknowledged that she appreciated the social atmosphere of the class and the
opportunity to “get together” with the other women. She said that they could talk through
problems and chat about their children. This eased tension and enhanced the women’s sense
of well-being. She also praised the convenience of the timing and location of the classes.
She described herself as feeling comfortable in NZ, as it is safe and “clean and
green”. She commented that “nobody bothers you”. At the conclusion of the course, she
wanted to continue with her studies and, some time in the future, attend university.
Participant 39 was a Somali woman who received the lowest score when assessed.
She was older than 50 years, had been in NZ for between one and two years, and had never
attended school or worked.
She wanted to attend the course as she wished to have enough English and literacy
to do practical tasks. She described becoming able to do banking and shopping as
important. She attended this particular course because she liked the bilingual teacher, and as
her children were at school during the morning, a class in the evening would present
transport problems.
139
She described herself as feeling confident in class, as the atmosphere was good and
the teacher supportive. She explained her progress by commenting that she was unable to
write her name when she arrived at the class and that she could now do that and more. In
terms of why she liked the course, she said that it was bilingual, that the students were all
African and all women. She said that these factors meant that she could get help with
problems and talk with friends.
She identified cloze tests as her favourite classroom activity and said that she had
learnt how to use English to shop, socialise, help her children and go to the doctor. She
could use her newfound literacy skills to fill the gaps in a reading passage, copy and read
some English. She hoped to attend another course.
Tutor interviews
Both of the tutors were women with considerable teaching experience. Tutor 1 had
worked for the South Auckland ESOL Home Tutor Scheme for the previous three years and
had previously taught adult literacy. Tutor 2 had worked for many years in the Ministry of
Education in Mogadishu, Somalia, before teaching ESOL in Dubai. She was working on a
part-time basis for the West Auckland Home Tutor Scheme. Tutor 1 was NZ-born and
Tutor 2 was from Somalia.
In response to the first question regarding a definition of literacy, Tutor 1 described
it as the ability to read and write in one’s own language. Tutor 2 agreed but added that
literacy in a second language meant being able to manage everyday life in the new
linguistic context.
Question two explored the tutors’ philosophies of teaching. Tutor 1 felt that it was
important to be responsive to students’ needs in terms of the curriculum. She felt that
courses should include the functional language that is useful in everyday life. This would
support the development of the students’ participation and independence. Other long-term
goals would be to enhance the students’ quality of life and enable them to socialise. It was
also seen as important to establish a friendly and supportive classroom atmosphere. If the
tutor was able to assess and target the level of the students, progress could be achieved.
Tutor 1 acknowledged the difficulties of finding appropriate resources for adults learning
140
basic English. She gave the example of finding resources for teaching the alphabet, which
could sometimes be demeaning, as they were designed for children.
Tutor 2’s response was to highlight the significance of establishing a relaxed
atmosphere in the classroom and demonstrating respect for the students. By being
supportive and responsive to their needs, the tutor could support the students’ learning. She
described the process of moving from what was within the students’ experience to the
unknown when teaching new material and using all of the senses to achieve the necessary
repetition and memorability. Tutor 2 favoured activity-based learning in real situations.
The third question was regarding the teaching methods which are most effective in
the delivery of low-level ESOL courses. Tutor 1 recommended using materials that relate to
everyday life. She felt that new language should be introduced slowly, building on previous
learning. It was vital to allow sufficient time for new learning to consolidate. There should
be a logical progression within and between curriculum topics. Realia and pictures were
useful teaching aids. She felt that matching pictures with new words was a useful exercise,
as unfamiliar concepts could be illustrated. Re vision was a vital and regular component of
the programme. Due to the emphasis on developing literacy skills, she had included the
teaching of phonetics, consonant blends and diphthongs at the commencement of the
course. Finally, she thought it was important to balance the practice of the four skills.
Writing was the most challenging skill for all of her students.
Tutor 2 considered that it was invaluable to give classroom instructions in the
students’ first language, as this aided comprehension and enabled a variety of classroom
activities. This latter point was especially significant in low- level ESOL courses, as much
repetition was necessary. She was emphatic that the target language should predominate in
the bilingual classroom but that, for example, translation of a new word could enhance
student learning. She described the usual sequence when introducing an activity as giving
instructions in Somali for participating in a dialogue, and then the students conducting the
dialogue in English.
She agreed that students in English-only classrooms got bored and frustrated due to
their limited comprehension. She cited the example of delivering a language lesson in
Somalia to Home Tutor trainees. She would not allow them to write in English and refused
141
to speak English herself, which resulted in much confusion, anxiety and frustration, as the
trainees experienced how pre- literate students feel in conventional classrooms.
Tutor 2 also described how she designed many of the classroom materials herself,
especially at the beginning of the course to ensure their relevance and appropriateness for
the students. If materials were sourced elsewhere, she would modify them to reflect the
level and interests of the students. She explained that it was essential to use a large print
size in handouts as many of the students had poor eyesight.
Question four asked the tutors how they catered for the needs of low- level students.
Tutor 1 described typical classes of this kind as having at least two proficiency levels. She
commented that the impact of this was far greater in 12-hour weekly classes than in the 2hour ones that were previously the norm. She felt that the greatest challenge was to be
responsive to the needs of the students at the bottom end of the class. A technique she used
was to modify the level of difficulty of questions according to the student’s proficiency
level. Generally, she used the same classroom materials for all students. She had also found
it effective to vary the combinations of students in pair work. It was particularly successful
when more able students were paired with the students who were struggling. The learning
was enhanced when the pair shared the same first language. She has also found that
establishing teams based on gender created a lively competitive and dynamic atmosphere.
Tutor 2 addressed the pastoral element of her role in terms of catering for student
needs. She described how she often changed the timing of her students’ appointments and
accompanied them to meetings or appointments, acting as an interpreter and mentor. The
class also had discussions about current events such as the possible repercussions of the
September 11th disaster. They were all very afraid of a potential backlash, and discussed
how they should behave should they or family members be abused in any way. Using
emergency telephone services was recommended and became an activity practised in class.
When asked to detail the characteristics of students most likely to succeed in their
literacy and English learning, Tutor 1 responded that those students who were willing to
practice and revise would do well. They needed to be able to listen carefully and make the
necessary changes. They should be relaxed about making mistakes, accept the inevitability
of confusion, and test themselves in terms of their language learning. They should have
142
good powers of concentration and an understanding of the teacher’s expectations and
classroom routines. Having confidence, being young and having a good education were
perceived as significantly beneficial factors. The successful student also needed stable
personal circumstances, as illness in the family or responsibility for many children could
interfere with attendance and concentration.
Tutor 2 agreed that the younger students were more likely to succeed, as the older
students had poor memories and had difficulty retaining new knowledge. She commented
that the younger students were benefiting from the longer hours of study and the enhanced
opportunities for revision. They also had input into the content of the course and they
requested topics such as catching the bus, using the telephone and going shopping. The
tutor’s responsiveness to their needs had increased their motivation to learn. Being literate
in their first language was identified as a critical indicator of potential success. This
provided the foundation for learning a new script and set of language rules. Being able to
use the first language to teach the sounds and shapes of the letters in the alphabet was also
advantageous.
The fifth question was regarding outcomes for the students at the conclusion of the
course. Tutor 1 responded that some of the students were disappointed with their progress
and most wished to continue their English study. While their confidence in using English
had improved, few were sufficiently proficient or eligible for a TOPs course. Several had
made sufficient progress to successfully compose a letter to a school explaining an absence.
Tutor 2 hoped that her students would leave the course with mastery of some simple
tasks in English. They had learnt to write appointments and the date, make a shopping list
and fill in forms with personal information. They could use a calendar and identify days of
the week and months of the year. They understood alphabetical order and had experience of
using a dictionary. They had been introduced to reading maps, and could identify where
they lived and ask for directions. They had learnt to tell the time and would experience
catching a bus and reading simple timetables before the end of the course.
Question six related to the assessments being used in the course and their
relationship with curriculum development. Tutor 1 suggested that there could be a danger in
putting too great an emphasis on assessment, as the test becomes “an end in itself”. To
143
avoid this, she had endeavoured to incorporate the teaching of the necessary skills within
the topics. There was some concern that the topics were too broadly-defined. Examples
given were “write about familiar things” and “read a simp le story”. Furthermore, the actual
public signs that students should be able to read were not identified. Delivery of the
curriculum would have been facilitated by a more prescriptive outline of appropriate course
content. The tutor felt that the students did not understand the purpose of the test. Concern
was also expressed about the curriculum not being available prior to the initial test, and that
considerable time was lost teaching the language of tests such as “underline”, “circle” and
“match”.
Tutor 2 acknowledged the relevance of the assessment to the curriculum, but
pointed out that the latter was not formalised until after the commencement of the course.
She did not see this as problematic, as a teacher of literacy is experienced and wellequipped in terms of the content of the teaching and appropriate teaching materials. She
commented, “When you have a student who can’t read and write, who doesn’t know the
sounds or systems of the letters, that is where you start…. There is so much for the students
to learn, and this provides the basis of a curriculum”. The content Tutor 2 chose to teach
was reflected in the items included in the assessment.
When asked if any additional forms of assessment had been utilised, Tutor 1
described the informal assessment she used daily in the form of monitoring the
effectiveness of students’ learning. Tutor 2 also used informal tests such as matching the
sounds and shapes of letters, and dictation to check the students’ progress and allow for
reinforcement of new learning.
Classroom observations
Literacy classes were being delivered as blocks of three hours in the mornings from
Monday to Thursday.
Class 1 consisted of 11 students, six being female and five male. Eight were from
Vietnam, and there was one student from each of the following countries: Cambodia, China
144
and Iran. Students were seated at desks arranged as three sides of a rectangle and the tutor
led the class from the front. The teacher was a native speaker of English.
Class 2 consisted of 13 students. The majority were from Somalia and there were
two students from Ethiopia. The tutor’s first language was Somali but she was also fluent in
Arabic, which she could use with the Ethiopian students. The classroom was arranged in the
conventional manner with rows of desks facing the whiteboard from where the teacher led
the class.
The content, resources and related activities of Class 1’s lesson were as follows:
There was a brief discussion about fireworks at the commencement of the class. The
students then revis ed the use of “how long does it take to…” as a whole class exercise, with
one student questioning another using a prompt provided on individual slips. An overhead
transparency (OHT) was used to introduce the concept of safety, as well as the use of the
auxiliary verb ‘should’ plus an infinitive. This was practised orally, using a handout with
partial sentences. The students then completed this individually. The teacher then modelled
the use of the negative form 'shouldn't'. The class was asked to think of the advice they
would give to a new student attending this class for the first time. They worked in groups,
wrote their sentences on the board and the class worked together to make corrections.
Class 2 was revising dates of birth and dates of arrival in NZ. They began with pair
work, asking their neighbour about their date of birth. Their tutor then worked one-to-one
with students to check the accuracy of their reading of dates. The tutor then demonstrated
how to express a date in figures. The class completed a handout to practice. They then
listened to a tape and circled the appropriate date on a work sheet. These exercises were to
be followed by form- filling, using the new learning.
Both tutors attempted to link the prior experience of students to the content of the
lesson. Tutor 1 checked which side of the road was used for driving in the different
countries of the students and referred to the commencement of the course as a lead- in to the
work on ‘should’ and ‘shouldn’t’. There was an emphasis on speaking and listening in
Class 1’s lesson, but students also did some writing and reading. Practice of the skills of
reading and listening predominated in Class 2, but some writing and speaking did occur.
145
The level, content and pace of the lessons differed. Cla ss 1 focused on the structure
of English, and the content was generally at a higher level. The pace of Class 2’s lesson was
slower and geared to student comprehension. There was more one-to-one interaction
between tutor and student. There was greater attention to repetition and the successful
learning of individual students. Apart from one occurrence, where the mother tongue was
used to explain a new word and clarify instructions, both teacher and students in Class 1
used the target language only. However, the issue was raised by the students, when they
were referring to the class rules to be passed on to new students, that not speaking your own
language was preferable. Class 2 used Somali and English, with the teacher using Arabic to
clarify instructions and the lesson content for the Ethiopians on an individual basis. The
proportion of English versus first language speaking was approximately 60% to 40%.
The tone of the classes was different. The students in Class 2 were very involved in
their work and had been trained to do revision when the teacher was marking individual
work. They were motivated, attentive, and clearly felt some sense of achievement from
their learning and the completion of tasks. Class 1 was equally interactive, but the range of
students’ English proficiency and the lack of reference to first languages did alter the
energy level and degree of participation.
Summary: Case studies
This research project used multiple data sources to “illuminate the immediate
experience of the student in the lived culture of classrooms” (Nunan, 1992). It linked
educational theory and practice to identify the factors which promote successful learning.
The case study methodology provided data which revealed the detail of the educative
process: students’ experience of the course, the learning situations, tutors’ teaching styles
and classroom practice.
As described in the literature review, cross-cultural research is fraught with
difficulties. To undertake such research demands particular characteristics of the initiator.
Of crucial importance is an understanding of the culture and traditions of the participants,
and having long-term involvement with them in order to develop credibility with the
communities concerned.
The researcher conducting the case studies had established
146
working relationships with refugee communities, had developed cross-cultural research
methodologies for working with vulnerable groups and had managed on-arrival education
programmes for refugee families.
The research design and data collection was further strengthened by the use of
interpreters from the same ethnic background as the interviewees. Participants could
express their opinions in their first language and the cultural appropriateness of the
methodology could be checked and the accuracy of interpretation of questions and answers
confirmed. The bilingual interviews achieved a depth of detail regarding students’
perceptions of their learning not readily available to educators.
The method provided
access to individual student’s motivation for study, their own perceptions of progress, the
experience of arriving and surviving life in NZ, and their aspirations. Such invaluable
information regarding beginning students of English is generally not available to their
tutors, due to their limited English proficiency.
The use of the tutor as the interpreter for the Somali interviews may have influenced
the range of student responses. However, it could also be argued that her familiarity and
support may have alleviated the formality of the unfamiliar situation, and the fear and
mistrust which could be associated with interviews.
The case study aimed to describe, understand and explain the processes of literacy
gain by focussing on the perspectives of the people involved, and the natural context of the
interactions. The consultative approach used in the project successfully identified the
expectations, perceptions and experiences of the students and tutors involved in the literacy
and ESOL course. Their contribution to the study was to provide a “rich, descriptive, reallife, holistic account” (Burns, 1994: 479) of the process of English teaching and learning.
This has the potential to inform and improve future educational action.
Another aspect of the study which requires further comment is the utilisation of
bilingual delivery for pre- literate students. Clearly, there are often practical difficulties
which preclude this choice, such as the lack of suitably-qualified staff, and student numbers
not justifying an ethno-specific class. However, the case study does confirm the benefits of
this approach.
147
The bilingual tutor's class chosen in this study consisted of Somali and Ethiopian
women. As they themselves said, this created a relaxed and friendly atmosphere which was
conducive to the efforts of new learners. In addition, the bilingual delivery of instructions
facilitated a variety of teaching and learning activities. This also made it possible to link
new language features to the characteristics of the students’ first language. These features
of the bilingual classroom enhanced the students’ learning, and considerably reduced the
frustration and boredom evident in many low- level ESOL classrooms.
The Somali class also had a holistic quality in that, in addition to literacy and ESOL
learning, current issues could be addressed, practical support offered and cultural
orientation taught. It was possible to discuss the attitudes and practices of the host society,
and address problems common to all students in the class. In addition, the students
benefited from the interpreting, advisory and practical support offered by the tutor.
148
149
DISCUSSION AND RECOMMENDATIONS
The main purpose of the study was to gauge if literacy gains were being made on a
20-week literacy programme for pre- literate and low-level literacy adult ESOL students in
Auckland. The students were being taught at three different sites – managed by the West
Auckland and South Auckland ESOL Home Tutor Schemes. The students were receiving
instruction from bilingual and native English-speaking tutors, with a majority attending 12hour weekly classes and a minority, 2-hour weekly classes. Although there were 118
students enrolled in the programme over the course of the semester, 62 students completed
most of the tasks which were set as part of this project and, for the purpose of data analysis,
these 62 participants were selected.
Four main methods were used to study the students, the programme, and to
determine literacy gains. These included personal information profiles of the students, casestudy procedures, self-assessment procedures which were conducted at the beginning, the
mid-point and the end of the programme, and literacy tests which were conducted at the
beginning and end of the programme. The self-assessment tasks and the literacy test tasks
were parallel to each other in order to obtain a realistic comparison and to validate the selfassessment scales, which could then be used instead of actual tests in future programmes. In
addition, six case studies were undertaken to enable a qualitative look at the programme.
At the beginning of the programme, tutors and students were asked to fill in
personal profile forms, which provided an insight into some of the factors affecting
teaching, learning, and literacy gains among pre- literate and low- level literacy learners who
come from disadvantaged and often traumatic backgrounds. Eight tutors participated in the
project – two native English-speakers and six bilingual tutors. All the bilingual tutors had
arrived in NZ as adults and English was their second, third or fourth language. Despite this,
four of the bilingual tutors preferred to speak English bilingually with their own language at
home – an area that needs further investigation as language shift within the first few years
of immigration is fairly unusual amongst adult immigrants.
The tutors were well-qualified and experienced, and all were highly motivated to
teach those in their community who needed literacy help. They were very happy with the
level of support they had received from the ESOL Home Tutor Schemes and staff they were
150
working with, but expressed concern about continuity of funding, the transient nature of
classes, irregular student attendance, the inadequate number of hours spent each week in the
classroom, and the need for further in-service training. Some tutors were also worried about
student attention-span and forgetfulness – which may partly be due not only to the nature
and duration of classes but also to the background of the students. Finally, tutors wanted to
see a further revision of the curriculum, with clearer outlines, objectives and outcomes
linked to specific topics and task-types. They also wanted easier assessment procedures
because of student difficulty with these. A majority of students had never previously
experienced a test-taking session. Collaborative work on the tests and the process of getting
self-reports of literacy were perceived as real problems, needing to be worked into future
test procedures with learners of similar backgrounds at this level.
The student personal profiles also served to provide information about the students’
current perceptions of their English language proficiency, and their hopes and expectations
of the programme at its inception. All the students were first-generation immigrants and
refugees; most had arrived in NZ when they were over 21 years of age and had lived here
for less than five years. They had varied first languages and mother tongues, and wideranging first language literacy levels. They originated from seven different countries.
Over 90% of them had been unemployed in NZ, and prospective employment as
well as the need to use English in a range of survival situations such as filling out forms,
understanding TV, and adapting to NZ society had prompted them to attend the literacy
classes. Only half the participants had received a primary school education.
As with the bilingual tutors, the students too, showed evidence of language shift, as
over 40 percent of them stated that they preferred to use English bilingually with their first
language/mother tongue at home. Gender, country of origin and mother tongue seemed to
be the three variables most affecting language shift. Somali, Afghani and Cambodian
participants were most likely to maintain their home languages.
Bilingual tuition seemed to be in demand for over two thirds of the participants at
the start of the course – a question that would have been good to follow up during final
course evaluations. Participants were divided in their opinion of how the host community
perceived them if they spoke their home language in public. Half were comfortable with it,
151
but the others felt uncertain. Both English and first language proficiencies showed the
passive skills (listening and reading) being stronger than the active skills (speaking and
writing). First language literacy was stronger than English literacy, and had a significant
positive effect on the acquisition of English literacy. English literacy was also affected by
gender, with women displaying weaker abilities. Finally, the newest arrivals had less
competence than those who had arrived earlier in NZ, showing that length of residence had
a significant effect on English literacy in this group.
While it is acknowledged that ‘literacy’ means a great deal more than just reading,
writing and numeracy skills, for the purposes of assessment, these were the primary skills
assessed. Other aspects of literacy, particularly those influenced by the affective domain
and background variables, were covered in the personal profiles and case studies. These
included a gain in confidence in social interaction and progress towards the students’ aims
of preparing for further study, employment and providing more effective support of their
families. They increased their independence by developing coping skills for everyday life
such as using public transport and the telephone. The students also benefited from the
advice and mentoring provided by their tutors, and their cultural orientation was
strengthened in terms of their increased familiarization with the NZ lifestyle and
contemporary issues.
Although all three self-assessments were useful in this study for perceived literacy
gains, it was apparent that having two – the initial and the final would have yielded similar
information. However, having a self-assessment at mid-point allowed the follow-through of
trends in perceived gains by the students. The use of the scale in the initial self- assessment
showed that students tended to feel they were more able to perform numeracy tasks and
meet de-contextualised language demands than to produce language for real functions such
as the filling out of forms. This had, however, changed by the final self-assessments, with
student reports of progress matching their results to a far greater extent.
Students felt they had made significant gains in both reading and writing. An
independent-samples T-test showed that variances in perception of literacy gains between
those taught by English-speaking and bilingual tutors were non-significant, although gains
made by individuals within each group using a Wilcoxon matched-pairs test showed that a
152
majority of students taught by English-speaking tutors felt they had significantly improved
in their literacy ability by the end of the programme.
Variances in reports between the two groups receiving 2-hour and 12- hour weekly
instruction were significant only for the final writing self-assessment, with students in the
12-hour group feeling they had made significant gains in literacy in both reading and
writing.
When the students were actually tested, there was clear evidence of significant gains
having been made in literacy during the research period for the wider research sample as
well as for both groups: those taught by bilingual and English-speaking tutors. For the total
sample, this represented an increase of two levels on the 8-point descriptor scale developed
for this project. Reading skills were consistently better than writing in all tests for this
research sample.
Analyses performed between the groups receiving the 2- and 12-hour weekly
instruction indicated the benefits of having more contact time for literacy education.
However group variances between the two were not significant, which would indicate not
only the general importance of but also the gains to be made from any literacy instruction,
regardless of the number of contact hours.
When assessed and reported literacy skills were compared, the closest match was
between the final writing test and self-assessments. Twenty weeks of instruction, and
awareness of the content of literacy education had meant that students were able to assess
their writing ability with a greater degree of accuracy. There was evidence that the test and
self-report descriptors were generally working quite well by the end of the programme, and
the accuracy of the mid-point on the scales was clearly confirmed.
Recommendations derived from this report
1.
That student personal profiles at the beginning of the programme be retained for
planning purposes, so that student needs are appropriately and adequately catered
153
for, given the background, previous education, gender, length of NZ residence and
first language literacy.
2.
That the sequence of tasks in this curriculum be re- visited in the light of student
initial self assessment, so that skills they perceive to be easier are consolidated
before more difficult ones are introduced. This will reduce anxiety among students.
3.
That bilingual and native-English speaking tutors be retained, and further training be
offered to address their current needs. That tutors be encouraged to have a greater
role in curriculum and assessment design and development, lesson delivery, and in
identifying ways in which to enhance student motivation and retention.
4.
That classes have access to bilingual, English-speaking and EAL tutors so that they
have realistic models to work with, given the plurilingual, multi-cultural nature of
the NZ environment. All have their strengths in teaching various aspects of the
curriculum, and it is important that this resource be utilised for all students. Two
thirds of the students indicated they would like bilingual teaching at the start of the
programme, and this is clearly a preferred option at the beginning stages when
student language needs require special scaffolding, and especially when they face
insecurities with language learning, given their traumatic backgrounds.
Older
women in particular – who made up the bulk of this sample – are insecure about
beginning literacy and language learning skills.
5.
That students are given practice in whichever form is chosen to measure their
achievement (self assessment and /or testing). These students need to be given
practice in all aspects of self-assessment and testing methods, including timing, so
that test anxiety is reduced and results reflect their real knowledge.
6.
That funding be obtained for regular 12-hour weekly classes in preference to 2-hour
ones, but having a smaller number of hours as an option for students who are unable
to attend for more hours per week. Some literacy instruction is preferable to none at
all, as students made gains in the 2- hour classes, although not as significantly as
those in the 12-hour classes. All students in these 2- hour classes however, had some
154
primary school education – it is presumed the gains would not have been as great in
these classes if they had had no previous schooling.
General recommendations
1.
That first language and literacy skills be encouraged and supported, as there is a
significant relationship with English literacy skills.
2.
That students and tutors be made aware that home use of the mother tongue is
crucial for cognitive development and that a shift to bilingual language use with
English may threaten acquisition of both languages, particularly among younger
children and in the NZ-born generation.
3.
That low- level literacy and ESOL programmes:
•
Acknowledge that the diversity of learners requires a range of educational responses
•
Use a curriculum responsive to student needs
•
Use classroom materials and activities based on relevant, useful and real- life situations
•
Formulate curriculum and assessment tasks stemming from the profiles of students at
the beginning of the course
4.
That the student be given a progress and achievement report at the end of the
programme which clearly signals the areas in which the student is now competent.
This will comprise the final test results along with the test descriptors. It will give
the next teacher a clear indication of student ability.
Recommendations for further research
1.
That further work be done looking specifically at the variables which affect
language literacy and English literacy gains among refugee pre-literate and lowliteracy ESOL students, so that tutors are made aware of the factors which might
intrude on or further motivate their students and encourage greater gains.
155
2.
Collaborative learning is appropriate to the culture of these learners and therefore
methods of collaborative test-taking could be explored.
3.
The use and shift of home languages be documented and the variables which seem
to contribute to this be explored.
Conclusions
This research project was useful and worthwhile in that it allowed the gathering of
sufficient data for making sound judgements about literacy gains for 62 students enrolled
in a programme for pre- literate and low- level literacy ESOL students. It was clear at the
outset that test anxiety meant that testing procedures were not as stringent as they could
have been, as students often worked collaboratively in order to ‘do well’ for their teacher.
However, by the end of the programme students were more aware of the testing
procedures, and the self- assessments and test results were more evenly correlated. The
following aspects of the assessment procedures used will be valuable in future
programmes: student personal information profiles which gather pertinent background
information on the students; a revised initial self-assessment which could form part of the
profile and which reflects prior knowledge; a fina l self-assessment and/or literacy test to
acknowledge progress and literacy gains; and a clear reporting procedure which reflects
student achievement and reports their ability to perform functional tasks in the real world.
This would be helpful for both record-keeping and for passing on to the next class teacher.
This report indicated the very real and significant literacy gains that learners on this
programme made. It is therefore important that the issues of student retention, continuation
of funding, on-going training of bilingual tutors, and continuity of learning for the students
be addressed so that these gains are not lost in the long term.
.
156
157
REFERENCES
Abbot, M. 1989. Refugee Resettlement and Wellbeing. Based on national conference on
refugee mental health in NZ, Wellington 12-15 May, 1988. Wellington: Mental
Health Foundation.
Allen, P., M. Fröhlich and N. Spada. 1984. The communicative orientation of language
teaching: An observation scheme. In Handscombe, J., R.A. Orem and B. Taylor
(eds.) On TESOL: '83: The Question of Control. Alexandria, Virginia: Teachers of
English to Speakers of Other Languages, Inc.
Allender, S. C. 2001. Adult ESL Learners with Special Needs: Learning from the Australian
perspective. Victoria: NCLE, Adult Multicultural Education Services.
Adelman, C., D. Jenkins and S. Kemmis. 1976. Rethinking case study: notes from the 2nd
Cambridge Conference, Cambridge Journal of Education, 6, 3: 139-150.
Altinkaya, J. 1998. Briefing paper to Interdepartmental Committee on Refugees - Access to
English language support (ESOL). Wellington: National Home Tutor Association
Inc. Unpublished paper.
Anderson, A. 1994. Studying across differences. In Denzin, N. and Y. Lincoln (eds.)
Handbook of Qualitative Research. California: Sage.
Bachman, L.F. 1990. Fundamental Considerations in Language Testing. Oxford University
Press.
Bachman, L.F. and A.S. Palmer. 1996. Language Testing in Practice. Oxford University
Press.
Bassey, M. 1999. Case Study Research in Educational Settings. Buckingham: Open
University Press.
Brindley, G. 1989. Assessing learner achievement in the AMEP. Prospect, 4, 3: 9-23.
158
Brindley,G. (ed.) 1990. The Second Language Curriculum in Action. Sydney: NCELTR,
Macquarie University.
Brindley, G. 1998. Assessment in the AMEP: Current trends and future directions,
Prospect, 13,3: 59 –71.
Brod, S. 1995. Outreach and Retention in Adult ESL Literacy Programs. Washington DC:
ERIC Digest, ED383241, Adjunct ERIC Clearinghouse for ESL Literacy Education.
Broome J., S. Hardman, M. Hayward and K. McDermott. 1998. Refugee needs in the
Auckland area. Unpublished paper.
Burns, R. 1994. Introduction to Research Methods. Melbourne: Longman Cheshire.
Burt, M. and F. Keenan. 1995. Adult ESL Learner Assessment: Purposes and Tools. ERIC
Digest EDO-LE-95-08, 1-5.
Cambourne, B. 1992. Tests and testing. In Derewianka, B. (ed.) Language Assessment in
Primary Classrooms (Chapter 1). Sydney: Harcourt Brace Jovanovich.
Clark, J.L.D. 1975. Theoretical and technical considerations in oral proficiency testing. In
Jones, R.L. and B. Spolsky (eds.) Testing Language Proficiency (pp 10-28).
Arlington: Centre for Applied Linguistics.
Cohen, L. and L. Manion. 1994. Research Methods in Education. London: Ro utledge.
Cumming, A. 1998. Skill, service, or industry? The organization of settlement programs
for adults learning English in Canada and Australia. Prospect, 13, 3.
Cumming, A. 1992. Access to Literacy for Language Minority Adults. Ontario: Modern
Language Centre, Ontario Institute for Studies in Education. ERIC Digest EDOLE-92-02,1-5
159
D’ Annunzio, A. 1995. The training of bilingual tutors in an English as a second language
program. Adult Basic Education, 5, 1, Spring.
Davison, C. 1996. The multiple meanings of literacy in the TESOL and adult literacy
professions: Problems of perspective? Prospect, 11, 2, August.
Denzin, N. and Y. Lincoln.1999. Case Study Research in Educational Settings.
Buckingham: Open University Press.
Eckermann, A. K. 1993. Research methods in social investigations: Qualitative methods,
Unit AMC 550. (Lecture notes). Armidale: University of New England.
Eckermann, A. K., D. Roberts and G. Kaplan. 1993. The role of participatory action
research in Aboriginal education. Conference paper presented to the World
Indigenous People’s Education Conference, Woolongong.
English Language Institute, 1992. ELI Proficiency Scale. Wellington: English Language
Institute, Victoria University of Wellington.
Fitzgerald, M. 1995. Using cultural informants in research project development. (Seminar
Notes). Sydney: Sydney University.
Galang, R.G.1982. Native Language Assessment of Asian American Children. Los
Alamitos, California: National Centre for Bilingual Research. ERIC Document
Production Service, Document No. 219 973.
Gardner, S., E. Polyzoi and Y. Rampaul. 1996. Individual variables, literacy history and
ESL progress among Kurdish and Bosnian immigrants. TESL Canada Journal,14, 1
Winter.
Goldman, A. and S. McDonald. 1987. The Group Depth Interview. New Jersey: Prentice
Hall.
160
Grant, A. 1986. Defining literacy; Common myths and alternative readings. Australian
Review of Applied Linguistics, 9, 2: 21-22.
Hagan, P. 1994. Competency-based curriculum: the NSW AMES experience. Prospect, 9,
2: 30-40.
Hakuta, K. and D. D’Andrea. 1992. Some properties of bilingual maintenance and loss in
Mexican background high-school students. Applied Linguistics, 13, 1: 72-99.
Hamel, J., S. Dufour and D. Fortin. 1993. Case Study Methods: Qualitative Research Series
32. California: Sage.
Hammond, J. 1989. The NCELTR literacy project. Prospect, 5, 1: 23-30.
Hammond, J. and B. Derewianka. 1999. ESL and literacy education: Revisiting the
relationship. Prospect, 14, 2: 25-39
Hood, S. 1990. Second language literacy: Working with non- literate learners. Prospect, 5,
3: 52-61.
Huerta Macias, A. 1993. Current Terms in Adult ESL Literacy. ERIC Digest, ED359750 15
Ingram, D.E. 1981. Testing practical language skills. Paper presented at the 1981 FIPLV
Congress‚Second and Foreign Languages in Education, University of Ibadan,
Nigeria, August 23rd - 29th, 1981.
Ingram, D.E. 1981. Australian Second Language Proficiency Ratings (ASLPR). In J. Read
(ed.) Directions in Language Testing (pp 108-138). Anthology Series 9 Singapore:
SEAMEO Regional Language Centre.
Johnson, A. 2000. Changing Skills for a Changing World: Recommendations for Adult
Literacy Policy in Aotearoa/New Zealand. Wellington: Ian Axford NZ Fellowships
in Public Policy.
161
Kalantzis, M., R. Gurney and B. Cope. 1986, A study of the relationship between first
language literacy and second language acquisition. Unpublished report, Canberra
DIEA cited in Hood, S. 1990. Second language literacy: Working with non- literate
learners. Prospect, 5, 3: 52-61.
Kalantzis, M. 1987. From first language illiteracy to second language literacy: NESB Adult
Learning. Prospect, 3, 1: 35-43.
Kalantzis, M., Cope, B. and Slade, D. 1989. Minority Languages and Dominant Culture.
London: Falmer Press.
Kasanji, L. 1994. Community language schools and their future. Paper delivered at the Fourth
Community Languages and ESOL Conference, Christchurch,1st - 4th September, 1994.
Kettle, M. 1998. Bilingualism bites the dust, in ‘Washington Diary,’ Guardian Weekly.
California, June 14th , p6.
LeBlanc, R. and G. Painchaud. 1985. Self-assessment as a second language placement
instrument. TESOL Quarterly, 19, 4: 673-87.
Lewis, J. 1990. Self-assessment in the classroom. In G. Brindley (ed.) The Second
Language Curriculum in Action. Sydney: NCELTR, Macquarie University.
Loo, C. 1982. Vulnerable populations: Case studies in crowding research. In J. Seiber (ed.)
The Ethics of Social Research: Surveys and Experiments. New York: Springer
Verlag.
Martin- Blaker, J. and S. Hardman. 2001. Jumping the Barriers: Language Learning with
Refugee Groups in New Zealand. Unpublished paper.
McDermott, K. 1997. A methodological journey: the evolution of a research model
appropriate for an exploration of the factors facilitating resettlement for government
162
quota refugees in Auckland, New Zealand. M.Ed. Thesis. Armidale:University of
New England.
McSpadden, L. 1987. Ethiopian resettlement in the Western United States: Social context
and psychological well-being. International Migration Review, 21, 3: 796-819.
Majka, L. 1994. A Review of Research on Refugees in the Austroasian Region. Melbourne:
La Trobe University.
Martin-Jones, M. 1991. Sociolinguistic surveys as a source of evidence in the study if
bilingualism: a critical assessment of survey work conducted among linguistic
minorities in three British cities. International Journal of the Sociology of
Language, 90, 37-55.
Murray, D.E. 1998. An agenda for literacy for adult second language learners. Prospect, 13,
3: 42-49.
Navarra, D. 1993. Literacy Assessment Tasks for Placement and Referral. Sydney, NSW:
AMES.
Nicholas, J. 1988. British language diversity surveys (1977-87): A critical examination.
Language and Education, 2 : 15-33.
Nunan, D. 1992. Research Methods in Language Learning. Cambridge University Press.
O’Grady, C. 1987. Use of the learner’s first language in adult migrant education. Prospect,
2, 2:171-181.
Paltridge, B. 1989. Placement testing: A study in self- assessment. Prospect, 4, 3: 27-38.
Patton, M. (ed.) 1990. Qualitative Evaluation and Research Methods. California: Sage.
Peyton, J. and J. Crandall. 1995. Philosophies and Approaches in Adult ESL Literacy
Instruction. ERIC Digest ED 3869608, 1-5.
163
Pittaway, E. 1991. Refugee Women - Still At Risk in Australia. Canberra: Australian
Government Publishing Service.
Ramm, J. 1993. Learners With Minimal Formal Education. Victoria: NCELTR, AMES.
Reason, P. 1990. Qualitative evaluation and research methods. In M. Patton (ed.)
Qualitative Evaluation and Research Methods. California: Sage.
Rivera, K. 1999. Native Language Literacy and Adult ESL Instruction, ERIC Digest,
EDO—LE-99-04, 1-5.
Roberts, C. 1994. Transferring literacy skills from L1 to L2: From theory to practice.
Journal of Educational Issues of Language Minority Students, 13, 209-221.
Roberts, M. L. 1999. Immigrant language maintenance and shift in the Gujarati, Dutch and
Samoan communities of Wellington. PhD thesis. Wellington: Victoria University.
Rolfe, T. 1990. Self- and peer-assessment in the classroom: A case study. In G. Brindley
(ed.) The Second Language Curriculum in Action (pp187-287). Sydney: NCELTR,
Macquarie University.
Schumann, J. 1978. Cited in D. Nunan. 1992, Research Methods in Language Learning.
Cambridge University Press.
Shadbolt, M. 1996. The longest journey: the resettlement of Ethiopian and Eritrean refugees
in Auckland. M. Phil.Thesis. Hamilton: University of Waikato.
Shameem, N. 1995. Hamai log ke boli. Language shift in an immigrant community: The
Wellington Indo-Fijians. PhD thesis. Wellington: Victoria University.
Shameem, N. 1998. Validating self-reported language proficiency by testing performance
in an immigrant community: the Wellington Indo-Fijians. Language Testing,15,
1:86-108.
164
Shameem, N. 2000. Methodology in a study of school language use in Fiji. New Zealand
Studies in Applied Linguistics, 6, 67-88.
Shameem, N. 2001. Many Languages, Diverse Peoples, One Nation, Aotearoa: A report on
education needs of community language teachers in Auckland, Aotearoa/New
Zealand. Auckland: School of English and Applied Linguistics, UNITEC.
Shohamy, E. 1983. The stability of oral proficiency assessment on the oral interview testing
procedures. Language Learning, 33, 4 :527-540.
Shohamy, E. and T. Reves. 1985. Authentic language tests: where from and where to?
Language Learning, 8, 1: 23-40.
Smith, H.A. 1994. English Language Acquisition in the Lao Refugee Community of
Wellington, New Zealand. MA Thesis. Wellington: Victoria University of
Wellington.
Smith, H. A. 1997. Ngeun dtem pha bo panya dtem phoung: Social network analysis and
the acquisition of English in the Lao refugee community of Wellington. New
Zealand Studies in Applied Linguistics, 3: 21-45.
Spada, N. 1990. Observing classroom behaviours and learning outcomes in different second
language programmes. In Richards, J. and D. Nunan (eds.) Second Language
Teacher Education. New York: Cambridge University Press.
Spada, N. and M. Fröhlich. 1995. COLT, Communicative Orientation of Language
Teaching Observation Scheme: Coding Conventions and Applications. Sydney:
NCELTR, Macquarie University.
Spolsky, B. (ed.) 1978. Approaches to Language Testing: Advances in Language Testing
Series: 2. Arlington Centre for Applied Linguistics.
165
Stanfield, H. and R. Dennis. 1993. Race and Ethnicity in Research Methods. California:
Sage.
Stevenson, D.K. 1985. Pop validity and performance testing. Language Testing, 2, 1: 4147.
Van Weeren, J. 1984. Language Testing: Making a Decision on the Basis of Observed
Behaviour and ‘Tapped’ Knowledge. In Practice and Problem in Language Testing
the
Interuniversitare Sprachtestgruppe. ERIC Document Reproduction Service,
Document No. 266 643.
White, C., N. Watts and A. Trlin. 2001. Immigrant and Refugee Experience of ESOL
Provision in New Zealand: Realities and Responsibilities. Palmerston North:
Massey University New Settlers Programme, Occasional publication No. 5.
Wickerts, R. 1990. Putting No Single Measure into the wider literacy picture. Prospect, 5,
3: 17- 26.
Wrigley, H. 1992. Learner Assessment in Adult ESL Literacy. ERIC Digest, EDO-LE-92 15
Wrigley, H. 1993. Innovative Programs and Promising Practices in Adult ESL
Literacy.ERIC Digest, EDO-LE-93-07 NCLE.1-5
Newspaper collage items from:
Central Leader
New Zealand Herald
West Weekly
Sunday Star Times
Indian Newslink
166
Sunday Star Times
167
APPENDICES
Appendix 1
Course Document
COURSE TITLE:
Bilingual Tutor Training Course
UNITEC Course Type:
LEVEL:
Assessed Short Course
4
STUDENT LEARNING TIME:
CREDITS:
30
300 hours
AIM
To provide trained bilingual tutors from a refugee background to teach essential
language and literacy, numeracy, and learning skills to small groups in schools and the
community.
PURPOSES OF COURSE
The stated aims in the programme contract are as follows:
• To recruit and train suitable applicants to become tutors within their own community
• To build job search techniques and to develop knowledge and experience of the NZ
work place
• For participants to gain work experience in the NZ labour market
CONTENT OUTLINE:
Life stages and learning; learning styles and backgrounds; needs analysis; tutoring
approaches for literacy, numeracy, learning skills; managing tutor groups; materials
development; using a/v equipment; understanding NZ schools and communities.
COURSE OBJECTIVES
The trainees will be able to:
1. Identify characteristics of adult and teenage learners and describe the basic concepts of
learning and teaching.
2. Identify and assess learning needs and respond appropriately.
3. Implement current approaches to language, literacy and numeracy teaching.
4. Work as bilingual tutors in the community.
INSTRUCTION
This course involves 300 hours of student learning:
•
•
•
120 hours class contact (six 20 – 25 hr block courses over 18 weeks max.)
20 hours supervised tutoring practice
160 hours of self directed learning
SPECIFIC OUTCOMES
• At least one literacy class would be established in each community
• Course graduates would be employed in their communities within three months of
course completion
• Quota refugees would gain access to first step provision
168
•
Subsequently such refugees would be enabled to go on to English language provision in
Beginner or Elementary classes
ASSESSMENT
1. Report on written observations
2. Portfolio of literacy and learning
3. Tutoring practice
MODERATION
The assessment will be pre-event and post-event, moderated by the teaching team
with the assistance of an external moderator from the community.
CERTIFICATION
A UNITEC model B certificate will be awarded for successful comple tion of this
course
SELECTION
Because of the need for community support and the fact that suitable candidates
may not readily present themselves, UNITEC staff requires the assistance of community
and statutory groups for the recruitment of suitable trainee tutors.
Selection from referrals are based on:
• an interview
• reading a given text and a response to it
• writing on a set topic
EVALUATION
The programme is formally evaluated by the participants at the end of the course, to
identify strengths and areas for improvement. The School of English and Applied
Linguistics, through the Language Teacher Education programme committee, is responsible
for the course, and reviews and acts on the evaluation findings. Informal evaluation during
the course also ensures the course meets the needs of the participants.
169
Appendix 2
Literacy Research Project: Personal Information Profile for Literacy
Tutors
Circle the answers where there are choices. Write an answer where there is a space
________________________________________________________________________
1
Circle how old you are
2
What is your country of origin? _________________________________________
3
What gender are you?
4
Circle how old you were when you arrived in New Zealand?
Born in NZ
5
Less than 20
Male
Less than 20 years
21-30
31-40
41-50
Over 50
Female
21 –30 years
31-40
41-50 Over 50 years
`Circle how many years you have lived in New Zealand
Less than 1 year
more than 8 years
1- 2 years
3 – 5 years
6 – 8 years
LANGUAGES
6
What language(s) do you prefer to use at home?
Mainly English
7
Both English and my language
Does your first language use the same letters as English?
Mainly my language
Yes
No
170
8
Below, write the name of all languages you know and choose a number to show
how well you can use the skills below in each language.
5.= Very well; 4.= Well;
Language
a.
b.
c.
d.
3. = Average;
Speaking
……………………
……………………
……………………
……………………
2. = So-so; 1.= A little; 0. = Not at all
Understanding
__
__
__
__
__
__
__
__
Reading
Writing
__
__
__
__
__
__
__
__
EDUCATION
9
What is your educational level?
Primary School
Institute of Technolo gy
Other
10
Secondary School
University
What are your qualifications and in what field?
Informal qualification in _____________________Certificate in_________________
Diploma in ________________________________Degree in __________________
11 What teaching experience do you have? (In any country)
What?
When / How long?
Where?
171
12 Level of support for teachers:
a What aspects of tutor support have been useful to you?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
b What further support do you need?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
c What aspects of teaching this course have you enjoyed?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
d
What problems have you encountered teaching this course?
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
172
Appendix 3
Literacy Research Project: Personal Information Profile for Students
Literacy Research Project
Personal Information Profile
Code # __________
Circle the answers where there are choices
For example Yes
Write an answer where there is a space or where you circle ‘other’
No
No
1. What is your first name? _________________________________________
2. What is your last name? _________________________________________
3. What name do you like to be called? _________________________________________
4. Circle how old you are
Less than 20
21-30
31-40
41-50
over 50
5. What is your country of origin? _________________________________________
6. What is your current address? _________________________________________
7. What is your phone number? _________________________________________
8. What gender are you?
9.
Male
Female
Circle how old you were when you arrived in New Zealand
Less than 20 years
years
21 –30
31-40
41-50
Over 50
10. Circle how many years you have lived in New Zealand
Less than 1 year 1- 2 years
3 – 5 years 6 – 8 years
more than 8 years
_______________________________________________________________
LANGUAGES
11. What language(s) do you prefer to use at home?
Mainly English
Both English and my language
Mainly my language
12. What is your first language (mother tongue)?
________________________________________
13. Does your first language use the same letters as English?
Yes
No
173
14. Below, write the name of all languages you know and choose a number to show
how well you can use the skills below in each language.
5. = Very well; 4.= Well; 3.= Average; 2.=
a.
b.
c.
d.
Language
Speaking
…………………………
…………………………
…………………………
…………………………
__
__
__
__
1
0 = Not at all
Understanding
Reading
__
__
__
__
Writing
__
__
__
__
__
__
__
__
e. Choose a number to show how well you can use the skills of reading and writing in
your first language
5 = Very well
4=
3=
Language
2=
1=
0 = Not at all
Reading
__________________
Writing
_________________
_________________
EDUCATION
15. Did you attend school in your country?
Yes
No
If yes, circle how many years you attended school in your country
Less than 1 year
1 – 2 years
3 – 5 years
6 – 8 years
If no, why did you not attend school?
________________________________________________________________________
________________________________________________________________________
16.
Did you study English before you came to New Zealand?
Yes
No
If yes circle how long you learnt English in your country
Less than 1 year 1 - 2 years
3 - 5 years
6 - 8 years
more than 8 years
174
PAID WORK
17. Did you do any paid work before you came to New Zealand?
Yes
No
If yes, what paid work did you do before you came to New Zealand?
_________________________________________________________________________
________________________________________
18. Have you had paid work in New Zealand?
(If no, go to question 29)
19. If yes, circle whether it was full-time or part-time.
Yes
No
Full time / Part time
If yes, circle how long you have had paid work in New Zealand
Less than 1 year
1 – 2 years
3 – 4 years
6 – 8 years
20. If yes, what kind of work have you done in New Zealand?
________________________________________________________________________
________________________________________________________________________
ENGLISH IN NZ
21. Have you been to other English classes in New Zealand?
If yes, who did you learn with?
How long?
(in weeks, terms, semesters or years)
Home tutor
Home tutor groups
Community education classes
AUT
Polytechnic
Other groups
Yes
No
How many hours each week?
……………
…………..
…………..
………….
…………..
…………..
……………
…………….
……………
…………….
…………….
…………….
22.
Have you ever been taught by a bilingual tutor in NZ?
23.
Would you prefer your teacher to speak your language?
24.
How do you feel about speaking your language in front of New Zealande rs?
comfortable
confident
uncomfortable
not confident
Yes
No
Yes No
other ____________
other ____________
175
25.Circle why you want to learn English:
To get a job
To fill out forms
To talk and understand New Zealanders
To talk and understand neighbours
To understand the T.V
To study further
To get New Zealand Citizenship
Other
.
176
NB:
NB:This
Thisscale
scalewill
willneed
needmodification
modificationininlight
lightofofresults
resultsononself-assessments
self-assessments
Appendix 4
Self-Assessment Schedule
177
Appendix 5
Literacy Curriculum
CURRICULUM FOR LITERACY PROJECT 2001
§ Write in the date as you complete these
§ Remember to teach these in a context
TOPIC
Teach Review
Personal information:
• Answering questions
• Reading the words
• Filling it in
Alphabet:
• Spelling own name S/W
• Spelling address
• Writing other names L/W
• Matching capital/small
• Ordering capital/small
• Circling letters
• Guessing how to say a word
(using the sounds of letters)
Date:
• Days of week + abbrev
• Months of year +abbrev
• Use a month calendar
• Use a year calendar
• Match no. & date with words
• Know dates of birth/arrival
• Know when? How long?
Appointments:
• Understand when
• Day/time
• Listen and write down
Numerals:
• Phone numbers:
-recognise written
-repeat what hear
-write what hear
-say/write own
• Prices:
-recognise, find, say
• Bills:
-power, phone, water, Dr
-find how much
-find when you pay
• Medicines:
-read how much
-read how often
Review
Review
178
•
Time:
-repeat and show (on clock)
-match hands and nos.
-digital
-listen and write
Public signs:
• Understand
• Read
Bus timetables:
• Find Mon-Fri / Sat/Sun
• Recognise which timetable
• Find what time you arrive after
leaving at …..time (mid travel and
end travel)
• If you want to arrive at …. What
time will you leave?
Alphabetical order:
• Using a dictionary, phone bk
-ordering names of sts – in line then
using words
-ordering words (Diff.letters
-ordering words (same 1st )
-ordering words (same 1st ,2nd)
-opening bk at a letter
-look at guide words – will .. be
before /after or on this page
Reading a story:
• Use language experience with own
experiences – read then write
• Read others' stories
• Answer multi-choice questions
Write a letter of explanation:
• Note to a teacher
-why child is absent
-excuse a child from sport
-write your child can/ cannot
-go on a school camp
Write about your family
§ Write a journal – what you did at
some time in the past
Grammar:
§ Simple present
§ Simple past
§ Prepositions
§ Pronouns
§ Wh’ questions –yes/no
§ ‘To be’ (I am…)
§ ‘Have got’ (with family)
§ ‘There is/are’
179
Appendix 6
Test Specifications
Purpose
§ To test the literacy gains made by low- level literacy learners who speak English as
an additional language, and who participate in classes over 20 weeks
§ To develop generalised measures of literacy gain that can be used with similar lowlevel literacy learners across similar programmes.
Constructs
§ Understanding written language in survival English contexts
§ Producing written language in survival English contexts
Focus
On language useful in day-to-day life in New Zealand,
§ understanding authentic reading texts
§ producing contextualised written text.
Test Objectives
§ To test the ability of students to understand written language in familiar text types
§ To test students’ ability to write English in everyday contexts
Test-takers
Adult learners of English who may have:
§ little or no literacy in their own language
§ little or no schooling in their own language
§ attended previous English classes in New Zealand
Level
§ low- level literacy in English
§ generally low oral skills in English
Topics
Personal and familiar topics studied in class which are appropriate to context:
§ personal information, dates and numerals, public signs, bus timetables, alphabetical
order, basic punctuation, understanding a simple story, writing sentences about
familiar things and taking a telephone message.
Register
Formal and semi- formal
Input
Written and authentic, where appropriate
180
Content
Reading:
Identification of:
- letters of English
- personal information words
- numerals
- public signs
- alphabetical order
Comprehensio n of:
- a bus timetable
- a simple story
Writing:
- alphabetical letters of English
- personal information responses
- basic punctuation (capital letters and full stops)
- dates
- basic telephone messages
- short letters of explanation
- about familiar things, places, events
- a diary entry
Text Types
Authentic where appropriate
Test Methodology
§ Test instructions are on a separate page in the tutors' information booklet.
§ All tutors attend a training session and go through instructions for each test
§ Tests are conducted by tutors who also teach the classes
§ Where appropriate, bilingual tutors give the instructions in their own language
§ The instructions include the varying number of times that questions are read aloud by
tutors, depending on whether the tutor is bilingual or native English-speaking. This is
clearly indicated in the instructions
§ Time for students to complete is signalled for each test
Marking
§ Answer booklets are provided
§ Writing tests are marked by trained ESOL teachers
§ The same markers mark both the initial and the final tests
§ All test scripts are moderated
181
Appendix 7
English Literacy Test Descriptors and Score Sheet*
A scale of 0 – 5 is used at each skill level, both on the self-assessment procedures
and to score the performance tests in reading and writing. 0 signifies no proficiency at each
level while 5 means the student is able to perform the required task ‘very well’. The tests
have eight skill levels, with specific tasks at each level at which students are required to
state and demonstrate their ability.
Student name:______________________
Class:____________
Reading
Score
Not
at all
Level
0
1
2
3
4
5
6
7
8
0
Very
well
1
2
3
4
5
No understanding of English writing. Unable to
demonstrate reading ability at even the most basic level.
Familiarity with formulaic words related to personal
information. Shows ability to recognise key words on a
form requiring personal information already known and
easily identifiable.
Initial reading ability, familiarity with letters, sounds and
shapes of letters of the alphabet placed in any combination
of letters to spell out names.
Initial numeracy ability. Familiarity with known dates and
written forms of the months of the year. Is able to identify
appropriate abbrevia ted forms of the months of the year
and circle familiar dates known on a calendar.
Minimum reading ability. Understands numbers placed in
any combination. Able to identify numerals used for
familiar purposes i.e. phone numbers, prices of goods and
services, numerical information on prescriptions, bills.
Emerging reading ability. Able to recognise and read
public signs for different purposes and essential services
i.e. doctor, supermarket, warnings.
Survival reading ability. Is able to read numerals and
words placed in an information table. Able to identify and
access meaning of numerals and simple words used for
familiar purposes i.e. destinations and times in bus and
train timetables, class timetable.
Basic reading ability. Is able to identify and locate words
and letters arranged in alphabetical order. Is able to use
numerals to demonstrate order of letters as in a dictionary,
an index of street names, a phone book.
Social reading ability. Is able to read a short simple story
within the first 500 word level and with familiar place
names, within the context of own experience. Is able to
read, understand and interpret numerals embedded within
the text. Understanding of text is at factual recall level
only i.e. ans wers require recall of clearly stated
information in the text and are either right or wrong.
* These descriptors will need fine-tuning in the light of the research results of the tests and self-assessments.
182
Writing
Score
Not
at all
Level
0
1
2
3
4
5
6
7
8
0
Very
well
1
2
3
4
5
Pre-literate in English. Unable to demonstrate writing
ability at even the most basic level.
Alphabet recognition. Ability to recognise letters of the
alphabet and the difference between upper case (capitals)
and lower case letters.
Initial writing ability. Ability to accurately fill out a
familiar personal information form requiring essential
information about own background and life.
Observation of writing conventions. Is able to appropriately
use basic writing conventions (capital letters and full stops)
in a meaningful, familiar context and within the appropriate
vocabulary range.
Minimum writing ability. Initial numeracy ability. Ability to
identify and write dates, days of the week, months of the
year. Ability to translate numerical order of months of the
year to the appropriate word form and vice versa. Ability to
identify and write key aspects of appointment times where
numerical understanding is required (day, date, time).
Emerging writing ability. Ability to write a short
meaningful letter of explanation on a personal matter to a
realistic audience/recipient. Content clearly reflects
reasons/excuse. Grammar and punctuation do not affect
meaning. Appropriate use of vocabulary. Le tter reflects
ability to write at some length.
Survival writing ability. Ability to accurately take down a
short cryptic message in an unseen situation e.g. a telephone
message when the context is not embedded in the discourse
and non-verbal communication is not a part of the situation.
Interlocutor, topic and accent may be unfamiliar but all
aspects of the message are included accurately.
Basic writing ability. Ability to write more extended prose
freely on a familiar topic which falls within personal and
social experience. Accuracy reflected by appropriate use of
punctuation and grammar. Fluency reflected by appropriate
length of text. Appropriate use of vocabulary.
Social writing ability. Demonstrates ability to write
extended and sustained prose on a regular basis for a longer
length of time (10 minutes), with fluency (length), interest
(content) and accuracy (punctuation and grammar).
Appropriate and accurate use of vocabulary.
183
READING TEST SYLLABUS
1. Reading
personal
information
words on a form
Tutor asks students to
identify 5 words found on
the form by asking where
they would write their
family name etc.
1. Students circle names of
5 letters
2. Knowing the
names and
sounds of the
letters
3. Reading the
date
No. of
questions
5
Time
Type of question
Possible score
Bilingual tutors say
questions twice;
English tutors say
questions 5 times
Write a number in a
circle
5
5
Tutors repeat twice
Circle letter
5
2. Students circle letters as
they hear 5 sounds
5
Tutors repeat twice
Circle letter
5
3. Students individually
spell out one name
1
Response in 5 seconds
and assess …/5
0=not at all
5= easily
1 minute
Spell name
5
Say name
5
Total_ /20> _/5
4
4. Students individually
pronounce one name
A. Match the abbreviations
with the word
1
B. Circle the date on a
calendar from an oral
and written sentence
4
Tutors read out
sentences 2 times,
then give 30secs to
circle each.
Circle the date from
4
oral/written information
2
1 minute
Circle the date from a
numerical date
C. Circle the date on a
calendar from dates
written in numerals
4
Match
Test Syllabus
Parts to question
Appendix 8
Section
2
Total _/10>_/5
184
Section
Parts to question
4. Reading
numerals
A. Identify phone numbers
and prices
B. Reading a bill
No. of
questions
7
Time
Type of question
Possible score
Bilingual tutors say
questions 2 times;
English tutors say
questions 5 times
Write a number in a
circle
7
2
Tutors read the
questions and give 2
minutes
Write the answers
2
Write the answers
C. Reading a medicine label
2
5. Reading
public signs
Tutor asks students to
identify 5 public signs by
reading a sentence
describing each one.
5
6. Reading a
bus timetable
1 & 2. Identify the correct
timetable
5
3 & 4. Identify time of
arrival
5. Identify time to catch
bus when arrival time
is designated
Tutors read the
questions and give 2
minutes
Bilingual tutors say
questions 2 times;
English tutors say
questions 4 times.
Leave one minute
between questions
Bilingual tutors say
questions twice;
native English tutors
say questions 4 times.
Leave one minute
after 1 & 2 and 2
minutes after 3-5
2
Write a number in the
box
1 & 2 Identify the
correct timetable by
writing the number
Total_10>_/5
5
5
3 - 5 Write time of
arrival
185
Section
Parts to question
7. Alphabetical
order
1. Number names
beginning with different
letters in alphabetical
order
2. Number names
beginning with the same
letters in alphabetical
order
3. Number words
beginning with the same
letter in alphabetical
order
4. Say whether words in a
dictionary would be on /
before/after a page
8. Reading a
story
Read a short story and
answer 5 questions
No. of
questions
5
Time
Type of question
Possible score
1-3
(6 minutes)
1-3 Number the words
in order
5
5
5
5
5
5
5
2 minutes after
instructions
10 minutes (tutors do
not read out any of the
story or questions, but
may introduce any
new words)
Write A if word is on
this page
Write B if word is
before this page
Write C if word is after
this page
Multi- choice: circle the
correct answer out of a
possible three
5
Total_/20>_/5
5
TOTAL ……/40
186
WRITING TEST SYLLABUS
Section
Parts to question
10. Writing the
letters of the
alphabet
A. Write a dictated sentence
(aim is to write small letters
accurately)
No. of
questions
1
Time
Type of question
Possible score
Tutors dictate
sentence once,
then say each
word once and
spell it twice
A. Write a sentence that is
dictated and in which
each word spelt
5
(0=no letters
correct
5= all correct)
B. Write familiar words
(aim is to write capital
letters accurately)
1
11. Writing
personal
information on
a form
12. Using
capital letters
and full stops
Write answers to as many
prompts on the form as
possible (including -- where
the answer is "No").
Mark capital letters and full
stops in a story
1
10 minutes
Complete personal
information
1
Tutors read the
story once then
give students 5
minutes
Mark capital letters and
full stops in a story
20 >
13. Writing the
date
1. Fill in the missing parts
of the dates (words /
numbers)
2. Listen to day, date and
time and fill in an
appointment card
4
4 minutes
1. Complete words /
numbers in the dates
4
6
Tutors read out
appointment
information three
times
2. Fill in the information
on an appointment
card after listening
6
B. Rewrite the above
sentence or write
personal information
in capital letters
5
(0= no letters
correct
5= all correct)
Total_/10 >_/5
15 >
_/5
_/5
Total _/10 > _/5
187
Section
Parts to question
No. of
questions
1
14. Writing a
short letter of
explanation
Write a short letter of
explanation including
address, date, greeting,
body, farewell.
15. Writing a
telephone
message
Listen to phone message
1
and write appropriate details
16. Writing
about a familiar
thing
Write about a given topic
familiar to oneself
1
17. Writing a
journal
Write a diary entry
1
Time
Type of question
Possible score
5 minutes
Write a short letter of
explanation
Play tape through
once (no writing)
then three more
times
Write:
• who the message is to
• who the message is
from
(selecting one name from
written options)
• the phone number
Content _/5
Grammar_/5
Punctuation _/5
Spelling _/5
Length _/5
Total /25 > /5
2 for each name
1 for phone number
_/5
Tutors and
Write about a given
Content _/5
students discuss
familiar topic
Grammar _/5
the given topic for
Punctuation_/5
2 minutes then
Spelling* _/5
students write for
Length _/5
10 minutes
Total_/25 >_/5
Elicit possible
Write for 5 minutes about
Content _/5
answers then give an experience or event in
Grammar _/5
5 minutes to write the past
Punctuation_/5
Spelling* _/5
Length _/5
Total_/25 >_/5
TOTAL …/40
* Spelling to be changed to vocabulary
188
Appendix 9
Impressions from an English-Speaking Tutor
The concept of testing produced anxiety in my students - they were concerned that
they wouldn’t perform well enough, perhaps they were worried that they wouldn’t be able
to continue in the class or that they would disappoint me. I repeatedly explained to them
why they were being tested and that it didn’t matter if they couldn’t do or complete a task.
I also explained to them that for research purposes we wanted individual work. During
testing, I separated the desks, continually monitored and reinforced that they must not help
each other, but it was an impossible task. It was not just that the learners who were having
difficulty with the task were asking more competent learners for information; the more
competent learners were offering information to others. However, there did seem to be
less collaboration in the second test than the first.
Learners without previous formal education are especially discouraged if they
can’t achieve a task. In class they always work collaboratively- the more competent
learners explaining and helping the others. At the beginning of the project I always
explained every worksheet using the white board or OHP and reinforced this individually.
By the end some of the class were able to cope with just oral instructions in some
instances. The learners without formal education tended to have the most difficulty with
abstract concepts and transferring skills from one task to another, which made coping with
a different task format in the testing difficult. This seemed to be more so with age. e.g.
alphabetical order. Some tasks that they had reviewed frequently and in various ways in
class, they still had difficulty with in the tests. For some learners retention of information
is very difficult, as well as the speed that they can listen and then process oral information
into written form.
189
Appendix 10
Recording Sheet
190
Appendix 11
Case Study Instruments
Schedule of questions for the student interviews
1. What are your reasons for studying English?
2. Why did you choose this particular course?
3. How confident are you about learning English?
4. Do you feel you are making progress? Why and how?
5. How satisfied are you with the course? What do you like most/least about the course?
What would you like changed?
6. What are the easiest and most difficult ways for you to study English?
7. Is it easy for you to attend this course? Why/why not?
8. Do you enjoy the social contact with other students? Why/why not?
9. How is learning English helping you live here? Do you feel well and happy living in
New Zealand? Why/why not?
10. What do you hope to have achieved on completion of the course?
11. How did you feel at the beginning of the course compared with now? Why do you feel
differently?
12. Are you satisfied with your progress?
Schedule of questions for the tutor interviews
1. What is your definition of literacy?
2. How would you describe your philosophy of English language teaching?
3. What works best in terms of the delivery of this course?
4. How do you cater for the needs of this particular group of students?
5. What are the characteristics of the learners you consider the most likely to succeed in
achieving English literacy?
6. Can you identify the outcomes of this course?
7. Is the assessment being piloted in this course useful to you in terms of curriculum
development?
8. What other forms of assessment are you using?
191
The framework for the classroom observations*
Time, frequency and size of class
Native speaker or bilingual tutor
Description of classroom activities
Participant organization
Lesson content
Student modality
Classroom materials
Use of the target language
Reference to students’ prior experience
Balance of English skill development
* Adapted from COLT Model (Allen, Fröhlich & Spada, 1984; Spada and Fröhlich,
1995)
192
Appendix 12
Information Sheets
Literacy Research Project
Participant Information Sheet
The Research Project: Measurement of Literacy Gains
The Purpose of the Research
The ESOL Home Tutor Service has agreed to run these classes. They are working with people from
UNITEC and AUT to do a research project. The aim of this project is to measure how much you learn to
read and write in English. To measure your progress, we need to test you at the beginning of the term to see
what you can do already and then compare these results with the results of another test at the end of the
term. It is important that you do both tests so we, and you, can see how much you have learned. Some
classes have lessons for 12 hours a week and some have 2 hours a week.
In the first test there may be many things you have not learned to do yet. Do not worry as we expect this.
The test at the end of the year will be similar but by then you will be able to do a lot more. You will get the
results at the end of the year and can see how much you have learned.
The self-assessment will give us information that will be compared to your test results. We will find out if
self-assessment is a useful way of knowing how much people have learned.
The profile will give information about some of the things that make it difficult for you to learn, like
worrying about family and jobs or finding transport to the class. You will have the assistance of a person
who speaks your language to help you with the profile. All the information you give is completely
confidential.
If the project is successful it will make it easier to get funding to continue classes next year. It also allows
the government to see the importance of making sure there are more places next year for people who need to
learn to read and write English.
The Participants
This class is one of eight classes paid for by the Ministry of Education for one year as a special project. The
classes are in West Auckland and South Auckland
You will be asked to complete these forms
•
•
•
•
•
A consent form: When you sign the consent form you agree to attend the classes for the rest of the
year and also to sit the tests.
A personal profile giving information about yourself and your experience of learning in your own
country and also in New Zealand
A self–assessment of your English literacy ability
In class assessment tasks designed by your teachers, at the beginning, and end of the course
Some of you will be asked to take part in the case study that involves a one-to one interview and a
questionnaire.
Most activities will be done in class time. Some extra time will be needed for the case study involvement.
You can ask your teachers or any or the researchers for more information about the project.
193
Confidentiality
The information received will be available only to the researchers. No one else will see the forms or use the
information for any other purpose. There will be nothing in the final report that could identify you
individually. All the information gathered (test forms, personal profile, and self-assessments) will be kept in
a locked cabinet at the School of English and Applied Linguistics, UNITEC, Building 131. The information
will be kept for five years and then it will be destroyed
Researchers
The research is being contracted by: Judi Altinkaya, (ESOL Home Tutor Service).
She is responsible to the government for this project.
Jeannie Martin Blaker (contact 8463098, jmartin-blaker@unitec.ac.nz) of West Auckland ESOL Home
Tutors and UNITEC; Julia Castles, (contact 2789099, sthakESOL@xtra.co.nz) of South Auckland ESOL
Home Tutors;
They are organising the tutors, writing the tests to measure literacy gain, the personal profiles and the selfassessment form and overseeing the administration of the tests.
Keryn McDermott, (contact 3079999 ext 6666 kerynm@ihug.co.nz) of AUT Centre for Refugee
Education;
She is going to interview some people (who volunteer to be interviewed), in order to write case studies
Lois Bellingham (contact 8154321 Ext 8711) Head of the School of English and Applied Linguistics has
overall responsibility for UNITEC’s participation in the project
Jenny Carryer (contact 8154321 ext 8171, jcarryer@ihug.co.nz) of The School of English and Applied
Linguistics, UNITEC.
She is overseeing the management of the project, writing the literature review, gaining ethics approval from
the UNITEC Ethics Approval Committee and assisting with data gathering.
Dr Nikhat Shameem, School of English and Applied Linguistics,(contact 8154321 ext 8173), of UNITEC
is supervising this project. She is also responsible for collating and writing the final report. The research is
funded through a grant from the Ministry of Education to the National Association of ESOL Home Tutors.
This study has been approved by the UNITEC Research Ethics committee from August 15, 2001 to
March 3, 2002. UREC Ref: 013-2001. If you have any complaints or reservations about the ethical
conduct of this research, you may contact the Committee through the Secretary (ph: 09 815 4321 Ext
8571). Any issues you raise will be treated in confidence and investigated fully, and you will be
informed of the outcome.
194
Teacher Information Sheet
What the Literacy Project hopes to achieve
To develop measures of literacy gain that can be used with low level English language and literacy
learners
•
•
That are effective, valid and reliable
That are informed by research
The measures proposed are teacher-designed competency tasks (8 for reading and 8 for writing) and self assessment based on descriptions of these tasks. The validity of self- assessment will be measured by
examining the relationship between these two measures of assessment. These measures would be made
available to other literacy classes.
To gather information regarding the effect of a number of specified factors on the literacy gains made
by participants
Information about these variables will be gathered through the personal profile and the case studies. It
includes things such as literacy in first language and level of English language and literacy, length and type
of education, age, time in NZ, socio-economic factors, effects of bilingual teaching, goals of learners, and
attitudes to NESB speakers in NZ.
Literacy gains for the participants
The funding provided by the Ministry of Education through a grant to The National Association of ESOL
Home Tutors for the research project has provided money to run the classes, enabling approximately 100
learners to have the opportunity to improve their English language skills.
Publication of research findings
The learner benefits ultimately from the collective pool of knowledge on how best to deliver and assess
literacy education
Researchers
The research is being done by Jeannie Martin-Blaker (contact 8463098, jmartin -blaker@unitec.ac.nz) of
West Auckland ESOL Home Tutors and UNITEC; Julia Castles, (contact 2789099, sthakESOL@xtra.co.nz)
of South Auckland ESOL Home Tutors; Keryn McDermott, (contact 3079999 ext 6666
kerynm@ihug.co.nz) of AUT Centre for Refugee Education; and Jenny Carryer (contact 8154321 ext 8171,
jcarryer@ihug.co.nz) of UNITEC. The research is being supervised by Dr Nikhat Shameem, (contact
8154321 ext 8173), of UNITEC. The research is funded through a grant from the Ministry of Education to
the National Association of ESOL Home Tutors.
The Stages of the Project
Stage 1 March to June 2001
Literature review to establish and justify the content of the personal profile and measures of assessment
Development of the assessment tools and trials in a number of language classes
Ethics approval from the UNITEC Ethics committee
Stage 2 July to December 2001
Data collection through assessment instruments
Case studies
Literature review
Stage 3 November – December 2001
Data analysis
195
Stage 4 January 2002
Report Writing
This study has been approved by the UNITEC Research Ethics committee from August 15, 2001 to
March 3, 2002. UREC Ref: 013-2001. If you have any complaints or reservations about the ethical
conduct of this research, you may contact the Committee through the Secretary (ph: 09 815 4321 Ext
8571). Any issues you raise will be treated in confidence and investigated fully, and you will be
informed of the outcome.
196
Appendix 13
Consent Form
Literacy Research Project: Consent form
This project looks at how we assess literacy gains of people with low levels of English
language and literacy. It also aims to collect information on factors that influence literacy
learning. The research is a joint programme between UNITEC, AUT and the National
Association of ESOL Home Tutors.
The research coordinator is Jenny Carryer and she can be contacted at UNITEC Building
136, Phone 8494180 ext 8172 email jcarryer@unitec.co.nz
Please read, fill in and sign the consent form
•
I have read and understood the information sheet about this research project
•
I have had an opportunity to ask questions about this project, and have them
answered.
•
I have had enough time to think about whether I want to take part in this project.
•
I undertake to take part in the classroom tests, self-assessment and supply a
personal profile
•
I understand that the information I have given will be used in this project and for
no other purposes.
•
I understand that no information I give will, in any way, affect my participation or
assessments of work done in the course
•
I understand that the information I have given is confidential and that nothing that
is written in the final report will identify me.
•
I understand that I can withdraw from the project without this affecting my access
to any services provided by any of the research bodies (UNITEC, AUT and
National Association of ESOL Home Tutors). If I choose to withdraw, all
information that I have given will be destroyed.
•
I understand that copies of the final report will be available at West Auckland and
South Auckland ESOL Home Tutor offices, The Centre for Refugee Education,
197
Mangere (AUT) and The School of English and Applied Linguistics (UNITEC). A
copy will be lodged in the UNITEC library.
Name of participant: ________________________________________________
Signature: ___________________________ Date: ________
The project was explained to me by ___________________________________
Signature: __________________________ Date: _________
This study has been approved by the UNITEC Research Ethics committee from August 15, 2001 to
March 3, 2002. UREC Ref: 013-2001. If you have any complaints or reservations about the ethical
conduct of this research, you may contact the Committee through the Secretary (ph: 09 815 4321 Ext
8571). Any issues you raise will be treated in confidence and investigated fully, and you will be
informed of the outcome.
198
Appendix 14
Literacy Tests: Accompanying Document
Includes:
1. Initial and Final Tests for Reading and Writing
2. Tutor Information
199