Annual Review
Project Title
: English in Action Bangladesh
Date started
Review date
: May 2008
: May 2013
Introduction and Context
What support is the UK providing?
English in Action is a £50 million programme being implemented over nine years (2008-2017)
What are the expected results?
English in Action (EIA) is designed to change the way that children, young people and adults in
Bangladesh learn English. EIA was initiated at the request of the Government of Bangladesh
and is funded by a grant from the UK Government.
The programme’s expected outcome is ‘to increase significantly the number of people able to
communicate in English to levels that enable them to participate fully in economic and social
activities and opportunities.’ This aims to is to contribute to ‘economic growth by providing
English language as a tool for better access to the world economy’.
The project aims to reach 25 million primary and secondary school children and adult learners
through innovative ways of learning spoken English using a combination of traditional and new
methods. The projects results include - Increase the numbers of schools, teachers and pupils participating in English in Action
primary interventions to 30,000 schools, 67,500 primary teachers and 6.75 million pupils
(3.29 million M; 3.46 million F) and secondary interventions 9000 teachers (69%M;
31%F) and 3.75 million pupils (48%M; 52%F).
- Employer satisfaction (with applicants’ / employees’ communicative English) to increase
from 68% of employers in the baseline to 80% by 2016/17.
- Increase motivation for learning and using English by students from 51% to 65%, and by
teachers from 64% of to 80%).
- Increase use of English Language news and media: online; broadcast or print based from
5% to 35% increase
- National education systems and structures take on the programme initiatives (training
methodology, revised curriculum, research, etc.)
What is the context in which UK support is provided?
English language teaching capacity has atrophied in Bangladesh due to the lack of trained
English teachers in a centralised education system that expanded rapidly in the 1970s. Few
people under fifty are confident, fluent English speakers, and English is the most commonly
failed examination subject at the secondary certificate level. Lack of English language skills is
regarded as a constraint to economic development. Job applicants and employees often lack
English language skills necessary to work effectively. A pre-appraisal mission recommended
that the problem be addressed at scale, through a project that would have substantial reach
and impact.
1
Section A: Detailed Output Scoring
Output 1: Primary teachers’ classroom practice and both teachers and students’ English language
proficiency improved; and
Output 3: Secondary Teachers’ classroom practice and both teachers and students’ English language
proficiency improved.
As the Outputs and the indicators are exactly the same for the two Outputs, except that one is
for primary and the other secondary schools, it was felt that it would be more meaningful to
address them together. The aim is two-fold:
(i) to improve pupils’ competency in English ; and to achieve this,
(ii) to improve teachers’ English language capabilities.
Both of these are addressed through raising the quality of language teaching practices.
Teachers’ own improved English language competence is both an enabling factor and an
output. As an enabling factor, it makes the results sustainable. The immediate beneficiaries
are the teachers and the students in the primary and the secondary education programmes of
Bangladesh.
Output 1 & 3 score and performance description:
Overall Score for Output 1: A +
Overall Score for Output 3: A +
Summary Progress and Performance on Output 1 & 3 (2012 – 2013)
Both output 1 and 3 have moderately exceeded expectation. Since the 2011 Annual Review,
EIA has prepared to up-scale the primary and secondary school initiatives by approximately
twenty-fold from Phase II (2008-2011) to Phase III (2011-2014). The present aim is to reach
12,500 teachers by 2014. Towards this target both the preparatory work in terms of quantity
and quality exceeds expectation. The following actions have been put in place to meet the
2014 target:


An updated model involving four thousand teachers in May & June 2012; and
New teacher educators in 35 Upazilas across the country, including core trainers in Dhaka
have been recruited and trained.
This major up-scaling plan has led to important qualitative refinements to all aspects of the
school interventions, including - more decentralized teacher support, more structured learning
materials & modules, and more effective use of mobile technology.
Key refinements include:





strengthening 'expert' subject and teaching knowledge in the teacher training materials;
using video to show authentic classroom practices of actual Bangladeshi teachers , and
including 'teachers voices' in discussing teaching and learning;
more emphasis on peer learning for teachers,;
strengthening the role of Head Teachers, with more explicit and guided involvement in
informal monitoring and support, through new Head Teacher materials and meetings, and;
better quality assuring (QA) procedures, involving local government staff, thereby
implementing a model through existing government structure to ensure teachers receive
appropriate support.
2
Progress against expected results
The expected results in annual indicators for 2013 are not given in the logframe. The
milestones of performance used in the logframe for 2014 are:
Performance indicators
Milestones for 2014
1. Reach: The cumulative
numbers of schools, teachers,
and pupils participating in
English in Action primary
interventions
Primary: 2,500 schools; 7,500
teachers (50%M, 50%F); 750,00
pupils (49%M, 51%F)
Yes
Secondary: 2,500 schools;
5,000 teachers (70%M; 30%F);
2.08 mn pupils (48%M; 52%F)
Yes
2. Practice: The number of
teachers evidencing
communicative language
teaching approaches in their
classroom practice
20% student talk of new EIA
cohort in a lesson; 60% of that
talk in English
Yes
Primary: a) 7.5% more teachers
at level 2 or above and 7.5%
more teachers are at level 3 and
above. b) 5% more students are
at level 1 or above and 5% more
students are at level 2 and
above.
Secondary: a) 7.5% more
teachers are at level 3 and above
and 5% more teachers are at
level 4 and above. b) 10% more
students are at level 2 and above
and 5% more students are at
level 3 and above.
Yes
3. English Language
competence: The proportion of
EIA supported teachers and
students reaching a particular
English Language
competence grade on Trinity
Scale1.
On track
Yes
Progress against indicators and milestones:
Indicator 1: Reach. 2014 Milestone: Primary: 2,500 schools; 7,500 teachers (50%M,
50%F); 750,00 pupils (49%M, 51%F) Secondary: 2,500 schools; 5,000 teachers (70%M;
30%F); 2.08 mn pupils (48%M; 52%F)
Progresses against the milestones for 2014 are on track. Up-scaling is already under way and
has reached 975 primary and 927 secondary school and 4,906 teachers (2974 primary; 1932
secondary) exceeding the target of first batch. The next cohort of 8,000 teachers will be added
in June 2013. Adding these additional numbers will result in exceeding the 2014 target for
Output 1.& 3.
1
The Trinity Grade scale has been developed by Trinity College to assess students’ English Language ability. It is a 12-grade
scale. For a brief overview, the first four grades on the scale can be described as follows:
Trinity Grade 1 – requires knowledge of greetings and some basic personal information
Trinity Grade 2 – speakers can offer and respond to short, simple questions, requests and statements relating to personal
details and situations
Trinity Grade 3 – speakers can exchange basic everyday information, asking and answering questions using simple sentence
patterns
Trinity Grade 4 – speakers can talk about a prepared topic, asking and answering questions about it
3
Indicator 2: Practice. 2014 Milestone: students are talking in the classroom for 20% of the
lesson time; 60% of the talking time is in English.
Progress against this output is on track. The progress has been assessed based on the
feedback collected through continuous quality assurance process. Though full research data is
not yet available, the quality assurance system is designed to monitor indications of classroom
practice being affected by EIA training, with an ambitious 100% sampling target over the
course of the intervention. QA findings indicate the project is on track to achieve the 2014
target:
Feedback collected through questionnaires indicates that:



Teachers report using EIA activities or practices in lessons either often, or in most of
their lessons (97% of primary teachers; 90% of secondary teachers)
Teachers report using EIA audio-visual materials, either often, or in most of their
lessons (97% primary, 86% secondary teachers).
On average, 95% of teachers report that they consider using the EIA activities and
practices in their classroom as important or crucial, to their professional development,
with over two thirds of primary teachers, and half of secondary teachers, considering
these as ‘crucial’.
Classroom observations, carried out in 748 classrooms between October 2012 to January
2013, indicate that:





In almost all observed lessons, teachers speak in English more than half of the time
(90% of primary teachers, 92% of secondary teachers observed)
In most lessons observed, students speak in English more than half of the time (69% of
primary lessons, 64% of secondary lessons)
In almost all lessons observed, most students participated in EIA activities (98% of
primary lessons, 97% of secondary lessons)
In almost all lessons observed, teachers attempted to use pair and group work (95% of
primary lessons, 93% of secondary lessons),
In almost all lessons observed, teachers asked questions from a range of students
(97% primary lessons, 100% of secondary lessons).
The QA data shows a high level of adoption of EIA activities, techniques and materials, by
teachers in the current cohort, which suggests that EIA is likely to exceed 2014 targets for
changes in practice..
Indicator 3: EL Competence. 2014 Milestone: Primary: a) 7.5% more teachers at level 2 or
above and 7.5% more teachers are at level 3 and above. b) 5% more students are at level 1 or
above and 5% more students are at level 2 and above.
Secondary: a) 7.5% more teachers are at level 3 and above and 5% more teachers are at
level 4 and above. b) 10% more students are at level 2 and above and 5% more students are
at level 3 and above.
Research is used to determine changes in English Language (EL) competence of both
students and teachers. .
The assumption is firstly, that the students’ English speaking ability should improve through
the use of EIA activities, materials and methods. Secondly, as the teachers continue to work
and become more familiar with the materials, their own proficiency and skills would improve
over time. The purpose of the research is to assess these assumptions.
4
Whilst there is no formal data on English Language competence for the current batch of
teachers (impact evaluation studies are scheduled towards the end of 2013, when teachers
will be completing their participation in EIA), the changes in practice noted above suggest that
teachers and students, are both having significantly increased opportunity to listen, speak and
communicate in English language, through changes introduced in classroom practice. That
these changes in practice are in line with changes seen in the previous batches indicates that
similar gains in English language competence may also be expected in the independent
assessments in 2013. The project is on track or expected to exceed expectations to achieve
this indicator.
The independent reviewers made two observations. Firstly, that spoken English is only a part
of the language skill. Sustainability of spoken English is relatively easy in a predominantly
English speaking environment but is extremely difficult in a non-English environment unless
the secondary and higher education environment naturally transforms itself into a multi-lingual
environment as in many of the major cities in India. Secondly, the impact of a comprehensive
approach combining all four competences i.e. speaking, listening, writing and reading, would
have a better chance of sustainability than only spoken English.
In light of the above observations they acknowledged that the EIA package actually does
include reading and writing materials. However their recommendation is to strengthen the
reading and writing components in the EIA curriculum, the EIA technology, in the classroom
and individual learner assessments.
Recommendation 1
Balancing reading and writing with speaking and listening: Communicative English should be
defined to include all 4 competencies. Sustainability of the EL competence and ownership of
EIA by Government of Bangladesh (GoB), especially by Directorate of Primary Education
(DPE) would benefit from EIA strengthened reading and writing competences in classroom
practice and classroom environments through a wide selection of reading materials available
inside the classroom. EIA needs to assess for itself how best this could be achieved within the
existing scope of the project. The current scope does not include funding to print large
amounts of reading materials, however EIA can promote more reading materials inside the
classroom and provide TA to the National Curriculum and Textbook Board (NCTB) to provide
such materials
The reviewers recommended:
(a) Incorporating more reading and writing content into the EIA technology (for example 10
developmentally and culturally appropriate stories, creative writing exercises etc.).
(b) Making available a wide selection of books (storybooks and interesting non-fiction) and
reading materials inside the classroom, emphasizing that National Curriculum and Textbook
Board (NCTB) has little expertise in this area. The textbooks, produced by NCTB are still very
weak and would benefit enormously from technical support. This could help improve the
textbooks to world standard.
(c) Integrating reading and writing into the delivery of the
teacher education package of the Government, and
teaching and therefore, in the
(d) Ensuring research on classroom practice and EL competencies which clearly captures the
level of competence acquired both by the teacher and learner in primary and secondary
schools.
5
(e) Log-frame adjustments to capture both reading and writing at the level of output and
indicator. And strengthening the “talk-time” indicator, by including the range of ‘talk’ leaners
are able to tackle demonstrating their speaking competence improvement.
Recommendation 2
EIA intervention does not address children’s differentiated learning needs in classroom
practice, which is one of the key elements of acquisition of sustainable competences. To
address this effectively the independent reviewer recommended the EIA intervention package
to , address each learner in a way that she/he is able to proceed according to her/his ability
and pace; and secondly. They also recommended teachers prepare lessons according to
ability groups instead of assuming that all children should be able to follow the prescribed
syllabus and move forward at the same time.
However this needs to be carefully assessed along with the overall objective and scope of the
project and against the curriculum and textbook of the government, because EIA materials
are all developed based on the national curriculum.
Though it needs to be noted that EIA intervention has done much to alert teachers to the
learning needs of their students, for example through developing skills in assessment for
learning. This is the first step on the long road to a fully differentiated approach to teaching
techniques.
Impact Weighting (%): 25%
Revised since last Annual Review? N
Risk: Low
Revised since last Annual Review? N
Output 2: Improve adults’ attitude towards and proficiency in English through media.
Output 2 score and performance description: A+
Progress against output 2 has moderately exceeded expectation. The reviewers were
impressed by the quality and variety of materials produced for the adult learning and media
component. There is strong evidence to indicate that Bangladeshi audiences use and
appreciate these products, accessing them across television, print and mobile phone platforms
as well as (increasingly but to a lesser extent) on the internet and face to face in English
Clubs. A further impressive achievement is the partnerships with key media actors in
Bangladesh and the considerable local capacity they have built in creative technologies.
Over the past year and following the successful launch of the BBC Janala’s Amar Engreji (My
English) Course, a second course has been developed and is in the process of rolling out
across mobile, web and print. The course shares a common core syllabus across all platforms
with content that focuses on improving functional, communicative English skills. The course
content is culturally relevant for Bangladeshi learners, and reflects the interests & diversity of
the population. Learning is designed to be personalised, allowing those using the service to
proceed at their own pace. The course provides progressive, step-by-step learning, so that
learners can track and evaluate their progress. All learners can undertake regular interactive
quizzes to self-assess their learning; and can receive a Course Report on completion, to
acknowledge their achievement.
The second series of the project’s Mojay Mojay Shekha (Learning with Fun) TV series
6
completed broadcasting during the year. A new TV series – Nijay Nijay Shekha (Learning by
myself) – has been developed and produced. The series follows eight real-life, beginner-level
English learners, combining factual entertainment with elements of “reality TV” and allowing
viewers to learn alongside the on-screen characters.
Over 600 English Clubs in 31 districts have started activities so far, with an average of ten
learners in each club. These English clubs are operating in partnership with local nongovernment organisations, many of whom are working with the poorest and most remote
communities. The clubs use EIA adult learning materials from the Amar Engreji Course and
are designed for peer-led learning environments where teaching expertise is not readily
available.
Other highlights during the year include a pilot community radio programme, which has been
completed and is being evaluated in partnership with 14 community radio stations, and a
major research study which sought to measure the effect of EIA media products on learners’
English competence.
In particular, completing the second Amar Engreji Course has strengthened the extent and
duration of engagement with language learning materials and is providing a progression route
as well as a second entry point, above beginner level, for adult learners.
Progress against expected results
Performance indicators
Milestones for 2014
On track
1. Cumulative numbers of
Bangladeshis accessing BBC
produced EIA media products.
A cumulative total of 8.5 million
people had used BBC produced
EIA media products regularly by
the end of March 2014
(65%male, 35% female)
Yes
2. Perception of reduced barriers
and increased positivity
towards learning
communicative English among
school leavers and adults
43% of people who have used
BBC EIA products will report
increased motivation to learn and
/or increased confidence to use
English in their everyday lives
(47%, 35%)
Yes
3. The percentage of people
exposed to BBC produced
media products showing an
increase in average
competence as evidenced by
speaking and listening test.
49% of people who have used
BBC media products showing
increased English Language
ability
Yes
Progress against indicators and milestones:
Indicator 1: Cumulative numbers of Bangladeshis accessing BBC produced EIA media
products. 2014 milestone: A cumulative total of 8.5 million people had used BBC produced
EIA media products regularly by the end of March 2014 (65% male, 35% female).
7
The output is well on track with signs of exceeding the target of 2014. The total number of
users of each media product across television, mobile phone, internet and newspaper is
tracked and shows continued impressive growth. At the time of the review performance was
well on track, even without counting television viewers, There are an additional 1.6m
committed users of mobile phone, web and newspaper products by February 2013. A
cumulative total of regular users will be produced for the 2014 Mid-term Review.
Progress reported for 2012-13 (including Amar Engreji Course 1 ), to February 2013:



Television ‐ 13.6 million people watched Mojay Mojay Shekha 2.
Mobile ‐ 3.3 million calls made to the service during the year. 700,000 users undertook
learning using the new course.
Website ‐ 780,000 unique visitors to the website during the year, with 140,000
registered users on the site.
Indicator 2: Perception of reduced barriers and increased positivity towards learning
communicative English among school leavers and adults. 2014 Milestone: 43% of people
who have used BBC EIA products will report increased motivation to learn and/or increased
confidence to use English in their everyday lives (47%M, 35%F)
The focus on reducing barriers to learning and using English has been informed by base-line
and continuing research, which shows that even people who have studied English at school,
where the focus has been on reading and writing without developing speaking and listening
skills, lack confidence to use English and are reluctant to study what is perceived as a difficult
subject. To assess progress against this milestone a major survey will be done in 2013,
feeding into the 2014 mid-term review. A number of smaller studies have also been done in
the past year, from which there are good indications that the component is on track for the
2014 milestone.
For example:




Impact evaluation of Amar Engreji Course 1 undertaken during the year showed that
“willingness to learn English” and “desire to learn English” both increased following
exposure to the course.
Qualitative feedback from Mojay Mojay Shekha 2 viewers demonstrates an interest in
and positive attitude towards learning English from watching the series.
“Mojay Mojay Shekha can inspire women because it provides them with the opportunity
to participate in the show and other women will get inspired to learn English having
been given the opportunity to watch them on TV.” Female, 20 Years, Manikganj
Interviews conducted with Mojay Mojay Shekha 2 viewers before and after the series
indicate an increased motivation to practice English with others, despite knowing that
they might make lots of mistakes. Further, fear of learning English appears reduced,
particularly amongst female, rural study participants.
Indicator 3: The percentage of people exposed to BBC produced EIA media products
showing an increase in average competence as evidenced by speaking and listening test.
2014 Milestone: 49%
Preliminary results of testing suggest that achievement is on track for 2014. Reporting against
this indicator requires the development of innovative methodology which, in itself represents a
gain for the project. Assessing the effect of the BBC Janala mass-media intervention (mobile,
TV, newspaper and webpage) on English competency, outside of a classroom setting, has
provided a number of challenges. In particular, the design of the speaking tests used initially
8
were reviewed by an international ELT testing expert and have been redesigned in
preparation for research on the impact of Amar Engreji Course 2. Despite challenges, results
showing progress :

49% of the sample demonstrated an increase in English competence (of any size)
between baseline and end line.

Prothom Alo newspaper readers’ English competence increased over a five month
period, with the level of competence increase correlating to their level of exposure to
BBC Janala lessons.

Mojay Mojay Shekha 2 viewers’ self-assessment of their own English competence was
higher amongst those who had watched more episodes and amongst those who had
watched for longer.

Speaking test results for viewers not engaged in other informal English learning
activities showed a small increase in competence, correlated to viewers’ level of
exposure to the TV series.
Additional analysis to demonstrate the effect of BBC Janala on learner’s English competence
is still needed, although the experience of conducting the recent year’s research and the
benefit of the expertise engaged to redesign the project’s speaking tests should further inform
progress against the 2014 milestone.
Recommendations:
The English in Action project is regarded by DFID as well-positioned to contribute to key
objectives for the Country Operational Plan and so is encouraged to be ambitious. This will
require a wider team effort to identify the most appropriate opportunities and to consolidate a
strategy to achieve them.
While the development impacts of learning at the school level are well evidenced and
understood in the longer term, the new Theory of Change (TOC) and the project’s critical
paths analysis have enabled t better understanding about how the adult learning component
can and should contribute to the higher order objectives. However, while the TOC has
influenced the project – especially the research agenda – more can be done to unpack the
assumptions at all levels of the logframe. This will reveal and define how changes are
achieved and what they can lead to for individuals and more broadly in society and the
economy.
Recommendation 3
By the Mid Term Review (MTR) in 2014, EIA to develop a roadmap for sustainability of the
adult learning and media component in order to make specific contributions to livelihoods,
inclusion and economic development objectives.
Impact Weighting (%): 25%
Revised since last Annual Review? N
Risk: Medium
Revised since last Annual Review? N
9
Output 4: The generation and effective dissemination of knowledge and evidence for what
works in improving English language competency and how this contributes to economic
development in Bangladesh
Output 4 score and performance description: A+
Progress against expected results
Performance indicators
Milestones for 2014
On track
1. Number of knowledge
products developed to assess
project at impact, outcome and
output level.
15 Research reports; 9 research
briefs; 4 studies assessing
outputs and outcomes in Phase
III for schools component and 6
for adult learning component; 5
case studies on befits and use of
English.
Yes
2. Amount of research
disseminated to key school,
upazila, district, divisional,
national and international
audiences, via appropriate
knowledge products and
platforms
A Research Dissemination
Action Plan developed and
followed
Yes
Progress against indicators and milestones:
Indicator 1: Number of knowledge products developed to assess project at impact, outcome
and output level. 2014 Milestone: 15 research reports; 9 research briefs; 4 studies assessing
outputs and outcomes in Phase III for schools component and 6 for adult learning component; 5
case studies on benefits and use of English.
Indicator 2: Amount of research disseminated to key school, upazila, district, divisional,
national and international audiences, via appropriate knowledge products and platforms. 2014
Milestone: Research Dissemination Action Plan developed and followed: 29 national and
international conference papers (OU: 18, BBC: 11); 4 papers in peer-reviewed journals (OU)
and 2 chapters in academic books (BBC); distribution of research reports to key stakeholders
within Bangladesh and to research databases of Research for Development (R4D) and Open
Resource Online (ORO) within OU; 3 policy seminars; and PSC/GoB meetings including with
units within GoB and meetings with teachers and TFs.
Progress and plans for research reports is on track and exceeds the 2014 expectations for
publications and presentations. This is particularly valuable in the light of the recommendations
of the 2012 Annual Review and the subsequent changes of the logframe.
The independent reviewers found that EIA project management are using monitoring and
research evidence as quality assurance across the implementation activities (monitoring), for
continual improvement of project design, for laying a sound basis for evaluation and for
dissemination of findings locally and internationally. Since the 2012 review, all aspects of this
work have been strengthened by strategic planning against the new Theory of Change. More
working on understanding the assumptions that underlie the TOC will be of further help.
10
The QA process for school interventions is being implemented, working through government
Education Officers and Teacher Facilitators (TF) at Upazila level. Substantive local capacity
building has taken place through a series of orientation workshops for senior primary and
secondary Education Managers (divisional, district, upazila), Primary Teachers Training
Institutes (PTI) and Secondary Teachers Training College (TTC) Principals and Academic
Supervisors, and a series of training workshops for Education Officers and TFs implementing
the QA processes in the field. Findings from the QA process have been shared with, in
particular, the Teacher Training & Support Unit and EIA Management. Internal workshops have
followed with Core Trainers and OU ELT academics to review and respond to these findings.
Quarterly upazila sharing meetings have also taken place, where findings from the QA process
are fed back and discussed with Upazila Education Officers (UEO) and TFs.
The successful partnership with the Institute of Education and Research (IER) at Dhaka
University has been continued with a new batch of MPhil students starting in January 2013.
Their first fieldwork has taken place in respect of the cross-project community study.
A cross-project dissemination action plan has been developed and research findings have been
disseminated at senior Government workshops, policy seminars and major international & local
conferences, such as the International Association of Teachers of English as a Foreign
Language (IATEFL), UNESCO, the Bangladesh English Language Teachers’ Association
(BELTA), the Nepal English Language Teachers’ Association (NELTA) and mLearn (the
conference on mobile learning).
Key achievements are:
 Implementation of the new quality assurance system at scale, tracking over 4,000 teachers
through a series of workshops and cluster meetings and through over 750 classroom
observations.
 Establishment of improved project management information system (PMIS).
 Capacity of over 500 TFs and UEOs developed and utilized in quality assurance processes.
 Further development of the research collaboration with IER at Dhaka University, including
five out of fifteen students who are on the MPhil programme on a self-financing basis as a
move towards sustainability.
 Cross-project research findings disseminated through GoB workshops, policy seminar and
at major international and local conferences and peer reviewed journal articles.
For the adult learning / media component, most of the research conducted over the past year
has been to evaluate the first Amar Engreji Course and to inform the development of the
second. Studies informed the development of products on print, mobile and internet platforms.
The diagram below shows the systematic user testing process for the first course.
Research plans exist to measure the impact of the new course, being launched at the time of
the review. A test designed by UK-based ELT specialists will measure improvement in English
language competences of the second Amar Engreji Course users.
Recommendations:
The review team suggests opportunities to use data from the project as evidence for policy and
strategy development in the education and training sectors in Bangladesh and globally. An
increased focus on learning outcomes has already been embraced and should go deeper, using
the evidence from this project and seeking relevance and applicability beyond English language
learning.
11
The immensely attractive English in Action media products, the innovations in their delivery to
school students, teachers and learners in the community, and the successes of the project
could be showcased to DFID senior management and ministers for information and advocacy.
Recommendation 4
The review team suggested DFID advise the EIA team on the most effective and useful options
for defining, calculating and presenting the cost per improved learning outcome for children and
adults, especially for policy dialogue in Bangladesh. This is needed in time for research to be
planned and results produced by the MTR in 2014.
Impact Weighting (%): 10%
Revised since last Annual Review? Y
Risk: Low
Revised since last Annual Review? N
Output 5: The institutionalisation and sustainability of EIA approach across schools and the
media sector
Output 5 score and performance description: A+
Progress has exceeded this output and is one of the strongest areas of the school component.
Considerable success has been noted on securing gains in practice and evidence for policy
making. The project is now well-positioned, to influence policy and strategy, both within DFID
and beyond.
Progress against indicators and milestones:
Performance indicators
1. Develop strategic partnerships
with key institutions and
building capacities to sustain
EIA interventions beyond the
life of the Project.
2. Influencing the development of
an English language media
sector through improvements
in the capacity to develop
products and programmes
3. English Language education
policy in Bangladesh
supporting communicative
language teaching practices
4. Influencing curriculum,
Textbook and assessment
changes supporting
Communicative English
Language Teaching
Milestones for 2014
On track
Key strategic partnerships
established and strengthened
capacity of partners in the areas
of teacher training, materials
development and research
Yes
Key organisations and 100
sector professionals have taken
part in ELT media capacity
building initiatives
Yes
Core policy holders stakeholder
bodies, including MOPME and
MOE, engaging in development
of new architecture for English
Language education policy in
Bangladesh.
EIA actively engaging with
Government of Bangladesh
institutions to reform the
curriculum, revise the textbook
and adjust the assessment
12
Yes
Yes
mechanism
Progress against expected results
Indicator 1: Developing strategic partnerships with key institutions / the formal and informal
educational community and building necessary capacities to sustain EIA interventions beyond
the life of the Project. 2014 Milestone: Key strategic partnerships established & strengthened
capacity of partners in the areas of teacher training, materials development and research.
Increased partners’ responsibilities for implementing EIA interventions.
Progress is well on track with clear evidence of strengthened engagement with the GoB and
an appreciation from the government side of the value of EIA technical assistance and
products. This is evidenced by:
 Requests from the Secretary of Ministry of Primary and Mass Education (MoPME) for the
EIA approach to be rolled out to all schools in the country.
 MoPME approval of technical cooperation between EIA and NCTB regarding textbooks,
student workbooks, teachers guide, audio-visual and other supplementary materials and
primary curricula.
 Directorate of Primary Education (DPE) invitation to provide technical advice on the content
of their six-day in-service training course, which has now been provided.
 Directorate of Secondary and Higher Education (DSHE), Director of Monitoring and
Evaluation to hold joint workshop in June 2013 for collaboration plans for monitoring and
evaluation.
 Joint research undertaken with NCTB.
 Substantial contribution to the development and piloting of the new pre-service teacher
training called Diploma in Primary Education (DPEd).
During the past year, the following activities have been undertaken:
 EIA Orientation and Cooperation workshop organised in collaboration with DPE in February
2013, with participation from MoPME, Ministry of Education (MoE) and 5 key GoB agencies.
 Academic contribution to the development of the Diploma in Primary Education (DPEd)
programme in collaboration with National Academy for Primary Education (NAPE).
 Directly working with NCTB on a range of specific activities, including input to new teacher
guide, participating in small-scale trialling of new textbooks.
 Responding to requests from DPE and DSHE to provide technical advice on aspects of inservice training and monitoring.
 Working with a major World Bank and GoB secondary sector initiative, in reviewing their
ELT materials and in the development of a proposal for TV broadcasting for teachers.
 Policy Seminar in May 2012 on "Making learning effective through teacher professional
development – implications for policy".
Indicator 2: Influencing the development of an English language media sector through
improvements in the capacity to develop products and programmes. 2014 Milestone: Key
organisations and 100 sector professionals (Broadcast and ELT) have taken part in ELT
media capacity building initiatives
BBC Janala has continued to show success against this indicator and is on track to achieve
the milestone. With the 3rd creative media workshop, held in September 2012, 74 participants
have been reached, and another is planned for 2013.
13
The aim of the creative workshop was to bring together young Bangladeshi creatives, from a
variety of backgrounds including film, animation, photography, digital media and ELT, to
develop ideas for ELT media. The week‐long training allowed participants to develop a wide
range of ideas for low cost English learning media products and involves intensive
brainstorming, with inputs and advice from a range of senior creative professionals.
A secondary outcome of the creative workshop process is the links that the project has been
able to make with senior members of the ELT and media industries in Bangladesh. Following
feedback from these partners, an exploratory workshop was held in June 2012 with a view to
form an informal “alliance” of ELT and media organisations. Representatives at the workshop
set out and agreed a vision for English learning media in Bangladesh – this common goal is an
important part of meeting the sustainability goals of the adult learning component in the long
term and work will continue with key industry representatives to support and encourage
progress in this area over the coming year.
The plan for building a media ‘alliance’ around ELT and the plan for content distribution in the
next phase of the project amount to a sound strategy for capacity building and sustainability.
Indicator 3: English Language education policy in Bangladesh supporting communicative
language teaching practices. 2014 Milestone: Core policy stakeholder bodies, including
MOPME and MOE, engaging in development of new architecture for EL education policy in
Bangladesh.
The progress under Indicator 1 shows that the project is laying the basis for necessary
engagement and is on track to achieve this milestone too.
Indicator 4: Influencing curriculum, textbook and assessment changes supporting
Communicative English Language Teaching. 2014 Milestone: EIA actively engaging with
GoB institutions to reform the curriculum, revise the textbook and adjust the assessment
mechanism
Progress shows a strong basis for achieving this milestone. However, reforms on the
curriculum and textbooks for one subject are unlikely to lead to reforms in assessment, which
will be necessary across the curriculum. So, a focus on classroom practice and teachers’
understanding of how students are learning or failing to learn is important, and an emphasis
on differentiated learning is essential but not sufficient to drive a much wider and deeper set of
reforms. Assessment systems are key as they determine incentives for particular behaviours
and can undermine progress at the school and classroom level. For this reason, the project
progress needs to be seen as and deployed as evidence for the wider sector dialogue and
policy reform.
Recommendations:
The project has identified and created many valuable opportunities for sustainability of gains
beyond those suggested in the original approach and the listing of assumptions and risks. For
both components and the project as a whole, taking full advantage of these opportunities
during the next phase will be crucial.
The project should not be seen in isolation but in terms of how it contributes to a range of
development objectives. For this reason, the EIA ‘brand’ could soon become a hindrance and
is in fact already being toned down for more effective integration within a harmonised and
aligned dialogue on policy and planning, led by DFID. On the other hand, the powerful
reputation of the BBC will continue to signal quality for its distinctive products.
14
Recommendation 5
The reviewers recommended that DFID and EIA develop a roadmap for strengthening the
strategic contribution of the project as a whole to wider and higher order objectives in the DFID
Country Operational Plan, including aid effectiveness principles of alignment and
harmonisation. This should be done by the time of the MTR. Efforts should be refocused and
intensified with these changes reflected in individuals’ annual objectives and results reported
to the MTR. However this recommendation needs further discussion and analysis within DFID
and the project.
Impact Weighting (%): 15%
Revised since last Annual Review? N
Risk: High
Revised since last Annual Review? N
Section B: Results and Value for Money.
1. Progress and results
1.1. Has the logframe been updated since last review?
Yes.
1.2 Overall Output Score and Description: A+
The English in Action project is performing strongly and consistently across all outputs. It is
well on track to produce results that meet and in fact exceed expectations. . The project team
has clearly done very well, despite delays in GoB approvals, t and remained on track with very
good quality delivery, in both the classroom behaviour change and in the media impact.
An additional area that exceeds expectations is that of quality, with training, print products,
innovative use of technology, quality assurance/monitoring and world standard research all
contributing beyond what can normally be demanded of a sector-based project.
Management of the team and capacity building also deserve the strong endorsement of the
review team. What cannot be under-estimated is the way in which so many challenges e.g.
delay in government approval, new textbook resulting in adjustment in training materials, have
been faced and overcome.
There remain strategic challenges, as outlined in this report, but there is confidence that the
team, with DFID’s support can take these on board and respond accordingly so that the
outputs will clearly contribute to achievement of the outcome.
1.3 Direct feedback from beneficiaries
The review took place at an exceptionally difficult time in Bangladesh and the planned field
visit to Sylhet had to be cancelled because of threatened national strikes. Visits to only two
schools in Dhaka were possible, so that the review team had little opportunity to talk to
teachers and students. Feedback received from teachers, one head teacher, one deputy and
a dozen school students was all positive (and conducted in English); and active classroom
practice was observed. It is difficult however to draw broader conclusions and so the reports
of media user surveys and of ‘teacher voice’, for example, are valuable information on how
15
beneficiaries perceive the project.
1.4 Summary of overall progress
An action plan in response to the 2012 Annual Review recommendations has been developed
and implemented, A summary of progress against the action plan is shown below grouped
into six focus areas.
Revision in the project framework
 The project logframe has been updated in light of the 2012 Annual Review
recommendations, in particular for Impact/Goal and Output 4, and simplified, where
possible.
 A Theory of Change: from Learning English to Economic Development, based on critical
pathways, has been developed and endorsed by DFID.
Institutionalisation and sustainability of schools programme, including advocacy
Since the 2012 Annual Review, the project made significant progress in building relationships
with GoB; getting engagement with different mainstream activities, including textbook
revisions, pre- and in-service teacher training and monitoring and evaluation; and undertaking
a number of advocacy initiatives at different levels, drawing on the practical experiences of the
project.
 A series of reflections and stories from a range of EIA participants (including teachers from
the pilot phase) have been collected.
 EIA developed plan for appropriate and effective methods to gather, interpret and
disseminate “Teacher Voice” through self-reporting of experiences and issues
(questionnaire, focus group discussion, interviews etc.)
 In parallel, project staff have undertaken classroom observations and deeper interviews
with participants.
 These are being used to prepare a set of actions for reconnecting to pilot teachers,
revisions to teacher professional development materials and to inform plans for local
institutionalisation and sustainability.
 In parallel, champions amongst teachers have been identified.
 GoB workshops/meetings are drawing on the research evidence – targeted at central
Ministries and education departments and agencies and at local level at District and Upazila
Education Officers.
Adult learning: assessment, content and sustainability
 An English language test, linked to course syllabus, has been designed with an
independent ELT expert and linked to the Common European Framework of Reference for
Languages.
 In advance of Adult Learning Course 2, new additional materials are being published on
the BBC Janala web and print platforms to maintain learners’ momentum.
 Potential models for the future sustainability of access to and development of ELT media
content have been presented to DFID.
Research Dissemination & Project Communications
 A comprehensive participant analysis, communications and research dissemination
strategies, relevant partner or participants engagement plan setting out key activities
within Bangladesh has been completed.
 An expert has reviewed and reported on EIA’s dissemination and, in a second report, set
out a proposed framework for dissemination action plan.
16





Drawing on this, a cross-project 2013/14 dissemination action plan has been developed.
Strategic focus has been adopted to target dissemination at ELT, international
development education research/teacher education and technology/mobile learning
audiences (international, regional and local), in addition to GoB and other local major
players.
Policy seminars are drawing on the research evidence – in May 2012 on "Making learning
effective through teacher professional development – implications for policy" and planned
in third quarter of 2013 on “Assessment of English in Schools”.
A cross-project communications team has been set up, which meets quarterly and
regularly updates the DFID Communications Team. A number of significant joint outputs
have been launched, such as the new EIA website, EIA brochure, community fairs.
Brand equity research has been undertaken and the value for money of promotional
materials has been reviewed. All promotional items follow DFID guidelines and EIA
Communications Strategy.
Social Inclusion
 An audit of the Social Inclusion Strategy Action Plan has been undertaken and 2013/14
actions prepared.
 DFID held a social inclusion session with BBC for broader technical understanding and
addressing inclusion issues in the adult learning materials
 The creative brief for Year 5 TV production was extended to socio-economic group E and
research undertaken into further ways of reaching the poorest. The TV series developed
during the past year was specifically targeted at the poorest and will contain a strong call to
action to publicise the mobile platform as well as other BBC Janala content.
 Further the selection criteria of schools include a strong mix of diversified demographic
settings, socio-economic conditions, male-female ratio and varied ethnic communities. All
project reporting should show disaggregation of data for gender, rural/urban, sub-national
and socio-economic categories.
Value for money
 The cost per child or adult with improved learning outcomes is being calculated to assess
the cost-effectiveness of project interventions.
 Particular focus has been given to review the unit cost of materials per teacher for the
schools component and unit costs for each media component and for the English Language
clubs for the adult learning component.
In addition to the Annual Review recommendations, an environment strategy is in progress
and will be finalised soon.
1.5 Key challenges
The operating environment faces additional instability and this has been reflected in the
revised risk assessment. A related key challenge is to build sustainability for all project
achievements within and beyond the state sector, by extending the working definition of
sustainability beyond institutionalisation, by continuing to manage risks effectively and to
strengthen and develop partnerships and innovation.
Other emerging themes from this review are to challenge the project to:

aim for and analyse the improvement of learning achievements beyond the specific outputs
for school students and adult learners, so that these are seen to contribute (at Outcome
level) to social and economic opportunities;

to maintain the very strong management and performance of the project through the next
17
period of scale-up and consolidation and despite the threat of political instability;

further to work on the assumptions within the new Theory of Change;

and (by doing this) to win the support and leadership of DFID beyond the education sector
– to help achieve and understand the gains at outcome and impact levels.
Additional funding request
English in Action and DFID have identified areas where project successes can be increased
and ensured in the final sustainability phase through additional funding. The focus will be on
capacity building within GoB teacher education and monitoring structures, and on supporting
institutionalised implementation in the final phase (Phase IV) and on extending adult learning
materials to provide industry-specific learning opportunities at higher levels.
Since the last annual review, EIA has deepened engagement with a wide range of GoB
ministries, directorates and agencies. These experiences suggest that the extent of capacity
building, support and monitoring required for effective institutionalisation of the schools
component at both national and local levels is significantly greater (by an order of magnitude)
than was apparent during project design. However, the current budget for Phase IV sees a
halving of the current Phase III budget with a ten-fold increase in the teacher reach target.
If, in addition to the increased targets, the project is to respond more effectively to the need
and demand from GoB and other partners for the full institutionalisation of EIA’s approaches
(both to strengthen impact and increase sustainability); and maximize impact and value for
money by extending the project’s reach, then additional efforts and funding will be needed.
1.6 Annual Outcome Assessment
The logframe measures success at this key level with the indicators and milestones below.
These are quoted in full because the question arises whether these measures do provide the
most useful balance of focus on the most valued outcomes of the project with the most cost
effective methodology for obtaining the information and the best ways of presenting it.
The review team would like to suggest that measures of competence need to be understood
by stakeholders in Bangladesh. This could mean a relatively simple ‘translation’ of Trinity
levels into description of competences. Having done this, is it possible to link the project
measures with GoB objectives and strategies, for example in the labour market, and with other
skill delivery and accreditation mechanisms in the public and private sectors?
In any case, data should be disaggregated by gender, poverty level, sub-national regions, etc.
to be of most value for policy makers. It is understood that this will not be a problem for EIA.
Outcome Indicator 1: Increases in proficiency in communicative English among the
population of Bangladesh. 2014 Milestone: Numbers of teachers, students and adult learners
classified at higher (improved) levels of proficiency.
Outcome Indicator 2: Employers’ satisfaction (with employees’/applicants’ communicative
English). 2014 Milestone: (Decreasing) numbers of applicants without and (increasing)
number of employees with necessary EL skills.
Outcome Indicator 3: Motivation for learning and using English among school population of
Bangladesh. 2014 Milestone: Number of students and teachers who report that English is
very important and difficult to learn.
Outcome Indicator 4: Use of English language news and media: online, broadcast or print18
based. 2014 Milestone: 30% improvement from baseline.
It is not possible to report accurately on progress towards the outcome measures but the
evidence presented of initial and small-scale investigations gives positive signs that progress
sound, towards the 2014 milestones.
Annual reporting would not be cost effective for these measures. However, with some further
work, the logframe and TOC can be improved to aid significantly the management of the
project and the process of the MTR, at all levels and especially for the outcome:
Recommendation 6
The logframe should provide annual indicators of progress, because this would make scoring
more precise. This review has focused on whether progress is ‘on track’ for the 2014
milestone rather than report against a set of indicators for 2013. Indicators should also be
specific, measurable, attainable, realistic and timely (SMART) as far as possible to facilitate
objective scoring.
Recommendation 7
As noted, the TOC will be more useful if assumptions are further unpacked to show how the
key changes between inputs – outputs – outcome – impact are expected to be achieved, with
research and QA focussing on testing these assumptions.
2. Costs and timescale
2.1 Is the project on-track against financial forecasts: Y
The project is on track against financial forecasts. Training of the first 4,500 teachers under
Outputs 1 and 3 in Phase III was set back by six months in 2012 due to a delay in GoB
approving the project, and this means in turn that the final 8,000 teachers in Phase III are
planned to start from June 2013. Activities and budget have been re-profiled and approved by
DFID to ensure outputs will still be achieved by end 2014. In order to mitigate the effect of the
planned reduced budget for the adult learning component, a request was made to DFID to
bring funds forward for this component from Phase IV into Year 6. This was recently
approved.
2.2 Key cost drivers
Key cost drivers of the schools component are teacher support, materials and equipment.
Significant reductions in materials unit costs have been, and are planned to be, achieved by
reviewing and, if appropriate, removing specific types of classroom materials (e.g. figurines),
streamlining remaining resources (e.g. flash cards) and achieving economies of scale through
greater quantities. In Phase IV, it is planned that equipment costs will be met from a variety
of sources; specifically, teachers will use their own mobile phones, the project will continue to
provide memory cards and PEDP III programme has expressed interest in supplying the
speakers.
For the adult learning component, key cost drivers relate to the production for and use of the
various media platforms: television, mobile phone networks, web and the English Clubs. As
Phase III draws to a close, the adult learning component will also be able to begin to evaluate
the “whole life” unit-cost of media platforms, as well as the year-on-year costs. These costs
are effectively decreasing every day, since every repeat TV broadcast or additional mobile
user accessing content produced in preceding years decreases the unit cost for those
products.
19
More details on unit costs and of the compliance with associated recommendations are
reported in the Value for Money (VFM) section below.
2.3 Is the project on-track against original timescale: Y
Despite many and serious challenges in the operating environment, the project team has
proved very flexible and determined, has continued to build capacity, ensure quality and
manage implementation within the plans.
3. Evidence and Evaluation
3.1 Assess any changes in evidence and implications for the project
There is a new Theory of Change.. This has had implications for the project and research
design and the new evidence will feed into analysis of contributions to outcome and impact.
At the root of the project is the question of ‘Why English now in Bangladesh?’ the definitive
answer lies within the demonstration of successes at outcome and impact levels. Meanwhile,
the review team was continually reassured by Bangladeshi stakeholders, as well as by the
research evidence, that English language skills are highly valued and that demand for their
acquisition is very strong.
3.2 Where an evaluation is planned what progress has been made?
No evaluation is currently planned. The Mid Term Review in 2014 is the opportunity to
monitor the project in depth as it enters the next phase.
4. Risk
4.1 Output Risk Rating: Medium (unchanged)
4.2 Assessment of the risk level
The 2012 AR noted that the main risk for the EIA programme was its sustainability. This has
been identified primarily as a question of institutionalisation (within government structures and
processes), potentially threatened by political instability which is outside the control of the
project. As noted elsewhere in this review, the project is now well-positioned to explore other
aspects of sustainability and to analyse a range of opportunities within and beyond the state
sector. This will also contribute to spreading and mitigating risk.
The comprehensive EIA risk management matrix has recently been updated. The revisions at
Impact and Outcome levels (a,b) and for Outputs 1 and 3 (c) are:



Major political instability at national level: probability increased from Low to Medium, with
some additional information on the upcoming elections and criminal court verdicts of the
war criminals.
Current political commitments to prioritise English learning for economic development not
maintained: from Low to Low-Medium, related to the upcoming elections and possible
change of power.
Failure to gain timely and effective access to schools and teachers: increased from Low to
Low-Medium.
20
4.3 Risk of funds not being used as intended
Risk of funds not being used as intended is low. Overall the project is managed well and costs
controlled. The project has experienced some delays in 2012 due to textbook revisions and
political instability in Bangladesh. This led to the project being off track against financial
forecasts. To recover these delays and ensure project targets are met, activities and the
budget have been re-profiled and included in Year 6.
4.4 Climate and Environment Risk
Not assessed.
5. Value for Money
5.1 Performance on VfM measures
The project continues rigorously to measure and monitor key performance indicators on Value
for Money through transparent and well documented processes. These indicators focus on the
cost of inputs (materials) and outputs (cost per teacher/student/adult learner reached) and thus
the VfM concepts of economy and efficiency. In the past year, further work has been done to
develop methodology for measuring quality, to be expressed as cost per learner with improved
learning outcomes. This is a challenge, especially outside a classroom or school context, for
adult learners with access to project material across a range of media platforms but the work
being done is innovative and will be valued by many stakeholders.
The 2013 Annual Review aims to strengthen the efforts since the last AR to develop a Theory
of Change which encompasses Outcome and Impact levels and makes recommendations
related to further development of methods and measures for improving VfM in terms of
effectiveness (maximising conversion of outputs to impact). The VfM concept of equity, too, is
being addressed and the recommendation on using disaggregated data relates to this.
21
5.2 Commercial Improvement and Value for Money
Following the 2012 review recommendation the project is seriously working towards monitoring
the cost per child or adult with improved learning outcomes. This will demonstrate how costeffective the intervention is at improving language skills of those reached. The definition of
‘improved learning outcome’ is being very carefully considered; for example, to ascertain its fit
with relevant DFID-used global indicators and to ensure that it can be clearly understood and
explained.
Creating a sustainable legacy can be seen as an element of VfM cost-effectiveness, so the
project needs to find a way to take creating a ‘legacy’ into account. EIA is not only seeking to
improve student learning outcomes, which could be implemented through a wholly project
mode, but also to make this change in ways that are sustainable to scale within the GoB
system at national and local levels. Therefore, activities may take longer and be more
resource-intensive but they will be sustainable and more cost effective in time.
New methodology is being developed to calculate the cost per adult learner with Improved
learning outcome. In recent months, a study was completed to track English language
competency increases amongst media product users. This data, alongside the learning gained
from the competency testing process, will enable the project to define improved learning
outcomes amongst adult users of EIA media products and develop a mechanism for tracking
the cost per adult with improved learning outcomes in the future.
Value for Money of the schools component is dependent on the number of schools reached
and the project’s effectiveness in improving students’ English competence. The project is on
target to reach 12,500 teachers by 2014 and plans to reach a further 64,000 teachers by 2017.
Based on these figures, the unit cost per teacher of delivering the teacher development
package reduces from £600 in Phase II to £302 in Phase III and to £23 in Phase IV. Similarly,
the cost per student reduces from £3 in Phase II to £1.20 in Phase III and to £0.20 in Phase IV.
Comparable data against cost per teacher is not available mainly because the teacher training
methodology and package differs from government teacher training programme. The table
below indicates the approximate unit cost of the various types of government teacher training
program and it should be noted that this does not include the material development costs
whereas EIA cost per teacher does.
Training
£ Unit cost*
119
64
44
ICT training
Subject based training
Sub cluster training
Teacher support network
leadership training for head
teacher
Pre service training (12 months)
Induction training for new
teachers
34
93
38,889
52
*Based on 2013/14 annual operational plan of the sector programme.
A request from the Secretary of MoPME has been made for the EIA approach to be rolled out
to all primary schools in the country, which will potentially further reduce significantly the cost
per student reached.
In response to the recommendation to ‘continue refining the materials to ensure they are low
cost, durable and potentially affordable for GoB to reproduce after the end of the EIA
22
programme. The indicative per teacher or pupil spend on materials under the Third Primary
Education Development Programme (PEDPIII) programme could be used as a benchmark
against which to compare project spend.’ The project has undertaken a cost comparison with
PEDP III, showing printed materials at a higher cost (although the extent and quality of the EIA
materials is higher) and equipment at a lower cost. Any monies spent by GoB on student
textbooks are highly unlikely to result in changes in student learning without associated teacher
training. For an additional £0.20 (primary) & £0.02 (secondary) per student, EIA can provide a
full set of teacher professional development and classroom resources to the teacher to support
their effective use of the textbook. A workshop with partners and stakeholders is planned in
June 2013 to review how the materials can be made more cost effective, yet maintain good
quality.
Finally in response to the recommendation to stop promotional items without a developmental
goal the project now only produces promotional items with a development goal and minimises
cost of all such items. For example the EIA notebooks and pens are produced and provided to
only to project related people i.e. teachers, trainers, education officers and unit cost has been
brought down from £2.7 to £0.96.
Under the adult learning component, in response to the 2012 recommendations, the project is
now measuring and monitoring unit costs for each platform. Where regular user data is
available – particularly for the web and mobile platforms – unit costs are tracked monthly as
part of BBC Media Action’s organisation-wide value for money reporting processes. The cost
per learner for each of these platforms has also been reduced contributing to economies of
scale. The unit costs for Year 5 (2012/13) are presented below alongside the Year 4 (2011/12).
Platform
Television
English club
Mobile – new users
Mobile – multiple users
Mobile – calls
Web – visits
Web – new registered users
Book & CD
Industry training
Total Project Cost Year 5
Total cost
£821,442
£182,374
£342,018
£175,089
£7,919
£39,211
£1,568,052
Total no of
users / units
13,619,200
6,000
544,822
294,476
3,396,917
604,723
60,290
135,400
26
Cost per
head Year 4
2011/12
£0.13
£55.73
£0.24
£1.32
£0.07
£0.79
£4.72
£0.52
£2,049
Cost per
head Year 5
2012/13
£0.06
£30.40
£0.63
£1.16
£0.10
£0.29
£2.90
£0.06
£1,508
%
change
-54%
-45%
166%
-12%
36%*
-63%
-38%
-89%
-26%
These show a reduction in unit costs for the majority of platforms, driven by:



Increased experience amongst local production and creative staff, many of whom have now
been working on the BBC Janala project for a number of years, reducing the need for the
project to pay for international expertise or to outsource key creative activities.
Cessation of project subsidy to retail platforms (books and CDs), which are now solely
maintained by commercial income received by private sector partners.
Absence of initial start-up costs for English clubs and industry training activities, incurred in
previous years.
* Unit costs for mobile users and calls have risen over the year, due to a reduction in call
volumes rather than an increase in expenditure in this area. With fewer TV outputs on air and a
much reduced marketing budget for the year, call numbers have declined. However, unit costs
for users who dial the service multiple times have decreased suggesting that, despite lower call
volumes, users are spending more time on the platform when they call.
23
As Phase III draws to a close, the adult learning component will also be able to begin to
evaluate the “whole life” unit-cost of media platforms, as well as the year-on-year costs. These
costs are effectively decreasing every day, since every repeat TV broadcast or additional
mobile user accessing content produced in preceding years decreases the unit cost for those
products.
5.3 Role of project partners
The management of the project by BMB remains an area of strength for EIA. Quality
assurance through tightly managed, transparent processes, excellent human resources, well
documented procedures and very detailed monitoring combined with learning from a wide
range of world class research activities. BMB is, with the largest part of the EIA team, also is
one of the three main implementing partners, with the Open University (OU) and BBC Media
Action. This gives a slightly complicated structure and it has been necessary from time to time
to check the balance of roles and communications across management, implementation,
monitoring and research. Document and information management by the project team appear
excellent with documents well-categorised and easily available to the review team.
Recommendation 8:
One further improvement, starting with preparations for the mid-term review, would be to
ensure that reporting against strategic outputs 4 and 5, as well as against outcome and impact,
clearly bring the contributions of the two components together, as an aid to cross-project
planning for synergies.
It is important to point out that since the beginning of the project, cross-project strategic
exchanges and peer support/feedback has contributed significantly to the generation of new
ideas and enhanced the quality of both the media and schools components since the start of
the programme. The joint strategic learning environment should be considered a critical factor
behind the success of EIA as a project.
The cross-project linkages go beyond joint strategy development and learning and include joint
coordination of activities. A good example is the Research Monitoring & Evaluation
Coordination Group (RME-CG), which plans, approves and coordinates all RME activities that
happen across the project and supports the implementation of joint research activities. This
was also reflected through changes in the logical framework early 2012 and taken a step
further through the collaborative development of a Theory of Change document produced in
2013. The joint EIA Research Dissemination Strategy and joint research activities are covered
under Output 4 and the joint EIA Social Inclusion Strategy is also noted elsewhere in this
review.
BBC-produced media content is integrated and embedded within the English Language for
Teachers (EL4T) course provided to teachers, through material available on teacher’s mobile
phones and workbooks. There is also joint work on delivery, with the BBC Janala English Clubs
launching in schools, supported by the EIA schools’ component, and joint capacity building in
the creative workshops organised under Output 2.
5.4 Does the project still represent Value for Money : Y
English in Action does represent Value for Money and this is expected to increase over the life
of the project, especially by factoring in the quality of products, the extent of reach and the
sustainability of innovative delivery, behaviour change and language competence along with a
pattern of declining unit costs.
5.5 If not, what action will you take?
n/a
24
6. Conditionality
6.1 Update on specific conditions
n/a
7. Conclusions and actions
The review team was highly impressed by the professionalism of project management and
implementation. The project was originally designed primarily as technical assistance and it
brought on board the Open University and BBC Media Action as well as creating a team of
local EIA project staff and partners. This has helped ensure that sound evidence from
research and from activities on the ground are produced and continually fed back to inform
practice and adjust directions. Information management is extremely good.
The implementation of the schools component has demonstrated that behaviour change in
the classroom to create an effective learning environment can happen and to scale. The
adult learning and media component has generated enormous interest and appreciation
across Bangladesh and for all socio-economic categories. This is clearly laying the basis for
sustainability but there are still challenges ahead. To ensure sustainability, the techniques of
communicative language teaching will have to go beyond the English classroom to the other
subject areas and influence other key aspects of education, such as assessment. The
concept of sustainability will also need to be extended beyond the state sector and have real
life impacts on poor peoples’ access to livelihood opportunities.
8. Review Process
The review team of one international and one national consultant worked with EIA and DFID
Bangladesh for 12 days from 27th April to 10th May. Over one hundred documents were made
available to the team and presentations on annual progress were made by the implementing
partners. Despite the logistical and security challenges, the team observed lessons and talked
to teachers in one primary and one secondary school. Separate meetings were held with all
the project partners and with EIA senior management. There were also meetings with officials
of DPE and NCTB, IER, with Prothom Alo and the ADB.
The review team would like to thank the whole EIA team for their exceptional efficiency,
patience and flexibility during the review.
25