Iwik Pratiwi / 06
THE EFFECTIVENESS OF THE TRAINING COURSE FOR TOEIC
IN VOCATIONAL FACULTY OF BRAWIJAYA UNIVERSITY
CHAPTER I
A. BACKGROUND OF THE STUDY

To face the challenge of being an entrepeneural university and prepare students
competitiveness in the global market, Brawijaya University requires students of
all faculties to have adequate English proficiency prior to graduation, indicated
by their scores on TOEIC. (Pedoman Pendidikan Universitas Brawijaya, 2012).

Test of English for International Communication (TOEIC) is an instrument
developed by Educational Testing Service (ETS) to assess non-native English
proficiency and is currently used in 60 countries and taken by more than 4.5
million people per year (Trew, 2007).

Since 2012, English Compulsory Course for the first year students of Vocational
Faculty is conducted as a training course for the TOEIC (Pedoman Pendidikan
Vokasi, Universitas Brawijaya, 2012).

Apparently, assesment instrument affects educational practices and beliefes
(Cohen, 1994), therefore, it is important to implement a systematic attempt to
gather information about the course, to make any judgement or decision on its
effectiveness (Lynch, 1996).

On a study of the Effect of Three Month TOEIC Training Program, Tanabe (2010)
suggested that the combination of quantitative and qualitative data would work
as clues in evaluating similar programs.
B. STATEMENT OF THE PROBLEMS
General

Can Training Course for TOEIC improve students English Proficiency?
Specific

How much improvement do students make by taking Training Course for TOEIC?

What do students think of Training Course for TOEIC?
1

What do instructor think of Training Course for TOEIC?
C. OBJECTIVES
General

The study is aimed at measuring the effectiveness of TOEIC Training Course held
for students of Vocational Faculty, University of Brawijaya.
Specific

It investigates the degree of TOEIC score improvement of students following the
Course.

It also finds out what students think about the course and

what the instructors think about the program.
D. ASSUMPTIONS

All instuctors are assumed to be equally competent in teaching English.
E. OPERATIONAL DEFINITIONS OF TERMS
1) The Training Course for the TOEIC is an English Course conducted to implement
Vocational Faculty curriculum for the first year students (Pedoman Pendidikan
Vokasi Universitas Brawijaya, 2012). It is offered as an integrated English
instruction in which the four primary skills of listening, reading, speaking and
writing, as well as other related skills (vocabulary, spelling, pronunciation,
syntax, meaning and usage) are interwoven in the instruction.
2) TOEIC (The Test of English for International Communication ) developed by ETS
(Educational Testing Service), is an English proficiency test designed specifically
to measure the everyday English skills of people working in an international
environment which covers real-life settings such as meetings, travel, telephone
conversations, etc.
3) Effectiveness is the extent to which Integrated Skills Teaching in Training Course
for TOEIC can help improve students proficiency in English. This improvement is
measured from students gain on TOEIC scores.
2
CHAPTER II
REVIEW OF RELATED LITERATURE
A. REVIEW OF INTEGRATED SKILLS EFL.

Language learning strategies: What every teacher should know.. Boston, MA:
Heinle & Heinle. Oxford, R. (1990)
The integrated-skill approach, exposes English language learners to authentic
language and challenges them to interact naturally in the language. This
approach stresses that English is not just an object of academic interest nor
merely a key to passing an examination; instead, English becomes a real means
of interaction and sharing among people. Whether found in content-based or
task-based language instruction or some hybrid form, it can be highly motivating
to students of all ages and backgrounds.

Teaching by Principles. USA: Pearson ESL. Brown, H. Douglas (2000).
B. REVIEW OF COURSE EVALUATION AND PREVIOUS STUDIES

Lynch, Brian K : Language Program Evaluation: Theory and Practice.
Course Adaptive Model (CAM) for language program evaluation is meant to be
flexible adaptive, heuristic. It provides the basis for arguing for the important
contribution that program evaluation can make.

Tanabe,H. The Effect of a Three-Month TOEIC Training Program in Tokyo
Polythecnic University, T-Kougey research Journal.Vol-2-29-2001, Tokyo
Polytechnic University.
In the three months program of teaching TOEIC, there was not a clear evidence
that the course directly improved the scores of the students. This came from the
lack of quantitative data because of the smallness of the number of the
participants in the TOEIC practice course. However,the questionnaire given
showed positive motivational changes in the students and this might work as
clues to improve this course in the next semester.
C. REVIEW OF TOEIC AS AN INTERNATIONAL STANDARDIZED TEST
A Teacher's Guide to TOEIC®Listening and Reading Test Preparing Your Students for
Success. Oxford University Press. Trew, G. (2007) :

There are two formats of TOEIC : 1) Secure Program (SP) –The standard and
most common form of the tests administered directly by the TOEIC Steering
Committee or their local representatives. 2) Institutional Program (IP) which are
3
administered by individual companies and schools on days and at locations
decided by the institutions that conduct them. IP tests are generally used for
assessment of new employees, as a placement test for in-company English
lessons or for self-development.

TOEIC Listening and Reading Test consists of 200 multiple-choice questions
divided between four listening and three reading parts and takes two hours to
complete. The test is paper-based, with students using a pencil to mark their
answers on a special machine-readable answer sheet. The test is scored from 10
to 990 and there is no pass or fail mark. The following chart showing the
approximate correlation between TOEIC scores and levels of ability is based on
ETS research.
D. REVIEW OF TOEIC CHARACTERISTICS, VALIDITY AND RELIABILITY. (ETS 1998a:I-V).
 The TOEIC tests measure a person’s ability to communicate in English in the
context of daily life and the global workplace environment using key expressions
and common, everyday vocabulary.
 The TOEIC tests, which cover reading, listening, speaking and writing, relate to
one another in complementary ways.
 None of the TOEIC tests are regarded as an appropriate substitute for any one of
the other.
 Different (alternate) forms of the same test yield scores that have the same
meaning, timing, security and testing conditions are maintained across
administrations.
 The difficulty of test questions is appropriate for efficient testing,
 Speaking and writing scores do not depend on who (among many trained
readers) scores the tests, and enough aspects of English language proficiency are
tested to allow a test taker’s ability in all four language skills to be determined.
 TOEIC is claimed to be a highly reliable test, estimated by ETS to be within 25
points for one Standard Error of Measurement
 The TOEIC is successful in minimizing reliability issues,done in part by having the
test clearly organized, including clear instructions and by attempting to provide
non-culturally based inputs.
4
CHAPTER III
RESEARCH METHODOLOGY
A. RESEARCH DESIGN
 It is an evaluative research: measuring the program accomplishment on the
basis of its stated goals and objectives. (Neuman, 1997)
 It employs a Pre- Experimental Research or One-Group Pretest-Postest Design.
The effectiveness of the program is measured by comparing the average score of
the pretest and the postest. (Latief, 2012)
 A combination of summative and formative evaluation will be applied to get
adequate information about the program.
B. POPULATION AND SAMPLE
 The population of the study includes 563 students of Vocational Faculty taking
Training Course for the TOEIC.
 The sample will cover 20% of the population or 113 students.
 Stratified Sampling will be applied since there are 20 groups of students divided
into three sub population, each represents students level of proficiency (Novice,
Elementary and Intermediate).
 The relative size of each level is controlled to guarantee representativeness or
fixes the proportion of different strata within a sample (Neuman, 1997).
Levels
SAMPLE OF STUDENTS OF VOCATIONAL FACULTY
TAKING TRAINING COURSE FOR TOEIC
STRATIEFIED BY ENGLISH PROFICIENCY LEVELS
Population
Stratified Sample
N
Percent
n
Novice
180
31.97
36
Elementary
360
63.94
72
Intermediate
23
4.08
5
Total
563
100
N = the total number of population ;
5
113
n = the number of sample
C. RESEARCH INSTRUMENTS

TOEIC Institutional Test as required by International Test Center will be used as
the Pretest instrument.

TOEIC Standard Test which is an alternate form of the same test will be used as
the Posttest instrument.

Questionnaire will also be used to gather informations about what students and
instructors think about the course implementation which covers the
effectiveness and the appropriateness of the syllabus, processing time, course
materials and other course components.
D. DATA COLLECTION

The IP-TOEIC administered by ITC at the beginning of the semester before
students attend the course, will be used as The Pretest instrument to get the
preliminary data of students English proficiency

The SP-TOEIC, administered at the end of the instructional sequence by ITC, will
then be used as the Post Test instrument to get the data of students English
proficiency at the accomplishment of the course.

Students are also required to respond to a questionnaire concerning their efforts
to succeed in the Training Course for TOEIC and their opinions about some
aspects of the course.

Another questionnaire is assigned for the instructors focusing on aspects of the
teaching of Training Course for the TOEIC.
E.
DATA ANALYSIS

A simple basic statistical techniques will be used to analyze the research data
which covers organizing, comparing and drawing conclusions

The data will be organized using charts and graphs for both pretest and posttest
result. While the data gathered from questionairre will be organized as a rating
scales and percentage of opinions.
6

The data analysis for summative evaluation is the comparison of means scores of
the Pretest and Postest. The gain will then be used to decide the improvement
degree of students proficiency.

The data gathered from the questionairre as the formative evaluation will be
analyzed through a series of comparison examining the relative effectiveness
and the respective feasibility of the course.

Lists of recommendations and solutions will be discussed on the rationale of the
data analysis.
LIST OF REFERENCES









Brown, H. Douglas (2000). Teaching by Principles. USA: Pearson ESL.
Educational Testing Service. (1998a) TOEIC Technical Manual. Princeton NJ: Educational
Testing Service
Lynch, Brian K (1996) Language Program Evaluation: Theory and Practice.
Cambridge University Press.
Neuman, W.Lawrence (1997). Social Research Methods. Qualitative and
Quantitative Approach, University of Wisconsin, Whitewater.
Oxford, R. (1990). Language learning strategies: What every teacher should know.
Boston, MA: Heinle & Heinle.
Program Pendidikan Vokasi, 2010. Pedoman Pendidikan Vokasi Universitas
Brawijaya tahun 2010-2011,
Rogers, B. (2006). Introductory Guide to the TOEIC Test. Boston: Thomson.
Tanabe,H. The Effect of a Three-Month TOEIC Training Program in Tokyo
Polythecnic University, T-Kougey Research Journal.Vol-2-29-2001, Tokyo
Polytechnic University.
Trew, Grant, 2007. A Teacher's Guide to TOEIC®Listening and Reading Test
Preparing Your Students for Success. Oxford University Press.
7