Example of the research design part of a thesis (Education)
Example 2
RESEARCH METHODOLOGY
In the previous chapter, the effects of IGI promoting discovery-learning, procedure of
IGI, and the effectiveness of IGI on SLA and motivation are discussed. From the
insights, parameters for designing materials, tests, and questionnaire are drawn out.
Chapter Three is divided into five sections. Research design and subjects and program
description sections provide general information about the study. The next three parts,
materials, instruments, and procedures of the research are described in details.
3.1. Research Design
This study is carried out with a one group pre-test and post-test design. In this
experimental research, the implementation of discovery-learning through IGI
(independent variable) will be monitored, and acquisition about English tenses and
learners’ motivation (dependent variables) will be measured.
The grammar points in this research are English tenses. A pre-test and a post-test are
designed to be the same. An anonymous questionnaire on motivation is delivered after
the treatment instruction.
3.2. Subjects and Program Description
The researcher, who is also the teacher, carries out the study in six classes in Course 28
and Course 29 at CFL in 2006. The experimental course, Level-Seven Grammar Course,
is the first of three courses for Level C (low-advanced-English level). The grammar
course is offered to the students for three 45-minute periods on each occasion during nine
weeks.
The students are randomly chosen and can be the representatives for the research.
The total number of the students was 227, but the authentic number of participants for the
present study was 133 because some students did not satisfy the conditions of data
analysis (see part 3.4 of this thesis). The participants were from 14 to 26 years old,
including 71 female and 62 male non-English majors. Most of them were undergraduate
students of Cantho University; some were high school and junior high school students in
Cantho city. The students in the course came from three sources. Some of them have
passed the exam of Level B (intermediate-English level) at CTU; some have gotten the
Certificate of Level B at other centers of foreign languages; some have just taken the
placement test of CFL. Their educational background of general knowledge and English
was varied. This is also the primary reason why the research had only one experimental
group.
3.3. Materials
Based on the parameters concluded from the part Procedure of Implicit Grammar
Instruction (part 2.3 of this thesis), and the first four chapters of the main course book
Mosaic One: A Content-Based Grammar (Werner, 1996), the detailed lesson plans have
been designed (see Appendix one). To promote discovery-learning in the treatment, three
stages of tasks must be concentrated. First, noticing and awareness of form are designed
in Setting the Context. Next, exercises in Thinking about Meaning and Use are
discovery-based, and they encourage learners to use their grammatical knowledge and
English skills to express their remarks and finish the tasks. Finally, the three parts
Practicing, Using What You’ve Learned, and Taking a Step Beyond are production tasks
that move from controlled to free language activities. Besides, Review Exercises is a
consolidation part of the materials. It gives students a chance to practice test-designed
exercises.
The texts serve as the handouts for the learners in the treatment course of the
research. The materials consist of seven topics and three review exercises (see Appendix
1). Beside Mosaic One: A Content-Based Grammar (Werner, 1996), some activities and
exercises in the materials are collected and adapted from the following books:
(1)
Grammar Practice for Upper Intermediate Students (Walker & Elsworth, 2000)
(2)
Grammar Sense 3 (Bland, 2003)
(3)
Grammar Sense 3 – Workbook (Blackwell & Davy, 2003)
(4)
Nitty Gritty Grammar - Sentence Essentials for Writers (Young & Strauch,
1998)
(5)
Oxford Practice Grammar (Eastwood, J., 1999)
(6)
The Anti-Grammar Grammar Book - A Teacher’s Resource Book Of Discovery
Activities For Grammar Teaching (Hall and Stepheard, 1991)
(7)
Understanding and Using English Grammar – Workbook (Azar & Azar, 1990)
The students receive the handout of each lesson at the beginning of the class meeting.
In general, each topic discusses in one class meeting (three 45-minute periods). There are
five sections in each topic. Here are the detailed descriptions of each part.
Setting the Context presents a brief passage that highlights specific structures
covered in that section. “The text is used not only for comprehension and for setting up a
topic for use later in the unit, but also to make the students familiar with the new
language before teaching it later” (Harmer, 1987, p.30). It can be used to introduce key
vocabulary and to make sure the students have a general understanding of the content.
This part is very necessary to cover in class because it provides the context for students to
notice and raise awareness of the language form, and a tool for them to discover the
“rules” underlined. In Stern’s (1992) opinion, “put whatever grammatical feature is
being taught into a meaningful context of practical use so that the meaning is never in
doubt” (p. 144). Besides, “they see the language in its proper (authentic) context and are
then led to understand how it works” (Harmer, 1987, p.36).
Thinking about Meaning and Use is the main section that conveys the discoverybased approach. This part serves not only as language input for students to expose but
also as opportunities for them to activate their own grammatical knowledge of language.
All tasks encourage students to analyze how we use the target structure and consolidate
students’ understanding of all aspects of the structure. Students are helped to involve and
use their reasoning processes to discuss about the target structure in order to find out
rules. This part will be “a good reinforcement to what the students have been learning”
(Harmer, 1987, p. 37) and encourage them to be actively involved in the acquisition of
grammatical knowledge about of English verb tenses.
Practicing includes two controlled kinds of exercises of the target structures
that are selected from a variety source of grammar practice books. One is
supplying the correct verb tenses in a passage or a dialogue, and the other is error
analysis. Both kinds of exercises are aimed for recognition and reproduction.
Specially, in the later type of exercises students are asked not only to judge but
also to correct the language while in Mosaic One: A Content-Based Grammar
these exercises are only for error judgment. Most exercises can be used for either
oral or written work; they can be done as a class, in pairs, in small groups, or as
homework.
Using What You’ve Learned activities are chosen from Mosaic One:
A
Content-Based Grammar. In this textbook, there are many activities. From our
empirical experience, however, we cannot cover all because of time limitation.
Moreover, we have to adapt them to meet students’ needs and to suit Vietnamese
situations. Thus, our adapted activities of realistic and motivating contexts offers
students more time and opportunities to use target structures and vocabulary in
more natural, personalized communication, to apply their skills to relevant, realworld situations. This kind of communicative activities engages students in active
learning and appeals to a wide range of students’ abilities and interests, allowing
them to adjust their perceptions of the language input they have received.
Taking A Step Beyond is a new section in A Content-Based Grammar Course
- Level 7. The activities in this part provide students with suggestions of looking
for the target structures in authentic materials from magazines, newspapers,
Internet, or other sources. Sharing and discussing these materials with other
students will help to reinforce and contextualize the linguistic items covered.
They become a bridge between the controlled, structured language of the
classroom and real-life language outside the classroom.
Review Exercises appear after two topics. These exercises are carefully
selected from a rich source of grammar books. They highlight structures covered
in the two topics, recycle previously studied material, and integrate it with newly
practiced material. There are three kinds of exercises in this part: 15 multiplechoice questions, a passage or a dialogue to supply the correct form or (modal)
verbs, and 10 sentences for error analysis. These review sections can be used in
class, for homework, or as the basic for quizzes. This part serves as preparation
for students to take grammar tests in the Certificate Level C exam at the Center of
Foreign Languages, Cantho University. Through repeated exposure to and
practice with this standardized test format, students will gain more confidence and
experience less anxiety in actual test-taking situations.
To ensure these lessons comprehensible to the subjects, a pilot study was randomly
conducted with 36 students in Course 27 at CFL in 2006. All of the tasks have been fully
tried and tested. The results of this pilot study shows that all texts are appropriate to the
level, fully comprehensible for the students, and suitable to the current study.
3.4. Research Instruments
The present research applies both qualitative and quantitative approaches. There are two
research instruments: the scores from the pre-test and the post-test and the learners’
responses to the questionnaire. The first instrument will measure learners’ acquisition on
English tenses. The second is designed to explore learners’ motivation in learning
grammar after implicit grammar instruction with the implementation of the discovery
technique.
3.4.1. Tests on Acquisition About English Tenses
Designing Tests: From the discussion in the above part about Implicit Grammar
Instruction and Second Language Acquisition (part 2.4 of this thesis), the
achievement test on English tenses will be recognition and judgment tasks. The
task types selected for this project are the most popular kinds of grammar tests.
Some parts of the test follow the format of the test for Level C certificate at CFL.
Furthermore, these frames exert a very high degree of control over the specific
form of the limited production responses. The test questions are efficient at
measuring the learning of the objectives in the course program.
The pre-test and the post-test of the study are the same, but the part order in the
two tests is changed to distract wise-test students. The tests are a kind of discrete
point test types focused on only the uses of English verb tenses. They include
five parts. Here is the description of the test parts in the pre-test (see Appendix 2).
The first and the second part are multiple-choice recognition tasks that are
similar to the CFL test format. The first task consists of 26 incomplete sentence
stems followed by four multiple-choice options for completing the sentences. The
questions are selected and adapted from three sources:
(1). Mosaic One: A Content-Based Grammar (Werner, 1996)
(2). Understanding and Using English Grammar – Workbook (Azar & Azar,
1990)
(3). Teaching Outline and Supplementary Activities (2005)
The test type in the second section is the same as the first task, but it is
presented in an incomplete passage with 14 blanks. This part is adapted from the
Review Exercise 4 on page 141 of Mosaic One: A Content-Based Grammar
(Werner, 1996). The test takers must understand the context, then complete it with
the correct answers from the four given multiple-choice options for each blank.
Test takers’ ability of recognizing the correct uses of verb tenses is evaluated in
these two sections although some items in this part test only verb forms (see Table
3.1).
Part 3 and Part 4 of the pre-test are designed to check learners’ recognition of tense
meanings in discrete sentences. Students have to fully understand the meanings of
English tenses to fulfill the tasks effectively. Part 3 consists of ten sentences that ask
learners whether the paraphrase sentences are true or false. In Part 4, there are two
options for each of ten sentences, and test takers have to choose one. The two parts
examine learners’ understanding the meanings of English tenses. They are selected from
exercises in Grammar Express – For Self-Study and Classroom Use (Fuchs & Bonner,
2002).
The last part is an error analysis exercise – a kind of judgment task as a means of
measuring learners’ internalized knowledge (Coppieters 1987, Masny & d’Anglejan
1985, Schachter 1989 cited in Renou, 2001). In this part, test takers are asked to identify
what is wrong with the use of verb tenses in ten discrete sentences. The sentences are
selected for two books: Mosaic One: A Content-Based Grammar (Werner, 1996) and
TOEFL Grammar Workbook (Lim, et al., 1992). This is an open-ended task with no
indication of which words or phrases are potentially incorrect. Then they have to find
and correct typical mistakes that students make when they use English tenses. Not only
test takers’ ability of recognition and judgment of the right structures but also their
capacity of reproduction are required in this task. Thus, students need to use their
grammatical knowledge to analyze and then apply in the task. However, some sentences
just aim to learners’ attention to verb forms.
Table 3.1: Grammatical Dimensions Tested in the Pre-Test
Grammatical Dimensions
Items in the Pre-test
Form
1, 6, 11, 16, 21, 25, 61, 63, 66, 67, 69
Meaning
41 – 60
Use
Remaining items
Scoring Tests: The perfect score on the tests is 80 points. For the recognition production
task (Parts 1-4), one point is awarded to each correct answer, and no point for incorrect
answers, for a total of 60 points. For the judgment task, the maximum score for each
sentence is two points. One point is given for correctly identifying the error. Two points
are given for identifying and correcting the error. This part has 20 points in total.
Scorer reliability is built into the test by having responses scored objectively
with scoring key. The answer key in parts 1-4 is carefully checked, so test takers
have only one choice, and there is high reliability. In part 5, if test takers
recognize the errors correctly and produce grammatical correct responses that are
acceptable and are not included in the scoring key, the key might need to be
revised after trailing.
Piloting Tests: After designing the tests, we have to “test the test” (Harmer, 1987,
p. 59). The test draft was shown to two English teachers at CFL. They carefully
read and gave suggestions for improvements. Then the revised draft was piloted
in a class at CFL of over thirty students with the same level. After that, some
students taking the piloting test were interviewed to identify any unclear or
ambiguous instructions. The test length, allotted time (60 minutes), and test
instructions were appropriate and clear for all students. Finally, the results from
the try-out in class were analyzed. The results were good. There was no problem
of the input. Shortly, the test procedure and the test content seem appropriate to
the purpose of the present study: measuring learners’ acquisition of English
tenses. We have the authentic tests for the study (see Appendix 2).
3.4.2. Questionnaire on Motivation
As mentioned in the part Implicit Grammar Instruction and Learners’ Motivation (part
2.5 of this thesis), Watanabe’s motivation questionnaire focusing on value and
expectancy components is adapted to use in the current study. However, Watanabe’s
questionnaire aims to general language learning, so it was modified to focus on only
grammar learning motivation. The adapted questionnaire is composed of three factors:
value components (intrinsic motivation and task value), motivational strength, and
expectancy components (expectancy and language aptitude).
A draft of adapted questionnaire was designed with 20 closed-item questions in five
rating points (strongly disagree, disagree, neutral, agree, and strongly agree). One of the
reasons for the choice of this kind of instrument is that the instrument may offer a high
reliability and it may be easy to be quantified and analyzed and cost the least time to
collect and analyze (Mackey & Gass, 2005). This draft has been piloted in two classes of
57 students with the same setting and background. The Cronbach’s Alpha of the pilot is
.59. Five questions have been deleted to improve the total reliability of the inventory
slightly to α = .74. With the alpha, there is no need to reconstruct the items in the
inventory. The questionnaire used for the authentic study has 15 questions relating to
motivation in learning grammar, especially with IGI promoting discovery-learning.
Table 3.2: Items in authentic questionnaire on motivation in learning grammar
through IGI
Clusters
Value components
Sub-clusters
Items
Intrinsic motivation
1, 7, 11 (R), 4
Task value
8, 2, 5, 9, 15
Motivational strength
Expectancy components
14 (R), 12
Expectancy
3, 10, 6 (R)
Language aptitude
13
For the detailed questionnaire and description of the adapted questionnaire, see Appendix
3.
3.5. Procedures
This section presents three procedures of this empirical research: treatment instruction
procedure, test administration procedure, and questionnaire administration procedure.
Treatment Instruction Procedure: The study is conducted during regular class time. The
students are not informed about the research. All students receive an equal amount of IGI
on English verb tenses from the same instructor, who is also the researcher.
The Level-Seven Grammar Course has nine class meetings. Each meeting per week
lasts for 135 minutes. The pre-test, seven topics about English tenses and three review
exercises are discussed during eight weeks. The post-test, the questionnaire, and some
remarks and feedback of the course occur in the last meeting occasion. Moreover, three
review exercises are set as homework; first because the time in class is limited and
because students need time to absorb the knowledge they recieve. In class, students have
to check answers together and exchange opinions. The teacher only takes part in the
discussion whenever the students ask for an explanation or when they cannot come to a
final conclusion.
Test Administration Procedure: The participants complete a pretest at the beginning of
the course. The teacher gives some explanation about the purpose of the test as a
diagnostic test before the course and does not say about the purpose of the research.
Post-testing occurs after the instructional treatments. The two tests take place during the
timetabled class hours. During the testing time, students do not permit to use
dictionaries, exchange information to each other, or ask for help from anyone. All the
tests are collected and kept well. The researcher is the first rater, and another teacher is
asked to check for the second time to avoid any missed answer. The pre-test do not give
back to the students until the end of the course. The photocopy of the two tests is given
to the students at the tenth week – an extra meeting of the course. Students might check
their scores and give any feedback.
Questionnaire Administration Procedure: The questionnaire is administered to students
for about 20 minutes after the post-test. This questionnaire is anonymous, but the
students’ code is written on the top of the page before it is delivered to them. Students
are first given a brief verbal explanation of the purpose of the survey, followed by
specific clarification of the each question. All of the completed questionnaires are
collected immediately at the end of the class. The students fill out the questionnaire
without a time limit. None of the students denies participating. The responses of the
questionnaire keep confidential. All the items are checked carefully for the missing
items. Therefore, there is no missing item in the questionnaire data.