English Proficiency of Undergraduate
Engineering Students:
Assessment and Influencing Factors
2011 ThaiPOD Annual Conference
28-29 July 2011, Bangkok
Kuntinee Maneeratana, Ratchatin Chanchareon
Nopdanai Ajavakom, Naebboon Hoonchareon
Saowaluk Rungsup and Angkee Sripakagorn
Department of Mechanical Engineering
• 3 B.Eng., 1 M.Eng. & 1 D.Eng. Programs
Bachelor Degrees
• B.Eng. in Mechanical Engineering (since 1933)
– Admit 75 Students annually
• B.Eng. in Automotive Engineering (since 1994)
– Admit 15 Students annually
• B.Eng. in Naval Architecture and Marine Engineering (since 1990)
– Admit 10 Students annually
Previous Work on English Language Skills:
ThaiPOD2010
Importance
 Required outcome
 Crucial for employability
Exposure
 Two 3-credit English courses in the first year
 English textbooks, exercises, exams and other media
Assessment
 Two 3-credit English courses in the first year
 CU-TEP in the fourth years
 (CATME teamwork assessment in the senior project)
“Indicators of English Proficiency for Undergraduate Engineering Students”
K. Maneeratana and A. Sripakagorn
Previous Work on English Language Skills:
ThaiPOD2010
CU-TEP
 The Test of English Proficiency of Chulalongkorn University
 measures the ability to use English for academic purposes
 measures in listening, reading and writing.
 offers TOEFL-equated score.
 offered, free-of-charge, to all 1st and 4th year students.
“Indicators of English Proficiency for Undergraduate Engineering Students”
K. Maneeratana and A. Sripakagorn
Previous Work on English Language Skills:
ThaiPOD2010
Conclusions
 No significant differences between the three programs.





English grades do not significantly affect grades in other
courses.
Medium to high values of correlation coefficient between
first-year English grades and fourth-year CU-TEP results.
CATME: lacks of required skill, confidence or both in reading
complicate texts in general contexts.
The class averaged TOEFL-equated score is 500.
Measures to improve English skills should be investigated
and implemented.
Research Problem
To explore the English proficiency of students entering and leaving
the three bachelor programs
...as well as ...
motivation and practice of English in-between.
2 English
courses
Motivation & Practices
program
4th yr
3rd yr
1st yr
2nd yr
ME
Entering
CU-TEP
Leaving
CU-TEP
Previous work
Present work
Population & Procedure

Class of 2006
◦ Studied in ThaiPOD 2010
◦ inquired about motivation and practices by a questionnaire
(email).

Class of 2007: encouraged in the senior project course to
submit the following information:
◦ CU-TEP in the first year
◦ CU-TEP or other standard tests in the fourth year
◦ Questionnaire on motivation and practices
No. of 2007 Students
No. of 2006 Students
English Grades
40
30
Mean 2.92
SD 0.66
Mean
2.87
SD
0.50
Mean 3.15
SD 0.66
Mean 2.86
SD 0.62
20
10
Foundation
English (FE)
I & II
0
40
30
20
Experiential
English (EE)
I & II
10
0
1.0 1.5 2.0 2.5 3.0 3.5 4.0 1.0 1.5 2.0 2.5 3.0 3.5 4.0
English I Grades
English II Grades
English Grades

Change from Foundation English (FE) courses to Experiential
English (EE) courses between the classes of 2006 and 2007
students

Flatter grade distribution for the first courses

The Pearson correlation coefficients r between x and y,
r ( x, y ) 

 ( x  x )( y  y )
( x  x ) (y  y )
A slightly higher
as compared to
2
2
, 1 r  1
r = 0.664 between FE I & II
r = 0.575 between EE I & II
English Grades

Grades should be a very good indicator of the proficiency.
BUT


Specified outcomes, assessment methods, and grading
standards are not readily available.
Very rough information
◦ Grades only, no raw scores
◦ No details on different skills
English grades:= Indicator of overall level of
proficiency for entering students only
correlation r = 0.782
650
600
550
500
avg 4th yr
4th Year TOEFL-Equated Score
CU-TEP: From 1st to 4th Year
No significant
& systematic
improvements
during the three years !
450
400
avg 1st yr
400
450
500
550
600
650
1st Year TOEFL-Equated Score
30
60
25
50
20
40
15
30
10
20
Listening
r = 0.726
5
5
10
15
20
25
1st Year Results
25
20
15
10
10
5
0
0
0
0
30
Reading
r = 0.670
4th Year Results
4th Year Results
CU-TEP: From 1st to 4th Year - by Skills
30
0
10
20
30
40
50
1st Year Results
60
Writing
r = 0.442
0
5
10
15
20
25
30
1st Year Results
In the proficiency monitoring,
individual skills must be considered separately.
• 4th year
– 62 Students
– Average 487
4th yr inc. other tests
– 87 Students
– Average 499
US University
requirement
5
0
No. of 4th Year Students
• 1st year
– 62 Students
– Average 477
No. of 1st Year Students
CU-TEP: career requirement
2010 Graduates
CU-TEP
TOEIC
IELTS
5
0
400
450
500
550
TOEFL-Equated Score
600
650
Situation
improvement is needed.
 without tangible incentives from the curriculum?
 rely motivation and (to be recommended) practices.

2 English
courses
4th yr
3rd yr
1st yr
2nd yr
Motivation & Practices
Entering
CU-TEP
Leaving
CU-TEP
Previous work
Present work
Motivation & Practice Questionnaire
• Population
– 2009 Graduates: After graduation, 27 answers
– 2010 Graduates: Before graduation, 80 answers
• Questions
– What are the main activities that affect your CU-TEP
results?
– What are the main motivations for improvement?
• Method
– Find frequent answers from the high, average and low CUTEP groups (25, 50 and 25% classification).
– Identify the effective practices for improvements.
Motivation
• Top motivations
– Graduate studies
– Employment in multi national company in Thailand
– Entertainment
– Employment abroad
• No appreciable differences between groups
Practice of English
• Top practices
– Tuition in English institutes
– Movies/Entertainment
(Free screening available)
– Textbooks
(most important curriculum-related)
– Self-Study
(online tuition program)
– Internet Media
• Tuitions and entertainment are popular in all groups.
• Low-score groups favor textbooks more than others.
• High-score groups study on their own more than others.
Third Languages

Popular third languages
◦ Japanese, Chinese, German for employment and study in
70-90% of students

Actual proficiency (very loosely defined)
◦ 20-35% have some basic knowledge
◦ No functional proficiency, except for daily uses at home

A nice edges for students with extra drives.
In general, seem not to put real efforts into the skills
The department should concentrate on English only.


How To Motivate Student:
Attention & Relevance






Know Yourself: 2nd yr students in particular (Orientation)
CU-TEP Results announcements to both students and
lecturers.
Specific Requirements for graduate studies and employments
Successful strategy and practices, i.e. Petroleum Engineering
program
Opportunities in the Faculty, i.e. free movie screening, free
online course & certificate.
Know Your ‘Enemy’
APEC Engineers
ASEAN Co-Operation
How To Motivate Student:
Confidence & Satisfaction


Know Yourself:
Opportunities in the Faculty, e.g. attend seminar/lecture by
English-speaking guests.
Join international club/activities (Inter. Program)
Conclusions

CU-TEP result provides the most convenient indicator for
the department-level monitoring and assessment.

At present, English proficiency level must be improved.

Students must be motivated to improved their skills by
continuous exposure and utilization of English.

Awareness (Attention+Relevance) is the first-line action.

Improvements required extra and concerted efforts from the
Department.
Acknowledgement
This work is supported by
• 2103499 Mechanical Engineering Project,
• Department of Mechanical Engineering
• Knowledge Management (KM) Activities of the Faculty of
Engineering, Chulalongkorn University.
Q &A
Kuntinee Maneeratana, Ratchatin Chanchareon,
Nopdanai Ajavakom, Naebboon Hoonchareon*,
Saowaluk Rungsup and Angkee Sripakagorn
Department of Mechanical Engineering,
*Department of Electrical Engineering,
Chulalongkorn University
Bangkok, Thailand
kuntinee.m@chula.ac.th
angkee.s@gmail.com