Qualitative Inquiry & Digital EFL Learning
A Theory-Driven QI in Response to Recent Paradigm Shifts
in TESOL and on the Internet
Chin-chi Chao
Associate Professor
National Chengchi University
國立政治大學副教授招靜琪
Presented at the National Chung Cheng University, December 30, 2008.
1
Agenda
1.
2.
3.
4.
Paradigm shift in TESOL: more open to QI
Paradigm shift on the Internet: Web 2.0
The case of a theory-driven QI
The contributions of QI
2
1. Recent Paradigm Shift in TESOL
3
Recent Debate  Paradigm Shift






Applied Linguistics (Special issue, 1993),
Modern Language Journal (1994),
TESOL Quarterly (1997),
Paradigm Shift: Understanding and
Implementing Change in Second Language
Education (Jacobs & Farrell, 2001).
The Social Turn in Second Language
Acquisition (Block, 2003),
Cognitive and sociocultural perspectives: Two
Parallel SLA Worlds? (Zuengler & Miller, 2006)
4
Key components of the PS
Jacobs, G. M., Farrell, T. S., (2001).
•
•
•
Focusing greater attention on the role of learners rather
than the external stimuli learners are receiving from
their environment. Thus, the center of attention shifted
from the teacher to the student. This shift is generally
known as the move from teacher-centered instruction
to learner-centered or learning-centered instruction.
Focusing greater attention on the learning process
rather than on the products that learners produce. This
shift is known as a move from product-oriented
instruction to process-oriented instruction.
Focusing greater attention on the social nature of
learning rather than on students as separate,
decontextualized individuals.
5
• Focusing greater attention on diversity among
learners and viewing these differences not as
impediments to learning but as resources to be
recognized, catered to and appreciated. This shift
is known as the study of individual differences.
• Focusing greater attention on the views of those
internal to the classroom rather than solely
valuing the views of those who come from
outside to study classrooms, evaluate what goes
on there and engage in theorizing about it. This
shift led to such innovations as qualitative
research - with its valuing of the subjective and
affective, of the participants' insider views and of
the uniqueness of each context.
• Along with this emphasis on context came the
idea of connecting the school with the world
6
beyond as a means of promoting holistic learning.
• Helping students to understand the purpose of
learning and develop their own purposes.
• A whole-to-part orientation instead of a partto-whole approach. This involves such
approaches as beginning with meaningful whole
texts and then helping students understand the
various features that enable to texts to function,
e.g., the choice of words and the text's
organizational structure.
• An emphasis on the importance of meaning
rather than drills and other forms of rote learning.
• A view of learning as a lifelong process rather
than something done to prepare for an exam.
7
8 Changes as Part of the Paradigm
Shift in Second Language Education
•
•
•
•
•
•
•
•
Learner autonomy
Cooperative learning
Curricular integration
Focus on meaning
Diversity
Thinking skills
Alternative assessment
Teachers as co-learners (Jacobs & Farrell, 2001)
8
Paradigm Shift Observed by 2006
Zuengler & Miller, 2006
“Ontological” – “basic questions about the
nature of reality (Denzin & Lincoln, 1998, p.185)
1. Cognitive vs. Sociocultural Understanding
of Learning
2. Positivists vs. Relativists over how to
construct SLA theory
9
Traditional Scientific Understanding
•
•
•
•
•
establishment of laws or patterns
that exist across contexts,
as a deductive system of reasoning
that is rule-based
and thus independent of the forces
(Linda Watkins-Goffman, 2006, p.2)
10
Contrasts between positivism
and post-positivism (Jacobs & Farrell, 2001)
Positivism
Post-Positivism
Emphasis on parts and decontextualization
Emphasis on whole and contextualization
Emphasis on separation
Emphasis on integration
Emphasis on the general
Emphasis on the specific
Consideration only of objective and the
quantifiable
Consideration also of subjective and the nonquantifiable
Reliance on experts and outsider knowledge-researcher as external
Consideration also of the "average" participant and
insider knowledge--researcher as internal
Focus on control
Focus on understanding
Top-down
Bottom-up
Attempt to standardize
Appreciation of diversity
Focus on the product
Focus on the process as well
11
Sociocultural Perspectives on LL
• View language use in real-world situations
as fundamental, not ancillary, to learning.
• Language not as input, but as a resource
for participation in daily activities
• Participation as both the product and
process of learning (Zuengler & Miller, 2006)
12
The Changing Focus of Research
It is about:
• the attempt to adapt the self into a new context and
a new world
• the struggle for participation in a new social
environment
• participation as a metaphor for learning a new
language
• participation as described by the individual’s
narrative can be interpreted as a metaphor for
acquiring a new identity
NOT just
• in the usual sense where the lexical, grammatical,
and semantic systems are learned or acquired
(Watkins-Goffman, 2006,p. 1)
13
Comparison
14
4 QI
Approaches
15
12 QI
Characteristics
16
2. Paradigm Shift on the Internet
17
18
19
20
Networks of Personalized Learning
•
•
•
•
•
•
Blogs
Podcasts
MySpace
Flicker
RSS
Delicious
21
Tools for Collaborative
Language Learning
•
•
•
•
•
•
•
•
Skype
Google Talk
Chinese pod
Live Mocha
Voice Thread
Yack Pack
Dotsub
Chinswing, etc.
22
我的學習
點數
貢獻分
跟我一樣
在學這個
語言的人
23
有人看過我po
出去的功課
24
25
26
27
28
Community
Culture
Dialogue
Identity
togetherness
Discourse
29
Toward an Understanding of Computermediated EFL Writing Experience
Through Vygotiskian Perspectives
Chi-chi Chao
National Chengchi University
Chao, C. (2007). Toward an understanding of computer-mediated EFL writing
experience through Vygotskian perspectives: A monograph published by Taiwan
Journal of TESOL. Taipei, Taiwan: Crane.
30
Purposes
1.
2.
to explore how Vygotsky’s socio-culturalhistorical theory (SCT) could serve as a
useful framework to the study of EFL
learning experiences
to understand how EFL learning is
supported by automated writing evaluation
(AWE) programs: first without an inquiry
community and later with one
31
The Philosophical Underpinnings
and QI Tradition



The philosophical underpinning of the research reported
here is interpretivist in nature. The aim is not to offer
causal explanations, but to understand the experience by
way of “reconstructing the self-understandings of actors
engaged in the action” (Schwandt, 2000, p. 191).
In other words, the focus is on grasping the meanings that
constitute the action of learning to write in English in the
AWE mediated learning environment, seeking to discover
some of the essence of that experience through intensive
study of individual cases.
Taking a theory-driven approach
32
Social Cultural Historical
Theory (SCT)



Lev Semenovich Vygotsky
(1896-1934)
a Russian Jewish
developmental psychologist
and the founder of culturalhistorical psychology
his major works span 6
volumes, written over roughly
10 years Mind in society: The
development of higher
psychological processes (1978)
33
Definition of SCT
[D]espite the label “sociocultural” the
theory is not a theory of the social or of
the cultural aspects of the human
existence. …it is,… rather, a theory of
mind… that recognizes the central role
that social relationships and culturally
constructed artifacts play in organizing
uniquely human forms of thinking (p. 1).
(Lantolf, 2004, cited in Lantolf & Thorne, 2006)
34
The Social Formation of Mind

“We want to be clear on this point. The
argument is not that social activity
influences cognition, but that social
activity is the process through which
human cognition is formed” (Lantolf &
Johnson, 2007, p. 878).
35
An Important Quote
Any function in the child’s cultural development appears
twice, or on two planes. First it appears on the social
plane, and then on the psychological plane. First it
appears between people as an interpsychological
category, and then within the child as an
intrapsychological category. This is equally true with
regard to voluntary attention, logical memory, the
formation of concepts, and the development of
volition — [I]t goes without saying that internalization
transforms the process itself and changes its structure
and functions. Social relations or relations among
people genetically underlie all higher functions and their
relationships (Vygotksy, 1978, p. 57).
36
Key Concepts



Mediation
Internalization
Zone of proximal
development (ZPD)
37
Piaget: the child as lone scientist vs.
Vygotsky: the child as apprentice
Q1. Describe as thoroughly as possible the image
you have for the word “apprentice.”
Describe her work environment, tools she uses,
and the people she might work with. Compare and
contrast this image with that of a pupil in the
classroom and then that of an EFL learner on the
Internet – What situation would an EFL learner be
an apprentice?
38
窯廠學師仔
囝仔工又稱「學師仔」(學徒)，一般師仔工(學徒工)大
都負責工廠較不重要工作，或是跟在作瓷仔(陶瓷)師
傅身邊當助手，工作的同時邊學習，一些較靈巧的囝
仔工二、三年出師(學成)了，擔任師傅。而有些師傅
則向老闆請負(包工、分租)，等較有資本後自行設立
小窯廠，全家人投入陶瓷製作生產，他的小孩又成了
囝仔工，生意較好時另又招了一些囝仔工。
台灣陶瓷數位典藏計畫
http://digital.ceramics.tpc.gov.tw/Web/yingo200/story/work/work.htm
39
Development occurs when we incorporate tools for
thinking from our society.
高雄市立美術館圖片檔 (古勒勒‧達比烏蘭)
http://www.kmfa.gov.tw/Data/Image/古勒勒與他的學徒Engzngz.JPG
40
Development occurs when we
incorporate tools for thinking
from our society.

Q2. Describe an experience
(i.e., yours or your students’)
that can verify this
statement. Then, describe
an experience in learning a
foreign language which can
also respond to this
statement.
41
• Vygotsky’s emphases: First. the
symbol systems come to us from
others rather than within ourselves.
Second, the symbol systems are not
just used in our thinking but completely
reorganize our thinking.
• …It [language] is no longer speech for
communication; it is now a personal
psychological tool that changes all her
thought processes.
42
• In traditional learning theory (note: i.e.,
behaviorism) and social learning theory,
society is thought to influence and shape
the child, but in Vygotsky’s theory, the child
is a part of society and a collaborator in his
learning with adult mentors.
• For Vygotsky, we can’t function on an adult
level without the culture of which we are a
part bringing us along and providing what is
necessary. This conceptualization
acknowledges a deeper level of social
interaction than the simple social influence
and conditioning envisioned by learning
theory (note: i.e., behaviorists).
43
Influences








(just a few)
Activity theory (Leontiv, Engestrom)
Anchored instruction (Bransford et al)
Cognitive apprenticeship (Collins, Brown & Newman, 1987)
Distributed cognition (Hutchins, 1995)
Dialogic inquiry (Wells, 1999)
Dynamic assessment (Holt & Willard-Holt, 2000; Lantolf)
Situated Cognition (Brown, Collins, & Duguid, 1989; Lave
& Wenger, 1991 )
Social constructivism (bringing together the work of Piaget
with that of Bruner and Vygotsky)
44
Some of the authors






Jerome Bruner,
Andy Clark
Michael Cole,
James Lantolf
James V. Wertsch,
Gordon Wells,
45
Features of SCT-inspired research
•
•
•
•
Emphasizing the context: taking a holistic view on
the “ecology” of language learning environment and
attempting to capture “multiple dimensions of
context and multiple levels of discourse.”
The complexity of the language learning
environment is elucidated by adopting theoretical
frameworks from disciplines outside of TESOL.
Issues such as power, identity, culture and gender
are investigated. Agency is another such issue
interested to researchers. (Tsui, 2008, p. 41)
Attempting to understand how technology
transforms human action (e.g., learners’ and
teachers’) and what activities people are engaged in
with the assistance of technology in the particular
social cultural context.
46
Many studies have adopted SCT-related
theoretical framework in designing and examining
activity structures for CALL
• Self-access language learning (Hoven, 1999),
• CMC (computer-mediated communication)
(Meskill, 2005; Shin, 2006),
• Telecommunication projects (Belz, 2002; Lee,
2004),
• Project-oriented CALL (Jeon-Ellis, Debski,
Wigglesworth, 2005)
47
The Mediation/Tool: AWE

AWE (Automated Writing Evaluation)
programs is a kind of computer
software designed to evaluate
compositions, supposedly as efficiently
as human raters.
48

Previous research has mostly centered
on psychometric aspects of the
software, comparing the quality of
machine generated feedback and
evaluation with that of human raters
(Warschauer & Ware, 2006).
49


Nowadays AWEs have been advocated
as “web-based writing instructional tool”
(Vantage), expanding from the original
purpose of evaluation to that of
instruction.
There is thus a need to understand how
it supports learning in the sociocultural
context of language learning
environments.
50
ETS Criterion http://criterion1.ets.org/cwe/
51
Demo: http://www.ets.org/Media/Products/Criterion/tour2/critloader.html
52
Vantage MY Access http://www.vantagelearning.com/myaccess/
53
54
Scoring scales
used by My Access & Criterion (1)
My Access



Focus & Meaning The extent to
which the response demonstrates
understanding of the text and the
purpose of the task, and makes
connections between them
through a controlling or central
idea.
Content & Development The
extent to which ideas are
elaborated with specific, accurate,
and relevant details (facts,
examples, reasons, anecdotes,
prior knowledge).
Organization The extent to
which the response establishes
purposeful structure, direction,
and unity, including transitional
elements.
Criterion

Organization & Development






Introductory Material
Thesis Statement
Main Ideas
Supporting Ideas
Conclusion
Transitional Words and
Phrases
55
Scoring scales
used by My Access & Criterion (2)
My Access
Language & Style

The extent to which
the response
demonstrates
effective and varied
sentences and word
choice appropriate
to the intended
audience
Criterion
Style

Repetition of Words

Inappropriate Words or
Phrases

Sentences Beginning with
Coordinating Conjunctions

Too Many Short Sentences

Passive Voice

Number of words: (number)

Number of sentences:
(number)

Average number of words
per sentence: (number)
56
Scoring scales
used by My Access & Criterion (3)
My Access
Mechanics & Conventions

The extent to which the
response demonstrates
control of conventions,
including paragraphing,
grammar, usage,
punctuation, and
spelling.
Criterion
Grammar

(problems to be detected)

Fragments

Run-on sentences

Garbled Sentences

Subject-Verb Agreement
Errors

Verb-Form Errors

Ill-formed verbs

Pronoun Errors

Possessive Errors

Wrong or Missing Words

Proof read this!
57
Mechanics

Spelling

Capitalize Proper Nouns

Missing Initial Capitalized
Letter in a Sentence

Missing Question Mark

Missing Final Punctuation

Missing Apostrophe

Missing Comma

Hyper Error

Fused Words

Compound Words

Duplicates
58
Usage
 (problems to be
detected)
 Wrong Article
 Missing or Extra Article
 Confused Words
 Wrong Form of Word
 Faulty Comparisons
 Preposition Error
 Nonstandard Verb or
Word Form
59
Feedback Wording

My Access

Overall 5: On a scale of one to
six, your response to this
assignment was rated a 5.
Your response was evaluated
on the basis of how well it
communicates its message
considering important areas of
writing including focus and
meaning, content and
development, organization,
language use and style, and
conventions and mechanics.




Criterion
Score of 5: Skillful Performance
Tells a clear story that is welldeveloped and supported with
pertinent details in much of the
response.
Well organized with story
elements that are connected
across most of the response; may
have occasional lapses in
transitions.
Exhibits some variety in sentence
structure and uses good word
choice; occasionally, words may
be used inaccurately.
Errors in grammar, spelling, and
punctuation do not interfere with
understanding.
60
STUDY 1:
AWE AS THE SOLE MEDIATION
61
Research Questions



How helpful is the opportunity to use the
AWE program in a self-access mode?
How helpful is the AWE feedback system in
developing writing skills in a self-access
learning mode?
What major instructional support may be
necessary to help learners develop writing
proficiency with the AWE program?
62
The Context

In the spring semester of 2005, for
promotional purpose, the Taiwanese
representative of Criterion offered the
author’s affiliated department 265 free
accounts which allowed students
unlimited use of the AWE for a period of
twelve months.
63

Presented as paradoxes, the assertions
below are derived from the learners’
self-report in interviews and comments
in face-to-face meetings.
64
Q1: How is the opportunity to use
the AWE program helpful?

The learner liked the opportunity to
practice writing at their own pace, but
very few of them actually took
advantage of it.
65
Topics
Due Date
Number of People
who wrote this
topic
Table 4.1. Frequency of use
Required
Number of
submissions
Goals
Feb 7
164 (64.82%)
375
Technology
Feb 13
84 (33.20%)
167
Special Object
Feb 20
89 (35.18%)
206
Teaching Styles
Mar 7
63 (24.90%)
118
Guest Speaker
Mar 21
44 (17.39%)
98
Make a Change
Apr 4
26 (10.28%)
53
Self-selected topics-- 54 essays (10.30%)
31 (12.25%)
Total:
Number of people in the program: 253
People who wrote at least one topic: 179
(70.75%).
People who finished all six required topics:
13 (5.13%)
470 required essays (89.69%)
524 essays in total.
1147 submissions
Revision: 2.19 times per
topic
66
70%
60%
Percentage
50%
40%
30%
20%
10%
0%
7-Feb
13-Feb
20-Feb
7-Mar
21-Mar
6-Apr
67
Q2. How is the AWE feedback system
helpful in developing writing skills?
•
•
The feedback system is adequate in areas
which a definite comment is possible and
which are included in the criteria, but it is
inadequate in other areas which are more
complex or illusive.
Those who made the most of the program
chose to trust the feedback and used
metacognitive skills extensively, although they
were fully aware of the program’s
insufficiencies.
68
Q3 What kind of instructional solutions
may be necessary to help learners
develop writing proficiency with the
AWE program?
• Learners liked to develop their writing in
private, but they also wanted a sense of
group.
69
DESIGNING A COURSE WITH SCT
70
Three Teaching Approaches
to EFL Writing
1.
2.
3.
The text-oriented approach to writing “focuses on
the products of writing by examining texts in various
ways, either through their formal surface elements
or their discourse structures” AWE
The writer-oriented approach focuses on the writer
and describes writing in terms of the processes used
to create texts, including the views that writing as
personal expression, writing as a cognitive process,
and writing as a situated act.
The reader-oriented approach emphasizes the role
that readers play in writing, including such views as
writing as social interaction, writing as social
construction, and writing as power and ideology.
71
The SCT Touch in the Course
• lies in its emphasis of writers as unique
contributors and creators of world
knowledge, who can best be supported by a
learning community that encourages
constant dialogic interaction through which
participants and the instructor engage
collaboratively in the inquiry into EFL writing
and reflectively in the actual practice of
writing.
72




The design of the course follows
Gabrielatos’s writing skills development
cycle, including -awareness-raising,
support,
practice, and
feedback as the key activities
73
STUDY 2:
MEDIATED LEARNING WITH AND
AROUND AWE
74
The Context






A part-time English writing course which featured using My
Access as a tool for self-regulated learning.
Nine people enrolled in the writing course
They were teachers, government officials, students, college
professors, or bank clerks who were interested in advancing
their English writing skills.
All of them had college degrees, with five (5) of them having an
advanced graduate degree, including one PhD in architecture
and one MFA in fine art.
Their writing proficiency levels, based on their self-report on the
first day of the class, ranged from low intermediate to high
intermediate.
As this is a smaller class than the previous year and students
have higher education levels, it was expected that these
students would be able to use more metacogintive strategies
and that it would be more likely for the learners to have intense
interaction as a class.
75
Research Questions
1.
2.
3.
How do two learners regulate their learning
through the mediation system formed by the
AWE program, the teacher, and fellow
students?
How do they regulate their learning with the
mediation of the AWE program?
How do they regulate their learning with the
help of the teacher and peers in the class?
What meaningfulness do they derive from
their experiences in this writing course?
76
Key Informants: Anne & Grace



Two reasons to choose these two learners as the
cases focused in this study -(1) they were among the most motivated in the class.
They were comfortable about articulating their
thoughts in class and in face-to-face conferences,
which allowed me many opportunities throughout the
course to understand how they regulated their
learning in this context.
(2) they happily accepted my invitation the moment I
asked. The rapport existing between each of them
and me was expected to lubricate the data collection
process. Below is the two learners’ background
information.
77
Anne




In her mid fifties. Married with two children.
Job: In the nursing profession. She had two
years’ experience studying for a master’s
degree in the U.S. about fifteen years ago.
Goal: Passing GEPT
Focus: Text-based writing techniques
78
Grace





In her early forties, married with a teen daughter.
Job: a computer programmer in a well-established
bank for fifteen years.
Goal: Purely for interest
Focus: Developing content.
Comparatively speaking, she writes better than
Anne and many other members in this class.
79
Data Sources


Interviews, class field notes, learner’s
written works, and records of conference
with the teacher.
Data taken directly from the AWE system:
number of prompts used, number of scored
essays, number of pending essays, scores
for the first try and the best try, average
scores, and the date when the participant
input the system for the last time.
80
Data Analysis




Construct profiles for the two learners using
and triangulating various data sets
Categorization based on critical incidents
Interpretation
Member checking
81
Critical Incidents
Q1 How do they regulate their learning with the
mediation of the AWE program?


Using AWE functions and feedback
Analyzing techniques in model essays
Q2 How do they regulate their learning with the help of
the teacher and peers in the class?

Adopting external resources to enrich writing
Q3 What meaningfulness do they derive from their
experiences in this writing course?

Deriving insights from interacting with the instructor: class
activities and conferencing
82
What is learned?
83
Activity Theory
84
The Subject, Object, & Outcome

The learner’s goal for learning how to
write played an important role in
determining how they interacted with
the tool.
85
The Mediation
Second, AWE is not a neutral tool: It
serves a specific type of learner, who
is perhaps more like Anne than Grace
in terms of goals and proficiency levels.
86
The Community
Not everybody learned the same
thing with the same software.
87
Four levels of contradictions in a
network of human activity systems
88
Theoretical Implication
1. The concept of mediation in light of the
two studies
•
The AWE tool
89
Theoretical Implication
2. AWE as mediation in the intrapsychological
categories
90
Pedagogical Implications


Design of AWE: Supporting thoughtful engagement,
in the process of ‘learning by doing’ and ‘learning by
reflection’
John Seely Brown (2000) argues that the point is
designing tools to support functions that are most
comfortable and natural to human learning in
everyday environment. He says, “Our challenge and
opportunity, then, is to foster an entrepreneurial spirit
toward creating new learning environments-- a spirit
that will use the unique capabilities of the Web to
leverage the natural ways that humans learn”
(p. 13).
91
Pedagogical Implications



Design for social interaction
It would be necessary to explain to
learners the importance of interaction
so that they do not think of reflective
conversations as a waste of time.
More research is necessary to
investigate learner difference in
response to the emphasis of interaction
and reflection in language learning
environments.
92
The Contributions of QI
93
The schism between research and
practice is a major challenge facing
applied researchers.
QI holds potential to
 allow documentation of the challenges encountered
in implementing interventions designed to change
or reform existing practice
 pay attention to cultural and contextual factors
 acquire the benefit of formative research, attending
to the specific needs and resources of the target
population
( Nastasi & Schensul, 2005, pp.186-187)
94
QI


[is] critical for documenting the adaptations necessary for
application of interventions to real-life contexts, and for
identifying core intervention components which are related
to desired outcomes.
can help researchers to describe various manifestations of
intended outcomes that may not be reflected in
standardized instruments, and to identify unintended
positive or negative outcomes for the individual and
institution/community.
( Nastasi & Schensul, 2005, pp.186-187).
95
The researcher as a learner



A key contribution of qualitative research is the
development of theories and concepts that can aid
an understanding of education…
Within a qualitative worldview all knowledge is
partial, situated, and contextual.
Our own learning as researchers and practitioners
never ends; the more we know the more there is
to know (Giangreco & Taylor, 2003, 135-136)
96
97