Danah Henriksen
CEP 900 – October 20, 2003
Article Critique Japanese and American Students Differential Recognition Memory
for Teachers’ Statements During a Mathematics Lesson
A. Theoretical Perspective
The article “Japanese and American Students Differential Recognition Memory
for Teachers’ Statements During a Mathematics Lesson”, by Yoshida,
Fernandez, and Stigler (1993), studied Japanese and American students
recognition memory for statements made during a videotaped lesson. The
purpose in examining these differences was an effort to get at some of the
reasons behind, and effects of, the way students notice relevant and irrelevant
statements made by the teacher. Differences in the structure and design of
mathematics lessons between Japanese and American instruction, was
deemed to be a significant factor. The authors note that Japanese lessons are
characterized by a more coherent and organized flow with a single and
significant purpose, placing them in contrast to American lessons, which are
less cohesive and more jumbled by topic shifts and irrelevant interruptions.
With consideration of these ideas, the authors sought to investigate the
processes by which Japanese and American students learn from instruction,
and how lesson design and organization impacts students’ thinking and
learning in different ways.
Speaking in very general terms, I would say that the authors have constructed
their theory and research under a cognitive type perspective. Given the focus
on learning processes and mental representations, as well as the way that
lesson structure impacts thinking and knowing within an individual’s mind, the
article’s basis falls loosely within the domain of a cognitive framework. The
cognitive perspective however, is a general and broad context in which a more
specific theory is positioned. More specifically, Yoshida et al. are working
within a conceptual framework that is particular to their research issues. The
theoretical framework for their study posits a two-stage process of learning
from instruction in the classroom. These two stages assume that 1) The first
thing that students learning via a lesson must do is to construct a mental
representation of the events in that lesson. 2) Then, with this representation
serving as the basis for learning, the students construct their own knowledge
(in a practical and conceptual sense) of what the teacher expected to put
across. These two steps in learning are central to the idea and function of this
study in light of the fact that several factors are mentioned as affecting
childrens’ initial representation (Yoshida et al., 1993). The characteristics of a
lesson are one of these (i.e. is it coherent and purposeful?), in addition to
student’s prior knowledge of the subject matter (i.e. do they already possess
strong math skills?), and the expectations that students bring about lessons
(i.e. what do they expect in a lesson and how should they learn and interact
with this?).
This framework seemed to work relatively well for the purpose of allowing the
study to compare the ways that Japanese and American students learn
mathematics. Since the study focuses on how children are impacted by the
characteristics and design of a lesson, it makes sense to assume that they
form different representations based on instruction and thus to study the way
that different types of lessons are constructed to form knowledge and
understanding.
This framework also implies the need for the study and the importance of its
purpose in exploring the process of students’ learning via a lesson.
Mathematics is an important subject, and insight into the way students think
and learn, contingent on lesson coherence, could potentially offer valuable
information. An understanding of how lesson structure impacts mental
representations seems like an important target of study, given the possible
benefit of gaining prescriptive evidence and implications for future math lesson
design. Despite the value in the aims of this study and the possible significant
benefits of properly addressing its research issues, there are quite a few
problematic areas in this research, which lead to the fact that the findings may
be questionable or less significant than their objective initially implied.
As mentioned previously, the conceptual framework seems well formulated
with respect to investigating the issues at stake, yet there is a significant point
of contention with regard to lack of connection to relevant theory and prior
research. The authors put forward their conceptual framework clearly, yet I
feel they failed to support it with discussion of existing theory or prior research
citations. Without these points they leave the reader with no strong basis for
the credibility of their theory, putting their work on less reliable ground. I for
instance, am not reading this article with a strong background in issues of
mathematics learning, and it is difficult to freely accept their theory without
any explanation of what their idea of the learning process was derived from,
whether from long-standing existing theory or recent research developments.
In their most initial discussion of background, Yoshida et al. make some cited
references to support some of their points of difference in Japanese and
American students and lessons, and this is helpful in supporting the
established patterns of difference in lesson styles and signifying possible
impact in student learning. However, after this point in the article, the
authors fail to note or discuss any other theory that might explain their own
framework or hypothesis. Granted, it is acknowledged that the current study
is an early effort in examining the processes in Japanese and American
learning from classroom instruction (Yoshida et al., pg. 610, 1993). However,
one must still assume that other existing concepts or research lead them to
this point, and it would have been helpful to include greater explanation of
these, or citations for the reader. Therefore, it did seem that although ties to
existing research were initially and briefly touched on, there was not enough
depth on this point, which had a weakening effect upon their conjectures.
In terms of clarity and appropriateness of research questions and hypothesis I
do believe that Yoshida et al. are within bounds of suitability. They seek to
understand differences in students’ mathematical thinking and learning based
on cultural differences in lesson design and coherence. Given the fact that
they did discuss evidence to the fact that Japanese and American math lesson
are inherently different in their organizational features, it seems reasonable
and appropriate to study these differences and consider how they account for
learning and performance differences in math. Clarity is a factor only in the
sense that their hypothesis is rather broad and not a particularly definitive
statement. They seek to “pull apart and document some of these
hypothesized relationships”, which is a fairly wide-ranging statement of
hypotheses (Yoshida et al., 1993). Their research issues are complex making
it more difficult to clearly outline their theory in a specific statement, but I felt
their argument would be more powerful if clearly and succinctly stated before
entering a detailed discussion of framework.
B. Research Design and Analysis
In order to investigate their hypotheses, Yoshida et al. used a recognition
memory task to compare how American and Japanese students represented a
single videotaped lesson differently. The lesson was designed in the more
“coherent” Japanese format, and the videotape shown was of an actual
Japanese lesson, constructed in a comparable English version. With respect to
this aspect of design, I have some issues of doubt. The fact that the lesson
shown was originally a Japanese lesson is definitely a possible confounding
factor, in that Japanese students are likely to be familiar with the lesson
structure and more naturally inclined to do well with this type of lesson. If a
traditional American lesson format were given, it is possible that the findings
would go in the opposite direction. This familiarity variable is significant
enough to potentially skew the results, yet the authors’ mention it only briefly
initially, enough to comment on the fact that it may be a problem. They do
come back to it in the final discussion, but only enough to say that familiarity
with the lesson structure is a possible explanation for their results, but that
another study would be needed to determine this. Although I think it is
helpful that the authors’ acknowledge this problem, it did seem like a rather
offhand dismissal of a critical point of difficulty, and one that may likely impact
results.
Another source of concern is in the characteristics of the actual lesson shown
and the way it was presented. The lesson was presented on videotape, which
is not at all similar to or representative of the way that students are used to
learning in a classroom. Students were told to pretend that they were actually
part of the lesson and to behave exactly as the teacher in the tape instructed.
While I appreciate that Yoshida et al. described this process and the perceived
reactions and interest of the student subjects, I cannot help but feel that this
is a very artificial manner of instruction to students, and it may be difficult to
apply this atypical learning to a description of classrooms elsewhere (in either
the US or Japan). I understand that the use of videotapes in this instance did
make it easier to control for differences in the lesson presented to two cultural
groups being studied. However, I believe there is incongruence between the
study design and actual student expectations of a typical math lesson, which is
a concern regarding the validity and applicability of the study.
In addition to the mode of presentation via video, it should be noted that the
lesson given was also quite short. Though originally planned as a 45-minute
session, it was shortened to 15 minutes to make it more suitable for research
purposes. In light of this unusually short length for a mathematics lesson,
there arises the question of how applicable these results are in generalizing
them to a standard classroom example, which is certainly longer than 15
minutes.
Though there is some potential room for debate on questions of study design
and so forth, I strongly feel that the sampling methods used were not
adequate and did not lend themselves to generalizability. The final sample
selected, after pretests had screen out the highest “ceiling” achievers in all
groups, included 18 Japanese fourth graders, 26 American fourth graders, and
16 American sixth graders (who were included to balance out the prior
knowledge of the group, as they were closer in math knowledge and skills to
the Japanese fourth graders).
The sample size in and of itself was therefore quite small; with between 16-26
participants in each group it is not likely that effects in this small sample can
be considered applicable to a greater population. Perhaps more problematic
are the characteristics of the participant groups themselves. All were drawn
from one region, near the Chicago area, with the American students coming
from self-contained classrooms at the University of Chicago Laboratory School,
and the Japanese students coming from a Day School in a Chicago suburb.
There is a strong likelihood that students at a University laboratory school will
not be comparable to typical American students elsewhere around the county.
It is also likely that Japanese students who are living and studying in United
States may not be representative of students living and learning in Japan.
The authors are seeking to study the effect of a Japanese lesson as taught
within that cultural framework, yet the sample population they have selected
is not entirely representative of students living or being schooled in Japan. In
this way, I see some problems with the generalizability of the sample. It does
not appear to be either large enough in size, or representative enough of the
target population, to meet the claims of the authors.
Despite some of these inherent problems with the research design, there were
several counts on which Yoshida et al. were careful and precise, concerning
procedure and materials. In the construction of the videotape itself, they took
great caution to ensure that very similar lessons were provided both to
Japanese and American students. The tape was translated by a native English
speaker with fluency in Japanese, and to doubly ensure accuracy it was
reexamined by a native Japanese speaker who was fluent in English. They
retained the exact content with no changes to correct ambiguity. To reshoot
scenes in which Japanese was written on the blackboard, an actor was
substituted, with attention to matching details such as clothing, camera angle,
timing actions and length of original video. Yoshida et al. do deserve credit for
taking such exacting measures to replicate instructional materials between
study groups. However, such cautious detail for similarity on this aspect leads
me to wonder why they would ever use instructional video in the first place, as
it is really does not bear much similarity to authentic classroom instruction.
On this point, another possible issue arises with materials, by the fact that the
tape was dubbed into English. Though the translation was completely
accurate, it is possible that the dubbed quality may have even further added
to the unrealistic quality of this lesson.
On other notes of measures and procedures, it seemed that the pretest and
posttest were relatively sound methods, relating to Yoshida et al.’s purpose.
They needed to control for possible interference of prior knowledge and their
pretest seemed a good way assess and equalize the group. Though only one
question was given in this prior assessment, this was actually a good match up
to the rest of their instruction with its focus on coherence and a single
instructional goal. The design of the posttest was quite similar, with subtle
shifts in language, and one new question added which required students to
apply any new learning and helped the authors’ to measure this.
I was a bit more unsure or skeptical of the other measure used, which was a
recognition memory test. This is not so much a criticism of their actual
measure, which quantified the number of relevant and irrelevant statements
noticed by Japanese vs. American students, and appeared to be a sound
enough measure of what students were actually noticing and recalling. My
uncertainty is more on the point of operationalizing lesson coherence, or
comprehension and mental representations, as statement recognition. For
one thing it is difficult to delineate what is relevant in a child’s line of
reasoning, and though the focus is on the teacher’s statements, the child is
the interpreter of these.
Concerning the issue of data analyses, I believe that statistical assumptions
were fairly well met. Despite the fact that other discussed factors in the study
may lead to question the results of some analyses, it would appear that
Yoshida et al. are relatively secure on statistical ground. They used repeated
measures of variance (ANOVA) to confirm reliability of the pretest and posttest
results. With their small sample, one might expect greater difficulty in
showing statistical significance, yet they were able to clearly show a gain from
pretest to posttest of p<.0001. In anticipating questions on their scoring, the
author’s rescored the pretests and posttests for answer accuracy only,
excluding the demonstration of solution strategies, which they felt Japanese
students might benefit more from. The fact that the findings remained the
same appears to be a statistical indicator of scoring validity. Repeated
measures ANOVA was also used with statement type to analyze recognition of
teacher statements. The findings here revealed that Japanese students
showed a statistically significant difference in their ability to recognize
irrelevant and relevant statements, p<.0001. American sixth and fourth
graders however, did not show any difference in their ability to recognize such
statements, p=.82. These statistical findings are predicted in the authors’
initial assertions about cultural patterns of statement recognition, and their
statistical analysis seems to have been thorough.
C. Interpretation and Implications of Results
The authors did include some discussion of methodological and conceptual
limitations, which was helpful in considering some of the problems within their
study. Rather than specifically setting aside a space in which to discuss their
limitations or prospective problems, they were inclined to incorporate or
acknowledge these limitations in the form of potential alternative
interpretations, throughout the text of the article. Yoshida et al. tended to
anticipate certain areas of questionability and respond to those that they
could. For example, they weighed the possibility that their patterns of results
for pretest and posttest were a function of their scoring system, and countered
with a more rigorous test of results (omitting solution strategies). They were
careful to acknowledge that their results were only limited to the area of
mathematics and should not readily be thought of in terms of all school
subjects without further research. Additionally they did acknowledge the fact
that the lesson was originally written and designed in Japanese, and Japanese
students may have benefited more greatly as this fell naturally within their
schema for math learning. Although on this point, I do think that the authors
tended to shy away from confronting the significance of the flaw. Rather they
commented that they did not feel this explanation was the case, and that
more research would be needed to actually confirm anything.
There were also other areas in which they could have been more thorough in
their discussion of limitations. The sample size/characteristics and issues of
generalizability, discussed previously with respect to methodology, is quite a
significant limitation that was never discussed by Yoshida et al. Similarly, the
authors should have acknowledged the fact that the videotaped setup of
classroom instruction, with its lack of correspondence to actual classroom
settings, could have posed a generalizability problem.
Yoshida et al. did demonstrate a marked difference in the way that American
students have a greater propensity for noticing irrelevant (unrelated to
content) statements made by the teacher, whereas Japanese students focus in
on only what is relevant, and this was clearly reflected in their reported
statistical levels of significance. The explanation of why they were measuring
for these differences does explain the fact that cultural differences in lesson
design would lend themselves to such recognition patterns. Their focus is on
how students use these specific statements of relevant or irrelevant
information to form a mental representation, and how their schema for
mathematics learning impacts this. Generally speaking, I would say that what
they concluded was not inconsistent with their results as reported. However,
some of the previously noted problems with the sample and study design may
indicate the fact that some of their claims were overstated. It is difficult to
make the sweeping statement that most American students don’t discriminate
irrelevant from relevant statements, or that Japanese students have cultural
memory patterns, based on 18 Japanese students and 42 American students
all from one Chicago suburb. Additionally, the authors did not offer any
convincing evidence to explain why it is a negative thing for students to notice
more than only statements exacted to one line of reasoning. This one line of
reasoning may fit well in a Japanese culture and math class, yet may not be
feasible given the differences and variability of an American classroom.
Whether or not this is true, it remains a question left open by Yoshida et al.
Despite my disagreement with some of the authors’ claims, I would say that
they remained within their original theoretical framework in concluding their
results. Their two-step process for learning focused on the formation of
students’ mental representations, the knowledge constructed by this
information, and the impact of a coherent vs. changeable lesson plan. In their
final explanation they were careful to relate their ideas and analysis back to
the original theoretical base. They state that the reported patterns of
recognition result from differences in students’ schemata for learning math.
Because Japanese students’ routinely receive coherent instruction, the authors
feel that they are more capable of identifying the salient points of a lesson,
and getting to the heart of what the teacher intended to convey.
In my view, some of the inherent design problems in the way this study was
enacted make it difficult to extrapolate much significance to a greater
population. That being said however, I do feel that the authors’ have an
interesting set of ideas and line of reasoning that could potentially be built on
through further research and better methods.
The aims of the study have
value in the sense of investigating the process of mathematics learning from
classroom lessons. The authors are trying to get at issues of how lesson
characteristics impact the way students think and learn. Developing
awareness of a beneficial type of lesson structure for American classrooms
seems like a valid goal, and perhaps with a more realistic research design,
there could be potential descriptive evidence of how mental representations
are formed in the classroom, or possibly prescriptive evidence in implications
for future math lesson design. As it stands however, I would not feel
confident in trying to generalize this study on a broader scale.
Yoshida, M., Fernandez, C., & Stigler, J. W. (1993). Japanese and american
students' differential recognition memory for teachers' statements during a
mathematics lesson. Journal of Educational Psychology, 85(4), 610-617.