A Novel Technology to Investigate Students’ Understandings
A Novel Technology to Investigate
Students’ Understandings of Enzyme
Representations
By Kimberly J. Linenberger and Stacey Lowery Bretz
Digital pen-and-paper technology,
although marketed commercially as
a bridge between old and new notetaking capabilities, synchronizes
the collection of both written
and audio data. This manuscript
describes how this technology was
used to improve data collection
in research regarding students’
learning, specifically their
understanding of enzyme-substrate
interactions as depicted in textbook
representations. Students were
provided this technology during
individual interviews and were
permitted to annotate multiple
representations of enzymes and
substrates, as well as to generate
their own representations. The
ability to digitally revisit the
sequential student drawings was
valuable in analysis of the research
findings. Innovative and novel uses
for this technology are discussed
for both discipline-based education
research and classroom practice.
O
ften the tools that are
used to collect data are
buried deep within the
methodology of a research article, if mentioned at all.
We recently began to use a new
commercial device that could revolutionize data collection for semistructured interviews. The purpose
of this manuscript is to share our
findings regarding this research
tool with the community of science
education researchers and science
teachers.
We are currently investigating
biochemistry students’ understanding of enzyme-substrate interactions (Linenberger & Bretz, 2012).
Because no assessment existed
to specifically measure students’
knowledge of these interactions, we
developed a concept inventory to
do just that (Bretz & Linenberger,
2012). The first step in developing this concept inventory was to
interview students using a semistructured interview guide (Bowen,
1994; Bretz, 2008). Part of this
interview asked students to discuss
two representations, each depicting a
different aspect of enzyme-substrate
interactions. One of the representations used in the study is found
in Figure 1. While analyzing data
collected from these semistructured
interviews, we encountered several
challenges, specifically with regard
to interpreting drawings made by the
students during the interviews. This
paper describes these challenges,
how the new technology resolved
these difficulties, and the applicability of this technology to other science, technology, engineering, and
mathematics (STEM) disciplines.
Methodological challenges
Interviews were both audio and
video recorded. Students were permitted and encouraged to draw on
the provided representations. Blank
sheets of paper were also available, if needed, to further explain
their ideas. Any such drawings or
notes were collected and scanned
as additional pieces of data. These
rather traditional data collection
techniques, however, created unanticipated challenges in the data
analysis.
The pencil and pen markings of
students who drew or made notes
were not always visible after scanning the images. We substituted a
fine-point marker to ensure that the
image was readable and provided
multiple colored markers that could
be distinguished from both one
another and the paper. However,
difficulties still remained. Students
were repeatedly interacting with
the representations during a single
interview. Consider the copy of the
representation (Figure 2) collected
after an interview with Jason, a junior biochemistry student. Discerning post hoc the meaning of each
marking to Jason in real time during
the interview was difficult because
he continued to draw over his own
markings. We subsequently modified
our protocol to use separate sheets of
Vol. 42, No. 1, 2012
45
paper for each phase of the interview,
thereby minimizing the problem of
multiple markings.
Although videotape captured
student markings in real time, it was
still exceedingly difficult to correlate
markings on the representations with
time stamps on a transcript of the
interview. Students moved their head
or hands, sometimes obscuring the
view of the camera. The transcripts
of Jason’s interview offer an example
of this difficulty as he explained what
he found confusing in Figure 1. It is
almost impossible to discern post
hoc which marking from Figure 2
corresponds with which pronoun or
phrase:
Interviewer: Ok. Alright, so what
about the image do you not understand or find confusing?
Jason: That, that, um, where
this is pointing to. I don’t know
what is going on in here. I don’t
know. Um, this is a little weird
how—’cause usually the enzyme
doesn’t change, well I mean it
could change conformation, but
these are not the actual active site
part of the enzyme. Like, these
two are separated. I don’t know
what’s going on in here. It just
interacts with the substrate and
not with itself. I mean it could,
it does interact with itself, but it
shouldn’t change its conformation or like eventually it’s going
to have to go back to its original
conformation. I’m just wondering. I mean, I don’t know, this is
just a little weird.
To remedy any potential misinterpretation of data, we revised our
protocol to incorporate digital pen
and paper technology (DPPT; Anoto
Group AB, 2011) simultaneously.
Although a pen tablet or graphical
tablet interface could be used as
well, we were eager to avoid any
additional “learning curve” while
exploring student understanding of
46
Journal of College Science Teaching
the representations, as it has been
shown that the farther the interface
is from a traditional paper-and-pencil
format, the larger the cognitive load
for the student (Oviatt, Arthur, &
Cohen, 2006). For this study, the
Livescribe Pulse smartpen (http://
www.livescribe.com) was selected
for its ease of use, cost-effectiveness,
and availability. The remainder of
this article describes our novel use of
DPPT (beyond its intended commercial applications) and its potential
for revolutionizing interview data
collection in the context of investigating students’ understanding of
enzyme-substrate interaction.
Technological solution
Used in the same manner as an ordinary ballpoint pen, the digital pen
is a computer that literally fits between the fingers of a student. A microphone embedded in the top of the
digital pen captures not only what
is being said by the student holding
the pen, but also what is being said
by anyone nearby (e.g., researcher,
teacher, or other students). The recorded audio can be downloaded
and converted to MP4 or WAV
files. An infrared camera at the tip
records anything a student might
draw or write but does not capture
previously existing ink or artwork
FIGURE 1
Representation of the backbone of trypsin interacting with the
peptide substrate. The active site, binding pocket, and disulfide bonds
are additionally labeled.
From Chemistry: The Molecular Nature of Matter and Change (4th ed., p. 709), by M. S.
Silberberg, 2006, New York, NY: McGraw-Hill. Illustration by Irving Geis. Rights owned
by Howard Hughes Medical Institute. Not to be used without permission. Reprinted here
with permission.
A Novel Technology to Investigate Students’ Understandings
(e.g., the enzyme representations
we provided to students). Rather,
the camera tracks only where ink is
written. The digital pen uses a dot
positioning system to translate the
images taken with the infrared camera into a coordinate system to track
where the student has drawn on the
page. (Because the Livescribe product includes dot paper with several
distinct dot patterns, our research
group is able to simultaneously use
one pen across multiple projects
while keeping data separate by project.)
Specific to the Livescribe platform, at the bottom of each piece of
dot paper is a tool bar, with buttons
to record, pause, and stop during the
interview. Once the interview is completed, tapping on any portion of the
drawing with the digital pen triggers
the audio associated with that piece
of the drawing to play on the digital
pen, which comes equipped with a
built-in speaker. The digital pen can
collect up to 5 hours of both voice
and written data.
Developed in 2007, the Livescribe
platform is compatible with both
Windows XP and Mac operating
systems; it comes in 2GB, 4GB, and
8GB models (Livescribe, 2010; see
note at end of article). An electronic
file containing both the audio and
drawings can be uploaded to the
Livescribe desktop (Version 2.0).
Clicking on a computer screen over
any portion of the drawing plays the
associated audio, similar to touching
the dot paper with the digital pen itself. As the audio plays, the drawing
appears in real time. Drawings can
be exported as PDF files and also
FIGURE 2
Representations of the serine protease trypsin interacting with a
peptide collected from the interview with Jason.
as “pencasts” that can be embedded
into blogs, course pages, or e-mails.
A sample pencast from this research
study can be found at the following
link: http://www.livescribe.com/
cgi-bin/WebObjects/LDApp.woa/
wa/MLSOverviewPage?sid=sf7KC
VrfwdfD.
Research tool
In order to capture students’ prior
knowledge about enzyme–substrate
interactions during Phase 1 of the
interview, a blank sheet of dot paper
was provided to each student to write
notes or make drawings on to further
explain their ideas about enzyme–
substrate interaction. Students were
given a brief tutorial on how to use
the digital pen. Two-thirds of the
students used the digital pen during
Phase 1 of the interview, and 86%
of the 25 students interviewed used
the digital pen at least once during
the interview. An excerpt from Megan’s use of the digital pen during
Phase 1 of her interview is found
in Table 1. The table shows screenshots from the Livescribe desktop
when she was asked to elaborate on
what she meant by the active site
and its role during inhibition. Note
that unlike the challenges we faced
with interpreting Jason’s interview
transcript, the information captured
and displayed in Table 1 enabled us
to accurately correlate what Megan
said with each part of her drawing.
In interviews we conducted prior
to using Livescribe, we might have
had to read through 30 or 40 pages
of a transcript to find what a student
had to say about a particular part of
a representation.
During Phases 2 through 4 of
the interview, students were asked
to describe their understanding of
representations we provided (e.g.,
Figure 1). We tested several methods
to optimize our use of the digital pen
and dot paper to collect data about
students’ understandings of these
representations. These methods
included using an overhead projecVol. 42, No. 1, 2012
47
TABLE 1
Paper replay session.
Drawing*
Transcript of student interview
(explanation of drawing)
Time stamp
Well, we draw this.
4:12–4:25
Like this is our enzyme and it’s in solution with our substrate.
4:29–4:35
That line’s not supposed to be there. And they fit with one another.
4:39–4:44
And so this is how the enzyme and substrate form the complex.
4:45–4:53
This is the enzyme complex so this is the active site and they’re similar.
4:54–4:58
For example, going back to the inhibition you can have
4:59–5:04
something that looks like that. That’s the inhibitor. They’ll have the same
affinity for the active site but they’re two different things. And so they
compete for one another with the enzyme.
5:05–5:17
*Note the darkening of the images in the left-hand column as Megan explains what she draws.
48
Journal of College Science Teaching
A Novel Technology to Investigate Students’ Understandings
tor to project the image onto the dot
paper, using a light table to shine the
image through the sheet of dot paper,
and printing the dot paper onto an
overhead transparency and laying
the transparency over the image.
Ultimately, we chose to print the images used in the study onto individual
sheets of dot paper.
Implications for teaching
and further research
So far this manuscript has focused
solely on the implementation of
the digital pen as a data collection
technique. However, the effectiveness and applications of this suite
as a curriculum and teacher aid
in chemistry (and other science)
courses merit further investigation.
A review article by Van Schaack
(2009) described previous work
that would suggest that the Pulse
technology would be an effective
learning tool. Van Schaack offered
a multitude of ideas regarding the
digital pen for homework support,
audio/tactile graphics for blind students, assessing class participation,
and assessment of reading fluency.
To date, only one study has been reported thus far, comparing the pros
and cons of posting the instructor’s
pencast with a Livescribe pen to
screencasting PowerPoint slides
with a tablet PC in contrast to the
more traditional method of posting video of lectures (Benedict &
Stasko, 2009).
Livescribe makes their platform
available online for users to develop
and download new applications.
Currently, applications exist to draw
and recognize both amino acids and
acyclic alkanes. These applications
allow the student to draw a structure
and then name what has been drawn
or, conversely, to name a structure
and then assess the accuracy of the
structure drawn by the student as a
response (http://pen.intergrader.net).
However, the creation of disciplinespecific applications for the digital
pen remains largely unexplored.
There are multiple possibilities
for the Livescribe Pulse technology in science education. Biology
educators could use the technology
to explore student representations of
cell, animal, and plant anatomy or for
further investigations into students’
understanding of metabolism. Researchers in physics and engineering
could use the technology to track
student problem solving or creation
of representations of circuits, mirrors, or free body diagrams. Possible
uses within the Earth sciences would
include topographic maps, stratigraphy, or weather maps. Student groups
could make drawings and/or analyze
representations, upload their digital
file with audio to a course website,
and engage in peer critique of one
another’s work.
Although we have discussed just
one aspect of how this technology
could be used to improve the collection of qualitative data through semistructured interviews, the purpose of
this manuscript is to disseminate information about our technique so that
others may apply it in their research
and use it to better assess their students’ learning. We look forward to
reports of innovative implementation
of this technology describing effective ways for instructors and teachers
to implement it in their classrooms. n
Note: The authors have no direct
or indirect commercial interest in
the Livescribe product, nor do they
intend to develop a formalized relationship with the company at any
time in the future.
References
Anoto Group AB. (2011). Anoto paper
applications. Retrieved from http://
www.anoto.com/paper-applications.
aspx
Benedict, L., & Stasko, D. (2009).
High tech low tech: A look at digital
versions of chalk talks for the
general chemistry classroom. Paper
presented at the Northeast Regional
Meeting of the American Chemical
Society, Hartford, CT. Available at
http://www.usm.maine.edu/~dstasko/
research/NERM-talk.html
Bowen, C. J. (1994). Think-aloud
methods in chemistry education:
Understanding student thinking.
Journal of Chemical Education, 71,
184–190.
Bretz, S. L. (2008). Qualitative
research designs in chemistry
education research. In D. Bunce
and R. Cole (Eds.), Chemistry
education research: Nuts and bolts
(pp. 79–99). New York: Oxford
University Press, ACS Symposium
Series.
Bretz, S. L., & Linenberger, K.
J. (2012). Development of the
enzyme-substrate interactions
concept inventory. Biochemistry
and Molecular Biology Education,
40(4), 229–233.
Linenberger, K. J., & Bretz, S. L.
(2012). Generating cognitive
dissonance in student interviews
through multiple representations.
Chemistry Education Research and
Practice, 13, 172–178.
Livescribe. (2010). http://www.
Livescribe.com/
Oviatt, S., Arthur, A., & Cohen, J.
(2006). Quiet interfaces that help
students think. Proceedings of
the 19th annual ACM symposium
on user interface software and
technology, 191–200.
Van Schaack, A. (2009). Livescribe
in K–12 education: Research
support. Retrieved from http://
www.Livescribe.com/media/
pdf/education/Livescribe_K-12_
Research_Support.pdf
Kimberly J. Linenberger is a postdoctoral research associate in the Department of Chemistry at Iowa State University in Ames and was a student in the
Department of Chemistry and Biochemistry at Miami University in Oxford, Ohio,
when this paper was written. Stacey
Lowery Bretz (bretzsl@muohio.edu) is
the Volwiler Distinguished Professor of
Research in the Department of Chemistry
and Biochemistry at Miami University.
Vol. 42, No. 1, 2012
49