A Novel Technology to Investigate Students’ Understandings A Novel Technology to Investigate Students’ Understandings of Enzyme Representations By Kimberly J. Linenberger and Stacey Lowery Bretz Digital pen-and-paper technology, although marketed commercially as a bridge between old and new notetaking capabilities, synchronizes the collection of both written and audio data. This manuscript describes how this technology was used to improve data collection in research regarding students’ learning, specifically their understanding of enzyme-substrate interactions as depicted in textbook representations. Students were provided this technology during individual interviews and were permitted to annotate multiple representations of enzymes and substrates, as well as to generate their own representations. The ability to digitally revisit the sequential student drawings was valuable in analysis of the research findings. Innovative and novel uses for this technology are discussed for both discipline-based education research and classroom practice. O ften the tools that are used to collect data are buried deep within the methodology of a research article, if mentioned at all. We recently began to use a new commercial device that could revolutionize data collection for semistructured interviews. The purpose of this manuscript is to share our findings regarding this research tool with the community of science education researchers and science teachers. We are currently investigating biochemistry students’ understanding of enzyme-substrate interactions (Linenberger & Bretz, 2012). Because no assessment existed to specifically measure students’ knowledge of these interactions, we developed a concept inventory to do just that (Bretz & Linenberger, 2012). The first step in developing this concept inventory was to interview students using a semistructured interview guide (Bowen, 1994; Bretz, 2008). Part of this interview asked students to discuss two representations, each depicting a different aspect of enzyme-substrate interactions. One of the representations used in the study is found in Figure 1. While analyzing data collected from these semistructured interviews, we encountered several challenges, specifically with regard to interpreting drawings made by the students during the interviews. This paper describes these challenges, how the new technology resolved these difficulties, and the applicability of this technology to other science, technology, engineering, and mathematics (STEM) disciplines. Methodological challenges Interviews were both audio and video recorded. Students were permitted and encouraged to draw on the provided representations. Blank sheets of paper were also available, if needed, to further explain their ideas. Any such drawings or notes were collected and scanned as additional pieces of data. These rather traditional data collection techniques, however, created unanticipated challenges in the data analysis. The pencil and pen markings of students who drew or made notes were not always visible after scanning the images. We substituted a fine-point marker to ensure that the image was readable and provided multiple colored markers that could be distinguished from both one another and the paper. However, difficulties still remained. Students were repeatedly interacting with the representations during a single interview. Consider the copy of the representation (Figure 2) collected after an interview with Jason, a junior biochemistry student. Discerning post hoc the meaning of each marking to Jason in real time during the interview was difficult because he continued to draw over his own markings. We subsequently modified our protocol to use separate sheets of Vol. 42, No. 1, 2012 45 paper for each phase of the interview, thereby minimizing the problem of multiple markings. Although videotape captured student markings in real time, it was still exceedingly difficult to correlate markings on the representations with time stamps on a transcript of the interview. Students moved their head or hands, sometimes obscuring the view of the camera. The transcripts of Jason’s interview offer an example of this difficulty as he explained what he found confusing in Figure 1. It is almost impossible to discern post hoc which marking from Figure 2 corresponds with which pronoun or phrase: Interviewer: Ok. Alright, so what about the image do you not understand or find confusing? Jason: That, that, um, where this is pointing to. I don’t know what is going on in here. I don’t know. Um, this is a little weird how—’cause usually the enzyme doesn’t change, well I mean it could change conformation, but these are not the actual active site part of the enzyme. Like, these two are separated. I don’t know what’s going on in here. It just interacts with the substrate and not with itself. I mean it could, it does interact with itself, but it shouldn’t change its conformation or like eventually it’s going to have to go back to its original conformation. I’m just wondering. I mean, I don’t know, this is just a little weird. To remedy any potential misinterpretation of data, we revised our protocol to incorporate digital pen and paper technology (DPPT; Anoto Group AB, 2011) simultaneously. Although a pen tablet or graphical tablet interface could be used as well, we were eager to avoid any additional “learning curve” while exploring student understanding of 46 Journal of College Science Teaching the representations, as it has been shown that the farther the interface is from a traditional paper-and-pencil format, the larger the cognitive load for the student (Oviatt, Arthur, & Cohen, 2006). For this study, the Livescribe Pulse smartpen (http:// www.livescribe.com) was selected for its ease of use, cost-effectiveness, and availability. The remainder of this article describes our novel use of DPPT (beyond its intended commercial applications) and its potential for revolutionizing interview data collection in the context of investigating students’ understanding of enzyme-substrate interaction. Technological solution Used in the same manner as an ordinary ballpoint pen, the digital pen is a computer that literally fits between the fingers of a student. A microphone embedded in the top of the digital pen captures not only what is being said by the student holding the pen, but also what is being said by anyone nearby (e.g., researcher, teacher, or other students). The recorded audio can be downloaded and converted to MP4 or WAV files. An infrared camera at the tip records anything a student might draw or write but does not capture previously existing ink or artwork FIGURE 1 Representation of the backbone of trypsin interacting with the peptide substrate. The active site, binding pocket, and disulfide bonds are additionally labeled. From Chemistry: The Molecular Nature of Matter and Change (4th ed., p. 709), by M. S. Silberberg, 2006, New York, NY: McGraw-Hill. Illustration by Irving Geis. Rights owned by Howard Hughes Medical Institute. Not to be used without permission. Reprinted here with permission. A Novel Technology to Investigate Students’ Understandings (e.g., the enzyme representations we provided to students). Rather, the camera tracks only where ink is written. The digital pen uses a dot positioning system to translate the images taken with the infrared camera into a coordinate system to track where the student has drawn on the page. (Because the Livescribe product includes dot paper with several distinct dot patterns, our research group is able to simultaneously use one pen across multiple projects while keeping data separate by project.) Specific to the Livescribe platform, at the bottom of each piece of dot paper is a tool bar, with buttons to record, pause, and stop during the interview. Once the interview is completed, tapping on any portion of the drawing with the digital pen triggers the audio associated with that piece of the drawing to play on the digital pen, which comes equipped with a built-in speaker. The digital pen can collect up to 5 hours of both voice and written data. Developed in 2007, the Livescribe platform is compatible with both Windows XP and Mac operating systems; it comes in 2GB, 4GB, and 8GB models (Livescribe, 2010; see note at end of article). An electronic file containing both the audio and drawings can be uploaded to the Livescribe desktop (Version 2.0). Clicking on a computer screen over any portion of the drawing plays the associated audio, similar to touching the dot paper with the digital pen itself. As the audio plays, the drawing appears in real time. Drawings can be exported as PDF files and also FIGURE 2 Representations of the serine protease trypsin interacting with a peptide collected from the interview with Jason. as “pencasts” that can be embedded into blogs, course pages, or e-mails. A sample pencast from this research study can be found at the following link: http://www.livescribe.com/ cgi-bin/WebObjects/LDApp.woa/ wa/MLSOverviewPage?sid=sf7KC VrfwdfD. Research tool In order to capture students’ prior knowledge about enzyme–substrate interactions during Phase 1 of the interview, a blank sheet of dot paper was provided to each student to write notes or make drawings on to further explain their ideas about enzyme– substrate interaction. Students were given a brief tutorial on how to use the digital pen. Two-thirds of the students used the digital pen during Phase 1 of the interview, and 86% of the 25 students interviewed used the digital pen at least once during the interview. An excerpt from Megan’s use of the digital pen during Phase 1 of her interview is found in Table 1. The table shows screenshots from the Livescribe desktop when she was asked to elaborate on what she meant by the active site and its role during inhibition. Note that unlike the challenges we faced with interpreting Jason’s interview transcript, the information captured and displayed in Table 1 enabled us to accurately correlate what Megan said with each part of her drawing. In interviews we conducted prior to using Livescribe, we might have had to read through 30 or 40 pages of a transcript to find what a student had to say about a particular part of a representation. During Phases 2 through 4 of the interview, students were asked to describe their understanding of representations we provided (e.g., Figure 1). We tested several methods to optimize our use of the digital pen and dot paper to collect data about students’ understandings of these representations. These methods included using an overhead projecVol. 42, No. 1, 2012 47 TABLE 1 Paper replay session. Drawing* Transcript of student interview (explanation of drawing) Time stamp Well, we draw this. 4:12–4:25 Like this is our enzyme and it’s in solution with our substrate. 4:29–4:35 That line’s not supposed to be there. And they fit with one another. 4:39–4:44 And so this is how the enzyme and substrate form the complex. 4:45–4:53 This is the enzyme complex so this is the active site and they’re similar. 4:54–4:58 For example, going back to the inhibition you can have 4:59–5:04 something that looks like that. That’s the inhibitor. They’ll have the same affinity for the active site but they’re two different things. And so they compete for one another with the enzyme. 5:05–5:17 *Note the darkening of the images in the left-hand column as Megan explains what she draws. 48 Journal of College Science Teaching A Novel Technology to Investigate Students’ Understandings tor to project the image onto the dot paper, using a light table to shine the image through the sheet of dot paper, and printing the dot paper onto an overhead transparency and laying the transparency over the image. Ultimately, we chose to print the images used in the study onto individual sheets of dot paper. Implications for teaching and further research So far this manuscript has focused solely on the implementation of the digital pen as a data collection technique. However, the effectiveness and applications of this suite as a curriculum and teacher aid in chemistry (and other science) courses merit further investigation. A review article by Van Schaack (2009) described previous work that would suggest that the Pulse technology would be an effective learning tool. Van Schaack offered a multitude of ideas regarding the digital pen for homework support, audio/tactile graphics for blind students, assessing class participation, and assessment of reading fluency. To date, only one study has been reported thus far, comparing the pros and cons of posting the instructor’s pencast with a Livescribe pen to screencasting PowerPoint slides with a tablet PC in contrast to the more traditional method of posting video of lectures (Benedict & Stasko, 2009). Livescribe makes their platform available online for users to develop and download new applications. Currently, applications exist to draw and recognize both amino acids and acyclic alkanes. These applications allow the student to draw a structure and then name what has been drawn or, conversely, to name a structure and then assess the accuracy of the structure drawn by the student as a response (http://pen.intergrader.net). However, the creation of disciplinespecific applications for the digital pen remains largely unexplored. There are multiple possibilities for the Livescribe Pulse technology in science education. Biology educators could use the technology to explore student representations of cell, animal, and plant anatomy or for further investigations into students’ understanding of metabolism. Researchers in physics and engineering could use the technology to track student problem solving or creation of representations of circuits, mirrors, or free body diagrams. Possible uses within the Earth sciences would include topographic maps, stratigraphy, or weather maps. Student groups could make drawings and/or analyze representations, upload their digital file with audio to a course website, and engage in peer critique of one another’s work. Although we have discussed just one aspect of how this technology could be used to improve the collection of qualitative data through semistructured interviews, the purpose of this manuscript is to disseminate information about our technique so that others may apply it in their research and use it to better assess their students’ learning. We look forward to reports of innovative implementation of this technology describing effective ways for instructors and teachers to implement it in their classrooms. n Note: The authors have no direct or indirect commercial interest in the Livescribe product, nor do they intend to develop a formalized relationship with the company at any time in the future. References Anoto Group AB. (2011). Anoto paper applications. Retrieved from http:// www.anoto.com/paper-applications. aspx Benedict, L., & Stasko, D. (2009). High tech low tech: A look at digital versions of chalk talks for the general chemistry classroom. Paper presented at the Northeast Regional Meeting of the American Chemical Society, Hartford, CT. Available at http://www.usm.maine.edu/~dstasko/ research/NERM-talk.html Bowen, C. J. (1994). Think-aloud methods in chemistry education: Understanding student thinking. Journal of Chemical Education, 71, 184–190. Bretz, S. L. (2008). Qualitative research designs in chemistry education research. In D. Bunce and R. Cole (Eds.), Chemistry education research: Nuts and bolts (pp. 79–99). New York: Oxford University Press, ACS Symposium Series. Bretz, S. L., & Linenberger, K. J. (2012). Development of the enzyme-substrate interactions concept inventory. Biochemistry and Molecular Biology Education, 40(4), 229–233. Linenberger, K. J., & Bretz, S. L. (2012). Generating cognitive dissonance in student interviews through multiple representations. Chemistry Education Research and Practice, 13, 172–178. Livescribe. (2010). http://www. Livescribe.com/ Oviatt, S., Arthur, A., & Cohen, J. (2006). Quiet interfaces that help students think. Proceedings of the 19th annual ACM symposium on user interface software and technology, 191–200. Van Schaack, A. (2009). Livescribe in K–12 education: Research support. Retrieved from http:// www.Livescribe.com/media/ pdf/education/Livescribe_K-12_ Research_Support.pdf Kimberly J. Linenberger is a postdoctoral research associate in the Department of Chemistry at Iowa State University in Ames and was a student in the Department of Chemistry and Biochemistry at Miami University in Oxford, Ohio, when this paper was written. Stacey Lowery Bretz (bretzsl@muohio.edu) is the Volwiler Distinguished Professor of Research in the Department of Chemistry and Biochemistry at Miami University. Vol. 42, No. 1, 2012 49