Personalized Extensions: Democratizing the Programming of Virtual-Physical Interactions MASSACH-USL-1-S INST by OF TECHNOLOGY Abdulrahman Y. idlbi LIBRARIES B.S. Computer Engineering, Damascus University (2009) Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 2014 Massachusetts Institute of Technology 2014. All rights reserved. Signature redacted Author Program in Media Arts and Sciences August 22, 2014 Signature redacted Certified by Mitchel Resnick LEGO Papert Professor of Learning Research Academic Head, Program in Media Arts and Sciences Thesis Supervisor Signature redacted Accepted by Pattie Maes Alex W. Dreyfoos Professor of Media Technology Interim Academic Head, Program in Media Arts and Sciences 2 Personalized Extensions: Democratizing the Programming of Virtual-Physical Interactions by Abdulrahman Y. idlbi Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, on August 22, 2014, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences Abstract This thesis describes a new framework that enables a wider range of people to extend and customize programming environments to support more personalized explorations, especially in the physical world. While the vocabulary of the Scratch programming language (http://scratch.mit.edu) can be extended to access hardware devices and web services through Scratch Extensions, creating an extension is not trivial and requires a significant level of technical expertise. This occasionally means having less personalized, more abstract language for those extensions. I describe and analyze the design of PersonalizedExtensions, an extension mechanism that allows the users of Scratch to personalize abstract Scratch extensions, and consequently help other users to have more meaningful learning experiences in the worlds which Scratch extensions allow them to access. I conclude by reflecting on some case studies where personalized extensions were used, and how they affected the learning experience of the users. Thesis Supervisor: Mitchel Resnick Title: LEGO Papert Professor of Learning Research, Program in Media Arts and Sciences This thesis is based on research supported by the National Science Foundation under grant numbers 1002713 and 1027848. Any opinions, findings, and conclusions, or recommendations expressed in this document are those of the author and do not necessarily reflect the views of the National Science Foundation. 3 4 Personalized Extensions: Democratizing the Programming of Virtual-Physical Interactions by Abdulrahman Y. idlbi The following people served as readers for this thesis: Signature redacted Thesis Reader !/ Eric Klopfer Professor Director, The Scheller Teacher Education Program Massachusetts Institute of Technology Signature redacted Thesis Reader Amon Millner Visiting Assistant Professor of Computing Innovation Director, EASE Lab Franklin W. Olin College of Engineering 5 6 Acknowledgements I would like to begin by thanking my advisor, Mitch Resnick. Your ideas have been a source of inspiration since I knew about Scratch and the Computer Clubhouse seven years ago. I am grateful for giving me the opportunity to be part of this adventure, and for supporting me in every possible way while exploring and following my passions. Thank you for your kindness, patience, and all the wonderful work you've been doing to empower youth. I am also grateful to my thesis readers, Eric Klopfer and Amon Millner. Eric was on a sabbatical and Amon was expecting a baby, but they generously provided thoughtful advice, reflections, and comments on my work. The members of the MIT Scratch Team and Lifelong Kindergarten group have been a big family filled with love, compassion, and creative ideas. Each has contributed in their own awesome way. A special note of thanks goes out to J. Philipp Schmidt, a conversation with whom inspired the idea behind this work; David Mellis, John Maloney, Sayamindu Dasgupta, and Shane M. Clements, whose work on Scratch Extensions was essential for this thesis; Ricarose Roque for valuable advice and lessons on conducting research and running workshops (and for being with Sayamindu the kind, caring elder siblings they have been); and Amos Blanton and Natalie Rusk for their love and support all the way. Gobosforever v My childhood and teenage LEGO building experiences, by myself or with my four brothers (collaborating peacefully most of the time), had deeply affected my interests in learning in the physical world and connecting it to the virtual one. Building Scratch/LEGO WeDo chain reactions was a favorite activity while being at LLK. Thanks to the folks at the LEGO Company for the creative construction opportunities they have been providing over generations. Thanks to SparkFun (especially Nathan Siedle, Founder and CEO) for generously contributing most of the electronics that were used in the workshops I conducted as part of this work. My gorgeous family... I.-i~I 4i1 -Ir eA .4s1 .r.. lIyJI Lic J Ly 4i1 Liyj .9 'x,9 .i~ - - 4iI ai 7 .Ii oiI . j 9J4 L PLL 44,S L : "ol - VS 4Jhi .4 LaJJ V UI a. 8 Contents 1 INTRODUCTION .......................................................................................................................... 1.1 13 THESIS OVERVIEW ....................................................................................................................... 15 2 OVERVIEW OF VIRTUAL-PHYSICAL CONSTRUCTION KITS............................................................ 17 2.1 SCRATCH 1.X M ODIFICATIONS AND EXTENSIONS............................................................................ 25 2.2 SCRATCH 2.0 EXTENSIONS ............................................................................................................ 27 2.2.1 HTTP Extensions ............................................................................................................... 28 2.2.2 JavaScript Extensions .................................................................................................... 29 2.2.3 Accessing Scratch Extensions ...................................................................................... 32 3 PERSONALIZED EXTENSIONS 3.1 PERSONALIZING SCRATCH EXTENSIONS......................................................................................... 3.1.1 3.2 ...................................................... 35 38 Procedures to Libraries, Custom Blocks to Personalized Extensions ............................. 41 HYPOTHETICAL USE CASE.............................................................................................................. 44 4 EXPERIENCING PERSONALIZED EXTENSIONS............................................................................ 51 .4.1 M ETHODOLOGY.......................................................................................................................... 51 4.2 W ORKSHOP FORMAT ................................................................................................................... 53 4 .3 F IN D IN G S...................................................................................................................................5 8 4.3.1 On Connecting the Virtual to the Physical................................................................... 58 4.3.2 On Personalized Extensions ........................................................................................ 4.3.3 On Setup and Support M aterials.................................................................................. 60 5 LOOKING BACK, LOOKING AHEAD ........................................................................................... 58 63 5.1 PERSONALIZING AN EXTENSION, BUILDING A COMMUNITY ............................................................. 63 5.2 LEVELS OF PARTICIPATION ............................................................................................................. 66 5.3 GUIDING THE DESIGN OF PERSONALIZED EXTENSIONS..................................................................... 67 5.4 PERSONALIZED EXTENSIONS AS SCRATCH LIBRARIES........................................................................... 68 9 5.5 EMPOW ERING SCIENCE EXPLORATIONS WITH COMPUTATIONAL TOOLS ............................................. 69 A IMPROVEMENTS TO SCRATCH'S "MAKE A BLOCK" ................................................................... 71 B W ORKSHOP M ATERIALS ............................................................................................................. 75 REFERENCES..................................................................................................................................87 10 In the name of Allah; the Most Gracious; the Most Merciful; God of Abraham, Moses, Jesus, and Muhammad, peace be upon them all. 11 12 P Introduction Nowadays, children are increasingly involved in various virtual activities. However, physical objects still have their own attractiveness. Our physical experiences happen way before our virtual ones and stick with us. Physical objects surround us everywhere and are more intuitive to manipulate and play with, and patterns of collaboration emerge more easily and naturally around them, even when we are talking about young children. This is why it makes sense not to forget about the interactions with the physical world as we are creating new learning adventures for the children in the virtual one. Michael and Ann Eisenberg wrote about the relationship that should be between the crafts and computer applications [1], which I think can be generalized to the kind of relationship that should be between the physical and virtual worlds: It's a natural desire to employ all one's senses and cognitive powers in the course of a single project. We do not feel that a love of crafts is incompatible with technophilia, nor that an enjoyment of computer applications must detract from time spent in crafting. The world is not, or 13 should not be at any rate, a battleground between the real and the virtual. It is instead a marvelous continuum, a source of wonders that blend and knead together the natural and artificial, the traditional and novel, the scientifically objective and the personally expressive, the tangible and the abstract. We anticipate a future in which ever more astonishing things will present themselves to our minds, and ever more astonishing ideas to our hands. Having construction kits that support creating projects that connect the virtual and physical worlds allows providing multiple entry points for people with various interests and backgrounds to start the learning and creation process. Additionally, learning about certain physical principles in the world can be more effective when developed through interactions with actual physical items. However, creating such connections and making them accessible for a wide range of users and creators has not been easy. Looking at the programming language and environment Scratch for example [2], creating interfaces between the virtual authoring environment and the physical world (through what is called Scratch Extensions) requires a relatively high level of technical expertise. This means only a small set of physical devices can be supported, and that the users of the environment have little choice in customizing the extensions to meet their creation and learning needs and styles. This thesis explores how the barrier of creating Scratch extensions can be lowered to allow people with diverse backgrounds and interests, including educators, hobbyists and teenagers, to build their own customized interactions with the natural world with a shorter learning curve. As extensions become easier to create, people will have the freedom to connect a wider range of hardware kits, including ones of their own design. Throughout this thesis I describe and analyze the design of PersonalizedExtensions, an extension mechanism that allows the users of Scratch to personalize abstract Scratch extensions, and consequently help other users to have more meaningful learning 14 experiences in the worlds which Scratch extensions allow them to access. I conclude by reflecting on some case studies where Personalized Extensions were used, and how they affected the learning experience of the users. 1.1 Thesis Overview In Chapter 2, I overview the various construction kits for children that combine both the virtual and physical worlds. That covers the different ways through which Scratch has supported hardware connections. In Chapter 3, I describe the motivation and design of Personalized Extensions, a lowfloor mechanism to offer more meaningful learning experiences in the worlds which Scratch extensions allow to access. In Chapter 4, I present case studies that describe the experiences of learners using Personalized Extensions. In Chapter 5, I reflect on the design of Personalized Extensions and the experience of their users, discussing design implications and future directions. 15 16 Overview of Virtual-Physical Construction Kits There has been a long history of creating construction kits that connect the physical and virtual worlds. When Seymour Papert and his colleagues at MIT developed the Logo programming language for children in the late 1960's, what we remember now as a "screen turtle" was actually a "floor turtle" (figure 2-1): a simple mechanical robot connected to the computer by a long cable. Commands like "forward 50" and "right 90" were used to control an actual robot instead of graphic images on the computer screen [ 3]. In the following decades several kits in various forms were created like LEGO/Logo, LEGO Mindstorms, and Crickets; which were designed to support computerized construction in the physical world. 17 Figure 2-1: Logo 'floor turtle"[4] LEGO/Logo was a computer-based robotics system that supports a variety of design activities. Children used LEGO bricks (including nontraditional ones like gears, sensors, and motors) to build machines that could be connected to a computer and programmed using a modified version of the Logo programming language (figure 2-2). LEGO/Logo was meant to encourage the children to learn through the design process as they created a variety of their own machines before programming them. It was "viewed as a 'multi-media construction kit,' allowing students to build and create in several different (though interconnected) media." [5] 18 PoCu : Computer and Serial Interface box interpret the input optaoo .. r Inpnp[ oup: Keyboard Sensors Mouse Action by LEGO components Computer screen display F igure 2-2: Principles of LEGO/Logo learning environment [6 A follow-up of the LEGO/Logo project was the Programmable Brick (figure 2-3). It was a tiny, portable computer embedded inside a LEGO brick, capable of interacting with the physical world in a variety of ways. The Programmable Brick was designed to support multiple activities, multiple input/output modalities, multiple (parallel) processes, and multiple programmable bricks that interact with each other. The design activities and application for the brick included autonomous creatures, active environments, and personal science experiments. Similar to LEGO/Logo, a program had to be written on a computer using a special version of Logo known as Brick Logo. However, the programs are to be downloaded afterwards via a cable to the Programmable Brick, then the cable can be disconnected as the programs remain stored on the brick [7]. This project led to the development of LEGO Mindstorms and the PicoCricket robotics kits. 19 SENSOR INPUTS OPERATION + ON/OFF + * * + /E SWITCH LOW-BATTERY UTR . STARTLE BUTTON SCREEN CHOICE STOP KNOB BUTTON Figure 2-3: The Programmable Brick from MIT [81 LEGO Mindstorms is a programmable robotics construction kit that was released in 1998 to become a commercial success. The kit includes LEGO bricks, gears, motors, sensors, and an intelligent brick which is based on MIT's Programmable Brick (figure 24); and has been designed to be particularly well-suited for developing mobile robots, and has been the foundation for many robot competitions around the world; most notably, the FIRST LEGO League. On the other hand, the PicoCricket kit, which was very similar to LEGO Mindstorms, was designed for making artistic creations (e.g. musical sculptures, interactive jewelry, and dancing creatures) with lights, sound, music, and motion (figure 2-5) [9].1 1 The PicoCricket kit was released in 2006 and has been discontinued since 2012. 20 4.t r (a) (b) 4 / Figure 2-4: (a) A robot made with EV3, the third generation kit in LEGO Mindstorms robotics line (b) An EV3 program mot . on Lm forever (a) set powe. 100 - bri (b) Figure 2-5: (a) A PicoCricketproject (b) A PicoCricketprogramcreated with PicoBlock 21 After helping develop the robotics kits mentioned above, the Lifelong Kindergarten group started developing Scratch: a programming language with an online community for children to create and share their own interactive media such as games, stories, and animations (figure 2-6). LLK designed Scratch to be more tinkerable, more meaningful, and more social than past programming languages [10]. Since launching the website in 2007, Scratch has become a vibrant online community with over 6 million projects shared by thousands of creators, primarily between the ages of 8 and 16 years old. ® U---- 't.bJ ..*~n a.' WH. O~0r b.O . t.- 6 bx:: *~.wIo.l~r6 as r I -. J .q Yb. ' IW" rM.B.~ u-ub01 iMuWP s (b) (a) Figure 2-6: (a) The Scratch community website (b) The Scratch programming environment Right from the beginning, Scratch has provided support for the PicoBoard (also known as Scratch Sensor Board): an electronic board with multiple sensors that allow Scratch projects to sense and respond to things going on in the real world (figure 2-7). Later, Scratch also added programming blocks for the LEGO WeDo robotics kit which has a motor and a couple of sensors. What makes Scratch different from the examples mentioned above is that Scratch can be used to control objects in both the virtual and physical worlds and make them interact in various ways. 22 light sensor F: slider set Whirl 4>g) sensor value effect to sop J % 0 Figure 2-7: The PicoBoard and a Scratch script using the board's sound sensor A Scratcher can for example create an interactive story or game like the Ferris wheel in figure 2-8. The Ferris wheel will keep running (both the real one and the one on the screen) until the LEGO minifigure is brought closer to the WeDo light sensor. Another example is building a digital-physical chain reaction: Something from the physical world triggers a sensor, which kicks off a chain reaction within the virtual world of the Scratch project (figure 2-9). At the end of that virtual chain reaction, a motor is activated, which continues the chain reaction in the physical world. That triggers the next WeDo / Scratch project on the next laptop, on and on for as many links as there are in the chain [11]. 23 - )file .} kJ dUt Share Hae Ly ~~jpm n :+ Figure 2-8: A Scratch/WeDo project by Rita Chen, Inna Koyrakh, and Jinha Lee; created for MAS.714, a course offered at MIT Media Lab in the Fall 2009 term [121 N, N ? Ake e.+: 00-0 00"I .- -is Lft- 0. d 1d, 'O I ~ Figure 2-9: A virtual/physical link from a Scratch/LEGO WeDo chain reaction However, to understand the real potential of connecting Scratch to the physical world, one has to look at the history of Scratch modifications and extensions, many of which were intended to access hardware devices using Scratch. 24 2.1 Scratch 1.x Modifications and Extensions The source code of the first generation of Scratch (1.x) has been freely available, 2 and this has allowed advanced Scratch users and enthusiasts to modify and extend the programming language in different ways [13]. The modifications created of Scratch (also called mods) would typically introduce new functionality to the language, notably controlling a hardware device, and in some cases extend the grammar. There was not a standard process to follow to create a Scratch mod. One could simply download the source code and modify it as they wish as long as they meet the Scratch trademark policy, license the code properly, and not upload projects created by the mods to Scratch website. The MIT Scratch Team would also create mods to experiment with new ideas without affecting the normal users. For instance, NetScratch [14] was an early modification that had new dynamic blocks to receive data from the Internet. In another mod, Scratch was extended to support image-processing primitives, so that children programming with Scratch could create their own image filters [15]. Some of these extensions made their way into the official version of Scratch - one such example being the extensions that made it possible to use hardware in the LEGO WeDo kit with Scratch [16]. Outside of efforts by the MIT Scratch Team, Chirp was a Scratch mod that focused on adding new features like exporting and importing scripts as XML and compiling projects as executable files (to make standalone applications) [17]. Another notable mod, Build Your Own Blocks (later called Snap!) [18] extended the language by allowing users to define new blocks (procedures and functions). Additionally, multiple BYOB projects could interact by sharing variables and broadcasts. 2 The first generation of Scratch was written in Squeak, an open-source implementation of the Smalltalk- 80 language. 25 BYOB also extended the grammar of Scratch by adding new computational concepts like programmatic cloning to let sprites duplicate themselves while the program is running, and new data structures like lists of lists. The MIT Scratch Team considered adding some of these features to Scratch but they were always cautious in establishing a balance between making Scratch more powerful by adding new functionality and still having a language that is easy to get started with. 3 Those are two of the three elements which the MIT Scratch Team focus on while designing Scratch: having a "low floor" (easy to get started), a "high ceiling" (opportunities to create increasingly complex projects over time), and "wide walls" (supporting many different types of projects so people with many different interests and learning styles can all become engaged) [19]. When introducing new features to Scratch, the Team tends to prefer maintaining a low floor to raising the ceiling; however, enabling people to create their Scratch mods allowed them to break away from those design constraints to try new ideas and concepts on their own. Mods not only extended Scratch within the realm of virtual world, but also they were created to connect Scratch to hardware devices, in order to provide a friendlier access to the physical world, as compared to a traditional hardware programming language like C or assembly. S4A is a mod that provides new blocks to handle sensors and actuators connected to an Arduino board. Enchanting is another mod that is used to program LEGO Mindstorms NXT robots. In contrast to most other mods, Enchanting removed some of the Scratch blocks, thereby making itself more specialized, and incompatible with projects created with the official version of Scratch. Apart from direct source code modification, another method to extend Scratch was introduced with version 1.3.1 through what was called the remote sensors protocol. When the remote sensor connection feature is enabled, custom Scratch blocks can be created with programs that talk to Scratch through a network protocol. Two notable 3 Some of these features (e.g. cloning and procedures) have been added to Scratch 2.0 in a modified form. 26 examples of Scratch extensions made using this protocol are JoyTail, which adds joystick support to Scratch, and Kinect2Scratch, which allows creating Scratch programs that communicate with Microsoft Kinect. (10) L in02139 temperature ffi .betkle .edu value of sensor AM F90 W-1 - "_ -h0 -scratch.mit.edu Elba: Blocks to control a hardware device Enchanting (LEGO Mindstorms), S4A (Arduino), and Kinect2Scratch (Microsoft Kinect) Blocks to access the web - NetScratch, Snap!, and Insanity. insanity 1 ^3s [90ipt vari -{ foo.txt b4s New control blocks structures - Snap! and data New operations - Insanity and Snap! Figure 2-10: Examples of blocks from different categories from Scratch mods It should be noted here that regardless of the method that is used to extend Scratch, projects created with a mod could not be shared on Scratch online community website. Links to get new mods can only be shared on the forums or Scratch Wiki, but they are not endorsed or curated by the MIT Scratch Team. 2.2 Scratch 2.0 Extensions Noticing the various mods and extensions created by Scratch community, the plans for the second generation of Scratch included creating an infrastructure to support building and sharing extensions to the functionality of the programming language. The design of the extension mechanism went through two main stages: 27 2.2.1 HTTP Extensions The main focus at the beginning was extending Scratch 2.0 to control external devices (e.g. robotics kits, musical instruments) and to access data from external sensor hardware (e.g. sensor boards). As Scratch 2.0 runs in the browser,4 it cannot interact with hardware devices directly due to browser security restrictions. To run an extension, a separate application, called a helper app, must be installed and run on the computer. Scratch communicates with the helper app via HTTP requests, and the helper app talks to the hardware [20]. In addition to the helper app, an extension description file is required to use the extension. The description file is a text file in JSON formats that includes the extension's name, the TCP/IP port number used to communicate with the helper app, and a list of Scratch block specifications (figure 2-11). Every HTTP extension needs a separate helper app to be developed, installed, and run; which makes these extensions difficult to create, maintain, and distribute. With the introduction of the next version of extensions, JavaScript extensions, the online version of Scratch 2.0 no longer supports HTTP extensions. 6 4 An offline editor was introduced later. s See http://www.json.org/ 6 The Scratch offline editor still supports HTTP extensions. 28 1 { 2 "extensionName": "A4S (Arduino For Scratch)", 3 4 "extensionPort": 12345, "blockSpecs": [ 5 1" [" 6 7 8 9 10 ", "set pin %nas 9mi.mode", "pinMode", 2, "Digital Input"], %n %n ", "digital write pin %m.highLow", "digitalWrite", 13, "high"], "analog write pin value "analogWrite", 3, 2551, [" ", "servo write pin degrees "servoWrite", 5, 1801, ["b", "digital read pin 'n", "digitalRead", 2], ["r", "analog read pin "analogRead", 0], [" ", %n %n", %n", %n", 11 ], 12 "menus": { "mode": ["Digital Input", "Digital 13 14 15 16 "highLow": ["high", "low"], Output", "Analog Input", "Analog Output((PW )","Servo"], }, } Figure 2-11: An HTTP extension description file -A4S (https://github.com/damellis/A4S) A4S (Arduino For Scratch) v se analog s i write pin s2 gtldigital 6)high write pin 3.255 servo write pin degrees Lanalog read pin Figure 2-12: A4S (Arduino for Scratch) extension blocks (https://github.com/damellis/A4S) 2.2.2 JavaScript Extensions The HTTP extensions were cumbersome to manage, share, and run as each extension needed a different helper app that has to be installed. A new method for extending Scratch was introduced which allows Scratch to be expanded to interact with hardware devices (e.g. LEGO WeDo or the Kinect), or access online web-services (e.g. the US National Oceanic and Atmospheric Administration - NOAA weather service). The extensions themselves are implemented in Javascript and can use a single browser plugin to access hardware devices [21]. A JavaScript program for an extension has basically the following parts: a cleanup function when the extension is unloaded; status reporting code to report missing hardware, plugin or unsupported browser; block and block menu descriptions and 29 corresponding functions for the blocks; and the code to register the extension. Figure 213 shows the simplest code possible for an extension. It defines an extension called "My first extension" with a single command block "My first block" that does nothing. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 (function(ext) { // Cleanup function when the extension is unloaded ext._shutdown = function() {}; // Status reporting code // Use this to report missing hardware, plugin or unsupported browser ext._getStatus = function() { return {status: 2, msg: 'Ready'}; }; ext.myfirst block = function() { I/ Code that gets executed when the block is run // Block and block menu descriptions var descriptor = { blocks: [ // Block type, block name, function name 18 [' 19 20 21 22 23 24 25 ', 'my first block', 'my_first_block'], ] 1/ Register the extension ScratchExtensions.register('My first extension', descriptor, ext); })({}); Figure 2-13: The basic code for ajavaScript extension [22] The JavaScript code for the extension includes an extension descriptor to define the extension blocks, which is similar in format to the HTTP extension's description file. It lists the blocks and their type, parameters, menu description, and corresponding functions (figure 2-14). 30 1 2 3 var descriptor = { blocks: [ ['w', 'turn motor on for 4 5 [' ', [' ', 6 7 8 1' 96n secs', 'allMotorsOn'], 'allMotorsOff'], 'set motor power %n', 'set motor direction ['h', 'when distance %m.lessMore ['h', 'when tilt %m.eNe %n', ['r', 'distance', ['r', 'tilt', [' 9 10 11 12 13 14 15 16 'motorOnFor', 1], 'turn motor on', 'turn motor off', ', ', %m.motorDirection', %n', 'startMotorPower', 100], 'setMotorDirection', 'this way'], 'whenDistance', '<', 20], 'whenTilt', '=', 1], 'getDistance'], 'getTilt'] menus: { motorDirection: ['this way', 'that way', 'reverse'], lessMore: ['<', '>'], eNe: ['=','not ='] 17 }, 18 19 url: 'http://info.scratch.mit.edu/WeDo' }; Figure 2-14: A JavaScript extension descriptor - LEGO WeDo extension Q LEGO WeDov t~nmotor tr motor tuon for 0 doetont secs hi a Figure 2-15: LEGO WeDo extension blocks In both methods, a Scratch 2.0 extension extends Scratch with a collection of command, reporter, and event blocks that can be used to interact with a particular device or source of information on the web. When an extension is enabled, its blocks appear in the "More Blocks" palette. An extension is limited to adding a set of new blocks to expand the Scratch vocabulary to read or control objects that reside outside Scratch (i.e. hardware devices or web 31 services), but cannot alter the language grammar nor affect the look or behavior of sprites or the stage. 2.2.3 Accessing Scratch Extensions Extensions that are officially curated by the MIT Scratch Team, also called approved extensions, are accessible through the Extension Library (figure 2-16). Developers building new extensions can share them by sharing Scratch project files that use these extensions. When an extension is added or a project that already uses extensions is loaded, the new blocks appear in the "More Blocks" palette. X Extension Library N1 Hardware LEGO WeDo PicoBoard OK Cancel Figure 2-16: Scratch Extension Library, from where approved extensions can be added to a project 32 0 Finch V Move Finch Finch bz left: right Hz for a Finch LED color R0 0 :0 K iT m. .r turn "' C Q LEGO WeDov 00Ms motor .ii~i . . -l~ 1 on bstHelle Finch temperature ft . ri Finch rlgt obstacle LFinch L~ se moo ""Finch "Fbsnch X acceleration r r r"" to dieto thsa left light Finch Y acceleration Finch Z acceleration n20 Finch Orientation (a) (b) --- Speech To Text v Weather extension v recognize current temperature in city current condition in city Boston, Boston, speech and 0@ wait MA MA (c) (d) Figure 2-17: Examples of Scratch 2.0 extensions (a) An HTTP extension for the Finch robot (b) JavaScriptextension for the LEGO WeDo kit (c) A JavaScriptextension for a weather web service (d) A speech-to-text JavaScriptextension that uses the browser'sAPI 33 34 Personalized Extensions In a conversation with a colleague about the different ways Scratch could support teachers in classrooms, he mentioned a potential use of Scratch by teachers to create their own science kits and share them with their students to explore various disciplines. Commercial science kits are expensive and the MIT Scratch Team had already been working on developing the mechanism to extend Scratch, mainly to connect it to hardware devices. It would be great if teachers could use equipment that was available and relatively cheap (e.g. Arduino board connected to various sensors) to design and build their own science kits, then use Scratch to control and access that kit to explore the world around. The challenge was that Scratch extensions could only be developed by some of the more advanced Scratch users, as well as developers of hardware kits who want to see their devices supported in the Scratch language. In many cases that would be just fine. Take the LEGO WeDo extension for example (figure 3-1) [23]; the parts of the hardware kit are known: motors, lights, distance sensor, and light sensor. Consequently, the developers of the LEGO WeDo extension could define blocks that are meaningful to the 35 young users and reflect how the hardware will be used in real life. If a Scratcher wants to know the distance between the distance sensor and an object, she simply uses the "distance" reporter block. LEGO WeDo - turn motor off set motor power when distance Q to < zwO (b) (a) Figure 3-1: (a) The LEGO WeDo robotics kit (b) The LEGO WeDo extension blocks 7 However, for an abstract hardware device like the Arduino board, one cannot predict how it will be used in the real world. An extension developer for the Arduino can only define generic blocks that access the basic low-level functionality of the board - mainly reading or setting the pins on the board (figure 3-2). A Scratcher cannot intuitively use the blocks to read a distance sensor that is connected to the board. She will have instead 7 Arduino (http://www.arduino.cc/) is an open-source physical computing platform based on a simple microcontroller board, and a development environment for writing software for the board. 36 to read the value of the pin to which the sensor is connected, using a block that reads "analog read pin 1". A4S (Arduino For Scratch) v set pin 2 _ "' dagnal "3255s Digital write pin nput 13 haluh digital read plnG A p an a log re ad (b) (a) Figure 3-2: (a) An Arduino Uno board (b) The A4S (Arduino for Scratch) extension blocks Figure 3-3 shows how a temperature sensor (TMP36) can be connected to an Arduino board, and how a script that uses the abstract extension blocks will look like. The example script sets the color effect for a Scratch sprite to the temperature value (measured in Celsius). The value read on the analog input (which is a 10-bit analog-todigital converter) has to be converted to millivolts, then plugged into the formulas indicated at the bottom of the figure. 37 E *C t nm . OSemp in -. - - O 00 -. 02 00 1 (ou0nm-5001/0 Temp in *F= [(Temp in *C) x 1.8 ]+ 32 Figure 3-3: Connecting a TMP36 temperature sensor to Arduino, and a script using the A4S extension to read the temperature 3.1 Personalizing Scratch Extensions In the previous example, the end users of the hardware kit could have been helped by providing programming blocks that had been personalized, given the fact that the configuration of the kit became known. To approach this challenge, I built on a new feature that was introduced in Scratch 2.0 to allow users to define custom blocks. Custom blocks are Scratch's version of what is usually referred to as procedures, functions, or methods in other programming languages [24]. 38 Scripts Costumes Sounds *Motion l Events Looks Sound * Pen ' Sensing ' Operators Sho * Control Data Mor Bck Make a Block pixels in 0 secs jump Add an .h5 tisspit 1i~. Extension Edit Block 0 jum hght pxes ntime sc ii F i v Options Add number input: Add string input: Add boolean input: Add label text: text !t' Run without screen refresh OK Cancel Figure 3-4: Adding and defining a custom block in Scratch 2.0 Custom blocks are created and found in the "More Blocks" palette, the same place where Scratch extensions appear when added. A custom block is shown in the palette for each Define hat block. To create a custom block, the "Make a Block" button in the "More Blocks" palette is used. A new block window will open, where one can name the custom block and define its parameters. A Define hat block appears in the scripting area where the script that defines the new block can be built. Figure 3-4 shows how a simple "jump" can be defined using standard Scratch blocks. Similarly, a new block can be defined using blocks from an extension. The definition of a 39 II "blink" block for the Arduino board using the A4S extension 8 looks like what figure 3-5 shows. The new "blink" block will make the LED connected to the digital pin #13 blink 10 times.9 Make a Block Add an Extension 1 'i o A4S (Arduino For Scratch) v 1 se 2i a igtl nu diia rie * digtalwrte inC 1 1 25ig digital read pin an.g read pin Figure 3-5: Defining a custom "blink"block that uses A4S extension blocks The creator of the project can now use the new block in his project, and even share the project so other Scratchers can use it as a whole or use the individual scripts (including new blocks) in it. For the creator of the project, being able to see the definition of the "blink" block makes sense: she created it and might be iterating on its design. Although there is no intuitive connection between "blinking" and "digital write pin 13" high and 1 I use the A4S (Arduino for Scratch) extension through this document to demonstrate creating Personalized Extensions based on it. A4S is being developed and maintained by David Mellis from the Lifelong Kindergarten group, and is publicly available on https://github.com/damellis/A4S 9 It should be noted that the Scratch script or program do not get uploaded to the connected hardware, in contrast to what is usually understood when talking about programming an Arduino board. When the program is run, Scratch maintains a live connection with the board, and commands are sent to the board in the real time. This paradigm keeps the programmable objects in Scratch and the outside objects (the connected hardware in this case) all in synch. 8 40 low, she still knows what that means. That could be also the case with some other users of the project, but for many others, they would just like to use the new block to make an LED blink. Their focus is what it does, not how it does it, which is similar to the concept of using a library in a conventional programming language. 3.1.1 Procedures to Libraries, Custom Blocks to Personalized Extensions Sharing new custom blocks with the community where their definition is directly visible to the users has its benefits, especially within the principles that are advocated for by the MIT Scratch Team regarding openness. 10 However, encapsulating and hiding some functionality has its own value. In one of my favorite papers on designing construction kits for kids, Mitchel Resnick and Brian Silverman talk about the importance of choosing black boxes carefully. In designing Scratch, much thought was put into choosing the basic building blocks. These choices do not only affect how easy or difficult it is to use the programming language (or the construction kit in general), but also "what ideas users can explore with the kit - and what ideas remain hidden from view." [25] In the case of a hardware platform like the Arduino, working on a low level using commands that deal with individual pins is good if that is what the user is eager to learn about (or expected to). 11 However, if the user is more interested in exploring with Once a project is shared on Scratch website, visitors of the website not only can run the project, but also "look inside" it to inspect the programming scripts, objects, images, and sounds; experiment with and modify them; use these elements into their own projects; or even remix the whole project. This aspect of Scratch design has always triggered discussions among the members of the community about sharing, remixing and giving proper credits. Additionally, the source code of Scratch has been freely available since its release. 11 Low-level commands can be viewed as abbreviations designed for a chip to understand (e.g. "analog read pin 1"), while higher-level commands are those designed for humans to understand (e.g. "turn LED 10 on"). 41 sensors in the world, higher-level commands that deal with the sensors directly will help the user in focusing on the topic in hand, encouraging deeper explorations. 12 However, as the developer of the original, abstract extension can only see the plain hardware board, but not how it will be used, she can only develop a Scratch extension with abstract, low-level blocks. But a user who has a specific use in mind for that piece of hardware, can create a modified version of the hardware extension, the same way she can create a custom block out of Scratch standard blocks, then black-box it to encourage certain learning experiences when the modified extension, which I will call Personalized [1: 23 blink extension by adlogi (unshared) " Extension, is shared with others. Scripts Costumes I Sounds ' Events Motion Looks ' Sound Pen Data 3Control ' Sensing Operators 3 y:.54 Make a Block x: 240 y: 180 Add an Extension A4S (Ardulno For Scratch) vi® sDi 3i s1 Stage SKo Dinoaurl p5 d3i a New backdrop anlgw h i -- I iaou 2r5e g 0e i au q= q Backpack Figure 3-6: Sharing a Scratch 2.0 project with the Scratch online community 12 It is worth mentioning that Arduino has been successful partly due to its encapsulation of various aspects of electronics and microcontrollers, which made it easier for makers and hobbyists to use [26]. 42 When a creator wants to share her project, she currently presses the "share" button (figure 3-6). Then, other Scratch users can view and open the project, and they will see exactly the same objects and scripts the project owner can. If the creator wants instead to share an encapsulation of the behavior, leaving the definition invisible to create a Personalized Extension, I add the "Export a Personalized Extension" and "Import a Personalized Extension" to the File menu in Scratch editor (figure 3-7). When a creator chooses to export a Personalized Extension, all the custom blocks definitions are saved to an external file, which can be shared later with other users. blink exten w23 by adlogi stumes Sounds p a vents (ur Control Sensing Operators y ion A4S - (Ardulno For Scratch) se3 Stage I backdrop Sprte1 Dinoauri p; s D gia r n d n New backd~rop: Lanalog red pin vau 25 4= Q Backpack Figure 3-7: A mockup of Scratch 2.0 interface to export and import Personalized Extensions When another user imports that Personalized Extension, neither the custom blocks definitions nor the original extension blocks that were used to define the custom blocks will be visible. The blocks that were defined as custom blocks will now be available as a new extension (figure 3-8). However, if the Personalized Extension is opened as a Scratch project (File -+ Upload from your computer) instead of importing it (File -+ 43 Import a Personalized Extension), the original extension and block definitions will appear again, and can be edited to modify or remix the Personalized Extension. d r 4a2 File I Edit , z~r Tips About Scripts Untitled-6 by d"og (unshared) J Costumes adlogi "1,0& v D Sounds 3 Events Control ' Sensing ' opertorS * Motion Looks Sound Pen I Data Make a Block x; 240 y: -180 , Add an Extension blink extension v i/p Now backdrop: a/ra =q 0 Backpack Figure 3-8: A "blink"Personalized Extension after being imported The current interface for creating custom blocks in Scratch serves as a good basis to build Personalized Extensions. However, a couple of features should be introduced to make Personalized Extensions support blocks of different types. Those changes are listed in Appendix A. 3.2 Hypothetical Use Case Nada is a high school teacher who wants her students to learn about some environmental issues. She wants to provide them with some tools to observe and explore these issues in the real world, experiment with it, not only read about it. One option is to seek a commercial science kit with various sensors which has software that does the data acquisition and present the data in a collection of graphs and tables. Even when a carefully-designed kit is easy to use, several challenges arise. Assuming the 44 financial cost is not an issue, Nada still wants to be able to customize the kit to fit explorations in different areas in the future, especially to meet unexpected questions coming from the students themselves. Additionally, she does not want her students to deal with dry, abstract data as the only option they have. Using abstract data to create graphs can be useful in many cases to represent and understand a phenomenon, but this kind representation does not always fit everyone. For example, when numbers that represent the readings of a carbon monoxide sensor are shown, students might not be sure what these numbers mean. They would wonder "Is a reading of 75 a bad thing? What caused this value? How does it affect the environment around us?" Allowing the students to create more concrete representations of the readings (e.g. projects with pictures where the readings are indicated by the color of the sky, the traffic on the highway, and the number of trees) can make certain ideas about environmental conditions more accessible. Nada wants to support and validate multiple ways of thinking and knowing about the subject in hand - both abstract and concrete [27]. She expects that some of her students would have better learning experiences by creating stories or animations that use what they observe in the real world. She would even like to have them build live virtual-physical interactions that combine the intuitiveness and concreteness of the physical world with the computational power the virtual world offers. Nada knew Scratch could be used to create personal representations of the data. She had also some experience using the Arduino platform after building some projects using online tutorials, and knew she could have some sensors attached to an Arduino board, then connect the board to Scratch to read data from the real world. She would lose some accuracy, and would have to build the Arduino kits herself (or with the help of her students), but once that is done her students can start their own explorations, using a kit that deals with many of the issues mentioned above. 45 As she wants to explore environmental issues, Nada builds a kit that has a carbon monoxide sensor, a light sensor, a temperature sensor, and a tri-color LED, all connected to an Arduino board; then connects it to Scratch by adding the A4S (Arduino for Scratch) extension available in the Scratch Extension Library (figure 3-9). X Extension Library All Hardware PftoBoard WeDo A45 OK Cancel Figure 3-9: Adding an extension from the Scratch Extension Library The A4S extension provides Nada with abstract block to access the board, with commands like "digital write pin 13" and reporters like "analog read pin" (figure 3-10), but her board now has very specific features and capabilities. She enhanced it by connecting several sensors to reflect certain needs and applications, using instructions from various online tutorials. While she can use the abstract blocks, she wants her students to use blocks that reflects how the board (or the kit) is used in the real world; not a bunch of electronic pins anymore, but sensors that read gas, light, and temperature values. A4S (Arduino For Scratch) v write 2digital pin *m* high 3 analog write pin bvalue 255 digital read pin Lanalog read pin Figure 3-10: A4S (Arduino for Scratch) extension blocks as shown in the "More Blocks" palette 46 uJ She defines the new blocks as she knows what sensors she is using and how they are connected to the board. She creates new blocks to report the values of the temperature sensor (temperature), the light sensor (light), and the carbon monoxide sensor (carbon monoxide); in addition to several blocks that can control a tri-color LED (turn LED on, turn LED off, and set LED color to) to indicate some status (figure 3-11). son FRO " Edit " -nips Scipts 70 My Envlroinmen Kits Z3 by adlogi(ushed) Costumes Moon * Looks Socund J Pen .mssu; A 4 About odlopl i n.c' Evnt , Cuntun lUSesinc 1 O~emt-,r Doata v - d (Z ----. Meke a Bc !n Sl240 L5 m8 '! dennt carbon monoxide et/can report analog read pine > n Ne bakrp Add an Extes~o US (Ardulno for Scratch)' e r: s to z. vs tt Backpack Figure 3-11: Defining custom blocks using the A4S extension blocks to create a Personalized Extension called "My Environment Kit" Afterwards, Nada exports the new blocks as a Personalized Extension she calls "My Environment Kit", and shares the resulting file with her students who import it into their projects to use the new blocks. 47 FRO W looking for green Edlt v Tips About Scripts j Costumes *Motion sEvents * Sound * Sensing 3Looks 3Pen a . r w r A Sounds Control 3Operators a - - -: -1 3- Make a Block My Environment Kit " Add an Extension Ks4 a r/~ Ii Backpack Figure 3-12: A project using the "My Environment Kit" Personalized Extension Some students will use the Personalized Extension to create Scratch projects that show graphs representing collected data, but other will seek different forms of representation, including pictures and animations (figures 3-12 and 3-13). I Figure 3-13: Screenshots from a hypothetical Scratch project, where the creator chooses to represent the readings of a carbon monoxide sensor by the color of the sky, the traffic on the highway, and the number of trees, indicating changing environmental conditions [28] Moreover, Nada can now share her creation - the environment kit - with other interested educators and students. She can do that by sharing the Personalized Extension and how she designed and built her Arduino-based kit (figure 3-14 and Appendix B for more examples of cards that can be shared to explain how to use the kit). 48 Temperature Sensor 7t. Lr- Components TMP36 - Temperature Sensor " F SO . - - 31 i-F The ' yellow wire - 4- ..--. t---; ---- an be connected to any othe ANAIOG IN pins, numbered Ao-A&. Blocks Example - 71 - Tell the board which analog pin the temperature sensor is connected to. -ft.. --4 -+... Get the temperatrein Fahrenheit. 1 -IT I saratch.mit.edu gthub.com/dameHis/A48 aratch.mit.edu iT. r github.com/daneis/A4S Figure 3-14: A cardused to support the learnerswhile using the "My EnvironmentKit" Personalized Extension 49 4- 50 Experiencing Personalized Extensions In this chapter, I describe a number of case studies of abstract and personalized extensions being used to connect Scratch to the physical world, specifically, via Arduino boards. The audiences varied: high school students, college students and educators. I look at learning outcomes and challenges that arose while using the extensions to connect Scratch to the physical world. 4.1 Methodology To understand the efficacy and implications of my design, I conducted several informal studies with students and educators, and gathered feedback through face-to-face conversations, and discussions on feedback forms. I worked with educators to get their feedback and perspective on the potential use of Personalized Extensions in their classrooms, and with students who had no previous experience with electronics to observe their first experiences with Arduino while using Personalized Extensions. A listing of the contexts that I used for my studies follows: 51 Workshop 1 The first study took place at the Media Lab with educators who had attended Scratch Educator Meetups. 13 It was a three-hour workshop with three participants, all of whom were familiar with Scratch, but only one had worked with electronics and Arduino. Workshop 2 The second workshop lasted four hours with a different audience. It was held at the Flagship Computer Clubhouse at the Boston Museum of Science with seven participants:14 the Clubhouse coordinator and six high-school students who were volunteering there. The youth were new to Scratch and Arduino. Workshop 3 This workshop was also four hours long. It took place at the Media Lab with five college students. They were new to Scratch, but a couple had some experience of coding and none of Arduino. Workshop 4 This one was of shorter duration (90 minutes). It happened during the Scratch@MIT conference. There were 16 participants, mostly educators who were familiar with Scratch. Most of them had little or no Arduino experience. See http://scratched.gse.harvard.edu/discussions/scratch-educator-meetups 14 The Computer Clubhouse (http://www.computerclubhouse.org/) is a creative and safe out-of-school 13 learning environment where young people from underserved communities work with adult mentors to explore their own ideas, develop new skills, and build confidence in themselves through the use of technology. 52 Figure 4-1: Picturesfrom the Personalized Extensions workshops 4.2 Workshop Format The topic of all workshops was connecting Scratch to the physical world via Arduino. A workshop starts with 30- to 45-minute introduction to Scratch if the participants were new to it. That is followed with a brief introduction to the Arduino board showing how it could be used with Scratch to read sensors and control LEDs (figure 4-2), how to use a breadboard, and how to import an extension. There were two aspects of the Personalized Extension experience I could study: creating Personalized Extensions, and using them to explore the physical world. Extension creators had to be familiar with both Scratch and Arduino, so I decided to focus on the 53 larger audience of extension users. Most of the workshops focused on how the participants' first experience with Arduino would be when a Personalized Extension is used. There was a case where a participant moved to use an abstract extension after using the personalized one. In the last workshop, participants started by trying the abstract extension,15 and then moved on to use the Personalized Extension. Figure 4-2: A high striker example using Arduino with a Personalized A4S Extension to read a forcesensitive resistor and control LEDs while running an interactive animation on the screen The abstract extension that was used was A4S (Arduino for Scratch), which is an experimental HTTP extension that can be used with Scratch offline editor [29]. A4S is written in Java. The Personalized Extension was based on A4S and had the blocks shown in the figure 4-3. 1s See the first handout in Appendix B. 54 A4S (Scratch for Arduino) v connect light connect button A4S (Arduino For Scratch) v 13 Dt2. dgal3 write pin h255 sensor to pin AO to pin 7 S . se. pED 0 digital read 10 10:10 analog read pin (a) (b) Figure 4-3: (a) A4S extension blocks (b) A4S personalized extension blocks After an introduction to Arduino, the participants were given handouts that explained how to connect 3-5 sensors and two kinds of LEDs to an Arduino board (see Appendix B). They were asked to try connecting a sensor then an LED and program them as a start. Afterwards, they were encouraged to try different sensors or expand what they already had connected into larger projects using craft materials and LEGO bricks they had around. For example, one of the participants decided to integrate the tricolor LED and force sensor he had connected into a robot. To complete his project he wanted to add another LED, which led him to explore using the abstract extension with the personalized one (figure 4-4). 55 N P4,01 ;r -MW A. Figure 4-4: A robot expressing anger when its head is poked. A Scratch program plays a sound while the eyes (tri-color LEDs) changes color from green to red. The head is hiding a force sensitive resistor under a LEGO plate. Video on http://youtu.be/FroOEUSKpdU The handouts were inspired by Scratch Cards, which are used to help beginners in using Scratch (figure 4-5). The handouts were designed to be simple, created with software 56 available for everyone, so teachers knew they could easily build support materials if they were to produce their own Personalized Extensions (figure 4-6).16 M4o~Jfl -Moving .An Mol, ~n pt1MDIl Clkk ______ tooe the sprite Ikbrry Choe sprite that Lmoeosum_ You can change its Mutat has 2 tytttt. QictM( ttw Figure 4-5: A Scratch Card LED LED X20 t 330 c . n)- n* 7a~a ij- + . h4T4 wbsd 21-P T.1~ to thqiae conce toi (a..7, n - Turnan LED anor o1ELEs are numbe.ed accrding to the DIOFYALpin 4: ea. t acrateh.mdt.edut aithb.ao/deneBa./A48 Figure 4-6: A handout that was used to support creating with the personalized A4S extension The handouts were produced mainly using Google Docs (https://docs.google.com), and Fritzing (http://fritzing.org) was used to create the breadboard view of the circuits. 16 57 4.3 Findings My focus was observing how Personalized Extensions could lower the barrier for Scratchers as they make connections between the virtual and physical world. I developed the following points by observing the participants during the workshop and by face-to-face conversations. At the end of every workshop, I used a reflection tool called Green-Yellow-Red, asking the participants about what went well, what was fair or questionable, and what could be improved; then opened the floor to final remarks. 4.3.1 On Connecting the Virtual to the Physical Making connections with the physical world provides additional entry points for learners and supports a multiplicity of styles and experiences. A participant who was using Scratch for the first time said the blocks (both standard and personalized) were intuitive to use. However, it was the connection to the Arduino what made Scratch make sense to him as a college student. He later added that he liked "seeing [his] program going into something real out in the world." 4.3.2 * On Personalized Extensions Many participants expressed that the Personalized Extension made using the Arduino board easier. One educator who had previously attended Arduino workshops said she felt for the first time she could replicate what she learned to share it with her students. A college student said she had felt intimidated to try [learning] electronics before, but liked her experience as a beginner using the Personalized Extension. " The Personalized Extension made the learning and building experience more personal, as creatorscould focus on what they can create, ratherthan how to make it run in the first place. A college student described her experience saying: "Come 58 out with something, and create it, and make it your own". Another added he felt he was "only limited by his imagination, not the tools." * Personalized Extensions can also provide a multiplicity of entry points, whether depending on interests or levels of expertise. An educator mentioned she had students with different skills and interests attending her computing class. With offering abstract and personalized extensions, each of her students could choose the level of abstraction they are comfortable with, while exploring similar computational ideas. * PersonalizedExtensions allowed for a more participatory, engaging approach to the workshops. Because abstract behaviors were black-boxed, there was no need to explain them right from the beginning. The young participants particularly liked that they did not have to be lectured at the beginning to explain how things worked. Instead they could start the hands-on activity quickly. * The abstract and personalized blocks can complement each other, even for the same learner. In the third workshop, one of the participants wanted to use two tri-color LEDs in his project. To make a balance between the capabilities of the Personalized Extension and its simplicity, it provided blocks to control only one tri-color LED. To control another, the participant had to import the abstract extension and use its blocks with my help. He was eager to go through the process because he had then a clear idea to bring into realization. His scripts to control the LEDs had a combination of blocks coming from the personalized and abstract extensions, and looked like the script in figure 4-7. His comment on the experience was that the abstract blocks revealed the actual functionality and provided higher flexibility in using the board, but it was better to use the modified extension to get him started. 59 c nnect force sensor to pin Ao en force sd robot Oeotetat Pl yRGB s udset LED to red analog write pin value analog write pin value analog write pin value s set RGB LED to h green analog write pin value b analog write pin avalue write pina value analog Figure 4-7: A scriptusing blocks from the original and personalized A4S extensions S One of the main challenges that face the creator of a Personalized Extension is choosing his kit contents and extension blocks carefully. O ffering few options can be over-constraining; offering too many can be overwhelming. In my case, the kit that was used with the Personalized Extension had five sensors (light, temperature, force, potentiometer, and push button) and two kinds of LEDs. Some participants said it was a bit overwhelming in the beginning to have so many sensors available. 4.3.3 On Setup and Support Materials Providing a good tool is not enough. Creating good support materials and guidelines for its use is essential to have a good learning experience. * Many educators expressed that the handouts that were provided served well in leading the participants as they were building their first circuits. The handouts 60 themselves evolved over time to address issues raised or experienced by the participants. For example, actual images of the sensors with polarity information were added because some participants found difficulty in recognizing the drawings that were on the layouts. * Most of the trouble the participantsfaced was due to difficulties in troubleshooting and debugging the hardwareproblems. Some participants expressed the need to have a checklist of techniques to be tried when something goes wrong with the hardware part of their projects. * Some high school and college participants asked for more examples and applicationsthat show variousways of using a specific sensor. * The participantsin the third workshop (college students) had the chance to use craft materialsin their projects. They liked that, and it was reflected in the styles and diversity of the projects they came up with.1 7 LEGO bricks and craft materials were placed on the worktable they were sitting at. I learned my lesson the hard way in the second workshop (high schoolers) where the participants were surrounded by all kinds of craft materials on shelves and cabinets around the room, but not in immediate reach of their hands on the table where they were working. They ended up using none. 17 Check these videos: http://youtu.be/FroOEUSKpdU, http://youtu.be/Wa426j8DIv4. 61 http://youtu.be/KTHd08-sTOk, and 62 Looking Back, Looking Ahead When I started thinking about the topic of this thesis, there were several themes I was interested in exploring with regard to connecting Scratch to the physical world: I wanted to learn how people with different backgrounds could design and create their own extensions, how they would use these extensions in projects that are personally meaningful, how the creators and users of the extensions would share their creations and experiences, and how they could be supported through these experiences of creating, sharing, and building with these extensions. Obviously, I could only explore a small set of these aspects through this work. In this chapter, I reflect on the lessons I learned and the insights I gained as well as future directions in design and research. 5.1 Personalizing an Extension, Building a Community In the hypothetical use case in Chapter 3, the high school teacher, Nada, created her own kit (a variety of sensors connected to an Arduino board), and shared it with her students to learn about some environmental issues. She turned the generic board (with its 63 abstract extension blocks) into a toolkit that can be used with blocks that give a concrete sense of they can do ("turn LED on" vs. "digital write pin"). Afterwards, Nada could share her experience and her students' (i.e. the Personalized Extension, guides to assemble the Arduino kit, projects created by the students, etc.) with other educators who are interested in this area, and who in turn can use and remix the Personalized Extension and the materials related to it. Communities around various aspects of the Personalized Extensions experience are expected to emerge. The creators of the Personalized Extensions can share their creations on ScratchEd for example, which is an online community where Scratch educators share stories, exchange resources, ask questions, find people, and discover events. 18 ScratchEd can be used to share the Personalized Extensions and the specifications of the hardware kits associated with them. Extension creators can also use the platform to share their reflections on design practices and experiences, and get feedback from other designers or users of the extensions. Other educators or hobbyists may introduce changes to existing Personalized Extensions to improve the user experience or add new functionalities (figure 5-1). The educators using the Personalized Extensions can use the same platform, ScratchEd, to share their stories and experiences with the extensions. They can also share new materials, guides, and lesson plans they develop to be used with them. They may also add their requests asking for new Personalized Extensions (or improvements to existing ones) that support more learning experiences or explorations in the physical world. 18 ScratchEd (http://scratch-ed.org/) 64 connect button setn pn as wrie fu- connect button sensor to to AO pin 7 +" on a nu digital write pin dagnal to pin 7 COnneCt pin 13 high vau 25 analog read pin Figure 5-1: Creators of Personalized Extensions can build on abstract approved extensions or remix existing Personalized Extensions Young Scratchers, who might not be familiar with ScratchEd, should be able to share their projects that use Personalized Extensions on Scratch website (because they are basically custom Scratch blocks made with approved extensions, which are shareable on Scratch website). Through the project page (figure 5-2), the creator and viewers of the project can share their notes, comments and feedback on the projects, including the use of the Personalized Extensions. 65 wwrrm A.. Ls W, *AS .a as rr ~i~~wmacuapmmum .a rwr~a ~ vow Figure 5-2: A Scratch project page 5.2 Levels of Participation Before introducing the Personalized Extensions, there were only two levels of participants in the Scratch extensions ecosystem: the few developers who could develop extensions using JavaScript (to be curated by the MIT Scratch Team in Scratch Extension Library), and Scratchers who used those extensions. With the introduction of Personalized Extensions, a new layer of Scratchers is added in between, which is the Scratchers who can use the standard extensions to create new personalized extensions and share them with the rest of the community (figure 5-3). This layer of participation is added by democratizing the process of creating extensions, allowing everyone to extend the language to explore worlds in their own language to meet their own needs and interests. 66 I Many Scratchers -Imported through Personalized Extension Create with Personalized Extensions (or approved Scratch extensions) files (or from Scratch Extension Library) ( Some Scratchers 'Created using Scratch (using Create Personalize dI Extensions standard blocks or available extensions) *Shared through Personalized Extension files Create Standard Extensions Small group of developers -Created using JavaScript *Shared through Scratch Extension Library -Have to be approved and curated by the MIT Scratch Team Figure 5-3: The three levels of participation with Personalized Extensions. Derived from a graphic created by Natalie Rusk of the MIT Scratch Team to represent three levels of participation with extensions in the current version of Scratch (extension developers, extension testers, and extension users) 5.3 Guiding the Design of Personalized Extensions The MIT Scratch Team puts lots of thought and effort into the design of the Scratch programming environment and blocks to achieve several overarching goals that include: helping children become more fluent and expressive with new technologies; helping them explore important concepts through their expressive activities; and, most broadly, helping them become better learners [30]. The MIT Scratch Team has the power to make design decisions that support these goals, usually stressing the importance of having a "low floor" (easy to get started), a "high ceiling" (opportunities to create increasingly complex projects over time), and "wide walls" (supporting many different types of projects so people with many different interests and learning styles can all become engaged) [31]. However, some of this power is delegated to Scratchers as they get enabled to design and create Personalized 67 Extensions, and share them with the Scratch community; and with great power, comes great responsibility. As Scratchers create Personalized Extensions, they have to make decisions about the functionality they want to reveal or hide, which affects what ideas can be explored with the extension (and the associated kit). These design decisions affect how simple or overwhelming the extension is to use, an aspect that usually contests with the capabilities offered by the extension. The question of how to support the creators in making good design decisions remains an open area for exploration. 5.4 Personalized Extensions as Scratch Libraries While the Personalized Extensions were inspired by the need to make personalized explorations of the physical world more accessible, they do not have to be used solely for that. They do not even have to be modifications of approved extensions. A Personalized Extensions can generally be seen as an encapsulation of complex behaviors defined in Scratch. A Scratcher might build a gaming Personalized Extension, which has for example blocks that implement trajectories among other things (figure 54). Other Scratchers would see this extension as a gaming library and import it to use its blocks in building their games without worrying about the mathematics behind them. There would be others who would like to modify the implementation of those functions to meet different needs, and they would be able to do that by opening the Personalized Extension as a Scratch project instead of importing it. 68 - Figure 5-4: The definition of "launchwith velocity" custom block, which can be used to create a Personalized Extension (Based on code from "Gift Toss" project by Paddle2See http://scratch.mit.edu/projects/2238694/) 5.5 Empowering Science Explorations with Computational Tools After introducing Personal Extensions, a rich area to study is how teachers and students would use this tool to support their science explorations. Realizing that they have a computational tool to access the physical world, teachers are expected to start creating their own science kits that can be connected to Scratch and meet their needs in the classroom. With Personalized Extensions, the students would be able to focus on using the science kit in hand, which would be developed by their teacher, to explore various areas of interests without being overwhelmed by learning how to use the kits beforehand. It will also be interesting to see how the students will make use of having Scratch connected to their physical-world explorations. Scratch was designed to be more meaningful and more social than other programming environments. More meaningful means allowing people to create projects that are personally meaningful to them through supporting many different types of projects (stories, games, animations, simulations) and making it easy for people to personalize their projects by importing 69 photos and music clips, recording voices, and creating graphics. More social means building a community around Scratch where people can support, collaborate, and critique one another and build on one another's work [32]. These two aspects of Scratch combined with Personalized Extensions and the right hardware tools would allow for multiple pathways and representations (which correspond to different styles, interests, and backgrounds) of exploring the world around 19 and sharing these explorations with diverse audiences around the globe. 19 An example of personally-meaningful projects to explore the physical world is provided by the end of Chapter 3. 70 Improvements to Scratch's "Make a Block" The current interface for creating custom blocks in Scratch serves as a good basis to build Personalized Extensions. However, a couple of features should be introduced to make Personalized Extensions support blocks of different types. The first of them is supporting the creation of reporters: blocks that return values (figures A-1 and A-2). Scratch currently supports only the creation of command blocks. 71 touching I've reached howor ld the ceiling! 2 Figure A-1: Examples of Scratch reporter blocks and how they can be used Figure A-2: A mock-up of the definition of a reporter block Another feature is defining pull-down menus to be used as block parameters (figure A-3 and A-4). Currently, custom blocks can have numbers or strings as arguments. Restricting the value of an argument to the elements of a menu can simplify using the new block and eliminate the need to deal with unexpected values. 72 I New Block New Block . Options Add number input: Add string input: s Options Add number input: Add boolean input: Add string input: Add label text: Add boolean input: Add menu input: Add label text: 1 text W Reporter block Run without screen refresh I Run without screen refresh OK text Cancel Cancel OK (b) (a) Figure A-3: (a) The current window for creatinga new custom block (b) A mock-up of the window for creating new custom blocks thatsupport reporterblocks and using pull-down menus as parameters New Block 0 v Options Add n : Add string input: Add boolean input: text Add label text: Add menu input: * Reporter block O Run without screen refresh OK Cancel Figure A-4: A mock-up of defining a pull-down menu for a new custom block 73 II red '-. analog write pin value analog write pin Wvalue Cooanalog write pin value m analog write pin analog write pin W value IM value 40 (b) (a) Figure A-5: (a) A mock-up a script defining a new block using a pull-down menu as a parameter (b) the resulting new custom block 74 Workshop Materials I list here the tools and materials that were used in the workshops described in Chapter 4. The participants used the Scratch 2.0 offline editor 2 0 to work with abstract and personalized Scratch extensions for Arduino boards. The abstract extension was A4S (Arduino for Scratch), a preliminary HTTP extension for talking to Arduino boards running Firmata that is being developed by David Mellis from the Lifelong Kindergarten group. 21 The personalized extension was a modification of the A4S code (figure B-1). 2 2 The workshop participants had kits containing the following parts: 20 21 22 " Arduino Uno board " Breadboard - Jumper wires See http://scratch.mit.edu/scratch2download/ See https://github.com/damellis/A4S See http://bit.ly/A4Scratch 75 U' Resistor kit Assortment of LEDs Common cathode RGB LEDs Mini photocell Momentary Push Button Switch - 12mm Square Trimpot 10K with Knob TMP36 - Temperature Sensor Force Sensitive Resistor A4S (Scratch for Arduino) v eno conctlgh t inA cm button A4S (Arduino For Scratch) v Digital Input 2 set hI h digital write pin 3rt anlo digital read pie analog read pi 255DaueD pressed turn LED 2 on turn LED 2 off turn on turn off RGB RGB LED LED -0e m (a) 1 (b) Figure B-1: (a) The original A4S extension blocks (b) The personalized A4S extension blocks 76 The participants were provided with handouts (figures B-2 to B-9) that showed how to connect different components to the Arduino board and the associated Scratch blocks. The handouts were designed to be simple, created with software available for everyone, so teachers knew they could easily build support materials if they were to produce their own Personalized Extensions. The handouts were produced mainly using Google Docs (https://docs.google.com), and Fritzing (http://fritzing.org) was used to draw the breadboard view of the circuits. 77 H4 0 t M t 0 i - ~1i - - . - - -_- IT -. _ - ._ - Q .u . II + .1--------------------------------- --it1ii .- .l... ... 1 . j '. 717 - . Figure B-2: A handout for connecting a temperature sensor using the abstractA4S extension 78 I b ja" 0 j" { JI 4'! " 7 V.i " " " " " " " " " " " " " " " " " " " " " " " .... " Oka " " 000 " ........ ............... if iii T 9a ,. .0 t. t 1 J 'I 0 Figure B-3: A handout for connecting a temperature sensor using a personalized A4S extension " 79 e 44 o IW F -1- .. N _-I a., A fE-4. S FigureB-4: forconnecing handot apotentomete usinga peroaie 80 4 .._..... i E xeso 8 8 U I H I ,0 Ar 0t y, f - -~~ PT - YTh I :1 p lb. II c' s Figure B-5: A handoutfor connecting a photocell using a personalizedA4S extension 81 I U,~ V1 i rN 8 FF 0 1 7!CV 1~ sire - a III I 0 5 - " .w 3. _1 00 O v C O C d lI 8 K i I 0 I Figure B-6: A handoutfor connecting a force sensitive resistor using a personalized A4S extension 82 Ir ~LI ............ 4~-_. Ia!' " a j Il Q I. 1 i I Figure B-7: A handoutfor connecting a push button using a personalized A4S extension 83 .I I 0 8 1- --- 1~~ 0 ir 42 " ~U - t7---I ( - - - - ~ - - An - I - ill *a U m . - tV i }t~l:}Ej H ~ 'A Figure B-8: A handout for connecting an LED using a personalized A4S extension 84 I d 0 *1 0b Ike it ~ - jill _ "... _ 1111 _L _ .... -.-..--- - - 0 .r I _ -V. I 02 1R1 V M I 0 zztzx *1 - -- 4-- I t LiJ ~ ~K~< -- iiI a 0 vau -- ~-r- 1 111121.1 U V..A1......... III 1~1 ) E1L7 .711% -~ I *I----..--'--.--. M 0 Ia wzzt ~ a. iii Figure B-9: A handout for connecting a tri-color LED using a personalized A4S extension 85 86 References [1] Eisenberg, M. and Eisenberg, A. (2000). The Developing Scientist as Craftsperson. In N. Roberts, W. Feurzeig, and B. Hunter, eds. Computer Modeling and Simulation in Pre-College Science Education, NY: Springer-Verlag. [2] Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. [3] Resnick, M., and Ocko, S. (1991). LEGO/Logo: Learning Through and About Design. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing. [4] Papert, S. (1980). Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc.. [5] Resnick, M., and Ocko, S. (1991). LEGO/Logo: Learning Through and About Design. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing. [6] Lego Dacta. (1993) ControlLab Setup Guide and IntroductoryExplorations. Enfield, CT: Lego Systems, Inc. [7] Resnick, M., Martin, F., Sargent, R., and Silverman, B. (1996). Programmable Bricks: Toys to Think With. IBM Systems Journal,vol. 35, no. 3-4, pp. 443-452. [8] Resnick, M., Martin, F., Sargent, R., and Silverman, B. (1996). Programmable Bricks: Toys to Think With. IBM Systems Journal,vol. 35, no. 3-4, pp. 443-452. [9] Rusk, N., Resnick, M., Berg, R., & Pezalla-Granlund, M. (2008). New Pathways into Robotics: Strategies for Broadening Participation. Journal of Science Education and Technology, vol. 17, no. 1, pp. 59-69. [10] Resnick, M., Maloney, J., Monroy-Hernindez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. 87 T [11] Blanton, A. (2012, February 21). Physical-Digital Chain Reaction using WeDo Robotics with Scratch. Posted to: http://scratched.gse.harvard.edu/resources/physical-digital-chain-reactionusing-wedo-robotics-scratch [12] Brennan, K. (2009, October 3). We Do WeDo. Posted to: http://scratched.gse.harvard.edu/stories/we-do-wedo [13] Scratch 1.4 Source Code. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki: http://wiki.scratch.mit.edu/wiki/Scratch_1.4_Source_Code [14] Stern, T. I. (2007). NetScratch: a Networked Programming Environment for Children. MIT Masters Thesis [15] Eastmond, E. (2006). New Tools to Enable Children to Manipulate Images through Computer Programming.MIT Masters Thesis. [16] LEGO WeDo Construction Set. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki: http://wiki.scratch.mit.edu/wiki/LEGO_WeDo_Construction_Set [17] Jens. (2008, February 27). Chirp: A Community Edition Based On Scratch. Message archived at: http://scratchforums.blob8108.net/forums/viewtopic.php?id=3779 [18] Harvey, B. and M6nig, J. (2010). Bringing "No Ceiling" to Scratch: Can One Language Serve Kids and Computer Scientists. Proceedings of Constructionism 2010. (2010). [19] Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. [20] Maloney, J. (2013, September). Creating Scratch 2.0 Extensions [PDF document]. Retrieved from the Scratch Wiki: http://wiki.scratch.mit.edu/w/images/ExtensionsDoc.HTTP-9-11.pdf [21] Scratch Extensions. (2014). Retrieved August 26, 2014 from LLK GitHub repository: http://lk.github.io/scratch-extension-docs/ 88 [22] Scratch Extensions. (2014). Retrieved August 26, 2014 from LLK GitHub repository: http://lk.github.io/scratch-extension-docs/ [23] LEGO WeDo Construction Set. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki: http://wiki.scratch.mit.edu/wiki/LEGO_WeDoConstructionSet [24] Custom Blocks. (n.d.). Retrieved August 26, 2014 from the Scratch Wiki: http://wiki.scratch.mit.edu/wiki/CustomBlocks [25] Resnick, M., and Silverman, B. (2005). Some Reflections on Designing Construction Kits for Kids. Proceedings of Interaction Design and Children Conference, Boulder, CO. [26] Torrone, P. (2011, February 11). Why the Arduino Won and Why It's Here to Stay. Retrieved from Make Magazine blog: http://makezine.com/2011/02/10/whythe-arduino-won-and-why-its-here-to-stay/ [27] Turkle, S., & Papert, S. (1991). Epistemological Pluralism and the Revaluation of the Concrete. In Constructionism, edited by I. Harel & S. Papert. Ablex Publishing. [28] Resnick, M., and Rusk, N. (2009). Sensing 2.0: Integrating Sensor Data with Interactive Media to Engage a Broader Audience in STEM Learning. Unpublished project proposal. [29] Mellis, D. A. (2014). A4S (Arduino For Scratch) [code repository]. Available from https://github.com/damellis/A4S [30] Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. [31] Resnick, M., Maloney, J., Monroy-Hernandez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. [32] Resnick, M., Maloney, J., Monroy-Hernindez, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., and Kafai, Y. (2009). Scratch: Programming for All. Communicationsof the ACM, 52:60-67. 89 90