Using LEGO MindStorms in CS 1 and 2
LMICSE Workshop at SIGCSE 2006
March 4, 2006
Who are We? The LMICSE Project
•
LMICSE: Lego MindStorms in Computer Science Education
•
Goal: Development of MindStorms based materials for use
throughout the undergraduate CS curriculum
•
Principal Investigators:
•
•
•
•
Frank Klassner (Villanova University)
Pamela Lawhead (University of Mississippi)
Myles McNally (Alma College)
Website: http://www.mcs.alma.edu/LMICSE/
Workshop Schedule
•
•
•
•
•
•
Introductions
Overview of the Current MindStorms Platform
Using MindStorms in CS 1
•
Using MindStorms in a Closed Lab Environment
•
Overview of the Turtle Sequence
Using the BlueJ IDE with MindStorms
Using MindStorms in CS 2
•
GridWalker Module Overview
•
Random Walk Lab
•
Wavefront Propagation Lab
A Glimpse at Lego MindStorms NXT
The Current MindStorms Platform
LMICSE Workshop at SIGCSE 2006
March 4, 2006
The Lego MindStorms Kit
•
•
•
•
•
•
•
•
The RCX (the brains)
Two Geared Motors
Two Touch Sensors
One Light Sensor
A USB IR Tower
A CD-ROM
The Constructopedia
Lots of Lego Pieces!
RCX Brick and Standard
Sensors and Motors
A Basic Robot Design
•
•
The Constructopedia is a building
guide.
Shown at left is the basic Roverbot
from the Constructopedia
•
A robust design
•
Allows attachment of any of
• double bumper (shown)
• wheel sets (shown)
• single bumper
• light sensor
• tank like treads
•
A recommended design starting
point
Basic Roverbot Configuration
The RCX Itself (1)
•
•
•
•
•
•
•
Hitachi H8 series microcontroller
8 bit CPU at 16 MHz
32 KByte RAM
16 KByte ROM for the BIOS
Serial I/O (input/output)
ADC (analog digital converter)
Built-in timers
RCX Brick
The RCX Itself (2)
•
•
•
•
•
•
•
Liquid crystal display (LCD)
Four push buttons
One small internal speaker
Three sensor inputs
Three motor outputs
One infrared (IR) interface with a
proprietary protocol
Requires 6 AA batteries
RCX Brick
Actuators
•
•
•
With Kit
•
Geared Motors
Available From Lego
•
High-speed Motors
•
Mini-motor
•
Light Brick
Available third party
•
Servo Motor/Servo Controller
Servo Controller w/
2 Servo Motors
Ungeared and
Geared Motors
Sensors Basics
•
MindStorms sensors come in two types:
•
Active: RCX supplies power
• Example: Light sensor (supplied with kit)
•
Passive: RCX just detects changes in resistance
• Example: Touch sensor (supplied with kit)
Sensors Available from Lego
•
•
•
Touch
•
Passive
•
Supplied with Kit
Light
•
Active
•
Supplied with Kit
Rotation (Angle)
•
Active
•
Add-on
Touch, Light and Rotation Sensors
Third Party Sensors (1)
•
The sensor interface is well documented
•
Many sets of building instructions for sensors exist, for example
see Michael Gasperi’s website
•
http://www.plazaearth.com/usr/gasperi/lego.htm
•
Two commercial suppliers we know of:
•
http://www.mindsensors.com
•
http://www.hitechnic.com
Third Party Sensors (2)
•
•
•
•
Sonar (Ultrasonic range)
•
Available both from
Mindsensors and Hitechnic
Magnetic Compass Sensor
•
Available from Hitechnic
Active and Passive Multiplexer
•
Available from Mindsensors
And many others
Mindsensors Active Multiplexer
Hitechnic Sonar
Hitechnic Compass
Software Architecture
Typical Software Architecture Levels
Software Architecture
•
System ROM Layer
•
Low Level Control Software
•
Allows a second piece of software - the firmware
•
Can be bypassed for direct control of the hardware
Lego Firmware
•
•
•
•
The Lego firmware provides an environment for the execution of
programs on the RCX.
Must be downloaded (just once) before programming the RCX.
Other firmware solutions are available.
Lego Firmware v. 2.0 permits
•
5 separate programs
•
10 threads
•
32 global variables
•
16 local variables per thread
•
Broadcast IR protocol
Alternative Firmware
•
•
•
Firmware replacements further push the limits of the RCX
Popular replacements are tied to alternative programming
environments
Examples
•
LejOS (Java)
• see http://lejos.sourceforge.net/
•
BrickOS (C/C++)
• see http://brickos.sourceforge.net
•
RCXLisp (Lisp)
• Part of the LMICSE project
• See http://robotics.csc.villanova.edu/
Using MindStorms in CS 1
LMICSE Workshop at SIGCSE 2006
March 4, 2006
The Turtle Laboratory Sequence
•
•
Designed for teaching programming fundamentals
Based around the “Turtle,” a LOGO-like abstraction from the
details of motors and sensors
A Turtle
The Turtle Laboratory Sequence
•
Original work was done by Scott Anderson (Wellesley) with
support from Frank Klassner (Villanova)
•
Coded the Turtle software
•
Created drafts of seven CS 1 oriented labs
•
Current work by Myles McNally (Alma)
•
Simplified somewhat the Turtle software
•
Twelve labs oriented towards CS 1
•
Each lab was used in a fall, 2004 CS 1 course
•
Each lab has now been refined based on that
experience and videos added
The Turtle’s Basic Abilities
•
•
•
Motion
•
Move forward or backward (timed or continuous)
•
Turn left or right (timed or based on degrees)
• Can calibrate turning time required to turn n degrees
•
Stop
Events
•
Can respond to events
•
Simple event queue model
• NONE (queue is empty), RIGHT, LEFT, BOTH, VIEW,
PGM, RUN
Singing
•
Can play a tone for a duration
The Physical Turtle (1)
•
Many designs could be used
•
Requirements are
•
Differential drive
•
Left motor in motor port A
•
Right motor in motor port C
•
Left and right front mounted bump sensors
• Left touch sensor in sensor port 1
• Right touch sensor in sensor port 3
The Physical Turtle (2)
•
The design we recommend is based on designs from the
Constructopedia
•
The Roverbot Driving Base (p. 12)
•
The Wheel Sets (p. 17)
•
The Double Bumper (p. 30)
The Standard Roverbot
The Physical Turtle (3)
•
•
Then make the following modifications
•
Remove the front (smaller) set of
wheels
•
Add a slider to the bottom of the
bot between where the front
wheels had been
•
Move the RCX itself further back
on the base, so that its weight is
more centered over the remaining
wheels
These changes improve the
exactness of turns
The Turtle “top and bottom”
The Current Turtle Laboratories
•
•
•
Java-based (in particular LeJOS)
Covers topics found in a modern, object oriented CS 1 course
•
Basic types and expressions
•
Flow of control
•
Classes and methods
•
Arrays, stacks and lists
•
Interfaces
•
Inheritance and abstract classes
•
Polymorphism
Loosely follows the topic order in the Lewis and Loftus text, but
could be used with almost any object oriented text.
Turtle Laboratory Topics (1)
•
•
•
•
•
Sequential Control: Use straight-line code and a "Turtle" robot to
move in a few geometric figures.
Variables and Expressions: Use more advanced code and
variables to create more interesting shapes.
Methods: Use methods to separate code into parts and also use
the Random class.
Methods with Parameters / Scope: Use methods with
parameters and returns, instance variables, and the Math class.
Classes: Define a class that allows musical notes and rests to be
represented and played back by the RCX.
Lab 4, Task 5
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
Right Angle Random Patrol
Turtle Laboratory Topics (2)
•
•
•
•
Event Driven Programming: Work with the basics of event
driven programming and focusing on using decision structures to
respond to events.
Loop Control Structures: Work with each of the loop control
structures in Java in the context of event processing.
Using Interfaces: Define interfaces then implement them to run
races and to draw figures with the robots.
Array Structures: Use arrays to record inputs from the user and
then traverse a course using the recorded values to know when to
move and when to turn.
Lab 6, Task 3
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
Use Events to Drive the Turtle
Turtle Laboratory Topics (3)
•
•
•
Navigation: Implement a navigation interface which allow the
Turtle to go to positions in its world, then input a series of
positions and have the Turtle visit them.
Inheritance: Define several classes that handle notes and rests,
and an abstract class that each is an extension of.
Sorting and Polymorphism: Use polymorphism with various
sorting algorithms.
Lab 12, Task 2
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
Travel to Points Sorted by Horizontal Position
Programming the Turtle: LeJOS
•
The basic programming environment is LeJOS
•
an open source project
•
based on a subset of Java with extensions for the RCX
• missing Java features hard to predict (i.e., no switch
statement)
•
contains some advanced robotics libraries
• navigation (odometry)
• behaviors (subsumption architecture)
• image processing (vision)
•
available at lejos.sourceforge.net
Where to Learn More about LeJOS
•
The LeJOS Tutorial: an online resource which is part of the LeJOS
project (on their website).
•
The LeJOS PowerPoint at the LMICSE site
•
•
http://www.mcs.alma.edu/LMICSE/Workshops/alma2/powerpoin
ts/LejosProgramming.ppt
Books such as:
•
Core Lego MindStorms Programming, by Brian Bagnall
(Prentice-Hall, 2002),
•
Programming Lego MindStorms in Java, by Giulio Ferrari, et al
(Syngress, 2002),
•
and many others.
Programming: The Turtle Abstraction
•
LeJOS provides the ability to directly control motors and poll
sensors
•
The Turtle abstracts from these low level routines, allowing users to
direct the Turtle move forward, turn, and register events.
•
This results in simpler, more readable programs.
An Example LeJOS Program
•
Notice the use of low level control
import josx.platform.rcx.*;
public class Patrol {
public static void main(String[] args)
throws InterruptedException {
Motor.A.forward();
while (true) {
Motor.C.forward();
// go forward
Thread.sleep (5000);
Motor.C.reverse();
// turn around
Thread.sleep (1000);
}
Motor.A.stop(); Motor.C.stop();
}
Patrol back and forth (v. 1)
The Corresponding Turtle Solution
•
Notice the simpler, more direct coding solution
public class Patrol {
public static void main(String args[]) {
while (true) {
Turtle.forward(2000);
Turtle.turn(1000);
}
Turtle.stop();
// go forward
// turn around
// stop all motors
}
}
Patrol back and forth (v. 2)
Using Calibration
•
Once calibrated, the Turtle can do degree-based turns
public class Patrol {
public static void main(String args[]) {
Turtle.calibrateTurn(360,3000);// calibrate
while (true) {
Turtle.forward(2000);
// go forward
Turtle.left(180); // turn around
}
Turtle.stop();
// stop all motors
}
}
Patrol back and forth (v. 3)
Using the Turtle Event Model
•
Use the nextEvent() method and the Turtle event constants
public class MoveForward {
public static void main(String args[]) {
Turtle.forward();
//
int event = Turtle.nextEvent();
while (event != Turtle.RUN){ //
event = Turtle.nextEvent(); //
}
Turtle.stop();
//
move forward
until RUN button
is pressed
stop all motors
}
}
Move forward until the RUN button is pressed
Using the BlueJ IDE with MindStorms
LMICSE Workshop
June 14 - 17, 2005
Alma College
What is BlueJ?
•
•
•
•
•
Freeware developed by Michael Kolling and others
A popular integrated development environment for Java
Often used in introductory CS courses
Available at www.bluej.org, easy to install
Go ahead and open BlueJ on your PC
Setting up BlueJ for MindStorms
•
•
Two requirements:
•
Install LeJOS
• Available from lejos.sourceforge.com
• Just do the basic install
•
Add the BMJT plugin to BlueJ
• Available from ddi.uni-paderborn.de/mindstormstools/bjmt
• Good instructions available online
These steps are already done for you on your PC!
Once the BMJT Plugin is installed
•
•
set the path to
the JDK
You need to configure it
This is already done for you on your PC!
choose
configure
there is a 1.5 option
in newer versions
set the path to
LeJOS
just say usb
Firmware
•
•
•
•
•
•
Now you can download the firmware to the RCX
Plug the tower into a usb port
•
if on a PC you may need to install the usb drivers
Place the RCX in front of the tower, turn it on
Choose Firmware
You will see a progress dialog on the PC
On the RCX you will see an ascending series of numbers
choose
firmware
Compiling an Existing File
•
•
Right-click (Command-click) on the file icon
Choose Compile from the pop-up menu
choose
compile
not this
one!
Downloading a Project
•
•
•
Robot must be on and in front of tower
Right-click (Command-click) on the file icon for the main class
Choose Download from the pop-up menu
choose
download
Try it Out!
•
Place the robot in its play area.
•
Press run.
•
It should move forward until it hits an obstacle.
Hands-on Time!
•
Exercise 1: Modify the GoForward program so that the Turtle moves
forward, turning away from obstacles when necessary. That is,
when the left bump sensor is pressed, back up and turn right, and
likewise for the left bump sensor.
•
Exercise 2: Craft a program in which you enter a number by
pressing the right bumper that many times, and then have the Turtle
travel around in a square shape that many times. Signal the end of
your input by pressing the left bumper. The Turtle should then begin
to move.
•
Or do any of the lab exercises from the LMICSE CS 1 area.
Using MindStorms in CS 2
LMICSE Workshop at SIGCSE 2006
March 4, 2006
The GridWalker Labs
•
•
•
Introduce AI themes in CS 2:
•
robotic systems
•
graph searching algorithms (without
the graph)
•
occupancy grids and path planning
Use basic user defined data structures
like stacks and queues
Reinforce material learned in CS 1
such as
•
classes and inheritance
•
two dimensional arrays
•
exceptions
GridWalker Robot with Sonar
The Projects
Random Walk (and Back)
Wavefront Propagation
Classical Search Algorithms
Adding Sonar
Project Dependencies
Occupancy Grids
•
An occupancy grid is a discrete grid that represents the
environment.
•
Each cell in the grid is either
•
occupied (part of an obstacle).
•
empty (part of free space).
•
If any part of an obstacle is in a grid cell, then the cell is
considered to be occupied.
•
Grids representing real world environments are often fine grained
(cells representing 4” to 6” square area).
Occupancy Grids
A 14 by 8 grid
Black figures are the obstacles
Gray areas are the occupied cells
Basic GridWalker Environment
Wood Blocks Mark Occupied Cells
Project 1: A Random Walk (and Back!)
•
The robot goes for a random walk on an occupancy grid
containing no obstacles.
•
Then retraces its steps back to the starting point.
•
Basic strategy is to use a stack:
•
At each point the robot moves one cell north, east, south, or
west (if legal).
•
On the way out, push the directions traveled onto the stack.
•
On the way back, pop the stack and travel in the opposite
direction.
GridWalker in Action
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
Overall Solution Architecture
BlueJ Class Diagram for Project 1
The Grid Walker Class
•
The heart of this project is the definition of the GridWalker class. It
contains:
•
a constructor that specifies the grid size and the starting point of
the robot
•
a goto method that takes as a parameter a grid location and
causes the robot to move from its current grid position to that
location
•
convenience methods north, east, south, and west that move the
robot one cell in the respective direction.
Implementing the GridWalker
•
Three possibilities:
•
Implement the GridWalker directly, using timing for distances
and turns
• but timing depends on battery strength, so is inexact over
time
•
Use one of the two LeJOS navigation classes
• Timed Navigator: same problem as above
• Rotation Navigator: this is the ticket!
• Requires two rotation sensors
• Details in paper
The Rotation Sensor
•
•
•
•
Thread an axle through the sensor
One revolution of the axle is 16 clicks of the sensor
So it can measure a changes in angle of 22.5 degrees
Which is why it is also known as an angle sensor
A Rotation Sensor
Mounting Rotation Sensors
• Here is a solution that is a simple modification of the basic
Roverbot:
Rear View of Roverbot Chassis
The LeJOS Rotation Navigator Class (1)
•
Rotation Navigator implements the Navigator interface
•
Important Navigator methods:
•
public void gotoPoint(float x, float y)
• Rotates the RCX robot towards the target point and moves
the required distance
•
public float getX()
• Returns the current x coordinate of the RCX.
•
public float getY()
• Returns the current y coordinate of the RCX
•
public float getAngle()
• Returns the current angle the RCX robot is facing
The LeJOS Rotation Navigator Class (2)
•
•
assumes differential drive, with rotation sensors (encoders) for the
left and right sides in sensor ports 1 and 3.
the constructor:
public RotationNavigator(float wheelDia, float AxleWidth, float ratio)
the
diameter
of a wheel
the distance from the
center of the left
wheel to the center of
the right wheel
the ratio of
encoder
revolutions to
axle revolutions
Grid Walker Implementation (1)
•
Define GridWalker to extend RotationNavigator
•
Have instance variables for
•
•
initial grid position (x and y)
•
current grid position (x and y)
•
the number of columns and rows in the grid
•
the size of a grid cell (assume it is square)
The constructor is passed this information, plus the information the
RotationNavigator class needs
Grid Walker Implementation (2)
import josx.robotics.*;
public class GridWalker extends RotationNavigator {
int currentH, currentV, initialH, initialV;
int dimensionH, dimensionV;
int cellSize;
public GridWalker(int cH, int cV, int dimH, int dimV, int cSize,
float wDia, float wBase, float ratio) {
super(wDia, wBase, ratio);
initialH = currentH = cH;
initialV = currentV = cV;
dimensionH = dimH;
dimensionV = dimV;
cellSize = cSize;
}
The beginning of the class definition and the constructor
Grid Walker Implementation (3)
•
In implementing the the goto method:
•
need to check that the new location is legal (i.e., in the grid)
• if not, throw an exception (optional, but a good opportunity to
reinforce the use of programmer defined exceptions)
•
need to convert grid locations to absolute locations
• RotationNavigator assumes the robot starts at 0,0 with an
orientation of 0 degrees
• GridWalker allows the programmer to specify any cell as the
starting location, but still assumes orientation of 0 degrees
Grid Walker Implementation (4)
public void gotoCell(int h, int v) throws OutOfBoundsException {
if (h >= 0 && h < dimensionH && v >= 0 && v < dimensionV) {
gotoPoint(gridHToPoint(h), gridVToPoint(v));
currentH = h;
currentV = v;
} else
throw new OutOfBoundsException(h, v);
}
private float gridVToPoint(int n) {
return (n - initialV) * cellSize;
}
private float gridHToPoint(int n) {
return (n - initialH) * cellSize;
}
The goto method and helpers
Remaining Implementation Details
•
Add the north, east, south, and west methods to the GridWalker
class
•
Implement a stack of ints (or Objects - Integers)
•
Then in the main class
•
First (random walk)
• Generate a series of random ints between 0 and 3
• Travel in that direction and push on stack
•
Second (travel back)
• Pop the stack until empty, traveling in the appropriate
direction
Project 2: Wavefront Propagation
•
Design a program that will move the robot from a starting point to a
goal point
•
using the shortest possible path
•
navigating around obstacles
•
only moving north, east, south, or west (Manhattan movement)
•
Obstacle locations are known in advance
Propagating the Wavefront
•
•
•
Image a wave moving out from the goal cell.
When the wave first reaches each of the other cells, it is labeled
with the time it took to reach it.
The wave moves Manhattan
Propagating the Wavefront (2)
• If there are occupied cells, the wavefront simply flows around
them.
Propagating the Wavefront (3)
•
The path from any cell to the goal is implicit in the grid labels:
•
Until at the goal, move to an adjacent cell with a “smaller” label
•
There may be many different equal length paths
Overall Solution Architecture
BlueJ Class Diagram for Project 2
Propagating the Wavefront (3)
•
Initialisation
•
Allocate a new two dimensional array D the size of the
occupancy grid.
•
Set cells in D which correspond to occupied cells to high values
and all others to -1.
•
Set the value of the goal cell to 0.
•
Enqueue the location of the goal cell (a point).
•
While (queue is not empty)
•
Dequeue a point L.
•
Find the value V associated with point L in D.
•
Find the neighbours of L with a -1 value.
•
Set the values of these neighbours to V+1.
•
Enqueue each of these points.
Wavefront Propagation Implementation (2)
public void setValues (Queue q, int [][] grid) {
q.enqueue(new Point (goalX, goalY));
grid[goalY][goalX] = 0;
while ( !q.empty() ) {
Point currentP = (Point) q.dequeue();
int x = currentP.x;
int y = currentP.y;
int newX = x+1; // go east
if (newX < grid[y].length && grid[y][newX] == -1) {
grid[y][newX] = grid[y][x] + 1;
q.enqueue(new Point(newX,y));
}
}
// also need cases for the other three directions
}
The base method for setting the wavefront values
Wavefront Propagation Implementation (3)
public void travel () {
int curX = startX;
int curY = startY;
while (curtX != goalX || curY != goalY)
int curValue = grid[curY][curX];
{
if (curX+1 < grid[0].length && grid[curY][curX+1] == curValue - 1)
curX = curX + 1;
// move to the east
else
// cases for the other three directions
robot.goto(curY, curX);
}
}
Making the robot move to the goal location
Project 3: Adding Sonar
•
•
Begin with an occupancy grid in which all cells are marked free
Dynamically recompute the route as the robot encounters obstacles
during its travel
40 khz sonar sensor from Mindsensors.com
Sonar Basics
•
Sonar sensors are often used in robots for obstacle avoidance,
navigation and map building
•
Sonar sensors work by
•
•
•
emitting a short burst of ultrasonic sound (often 40 khz)
sensing reflected signals (if any)
computing object distance by using the elapsed time
Basic Solution Approach
•
Initialisation
•
Perform the wavefront labelling routine using an occupancy
grid with all cells marked free.
•
Do
•
Determine a direction to travel.
Orient robot in that direction.
•
Use sonar to check if next cell in that direction is occupied.
• If it is, update occupancy grid and rerun labelling routine.
• If it is not, move to that cell.
while (goal not reached and the cell label not -1)
•
QuickTime™ and a
MPEG-4 Video decompressor
are needed to see this picture.
Hands-on Time!
•
Exercise 1: Try out the Random Walk program. Then modify it so
during the random walk phase the robot never turns around and
goes back to the spot it just occupied, e.g., never goes south after
just having gone north.
•
Exercise 2: Try out the Wavefront program. Then modify it so that
the robot randomly chooses among the next legal locations during
its travel. That way it should travel different routes each time it
moves from the starting point to the finish.
A Glimpse at Lego MindStorms NXT
LMICSE Workshop at SIGCSE 2006
March 4, 2006
The Future of Lego MindStorms
•
Lego MindStorms NXT
•
Announced January, 2006
•
Will start shipping August, 2006
The NXT Intelligent Brick
•
•
•
•
•
Dual processor unit with more
memory
•
32 bit processor
• 256 Kbytes FLASH
• 64 Kbytes RAM
•
8 bit processor
• 4 Kbytes FLASH
• 512 Byte RAM
4 input ports / 3 output ports
Matrix display
Real sound speaker
USB 2.0 and Bluetooth®
Motors and Sensors
•
•
•
•
•
•
3 Interactive Servo Motors
•
Rumored to have 1 degree sensitivity
Ultrasonic Sensor
Sound Sensor
Improved Touch Sensor
Improved Light Sensor
6-wire cords
Supplied Programming Software
•
•
•
Icon-based drag and drop program development environment
PC and Mac compatible
Based on LabVIEW from National Instruments
Need to Know More about NXT?
•
Check out the Lego site
•
http://mindstorms.lego.com
•
Read what the bloggers are saying
•
http://www.bnxt.com/community/news/
Thank you for Attending Our Workshop!
•
Acknowledgements
•
This work was partially supported by a United States National
Science Foundation CCLI grant DUE-0126494
•
Thanks to the numerous students have worked on the LMICSE
project
•
Please consider attending our upcoming 3-4 day workshops
•
This summer at Villanova University
•
This fall at the University of Mississippi