Matt LeGrand
who are you and why are you in my kitchen?
Matt LeGrand
Rich Internet Application developer primarily working in Flex since 2005
and flash since 1999. In a former life I was a bit of a jogger. Mostly left
foot then right foot, but sometimes right foot then left foot.
NerdWarfare Inc.
NerdWarfare builds Rich Internet Applications (RIAs) and provides web
development and consulting services based on the latest Adobe tool
sets.
Multitouchup.com
Video tutorials and blog about multitouch on the flash platform. Links to
helpful tools for multitouch development and an overall great place to
peruse while enjoying your morning coffee.
simtouch project
Simulate touch events
Uses an Air application to simulate unlimited touch points over a socket
server (UDP).
Sim Connect
Required swc and class reference.
OpenSource details
The project is open source and is located at http://code.google.com/p/
simTouch
what’s in this talk for me
Multitouch for Flash 101
A bit about the past and a quick introduction to new multitouch events
and when to use them.
Coding Tips and Tricks
We'll look at a few code snippets showing common problems unique to
multitouch and how to quickly overcome them.
Tools, Hardware and Resources
We'll quickly look at a few tools for developing and testing multitouch
applications.
Google Trends for ‘Multitouch’
I thought Steve Jobs invented multitouch
These guys never worked at Apple
Nimish Mehta
1982 Flexible Machine Interface seems
to be the first true multitouch device.
Myron Krueger
1983 Video Place / Video Desk - vision based system that tracked the
hands and enabled multiple fingers, hands, and people to interact using
a rich set of gestures.
Bill Buxton
University of Toronto, Microsoft Research
Recent spike
Jeff Han 2006
Gave a Ted Talk in 2006 using a FTIR multitouch table.
http://www.perceptivepixel.com/
Apple iPhone 2007
released in May of 2007
Microsoft Surface 2007
released in November of 2007
Jeff Han’s Ted Talk
Evaluate single touch vs multitouch
Touting benefits of multitouch
Study out of UC Berkeley discovered that a multitouch workstation was
about twice as fast as the traditional mouse-based workstation at
performing multi-selection tasks.
Sorting with the mouse
The most significant increase in performance was seen when going
from the mouse-based work-station to the single finger work station.
That alone accounted for 83% of the performance improvement.
Twice as fast
Details: http://multitouchup.com/study-shows-multitouch-is-abouttwice-as-fast-as-mouse/
UC Berkely Study Results
Flash player 10.1 and AIR 2
New Multitouch Class
flash.ui.Multitouch
Minor changes to Capabilities
flash.system.Capabilities
Two new major events
TransformGestureEvents and TouchEvents
Gesture events
flash.events.TransformGestureEvent
Types:
GESTURE_PAN
GESTURE_SWIPE
GESTURE_ROTATE
GESTURE_ZOOM
Important event attributes:
offsetX and Y
scaleX and Y
rotation
gesturePan
gestureRotate
gestureSwipe
gestureZoom
Other Gesture events
flash.events.PressAndTapGestureEvent
GESTURE_PRESS_AND_TAP
The event is dispatched when a user has a finger down on an
object and taps it with another finger.
Important event attributes: tapLocalX and Y, tapStageX and Y
flash.events.GestureEvent
GESTURE_TWO_FINGER_TAP
This is the base class for the previous gesture classes.
Important event attributes:
GesturePhase which gives insight into the gestures progress.
(begin, update, or end)
Getsture event trace
Getsture example
Getsture example
Getsture example code
Raw Touch event
flash.events.TouchEvent
This event is similar to the standard MouseEvent.
Types include:
TOUCH_BEGIN
TOUCH_END
TOUCH_MOVE
TOUCH_OUT
TOUCH_TAP
TOUCH_OVER
TOUCH_ROLL_OUT
TOUCH_ROLL_OVER
Limitations
This class is currently only accessible by the Windows 7 operating
system, iPhone and Android. Not available on OSX.
Touch event trace
Touch event code
Touch event example
Cross platform you say?
Apple’s uni-body trackpad
Apple's uni-body Mac book track pad only supports gesture events.
The developer does not have access to touch point x/y positions.
Windows 7 hardware
With Windows 7 and capable hardware you can listen for raw touch
events or gestures.
iPhone, Android and the Playbook
Like windows 7, mobile devices have access to raw touch points or
gestures.
We’ll need to double check that
flash.ui.Multitouch
The Multitouch class is a new public final class. It helps manage the
current environments multitouch support.
Example Usages
Multitouch.supportsGestureEvent (Boolean)
Multitouch.supportsTouchEvent (Boolean)
Multitouch.maxTouchPoints (bug in Android)
Multitouch.supportedGestures (Vector of Strings)
Capabilities.touchscreenType (TouchscreenType.FINGER, STYLUS, NONE)
Capabilities.touchscreenType (TouchscreenType.FINGER, STYLUS,
NONE)
Double check
Multitouch input mode
Multitouch.inputMode
Default is set to GESTURE.
Required for raw touch events.
Toggle at runtime.
Example Usages
Multitouch.inputMode = MultitouchInputMode.GESTURE;
Multitouch.inputMode = MultitouchInputMode.TOUCH_POINT;
Switch Multitouch modes
When to use what event
Click Events
If you don't change any of your code tap events get mapped over to
click events and your application should just work. Avoiding the
multitouch apis is also more efficient for flash.
Touch Events
Touch events give you raw x/y coordinates and allow you to have a
fine grain to your animations.
Gesture Events
Gesture events are easy to implement but vary by operating system
and you lose a bit of control over how your application will respond.
Use raw touch events if possible
Small suggestions
Roll your own gestures
If possible, use the raw touch points even if you are reacting to a
gesture.
Use Physics
Users expect digital content to be thrown, bounced and slid.
Avoid Flex
The spark.core.SpriteVisualElement seems to be responsive. Avoid
nested classes. Consider stopping event propagation. Consider Reflex
(http://reflex.io/) for more responsive applications.
Other big ideas to keep in mind
Where is the user?
In many multitouch surfaces the user might approach your application
from a variety of angles.
Size Matters
How many people are going to be interacting with your device at the
same time? 32' seems large but if you have 2-4 people in front of the
single screen you may need to rearrange your UI
Native code or javascript
Android, Cocoa touch and javascript all have strong multitouch apis.
(there are some cross browser issues with javascript currently)
Hardware list
Microsoft Surface $7k
Three items of interest
HP Touch (N-Trig)
TouchSmart 600t, TouchSmart TX2, L2105tm monitor.
Get them 'on the cheap' but they are dual touch input and slow
response times.
Microsoft Surface v2
The new Surface made by Samsung is thin and has an estimated price
tag of $7k.
3M 32 inch monitor
3M announced at CES that they'll be shipping a low latency 32"
multitouch monitor capable of 40 plus touch inputs. Estimated price tag
of $2500
3M releasing this summer est. $2500
Capacitive touch screens
Surface Capacitance
One side of the insulator is coated with a conductive layer. A small
voltage is applied to the conductive layer, resulting in a uniform
electrostatic field. When a conductor, such as a human finger, touches
the uncoated surface, a capacitor is dynamically formed.
Projected Capacitance
PCT technology is a capacitive technology which permits more accurate
and flexible operation, by etching the conductive layer. An X-Y grid is
formed either by etching a single layer to form a grid pattern of
electrodes
DIY and large MT displays
FTIR
Frustrated total internal reflection
LLP
Laser Light Plane
DI
Diffused Illumination (DI)
Contact info
Matt LeGrand
mlegrand@gmail.com
@matt_legrand (twitter)
http://github.com/mlegrand
NerdWarfare Inc.
http://nerdwarfare.com
Multitouchup.com
http://multitouchup.com
@multitouchup (twitter)