FROM WIMP AND SILK
TO SMILE
Exploring emerging interaction technologies
Today’s topics
• Understanding the rapid change of technology
• Engaging in the Jurassic Park Test
• Exploring some of the many new (=current) ways to
interact with technology
The world changes fast
• Creative destruction (Schumpeter)
• One economic development arises out of the destruction of an older
• This goes for technology too
• How many buy music? (CDs)
• How many rent a video? (in a store)
• We are moving from “hardware” (CD) to “software” (iTunes)
to “services” (Spotify for music, Netflix for movies)
• My first USB 2.0 memory stick was 128 MB
• You can barely store more than the sound of “air” in good quality there
The web changes fast
• From being
• World Wide Web
• You sit by a home computer/laptop and browse web pages,
using your keyboard and a mouse – or through your mobile
• …we now have
• A Web-Wired World
• Where you always are connected
• Where you interact in many new ways
• Where you (and your phone) constantly transmit information
• Fine, there are changes…
• How can we exploit then for building better/more modern systems?
Before we SMILE
• Let us have a look at some technology changes
• From “fatscreen” to “flatscreen” (change h/w <-> h/w)
• From “local storage” to “cloud storage” (change h/w <-> service)
• From “applications” (office, email) to “services (s/w <-> services)
• Office 365, Google Docs, Dropbox
• Changes come fast, and old technology is forgotten
• READY TO DO THE JURASSIC PARK TEST?
Jurassic Park test
• Which of you know what happens when pressing this?
• Which of you know what this symbol represents?
• Those of you with your hands up, kindly pick up your
SENIOR CITIZEN CARD when you leave this room...
• ”John”, 13
”...and then you have this symbol for
saving. I don’t really know what it is but
it is sort of square, with a somewhat
bigger white area in the upper half...”
More predictions
• Remember this!
• (Dialogue in 15 years)
-
-
Daddy, have you ever used
a keyboard?
Sure I have!
Why did you do that...?
• This is the next generation,
after you (watch out):
• Anna, 2 years finds a glossy magazine.
She immediately tries to ”change page”
by swiping…without success.
• She hands over the magazine to her
mother, with eyes saying ”this device is
not working…”
Changes in paradigms
• The 1980’s: WIMP
• Windows – Icons – Menus – Pointers (or point-and-click)
• We have lived by this paradigm for 35 YEARS!!!
• The mid 1990’s: SILK (Reddy)
• Envisioning what would lie ahead
• Speech – Image – Language – Knowledge
• We have, in part, seen S-I-L, but don’t see K (back-end)
• Now: we must include all the new ways of interaction
which are possible with mobile phones, tablets, consoles
• You should start to SMILE
SMILE
• In an effort to cover most of the “modern” – and by all
means, also traditional – modes of interaction we coined
• Speech
• Speaking to technology, and be spoken to by it
• Movement
• Mouse…ok, now swiping but also hands (gestures)
• Image
• Not only “photos”: “live” avatars, QR codes, face recognition
• Language
• Translations, and also “emotional” language
• Environment
• You, and your devices, interact…constantly
SPEECH
• Speech recognition: speak to your phone (SIRI; not very new)
• Speech generation: TTS – text to speech – units speak to you
• Emotions are also a language (MIT – Kismet – 1997)
• http://www.youtube.com/watch?v=8KRZX5KL4fA
• WAMI – speak to the Internet
• http://groups.csail.mit.edu/sls/research/vehicle.shtml
• Kinect – SpeechRecognitionEngine
• http://www.youtube.com/watch?v=9lePgTpjRto
• Experiments (Spring 2013): Stroop effect (WAMI och Kinect)
• Classic psychology experiment: say the COLOR of the TEXT
• BLACK BLUE RED BLUE RED GREEN BLACK BLUE BLUE BLACK
Motion/movement
• Catch movements in 2D, 3D
- Move a mouse pointer, swipe a tablet
- New ways to communicate (mostly 3D)
- Sensor fixed, object moving (Kinect)
- Sensor moving, object fixed (”scanning”, e.g. with Kinect)
- Major movements (as GPS in mobile) is dealt with later
• Experiments (2013)
• Connect the dots
• Kinect Fusion
Image
• Basicly: image analysis (detect patterns)
• Image filtering (blue-screen)
• Combine with scanning (interpret ”the surrounding”)
• Beyond ”flat” image: 3D/holograms, as here (but not really…)
• DVE: http://www.youtube.com/watch?v=Wl9og1O3onQ
• Miku Hatsune:
http://www.youtube.com/watch?v=dhYaX01NOfA
• Experiment (2013/14)
• ”The fake postcard”
Language
• Audio analysis
• As speech, but for a different purpose
• Also covers intonation and ”body language”
• http://www.nextup.com/acapela.html
• Tomorrow’s Google translate
• http://www.youtube.com/watch?v=QFL0_LsZiYQ
• For all you programmers: Google API (set language/text)
• http://translate.google.com/translate_tts?tl=en&q=hello+computer+experts
• Experiments (2013/2014)
• ”Oh dear” – measure pulse, comfort
Environment
• Which means ”everything around us...”
• In the new WWW we are transponders which constantly
and without realising it (?) interact with our surrounding
•
•
•
Mobiles/GPS/Geo-location is only one, among many,
exemples
http://www.pathintelligence.com/
”Smart” clothes is another
• Smartphones have geo-location, but not Kinect
• Connecting to WiFi brings the node’s position
Experiment (2013/2014)
Live battlegrounds (3-5 meters accuracy indoors, 20 m outdoors)
http://www.navizon.com/indoors-solutions
http://www.pacmanhattan.com/
WHAT? Not the end?
• Beyond SMILE we also have AH (as in AH, SMILE!)
• Data forms also change (more than e.g. audio)
• We are used to graphics, sounds, printouts
• A as in ARTIFACT (”things”)
•
3D printers are as low priced as 400 EUR (and up)
•
http://www.creativetools.se/hardvara/3d-skrivare?sort=p.price&order=ASC
•
•
Are you cold my son? Let me print you a sweater...
We have been dating for three months. I printed these flowers to
you but they had to be blue; I ran out of red ink…
• H as in HAPTIC (touch)
•
•
Anyone who remembers FORCE FEEDBACK games?
Mobile phones that VIBRATES are also feedback
Why is this important?
• Ask yourselves:
• In all the time you spend using the Internet during an average day
and night, how much is spent in front of a computer screen (as
compared to using mobile phones/tablets)?
• So, how do you think “tomorrow’s consumers” (you and
younger generations) will interact with the WWW?
• Then why concentrate on developing traditional websites?
Class discussion
• Form groups of 3-5 students
• Discuss a new idea (“mental prototype”) which will build
on one or more of the SMILE technologies
• Who is if for?
• Where and how will it be used?
• What is the value for the user?
• You have 15 minutes
Summary
• New technology is pushing, our demand pulling
• WIMP and SILK will still be around, but we (YOU!) must
also be ready to AH, SMILE as means for interaction
• The WWW is turning into WWW
• Many applications will (should) be Mobile first