PixelSense Interface
2014 - 2015 Texas State Electrical Engineering
Tyler Lyssy, Kyle Paul, Alex Leon
Problem Statement
• NASA requires an expandable interface for
controlling and monitoring the different
systems required for future space flight.
Project Overview
• Texas State University will develop a multiple
user interface that uses not only touch based
interaction, but also allows the user to place
physical objects on the screen as an alternative
form of interaction.
Design Requirements
• Must be able to allow up to four users to interact with the
interface simultaneously.
• The system will show statuses and updates in an organized
and orderly fashion.
• Must use tagged objects as input into the system.
• The system will also sync with the Smart Watch to use for
input.
• System must be designed to allow for easy portability to
NASA’s systems.
• The System must be expandable.
Hardware
Samsung SUR40
Protection Glass
LCD Glass Panel
with Integrated
Sensors
Optical Sheets
PixelSense
Processing
Backlight with
Visible and IR LEDs
Embedded
PC/Housing
5
Hardware
6
Hardware
Safety
• The main safety concern is the weight of the
SUR40.
• It weighs roughly 80 lbs. and is difficult to move.
• The aluminum stand it rests on allows it to lay flat
or at an angle.
• Adjusting the viewing angle should be performed
by two people.
• The aluminum frame also has some sharp edges
that could be potential hazards.
8
Ethics
Privacy
Ethical
andand
Privacy
Issues
• The interface would be used by NASA crew
members, with its primary purpose is to
increase efficiency, so no ethical issues are
present.
• With further development, each user has the
capability to hide certain information of
themselves from the crew tab by using their
password protected Smartwatch for privacy
concerns.
Software
•
•
•
•
The SUR40 is currently running windows 7.
Tried to install windows 8 with no success.
Microsoft surface 2 SDK.
Microsoft Visual Studio IDE for development.
Why?
• The SUR40 can detect
objects placed on the
screen
• This gives an
additional level of
interaction and could
increase productivity.
11
How?
•The Samsung SUR40 uses special integrated
PixelSense sensors and infrared LED’s to track
tags.
•The tags have a predefined shape and 8-bit
mapping scheme.
•8 bit tag values allow for 256 different tags.
•Each tag has a black infrared absorbent
background and a pattern of white infrared
reflective circles.
12
Tag Dimensions
0.25 in.
7
0.14 in.
0
1
6
0.16 in.
5
2
4
3
0.28 in.
00010111 = 0x17
= Data bit circle; White = 1 Black = 0
= 3 outer circles, Used for orientation
= Center circle, used for location (X,Y)
13
Pros vs. Cons
Pro
• Adds unique level of
interaction.
• Wide variety of
applications.
• 256 unique tags.
• Methods for manipulating
the tags are already
written by Microsoft.
Con
• IR Sensors are very
sensitive to ambient
lighting.
• Tags must lay flat on
surface.
• Limited to Surface 2 SDK
and windows 7.
• PixelSense is no longer
supported by Microsoft.
• Will not work for small
objects.
14
Object Recognition
• We will be tracking two separate types of items
parts and food.
• Each item will have its own tag ranging from value
0 - 255.
• Parts will range from 0 - 127 and food will range
from 128 - 255.
• When a user place an item on the display a
window will pop up displaying information for the
item.
15
Parts
• The Part interface will show
relevant information for the part
such as the name, size, mass,
and location that they are
stored in.
Picture of Parts interface
• There will also be two buttons
that will allow the user to add or
remove and item from the
inventory or watch a video on
how the part is used/installed.
16
Food
• The Food interface will show
specific information for the food
item including the name,
quantity, weight, and sodium
values.
Picture of food interface
• This window will also have a set
of buttons that will allow the
user to add or remove the item
from inventory or to view a
video on how to prep the meal.
17
Design
Zones
2
1
4
3
Design
• When a user presses one of the four buttons in their
respective zone a window will appear with one of four
windows depending on which button is pressed.
• Once the window is up the user can toggle between the
different views using the buttons.
• Only one window can be open at any time per user.
• The window can be moved, rotated, and resized.
• When the window is opened again it will reappear in its
respective zone.
Schedule
• The Schedule will show all of
tasks currently scheduled for the
day for all the crew members.
Picture of schedule interface
• Users will be able to add,
remove, or modify tasks in later
updates.
Systems
Picture of Systems interface
• The window will have selector
buttons along the sides to
choose which system to view.
Once the user selects a system
they will be able to monitor
and manipulate system specific
information.
• Systems will include C&DH, C&T,
ECLS, EPS, GNC and Thermal.
• The visuals for the system will
be place holders for the actual
systems interfaces.
22
Crew
• The window while display the
entire crew with basic non
private medical information to
help monitor crew health.
Picture of Crew interface
• Further development for the
Crew window would allow the
user to be able to log in using
there smart watch to edit what
information being displayed.
Vehicles
Picture of Vehicles interface
• The final button will display a
window that will show vehicle
specific information. The user
can monitor vehicles and
vehicle systems such as fuel
levels, tire pressure levels, etc.
• The window will have a column
of buttons along the side to
choose which vehicle to
monitor.
24
Validation Test Plan
• To test the functionality of the interface,
multiple tags were placed on the SUR40 as
well as going through and testing every button
that was available.
Communications
• The PixelSense and SmartWatch will communicate
with a WAMP (Windows Apache MySQL PHP) server.
• All of the the information for the systems, parts, food,
etc. will be stored in a database on the server.
• The Server will be hosted on a machine provide by
Texas State
• Protocols used for WAMP data transfer include HTTP
and PHP.
26
Project Timeline
Preliminary Design Completion of Integration with
Review
Parts and Food server completed
Install and
2/18/15
Interface
4/18/15
Documentation
3/18/15
Turnover
5/15/15
Feb.
May
4/22/15
3/11/15
Critical Design
Review
Crew and Schedule
Module Completed 5/8/15
4/1/15
Place Holders For
Senior Design
Vehicles and Systems
Day
Module completed
Turnover Documentation
• This will include a User Guide, the Requirements and
ConOps (Concept of Operations).
• The User Guide will be in PDF format and will include
diagrams demonstrating the different features of the
interface.
28
Stretch Goals
• We would like to add an entertainment aspect to the
interface, a game of some sort or activity that can
promote teamwork and camaraderie as well as
provide a release at the end of the day.
29
Outreach
• Texas State Senior Design Day (Dec. 2014 &
May 2015)
– Poster display which provided information about
the project.
– Device setup for hands-on demonstration
• Texas State University Newspaper
– A sectional piece describing the project.
– Informational video created about the project
Cost Analysis
SUR40
~$8,000 (current hardware on loan from NASA)
Microsoft Visual Studio
~$600 (Free with Texas State DreamSpark account)
Printing cost for tags
<$10 or ( Texas State has a Vinyl printer )
2 Surface Pro 3 tablets for
development
~$1600
Conclusion
• In the construction of the interface, Texas
State has created expandable interface for
controlling and monitoring the different
systems required for future space flight as well
as utilizing tag recognition on objects for time
efficiency and ease of access.