Development of the Kinect Virtual
Art Program to Physically Engage
Children with Severe Impairments
Laura Diment
Supervisors:
David Hobbs
Flinders University, Adelaide
Tom Chau
PRISM Lab, Toronto
Kinect Virtual Art Program (KVAP)
Hypothesis
The KVAP will physically engage children
with severe impairments.
Outline
•Benefits of art
•Software development
•Clinical trial
•Results
•Further Modifications and Uses
Background
• ~10% of the world's population have a disability1
Art develops2 - Communication Skills
- Problem Solving Skills
- Social Skills
- Emotional Skills
- Motor Skills
- Self-Expression and
Creativity
• No commercial virtual art programs exist for children
who cannot participate in mainstream art practices.
1Langtree, I
& L (2011), Disability News, Information and Resources, Disabled World, http://www.disabled-world.com/
M A. (2011), The Importance of Art in a Child's Development, http://www.barnesandnoble.com/u/maryann-kohlimportance-of-art/379002442/
2Kohl,
Adapted Art Methods
Kinect Virtual Art Program (KVAP)
Benefits of a Virtual Program:
•No mess
•No paint accidentally consumed
•Low material cost
•Impairment compensation
Kinect Technology
•Multiple cameras and an infrared laser
projector collect depth, movement and colour
information.
Original Design
1.5m
•3
depth regions for
menu access
Design Issues
•
•
•
•
•
•
Depth regions confusing
Accidentally change settings
Difficulty selecting menu items
Difficult to activate special effects
Lag
No audio feedback
Development
•
•
•
•
•
•
•
Microsoft SDK Beta -> full version 1.0
Single depth region
Improved visuals
Auditory feedback
Gesture activated special effects
Reduced lag
Auto/manual mode
Demonstration
Clinical Trial
• 5 Children from Trinity Gardens St Morris Unit
– Level IV- V GMFCS
– Level IV MACS
– Non-verbal
• Collaboration with Disability Studies staff and
students - analyse qualitative responses
• Program records quantitative data
Session 1
Participant
tracked limb
zoom level
Screen choice
1
all
x2
17”
2
all
x2
17”
3
all
x2
17”
4
all
x2
17”
5
all
x2
17”
Observations
• Nervous/confused
• Little response to visuals, some response to sound
• Minimal movements and attention ~ 3min
Data Collection
Quantitative
Qualitative
•Limb position on x, y & z axes
•Time
•No. of button activations
•No. of activate special effects
•Image at 3 minute intervals
•Session length
Signs of:
•Enjoyment
•Frustration
•Fatigue
•Other emotions
•Size of movements
•Speed of movements
•Percentage of screen covered
Data Analysis in depth
Angle Time (s) x_px y_px z_value (m) x (m) y (m)
y in metres = y pixels * z distance
510 pixels/metre
Calculate % of screen covered
in 3 min from bmp. image:
white pixels/total no. of pixels
P4 Session 2
Minute 2
Right hand
Left hand
Head
Left foot
Right foot
Minute 3
Minute 4
P4 Session 3
Minute 2
Right hand
Left hand
Head
Left foot
Right foot
Minute 3
Minute 4
P4 Session 4
Minute 2
Right hand
Left hand
Head
Left foot
Right foot
Minute 3
Minute 4
P4 Session 5
Minute 2
Right hand
Left hand
Head
Left foot
Right foot
Minute 3
Minute 4
P4 Session 5
Right hand
Left hand
Head
Left foot
Right foot
2
3
4
5
Observations from sessions
• Trends over the sessions:
– Movement increased for 4 out of 5 participants
– Enjoyment increased for 4 out of 5 participants
– 2 out of 5 participants got excited in anticipation
– Session length doubled
– Less prompting required
– More responsive to audio than visual
Further Modifications and Uses
•
•
•
•
Multi-user
Non wheelchair users
Therapy – range of motion, increasing activity
Potential physiotherapist and occupational
therapist analysis tool
Summary
•
•
•
•
•
Planned and developed KVAP software
Obtained ethics approval
Clinical trial
Positive results from analysis
Children created their own printable artworks
Children are engaging physically with the KVAP