Next up we have KAIST Korea Advanced Institute of Science and Technology from
Korea. It's their first time participating in Design Expo. Their project is
called Sparkle. It's a visual feature description system.
>> Hey. This is Sparkle from KAIST Korea. I'm Cheolho Jeon, and these are my
team members Jonghyuk, Sungbae and Sungwon. Let's start with how our idea
emerged and led us here. 285 billion people in this world, which is
approximately one in 25 people, are with visual impairments. Most of them have
visions -- most of them -- some of them have no visions at all. Most of them
have a sense of color. We conducted a contextual inquiry with people with
visual impairment. We followed them shopping and we volunteered to help them
shopping. And we found out this: I would like a pink [indiscernible] cardigan.
They say it's trendy, says a women with visual impairment. Trendiness and
fashion matter to sighted people. And not surprisingly they also matter to
people with visual impairment. But how do they wear and choose clothes without
visions? So this is the very question that led us here to Redmond. How do
people with visual impairment see clothes? With this question in mind we
conducted more interviews with people with visual impairments and we found out
there are two kinds of information during the shopping. First, nonvisual
information like thickness, material, texture, can be gathered directly by the
people with visual impairments through touch. Second, visual information,
however, can only be gathered indirectly through the help of another person.
Assistants like clerks, friends, family, have to, need to verbally communicate
and describe the colors and patterns to people with visual impairment.
And we noticed that there's a great deal of inconvenience during this process.
Especially around the colors and -- especially around describing colors and
patterns. Imagine you're describing these clothes to your friend. How would
you describe the pattern to your friend? What about colors? Please take a look
at these two colors. How would you describe the difference between two colors?
I can say the two colors are blue, and one is darker and one is lighter. But
those are not the only things we see. We know the two are similar but
definitely different. And it is hard for us to verbally describe the
difference. And this is what happens during the process. Assistants, even if
they're eager to help, they struggle to verbally describe the difference. And
we wanted to sparkle-up the situation. We decided to help and improve the
verbal description process. For that, we're going to use cross-power and
technologies. There are two people involved in the shopping process.
Assistants are going to use the Sparkle app to better describe what they see.
And people with visual impairment will use the Sparkle Band to better recognize
patterns in the clothes and this together is called Sparkle. And here's our
video, and after the video [indiscernible] will continue the presentation.
Thank you.
[Music]
>> Jill lost her vision just a few years ago. Despite her disability she still
actively enjoys a cultural life. Here she feels like she should go out and buy
some new clothes. [indiscernible] is a good brother but certainly not a fashion
star. [indiscernible] tells her brother to take her shopping. Her brother is
resistant at first, but then he wants to enjoy the weekend. As his sister
insists, he says yes. He is a good brother.
[Music]
>> The two arrive at the department store. [indiscernible] is trying to buy
some cool clothes for spring and summer. It seems like she has to pick. She
wonders how does the pattern look like. All the patterns are too complicated
for her brother to explain. She does not just stand there and watch. She uses
her Sparkle. The Sparkle then delivers information about [indiscernible]
displayed on the braille display.
[Indiscernible] uses the Sparkle app. The Sparkle Band sends a visual data to
the Sparkle app and the Sparkle app displays suitable description words of the
visual data on the screen. [Indiscernible] describes how the clothes look and
feel with more detail from the help from the Sparkle. With her good brother and
useful Sparkle she continues to shop. They shop and shop and shop until the
brother begged her to stop. It seems like it was a nice day, at least for one
of them, isn't it?
[Applause]
>> Let's talk some more about details. There are three main stakeholders in
our system. First, everyone who can see denoted by green or gray. Second, the
assistants of people with visual impairment denoted by green. And finally there
are people with visual impairment marked red. We will describe how each user
group contributes to and take advantage of our service. Let's say we have a
piece of clothing that has some complex patterns. Using the camera, Sparkle
band can transform the pattern information into braille that people with visual
impairment can feel with their fingertip. So, how does it actually work? We
have a software demo for this. The braille on the surface of the band will move
up and down to create a texture that resembles the pattern. The color of each
cell represents the height of each braille. So the wider the cell, the higher
the corresponding braille piece. Now, let's see how sighted people and
assistants are involved in our system. First, sighted people. When they shop
for clothes online they'll be asked to do a small task for the people with
visual impairment. The assistant will ask them to describe the colors they see
with a few words. This processed data will be stored at the central database
and it will be used by Sparkle app, Sparkle Band and other services. This
processed data can be useful for everyone even sighted people. For example,
Sparkle can offer advanced searching and rich color dictionary. With Sparkle's
rich color vocabulary assistant, made by people, we can now search with a Curry
like, three-piece baked tea and expect some clothes which match that
description. Furthermore, now we have a color dictionary that also contains
descriptors of emotions and feelings attached to colors. Designers can easily
reference and pick colors accordingly. As you saw in the video, the assistants
can use the Sparkle app to get recommendations and describe colors easier. Do
you still remember these colors? We got some help from the crowd through
surveys and this is what we got. Using the system, assistants can say about the
color on the right, this is a light blue color which reminds me cloudy day and
makes me relaxed. So people who acquired visual impairment can imagine colors
easily with the descriptions given and people born with visual impairment can
share the feelings that the sighted people feel from the color. And we will
talk some more about the future applications of Sparkle. First, independence is
important to all of us, but some of us don't fully enjoy it. We want to help
people with visual impairment to shop freely on their own with Sparkle. Sparkle
can drive them to the shopping mall they're looking for and let them choose
clothes and describe colors and patterns to them. And, second, clothes are not
the only ones that are hard to explain. Furniture, accessories, drawings.
There are tons of things that are hard to describe. Sparkle makes, can make
this describe easier. Thank you. Any questions?
[Applause]
>>
Thank you Korea.
Who would like to start?
>> Great job, guys. I really like this concept. I think you guys have done an
amazing job actually. You've got a ton of detail in here. There's the software
service that you've designed, a platform for crowd-sourced information and
subjective, qualitative descriptions of color. There's a hardware device in
here, it's obviously got its own gamut of scenarios that you've only lightly
touched on here. Not only that, the graphic communication work, the design of
your presentation is really beautiful, the character design and all of those
things. I think there's just tons to commend you on, and even looks like there
was some software image processing work done just to do some live video
presentation. And let me just say that quite a lot of that happened between
Monday and today. So even more impressive for how far it's come. So well done.
>>
Thank you.
>> So I think that was really interesting. I really like many aspects of this.
I thought the braille, the sort of representation of patterns by sort of a
tactile-pattern thing was really interesting. I don't know, it would be great
to see some kind of progression on that and testing and how can people really
feel it and what does it mean to have polka dots -- a lot of understanding
patterns is really interesting. I liked the way that you thought about color
and different ways of explaining that as well. I think it would have been great
to see -- I was not completely convinced by the way that the wrist thing went
with the app and the description of the color versus the pattern, I think that
felt a little bit kind of disconnected. Apart from that, it was really nice,
and you did a great job explaining it.
>> Yeah, I think this is really, really neat. I mean, it took me -- maybe I'm a
slow study -- it took me a while to get -- because the pattern that was taken
off the rack had texture to it. It threw me off. You probably already heard
this feedback. So I won't belabor the point. But once I got it I started to
kind of appreciate the kind of expansiveness of what you guys were going for
here. This is a lot of vision. I always worry in those situations it's a lot
in the platform. But if you can produce that entire experience, I see the
vision. I think it's really, really exciting. So what I was most excited about
was the end result. I thought that the crowd-sourced actual description was
really kind of very helpful, very meaningful kind of in that response. So just
getting that part right in terms of how people give you the right words and how
you discern that these are the right ones for this thing and for each of those
cases -- each color, each garment -- that's a very expansive problem, even if
you don't have hardware or anything else contributing to that. So I really,
really like that part. I see that as a piece of it that in and of itself has a
lot of value. I don't know if that helps you at all. But I do enjoy the
project. I like the branding. I like the look. Congrats. A lot of great
work.
>> Thank you.
[Applause]
>>
We have time for one question.
>>
[indiscernible]
Cindy.
>> So the question was if you guys did any thinking about shopping online,
because when you shop online you have to rely a lot on the metadata that the
website provides.
>> So we actually met and have communicated with people with visual impairments
and we asked about the online shopping. And the answer was, like, we were
depressed to hear that they do not enjoy online shopping as much as we do. But
they said that they use online shopping only with offline shopping. They go to
the off line mall and wear it and try it out, then go back to their home and
order it online because it's cheaper. And we did not quite think about how to
make the online shopping better. That's all. Yes.
>> Thank you.
[Applause]