>> Desney Tan: It’s my pleasure to introduce Julie Kientz. I think most of you or
many of you know her. Julie is out at the University of Washington, right across the
lake.
She is faculty in HCDE, Human Centered Design and Engineering, with affiliate
positions in let me look at the slides, [inaudible] and CSE. Her lab is really cool. It’s
chill.
The computing for healthy living and learning. And Julie has done a whole bunch of
really neat work in the health space around technologies and understanding for
parents understanding kids, for kids with autism, and for sleep disorders, and I
think she is going to tell us about some of this work today.
So take it away!
>> Julie Kientz: Thanks, Desney. Yeah. So in one of these if you ever have any
questions about who is doing sort of HCI type work at UW I’m very intimately
familiar with many of the departments, so I call it [inaudible] alphabet soup over
there.
So today I’m going to talk to you about health and well-being technology. So there
has been a lot of talk about this. You’ve seen lots of articles about how it’s going to
revolutionize everything around us and it’s going to save us time and effort in being
healthy. It’s going to save us money and we’re all talking about health care and how
much it costs and how much it’s going to be, how much technology is going to save
us money.
It’s going to allow for continuous self-monitoring, right? So you’ll be able to track
everything and there will be nothing that stands in our way in terms of being able to
understand ourselves.
It will be easier to use than traditional methods, you know, paper-based diaries and
those sorts of things are also cumbersome and so it’s going to be much easier to use.
We’re not going to have to visit the doctors often. You’ll be able to do this magical
transfer of information to our doctor, and it will make things wonderful in that way.
Ultimately this is going to lead to miraculous health outcomes. And so I say this with
a little bit of sarcasm because we haven’t quite met that vision yet. So a lot of people
are chatting health and technologies as being sort of this holy rail of making the
world healthier, solving the obesity problem, solving our sleep disorders, solving all
these different types of disorders, we haven’t quite made it there yet.
And so the reason for this is what we’ve been looking at, or some of the burdens that
have been associated with these technologies. And so in my work I’ve come across
seven different burdens that have been manifested in different types of technologies
that we found are preventing people from adopting these types of technologies.
And so starting with the physical burden, so this is an example of a pediatric patient
getting a sleep study. So this is actually we’ve got pulse [inaudible] strapped to his
finger, they’ve got a chest strap strapped to his chest, he’s got sensors taped all over
his face looking at his facial movements. He’s got EEG sensors strapped all over his
head and they’re expected to sleep with all this stuff on.
And this is okay for a single night. You know, things like sleep apnea can be
diagnosed in a single night, so that’s why people do it, but if you’re looking for more
behavioral stuff over the long term, no one is going to wear all these sensors to bed
every single night for years, which is a lot of times insomnia is something that you
have to measure for a very long term.
So this physical burden can stand in the way of these things as well. There’s also a
mental burden associated with things. So having just to remember to whip out your
diary and write that you ate that cookie, just to have to remember that has a lot of
cognitive load that can prevent people from taking advantage of a lot of these
technologies just remembering having to do it.
There’s also things like emotional burden. So people might be familiar with the Wii
fit, which you stand on the platform and you step on it and it tells you that you’re
obese, and not only does it tell you you’re obese but it makes your character look fat,
and it makes it all jiggly. So there’s been a lot of reports that they’ve been doing this
to children and to people who didn’t otherwise know that, and they’re suddenly
finding out that they’re very unhealthy through a technology, and it can be really
upsetting to people if they’ve gone through that.
So a lot of times this emotional burden might prevent people from using these types
of technologies. There’s also financial burden. So a sleep study can cost around
$5000 for a single night, so that’s obviously, and this is you know the U.S. and we’ve
got all sorts of issues with how much health care costs in the U.S. but that’s pretty
cost prohibitive for things.
But even things like Smartphone technologies, they’re still the cheapest plan that we
found is $300 for a cell phone and I think about $40 per month for unlimited
minutes and data plan. This is Walmart’s plan, so this is actually the cheapest that
we’ve been able to find.
And so that’s pretty cost prohibitive to have low-income families to be able to adopt
these sorts of technologies so there might be a financial burden associated with that.
And then there’s the time burden.
So technology is supposed to save us time, but if anyone has tried to keep a food
journal then they know how intensively you have to kind of track all these different
things, you know, what did I eat and how much was it? I have to measure that out.
What was in that? Did it have mayonnaise on it? I can’t remember. And so you have
to go through this whole process. And a lot of times even these quick things that are
easy to record still take time. So has anyone done the track your happiness
application?
So they send you a text message and you’re supposed to fill out this quick little
survey. It’s supposed to take less than two minutes or so, and the way it’s set up is
you’ll do about 50 of these and then they’ll ask you again in six months, you’ll do
another 50 of these.
So the first time I did it as great. I filled it all out. I was all yeah, two minutes, this is
totally easy, filled it all out, got this great report about it, and then I had a daughter.
And then I got it like three months after she was born, and I got it again, and the first
time I did it I answered every single one religiously, and then the second time
around it was just like my phone wasn’t near by, my hands are full, and it was like
even that two minutes just putting her down was like, I missed at least -- I had to go
through like 160 text messages to get the 50.
And they said I was actually pretty good in that regard. So even just that little twominute thing took a lot of time and effort to be able to track that sort of thing.
There’s also the privacy burden associated with these things. A lot of people in the
room might be familiar with this and people might be uncomfortable having things
recorded and understanding how, or what the technology ends up doing and where
it’s going.
Some people might be prohibitive in that way. And then finally there’s the access
burden. So for people with disabilities it might be difficult for them to actually use
different types of technologies because they can’t or it’s not accessible for them
either for visual impairment or for physical impairments, or anything like that.
So then we use things like [inaudible] or anything like that? I figured this audience
is probably pretty familiar with it. So thinking about sort of the time burden this has
come up for me also as well. So I’ve been interested in tracking sleep and the
[inaudible] is one of the easiest things that you can do to track sleep. You are
supposed to put it on your wrist, there’s a little sleeve that comes with it and you
put it on there and when you start and stop it telling it when you when to sleep at
night. And this is based on accelerometers. It’s actually the gold standard according
to the health industry and tracking sleep off the shelf, believe it or not.
It took 20 years for them to appreciate that this was actually a technology that could
be used for sleep, and there’s all these caveats about how it should only be used in
healthy patients and things like that. But as simple as this technology is, in our
experience we’ve actually given this to people and they almost always forget to push
that button or they forget to stop it. So even just this quick little thing that they’re
supposed to do, start and stop, that combines a lot of things, the mental burden, the
remembering having to put it on their wrist every night, and having to remember to
do those sorts of things can be prohibitive even if it sounds like it’s conceptually
simple.
So our kind of theory for the kind of stuff we’ve been doing is that if we can try and
reduce a lot of these burdens associated with these technologies, right, we’ll be able
to lower the barrier and get us closer to those envisioned uses that people have
been having for health technologies.
So that’s sort of the theme that we’ve been doing in a lot of our work. So first I’m
going to talk a little bit about how we understand these burdens. So we’ve done
some work on looking at different aspects of burdens. So lots of people have done
stuff on things like mental burdens, so we’re going to explore some of these more
subtle burdens as well.
So the first one I’m going to talk about is the emotional burden associated with these
types of technologies. So remember the WiiFit example of finding out that you’re
obese through a technology, and we’re actually kind of fascinated by this about like,
and we’d been designing a technology for parents and tracking young children’s
development, and one of the outcomes is that your kid could have a developmental
delay, or things like autism, and how do you find that out through a technology?
That just seems really uncomfortable. So this is one of the things that came up when
we did an initial study of this work is how do you convey this information that their
kid might be behind, that they might have delays, and how do you do that in a way
that is motivating enough that they’ll do something about it, but not going to make
them super, super obsessive or scared of checking the technology.
So one of my students went through and she interviewed a lot of people who have
been diagnosed with some sort of terminal or chronic condition, so she interviewed
people with cancer, with Parkinson’s disease, people who I think she also did
diabetes and all those sorts of things, and really tried to understand what helped
them when they received this diagnosis for the first time, and then also what hurt
them. So talking about both the good and bad about what happened to that.
And she’s hearing all these stories about, one person had used a personal health
record when she was pregnant and she got one of the notifications that her test
results for one of the prenatal tests was available, and all it said was result
abnormal. And there was no context to it, there was nothing that they could learn
about what does that mean? It was the test that they do for Down Syndrome, and all
it meant was that she had a greater than average chance, I think instead of like 1 in
3000 it was more than 1 in 200. And there was no explanation of that, it just said
result abnormal.
And it was like 10 o’clock at night, she couldn’t call her office to get an explanation,
she’s up Googling all night about what does this mean? And so that’s really kind of
heartbreaking to find that out. But on the flipside we don’t want to hide that
information from people either. If they have test results it’s important that they get
this information as soon as possible so they can be proactive about it and engage
with it and do something about it. So we have to try and strike this balance of
getting the information to them in a way but something that can acknowledge the
fact that this might be emotionally burdensome and helping them deal with it.
So in addition to interviewing the patients we also looked at the doctors. So we
interviewed a lot of people who had given these diagnoses, talked to them about
their training, we also reviewed the medical literature to look at how they train new
doctors about how to convey this information to people and we found a lot of really
great information.
So a lot of times doctors try to insist on doing this face to face, but at the very
minimum they try to do it over the phone or at the very minimum making sure that
they release test results that there is some sort of note associated with it that they
can provide the sort of explanation.
Some of the things that they find is, you know, again is wanting this information but
in a way that acknowledges that this could be sensitive, you know this could be hard
to hear but I just want to let you know that you may have the precursors to diabetes,
or something along those lines, or you have a tumor or giving that sort of
acknowledgement that they may be feeling something anxious.
Also we learned that using team language is important. So in stead of saying things
like you have cancer or you have to deal with this, you think about like oh, we’re all
in this together, we’re going to work together to make sure that you get the best
treatment that you can and talking about using this sort of team based language is
really important as well. And then also just simple things about connecting them to
where to find more information, what can I do about it? Because that seems to be
when we were talking to patients that seemed to be the biggest thing is that the first
thing was just feeling completely helpless and not knowing what they could do. So
just giving them some simple things about what I do next is sort of the most
important thing there.
So we’ve been trying to translate these into design guidelines and we’ve got some
work that’s under review for that, so I’m happy to kind of share that with people if
they want to see that more. The second [inaudible] we did was actually
collaboration with Dan here, so the student was looking at how people when they
put things like sensing [inaudible] technologies in the home how they might react to
that.
And so you can ask people just how they feel about it, but a lot of times people don’t
know what they’re doing and they also don’t think of all the different situations in
which they might be recorded and whether or not that’s going to be problematic.
And so what my student did was she did a cultural probe where she brought people
into a lab setting and interviewed them on, first she gave them their patient
education session on what sensing inference was and what kinds of things it
recorded, what types of applications you can use it for, and she got their initial
reactions. But then instead of just kind of taking that interview data and looking at
it she sent them out back to their home with what we called sensor proxies. And so
you can see this in the picture here, they’re just very simple motion-activated lights
that she put a little cover on, and she asked them to place them in four different
rooms in their house. And whenever the light turned on they were supposed to be
triggered to think about what were they doing just now and that had been recorded
how problematic would that have been?
So she kept them with a log book so they could write down their thoughts and then
she also worked on keeping a guest book so other people who had come into the
house and then have them take pictures of the different rooms that they placed it in.
And so that was really great because we were able to get their receptiveness to
these sorts of things in context and get them to think about these kinds of corner
cases. And so, for example, one of the lights had gone off and the participant’s child
actually saw it and was like oh what is that for?
And it sort of sparked this conversation. And she sort of explained the study that the
boy was seven years old and explained the study to them and the mother explained
like, well, I don’t know if I would’ve wanted to be recorded just then. Like that’s a
private conversation. And she hadn’t really thought about that case when we
interviewed her first. So again thinking about those sorts of things was really
important for that.
So we put them in the houses for about four weeks and prompted the users to
reflect on those things and then we brought them back into the lab to ask them, and
we actually found a lot of people sort of changed their mind about things that they
thought were problematic no longer became a problem by the end, because like oh
yeah I thought about this use case, right, so this actually would make it really
important. So I didn’t think I wanted cameras in my house, but then I thought about,
well, I have a child with disabilities and that actually would have been really
important to know what they were doing and if they were in some sort of
troublesome situation.
So not only did it make them more concerned about privacy for some things, but it
made them less concerned about privacy because they could see the application
having reflected on it for a long period of time. So the other thing was just like,
when I talk about the mental effort associated with this, there’s also just this thought
about just like thinking all the time about what’s being recorded and when and in
what rooms and trying to help them understand what all these things are can be
sort of a mental burden for that as well.
The third exploratory stuff we looked at was specifically in the context was
specifically in the context of an application design space. And so we’ve been
interested in doing a lot of stuff with sleep, and so we did a formative work in trying
to understand what this design space was and what’s already out there and what
people are willing to do and what sort of opportunities are. So we conducted a
large-scale survey and interviews to determine the acceptability of these
technologies.
So what are people currently using, what are they willing to use, what sorts of things
do they want to track and those sorts of things? And we use that information to
develop a design space framework. So this is combined with a literature to sort of
map out what the different opportunities and challenges are as well as what’s really
being done and what opportunities there are for additional design.
And so specific to the different user burdens they come across is that people were
pretty averse to wanting to wear things to bed. And so unless they had something
like they were used to wearing a sleep mask or something like that, they were pretty
aware that they didn’t want to be wearing stuff to bed. It might be uncomfortable,
you know people actually kind of wake up a lot if they think they’re being
monitored, so they were really conscious of not wanting to wear anything to bed.
Even just the simple little thing on their wrist was problematic.
Yeah?
>>: When you say not wanting to wear anything to bed, is it an issue of just how
invasive it is? Or like how comfortable it is?
>> Julie Kientz: It’s comfort, yeah. And so things that we button to clothing I think
people said that they would be okay with. So things like smart fibers and things like
that would be okay, but things like having to put something on their wrist or
especially on their head, which some of the approaches have had a head-worn thing
->>: But they have clothes that they could wear comfortable and pillows?
>> Julie Kientz: Yes. Yeah, yeah. So there’s other opportunities, but they didn’t like
the idea of having body-worn things because we showed them the different options
that were out there and so they were a little bit conscious of putting anything on
their body. And I know this is anecdotal so I shouldn’t talk about that, but just in my
own experience with wearing these things I’m always constantly worrying about it
coming off. I’ve worn the [inaudible] headband and it’s left a red mark on my
forehead because I was so conscious of it falling off that I pulled it really tight and I
was like walking around I looked like Cyclops during the day because it left this red
mark on my forehead.
And the other issue is that because the reason I was so conscious about it was
because if it slipped off in the middle of the night I lost the whole night’s worth of
data. And so and like I wore this for nothing I didn’t get any data from it.
>>: Do you have an intuition as to like how something like a pillow would be
accepted if the nature of how your face interacts with it doesn’t change but there is
something inside of it?
>> Julie Kientz: Yeah. So people are kind of picky about their pillows. Like, some
people will sleep on anything. We had also told stories of people who filled their
suitcase with their pillow and took them to every hotel with them because they just
had to have their special pillow to sleep. So I think it’s pretty variable from person
to person.
>>: So is it a lot of people that it’s like a very personal choice ->> Julie Kientz: I think so, yeah. The national sleep foundation does a lot of these
huge, huge pools that look at things like sleep environment and things like that, and
they’ve done one in terms of bedding and mattresses and things like that so I can
point you to that if you want really specific info on that. But yeah, just hearing from
the people in our study a lot of times people can sleep anywhere and a lot of times
people are so sensitive that they’re really averse to those sorts of changes.
The other thing is that just thinking about being recorded while sleeping is also
disruptive to sleep itself. So that was sort of problematic as well. Like, if they
thought about it, especially if they were like is this on? I just woke up do you think it
recorded me being awake? So there’s a lot of this sort of concern about this stuff
going on in their head about what’s being recorded in the middle of the night that
was actually doing the opposite.
So instead of actually helping them sleep better it was actually making them sleep
worse. And so some people we talked to are doing long-term sleep tracking got
used to it over time. So it is sort of something that you can get used to if you’ve been
doing it enough, but those initial weeks having lots of people try these things out
people were like yeah, I felt like it was kind of weird. I couldn’t stop thinking about
like, oh did it pick this up? I moved. Does it think I’m awake or does it think I’m
asleep?
Yeah?
>>: Did a couple of years ago, like, inadvertently they set a privacy settings on your
sleep stuff [inaudible]. Oh, activity says ->> Julie Kientz: Right. Yeah.
>>: That’s awesome, thank you for sharing.
>> Julie Kientz: Yeah. That’s actually a great point. So the bedroom we [inaudible]
things it came up with, so thinking about the privacy burdens that there’s a lot of
stuff that goes on in the bedroom that you may not want recorded as well. So when I
talk about the next study I’ll talk about some of the particulars that we came up
when we were recording in the bedroom environment, and those things certainly
came up for sure.
So this is our sort of testing the waters of what the design space for sleep was like.
And so the main things we were looking at were it needs to be really easy to use, the
other issue is that when people are going to bed or waking up in the morning, I don’t
know about you guys but when I go to bed at night I’m usually exhausted and I don’t
want to spend a lot of time doing anything before I go to bed. I just want to kind of
like brush my teeth, get my PJs, take my contacts out and go to bed.
And just even that little act of like, oh, did I get the, oh crap I left it downstairs it’s
charging I have to go downstairs and get it and put it on my wrist. That just adds a
sort of extra step. Same thing with, like, keeping a journal. On the other side when
you wake up in the morning the first thing I want is a shower and coffee. And so I
don’t want to necessarily be thinking about like, oh, how long did it take me to fall
asleep last night and how many times did I wake up? Oh I went to the bathroom at
like three in the morning, or my daughter woke me up or that cats came in.
Like sleep doctors want you to record all this information and it’s hard to do first
thing in the morning. And so that was sort of the nature of sleep itself sort of made
it difficult to do these sorts of manual recordings as well. So that was the sleep
technology stuff.
Any other questions at this point?
>>: What I take away from this is you basically need to set a technology that if you
use it properly it automatically rewards you with a coffee in the morning?
[laughter]
>> Julie Kientz: That’d be great. It has to be before a certain amount of time I’ll tell
you with one of our other technologies is if you do it too late in the day then that can
be problematic as well.
So that was sort of the formative work in sort of understanding what these designs
associated with burdens are about. So we looked at a lot of the literature for things
like mental burdens and things where we specifically delved into the emotional
burdens, the privacy burdens, and the physical burdens associated with sleep.
Yeah, Dan?
>>: Sort of a philosophical question so I’d be happy if you want tot punt this to the
end of the talk. Everybody basically accepts, sort of society accepts that managing
your weight has a clear impact on your happiness and your life span. Not every
body does think about it, but basically everyone is sort of on board with that. I think
largely thanks to the press, we’re on board with the fact that these things [inaudible]
your life span.
I used to be sort of asleep, I take it everybody knows how to sleep, I am totally on
boar with like sleep is up there with sleep is up there with managing your weight in
terms of your daily happiness and maybe not the same impact on your life span, but
certainly on your quality of life. It seems like [inaudible] people don’t seem to have
that same socially accepted with something that sleep really affects your quality of
life, and that itself is a barrier to adoption of like, certainly that regulates your
threshold of what is too uncomfortable to put on?
Like I said, very philosophical, so happy to punt it to the end of the talk. Just
thoughts on what one might do about that or if you agree or disagree, or ->> Julie Kientz: We were thinking about that, it’s not a problem Dan, it’s like your
plan. So one of the things that we’ve been thinking about is how to prioritize sleep.
So when I talk about some of our other studies we found people that were making
the choice to I want to go out with my friends, I don’t want to go to bed at midnight.
It’s like, even though that’s what the doctors say, I prioritize my social life over my
sleep.
Or if they’re thinking about like, oh I don’t get off work until 9 o’clock, and I need to
exercise at some point, I need to do that before I go to bed. So should I do that or
should I sleep? It seems like to me exercise is more important. And so there are
definitely these sort of trade-offs and sleep always seems to get the short end of the
stick for almost everyone.
And so the interesting stuff we’ve been engaging with a lot of people in the sleep
community is that people are really, really bad about knowing how sleep impacts
their life if they’ve never thought about it. So they’ve done all these studies where
the types of studies where they take undergraduates and sleep-deprive them for like
weeks at a time, and then they do all these sorts of objective measures about it.
And so -- they may pay them a lot, and the [inaudible] results for that are pretty
tricky, but so they do these sorts of studies and they say they get people getting like
six hours of sleep every night and they’ll have them talk about it, and when you ask
them they say like, oh yeah, I feel pretty good, you know, it’s fine. I think I’m totally
okay with it. But they do all these objective tests and it they’re reaction time is way
slower, their memory is slow, their mood has gone down, they’re having more
arguments with people, and so they’re just really bad about understanding the
impact that sleep has.
And so one of the projects we’re looking at is helping people actually realize that. So
I’ll talk about that in a little bit, but we have definitely been thinking about that
problem for sure.
So, okay. I think I have a stray animation in there, sorry. So I’m going to talk about
four different types of technologies that we’ve been looking at for reducing some of
the burdens with different health care stuff and I’ll start with two from the sleep
domain since we’ve already been talking about that, and I’ll talk about one from the
parenting domain and then one from the physical fitness domain.
So the first one we’re going to talk about is an application called shuteye. So this is
in collaboration with some Ph.D. students, Sunny Consolvo and Ben Greenstein,
[inaudible], and Nate Watson who is a sleep technologist. So he was actually a
participant in our initial sleep study. We did a contextual inquiry at the UW sleep
lab and he was just so fascinated by the thought of using technology for sleep that he
ended up becoming a collaborator in a lot of our different projects. So but when
talking with him he talked about this notion of what sleep hygiene is.
And so when you think of things like hygiene you think of dental hygiene, you know,
like brushing and flossing your teeth and those sorts of things, but it turns out
there’s such thing as sleep hygiene as well. And these are sort of the little things you
do throughout the day that can have an impact on your sleep at night. And so
there’s a whole list of things usually if someone complains of sleep problems the
very first thing they do is they give them this list of sleep hygiene recommendations.
And so this includes things like getting consistent timing of sleep is actually more
important than duration.
So things like going to bed at a consistent time every night is actually pretty
important. Getting between 7-9 hours of sleep is important. Not drinking caffeine
within 8-14 hours of bed time, not exercising heavily within three hours of bedtime,
also things like not watching the clock at night, only using the bedroom for sleep and
sex is actually listed on the list because you start to associate your room with like, if
you work on your laptop in bed you start to associate things with being awake at
night.
And so these things that you can do throughout the day that can have an impact on
your sleep. But the problem is like remembering to do all these things at any given
time is pretty problematic, and it’s hard to think about these things, and you know,
I’m pretty big on getting coffee every now and then and thinking about 8-14 hours,
well, if I go to bed at midnight it’s like, what does that end up being?
So these sorts of things are pretty hard to remember to do all these things
throughout the day. So what we want to do is look at how we can use these sorts of
time-based things and make it easier for people to remember these sorts of things.
And so this is just a subset of the things we looked at, but things like napping. You
know, napping is only really recommended in that kind of one hour after lunch sort
of window. And it actually doesn’t matter when you woke up or went to sleep,
there’s just a natural dip in your circadian rhythm right about the 2-3 p.m. window
time frame.
So that’s an important thing. If you’re going to nap that’s the only time of day that if
you do it it’s not going to impact your sleep later on at night. Meals, you know, three
hours before bedtime you shouldn’t have a heavy meal. Alcohol, this one is actually
counter-intuitive to me because you think about people drinking a glass of wine
before bed to sort of unwind. I mean, it really does help. You go to sleep, but as
soon as the alcohol oxidizes you get a spike back up in your circadian rhythm and it
wakes you up and it can disrupt your sleep through the rest of the night.
Then they also recommend doing relaxation activities in the hour before bedtime.
So doing like crossword puzzles or reading, or those sorts of things to sort of
unwind and do things before your day. So these are just sort of time-based things
that we have come across that are important for maintaining good sleep hygiene.
And so our design question was how can we make people aware of these sorts of
things? But [inaudible] this knowledge that people don’t want to record a lot of
things, they don’t want to spend a lot of time doing these things. So how can we do
it with the most minimal amount of effort possible on the part of the user?
So we didn’t want them to have to remember these things, we didn’t want to have to
make them wear anything, we don’t want to have to make them record anything
whatsoever. We tried to do to the most low-burden approach that we could think of.
So we developed an application that runs on android phones live wallpaper, and
what it is is it just runs in the background screen and this is a timeline throughout
your day. And so you’ve got the different sorts of activities that might impact your
sleep. So caffeine, napping, exercise, meals, alcohol, relaxation, we also have
nicotine use, but people could turn that on and off depending on whether or not
they used it, and you could also add custom things like medications or some people
had things like they though T.V. watching might have impacted so they could add
custom things like that.
Basically what it is is the idea that you can get a simple glance whenever you’re
looking at your phone, checking your text messages or emails, and get a sense of
what the current time is and whether or not they do those sorts of activities if
they’re going to have a negative impact on my sleep.
So the vertical line represents the current time, and if the bar for a given technology
is thick it means that if you do it’s going to be fine, if it’s thin it means that you
probably shouldn’t do that now at this time. Yeah, Scott?
>>: There’s no [inaudible]?
>> Julie Kientz: The important thing is that we only took stuff that there is actually
a research from the medical literature on using these things ->>: So the phone doesn’t tell you you shouldn’t be using things?
>> Julie Kientz: Right. Which is actually interesting because research came up since
we have done this that indicates that certain amounts of blue light can actually
disrupt your melatonin levels and so they are not recommending having anything
that emits blue light within one hour of going to bed.
So we could have it automatically shut off or something like that. They could have
added phone use as a custom thing if they wanted to.
>>: Or they could change the interface to be all red light at a certain time. Actually I
think there are some filters that you can get for MacOS that will remove all the blue
light that is emitted from your screen. So you can do those things as well. So the
idea is that you can also scroll and look ahead, right? So if you take a look at it and
say like, oh, coffee isn’t really recommended after about three o’clock, so you can
scroll ahead and look up oh, I’ve got a good amount of time for that.
And so the idea is that you can really quickly take an easy look at those things. The
only thing that we required is that when they first started using the technology they
just had to estimate their sort of sleep and wake times they normally do, and then
everything else was set. So they didn’t have to change anything, they didn’t have to
do anything at all once they got the initial technology set up.
And they can always go and read about these sorts of things too. So we had like the
specific literature that cited why this was the case they could learn a little bit more
about it as well.
And so we did a deployment study on this and so we did a four week deployment
with 12 different participants and so we brought them in and we changed our the
SIM card if they didn’t currently have an android phone, so they were using shuteye
on their primary phones that they were using for making phone calls and texting
and everything. So we mostly had a lot of qualitative findings on this one, but we
had an increased sense of awareness in general. So we asked them a lot of things at
the beginning of the study, you know, what sorts of things are impacting your sleep,
what do you know about alcohol use and asking them lots of questions, and a lot of
people sort of had these vague notions of it, but by the end of the study they just sort
of had this much more impactful sense of, oh gosh, I drink a lot of coffee and they
sort just had this bigger awareness to it.
The other thing is that because shuteye is cyclical and it reset every day, they also
sort of had this desire to almost stay within that schedule. They almost use it as sort
of a goal, right? Like I set this goal for myself that I’m going to be in bed by 10
o’clock. And they sort of saw the shuteye getting slowly towards the end and it’s
almost sort of the subconscious motivator to get them on a sort of a different type of
sleep schedule.
And then they had a better sense of just the different factors. So a lot of people
thought that alcohol was actually good at getting them to go to sleep and so they
understood that a little bit better.
We did some initial exploration into the efficacy of it. It’s hard to show behavior
change and efficacy in a four-week study, but we did a measure anyway. So we used
a standardized scale called the upward sleepiness score, which is just an eightquestion scale that results in a numeric value between one and twenty. And so we
did see improved scores for 8 of the 12 participants. Three people stayed the same
through the course of the study. And this person really ruined our results. So she
actually got worse. So she started out with pretty good sleep and then she ended up
getting worse sleep by the end of the study.
And when I talked about how people don’t prioritize sleep, she told us at the end of
the study I think I learned in the study that I don’t care about sleep. So she was
pretty young, she liked to go out and go party and things like that, and this just
helped her think like, oh, if I sleep I have to sacrifice things like alcohol and get to
bed at night. So we actually had a reverse impact on one person in our studies.
So that was a little bit disappointing but we did see some promising results for the
rest of the participants as well. The other interesting thing, and this goes down to
just knowing a little bit about how the different designs you have can impact
people’s perception of things. And so the recommendation says that alcohol doesn’t
have an impact until you’re within an hour or two of bedtime, right? So we just
showed that it was fine to use any time of day.
But if you look at this it looks like we’re encouraging alcohol use at 7 in the morning,
and so -[laughter]
People are like wow! I can drink alcohol now! It’s seven in the morning! So thins
like that are problematic. The other thing is because we had sort of a hard edge on
when things ended, it’s not actually the case that it’s a hard edge, it’s definitely more
of a window, right? People thought this was like, Oh my god, I’ve got to go get coffee
really quick before 3 p.m.
So I think if we redesign it we’d have more of a subtle transition as opposed to like a
hard stop. So just these little tiny design decisions were actually having an impact
on how people thought about things. And that was actually really fascinating to
think about. Yeah? Sure.
>>: When users were able to sort of set their own timelines, so ->> Julie Kientz: Yeah, so we used the literature-based recommendations for setting
the initial ones, but they could go in and edit it. So a recommendation for caffeine is
like 8-14 hours, which is a pretty big window. And some people if they have, you
know, it’s definitely 14 hours and they know it, but for some people it’s maybe a
little bit less than that. So they were able to go in and edit that I think if they needed
to.
>>: Did you find that they did that?
>> Julie Kientz: Not too many people did. You know, again it’s one of those things
like defaults are really strong, right? And so people they don’t go in and edit as
much as we would have expected. So that’s why the importance of setting really
good defaults is always an important thing. So yeah, great question.
All right. So any other questions on shuteye?
Yeah?
>>: Did you also find out how many people kind of violated the guidelines?
>> Julie Kientz: Yeah. A lot of people did. And so but the nice thing about it was
that they were kind of just conscious about it like, so I know I’m going to have this
coffee now and I know it’s going to impact me, so even if they didn’t follow them at
least they were thinking about the consequences of it. And so we thought they were
making more mindful-based decisions about the types of things that they were
doing.
So we didn’t have a way of measuring alcohol intake and so we couldn’t get a
quantitative measure of these things and it is based on self-report, which is a little
bit, you know, the limitations of that, but yeah, we did see some people who were
consciously violating it, but at least they were more mindful of it.
You know, it’s like, I drank coffee but I felt guilty about it at least. So -- yeah?
>>: Did you look at the time that people took to fall asleep? Because I’m sure that
this is a very complicated theory in terms that any awareness [inaudible].
>> Julie Kientz: Yeah, we didn’t because again we wanted to try to do like
completely no instrumentation whatsoever, and so I think some future work with
this would need to include some sort of sensing to look at this because right now it’s
just sort of set on some of their ideal sleep and wake times, which isn’t always what
people did.
So it might be a little more accurate if we had some sort of unobtrusive sensor that
[inaudible] and automatically adjust it and also sort of find out what those things are
as well, right? So when I say that caffeine is 8-14 hours, you know finding out what
it is for me might be important too. So if we had some sensing to make this a little
bit smarter then we can think about well, caffeine measurement actually is
happening more for you more about six hours because when you take it then it’s
going to be along those lines.
So I think with some extra instrumentation you can make this a little bit more
powerful, but our goal was to make the user do absolutely nothing, so I think it was
pretty successful and I at least getting people aware of those sorts of things.
And I think there is more you can do with it though. Any other questions?
Okay. I’m going to talk a little bit more about sleep as well, a different type of
approach. So when I talked about sleep hygiene -- okay. So when I talked about
sleep hygiene I mentioned things like the time-based ones, but there’s also
environmental-based ones. So things like sleeping in a room that’s fairly cool, free
from distracters like light, or sleeping in a room that doesn’t have a lot of sounds in
it. They recommend not having pets in the room with you.
They recommend like not keeping children in the bed with you. And so there’s all
sort of things about keeping your sleep environment very pristine, which people
who have kids and pets and things like that know that that’s very difficult to do. But
you wanted to help people understand what these are because some people can
sleep with their kids in their bed and it’s not a problem. So some people can also
just sleep in a warm room and it’s not a problem.
And so how do we find out these sleep recommendations are based on a general
population, so how do we help people find out what’s actually affecting your sleep?
So that way do you actually take the trouble of actually making sure that the casts
and things are kicked out in the middle of the night, or is it okay for you to actually
have them in bed?
So our goal is to help people understand their own sleep environment and also
what’s disrupting their own sleep. So we wanted to allow people to explore what’s
happening when their going to sleep at night in a home setting that minimizes the
privacy, physical, and time-based burdens. So privacy is really important in this one
because a lot of things go on in the bedroom like I mentioned that you don’t
necessarily want recorded. But it’s hard to figure out what your disrupters are
unless you’re recording.
So we have to be really conscious about the privacy concerns going on with the
recording in the bedroom, so we used a lot of the information from the privacy
study that we talked about to kind of think about how to design these sorts of things.
And so what we end up coming up with is a system called lullaby. We had a lot of
fun brainstorming lots of like sleep-based terms for the names of our projects. So
what it is is it’s a bedside [inaudible]. It’s about the size of a table lamp. It’s a
research prototype so it looks a little bit scary at times, but it’s got two motion
sensors, which they’re on posts that you can turn. So we recommend one is pointed
at the bed and one is pointed at the door to the entrance to the bedroom. There’s a
sensitive microphone, a temperature sensor, a light sensor, an infrared camera that
snaps a picture every 15 seconds so it gives you kind of night vision of what’s going
on in the room.
And so it’s got a tablet-based interface that will allow you to get a quick sense of
what’s going on in the room. So just a green of the noise level or motion that is
going on, if it’s green it means your condition is within the recommended range of
sleeping conditions, if it’s red it means it’s outside the recommended range. So you
can also get a quick snapshot of what’s been going on in the last five minutes or so.
There’s a quick snapshot of what’s going on in your room, and then there are
privacy controls. We wanted people to really easily be able to turn off the recording
and we also wanted them to be able to just quickly delete anything that had gotten
recorded right away as well. So with one button you can delete what happened in
the last hour.
Yeah?
>>: So how close was that to somebody’s head?
>> Julie Kientz: So we recommended putting it where you put your alarm clock or
so.
>>: So it’s pretty close?
>> Julie Kientz: Yeah. So it’s fairly close.
>>: Did you find that any of that electricity next to someone’s head was kind of ->> Julie Kientz: No, I didn’t. Our study recorded that. We were really, really
conscious about things like blinking lights and stuff. We actually had to go through
with electrical tape and tape off every little blinking light. A lot of the little power
adapters for the sensors have lights on them.
>>: Are there any issues with noise [inaudible]?
>> Julie Kientz: Yeah. So we had initially included an air quality sensor in the initial
design and that had a fan on it that sucked in the air particles and it made so much
noise that we were like okay, we have to get rid of this. So yeah, so we’re pretty
conscious about it. We used it ourselves at first and were like uh, I can’t sleep with
this air quality it’s a little bit noisy.
Same thing with sort of lights and things like that. In terms of electricity
interference we didn’t get too many people in our study that complained about that,
but yeah, I think that’s something that’s definitely worth exploring.
So the idea is that it’s capturing what’s going on while you’re asleep, which actually
ended up being pretty interesting because a lot of stuff happens in your sleep that
you don’t know about. So there’s been a whole class of sort of applications that have
looked at capturing different things, you know, things like meetings or time you’re
playing with your kids or classroom activities and being able to record and review
those things later, and we found that most people don’t look at information because
they were there. Only if they need to remember something important do they go
back and look at it, but if you’re unconscious during what happened people were
actually super fascinated by this so it’s kind of interesting.
I’ll talk about that in little. This is a more detailed view. So this gives you a read out.
So this is a sleep cycle that you had. So we had people wear the Fitbit [phonetic] and
so despite all the limitations I talked about with the fitbit that seemed to be the best
compromise for getting sleep sensing right now. But we’ve got some ideas for how
to remove that component. But for this particular study we did use the fitbit for
that. And so you’ve got your sleep throughout the middle of the night. And so the
fitbit is telling you all your awakenings.
And so what you can do is if you see awakenings you can go back and look at the
different sensor streams and see if there is anything there. So was there a spike in
the light levels, was there a spike in the motion levels, was there a spike in
temperature? For this particular example you can see there was actually a fan
running throughout the night so you see a lot of motion and things just blowing up
and picking up from the fan throughout the night.
They had a pretty disrupted sleep also throughout the night. So the other idea is
that if you see a spike there you can also slide this little bar here to that particular
moment and then just hit play and it will go back and replay both the audio and, it’s
kind of hard to see here but this is the night view of my student sleeping, but you
can actually see what was going on in that frame as well as playback the audio.
So you could say like, oh, that was when my cats actually starting scratching the
door, or that was when I got that phone call in the middle of the night from this
random person in Europe that forgotten pocket dialed me. So you can kind of see
what those things were that were interrupting your sleep. There’s also privacy
controls you can go back and delete big chunks of data at a time because again the
environment for that people are sleeping in can be pretty private and so we went
through and allowed people to easily do that.
It looks like your sleep is very remarkably the same throughout the night. So these
are just the different visual images. And people can just delete whole chunks of data
at a time. So they can just tap on things and just quickly delete them.
And so we did a small study with this one. So we did a two-week study with four
participants and so it’s always one of those things like how long do you run your
study for? And we’re actually pretty lucky that it’s pretty well regarded in the sleep
community that two weeks is about the time it takes to get a good snapshot of what
people’s sleep environment is. And so we did a two-week study with four different
participants and so each of them had the lullaby for two weeks. And so we saw
pretty high access rates. I mentioned that people are just fascinated by what went
on in their sleep.
So you see all these spikes in the motions and it’s like what’s going on there? I don’t
remember getting up. And so they actually went through and used the system quite
a bit. We saw one participant saw this constant spike in the audio stream and she’s
like what’s going with that? And it turns out she was coughing in her sleep and she
didn’t even know it. She went back and played the audio and she’d just hear this
[coughing] almost consistently throughout the night.
She didn’t, you know, she slept by herself. She didn’t have a sleep partner that she
slept with and so that was something that she didn’t know about herself. And the
other participant saw some motion and an awakening. It showed an awakening on
here and he was like I don’t remember that. What was going on? And he noticed
that the door to his room had been closed when he woke up in the middle of the
night, or when he woke up the next morning, and he was like I always sleep with my
door open, what’s going on there?
So he went back he used lullaby to go back and play. His cats had come in in the
middle of the night and he got up, kicked them out, shut the door, went back to bed
and had no memory of it whatsoever. So he actually discovered -- and that
happened like three times throughout the study and he was just, whoa, it was like
kind of really creepy.
Yeah?
>>: Do they find that having control over what data they can delete and take out
was enough consolation to get over the fact that there’s a camera watching you?
Because I imagine pretty readily it would trip the creepy factor.
>> Julie Kientz: Yeah, it was definitely varied participant to participant. Some
people were really conscious of it, like, one person had a really small apartment and
she changed in the room that it was going on so she was very conscious about
making sure the recording was completely turned off at that time. One participant
didn’t care. He was also the type of person that when we asked about like your
routine that they did before bed he was very forthcoming about the types of things
he did in his bedroom and made my poor graduate student blush like crazy.
And so for him it didn’t matter as much. And so I’ve got a view of the different
deleting patterns that people did as well, which I can show you here. And this is a
chronogram of what was going on throughout the night. So the light blue indicates
the sleep periods, and so the green is when they’re viewing the history screen. The
home screen was visible and the delete screen. So the first person was the one who
did changing in the bedroom so she was pretty conscious about going through and
deleting things.
This is that guy who is, like I mentioned, very forthcoming and he only deleted two
things across the two-week period, and was very open about what those things
were. And this guy he actually had his wife with him and they’d had a lot of private
conversations around bed time and so that was important for her, and he didn’t care
but his wife cared. And so she was very much like oh, make sure you go back and
delete that. And this guy was mostly just kind of playing around with things and he
was like ah, there’s nothing really there that I’m worried about.
So he just didn’t end up deleting anything for the rest of it. But yeah, that’s a good
question. So it was kind of interesting to see that people are actually bringing up the
history almost every day. So you might expect to see sort of a novelty effect where
people are really looking at it a lot in the beginning but people seem to be using it a
lot throughout the whole week, which is kind of promising for this that there was a
sort of fascination with what was going on in the middle of the night that people just
don’t know what’s going on.
And so we’ve been taking this work a little bit forward and looking at things. So
right now we’re just kind of show the raw data and allow them to do it but it’s kind
of hard to make those comparisons yourself. So we’re looking for likes like
correlations and doing alternate visualizations that show like just simple tech space
things like, you know, it looks like when the temperature is above the recommended
level that you’re having more awakenings, so these sorts of insights about what’s
going on with their data because people can get overwhelmed.
Some people are really fascinated and would really look at it in depth, but some
people were just like oh, it’s too much for me to look at. So yeah, those were some of
the things that we learned there.
We’re also looking at we’re working with some people in rehabilitative medicine at
UW on uses with multiple sclerosis patients because they tend to have a lot of
awakenings at night. Some of it is associated with their pain; some of it’s associated
with other sorts of things. So we’ve been using this to sort of identify the different
disrupters that are going on.
Yeah?
>>: It sounded like maybe some of the participants had [inaudible]. How did you
tease out the [inaudible]?
>> Julie Kientz: Well, the sleep was a wrist-worn thing so it was only worn on the
participants themselves. But the movement, yeah, there’s a lot of stuff from the
partner as well, so your partner is actually a big source of disruption based on our
literature that we found. So snoring partners is important, wakening up restless
partners. So yeah, those sorts of things should be captured and they were able to
look at that. But ->>: Yeah, I was thinking like I like sleeping when it’s cooler and she likes sleeping
when it’s warmer.
>> Julie Kientz: Yeah. So there is some of that as well. So definitely fighting over
the thermostat for one of the couples was there. And so yeah, that’s hard too
because then you have to think about finding a compromise for those things because
if you talk to the sleep doctors they’re very extremist and they’re like, well, sleep in
separate bedrooms. And it’s like, that’s not going to work for most people, right?
Most people like sleeping with their spouses, and yeah some people like sleeping
with their pets in their bed and there are things that people are making tradeoffs
for, and so this is why we think this sort of tool is useful for helping them
understand that.
For me, my partner was maybe moving all night but it actually wasn’t disrupting my
sleep. So I’m not going to kick her out into the guest room. Or my dog sleeps on our
bed and I was using this for a couple nights, and there was some motion going on
and she got up and did things that didn’t wake me up. So I was like, well, okay, I
think for me it’s okay to violate those principles because for me it’s not having that
impact. So the multiple partner thing is another interesting exploration of how to
merge the two.
Because yeah, fighting over blankets and things like that can be a problem as well.
Any other questions about lullaby? Okay.
Yeah?
>>: So the privacy concern that I’m curious, you know, what were participants’
expectations with how much the researchers would see or if they were using it for
long term how long would their data be kept ->> Julie Kientz: Yeah, yeah, yeah. So we have to at UW we have an interview
process. So we had tease out all these details about that. So we told them -- we sat
down with them before we ever did it and said like we’re going to show you how to
delete everything so you’re not going to see it, and some people said they didn’t care
and just said like use whatever data you want, I don’t care. But for others it was
important so we sat down with them and deleted all the data with them.
And the use case for that is just for their own personal use because they’re in a study
they were obviously given the technology back to us, but if it was for personal use
then I think it would be less problematic because there is this sort of trade off
between deleting your data and having the data to be able to watch and look at,
right? And so yeah, some people were a little bit conscious about he researchers
might get this data, but the one guy who didn’t care was just like if this was my own
I wouldn’t do anything about it I’d just let it record all night long.
So thinking about who is going to access the data is definitely an important
consideration as well. All right, so I’m going to move on to now a different point of
view. So sorry about the little animation thing there, it’s stray little animation I
didn’t think about.
So flipping around to a completely different thing, and something that’s been on my
mind a lot lately is how do we as parents sort of understand the sorts of things that
are going on in our kid’s life and know that things that they’re developing well and
things are on track? So I’ve been teaming up at Washington State initiative for their
goal is to track every child from birth through age five and get them regularly
screened for meeting different developmental milestones on track.
And so they want to make sure every single child in the state is covered. So right
now there is a similar system in place for vaccinations, so any time a child gets
vaccinated it goes into a central database and then any time that they go to a doctor
they can bring up those records. And so they want to make sure that they’re getting
them regardless of income, residency, or background. And so the screening process
is a 22 different questionnaires across a five-year period, and each question asked of
children have met 30 different milestones. And so by milestones I mean things like
at eight months does Samuel turn his head in the direction of a loud noise?
So if he’s not responding to that then that could be a sign of hearing loss, or also a
sort of cognitive impairment that sort of manifests as not reacting to sounds. At 14
months does Sally play with a doll by hugging it? So that’s sort of a socially
important thing to recognize that you hug things that look like humans or animals.
So that’s a social thing, so again things like autism might manifest by not showing
those sorts of activities.
And by two years can Ayesha kick a ball by swinging her foot? So that’s a physical
milestone and there might be a sign of a physical impairment if she’s not doing that
by that period of time.
So our question was how can technology support tracking all these things for every
child in Washington state over a five-year period? And one that reduces the
emotional and financial burden. So remember they want to get every single child in
the state of Washington regardless of their income or background, and speaking
with the people running this organization, you know, the people from the upper
middle class families were the ones that were fine, they were getting diagnosed
pretty early, getting their services pretty early. It was definitely the kids that came
from the under-served populations that were falling through the cracks and not
really getting diagnosis till about kindergarten or so.
And usually the idea is that if it’s by kindergarten it’s almost too late because then
[inaudible] functional language at that point together kind of keep on path with
those things. So there’s really this pressure, there’s actually a federal mandate that
any child under the age of three gets free services if they are diagnosed with a
developmental disability.
But as I mentioned earlier, thinking about your kid having a disorder or not being
ahead there’s a huge emotional burden associated with that. And so we had to think
very carefully about how to design this sort of system that parents would be
tracking this information, get feedback from people without making them obsess
about every little thing that their kid is doing and then also thinking about the
financial burden. So how can we make things that are accessible to a wide, wide
range of people? Washington state is very, very broad and we actually had our
participants gave us a four-page Excel printout about under-served populations, and
it included things like people from different cultures, people living on the tribal
reservations, people who are in homeless shelters or domestic violence shelters, you
know, so all sorts of ranges of people are living in Washington State.
So we had to think about how to approach this from a broad point of view. So our
solution is to do this sort of ecosystem of tools. So rather than one tool we actually
have a whole set of tools. So we did some qualitative work to understand what
types of things people had access to and try to get the whole gamete of things.
And so the idea is getting all this information to the centralized database. And so the
coalition we’re working with is worrying about once it’s in that database what do we
do with it? So they’ve got a whole system in place for following up with people and
getting them connected to a pediatrician or getting them a provider and getting
them connected with services.
So the problem we’re looking at is how did that data get into the information in the
first place? So a lot of populations we looked at had access to text messaging
services, so that was a pretty basic service. Websites were pretty important for a lot
of people. Lots of people are using facebook and twitter for social media stuff,
maybe not the lower income people, but a lot of young people are using those sorts
of things and those are the target populations.
But then for people who didn’t have access to technology, we have a public kiosk, so
the international community health services is doing this model where having a
kiosk in their waiting room has been really successful. And then the other issue is
that face-to-face is also important for a lot of these populations too. So we looked at
people who would go to migrant worker communities and go door-to-door and ask
them the screening questions. But right now it’s paper-based and so they have to fill
lout this thing, type it up and put it into the database.
And so we’ve got some students developing a tablet-based tool for doing that faceto-face communication. So we’ve been focusing on, we’ve got prototypes of all these
components, but we’ve been focusing on three in particular that we’re going to start
a study on this summer. So I’ll talk a little bit about those. So the web portal, this is
still under construction so this is a screenshot of its current state but I’ve got my
students who will be finishing it in a month or so. But the idea is that we have sort
of a timeline metaphor.
So one of the things that we noticed is that a lot of these milestones, while they’re
good for medical purposes, they’re also really important milestones to celebrate in a
child’s life as well. So when you fill out the questionnaire you have the option to add
it to a timeline that shows up about your child’s information. So it’s very much
about like, you know, chronicling your child’s life on these sorts of things.
And so you can kind of scroll and do those sorts of things. So we have the
developmental tracking information, but we also want to combine it with the
sentimental record keeping as well. So lots of parents are taking pictures of their
kid or tracking things like first trip to the zoo. So if we combine those sorts of things
we can reduce sort of that emotional burden associated with the medical tracking by
combining it with the fun stuff that parents also want to do.
So parents can add memories, videos, photos, add that to the timeline as well and
sort of curate the sort of timeline about things that are going on. We have a way of
embedding this onto a website. If you want to add it to your blog we can also make
publicly sharing things so you can add it to facebook. You know, we toyed with the
idea of posting each thing into facebook, but our research showed that people who
are friends of kids might worry about over-share of every single little thing that
their kid does. I know I’m very conscious about it myself having a daughter now I’m
very selective about the things that would annoy my friends.
So we decided to go with the route of they could share sort of the collection of
information as opposed to each and every milestone. The questionnaire is built in to
the system as well so you can answer the different questions about when you look at
a site does it look in the direction of your voice, and then you can click add to
timeline if you respond to that.
In terms of showing how progress is going we toyed with lots of different
visualizations for this, so things like graphs, or things like percentiles or all these
sorts of things about how do you present the information to people about how they
are doing?
But all those things had different types of problems and showing a graph of
someone’s progress can be really scary. And it turns out that the pediatricians don’t
even do that. They just tell them, yeah, things are looking good, things are close but
you might do these sorts of activities to encourage that behavior if you’re not seeing
it yet, but then there are also things where it’s like you might want to try and get
them screened for this particular disorder.
So we have a tree visualization here. So the tree is fully-grown they are on track and
everything is great, they are good to go. And this is going to be most participants, so
most of the screening stuff, you know, probably 85 percent of participants are going
to be across the board fine for how the tools designed and developed, but if they’re
in these sort of two middle categories and we show not that they’re behind, but that
their tree is not yet grown. So sort of have this metaphor of growth and that it’s still
a seedling so it’s got potential to grow, because there are things that you can do.
So especially if they’re in this middle state a lot of times they just haven’t had the
opportunity to do these sorts of things, so things like grasping small objects. I was
filling this out for my daughter and it said like, oh, grabbing cheerios, and I was like
well I haven’t even given her those yet.
So that kind of prompted me to go and find some cheerios and let her try and
practice with those, and then if they’re not yet growing that’s sort of the thing there.
So that’s what we’re trying to do to sort of minimize that emotional burden
associated with things.
We also have been using social media stuff. So part of thesis is that a lot of these,
once your child gets to be older some of these questionnaires are spaced out by six
months. And so a lot of times you might go to a website, use it a lot, and then forget
about it until you need to do it again. So in order to keep people engaged we
thought about trying to take advantage of things that people already are using. So
we also have a twitter-based version of Baby Steps and so the idea is that you follow
the birth month of your kid on Baby Steps. So my daughter was born in November
2012, and so I’d follow the Baby Steps November 2012 feed, and then I’d see things
like this that show up in my feed, you know, does your baby turn his or her head in
the direction of a loud noise and then a code word.
And then I can respond back to that and say yes, Maya turns her head in the
direction of a loud noise with that code. And so automatically we have a tool that’s
scraping that information and adding it to the database for them automatically so
they can kind of doing it when they are dong these sorts of other things.
And then similarly we also a text messaging based system that does the same thing.
So they can sign up to receive text messages and reply back to those and then we’ve
got a system that will write it into the database. And so it’s actually possible to use
the entire baby steps system without the website, without all of that. You can do it
entirely through text messaging as well.
So that was trying to get at sort of the different sorts of burdens there. So our next
steps with this is to study those three different technologies this summer, and then
finalize the functional prototypes at the kiosk, the face-to-face tool, and the facebook
tool, and we’re launching a 1.5 year trial. Because the hard part is to do these sorts
of short studies, childhood development takes a long time and so we’re having to do
the long-term study route without about 40 families starting this summer.
And we’re looking at things like the emotional impact of using these sorts of
technologies and how if we did a good job of sort of mitigating the emotional burden
of these things, whether or not we’re improving access to these for people with
financial problems. Looking at things like self-efficacy and also patient activation,
does the parent take an active role in managing their kid’s health care?
Questions about baby steps?
Yeah, Dan?
>>: This kind of seems at risk times 100 for everything, I’m a super believer in
[inaudible], on the flipside I also [inaudible]. Like when you’re at a museum it’s sort
of sad how disruptive smart phones can be even to the best of intentions, like you
take a picture you just don’t realize that 60 seconds can [inaudible]. I wonder if
you’ve thought about how to mitigate, recognizing the positive in doing this, I just
mean the mean goal is the developmental [inaudible], mitigating the potentially
disruptive impact of every extra reason you give a parent to touch a smart phone
there’s some real risk associated with it.
>> Julie Kientz: Yeah, I’m totally with you. I’ve seen how my daughter sees s smart
phone even at six months and it’s like she is fascinated by it. And I’m totally
conscious of it. And so it’s hard, it’s definitely a hard problem. We’ve been thinking
about it by having a variety of different things that they could do. So they don’t have
to use a smart phone, they can use some of the other things that they are doing. And
we’re definitely talking about a small amount of things that people are doing, you
know, 30 questions over a two-month span sort of thing is sort of, it’s not even one
question a day if you’re looking at spacing it out at that point.
But yeah, you’re right. If they get that text message while they’re playing with their
kid they might be compelled to pick it up and -- I think it’s still an open problem. I’m
not sure how to address it. We’re definitely conscious of it, but ->>: Have you considered how [inaudible] just defer as much as you can till the kids
go to bed? And like photos [inaudible] like you may use this as an opportunity to
defer all that texting and captioning until after bedtime.
>> Julie Kientz: Yeah, that’s a great idea. Right now we’re actually still testing the
prototype and we’re actually debating about what time to send the messages
everyday and I think that that’s actually a great idea to ->>: [inaudible] like when the kids go to sleep if they’re even vaguely consistent it’s
an opportunity to ->> Julie Kientz: Right, right. Yeah, yeah, yeah. That’s a great idea. So I will tell my
student working on it. Very cool. Other questions about baby steps? Well results
will be TBD since we’re launching the study this summer, but our pilot tests have
been going pretty well so we’re looking forward to that. I actually need to track my
daughter’s own stuff. The caveats of using a research prototype for things is that
there’s a glitch in the database and I lost everything, but it was nice to be able to go
back through and reflect on it a little bit better.
Also one of those things that it gave me sort of faith in the user-centered design
process because I’d been doing all this stuff before I became a parent, and then I
became a parent in the middle of it and then all of a sudden I became the user that I
was designing for, and it turns out that I was like hey, I actually do need this. And so
it was kind of nice to think about yeah, I was very scared that after becoming a
parent I was like oh this is never going to work and this is terrible, and it actually
turns out it’s pretty useful for those things.
And I am definitely biased [inaudible] but it has sort of been a nice thing to use for
my own daughter. So last thing that I’ll talk about is looking at sort of the access
burden. So we talk about physical fitness for people, you know a lot of things that
their doctors can do are sort of great for everyday people, but how do you help
people who might have a different impairment?
So one of my students Kyle is looking at how do we kind of lower the barrier to
health and fitness to people who are blind or low vision? And so one of the things
that we looked at she did a formative work to look at this and simple things like
even going to an exercise class can be really, really scary to people because they
don’t know what to ask, they might be embarrassed about asking for
accommodations for things. And a lot of times things just like a fitness instructors
just don’t know enough about how to correct that.
So just the thought of going to a class is sort of terrifying for them. They may have
never tried it, so how can we sort of get over that barrier of access for that? So the
design question we started out with, you know, how can we encourage physical
fitness for the visually impaired in away that reduces the access and emotional
burdens associated with those?
So what we’ve been doing we’ve been playing with connect. You guys are Microsoft
so you’re quite familiar with that technology I’m sure. So there’s a lot of connect
yoga games out there already, but we played a lot of them and almost every single
one of them requires some sort of visual thing. So it’s like here’s the on screen body,
it recognizes your thing and it’s like you move your arms to match up with the onscreen avatar, which works pretty well for if you can see it, but if you can’t see it
then there’s some problems with that.
And so she actually has been using the connect tool kit to give audio based input
only for these things, and so she’s been using just sort of basic geometry to look at
the motion of it and then translate that into an audio based feedback to people. So
in this case you’re keeping your arms, you know, moving your arms closer to your
head if they’re out here and moving them out. And yeah, things like turning your
hips slightly this way and -- it’s really fun testing this technology and my poor
student Kyle was like I was so sore after debugging this technology, but basically
being able to give feedback to people using only audio.
And so, you know, for example, you know her arms are at 45 degrees and need to be
raised to 80 degrees, and so it gave that feedback until it would say good job when
you’ve gotten to that particular point. We also had an audio-based tone when you
got to the right part it kind of gave you a happy little chime that you were there. So
we designed this study with a bunch of yoga instructors. So we actually had a yoga
instructor record them instead of just using speech-generated feedback we had
them record the different prompts.
They went through and evaluated the different poses and things that were going on.
So it was really great. We just kind of emailed random yoga studios and people
seemed totally excited about this and so they were able to donate their time to help
make this actually really a good thing. They actually had like CDs of yoga music that
plays in the background and everything too.
So my student went down to the Washington State School for the Blind in Vancouver
and used this with 16 participants, and we looked at a couple things. So participants
reported feeling more confident and more likely to try yoga class after playing it. So
they knew what the poses were and they kind of knew what the [inaudible] were so
they felt much more like hey, I think I could take a yoga class now, that’d be really
great.
I think because I understand the poses a little bit more I’m willing to try that out.
The other thing we did is we actually took video recordings of all their poses and
how the yoga instructors look with and without the audio feedback, just looking at
the correctness of their pose, the results were not as good as we had hoped in that
they were all pretty good, but the poses we were looking at were all fairly simple.
There’s a limitation of the connect that they had to do all standing poses, but lying
on the ground with a connect didn’t work out so well.
They also rated the game pretty highly on different standardized scales for
measuring games. So we got some pretty good results for that. So any questions
about the connect yoga study?
Yeah?
>>: [inaudible]
>> Julie Kientz: Yeah. So we were thinking about that but thinking about the
physical burden we’re not quite sure what to do with that. We’ve been mostly
focused on it because we didn’t necessarily need it. It seemed to be pretty effective
with audio only and when we asked about things like distracting the noise and
things like that appearing -- people who are usually vision impaired are used to a lot
of audio feedback and so for them it wasn’t problematic for them to not use the
audio feedback. But I think it would be interesting to explore if you had a
comfortable sort of hap tic system.
>>: So audio feedback as it is will replace the instruction, the fact that they can’t get
instruction visually, but the other key visual -- even with roughly normal vision yoga
is hard to balance and audio feedback is not going to help that. I mean [inaudible]
but if there’s nothing to grab on to or nothing and it looked like a potential hazard if
you don’t know the area around you. You’d have to clear the area and all that. So ->> Julie Kientz: I think that there’s some potential for that but we didn’t explore
that but that’s definitely room for exploration. But yeah, balance was definitely an
issue for when we did tree pose for one of them, which is a balancing pose, and for
them like doing this sort of pose was actually problematic for someone who can’t
see because a lot of the times when you ask them to do a pose one of the tricks is to
look at a spot so that was problematic for a lot of people with issues of balancing. So
we might explore something along that line. It’s a good suggestion.
Desney?
>>: What’s the set of linguistic primitives required to get someone to [inaudible]? A
preventative sort of work across the volume?
>> Julie Kientz: Yeah. So it was actually interesting too because people responded
much better to things like metaphors, right? So, you know, when you talk like move
your arms closer to your head there’s a lot of things that you might think about, like,
oh like this? Or like this? Or like that? And so we use a metaphor of like you’re
reaching up for something in a cabinet that actually worked better than just moving
your arms closer to your head. So we did a lot of iterating on that.
And the other thing that was interesting was like some of these metaphors don’t
really apply to people with visual impairment. So one of the poses was like move
your feet close apart like you’re on skis, right? And some of the people had actually,
you know, blind people do ski and actually it worked for some people, but a lot of
people had never even thought of skiing and like how you’re supposed to have it.
And so some of the metaphors worked pretty well and some of them didn’t.
And so the other thing that was interesting was that move a step forward was
interesting too because, you know, for us it might be like this, but a lot of times blind
people are used to taking such small steps like if they use a cane, that for them it
wasn’t a big enough step and so we had to think about, like, they actually responded
pretty well to metrics. So like, take a foot, move forward by one foot as opposed to
take one step forward. Yeah?
>>: Did you also do any experiments with tones? So things like minor or major or
low ->> Julie Kientz: Yeah, we did. Yeah we used different sort of chimes and things like
that, and we’d experiment with things kind of like sonar thing like you’re getting
closer and that was just too annoying for people. Yeah, yeah, yeah. But we did use
sort of -- but I think we could do some more with that in terms of like different types
of tones and how close you are, but just sort of like we did have a simple tone that
sort of when they got it right. But we would experiment with like play a tone until
you get it right didn’t work very well because people just kept getting frustrated
with that. So yeah, but we were playing a little bit with the tone and that, but I think
there is some more we can do there.
Again the limitation of this is that we only did six standing poses because again the
floor poses were hard. We had one person suggest putting the connect on the
ceiling to do that, so we might try and play with that next to see if that works.
Because the other thing that’s nice is you can think about things like situational
impairments. If you’re on a floor pose sometimes it’s hard for you to see anyway. I
remember my first yoga class they made us do the rabbit pose and it’s the one
where you’re like on your head and it’s like we recommend watching what other
people do before you try it out yourself because you can’t look around when you’re
in that pose.
So it might be nice if you can do that. Yes?
>>: If you put the connect on the ceiling when you’re down for the floor poses and
make them [inaudible]?
>> Julie Kientz: Yeah. Okay. Okay, yeah. That’d be great. That was the suggestion
that we got just a couple weeks ago so ->>: Yeah, sort of like [inaudible] or something like that that’s [inaudible].
>> Julie Kientz: Yeah and I think since you have a mat anyway then you’ve already
got an excuse for sort of instrumenting the floor, so I think there’s a nice
opportunity for that. Cool. Yeah this is a fun project. My student has been exploring
other sorts of making other sports more accessible for people. The next thing she’s
working on is rock climbing, which has all sorts of interesting things. We’re still in
the sort of explanation part of that, but that’s been a fun thing to work about as well.
So just kind of in summary. So I talked about four different applications and the
different types of burdens that we tried to address through the design of the system.
So shuteye looked at sort of the mental, physical, and time-based burdens, lullaby
looked at the privacy and physical burdens, baby steps looked at the emotional and
financial, and Eyes-free yoga looked at access and time and emotional burdens as
well.
So in all the work we’ve done we’ve kind of come up with some different strategies
and how to reduce the types of user burdens. So imbedding technologies into
already existing activities. So like parent are already tracking information about
their kids in terms of the sentimental stuff. By embedding it into that activity
already that can be one way of sort of reducing that.
So providing multiple technology options, so again like the baby steps, providing
anything that they’re familiar with or comfortable with can be a way of using the
system can also provide that. Removing the need for manual logging and tracking as
much as possible, but only to a certain point because a lot of times there is a benefit
to actually doing the logging that you become more conscious of it. So finding out
how you can sort of find this nice balance between automated and manual logging
as well.
Making, so for things like privacy burden, making the recording methods
transparent and make it really easy to delete things if possible and only record the
things that are necessary or central to your goals is important. Reducing the need
for on-body sensing. This most applies to sleep. There are other things that I think
on-body sensing works for, but that sort of thing can be problematic for things and
user memory is an important thing to think about.
So some next steps, so I talked a little bit about burdens, but one of the things that’s
hard is these are all kind of retrospective on the studies that we’ve already done.
And so one of the things that we want to do to sort of think about this is from a
model-based perspective. So can we model the different types of user burdens that
people might consider, and also what those trade-offs might be. So by maximizing
different burdens you might minimize others, or vise versa. So like maybe if we’re
so conscious about privacy then it’s going to make it really, really problematic in
terms of a mental burden, so looking at what those different trade offs are and how
to maximize those for different possibilities for different applications.
The other point is that sometimes people are willing to put up with different
burdens if they see the benefit for things. So people might be more willing to track
privacy based things if they know that there is a high need situation for it. So
thinking about the benefits as well.
The other thing that we’re doing is it’s hard to measure burdens other than asking
people. So we want to actually develop a validated measure that we can use for
measuring these different burdens for things. So there are things out there like the
NASA TLX and like the technology acceptance model that sort of fit into these, but
we’re interested in sort of having a combined tool that you can use for assessing
this.
And then finally with all the health technology we’ve been thinking about when they
do sort of HCI stuff it’s sort of interesting to explore this level, and the health people
that I’m working on keep saying like, yeah, this is a fun little project but how are we
going to test the efficacies? So my next step is sort of thinking about how to take
these to the clinical trial level. So we’ve already been exploring that with shuteye
just because it’s such an easy thing to deploy to large amounts of people that those
are sorts of things to look at that level.
Other projects if you want to ask me about later, I’ve been working with Fred Hutch
Cancer Research Center on a smoking cessation app, working on assessing
sleepiness through reaction time tests and such. Kind of gets Dan’s point about
maximizing sleep. So we need a way of prioritizing sleep, so we need a way of
assessing the impact of sleep on a regular basis. So we developed an application for
assessing sleepiness to objectively measure that.
And then 40 winks is the application for visualizing the impact of sleep deprivation
on people’s lives and hopefully helping them prioritize that over things. So with
that, just acknowledge all my awesome Ph. D. students and, of course the people
who paid my bills [inaudible]. So thanks.
[applause]
I don’t know how much ->>: Enough for one or two questions maybe.
>> Julie Kientz: Now that [inaudible], is there anything else going on that you’re
aware of that seems like the next exciting thing happening in sleep in the
commercial sense? Is it happening besides -- there’s going to be 20 things to do that
seem [inaudible] but anything else, you know, bedside or [inaudible] or anything?
>> Julie Kientz: So a lot of people are looking at things like load sensors under the
bed. There’s lots of people been using vision based systems as well to track kind of
movements. There’s an app that you can get, sleep cycle on the iPhone, that you put
the phone on the bed itself and it uses the accelerometers to measure movement.
Unfortunately they don’t give you any information but they claim that it’s been
compared to some of these gold standard things, but with any commercial
application it hasn’t yet been validated, you see reviews saying like oh this works
great for me and other reviews that say like, I was out of bed and it thought I was in
bed and so they do all these tests and they’ve done that.
But I think wearable stuff in terms of things in bed and into the clothing is a really
promising thing that sort of can get at things like breathing and heart rate since
those can be important for that would be pretty interesting. There’s a [inaudible]
shirt that’s out there. So they only have a start-up page like come back soon, so
that’s sort of something to look at.
We’ve actually been experimenting with doing it completely unobtrusively. So
we’ve been colleting a lot of data on the smart phone itself for looking at things like
last time they interacted with a phone, if it’s charging or not, if the alarm is set or not
and comparing that to a ground truth and seeing if we can predict if they’re asleep
or not, which so far we’ve seen some pretty promising results. I think we’re not
going to get to the point say down to the minute and you’re in deep sleep versus
REM sleep, but we’ll hopefully at least be able to get core sleep times and things like
that for longer term.
>>: Is it surprising that, like, given that every company that makes anything now
has apps and [inaudible]. I’m just surprised that I haven’t seen anything from like
mattress companies or pillow companies that are sort of in the sleep business, or
pajama companies trying to do app anything. Especially things like sleep number, I
was really surprised maybe it’s just like hospitals they just kind of always
[inaudible] but I ->> Julie Kientz: I think what’s part of that, and like I said like the aptograph
[phonetic], which fitbit and all these other ones use took 20 years for the sleep
community to finally recognize that that was actually a valid measure for sleep, and
it comes with all these caveats and FDA approvals and everything like that. And so I
think people are just, you know, people go for the commercial route but sort of slap
these little notices like for entertainment purposes only or something like that
because they can’t say it’s for actually diagnosing sleep for that reason.
Yeah, Scott?
>>: Do you know if anybody has messed around with the sleep number beds?
Because they’re like already built to sense what’s going on, right? They already have
air chambers that are already hooked up perfectly to sense what’s going on with the
bed.
>> Julie Kientz: Is that how they -- I don’t know my parents have one but they
actually have to manually control it.
>>: It’s whole bunch of air chambers that are built in such a way that if you put
pressure down in one place ->> Julie Kientz: Oh it actually does some smart thing. Oh, okay. [inaudible]
>>: [inaudible] we have which, like, when I [inaudible] I’m going to experiment with
every aspect of like, you have to, you know, you say I want to go to 70. It has to
actually sense to figure out where 70 is. It’s like a pressure vacuum.
>> Julie Kientz: And so it is specific to you, it’s not just a set like 70 for me is not the
70 for you. So it does sort of sense the --
>>: If we weight different things [inaudible] 70 resistance it has to sense how much
-- so it will give you 70 no matter what weight is on the bed right now.
>> Julie Kientz: Okay. So it’s not like ->>: So you can instrument those channels with the pressure changes in it it’s totally,
it’s a pretty straightforward instrument.
>> Julie Kientz: Yeah. We don’t have space at our lab for a mattress but ->>: Do you know if anybody’s done this?
>> Julie Kientz: I’ve never heard of that. I mean, like I said I think the main thing
instrumenting the bed is like looking at load cells under the bed itself. People have
looked at mats that you can put under there that sort of sense movements and
things like that.
>>: As long as the mattresses are so expensive, like they’re crazy, it’s no regular
mattress for some of them are so expensive I’m just surprised there’s not more, like
when you start with that expensive you kind of find reasons to throw in [inaudible]
more with the technology to make it more [inaudible] to your competitor I’m just
surprised the mattress the mattress is [inaudible].
>>: [inaudible]
[laughter]
>>: I think I proved it by his silence.
>> Julie Kientz: Yeah I didn’t know that they had that sort of, I thought it was just
sort of a standard thing. Yeah, that’s cool. Yeah.
>>: [inaudible] and I need a better napping spot than my couch.
>> Julie Kientz: Yeah, me too.
>>: She’ll be around for a little bit with me, you guys she’s running a little late so ->> Julie Kientz: Yeah, not too far. Thanks!
[applause]