>> Ranveer Chandra: It's my pleasure to welcome Jon for an interview here with MCRC.
Jon has been really prolific at University of Washington. He has published lots of papers.
He has gotten tons of awards; I had to write them on my phone just so I would remember,
including the Microsoft research Fellowship, the best paper nomination at UbiComp, best
paper at Chi and most recently the University of Washington College of Engineering
student of the year, it's innovator of the year, right? And his technology has been
licensed to Belkin as well. He has been doing a lot of practical stuff and that is what he's
going to talk to us today about.
>> Jon Froehlich: That's right, thank you. It's a pleasure to be here. Thanks everyone
for coming. So really the focus of my dissertation work has been on environmental
behaviors and so that is what I'll be talking about largely today. There is often a
profound disconnect between our everyday behaviors in the world and the effect that
those behaviors can have on us, our health or the environment around us.
So there have been two main parts of my dissertation on transit and on home resource
consumption, so those are really going to be the focus of my talk today. So I thought I
would start out today with some relatively startling statistics about consumption. In the
United States we probably consume 346 thousand gallons of gas per day, which that may
seem like a lot and it is. It's more than twice as much as Japan, China, Canada, Russia
and Germany combined. And this accounts for about 26% of our carbon emission
footprint, so there's also some sort of other environmental consequences. These problems
really are only becoming more significant with growing population, economic
development across the world. For example in Beijing they have their water
consumption which rose to nearly one trillion gallons of water, but the main problem here
is that their water supply infrastructure only supplies about 576 billion gallons of water.
So there is a big difference there, obviously. And so the Beijing government has gone
and started melting snow to provide fresh water to the citizens. They also import
freshwater by tanker ship so they are going to unprecedented lengths to provide this
scarce resource to their citizens. And these sorts of problems don't just affect
international cities but also here in America, really all along the Colorado River basin.
For example, Lake Mead, which supplies 90% of Las Vega’s freshwater. The intake
pipes of the water levels are supposed to go below the intake pipes in the next five years.
So these are big, big problems, right? And because of that they require multifaceted
solutions that incorporate economic, political, behavioral and technological solutions.
And I really have been focused on these latter two areas here, so behavioral and
technological, and it's really no coincidence that I selected the Toyota Prius to represent
technological innovation because they also have a behavioral component as well. And
that is in terms of their Prius interface, which you may have seen if you have ever been in
a Prius. It gives you instant feedback about your driving performance and this is really
led to a movement of hyper milers, so people are able to gain efficiency out of their
vehicles simply by focusing on the feedback that the car is giving them. And this is an
example of eco-feedback, which has been largely the focus of my dissertation work. And
these systems have three components, you, some kind of sensing system and some kind
of feedback system, and I will come back to this diagram later in my talk.
So what about sensing and feedback in the home? Well ordinarily if you think about
electricity we have to go outside of the home, right, to see this electricity meter that gives
us real-time information on consumption. But even then, even if we were able to look at
the meter, it is really in esoteric units that don't make much sense, so kilowatts per hour.
Another way that we get access to this information is through a traditional bill, but this is
temporally disconnected. These arrive about a month or bimonthly so they are
temporally disconnected from the consumption activity. Interestingly there's a start up
company called OPower which has been tremendously successful. They are taking the
same data that energy utilities have access to, all energy utilities, and they are just
reframing it. They are re-visualizing it in the bills and they have been successful, in fact,
2.5% savings just amongst their constituencies who are getting this reframing of the data,
this re-visualization of data. And this accounts for about 20 million tons of coal or the
yearly output of four nuclear power plants. So these are significant, and just by changing
the way that the information is rendered to the people, the home occupants.
So can we do better than paper? Well I think we can. This is actually a tremendous
opportunity now. We have a plethora of different kinds of displays from iPads and iPods
or Windows Phone 7 devices all the way to in the home, where we are starting to see
smart thermostats and different kinds of ambient displays, and not only that, not only
from the display standpoint but also from the sensing and inference standpoint, right? So
we are starting to see sensing and inference allow us to get access to different kinds of
human behaviors and data about different kinds of human behaviors. So returning now to
this diagram, there are really four underlying questions here. So there are two that are
embedded in sensing, what behaviors should we sense, and how? And for feedback, how
should we present this data back to you? There is a giant design space here. And then
finally what impact does this feedback have on your behavior?
So that was the intro of my talk. The next part of my talk is going to be on a design space
which really provides kind of a framework for thinking about how we might want to
build these different types of feedback visualizations on environmental behaviors. And
then I'm going to go through two concrete systems. The first is UbiGreen, which is a
system for transit behaviors, and the second is all about water consumption in the home.
So HydroSense is a water consumption sensor in the home, and then Reflect is a way of
presenting that information back to home occupants. And then finally I will finish with
future work.
So let's take a look at this design space. Really there have been a number of interesting
kinds of ways of rendering consumption information that have come out of design HCI
UbiComp communities in the last five or 10 years. This one for example, this power
aware cord pulsates and lights up depending on the amount of consumption that the
device or appliance that is plugged into it draws. Microsoft Hohms, there have been a
number of industrial players, Google Power Meter, Microsoft Hohm; they have websites
where you can get access to your electricity consumption information. Here we have the
Energy Detective. And notice this is a real-time display. Notice they use monetary units
here in addition to kilowatts per hour. So there are number of different ways that we
could really think about and brainstorm around rendering this kind of information to you.
What's interesting is when I started to really do some critical literature searches, I found
that the first display came out in 1970s. And coincidentally actually this probably came
from the University of Washington, Colin Bart, and basically what they did is they
instrumented a light bulb. They specially designed a light bulb which would illuminate
when home occupants were within 90% of their peak energy levels. And they found that
this was enough, this simple light bulb was enough to change people's energy behaviors
in the home. So I think that really gives rise to two kinds of questions. What makes an
eco-feedback design system effective, from a light bulb all the way to these more rich
kinds of displays we are seeing? And the second one is how can we better understand the
trade-offs, constraints and different kinds of motivational strategies that we use in these
eco-feedback designs?
To get at these two questions I conducted a very large literature search. And I didn't just
target this literature search and HCI and UbiComp, but I also extend beyond that and
looked in environmental psychology, behavioral psychology and some of the health
behavior literature about how we can affect behavior through information. So let's return
to this eco-feedback design space. There are really two points to this. The first is a
critical lens for us to evaluate and analyze existing systems. And the second is a design
framework to allow designers to approach something principally and think about the
trade-offs that they are making when they make these designs, because it's a giant design
space. So there are nine points here. And each one of these points has a number of sub
dimensions. So I will go through them briefly.
So inputs, how do we get access to the data? How does the feedback system get access to
the data, what does it look like, how often, what is the sampling resolution, is there any
kind of self-report? The next three data representation information access and display
medium, basically how is the data presented? And then the next five I think are pretty
much the most interesting, which kind of relate to what are the persuasive strategies that
are used to present a look at the data to provoke or promote certain kinds of behaviors?
So these are comparison, action ability, motivational strategies, social and behavioral
models. So let's use this, let's return now to the Toyota Prius display and ask why is the
Toyota Prius display effective? And then let's use that design space to sort of situate that
when we ask this question.
I think there are a number of reasons. When you're driving it's a constrained activity.
You're focusing on the driving activity and you are getting information about that from
the Toyota Prius display. It is also real time. So you're getting real-time information
about driving behavior. But I think what's most interesting perhaps is comparison, which
has been found in psychology to be particularly motivating for certain behaviors, and
they do this comparison in a number of different ways. They basically allow you to
compare yourself to past performance, right? So here you have a current mileage, an
instantaneous mileage that you're gaining out of the car but it also allows you to compare
that to sort of ground it. Is that good or not? Yep.
>>: Did you do studies of this Prius effect or are you just taking-- I mean I'm wondering
if [inaudible]?
>> Jon Froehlich: So this is somewhat anecdotal, but the University of California has a
large study now that NSF commissioned to actually explore this in a longitudinal fashion.
And we've seen a number of other car manufacturers adopt these same kind of strategies.
So there is evidence that it's...?
>>: [inaudible] doesn't mean much right? I mean customers might really like it for the
first two months but that doesn't mean it's enough for them to sell cars. That doesn't
mean it has longevity [inaudible]
>> Jon Froehlich: I think that's sort of currently up for argument, so that's why it's
currently been studied. But I would argue if you've ever ridden in a Toyota Prius or
you've talked to Toyota Prius drivers as I have, a lot of them say that it's actually affected
their behavior. So I can give you one anecdote personally from my sister. She has a
Toyota Prius. She was actually competing, which I'll get into a second with her spouse.
So by presenting the kind of instantaneous mileage information, she was able to have a
little competition with her spouse in the car and it was something that led her to drive
more performant as a result. So we come back to that towards the end of my talk if you'd
like. It is a good question.
So in this case, it allows you to compare your instantaneous mileage along with the last
30 minutes, right, to ground how well that mileage is, as well as to the historic average.
So in this case it's 51.4 miles to the gallon. So in this way this design space allows you to
analyze these existing systems as I mentioned but it also serves as a design framework to
think about how we can improve these systems. So here you can imagine comparing
yourself to others so just kind of the example that I gave you. So the car can actually
assist with comparing your driving performance with other people's driving performance
in the car to give you an idea if you are a performant driver or not or it could compare
you with somebody in the cloud, another way to be comparing to a goal. So this could be
tailored to your kind of driving behavior. The car could tailor that, or you could set
yourself. So those are just two examples about how you might improve this interface
through the design space. So that was a quick run through of the eco-feedback design
space and sort of how it allows us to think critically about designing these interfaces and I
am going to interweave it throughout the rest of my talk.
So now I want to turn towards a concrete system called UbiGreen. And UbiGreen is
focused on looking at different kinds of transit patterns, so commuting patterns. And
increasing awareness of people's transit habits and attempting to motivate green transit
behaviors. So there are really three questions here. What transit behaviors should be
sense? How can we sense these transit behaviors, and how should this data be fed back
to the users? So those are the three kinds of questions that I am going to go through this
part relatively quickly, but in terms of what to sense we are primarily interested in
commuting behaviors. So here we are looking at things like biking, walking and taking
the bus, riding the train and driving. And driving, it wasn't just driving alone it was also
driving with other people, which is the green activity. Driving alone is something that we
wanted to dissuade. So how can we sense these transit activities? Well at the time I was
actually working at Intel and they were working on this mobile sensing platform. Some
of you might actually be familiar with it. Basically it is a multi-mobile sensing board and
you wear it along with the belt and it sends data, transmits confidence intervals to a
computational unit. In this case a phone. So it is basically running. It has its own
computational unit here, as well as all of these sensors and it's sending, doing some
computations and sending those estimates through Bluetooth to a computational unit and
at the time we knew that, or we figured that this external sensor board would actually be
migrating down into commodity phones themselves. So it was something that Scott and I
looked at it and in graduate school and was taking the same kind of thing which required
external sensors and just running it on the iPhone with the accelerometer within the
iPhone. So there are clearly sort of advancements that have existed here since we started
working on this system.
So that's how we got walking and bicycling from this mobile sensing platform.
Remember we are also interested in taking the train, carpool, bus and driving alone and
for this we used GS seven-based cell tower algorithms to sort of infer when people were
in vehicles. And we also had a little bit of self report which would ask you if you were
with another person since we couldn't automatically derive that information. So how
should we visualize this data in an eco-feedback display? So this is the display itself and
it’s unlike a normal iPhone application, you know where you pull it out and you click on
an icon and it launches. This is always running and it's always running in the
background. So there's basically this pre-attentive awareness that this mobile phone
display is always available. And as you might take, pull the phone out of a phone call or
send a text message you are going to see this background change. And it is broken down
to you in a few ways. So the first thing is it shows what your current activity is, your
currently inferred activity. Then there is this animated evolving image that changes
depending on what your green actions are like. And then finally there is this value icon
bar that emphasizes secondary values that are associated with that transit activity. So
here we are emphasizing sort of saving money and exercise, so two benefits of walking.
So let's take a look at an animation of this over the course of a week. It starts on Sunday
the tree is relatively bare, and as people engage in green transit, this mobile phone adapts
and changes the display. The flowers sort of levels up into this flower mode and then
eventually the flowers bloom and…yep?
>>: Are you sure you're not draining the battery?
>> Jon Froehlich: That is a good question. We didn't actually look at that in this
particular case.
>>: Were you able to actually sense 24-7?
>> Jon Froehlich: It was about 11 hours of the day. So the phone would actually tell
them, the phone was smart enough to tell people about charging behaviors to remind
them to charge in the evening. As well as if they were leaving without wearing the
device, as you might imagine might happen from time to time, the phone would remind
them to put it on if they were, if it was disconnected. So those are good questions. So
then it would stay on this display until Sunday in which the display would reset and all of
our participants would get very disappointed.
In addition to the treelike display we also had a polar bear display and here it's basically
similar metaphors, just in this Arctic ecosystem where the ice flow get longer depending
on the green activities that people are conducting. So now I want to bring back this
design space and just sort of tell you and explain how it was useful in designing the
system and thinking about and evaluating the system. So data representation, we clearly
took a much more artistic approach here. But you can imagine a system that is like a
time series graph, right? Or something that's a bar graph, something that is much more
quantitative. But we took a much more artistic approach, that of visual complexity was
relatively simple, but it's not something that you could immediately understand. You
basically either needed to be taught it or learn it as you use it. The primary encoding was
graphical; in fact we didn't use any text. The measurement unit was the activity itself and
the view was categorical. A second part that we really emphasized was how is the
information accessed? And this was through this background display and here I will just
simply say that the update frequency was in real-time, which is critically important for
these kinds of displays and it was co-located with the activity that you were engaged in.
So you are getting information on the activity that you are currently engaged in.
For the last part that I wanted to touch on that I think is one of the most interesting and
almost deserves an entire talk on its own is motivational strategies. So here there are
things like, that come from behavioral psychology and persuasive technology but things
like just writing down a commitment to something basically means that you are much
more likely to engage in that kind of activity, so let me give you another kind of example
about descriptive norms. So I have been traveling a lot and descriptive norms are
basically the idea that you know seven out of ten people do this. So you're in a
circumstance in which are unaware what the normal action is and you see information
and that can be very convincing for activity. So there is a famous study about this with
hotel towel hangers. You probably have seen these in your hotels. So they basically try
to advise you about hanging up your account to save water and to save laundry detergent
so they don't actually wash the towel itself. So Cialdini and Goldstein did a famous study
where they used this standard environmental message on these little towel hangers and
they used a descriptive norm message, so here it says almost 75% of guests who were
asked to participate in our new resource savings program do so. So which do you think
resulted in more towel hanging in these rooms, the standard message or the descriptive
norm message? Right. So there was a 26% increase here and the critical part is it is just
a change in language. It's just a change in the way the information is presented to people
and it can have profound effects.
So obviously I believe there is really a role for technology to do the same kind of thing to
encode those kind of behavioral economics or behavioral psychology findings into the
displays themselves. In this case we used a lot of things from game design. So we used
things like rewards, a sense of narrative, a high degree of evocativeness and we also used
levels so people got a sense of progress throughout the week as they were engaged in
green transit.
So to study this we did a qualitative study over three weeks with 13 participants in
Pittsburgh and in Seattle and basically we were interested in just investigating the sensing
system, looking at the visual display, how did people react to it, how did people react to
the sense of actually being tracked, if you will, by the system? And then evaluate the
potential for it to influence behavior. So this is what it looked like as a participant. You
would have a mobile phone and you would have the mobile sensing platform and they
would be communicating constantly.
So this is a research website that we had that only the researchers can access. This was
not for the participants but this gives you a sense of what the back grounds of the mobile
phones looked like. This was on a Monday early in the week. And this is on a Saturday.
So people were able to get at different levels throughout the week, and some people even
got to the final level. So here we have flowers and fruit and the northern lights. So in all
we collected 8 million sensor events. We are basically logging everything on these
phones. In a thousand travel events, 72% of them were green. I am going to focus on the
quality of the results since it was a qualitative study.
In terms of accessing the information, people really like the fact that it was omnipresent.
It was very easy to get access to this information in terms of the data representation; they
liked this kind of artistic abstract representation. They liked how the stories or these
narratives were used. In fact a number of participants responded about wanting new
stories every week, so being able to download new stories or being able to share new
stories so that their phone would change stories from week to week. But the biggest
thing that came up was the need for more quantitative data. So as it turns out by creating
all these kinds of beautiful abstractions, it actually reduced the amount of information
that people could derive. So they wanted the ability to compare their performance from
week to week. And it was really hard to do that without a graph of some sort or without
quantitative metrics.
Another part was motivational strategy, so people had this sense of anticipation with
these stories. How would that narrative unfold over time and this was a sort of, a
gendered response here. A number of participants mentioned having negative feedback,
right? So the polar bear may be dying or the tree dying throughout the week. Most of
our male participants brought this up. This would still need to be sort of studied to see
how effective it would be. And I wanted to see the final stage I could get to. So it was
interesting was that even though we designed this as a real-life game, we never used
game like language or nomenclature when we were talking to our participants. This is
important because the game sort of emerged from the participants themselves. So they
saw it as a game and as the result of that there were some critical issues. Like, I don't like
getting incentives for getting points artificially by taking unnecessary trips. So you could
game the system simply by walking a lot, not necessarily using walking to replace
another kind of transit, which is what we were trying to support. Another part that came
is if I didn't get a leaf or a flower after, I felt like I was getting cheated out of my points.
So the inference system wasn't perfect and sometimes it would sort of infer the wrong
behavior and then people felt cheated. So these are two interesting findings I think that
have implications for the rest of my work and work like this. So that was UbiGreen. It
was the first system to semi automatically combine sensing and feedback on a mobile
phone for transit behaviors and had a number of findings that were then folded back into
this eco-feedback design space and used to build my own system which I am going to
talk about next.
So this is a bit of a transition now. I want you to go from thinking about transport or
transportation to home resource consumption. So in here our primary focus was inputs
and actionability. If you think about those mobile home displays another missing thing
was how actionable is the information? Maybe a more actionable representation of
information would actually be telling you a recommendation about a different kind of
travel mode, like a bus, that would be more efficient. So it didn't make any
recommendations. So I didn't have real actionable information in the mobile home
display. So how can we do something that is more actionable?
So what are the most water consuming activities in the home? I just thought I would
open that up since we are kind of transitioning here. Let's see if we can come up with the
top three.
>>: Toilet.
>> Jon Froehlich: What was it?
>>: Toilet.
>> Jon Froehlich: Toilet.
>>: Shower.
>> Jon Froehlich: Shower.
>>: Laundry.
>>: Dishwasher.
>>: Lawn.
>> Jon Froehlich: Let's stay indoors. But lawn is about 40%. So irrigation outdoors is
about 40%, 50% in Seattle. Probably because we really want to save up all of our water
until the summer and we go outdoors and just use it. So I think we had dishwasher,
laundry and shower or toilet. So we did pretty well. Dishwasher is down here. But I
think it's interesting, you know. I think it's difficult for audiences to respond to this
question despite the fact that I imagine all of us showered today. You know we all have
been relying on sort of modern infrastructure our whole lives and it's difficult for us to
answer this question, and I think part of that is there's just no good way to get information
about it. So I like to use this analogy. Let's imagine that we are going grocery shopping
but there are no price labels on anything, right? So we're just going grocery shopping, we
are filling up our shopping cart. We are estimating cost of the produce and the meat that
we have and then when we get through the checkout, we don't actually get a nice
itemized receipt from Safeway, like you ordinarily do, but instead you have to wait one
month, right? And you get this kind of receipt, which says look total food units 1527,
total price $642. That would fundamentally change the kind of transactions that occur in
stores, yet this is the same level of data that we basically get about resource consumption
in the home, right?
So what if you could get the same level of feedback in the home and that is really the
vision that we pointed to for this work that I am going to be presenting next. So I have
looked at electricity, gas and water but the primary focus of my dissertation is on water
so that is what I'll be talking about. So the vision is let's give people itemized feedback
about where water consumption is going in the home so this would be like in the kitchen,
this is in the bath. And it's not just down to the fixture level-- this is Scott's home by the
way. Scott’s beautiful new home. So it's not just at the fixture level, but really we want
it to be all the way down, to be able to discriminate between hot and cold because there is
actually an interesting relationship between energy and water consumption. You use
about 15% of your energy in your home just heating water. So we want to give people an
itemized breakdown of hot and cold if we can.
Unlike a traditional water meter which is like a turbine-based meter when you actually
have to cut into the pipes to sense or monitor how much water is been consumed, we
wanted to do something that is much more noninvasive. And remember traditional water
meter gives you one number and we want to disaggregate that number down to the fixture
or valve level. So we came up with a way of doing this through single point sensing just
using pressure. So you screw on the sensor to any kind of access point. Here you have
an existing water spigot, a 3/4 inch spigot. And then it identifies water usage activity
down to the individual fixture and it provides estimates of flow. So to sort of understand
how this works, I thought I would give you a short plumbing primer. This is probably the
second cheesiest slide of my deck so enjoy it. So most of us get public utility water, I
would imagine. How many people are on public utility water? Versus how many people
are on well water? Nobody. Interesting, well water, if you are on well water, usually
don't have idea because you are not metered. The only way you know how much you are
consuming is by how often you have to, you know, get rid of the stuff in your septic tank.
Anyway we get water from the public utility which pressurizes it. Pressure is important
because we have a pressure-based sensing approach so this pressure regulator which is
usually mandated by cities or states stabilizes the incoming pressure and protects your
pipes from high-level or supply level spikes, pressure spikes. This is kind of a canonical
plumbing layout, and the key here is that it's a closed pressure system, and by that I mean
when you open up a valve, you get instant access to water. You don't have to wait for the
water to propagate through your plumbing system and that's good because it's a good
conduit for our signal, and the second thing is that the hot water heater actually bridges
between the hot water line and the cold water line, so they are actually under the same
closed pressure system. Again that is good because we only want to use one sensor. So
let's give you an example. Here we are going to put HydroSense on this hose spigot and
we're going to flush the toilet and when you open up that valve or you flush the toilet, it
creates this kind of disruption in what was in equilibrium in this closed pressure system
and it presents this pressure wave. Now let's open up the kitchen sink, cold open valve,
so you can actually see, visually that these two fixtures create two different kinds of
pressure waves and now let's open the kitchen sink hot, and the signal actually travels
through the hot water heater so the high-frequency part is more dampened. So we can
actually use the fact that it is going through the hot water heater to discriminate between
hot water usage and cold water usage.
And I actually lied. I simplified things a little bit. It actually turns out when you open up
a valve the signal goes everywhere. And that is what enables a single sensing approach.
So we can actually… Yep?
>>: Can you distinguish between two things running parallel?
>> Jon Froehlich: So that is a, Rod’s question is about what happens if, you know,
someone flushes the toilet and someone else uses the kitchen sink or someone is
showering and then flushes the toilet, a compound event? I will get into that in a second.
So that is a good question. So because the signal goes everywhere we have actually
evaluated hydro sensitive in a variety of different access points so here is one on the host
spigot, the hot water heater, and an inside, these are 3/8 inch connections. This is below
a bathroom sink your kitchen sink. And these are great for apartments versus here for
houses. So let's take a look at the signal in more detail. There is this sort of dampened
sinal sort of wave form part both for the open as well as for the closed. And then the
third part of the signal that is of interest here is this delta, this pressure delta and that
basically allows us to get an estimate of flow, so flow rate. So greater delta, more flow
rate. I think I have a quick movie, so this is going to play at I think at 2X and then it's
going to quickly go to 8X. It's a very fast movie of me using water in someone's home,
but hopefully you can see the pressure signals here, as I'm activating and deactivating
fixtures. Real-time. So the important point here is just seeing the variety of signals.
This, by the way, is just an overview of what is showing down here, the variety of signals
that exist in someone's home.
>>: [inaudible] bathroom versus [inaudible].
>> Jon Froehlich: You mean the location?
>>: [inaudible] specific you know bathroom sink and say kitchen sink?
>> Jon Froehlich: Yeah. There are three levels that I will kind of get into so you can
imagine a system with the highest fidelity would be telling you how much water you are
using at the individual valve. So that would be valve level detail. Then you could go up
one level from then say, okay, I don't care about temperature; I just want to know which
fixture is been used. And then, so that would be upstairs bathroom sink versus
downstairs bathroom sink. So that is fixture level. Then you can imagine going to
fixture category level, which is just sink, a sink is been activated. And for each of those
levels there are different kinds of policy, I mean different kinds of stakeholders, people
who have different interests at those different levels. And I will get into this in my future
work as well.
>>: Are you able to distinguish the toilet upstairs as versus the one downstairs?
>> Jon Froehlich: That's right. So that would be fixture level; that is sort of the medium
grade. One level above that would be valve level, which would be saying, in the toilet
case, it only uses cold water. So this is an easy example. But it would be like bathroom
sink hot, versus bathroom sink cold. Yep?
>>: Can you distinguish between the two different ways that the valves are operated?
Like if the cold is fast or slow?
>> Jon Froehlich: So that's a question which I will get into in the second study that we
have, and if I don't answer it you can bring it up again. Yep.
>>: I am trying to remember [inaudible] anyway what they have done was we were
looking at energy and [inaudible] and the whole idea was at the end of the day [inaudible]
would provide some analysis to the user saying this is where the energy is been used and
for example you're using so much for washing and whatever. [inaudible] modeling based
on the similar [inaudible] but it was very rock summit area didn't actually capture
[inaudible] similar but actually [inaudible] electric signature. And determine sort of have
the user say yeah I use the [inaudible] and then [inaudible].
>> Jon Froehlich: So it's in general, a similar approach. We have calibrated model right
now and I talked about it a little bit in future work…
>>: What I'm saying is…
>> Jon Froehlich: My adivsor Schlage is really known well for doing electricity
desegregation over sort of voltage noise that is kind of modulated on to the power line.
That might be similar. I mean George Hart had the original work that I am familiar with,
electricity desegregation work.
>>: [inaudible] 1980s.
>> Jon Froehlich: Okay.
[multiple speakers] [inaudible]
>> Jon Froehlich: If they did, they were only looking at load profiles.
>>: They actually [inaudible] but they actually [inaudible] new model and they
eventually it was [inaudible] back model [inaudible].
>> Jon Froehlich: The hard thing about that is you have so many things in your home,
like all lights. So if they are working with the data that I think they are working with,
which I think they are just looking at load profiles, all lights basically look the same. So
it's hard for them, and there are a lot of different kinds of, like a laptop charger might
look similar to other kinds of chargers in your house. So it kind of depends on the model
that they are using. But in general, the approach is right, and so Victor was talking about
using the train model which is what we do, which is the point of this slide in fact. So I
have demonstrated that these different fixtures have different signals, and that allows us
to identify where water is occurring in the home and that's because the signatures
dependent on the fixture type, the valve type but also the propagation pathway. So the
plumbing system itself basically is a transfer function on the signal. That's good because
it allows us, it's highly discriminable, but it sort of bad if you think about cross
generalizable models across homes. So right now we are working with per train or per
home train calibration data.
Let me just give you a better example of kind of how the algorithm performs, works.
There are four parts here. Detect that water event has occurred, because for the most part
it's flat. It's just a flat pressure line. So we wanted to detect that a water event has
occurred. We want to classify the event as either open or close and then determine the
source of that event, toilet, shower, and then provide estimates of flow. So let's take a
look at this as sort of the pressure is coming in, we use this raw pressure signal. We
actually do some, a low pass filter and then we use a derivative filter and this derivative
signal; we are basically looking for these changes. We want to look at changes in the
signal so we can segment the event so the signals coming in here, we are looking for
these drastic sort of critical changes in pressure. So this gives us an idea that maybe there
is something happening. Then we look for a stabilization point. And then we can
segment again. So now we know that there is something that is happening. So we are
automatically detecting an event but we don't know if it's an open or close, sort of get that
categorical data, we actually look at two things. A pressure decrease and then a negative
initial derivative, and if that's the case, like it is here, then we know that it is an open
event, and we basically do the same thing for a close.
So now though we are going to get this pressure increase and then pause an initial
[inaudible], so now we have this close. Now we have open and close but we don't know
which fixtures are responsible for that water usage so let's take a quick look at that. So
we have this unclassified open event and as Victor alluded to we use a template matching
approach. So we have to have an existing library of templates in which to do our
comparison, which means we have to do calibration, which is kind of a challenge. And I
will get into that in future work. So does this work? So the first study was basically
looking at the feasibility of using pressure to do desegregation and our approach here was
controlled experimental trials in ten homes. We have two people go into homes and just
basically open up and close valves. And then we would mark that ground truth data on
these laptops. We do that manual annotation and we also collected flow data. In four of
the ten homes we also did flow data calibration studies using a calibrated bucket which is
basically the plumbing standard, if you will. And then with the ten test sites we had 706
trials, 155 flow trials and 84 total fixtures were tested. So for our classification
experiments we did 10 fold cross validation. Here the X axis is the homes and the Y axis
is classification accuracy so higher is better. Blue is open events and green is close
events. So here, yeah, we did pretty well. In some houses we have 100%. If you look at
this in aggregate, 99% for opens and 97% with closes. So we were pretty satisfied with
this. You can look at the data in another way out going to change the X axis. It is the
same data but by looking at it in terms of fixture category, and here we did pretty well
again. Shower closes were the least accurate. And that is because there are different
ways of closing your shower actually with the diverter valve that you have, depending on
the shower, so that one is actually a more complicated example. So in terms of the flow
inference we were basically looking for 10% error margins because we found empirical
studies of traditional meters, that's basically what they perform at. So anything about
90% is usually pretty good. And of course this is just basically a feasibility study so we
only did four homes, but you get above 90% accuracy there as well. Yep?
>>: What about the long [inaudible] signature. I would think a shower would be one of
the ones that you would look at [inaudible]
>> Jon Froehlich: Yeah, that is definitely true.
>>: [inaudible] your accuracy quite a bit more.
>> Jon Froehlich: That is very true. In fact we did not capture that at all because we
basically did the same exact amount of time per trial for each fixture here so we didn't…
>>: [inaudible]
>> Jon Froehlich: Since these were controlled trials, we would just go in and use your
shower and we would keep the valve open for 5 seconds. We didn't sort of
naturalistically use your shower.
>>: [inaudible] the signature. Would it be at the offset the leading edge, the trailing
edge, or both?
>>: When you first turn the valve on…
>> Jon Froehlich: So basically how long that high-frequency wave form…
>>: [inaudible] template matching, [inaudible] template matching on the leading edge,
on the trailing edge, or both?
>> Jon Froehlich: We were basically doing match filtering.
>>: On the onset of study, when you first turn the valve on?
>> Jon Froehlich: Yes.
>>: What about when you turn it off, also?
>> Jon Froehlich: I don't think we would.
>>: I have a comment, if we can get back to the slide before this one. So each of these
cases for example if you take a dishwasher or a [inaudible] or whatever. Not a bathtub or
shower but definitely toilet, there is a capacity, a volume of water that is used every time.
And these vendors could potentially [inaudible] have the capacity [inaudible] so then you
wouldn't necessarily [inaudible] you could… [inaudible]
>> Jon Froehlich: Right, and that is something I will get into as well. So I think there
are a number of things that we are not capturing. So that is what I thought, I think it was
Mike, was alluding to temporal aspects of, you know, a shower is going to be 10 to 12
minutes, versus a toilet is going to be 45 to 55 seconds. So there is a lot of information
that we are not capturing by doing this really simplistic template matching approach.
And I will get into that again later in the talk.
>>: You are very controlled on your sinks. You open them all the way?
>> Jon Froehlich: So that's kind of towards, sorry I don't know your name, but that gets
towards his question which was-- I mean we didn't actually control for it much. I had
undergrads and myself, I was involved. I mean we would just open up the valve and
close it. We weren't really thinking too much about how fast we were opening it or how
close. But we were still wondering about that question, which I'm going to get into in
just a second. So questions? Good. So, you know, the results of these trials
demonstrated to us that this is fairly feasible technique and more experimentation is
necessary to kind of get at the other questions that we've had about compound events, so
more than one thing happening at a time or how about changes in the way people use
different kinds of fixtures. So that is kind of what we are interested in. So kind of like
brushing your teeth, or shaving or bathing or even paw washing, the cheesiest slide in my
deck here. [laughter] Interestingly, when I actually show this to people after my talk, the
people are like, I'm so glad you showed that because I have a cat and they use water in
the home. And I just don't know what to do about it. And I go, like that's interesting.
[laughter]
Or the worst case, what happens if all of these things are happening at the same time,
right, your cat decides to use the water while you are brushing your teeth and bathing?
So those are the kinds of things that we are interested in studying in the second study here
where we wanted to see how well HydroSense can classify real world water usage.
That's what we care about, right? So we get a five-week deployment in five homes to
study this and so originally we were doing this manual annotation of all the water activity
in people's homes but we can't do that right. You don't want me following you around
like Desny and his wife and his kids every time they are using water in the home. So we
are not going to have like, an undergrad we could probably pay to do it but it just
wouldn't work out very well. So this isn't going to work so this whole study became
about how do we get ground truth labels on the water usage activities in the home? And I
am not going to have a chance to go through all of these different ways that we tried, but
if you get me on a one on one, we can talk about. So we just thought we will use these X
ten buttons. We will have people tell us whenever they are using water.
And it turns out we have a whole lot of time lapse videos showing how poorly this
performs, but I will just tell you that people are not good about providing ground truth
labels on their water usage. [laughter] it's a good movie. So in other words we had to do
an automated method. We wanted something where people weren't even in the loop
basically. So here the most important part in terms of hardware capability is something
that is water resistant, because we're dealing with water. And in terms of sensing
capabilities we wanted it to work across fixtures and appliances, detect open and closes
and here discriminate hot, cold and mixed. And these are discrete states. So hot, cold
and mixed. If you think about it we wanted-- this is actually a hard problem, right?
There are a lot of different kinds of fixtures, manually operated, but also these
electromechanical devices, your appliances. They use water in different ways. They
require different ways of getting ground truth data on that water usage. Let's say we were
focused just on the manually operated valves. Here the single-handed faucet is much
different than the dual handle faucet. So there are a lot of different fixture designs which
require, have different kinds of specifications for how water is been measured at each of
these fixtures. So after many failed attempts we actually ended up doing our own custom
ground truth sensor board. In fact gave Cone, who is entered here did this PCB for us.
We used xbee and here we using a distributed sensor network. We are going to basically
affix a sensor to every single valve that you have in your house and it is going to provide
ground truth labels on our pressure stream. These three are the value oriented sensors,
the hall effect sensor, reed switch, three axis accelerometer and then these to our
vibration sensors. They wake up the sensor board and send us the data. So we had a low
amount of transmissions usually in the home because there usually wasn't all that much
data and we also could keep our power levels at a minimum. In addition we also had the
kilowatt, modified kilowatt to analyze when power was been drawn from these
appliances which we would then correlate with water usage as well.
So those are the different ways that we got the ground truth data. This is my bathroom
sink. That is a reed switch; that is a magnet. So this is a binary, you know, binary data.
This is my advisor's kitchen. So this is an accelerometer. That would tell us the position
of a single handle faucet. This is a bath. This is a toilet; this is my toilet. And this is my
advisor's shower, and I think, this is important. It took a team of 3 to 4 people 2 or 3 days
to do each one of these houses and it is important when the spouse maybe comes home
and sees this that, you know, you have a joke ready. So the one I told Julie in this case
was, you know when you get in the shower, just be sure you don't touch the wires,
because you're going to get a little bit of a shock. So no, actually they were very lowvoltage. But anyway-- so I don't think Julie showered for about two weeks. [laughter]
So and this is what it looked like for the appliances. Notice that we used a thermistor as
well, so that we can actually look at what the output of the washing machine was, so we
could discriminate if it was a hot, cold cycle or just a cold, cold cycle and so on. So we
had background truth data as well. Remember all of this, sometimes people get confused,
all of this was done just so we can get ground truth labels on the pressure stream. It is not
like HydroSense requires this distributed sensor network. It is just to get those ground
truth labels and because we went to all this work we wanted to do to sensors per home so
we could scientifically basically validate well, how much of a jump are we going to get if
we have a second sensor, so we put one on a cold point, one on a hot point, and that's
what it looks like in an apartment. And so I just want to play this quick video. We
always had a day logger, you know, deployed to communicate with the sensor network as
well as the pressure stream so here the blue is the cold line and the red is the hot line and
then down here is the ground truth data. So I will just play a quick movie, [video begins]
so I am going to flush, the reed switch goes up here. And I am about to use the cold
water. [laughter] All right, brushing the teeth. [video ends] So for every house we
deployed this sensor network, Yep?
>>: Did you consider [inaudible]. It seems like it would be much less invasive.
>> Jon Froehlich: Yes, we thought about that; we thought about a whole bunch of ideas.
So in terms of using audio signals, is that what…
>>: Or you could strap it on to the pipes there as there are available.
>> Jon Froehlich: Underneath, right. So we actually did the deployments in an
apartment so that would make that kind of hard. But we started out using microphones
just to do the desegregation actually. So we had all of that code written; we could've
done it. What we found was people tend not to, like to have live mics in their home,
especially around the bathrooms.
>>: But you could just drop it on the pipes.
>> Jon Froehlich: Right. But we still want to have, make sure that we have valve level
information, right? So we want to know, so we would have to deploy that fairly
strategically. Here we could just affix it to the fixture itself. I think maybe that approach
would work. It would take some time to think about how you would discriminate
between all the different signals, but it might work. Okay. In addition to the sensor
network, we had the data logger and then the two pressure sensors. We also had a hydra
server where all the data was been communicated to our Web server where it would also
notify us if it sensor went down, because if you have done deployments you know how
expensive that can be in terms of time. We didn't want to waste any data. And we also
built a whole bunch of analysis code through visualization and through Matlab, to deal
with all the incoming data. So in total two apartments, three houses, 103 water valves
were instrumented in 15,000 water events. And what is interesting here is that there isn't
that much data out there at this level about water usage in the home so it is interesting just
to look at it descriptively. So how many times water valve activations occur? So here
you're seeing that, average event count per day. And you see this kind of jump here in
home three and that is because they have for occupants. The rest of the houses have two
occupants. Okay, so it's about 35 to 46 events a day over all. But what is more
interesting is how much of that had a hot water component. Well actually a startling
amount we thought, 60% in fact 75% here in apartment one. Remember I mentioned that
kind of interesting nexus between energy and water consumption. In terms of where
water is going, this is by event frequency down to the fixture level. You will see it's
basically a power law distribution and 85%, are just the first four kinds of fixtures,
kitchen sink, master bathroom sink, master bathroom toilet, and secondary bathroom
sink. Now how about compound events? What is interesting here is that there was no
data in which we could have built a model on before we ran this study on about how
often compound events occur because nobody had done this kind of study before. So
what you guys think in terms of how often do compound events occur in the home?
I think the question is dependent upon two things. One is how many people are in the
home and how often are they in the home together? But let's just see. Is it 80%, is it
10%.
>>: So I guess in the morning there'd be more compound events.
>> Jon Froehlich: That's true. There's actually this temporal component, right? In our
data set it was about 22%, right, nearly a fourth, which we thought was actually a lot.
But can you think about why this might be? Why is, what is the most common
compound…
>>: Compound, both hot and cold being used at the same…
>> Jon Froehlich: That happens, that's true to read what is the most common fixture
compound event?
>>: [inaudible]
>> Jon Froehlich: Flushing and…
>>: Washing your hands.
>> Jon Froehlich: Hopefully washing your hands, which we have data on, for everybody,
including my advisors, so about 42% of all bathroom sinks were in compound because of
flushing toilets. So because they are so frequent I thought would go through this really
quickly. Yeah?
>>: So you didn't find that the shower consumes most of the water?
>> Jon Froehlich: So this was by event frequency. So if you will remember we've
basically discreetized our state space, yeah. I don't think I'm going to have, well I will
get through this and maybe skip another part of my work. Because I think this is
interesting. In terms of how the signals sort of look, again the Y axis is PSI. So this is a
bathroom sink in isolation. This is a bathroom sink event with the toilet open. So you
will notice there are a couple of things. One it is slightly distorted, right? It's dampened,
and that is because there is water filling in the pipes. But what is interesting is the sort of
frequency component is actually relatively stable. And the other thing that is interesting
is that if you just look at temporally, which is what I thought you were getting at before,
this is about 2 1/2 seconds of use. And this example is drawn from our actual drawn truth
data set.
And actually if you see something that is a couple of seconds long, it is much more likely
to be a sink activity. And we weren't actually using that kind of data. So here is another
signal. And so we kind of thought how can we move beyond just template matching?
How can we capture a lot of these other signals, so duration of water activity, time of day,
which is kind of what came out there. Recency of use, number of uses, the overall
pressure drop and really importantly, the relationship between valve events. Right, we
know toilets and bathroom sink water activity is related in some way so how can we
capture that? And we brainstormed and we finally arrived at a more bayesian approach,
and here we deal with the same pressure signature library but now we have a sequence of
unknown pressure transients. Not just one. So we have a sequence, we have this most
likely valve sequence that we are basically trying to solve for. We have the conditional
probability term. And this is the match filtering and also uses some simple signal
features like stabilize pressure drop, but for our prior probability terms we have things
like bygrams, right? So what is the relationship between fixture activities? So the toilet
flushing and the water in the bathroom sink. And then we also had a grammar, so if you
see an open, you are much more likely to see a close, and this also gives us the ability to
match things. So we want to be able to match things so that we can actually get things
like water usage, duration and volume which are also critically important, right?
So in terms of classification over all at the valve level was 75.5%. So this is the same
kind of classification that we did before when I was showing you nearly 100%
classification accuracy. So as it turns out real-world water data is much harder to
classify. So we get this big drop here, but as I always mentioned we can go up one level
from that to fixture level. So this is upstairs bathroom sink, downstairs bathroom sink,
kitchen sink, at the fixture level. And we get a jump year to 89.5%. We can also go to
fixture category level, which is just saying sink. And then we get another jump to 96%.
So it kind of depends on what applications you want to build around this sensor. And
then in terms of two pressure sensors, we see a marginal but significant increase. So it
just sort of depends on what your investment is in trying to get access to this data in terms
of how many sensors you might deploy.
So that was for our second study which I think demonstrated that HydroSense is capable
of basically discriminating water events, real-world water events, especially at the fixture
category level. And as we went through and evaluated this we collected one of the most
comprehensive data sets of water usage in the world.
So I want to quickly go over this next step which is what should we do with this water.
And that is actually what I have been working on the last five months, which is this
display called Reflect and here there are a number of different areas I pulled on from this
eco-feedback design space. I am not going to have time to talk about this in great detail
but the key, going from that mobile phone display into something that we are placing into
people's homes; there are a number of differences. One is that it is a shared display now.
It is not just a personal display. And there are also a number of different kinds of
occupants in the home.
So there are kids and there are different family members and they all have different ideas
about what the data should look like potentially, and how that should kind of promote
different kinds of activities, so one of the things that we concentrated on is the aesthetics
of the device. We want people to bring this into their home and this is kind of pulling
upon Beth Minet’s work, and putting a frame around the display. So it is not just this
piece of computational technology, that it actually looks like a piece that you would put
into your kitchen, okay. And this is one of the interfaces. The interface overall looks like
this is broken down into two things. So that on the top you have the status bar and at the
bottom you have this rotating ambient display of different kinds of water depictions,
water visualizations. At this top part it has this news feed, weather, aggregate water
usage and the date and time. And I basically like to use this kind of analogy or story
about when you walk into your kitchen, why do you look at your microwave? It is just
an appliance, because it has a clock. So that is the same kind of attention I am hoping to
pull in here with this, and that is why I have a number of other things at the top of this
that is not just related to consumption. So when I am also rotating every 3 to 5 minutes
different depictions of the data. So this is a much more pragmatic representation of water
usage. So let's look at one of these called rain flow. Basically these flows, coming out of
the different fixture categories is dependent upon the recency of use. And then I also
show people their daily average and their goal in these basically what are bar graphs, but
they are cylinders which is kind of capturing the flow as it is coming out. So I am going
to load up a video, if it actually loads, to give you an idea of what this looks like. [video
begins]So this is what it looks like. So you will notice this is going to get slimmer right
now. And that is dependent on the recency of use and then I am going to control for the
amount of cylinders. So all of these things would change depending on the dependency
and then it's a highly interactive.
So this is a touch screen. Some people can, you know, run their fingers along the display
and all of the displays are sort of interactive in this way, all of the displays that I built.
So, you know, to contrast that one which is a bit more abstract, we had this time series
view where the Y axis is flow rate and the X axis is time. Again this is much more of a
pragmatic representation. And then perhaps the most abstract or artistic is this aquatic
ecosystem which is similar to what we did for UbiGreen. And here we think that it might
appeal more to kids. And this is really about kind of water is a game in the home. So
you start with Frank the fish and then there are different kinds of water saving goals and
if you meet one of those goals different things will happen in a scene. So Frank might
meet a mate and they might actually have kids. And basically this ecosystem is evolving
as people are using water and meeting their different water saving targets. So this might
be what the end of a couple of weeks would look like.
So in terms of evaluation, we are going to do two parts here. We are currently doing this
which is an in lab study. We are actually bringing people in and looking at objective
measures about comprehensibility. So do people comprehend the displays and how
quickly does it take them to answer questions about what they are seeing in the displays.
But we are also very interested in doing a field deployment which would link
HydroSense along with these Reflect displays and then deploy these doing another kind
of qualitative deployment just to see what people feel like when they are seeing these
visualizations in their home.
So in summary there are really four big contributions here. The first is this eco-feedback
design space that allows designers and people who are interested in building these
systems think critically about the systems that they are building and the kind of trade-offs
that exist there. And I also built and evaluated two systems. So UbiGreen for personal
transit patterns and HydroSense which provides the highest granularity of any water
monitoring system that exists right now and then also Reflect which takes this data and
presents it to people in order to promote water efficient behaviors in the home.
So how much more time do I have? So we started a little late. Okay. So I just want to
briefly touch before I go on to future work, just that there are a number of other areas,
some of which I collaborated with people in this room on. A number of other projects I
worked on in grad school that didn't really make it part of my core dissertation, so I didn't
put in my job talk. I worked on like visualization systems for source code repositories for
software engineers, all the way to these mobile tools to support field studies. And this is
perhaps what I am most known for so I thought I would go into this just a little bit. My
experience allows people to do field studies that combine sensing or objective data, so
this might be physiological monitoring about heart rate or it might be activity monitoring
along with self-report and it does that through this sensing triggers and actions
framework. So it is very easy, lightweight way of linking all of these things on the
mobile phone and allowing people to do field studies that ties together physiological
metrics with self-report metrics. Let me give you a quick example. So here we just
noticed that you finish your morning walk. How is your breathing rate? Here we have
three sensors, a location sensor, a heart rate sensor, and a human activity sensor. So this
xml/scripting interface, the XML part is kind of a more, or allows you to detect this stuff
through a lightweight mechanism, but we also have a scripting interface which allows
you to do things more dynamically, so tying behaviors together. So this action now, we
want to just ask this one survey. We notice that you just finished your morning walk how
is your breathing rate? And we're going to tie this stuff together through a trigger. And
this is where the scripting mechanism comes in. So here you will notice that we get the
sensor, this human activity sensor, we see if it just exited the walking state and that they
were in the state for 15 min. And if they were then we just ask them the self-report
survey. And this is important because we can get quantitative sensor data about heart rate
but we can't get information about subjective feelings of exhaustion. We need to have a
person actually give us responses about that. Okay so since 2007 there have been 95,000
website visits, 3000 downloads, 124 citations and what I am most proud of is the dozens
of studies that this has enabled across the world and these studies have had a social sort
of component which is a big part of my work as well. So people have used it to do heart
health and stress studies or preterm infants; this was at Irvine. There is also a researcher
in Australia who is using it for smoking cessation. So that is really meaningful to me and
basically means that my time was well spent in trying to open source this.
So now in terms of future work, I am just going to go through three areas, eco-feedback,
urban informatics and new HydroSense applications. So in terms of eco-feedback there
are three things, longitudinal deployment of HydroSense and Reflect. So in graduate
school I have been unable to do a longitudinal deployment to really assess some of the
behavioral changes that people might have, so I would really be interested in doing that.
And a lot of times when I give this talk people are really interested in the ethics of
persuasion. So what does it mean if we slightly manipulate the data? It could even just
be shifting the Y axis a little bit on a graph would result in very different kinds of
behaviors. So as a designer what does it mean when we have that kind of influence and
power? So there are some interesting ethical implications. And then finally applications
of eco-feedback to health behaviors, so I can get more into this on my one on ones as
well, but that is basically where I started on UB fit project is looking at technology and
promoting healthier behaviors.
In terms of urban informatics sometimes people call it smart city research. I am really
keenly interested in the digitalization of infrastructure so the smart grid or traffic sensors
and all this information, in fact it looks like the parking garage at building 99 also has
some kind of sensing. I don't know if it works or not about full, no it doesn't work
[laughter] but it is those kinds of sensors that I am interested in. So I looked at this in a
number of ways and I don't really have too much time to get into it, but this was through
shared bicycling so as it turns out cities like Paris and Barcelona, even London now have
these shared bicycling systems, and these are third-generation systems. And by thirdgeneration I mean they are digitalized. So you actually have a smart like RFID card that
you slide and when you slide it, it creates is digital transaction that we can sniff. Right
this little digital footprint of where people are in the city and then it gives them access to
this, to these bikes. So it is that digital transaction that I am interested in because we can
scrape that and use it to do interesting things about where people are in the city. For
example, here we just looked at where the most active stations were. So this is a
lightweight way of getting a sense of the city in real-time or when the most active times
of the day are. So you can actually just see from shared bicycling that morning commute,
late Spanish lunch, so this is 2 PM, not 12 PM as you would expect it to be, and then the
evening commute. But what is interesting is what happens if we combine shared
bicycling with bus data, like that Orca card data or the subway, right. So we started
doing this as well, so we have the London, let's see the Oyster card I think it's called, the
London underground data that data set. And interestingly here we did that same time
series analysis. You only see two spike patterns. So this either reflects a difference in
the ways that people are appropriating, you know, shared bicycling versus the
underground, or it's a difference in culture in the cities, London versus Barcelona.
So there are a lot of interesting questions about this like how should this real-time
information be visualized, where should it be stored, how should it be accessed. You
know, can we use this to automatically determine anomalies in the city?
And then finally new HydroSense applications, so the first one would be leaks. Even in
our ground truth data set, we actually saw this a little bit. In particular we saw it through
toilets which account for 30% of residential leaks. So the leaky flapper valve that you are
supposed to replace every 4 to 5 years as I'm sure you do. So we were actually able to
see this in our ground truth system, so we can automatically detect that and I think that is
an interesting question. What about the kind of behavioral patterns that water activity
reflects? So just by looking at these water routines in three homes we can see when
people are going to work, when they are doing things in the kitchen, when they're going
further bedtime routine. But how predictable are their behaviors? And the predictability
of this stuff speaks directly to applications like aging in place. So imagine using
HydroSense just to give you an alert, it is 9 AM. Your grandmother hasn't actually used
the kitchen sink or hasn't used the bathroom as she usually does. Or a different kind of
system would be, you know, there is a link between dementia and water usage. So
people who are struggling with some cognitive impairments might forget to flush the
toilet, they might change debating practices and some of that can come out through just
simply looking at water practices. Yes?
>>: I am kind of surprised you said [inaudible] big-league yes, a big drip, drip, drip over
24/7 would be a big water consumer but I'm kind of surprised you said you could sense it
with the way that you did this.
>> Jon Froehlich: We can't sense anything that drips. Because is not going to create a
pressure sensor, pressure signal. A leaky flapper valve is going to though. We need
anything that is basically going to create a signature on the pressure line. So a leak,
something that is just leaking out of your tap is not going to create any sort of disruption
in the closed pressure system.
>>: But it is a leak.
>> Jon Froehlich: No it is a leak, but it is not something that we can sense automatically
with HydroSense. You're going to have to use some other technique. But that's a good
question. Privacy implications. So yes, water activity reflects, because it's a fundamental
ingredient of life, it reflects different kinds of activities in the home so there are privacy
implications here as well. So imagine that, you know, you come home and you see this
display. It is 6 PM and you know that you have a 12-year-old son and wait what is this?
The water activity around noon and 1 PM, right? They were supposed to be at school.
And it's only because of the way that we visualize the data, right? The fish display
wouldn't reveal this kind of thing. So as you think about this new sensing technology and
feedback technology it can reveal different kinds of patterns about our lives that we may
not want to be revealed.
And then finally and we touched on this a little bit in terms of how can we train up
HydroSense quickly? So how can, what kind of training set do we need? How quickly
can we train it up? Can we do an unsupervised or a semi-supervised learning approach?
And part of this will actually be this generalizable model that will have some heuristic
information about oh well, we know that showers are 10 to 12 minutes. We know that
bathroom toilets are 45 to 70 seconds. All of that information we haven't really captured
yet and tested to see if it's generalizable.
In closing I would just want to present a vision which is we have kind of evolved over
time to have different senses of our self. And we can even get objective senses of our
self from looking at digital photographs of us or through mirrors. We can get a different
sense of our self, but technology presents this unique opportunity to collect all sorts of
information on our eating habits, our fitness levels or even environmental activities and
then we can use technology to feed this information back to us to make us better people
for to make the environment better or to lead healthier lifestyles. So that's my talk. I
want to point out that James Landay and Shwetak Patel are my two advisors and I have
worked with a number of collaborators, including some people in this room. And thank
you.
[applause]
>> Jon Froehlich: Yes?
>>: I am wondering back to the Prius displays, why haven't those taken off beyond the
Prius?
>> Jon Froehlich: I think that's a good question. So Ford has an eco-display now. You
know one study that I am really interested in possibly doing, even just talking to the
designers so how did they come about, how did they think about these displays? The
Toyota Prius display really hasn't changed so I was just riding in the first Toyota Prius
model. I took pictures of it. It's largely the exact same visualization. So that's kind of an
interesting thing. Why haven't they adapted over time?
>>: What I meant was why aren't like a lot of cars that Toyota makes coming with that
display if it's such a great…
>> Jon Froehlich: Twenty years ago to track your gas mileage you had to look at your
speedometer, you go to the gas station, you balance your checkbook, you try to figure out
what your gas mileage is. Now even if I buy a brand-new car, I am not talking about
hybrid, a regular car, what does it tell you? It tells you what your mileage is, and that
allows you, it is actually very empowering information. The EPA sticker label on this car
said it was 24; I am only getting 18. So either EPA is inaccurate or I am not driving very
well. So I would offer that you are getting a lot more information if you think about it
then you were 20 years ago. There are a lot of changes in the car, and the other part is we
are keenly aware of the price of gas. Most people right now know that it is over four
dollars. It is blasted at you everywhere. You have a lot of different information sources
that you are drawing from for gas. I mean why is it that we just sort of inherently when
we just discuss MPG you guys, you know, we know what it is, mileage. It's because we
have it in normal conversation. We don't usually just talk about our kilowatts per hour.
Say how performant is your house? Right? Yet we talk about that with our cars. So it is
just an interesting thing. And I think part of it is access to that kind of information;
which we will get, we are going towards that. Yes?
>>: [inaudible]
>> Jon Froehlich: I think there are a couple of components. We are all going to change
our behavior if you increase the price of something. So in Seattle we talked a little bit
about the price, or water consumption behaviors change in the summer. We get a 40%
increase in water consumption. Think about that peak load change in the summer in
Seattle and what that means for the infrastructure. So what do they do from a utility
standpoint? They jack up the price of water. We are in a tiered pricing, at least in the
city of Seattle. So your summer water if you go over certain amount, and that is
compared to what you usually use during the winter, so that is how they figure it out.
And then you go up into a larger price point. And that actually changes. So this started,
in fact our urban planning department gets access all of the data and they have shown that
it does change behavior, you know, a couple of percentage points depending on
demographics. So price fundamentally does change. For water it is an interesting one
because you can only change the price point so much. It is an ethical issue. Everyone
deserves access to water so the city of Albuquerque, the last five years, each year they
have increased the price of water by 8.5%. They can't continue that. They are out of
fresh water sources. They are looking for alternative ways of driving down demand.
Yep?
>>: The eco-feedback design space looks too complicated to me. There is too much to
consider their. [inaudible] Do you have a sense of what are the two biggest things I need
to do here [inaudible]
>> Jon Froehlich: We haven't actually deployed it out and worked with designers to see
how they will acquire and use this in their designs. So that is one thing that I am really
interested in doing. In terms of what I think is most important, part of that draws on
psychology literature. The temporal component, having quick access to that information,
which is what I think the mobile phone really affords is important. So when you are at a
decision point or when you, since the mobile phone is always with you, it offers you the
kind of thing. The other is spatial co-location, so giving it while you are engaged in the
activity. So there is this temporal connection and this spatial connection to the activity.
Those two things I think are most important. So if you walk into a restaurant and I
present you with the information as you are making a choice about the menu, it depends
on what that information might be. I think calories is actually a bad part because people
don't actually know what a calorie is, but that can change your course of what you choose
on the menu. And it does. There have been a lot of studies. Just by putting a box around
something on a menu, drives up consumption for that particular thing. So there are a lot
of subtle things that people do even when they just look at the menus and choices about
how to change your behavior, and we can take advantage of those same things on our
mobile devices or other feedback displays.
>>: You mentioned location. So where are you going to put your device?
>> Jon Froehlich: So if you remember that a kitchen sink I think was like 30% of all
activity. So the kitchen is just a very, very highly trafficked place. So I think is a great
place for ambient display. So it would be great if it was on the fridge or something but
we are putting it kind of on kitchen tables, just one, for cost mainly. Yep?
>>: So I was thinking about [inaudible] what we are learning here, right? And one of the
dimensions [inaudible] apologize was the part that I heard you didn't cover was so the
demographic you are going after. And so I was thinking about the United States versus
New York versus Asia, so I'm thinking the same sort of metrics you use to change
behaviors are not, doesn't seem to me that those may be appealing to somebody from a
different culture so demographic you're going after [inaudible]
>> Jon Froehlich: Absolutely. So I have a workshop paper actually on looking at that
with my advisors in China right now. So we are looking a lot at different kinds of
cultural practices around consumption. Japan is an interesting one because it has seen,
obviously it is a more social culture, but it has really seen, frugality is seen as something
that should be, you know, a push towards. You should be achieving frugality, whereas in
America we are more about materialism and consumption. So we want to demonstrate
our consumption by buying a Hummer, whereas in the Japanese culture it's not really
about that. It would be basically the opposite. So I totally agree with you. Yes?
>>: That's odd because mostly [inaudible] in Japan so I don't know why there is
[inaudible]
>> Jon Froehlich: I think we wouldn't, would you accept the fact that there are different
kinds of cultural components? Okay, so I think that's the major point. We can tease apart
different parts if we want, but that is the major point. I think Victor's point is right which
is there are on the fundamentally going to be differences, and it kind of goes towards the
interface. We have generally just one interface for everybody. And it shouldn't be that
way. It should be strongly personalized and that's one difference between paper and
technology. It should be personalized to us and then changed over time.
>>: Just to follow-up. Have you ever researched, did you do [inaudible] process to your
research?
>> Jon Froehlich: So oftentimes I pulled that out, but even with UbiGreen. So a lot of
times what we would do is we would do an online surveys, we would do formulative
studies where we would bring people in and have them look at different visualizations,
different kind of renderings of the graphics. We did in [inaudible] study with where we
gave people mobile phones that would actually track where they were in real time and
then ask them surveys about what kind of transit mode they took and why, to try to build
and have evidence for the kind of display decisions that we were making. Thank you.
[applause]