>> Sharad Agarwal: Okay. Well, thank you all for coming. It's my pleasure to
introduce Professor Rajesh Balan from Singapore Management University. I
think most of you already know him. He's a professor at Singapore Management
University. And he's going to be telling us about some of his recent work on
adaptive power management displays, particularly LCDs, and he's also around
for the rest of today if any of you want to talk to him. So please.
>> Rajesh Balan: Cool. Thank you. I know everyone here, actually. Hi. Hello.
Okay. So actually this talk is about what we presented Mobi Sys last year, how
do we save power on LCD screens without actually impacting user experience.
Actually gave a preview talk about two years ago probably in this same room. I
think Stefan is sitting over there.
And this time around I actually have a full demo, et cetera. You wanted to see
whether this would be something that you guys would want to take up, right? It
actually works quite well. So let's go into the problem itself.
So so all of you know the display is the number one power drainer on most
mobile devices. I think this hasn't changed regardless of whether tablet, phone,
LCD, et cetera. All that this number is still the same just everything has come
down, but the screens are getting larger.
This number is still the same. Usually the numbers are about 40 to 50 percent of
your power comes from your screen. Any savings you make here is significant.
A lot of people focus on the CPU. Turns out the CPU doesn't really cost that
much these days, network, only so much you can do on the network. The
screen, let's see what we can do. So one of the screens and S and generally
there are two main types of screens. The LCDs, which are used by everyone.
OLED used by Samsung and they don't give it to anyone else.
>>: Consider the entire life of the phone between charges, what is the percent
that it's good for display?
>> Rajesh Balan: Probably more. Because actually not clear. Right? You
actually raise a fair point. This is when it's actually being used. When it's being
used about 50 percent over the life of the phone depends on how often your
screen is off. But if you're using it it's going to be this larger depending on what
else you're using. Because if you're not using network this dominates.
>>: I was wondering, because all the time it's got to be listening to CDMA or
whatever.
>> Rajesh Balan: That actually turns out to be quite cheap. The GSM stuff is
actually quite cheap. Believe in Wi-Fi. This thing will dominant. Wi-Fi is not
notoriously expensive not sending data. Idle state really expensive. 3G is really
cheap on idle state terribly expensive when you're sending which is the opposite
of Wi-Fi.
So not clear. We have to do it, right? But this is where we're looking right now. I
don't actually have the answer for you. We did it when we're actually using the
phone. We're using the phone 50 or more, right? Usually. Depending on what
your other components are doing.
>>: How much ->> Rajesh Balan: GPS. GPS not here. This is just watching YouTube video.
GPS is actually is an interesting one because it is a power hog. But most phones
are smart enough not to pull it continuously. Most apps are smart enough not to
do that. The only app that will kill your phone dead is Google navigation.
Because that one I think turns GPS on to the max and that one will, my phone
will die in an hour if I turn on GPS navigation. Google navigation if my S 2 will
die in an hour, guaranteed. It becomes hot. It will really become hot because it
starts pulling in GPS at 100 hertz or whatever ridiculous rate it's doing. Most
apps are not that stupid. Accelerometer as well. If you put on the accelerometer
100 hertz forget about it. Your phone will die in an hour, hour and a half. So
sensors are not included here. This is just the three main things, if you turn on
your sensors, that adds a whole different ballgame which is [inaudible] here at all.
What else is there? Sensors. Audio we haven't looked at. Actually nobody
looked at audio. Most people leave their phone off. I don't actually know how
much the audio component of it consumes.
All right. So let's -- so our key idea and solution, which I'll just cover very quickly,
then I'll actually show you a demo. I think it's more interesting to show how it
works actually in practice then we can go to the gory details. I always find it
more interesting to show you how it works then go through the gory details go
through the gory details forgot and everything. Key idea is very simple. We play
games with your eyes. It turns out the human visual system can easily be fooled,
because what we're doing is not the same. Let me be very clear. The image we
produce at the end is not the same. It is close enough. It's close enough that
through the human visual system it looks the same but it's not the same. What
we do is we brighten the image in castle ways and then we dim the LCD,
backlight. Because what consumes power in LCD displays turns out to be the
back light. The way LCDs work, quick prime of LCDs they always have a
constant on backlight, filter in front and filter determines what color you see.
What they do they apply filters in front of backlight if you want black that's
basically a black filter backlight is still on. So that's not very efficient.
Which is why LCDs have better brightness and all that, and they have no worst
case performance. All that on the other hand is individual pixels, which have
three diodes each of them you can turn them on and off.
>>: I'm all LEDs that's the difference. OLED is individual pixels. Each pixel has
three diodes, difference is the kind of technology they use from what I
understand. And ->>: Fails miserably. OLED basically only two approaches which I will actually
have a paper on that coming up soon. But there's two approaches. OLED one is
to change your colors because it turns on the different colors multi power. Green
is the cheapest for most devices. Black is the cheapest, but that's my second
approach. You dim the image. Darken and turn things black. That's the one we
are using. People like Lynn Zong have changed the color map, made everything
green tinged. You still see everything greenish. What I've done is I've turned
things off so they're quite complementary.
>>: Did you have like a separate set of glasses that reduce the weakness ->>: But then you have to pay the power for that. Right? So somewhere along
the way somebody pays. LCD is quite nice because you have the backlight so
you can play games. You don't actually have to change the colors. You don't
actually have to remove the image. What I'm showing you here actually is
30 percent to 40 percent savings with no effect on quality. Which is awesome.
You get it for free. You just get it for free, basically.
>>: [inaudible] dark background. How well it's going to work with bright
background.
>> Rajesh Balan: You'll see it here. Right. It just works. Actually with light
background it actually works better because you can actually dim the backlight
even more. You have to prevent saturation of the colors, right? The key thing
with light backgrounds you have to prevent saturation. It's one of the things we
work on here. Because if you brighten it too much you saturation everything.
You dim it still saturate it. That's the issue.
So what we do starting off we use a game because a game is 30 frames a
second. All of research here. Hello. So we use the game. This is what the
game looks like. We brighten it. And then we darken the backlight. So in the
end number one and number three look similar. Do pixel checking you'll see
they're different. When you're actually using it, you know if you're not going to do
microscopic examinations, no difference whatsoever. Right? So before let me
actually show you the demo before we do the rest. I think just to show you guys
what's happening. What did you come up with? There's only five ->>: People who are watching remotely. Are there people watching remotely?
Okay.
Okay. So let me just start a YouTube video for people watching remotely.
>>: The second scale is the backlight intensity.
>> Rajesh Balan: Let me start it again. Come on. Today's it's not working day.
>>: [inaudible].
>> Rajesh Balan: Yeah. What am I thinking of?
>>: Introducing you to our system.
>> Rajesh Balan: Do I need the audio on?
>>: As implemented on the [inaudible]. The algorithm dynamically brightens the
scene. There is saving power on LCD screens maintaining quality and
brightness let me first introduce you to UI screen right now. The first scale on the
UI shows the artificial brightening of the game scene we had. The second scale
is the back light of the intensity of the device itself and it shows the power saved
by this algorithm. This is directly dependent on the backlight intensity. Now let
me show you the algorithm in action. Initially we show you the effect of just
increasing the artificial brightness of the game scene. We increase this from a
scale of one to three.
We now reduce just the backlight intensity itself so we can see how it looks. And
you can see the estimated power saved based on this backlight level. And we
also performed experiments using latent cells to determine the mapping between
these two metrics such as the brightness of the scene remains relatively the
same. Here we show you that mapping.
We have also conducted a user study to estimate the maximum threshold
brightness that can be added to a scene depending on the brightness of the
scene itself. Here we show you that the result of that experiments. We have
determined two separate thresholds. One is called the quality threshold showing
you here. In that basically users are asked point out the point at which the
quality starts deteriorating.
The second threshold is what we call the playable threshold which is point at
which the game stops being quite as playable. Here we show you that threshold.
The playable threshold ends up saving more power but the quality loss is a little
bit more noticeable.
Thank you for watching.
>> Rajesh Balan: In fact, that's the whole secret. You want to do nonlinear
brightness otherwise you'll saturate like crazy. We compensate with the LCD
back light. And the bright area still looks fine. Now you might -- you probably
save less power to answer your question you can't play two mediums too bright
save about 20 percent. If you're willing to actually jack it up to extreme levels,
right, say you still want it to be playable but you're okay with, you're okay with it
being a little bright for some definition of bright now, again, this is -- now, this is
saturated. It's still quite playable, though. If you actually care about your power
because now you're saving 60 percent. Right? But this is saturated. Now you
can actually see the effect when you take it to extreme levels, right? Everything
looks there. It's there but you don't see the contrast difference between bright
and light. This is the issue. How do you actually get it so that the contrast levels
are correct and save as much as you can. Turns out you can get about 28 to 35
to 40 percent on most scenes. And we intentionally chose a game because this
is probably the hardest thing to do because it's completely dynamic. The
person's moving all over the place. So this is just the controls. This is not doing
anything. The game's actually running fully here. Everything's actually on
Android. This is just our back end interface to toggle the settings.
Everything's running here. Everything's local. This is just ->>: [inaudible].
>> Rajesh Balan: Not very high. What you see is we do is quite simple. That's
the whole part. If you actually waste a lot of time -- yeah. This is playable. As
you say this looks quite playable, actually. And if I didn't tell you this was
modified you probably wouldn't know.
>>: [inaudible] how would that look like? Say browser?
>> Rajesh Balan: This actually works even better because that one there's no
dynamic change. You don't have any content so your power savings for that is
actually constant. We haven't implemented it because we looked at it, that's
really simple. Because there's no dynamic manipulation when you're in a game.
When you're in a browser, et cetera, everything looks the same. It's actually
quite easy. You will save about 35 percent. Now....
>>: Power saving, kind of dark.
>> Rajesh Balan: Yeah. When your screen is -- no. When your backlight is high
we get the most savings. When your brightness is high we actually get the most
savings. Brightness already low there's not much we can dim it either. Turns out
the other way if you want those people set brightness to lowest level we can
probably save more than that unless we turn it off. If you're one of those who
leaves it on or whatever the default is, yeah, we can help you. Kung Fu shirts,
we can help you.
>>: In our case would it apply directly to video as well.
>> Rajesh Balan: Everything. We're applying directly to the frame buffer in this
case. This is basically a video, running at 30 frames a second.
>>: [inaudible] watching YouTube video applying the same optimization.
>> Rajesh Balan: No problem whatsoever. That's why I picked the game.
Because games are actually 30 games per second and they have real time
constraints.
>>: Modify the content? That is the video.
>> Rajesh Balan: Sure, definitely. We could modify the content you could make
it better. But the whole point this doesn't need to do that. This is completely
content. It's completely application dependent. It runs fully on the laptop. We
also have it running on laptop which I can show you if you care. That's the laptop
version. This is actually Quak 3, the Android version. We actually have Quake
3, laptop version. Implemented slightly differently, because Android accepts
Linux. We have the Linux version. We actually did it on a giant 42-inch LCD TV
for a demo I did two weeks ago. It also runs on TVs. We showed it to a teleco.
They're like we can put it on set top boxes because most TVs are LCD now.
OLED is coming but it's not there yet. The video, the video is actually on the
phone. So ->>: [inaudible].
>> Rajesh Balan: You make it, you can make the content darker so you could
brighten it. The thing this will not work well on is bright content because you're
quickly at a saturation point. Right? Because if you took something that's very
highly white, like this screen, there's not much more you can brighten it this light
because it will saturate that's what you show on the second demo. Everything
got raised so high saturated, all looked like one color at the end. You had no
contrast. Whereas something like this, easy. Right, because you still have a lot
of layers you could go up the scale, the back is even easier, black TV is even
easier.
Okay.
>>: It's hard to -- looks very similar to my eye, but what about my unconscious, I
find that when I play -- when I play Quake 3, eventually I get queasy. Do people
get queasy sooner.
>> Rajesh Balan: No faster than previously. If they get queasy this one helps. If
they don't get queasy this one hurts. We did a user study. We did a user study
with a lot of people. In the Mobius paper we did it with 60 people. Running on
the laptop actually because the problem with this tablet is it's incredibly hard to
play because tablet controls suck. So we actually did a user study on laptop with
mass -- playable just the same. Gamers nongamers everybody, they couldn't tell
the difference. We didn't tell them what was what. We just let them run two
versions and, yeah, feel exactly the same.
>>: [inaudible].
>> Rajesh Balan: No. All measured using external volt meter. We do not trust
software. This is the paper that convinced me of all the software powered savers
all crap. Software power, power measure -- this one is celebrated based on
previous results. So we know what the settings it should give us now. It's all
calibrated on previous results. It's not 100 percent accurate. But it's all based on
previous measurement. We took every possible value of brightness, gamma
correction.
>>: Brightness.
>> Rajesh Balan: So we measured all of that previously. So this thing just goes
to the previous measurements, right, for the same device.
>>: These are different power [inaudible] LCD. I don't know the operator, the
video screen --
>> Rajesh Balan: Retina, LCD. IPS.
>>: But is it the power is the same? Because [inaudible] is totally different.
[inaudible] numbers it's different.
>> Rajesh Balan: [inaudible] and all that basically have the same, cannot use
this because there's no backlight. For [inaudible] you basically have to do either
as I mentioned to him you either change the colors themselves or you darken the
image. You turn off the pixels, whereas with IPS you still have a backlight how
much does that save you not clear you have to calibrate for the device it will save
you quite a bit. On the LCD type things the backlight is on no matter what you
do. So what we're saving is backlight power.
Dimming the backlight. We trade off CPU to brighten the image, and the cool
thing is -- and that is why the, if the product guys are here they might be able to
help me out as well. In almost all graphic stacks, the last part already does an
image manipulation. That is where we put our stuff. So it may not actually be
that expensive. If you're actually in a graphics pipeline. Because all they're
doing is applying a gamma protection. Lambda buffer which happens already in
most graphics pipelines if you do it the right way it should not be that expensive.
Theoretically it should not be that expensive. Again we'll see what it is. But even
with our implementation in software, we are still getting these numbers. We get a
little bit more. Now, this already explained how LCDs cover. So just to give you
a bit of background, the things we tried. Firstly naively dim the display doesn't
work you'll see flicker we've tried this. Naively dim the display based on what we
see you'll get a lot of flicker. The human eye is very -- it's very good at picking up
flicker. I think it's one of those early reactions. We like seeing things out of sync.
We can detect when something flickers very easily. It cannot flicker. This
doesn't flicker. You can compensate with increased brightness. You can linearly
apply. You can apply a linear function. Problem is brightly saturated, like this.
So what happens if you don't see the difference between the edge and the
insight. Looks like uniformly white, because everything has reached 255 level
brightness and everything just becomes the same. So what we actually do is we
use non-linear brightness functions, something called gamma in this case there's
lots of non-linear gamma brightness functions we use gamma seems to work.
There's other ways you can use this as well. What gamma does it basically
applies a curve. It brightens the bright areas less. It brightens the dark areas
more.
Now what happens if you lose contrast, doesn't saturate. That's why the image
is not the same. I'll show you later with the histogram. You'll see definitely later
it's not the same. This is what happens. We have the game. We brighten it.
Looks kind of bright. If you actually pay attention, the contrast has changed. The
contrast up here changed a little bit. If you actually pay halfway attention to. But
the key thing it's not so bad that you notice it unless you raise it to extreme levels
like we did in one of our demos.
>>: Can I ask a question? I don't play a lot of games. This probably is very
naive. So that was the original, the left was the original and the right is after we
applied the gamma. Why does the original look better than that?
>> Rajesh Balan: Because the original has got more contrast. That one is too
bright. You're looking at it and going it's too bright and some of the contrast is
less. The difference between the bright areas and dark areas is not as wide as it
used to be.
>>: For most people they would agree that the stuff on the left looks better than
the stuff ->> Rajesh Balan: Definitely. Definitely. If you don't -- if you don't compensate.
The key thing is you haven't compensated yet. We haven't compensated. We
give them the thing on the right.
>>: Remember [inaudible] asking the question about two pictures that you
happen to put on the screen right now. So one on the left one on the right.
>>: You have to remember it's not just what the user wants it's what the game
developer wants. There's certain semantics in a game where one certain regions
of the scene to be dark where I might have an enemy hiding. If I go change the
brightness, right, then the user can go see the enemy that's hiding over there.
That changes the semantics of the game.
>> Rajesh Balan: Yeah, we're trying not to change anything, and let me show
you what happens. So this is the original histogram. This is your standard
brightness, pixel histogram. Looks like this. All you need to do is look at the
curves, original image. Compensated image. This is the gamma equals two.
And this is after we actually reduce the backlight. We do a backlight reduction.
Now, let me put them together, right? Looks reasonably similar, the curve. But
they're not identical. This is the key thing. We're not giving you identical images.
I know that's almost impossible. But we're giving you something that is for most
people it's almost you can't tell the difference. Especially in these things going
back. Right? So this is what you ask, right?
Stefan, the middle image you can see is clearly different.
>>: That's not what I asked. I understand. I asked a different question. Your
point is well taken.
>> Rajesh Balan: So this is what we're doing. So it's not the same. It's close
enough. So we're following the individual system. Now, let me give you some
results, right? I already showed you. We tested on RBG game itself just for fun.
There are basically three challenges to do this, right, which is one is
understanding how backlight, et cetera, gamma, et cetera, effect the power
savings same thing -- we're to understand for L C Ds. We have to understand
human thresholds this is very important you can't naively apply this. You have to
know what humans can actually accept. Where is the level beyond which bad
things happen. Where can you draw the line? Very important and finally how do
you dynamically apply it. Turns out this was the easiest part. Once you know
the part, applying the solution turns out to be really simple. You look at the loop
and you go, yeah, every system has this. So first things first. We looked at how
you can save power with backlight. Turns out on many, many devices we tested
on two phones and one laptop.
Probably other devices as well. Turns out to be fairly linear. Now, the
percentage as you changes a bit, as [inaudible] said, it's not that much like, on
HD Hero saves a lot more than HD Magic. But the line is linear.
The more power we apply to the backlight, the more power consumed by the
display. Turns out this was linear. We measured this using external voltage
meter.
There is some fuzziness when you go up to some values. But by and large it's
linear, which made us happy. Now, how do you actually handle gamma? Turns
out gamma is a very, very tricky function. Because it is non-linear, the gamma
value actually goes from 0 to 10 but if you apply 10 you'll see nothing but white.
Turns out the useful range is actually between 0 to 2. That's it. Right? Because
it goes this way.
It quickly saturates around this point, right? Because it is nonlinear. The less
you apply of it the more effect you get. Once you reach past a certain point you
reach a point and applying more doesn't help.
This actually made us very worried, because this told us that we needed to
calibrate with humans, because every setting, the difference between 1.1 and 1.2
was very, very big. It wasn't linear. You couldn't just naively apply it. So what
we did ->>: Can we go back? So difference by difference you mean that the wide ->>: What we did was we applied the gamma value of 0.5, this is the backlight
level that will give us the same brightness for the original image. We started with
image at 255. We applied gamma. This is the backlight level we set to to get the
same brightness. Turns out you needed to go lower and lower. Very fast. As
you applied a little bit more gamma, brightening it a little more. Once it got to a
certain point it saturated. Couldn't get lower than that. Image was completely
wiped out. And it saturated really fast. I mean, this range is really fast here, this
curve.
It drops really, really fast. So which is why you look at levels here the gamma is
like 0.64, 0.65. Levels are very low.
Now what we needed to do is we needed to map the gamma to what users could
actually see what they thought was useful. That's actually a big hole in our
evaluation right now. Our user threshold actually generated only from five
people, five-grad students. We got them in because this actually needed proper
calibration. Brought them in with property light sensors, everything. Took about
half an hour per person. Possibly an hour. We gave them a whole bunch of
images of different brightness levels, compensated said does the brightness look
the same to you, does it look the same to you, same to you and we actually
calibrated all the, what they thought was equal brightness with proper light
sensors, et cetera, so we brought in a proper light meter.
Now, this calibration is where this thing by and large will break down. If our
calibration doesn't work, if we didn't find the representative users, it means what
we show you, Stefan won't like it. Shared won't like it. I won't like it because
we're not the same as these people.
If this is actually deployed this step may have to be relooked. One criticism these
are all students. They all have reasonably good eyesight. What happens if you
go with people -- well, they're all young, at least. Their eye systems are at least
reasonably young. So what happens? You go with more elderly people, you go
to people who are colorblind we don't know. Frankly we don't know.
>>: Do you see much variance amongst these students?
>>: These students, no.
>>: Do you have any reason to believe that the ->> Rajesh Balan: I can't answer that. Honestly I can't answer that because we
don't know. Could be the case that some people perceive brightness quite
differently. I'm sure that's true.
>>: Chose that are very sensitive.
>> Rajesh Balan: I'm sure that's true. If you have individuals who perceive
brightness quite differently or people are colorblind what we're doing could -- it's
not clear whether you look the same. It might look the same as in the original
thing already -- the next one might suck just as much or it could be quite
different. That one we don't know. What I'm saying is I don't know whether
modification would make it worse or make it better.
>>: You can imagine having a calibration step much like you do on your phone.
>> Rajesh Balan: Set brightness level, right.
>>: But you kept talking about but we had a light meter, we had a light meter, will
it not work if I don't have a light meter?
>> Rajesh Balan: It will not work. It will actually work well with light meter just
being really careful. For this we're being really careful. I suspect if you actually
deploy something like what XBox does or something in our game, set the
brightness until this thing is barely visible it will work just as well. Since we
already know what goes on the back end. Since we already know what goes on
the back end we can probably calibrate it by saying please set the brightness at
some level we know all the other points.
So this is what we got. We ended up with two different thresholds. Conservative
is what we showed you. It's playable as in there's no difference of quality. This
is point where they said quality does not change. Right?
So these are the gamma values. Notice everything is below two. More or less.
Very, very low amounts of gamma being applied for different image brightness.
This is where they said, yeah, it looks bad but it's still playable. That was the
second level I showed you. Looks bad but still playable. It's not the case where I
look at this. I can't see things that I need to see. Looks a bit washed out but it's
still quite playable. Enemies can still play, things I can still shoot.
So those are the two levels. That's how we calibrated it. The actual algorithm
itself is quite simple. It's a standard feedback look. We calculate the average
brightness of loss examples, X is tuneable, and this game I think it's 200
samples. Because it doesn't do it more than once every two, three seconds
because otherwise you might see flicker.
>>: [inaudible].
>> Rajesh Balan: We do it on the whole image.
>>: So X samples means X images, frames.
>> Rajesh Balan: Frames, 200 frames. Look at 200 frames, average brightness
of 200 frames. Then we calculate if it's more than X samples, we calculate what
the average brightness was and apply compensation. If not, we leave as it is.
Then we sleep for 5 milliseconds. I think right now we're doing it every 200
milliseconds. I think the Android is 400 milliseconds. Once every half second,
roughly, maybe 500 milliseconds.
If you do it too often, your CPU load goes up and you also tend to get flicker. So
this is something that has to work, and we have something that works. So okay.
What we did to test it is we tested it on a phone and a laptop. We gave it to
users, and we also measured the power. So what we did is this is unmodified
level. We give with nothing modified obviously. We give with two levels, one is
full quality. One is playable. So those are our two thresholds.
The thing where we claim there's no difference and one where we claim there is
a difference still usable. Even though it saves a lot of power. These are the
numbers we got. By default you save nothing. When you actually turn on the
power savings, you get 49 percent. We get less on a tablet because a tablet is
actually, the tablet actually does things a little differently on the LCD, on the
laptop we actually got more power savings because the laptop the display is
larger.
Turns out the larger the display the better the system. Kind of makes sense.
Because the CPU, everything is about the same in some level.
And aggressive we got 70 percent. So on a TV we actually do really well, when
we actually did 42-inch TV. We didn't measure the power but I bet you we'll do
quite well.
So that part is easy. Then we actually gave it to users, and we said here let's
play games. Turns out lots of people want to play games we got overwhelming
response to this. We had 60 people managed to get them all in and out in two
days. We also played three versions. One normal. One with our system in fully
playable no difference and one with the system set to the high setting where it's
saturated but still playable. We didn't tell them which was which. They all got
calibrated by playing the game a little while first. So they knew how to play the
game. They saw what it looked like by default. The key thing to take away is
that the default and the very playable version within the noise. They turned out
to be exactly the same. Right? There is no difference between the two. We did
a T test, statistically insignificant, the difference, just artificial. This is statistically
is significant. The difference obviously when they saw the bad version they said,
yeah, this kind of sucks. Didn't suck that much, but they did say, yeah, it kind of
sucks, below neutral.
But these two are almost exactly the same couldn't tell the difference, that's what
we wanted to see looked at it, yeah, to us they look the same. The standard
deviation, everything is about the same. T test showed discretely insignificant.
In terms of power, right, so this is the big take-away. In terms of power, the
difference in aggressive and conservative is very different. Difference in between
30 percent savings and 69 percent savings, huge. Obviously you want to go
aggressive. But it kind of sucks. But as far as the users are concerned they kind
of hated it. So these are maybe two settings you could set in practice, if you
want to go extreme. Or maybe you don't even give it.
What's really interesting is this difference between conservative and default.
Basically you get all of this for free. This gap between 0 to whatever this number
is, it's basically for free as far as I could tell, because as far as the users are
concerned they couldn't tell any difference. We had 60 users. We had gamers,
non-gamers, everybody, and averaged out the T test, no difference. And they
didn't configure -- the people configured it didn't play the game, by the way. The
people who from whom the settings were taken from were not our user study
participants. These were 60 undergrad, the people who did the calibration, all
grad students.
That's basically the big take-away, that somehow we have a system that seems
to give you about 25 to 35 percent, maybe 40 percent. Almost for free. It's not
the same image. Right? We have to be quite clear. But it seems to be -- seems
to fool people enough that it actually works.
>>: So before you go into the other technologies, so when you presented the
algorithm, you said that you're doing this every 200 milliseconds.
>> Rajesh Balan: 500. Every half second.
>>: Why not every frame?
>> Rajesh Balan: It's very computationally expensive. Because -- so Stefan
you're actually correct. We could if we had a smarter way of doing it. If we were
in the kernel we could do it in every frame if we were in the right place. We're
doing it in the software right now. End user space. We don't want to do the
contact switch every so on. Because right now the way we're doing it is quite
expensive.
So if we actually did it in the kernel we could do it every frame and probably even
do better but we haven't investigated this, because we haven't gone into the
frame buffer, low level bits of Android to actually do this.
>>: Just to understand, if you were doing it every other frame does it mean you're
missing on half the potential of energy savings.
>> Rajesh Balan: Possibly. It's an interesting question. It depends how often
your brightness changes between frames. Frames are very small, though,
because these are 30 frames per second. Your brightness doesn't usually
change that quickly, because it's all user-driven. The user has to move. The
users don't move that quickly. So we didn't see any real need to do it every
frame. Plus it was very expensive. In a real implementation, we possibly could,
but again I don't think you'll gain that much, because the brightness really doesn't
change that fast.
>>: Changing frequency would affect the average power at all because half the
time you're not changing soon enough from a level that you would, say, more
power and half the time you're not going ->> Rajesh Balan: Yes.
>>: All the checking more often would do is you'll have higher dynamism and
those users who can notice ->> Rajesh Balan: Possibly flicker, exactly. I think that's true. You might
introduce flicker if you change too quickly as well. It's an interesting question. I
don't know the answer. This will require more in kernel implementation, where
you can actually overcome the overheads.
>>: You could do it faster if you sample just instead of checking the average
brightness of all the pixels if you just pick 20 random pixels and average -- if you
pick them all over the place, it's probably going to be approximately ->> Rajesh Balan: I think actually the tablet does that now because Android's
kind of slow. The laptop we actually do more because the laptop has a better
CPU. Linux gives you more bits I think on the Android implementation we do sub
sample, right? I think on the Android it's every two or three seconds this thing will
show it's five milliseconds. It's jerky, get quite jerky if you do it every 500
milliseconds. So that's some holes in the way we implement it. This is proof of
concept.
>>: [inaudible].
>> Rajesh Balan: No, I don't know the answer. Good questions. So all the
displays I'm actually working on right now. So we're actually working on what
you can do when you dim the display in strange ways. So I actually have a
solution in the Android kernel that's being user study happening in 12 hours back
home. My minions are working on it for Mobi sys. We actually have something
in the kernel that works on all applications and games, dynamically, darkens the
image. The idea is for more apps the top is the most important portion, the
bottom part is history, all the parts. If you could dynamically darken it in a way
Facebook most of the bottom parts is all the content. Infinitely scrolling. If you're
willing to change your user experience a little bit, all that you have to -- there's no
way to do it without changing your user experience as far as you can tell because
each pixel is individually lightened. If you're willing to change your user
experience a little bit, the numbers are getting 40, 60 percent power savings if
you're willing to change your user experience a little bit. We don't know if they
will. We do the user study today, see if they will.
But numbers are getting quite cool. 45 percent. It's a very naive approach.
>>: Compared OLED and LCD, what's the power characteristic.
>> Rajesh Balan: It's very hard to say that. On average, I've looked at so many
papers, Linius is probably the expert on this [inaudible] might be doing the run
stuff on this. On average it's 40 percent less expensive than LCD, but worst case
it's higher. If all you see is a white image, all that consumes more power than
LCD because LCD is constant power. Not affected by image at all. OLED is
really affected by image. If your background is black you're probably 70 -- if your
background is mostly black, it's probably 60 to 80 percent the power of LCD. If
your image is bright, white, it's about 20 to 30, maybe 40 percent more than LCD.
On average about 40 percent less. OLED has this huge variation, which is why it
surprises me that your Windows Phone by default everything is white
background.
>>: We're using ->> Rajesh Balan: 7.5.
>>: We're using LCD.
>> Rajesh Balan: Yeah.
>>: The markets ->> Rajesh Balan: Convince something to put in OLED version.
>>: Right. The sense of the Windows Phone 8 ->>: [inaudible].
>> Rajesh Balan: Yeah, but most of it previously was white. Which is fine on
LCDs, sucks on OLED. It's an interesting question he asks me is it all that
common? The answer is yes or no. It's yes on every sampling device. It's
known everywhere else. They refuse the share. But they have the most popular
phone on the market now. The S-3, I think, and dominate the smartphone. They
took over [inaudible] phones. We'll see. It's not clear where the war is going to
go because they want all the patents on that. So that's why Sony -- Sony was
using it, but they're licensing it. Apple and Samsung don't talk to each other, so
that will probably never happen. At least not in the near future. HTC tried to
license it. I think they failed as well. So we'll see. We'll see. But it's very hard to
find an OLED tablet. We tried. There's 1.7 inch, we could not get it working. It
was such a pain.
>>: Samsung.
>> Rajesh Balan: Only Samsung, the Galaxy tab, are all LCD. Only one Galaxy
tab 7.7 is OLED. We looked high and low.
>>: Why is that?
>> Rajesh Balan: I have no clue. I need to track someone down.
>>: Is that because of the cost of larger OLED displays.
>> Rajesh Balan: Possibly. Supposedly OLED is easier to manufacture. It can
be curved and thinned we see OLED TVs coming out they're like this. It's like
wow.
>>: More concerned about yield.
>> Rajesh Balan: Not clear. Not clear. But for some reason even the tablets are
not OLED except for one. We couldn't get it working for Mobi Sys my grad
student crawled up in a fetal position after one week he said the code doesn't
work. I said fine. Let's do an S3. We're working on a Galaxy and S3. That's
fine. When they crawl in fetal position, you have to help them. Can't do anything
after that.
So, yeah, we're looking at a whole bunch of power techniques for various
devices. So LCD made us happy. And this actually can be applied. So that's
why I presented this and Alec and Victor said come talk to your product guys. So
okay. I guess they're busy I'll try again.
>>: Is it a user ->> Rajesh Balan: Right now we're doing a very naive thing. We're actually doing
it in the application itself. We haven't put it as a service or anything like that. We
could. We figured the right people to do this, either you guys or Samsung. So
I'm also talking Samsung, because this really should be in the kernel or the
Android operating system. Google, you guys, Samsung are the right people to
do this. Doing this every application, no app developer wants to do this, because
it's just going to be more work for them. Right? This really shouldn't be an app
service. It should be a system service at some level.
That's the way we looked at it.
>>: Operating system, do you mean the driver or ->> Rajesh Balan: It should be -- this is a good question. I know where this ends
up and Android at some level. I don't know about Windows. Android for some
level, it should go -- should go into the kernel bits that actually output to the LCD
screen itself. There's a place inside the kernel where there's a video driver which
actually does the last manipulation of the frame buffer to put into the LCD. That's
where you do the gamma correction.
>>: [inaudible].
>> Rajesh Balan: It's the last step in the graphics pipeline. Right in the open GL
pipeline. Right before you output to the buffer you'll do gamma correction.
>>: That's not done in like ->> Rajesh Balan: It might. But you have to send the right command. I think on
some of the newer phones if you're playing games it's done on the GPU. They're
called tesselation lighting TNL. You want to put in the last TNL step. But this
would require the graphics experts to figure out where to put it.
At some level all of this, because this is just a brightness compensation, is
already there in the pipeline. You have to find the right place and just tweak the
settings to use this and you get it for free.
But I don't know where the right place is because every pipeline does it
differently. Every device does it differently at some level. So what we're doing
right now is quite naive in the software. Using a command parameter. Changing
the gamma and brightness. X Ren R for those who use Linux, that's what I'm
using to change this now. So system call. Just kind of stupid, but it works.
>> Sharad Agarwal: Is that it?
>> Rajesh Balan: That's all I wanted to say.
>> Sharad Agarwal: Thank you very much.
>> Rajesh Balan: Thank you.