>> Ashish Kapoor: So it’s my pleasure to welcome Jacopo Staiano, Saltiano --.
>> Jacopo Staiano: Staiano.
>> Ashish Kapoor: Staiano, my apologies, who is visiting us from University
of Trento. He is very interested in research and human behavior modeling,
specifically with focus on effective computing. And today he is going to
talk about some of the interesting work that he has done on social networks.
Okay, Jacopo.
>> Jacopo Staiano: Thank you. So first of all I have to prepare you. So
this talk was that I will give at [indiscernible] in a couple of weeks and
therefore it’s quite short. So I will talk about this work and then I will
give you some hints on other works I have been doing lately. So this is
involving data from smartphone’s and afterwards I will give you just a quick
outline of a data collection we have been doing using [indiscernible]
sensors. And then finally I will talk about a system with a device for user
experience simulation using a web cam. So it’s, there will be a jump, I have
to fill one hour some how.
So, okay, the title is basically what we try to do is we asked ourselves if
there is any relation between the social networks that a person builds around
itself and is included within. And basically we asked how effective egonetwork structure features are in predicting user’s personality? And most
importantly what are the benefits of using activity-based data verses surveybased data?
So personality is seen as a static and internal determinant of behavior that
means the difference is our disposition towards belief, actions and attitude
information. And it’s often used to explain and predict behavior. The model
we have been using is the “Big Five” factor model, which basically defines
the five traits that are chosen as being the most [indiscernible] as
possible.
And those five traits are extroversion or introversion which basically
describes a person on his/her sociability verses shyness; emotional stability
verses neuroticism, so if a person is calm, unemotional or anxious.
Agreeableness or disagreeableness, so if friends, cooperative or it’s a
faultfinding attitude. Conscientiousness on the other hand is a trait of
people who are organized while unconsciousness people don’t really care much
and are inefficient in doing their duties. And finally a trait related to
creativity which is openness to experience. So distinguished people who are
intellectual and insightful verses people who are unimaginative.
So why do we go for personality? We go for personality because it’s linked
to preferences and attitudes. It’s relevant for user modeling and
recommendation systems. And it basically is a factor if you want to devise a
persuasion system because adoptions that I might take as an extrovert person
are probably driven dynamics if I compare it to a guy who has a different
personality than me.
One thing we have to, I want to stress about, is that there have been
previous works in both the social psychology and the social computing fields.
The main difference between these two fields is that while social
psychologists use traditional social networks in the meaning that those
networks are built using [indiscernible] which you indicate friendship ties
or relational ties with other people.
And from the social computing perspective we instead use social events that
are basically like a call I make to someone or if I am wearing a device that
tells me I am around your right now. Those are events, I am not self
reporting that I am friends or that I know any of you, but there is a device
that’s logging somewhere that something is happening between me and the other
people that are around me.
So, [indiscernible] I have found that psychological predispositions, so
personality, are linked to other network characteristics. Also there have
been works made on mobile phones that are related to the work on
[indiscernible]. So Chittaranjan and [indiscernible] found out that actorbased features, that means code durations, how many codes they do, can be
predictive of the Big Five traits.
On the other hand De Oliveria and Oliver found that using those actor-based
features and augmenting them with the networks structure features you up the
significant performance improvements.
So we basically had access to a database which was collected at MIT a couple
of years ago, last year. It’s composed of an international sample of more
than 400 students. And those students are married graduated students living
in a student residency. And in the experiment there are this unit and this
partner taking part of the experiment. So you have data also from people who
are not really into that community, even though if you look at it tends to be
quite [indiscernible]. So this is one of the limitations I will talk about
later.
There is a vibrant community life, so social events and facilities. And there
are some that are sub communities, so social groups based on the ethnical or
geographic province of the participants.
So a rough view of the system that we use to collect the data. It’s
basically this is an application that is fairly downloadable on Android
systems. It’s called the funf and basically it allowed us to gather some
data which I will describe in the next slides. And that data was augmented
through the users Facebook app which was getting information about the people
who allowed us to do that.
And importantly it was, the data we had was augmented with the
[indiscernible], leasing [indiscernible] of the people who were participating
in the experiments. Of course all of these were complemented with the
surveys that we use to submit to the participants.
So the platform itself is composed of a number of probes which are, let’s say
the sensors for which it interfaces with the subject. And then there are
applications. There is a market application, there is basically all means to
know what the user is doing with that smartphone. So if I install a new app
how long it takes for you, my best friends, to adopt it too.
If I keep using it how long it takes to spread over my --. This is one
example. And of course all surveys and notifications, or whatever logged as
well. So this work is based on the pilot phase of the study. So we isolated
the pilot subjects, 53 subjects. We are using the --. We are logged for 8
complete weeks. The data we used are the Bluetooth hits, call logs and
sociometric survery data.
So the Bluetooth hits were basically driven on, derived from obviously the
Bluetooth sensors, but they were filtered on the signal level in order to try
to avoid, as much as possible, false positives. The Big Five questionnaire
data was used to gather the ground truth for the personality traits.
So from this data we built three networks; one was based on call logs, the
other on Bluetooth and then we have the survey one. And we also merged the
call and Bluetooth data by taking the intersection of the two networks and
built, let’s say a digital network based on the entire digital information we
had; while the last one was built on the sociometric survey, which was
basically, by the traditional means, asking what your relation is with the
other people.
So just at a glance the distribution of the ground truth which we had used to
[indiscernible] and we extracted a set of features that are related to the
structures of the network the subject was in. We focused mostly on the egonetwork structure features because our hypothesis was that the structure of
the ego-network of a guy who tells something that could [indiscernible] the
prediction ability of a model.
So we started centrality measures, small world efficiency measures and the
triadic and transitivity measures. In particular, sorry this is a fixed
slide. In particular we have started the standard centrality measure. Then
we have started the local efficiency, nodal efficiency and then the mean
local/nodal efficiency of the ego-network of the subjects. And with respect
to the triadic and transitivity measures we extracted two different
definitions of the triadic [indiscernible].
The first was formalized by Davis and Leinhardt’s and basically it’s account
of different configurations of triplets in your own subject ego-network.
While in the Kalish and Robins’ definition this configuration is based on a
strong/weak, strong/weak assessment of a given tie and in a different
configuration in which the ego would be present. Other than that
[indiscernible] created the [indiscernible] efficient, the global
transitivity and the mean/local transitivity. Meaning the mean of the local
transitivity so, for alters of the ego i.
So, to give quickly some of definitions; the centrality measures are the
degree which is defined just as the number of alters [indiscernible].
Closeness, I don’t know if you are interested in this, but I wouldn’t spend
much time on this, so it’s the [indiscernible] of the sum of the distance to
the alters, betweeness, [indiscernible] which is a relative drop in
efficiency when i is taken out.
Then we have efficiency measures which measure basically the response of the
egonet when the node, the ego is removed. And another feature which is the
inverted harmonic mean of the path length. So the closer the i is to its
alters the higher its nodal efficiency. And then we calculated values
relative to the entire egonet of i.
Finally we computed the triatic measures that I described before and
transitivity measures. So our experimental setting was based on the 4
network types. So, the network based on the co-network, the co-data, the one
based on the Bluetooth data, the [indiscernible] and then finally the survey
one.
We evaluated 8 feature types, so centrality, centrality plus efficiency
measures, transitivity, [indiscernible] the combination of them. After 8 we
had the binary test classification task for which we used the Random Forest
algorithm and we used the bootstrap, a bootstrap strategy. So basically we
embedded these in a leave-one-out setting, so basically at each iteration we
will take a subject off then for other times resample the set; the data set
that was remaining and then using that one as training and to predict the
left one out subject.
And this was suggested by a paper --.
>>: What is the binary classification test for?
>> Jacopo Staiano: So it’s basically for each trait.
introvert.
So you has extrovert,
>>: I see, so you didn’t do like a 5 class classification for access?
>> Jacopo Staiano: No, no, you have 5 binary classification tests basically.
>>: Got it.
>> Jacopo Staiano: And then we tried to explain our results by running some
correlation tests, so global surveillance analysis and some analysis of
marginal means.
So the results are, I will skip the boring tables and go more into the juice.
So, global analysis surveillance was performed. So we ranked the accuracy
figures and analysis surveillance was run on a design which was 5 traits by 4
networks by 8 feature sets that we defined. What we got out of this analysis
was that basically the Bluetooth network, the centrality with efficiency
features on openness and agreeableness were coming out as being the best
performing ones; in particular running a pairwise comparisons marginal.
On marginal’s you find out that this is the ranking of the, for example, for
the network types you will find that Bluetooth network would be more in
informative, then the merged one, then the survey, then the call one. Then
regarding the feature set, as I said before, the centrality and the
centrality plus efficiency were above the other ones. And with respect to
the trait then openness and agreeableness were over the extroversion and
consciousness.
Now going to the traits we ran 5 ANOVA’s, so one per trait. And what we
found out is that basically the network type is a significant influence on
all traits as you can see from this table. And the stable i accuracy of the
centrality measures somehow counteracts with the good performance of
transitivity measures, especially on extroversion of Bluetooth and on the
other social traits. So agreeableness as you see here and openness, which
kind of makes sense because those are social traits. So the transitivity
gives a measure of how close together are your others.
So for example this is a graft showing the performance on extroversion. The
blue line indicates the Bluetooth network and this is the peak over there
which indicates the transitivity set of features.
>>: When the Bluetooth is telling you how, I can go back, the Bluetooth data
is telling you how --. Yeah, so the Bluetooth data is telling you how
extroverted they are and how are you figuring that? Is that by how many
times their Bluetooth is within the presence of others?
>> Jacopo Staiano: So basically the Bluetooth data is telling you, “Shall we
use it as a proxy for proximity”?
>>: Okay.
>> Jacopo Staiano: And basically it was, so the smartphone will act in an
opportunistic way. So each, at most, at least every 5 minutes it would sense
who’s around and log them. And then as a post processing step you would use
some filters to eliminate a lot of false positives because with Bluetooth you
have like a 10 meter range or so. And if you have a really tiny wall you
also have a heat. But then if you filter by the signal level, for example,
and you, you can get a decent proximation of the proximity network.
So for example one result that came out was that basically, as you can see
the extroversion for example now here we are focusing we saw on the peak on
the previous slide, so extroversion on the previous network. And you can see
there is a positive correlation between extroversion and the full, the
complete triad we can say. What is negative otherwise and this just makes
sense because it’s a measure of the [indiscernible] efficient.
And we explain these values by saying that probably extrovert people like to,
let’s say filter out people who don’t fit in their network. So like
extrovert people are probably better off with other extrovert, which also
somehow makes sense. And it’s actually supported in the literature by a work
from Hallinan and Kubitscheck. So they show actually that friendly students,
so extrovert students, show a lower tolerance for intransitivity triads and
tend to remove them over time, so this might be an explanation of the results
we got.
We also tried to compare our results with the actor based features we had.
We couldn’t directly compare with other works, for example the one by
Kubitscheck, because we, in our data we only have width; we don’t have for
example the duration of a code. That’s not represented in our data
unfortunately so we couldn’t really extract all the features they used.
Anyway we extracted all the features we could from our data and what we found
out was that basically the network features, network based features always
led to better results on all traits.
Turning to the limitation of this work, one is definitely the small sample
sites because we only have 53 participants. The population is quasihomogeneous and importantly we have no logs about interaction data with nonparticipants. So we, we can’t really tell if a guy has a really sparse
network because he just doesn’t use his phone for calling or because he only
calls people who are not participating in the experiment. So this is a
pretty strong limitation to our work.
So the take away messages are basically the structural egonet features are
important, look important for personality studies. The data, activity based
data, seems to be more informative than surveys based data. And
triadic/transitivity measures are very relevant for extroversion, especially
in a [indiscernible] network like the one we built on the Bluetooth data.
And centrality and efficiency were relevant for neuroticism and the
agreeableness, overall on all traits, but on neuroticism the proximity based
network was more relevant than the one built from call lots.
So this is the first part of the talk. If I don’t have any questions on this
then I will go on with the other. Yeah?
>>: I think you actually answered this, but I might have missed it. So for
introverted people would, if you looked at like their local social network
would their social network graph have lower diameter than extrovert people?
>> Jacopo Staiano: Sorry?
>>: Would the social network graph of someone who’s introverted have a lower
diameter than that of someone extroverted?
>> Jacopo Staiano: A low diameter? So you mean there are more --? Yeah, no,
it should be the opposite I think. Yes. I guess it should be the opposite
because basically if we go back to the, to this one.
So for example here we are talking about the extroverted people; so the
opposite of the introverts. You are on either one of the spectrum. And it’s
very negative [indiscernible] for example with let’s say this setting of
triads or this one. Let’s say the non-closed triads. So I expect, no your
right, I would be longer. You mean the diameter?
>>: For extroverted, it would be longer for extroverted people?
>> Jacopo Staiano: No for introverted.
You mentioned that right?
>>: No I said it the other way around. The diameter for an introverted
person, I am sorry, should be, I feel should be shorter or smaller. And for
an extroverted person it should be wider.
>> Jacopo Staiano: Yeah in --.
>>: So it’s opposite then?
>> Jacopo Staiano: Yeah, our results suggest the opposite, because it looks
like the extroverts have a tighter network so the diameter is short right?
>>: Yeah, okay.
It’s just, it feels like --.
>> Jacopo Staiano: So let me see if you have a sparser network --.
>>: I feel like if someone is extroverted they should be more likely to make
friends quicker and with wider groups so they have like a longer path for
their social network. So they have more [indiscernible] components in their
social network.
>> Jacopo Staiano: So introverts should have a bigger diameter?
diameter you define the --?
So by the
>>: Yeah, but my point if someone is introverted and they have a small close
group of friends who are all probably next to each other so their diameter is
probably lower.
>> Jacopo Staiano: All right, yeah, that could be the case too.
>>: Okay, so yours is just the opposite of what I --?
>>: I think what you are saying is like extroverts friends, all like
different people, also know each other.
>>: Yes.
>>: So it’s shorter because it’s --.
>>: Because it’s more tight-knit, so that was my intuition.
are opposite of what my intuition is, so I was curious.
But his results
>> Jacopo Staiano: Yeah. So if there are no more questions left I will give
you, so I will give you a quick glance of actually our latest work which was
basically a data collection we ran with the [indiscernible] sensors. So the
sensors are the ones you see over there and the SocioMetric Badges. And
those sensors are equipped with microphones, the Bluetooth, an infrared
sensor, and an accelerometer.
What we did, again, is that basically we recruited the volunteer, actually
entire search units in our private research center, and we got over 55
participants over 6 weeks that were wearing these badges Monday through
Friday on the working hours. And we used an exponential sampling strategy so
that every day, three times a day they would receive on their mobile phones,
or on their e-mail box, a short questionnaire meant to be like 2 minutes, 3
minutes.
And so we have a bunch of variables logged into survey data. We have the
phone and e-mail logs and of course the sensory logs. The data we get is a
behavioral from the badges and from the mail and phone logs. So, how people
interact with each other by technological means or physically in the case of
infrared sensors that takes, for example, the face to face communications.
On the other hand we have the survey data which are pretty comprehensive and
they include personality, affect and loneliness, creativity and productivity
and situational items which we will use in order to reconstruct the mood, the
context in which the person was acting within. So this was just a quick
glance.
And now we will turn to a paper I presented a couple of months ago in
Newcastle which is quite different from what I said before is still related
to behavioral analysis, automated behavioral analysis.
In this case instead of using smartphone’s and data gathered through mobile
phones we used basically a webcam and the user is actually a subject testing
a new prototype for a softer system, so a whole new interface.
So in contrast with the feature you have on the left there in user experience
studies the emotions which are listed are typically low-intensity, mixed
nature, they are dynamic and simplified.
The research goal for this work was trying to understand how emotion-based
mere measures could support the user experience evaluation. We tried to find
the validation for new measurement tools and the overall goal was basically
to build some non-intrusive methods in contrast with the EMG, for example, to
seek on the subjects face in order to exploit emotion in user experience
evaluation.
So emotions are used to be measured in different ways. One way is the
questionnaire measures, which has pros and cons. So on one end it’s
ecologically valid on the other hand it doesn’t capture interaction dynamics
because it only happens afterwards, when the interaction is over.
On the other hand psycho-physiological measures like EMG sensors have the
benefit of capturing interaction dynamics, but on the other hand they require
expensive equipment and they are not ecologically valid because you have to
call a subject, call him in the lab, and stick the stuff on it which might
bias him and so on.
So what we propose is a system that basically automatically monitors the
emotional state of the users during the interaction. It is cheap in the
sense that it only needs a consumer level webcam and a computer. It’s noninvasive because there is no sensors attached to the user and it can
potentially be run at the user’s place. What is provided by the core of the
system is basically the detection of minute changes in facial activity during
the interaction.
So this is how the initial prototype is designed. So in a very simple way
there is the user in front of the camera. The camera gets video feed. There
is a module that tracks the facial motion. And those facial motion features
that are extracted, they are indexing. So basically there is a postprocessing step.
Then there is a --. We devised a calibration phase and a prediction phase.
Basically in the calibration phase we followed literature and the assigned
two really quick tasks who were prone to elicit the reactions that we wanted
to predict on the, for the automatic UX assessment.
So I will go into details now. So the facial features are based on the
[indiscernible] facial action coding system by Ekman, which in ‘78 defined
the set of more than 40 action units that are responsible for building for
facial expression taxonomies.
Our system uses motion units. Motion units basically are a subset of those
action units. And the relation between motion units and action units is that
while action units gives you a coordinate of a certain point in time our
system gives you also the [indiscernible] component so the amount of distance
you had from the previous position to the next one.
So this is an example that shows the muscles on the face of a human face
which are the most important muscles in the [indiscernible]. Anyway, this is
what the system looks like. This picture here only shows that it works, that
it used to work with really low, low-fi cameras, which are not in commerce
anymore right now. So that’s a good thing. That’s how it looks like for
real. And now I will show you a video.
So this is the tracking system in action. So basically how it works is that
it fits a 3D mask on the users face and basically we use it in order to track
certain points which are the ones at the joints of the face which are most
related with the changes in the facial expression. Like this one is an
impression that we want to take. We want to detect exactly those kinds of
things, while the user is playing with a new interface. There should be
another one, like this. So within those are signals that might, you know,
combine some information about the problems that the user is experiencing
with your new interface.
Okay, so basically, sorry for the mess with the video. So basically we have
our focus was on usability and in the literature we found out that some
combination of features, facial feature motion, were found to be correlated
with the confusion and frustrations. And since those were relevant to our
study we decided to basically following the leaders, to build compound
variables over the features that we had.
So here is a combination of them. And so we computed those indexes for
confusion and frustration, but it’s important to say that we don’t want to
detect frustration or confusion. We only use previous works in order to
build a new feature based on the features that we extract and then go on with
it.
The activation indexing stage that I spoke about before basically it uses a
technique which we found it literally to be very common on EMG signals. So
basically a feature would be consider active to given frame in time if its
value is above it’s main plus sum deviation value. So we stick with that
definition and we post-processed all the features at the start and that’s how
we built our vectors of features.
So we ran a couple of studies for this software and the first one we used 15
MSc students who had never used before a set of media players that we chose.
We gathered performance measures and user experience self-reports, and of
course we got UX_Mate output from the video data.
What we found is that there were significant correlations between the data we
extracted and the number of errors. We identified which were the relevant
motion units which were mostly related to the mouth corners and the eyebrows,
as expected. There was a weaker relation between subjective evaluation and
the number of errors. And it looked like the, the questionnaires were biased
by the last task which was the most challenging. Also there was a strong
inter-individual variability.
What we also did is that we took one-third of all the face videos that we
recorded and we gave it to three observers and asked them to do UX_Mate’s job
to see if she could say, “Okay, the guy is experiencing some difficulties”.
And what turned out is that two of them were kind of random. One was
performing well so we asked her what, do you have any specific training on
the facts systems on the Ekman coding system. And it turns out she had. So
we were like, “Okay, this might work”, because I mean if you were random as
well it does make sense that we try to make a system that approximates that
stuff. But these actually gave us the motivation to further investigate the
problems.
So we designed an evaluation study with twice as many students. So we got 31
undergrad’s, we used 3 academic social networking websites because we didn’t
have the resources to build the interfaces. And so those were Academia,
[indiscernible], and ResearchGate.
And we introduced the calibration tasks. So we designed two tasks in order
to get basically the training data for building our models. And while both
the screen action of the user and the synchronized face video have been
released so they are online if someone wants to work on it.
So this is an example of the calibration faces. So basically the guy would
be randomly, the subjects would be randomly shown this sequence; either first
that one or first that one. The order would be random. And here following
[indiscernible] you can see that the wild one is the cluttered one over
there. So it’s more difficult to find a double room at the Holiday Inn in
Bradley in that set up, rather than in this set up where everything is like,
you know, visually organized. So we used these in order to gather our
training set.
This is an example of what happened to the guys. So that’s what the guy is
watching. So the non-cluttered one, that’s, you know --. So there is
something there. And actually when we checked the numbers we had we found
that, so these motion units are the ones related to eyebrows and to the mouth
corners, those are the compound ones that I talked about before, the
frustration and confusion one.
So you can see that in the difficult condition there are higher activation
levels of the facial motion; so we ran, we built our models. We tried to
predict using all of the calibration phase data. We tried to predict the
actual experience of the user on the 3 networking websites.
The results were not so great, even though not either too bad for a first
run. So we plan to actually improve UX_Mate and this can happen, for example,
in trying to fix better calibration tasks more suited for what you want to
detect on the real, on the real tasks. Probably we should extend these to
other interaction feelings, for example engagement during video games might
be an option.
And probably new compounded variables need to be identified in order to
extend it. So after all this is probably your face now. So the outline is
that we present a cheap, non-invasive way to try to get the automatic user
experience assessment. And we released the database and hopefully somebody
will use it. So, that’s it now.
>>: Do you feel that there is room for improvement in terms of the actual,
like, mark-up of facial features? I mean do you think that tool is good
enough? And I am sorry, I like missed a little bit of how you developed that
tool or what that would look like.
>> Jacopo Staiano: Okay, so I think there is --. I mean the tool does a good
job, but right now I think that in that field you have a lot of tools that do
actually probably also a better job. So, I mean from a research point of
view I wouldn’t go improving that tool as so much researching it.
>>: Yeah, I mean even just like looking at the video for example, it looked
like it wasn’t looking that much at the forehead where it looks like a lot of
these kind of confusion wrinkles are. So, yeah I don’t know. What we are
looking at here, how do you pull the actual number out from --?
>> Jacopo Staiano: So what the number represents is that at each frame you
would have a displacement which gives you the direction and magnitude of a
movement of a certain facial feature point with respect to any
[indiscernible] point, which is the moment before the calibration task there
was a cross in the frame, at least there at me. The screen should --. But
yeah, I mean this system is not perfect, but it works reasonably well for out
purposes. It’s true that there are a bunch of other systems or commercial
systems or I mean I think of the ones that Google has, Microsoft has,
Facebook has. They are for sure 10 times better than this one, but we don’t
have access to them so.
>>: Have you considered like how this thing would play out in like more
natural settings instead of like, this is fairly artificial. You bring
people in --.
>> Jacopo Staiano: No, this is --.
but --.
Okay some of them are running in offices,
>>: [indiscernible].
>> Jacopo Staiano: Oh, you mean [indiscernible] here, the guy.
>>: I mean it seems, I don’t know, it seems like there aren’t much
distractions. The person is really like doing what they are doing right?
>> Jacopo Staiano: Oh, yes.
>>: But in more natural settings it might be more complicated where you are
in a social situation and there are marked different types of signals that
you are exposed to which --.
>> Jacopo Staiano: Yeah, more noise.
>>: Yeah, which --. I mean detecting emotion from facial features and those
types of environments. Do you have any idea as to how --?
>> Jacopo Staiano: Well the thing is that here the, the whole point of this
is to have some evaluation of a new interface prototype that I have. So I
would coax people in and say, “Hey”.
With what we foresee is that with a system like this, which can run on your
laptop, I can just give you access to my interface prototype and tell you,
“Hey do it at home when you have 5 minutes”. You would do it; of course, if
you started drinking your coffee and other stuff we will get random results,
but yeah. It’s the trade off with the, you know, ecological ability here and
--.
>>: Right.
>>: How much does it depend on how --?
top of the monitor?
I mean if I moved the camera to the
>> Jacopo Staiano: The camera is on top of the monitor.
>>: Oh, is it on the top or --?
>> Jacopo Staiano: Yeah, this is [[indiscernible] like this.
embedded camera.
So you have the
>>: The people are really having bad movement set-up.
>>: Yeah, I mean it seems like that. How sensitive is it too --?
well I guess all the [indiscernible] cameras over there, but --.
I mean,
>> Jacopo Staiano: You mean on the position of the camera?
>>: Yeah, yeah.
>> Jacopo Staiano: Well it’s, I mean it’s --. I would say like you have at
least a moment. If you talk about a tail I think you have a kind of --.
>>: What about if I like move this piece?
Is it [indiscernible].
>> Jacopo Staiano: Yeah, I mean it gets up to I would say like this distance,
like 1 meter and a half from the camera should be fine. And the problem is
with the partial occlusions like if you start to do this. I mean the
tracking would still be there. It is [indiscernible], but you should filter
them out because you don’t need it.
>>: So I am just wondering do people actually show a lot of expressions
during these tasks or?
>> Jacopo Staiano: It’s low intensity.
It’s precious.
>>: So I am just wondering is this worth spending effort into facial
expression analysis when people are, I mean there are other things that you
can do right? You can look at maybe their keyboard movements, maybe some
physiology. That might be an easier signal than some things, just a thought?
>> Jacopo Staiano: Yeah, yeah, probably, but I
an [indiscernible] professor and she was like,
usually done with 2MG sensors, one on the head
they measure”? They measure the activation of
measure because if it’s not visually you can’t
mean the idea was coming from
“Look arrangement studies are
and one on this and what do
those, which we can’t really
get it through the webcam.
>>: Actually the psychology literature there is a strong volume work around
the facial features and that emotion, like [indiscernible] Ekman, you sited
that [indiscernible]. But I think that you are right, that was the precomputer used times, so it’s not clear how those things play out in today’s
information world.
>>: So maybe if you had like those badges. So when people are interacting
with each other it seems that people would be more expressive with their
facial expression because they are trying to communicate a certain thing to a
different person. So in those kinds of situations these might be more
explicit.
>> Jacopo Staiano: So for example, so for this, so for this study. So you
have both the videos of the guy and you have the video of the screen of what
the guy is doing because that’s what annotators used later to say, “Hey it
didn’t take the right part to upload the photo over there”. But of course, I
mean, you could devise your interface prototype or whatever, in order to log
if the guy was doing a wrong action. Then you wouldn’t need facial
expression, you wouldn’t need to be a self contained story.
>>: Have you thought about just running this over time with what works and
logging how they felt throughout their --? So I know that I get passively
frustrated, I don’t realize that I am frustrated when I am working, and it
would be really interesting to go back see that I was frustrated when I was
writing this e-mail, or like when this person texted me, or something like
that.
>> Jacopo Staiano: All right.
>>: Something like that?
>> Jacopo Staiano: I mean, no.
>>: So self reflection.
>>: Yeah, yeah.
>> Jacopo Staiano: Do you think you would find people to participate in
experiments like that?
>>: Yeah, I think people would be interested in doing it for themselves.
Because I think in the social setting you have a more loudly expressed
facial, you know, communication. But I think on your own you have a more
honest one, because there is no one there so you probably --.
>> Jacopo Staiano: Right, you are less biased.
>>: Yeah.
>>: I have a feeling that when I am on my own, when I don’t like have --.
But I may be wrong; it might be like a subconscious thing.
>>: Yeah, feel that way and then when someone walks in I am like, “Oh I must
adjust my face”.
>>: There is a guy who did like an art project where he constantly took
pictures of himself, and he got in a lot of trouble because he took pictures
of, he got all the laptops at the Apple store in Manhattan to record everyone
and he got in a lot of trouble, but he said he wanted to do that project
first because he recorded himself and he found that he didn’t have any change
in facial expression. So that kind of goes along with what you were
thinking, that people don’t really change their facial expression that much.
But, it’s one example.
>>: Okay, let’s thank Jacopo again.
>> Jacopo Staiano: Okay, thank you.