>> Tom Zimmermann: So welcome everyone. It's my pleasure to introduce
Filippo Lanubile from University of Bari in Italy. And he's actually one of Software
Engineering Innovation Foundation of [inaudible] last year. And his project is
about social GFS to make GFS more social. And today he will tell us about his
progress.
>> Filippo Lanubile: Thank you, Tom, thank you. So Tom has already
introduced me. And I can start -- thank you from the second slide because Tom
was saying as here is written. I'm from University of Bari. So your question
might be where is Bari?
Many people in the US don't know it. So Bari is here. We are in South Italy on
the Adriatic Sea, on the heel of Italy. It's a myself place. I can tell you that you
might have a chance to visit Bari if you attend the international conference on
global software engineering in 2013. It will be organized in Bari.
Okay. The presentation today I will start looking at the relationship between
software engineering and collaborative supported cooperative work.
Then I will talk about the impact of distance and collaboration.
I will introduce and look in the concept of group awareness. And that is
something that proposed in CSCW and is much interest for software engineers.
I will provide this little summary of collaborative development environments and
compare them with respect to how they support the group awareness. Especially
the social awareness that is the kind of awareness I'm going to talk about.
And then I would propose -- I will present our -- what was our proposal
augmenting social awareness in collaborative development environments,
specifically the Team Foundation Server.
And then I will conclude. And there might be probably I'm even for a demo.
Okay. I start saying it's looking at software development is an intense
collaborative process. We actually had here during this week the workshop on
the future of collaborative software development. And probably the first one was
talking about software engineering as a collaborative, an intense collaborative
process was Frederick Brooks in the Mythical Man-Month, almost 40 years ago.
It was saying that a software development is an exercise in complex
interrelationships.
We have different types of collaboration within our team. We have
communication which is an exchange of information between members of a
team. Then we have a coordination, which is the orchestration of task so that
there are no holes and no repetition and all the task are -- go to achieve the
same goal. And we have control, which is the process to add to other policies,
guidelines, and can be done formally with standard formal meetings, also
informally as peer pressure.
And computer-supported cooperative work, we have the conference here this
week that finished just yesterday emerged from human computer interaction to
address the problems of collaboration in work groups because there were the
communication breakdowns. There were problems in coordination. There was
the lack of knowledge about activity of colleagues.
And this is something that actually -- actually it's the same problems that faced by
software developers in large scale teams. And this is especially true for global
distributed projects where the distance makes collaboration more difficult.
This explain -- having the same problems explain why there are many
researchers that work at the intersection of CSCW and software engineering
from the CSCW point of view, software engineering becomes a domain, a
specific domain of work. So we have the -- you have this conference there was
one session about software engineering, at least for each time slot.
And those we found in the swear engineering conferences, we found people that
look at problems of -- that from CSCW point of view.
About to explain -- to introduce the problem of distance when people
collaborating with either team they use this Dilbert strip where we have a
program manager who is explaining to his software engineering that there are
outsourcing their half of their programming work to Elbonia, an imaginary country
in the world, so they can take advantage of time difference, even cuts if the cost.
And the managers explain how it will be easy to do this. We will hand our
requirements at the end of our workday and get back the finished code the next
morning. And then we have a couple of Elbonians somewhere in the world. One
is say reading the e-mails from the manager. One again, I have no idea what
they want. And the other said let's pretend we died. And so that's -- because
that's -- that is the worst thing that can happen when you collaborate on a global
software project.
There are different dimensions of distance. We have a -- we can consider
geographical distance that is usually what we think about when we talk about
distance. People that are dispersed in different sides. And actually it can be
measured as the cost or effort to move across side to have physical meetings.
Sometime it can be very costly.
We have temporal distance that is a distance of temporal dispassion. That is the
effect of working at different time zones. Also can also be the effect of living in
different places with different work patterns. For example, in Italy, in South Italy
we do lunch break at one or two P.M. And in North Italy, people do -- make
lunch break at noon.
And so the intersection to organize meetings becomes reduced even if you work
on the same time zone.
Then we have also the sociocultural distance, which is a measure of how difficult
it is to understand the behavior of others and what they are saying.
And actually it also includes the linguistic differences that sometimes can be a
big problem. I was making the classification because there was a paper on a
communication of the ACM that was classifying the problems of working as a
distance, considering the three dimension of collaboration, communication,
coordination, and control, and the three dimensions of distance, geographical,
temporal and sociocultural. Actually in the original paper, the table was more
rich. This is simplified version.
And what and I mean here to highlight that what I'm -- we are addressing here
are the problems that we -- in coordination when there is -- people work distance
on a geographical basis. And the main problems are the lack of awareness and
the reduced trust. That what we -- I will now talk about.
Since talking about awareness, let's see what is awareness, generally group
awareness. This is a fundamental concept that was defined by Dourish and
Bellotti actually 20 years ago. Actually it was celebrated, this paper, as a seminal
paper on this conference.
And they find group awareness is the understanding of the activities of others
which provides a context for your own activity.
It is important because they make the individual contribution relevant to the
group. And this relevant awareness information, whatever the task domain, it is
-- basically awareness works with to kind of actions, the monitoring and the
displaying action. Everybody monitor what the others are doing. If it's relevant
for what you are doing.
And at the same time they let their action to be displayed so that the others can
be -- can monitor you. And some year later Carl Gutwin and others, they find a
classification for awareness four dimensions. They said that group awareness
could be the -- that is the information that provide awareness can be classified as
informal awareness that is the information about who is around, who is available?
It's very common now with instance messaging because it's the basic feature that
instance messaging introduced after the IRC chart. And then we have the
group-structural awareness that is information about who does what in a group,
which are the tasks, which are the roles.
Then we have the work space awareness, that is information about who is
creating and changing the artifacts which are shared in the group.
And at the end we have the social awareness, that what we are interested in this
presentation, that is the -- social awareness is the information that we maintain
about -- we know about otherwise while we are talking with them or in a social or
conversational context.
If I'm talking with something completely new to me, my social awareness is zero.
I don't know nothing. If I'm talking with Tom, I know that Tom -- what Tom is
interested to, who are his friends. And this can be useful. At least makes -- can
make the conversation better and also the -- and more general the collaboration.
What I will now present is a short service of collaborate development
environments and how they support group awareness and I will say specifically
social awareness.
Here I included some of these collaborative development environments, including
both the forges and tools that you can install.
The name collaborative development environments comes from a paper from
Grady Booch and Brown almost 10 years ago. He was talking about that the IDE
extended. He was asking to extend the IDE with collaborative features so that
programmers should not switch back and forth between different applications to
accomplish their work.
And then the first collaborative development environments were proposed for
open source software projects but then there have been so many others.
Sometimes they are called Application Lifecycle Management Platforms. The
names can change.
Basically what they are, project workspace with a tool set that usually include a
tracker, version controller, to communicate some way to make the right
documentation, dashboards, mechanism for event notification.
And this can usually -- it can work also as an enrichment of the IDE. I can
present just some event trackers became popular some year ago especially
because was the first one from what I know that introduced the Wiki to maintain,
to create and maintain documentation. And this was something considered new
at the time. And it was also one of the first one to introduce also the RSS feeds
for event notification. And Google called this popular, but it's nothing special. I
mean, very basic features. It's very simple to use, but nothing that probably is
the simplicity that makes it success.
And we have Code Plex from Microsoft. When Code Plex -- I don't know if this is
the first one, but I noted something new as web based collaborative development
environments was the ability to follow projects that is kind of awareness that you
can have, so that you can have if you -- it's a way to -- it's an alternative way to
RSS feeds for event notification. But you have the events of the projects that you
are follow -- following in a common activity stream which is part of your home of
your -- of your home page in the Code Plex.
We have also assembler. We put highlighted in bold some features that are
borrowed from social software that have been introduced in this environment, for
example here you can -- with Assembla you can follow not just projects but you
can follow specific software artifacts. You can follow a work item. So that can be
more specific.
And then you have an activity stream. Another thing new that other than RSS
feeds for event notification, you can also have even notification on Twitter using a
plug-in.
Here is the IBM Rational Team Concert which has been up on a platform called
Jazz. Actually Jazz was initially proposed as to extend the eclipse. So the IDE
with the presence awareness synchronous communication. Then it became
much more. It became an environment, a full environment. But at the beginning,
this was Jazz, specifically for presence awareness. And something borrowed
from social software that's been introduced in Rational Team Concert is the
ability to tag work items, to add freaking words as a way of class -- over as a free
classification like you can do in deletions for example.
And actually there has been work in the software engineering by Margaret Story
and Chris Strue [phonetic][inaudible] I'm not sure to pronounce well. Maybe not.
And where they have followed how developers in IBM were using tags. And they
were using tags in work items not just for as a classification way, but they were
using as a form of indirect communication to see what was important about the
work item and -- or not.
You have your Team Foundation Server. And I would say it doesn't add anything
to what we have seen before, at least from what I know. There is the SharePoint
based dashboards, but we have features that -- there is a -- I've never used. I
know that this is a plug-in for presence awareness and synchronous
communication and you can use MSN or Skype. I never tried it.
And finally we have GitHub that is I will say it was developed as the now forge for
projects using Git as a decentralized version controlled system. But what is
really new about GitHub is how many different ways you can use the follow
features. And you can really develop a social network of programmers. Because
you can -- okay, follow projects, but you can also follow specific developers,
follow a developer was something used and it means that I can see what
developer working on different projects do or the events in my timeline.
And I can also see what are the connection of other developers. Which
developers are following other developers? And this something is also has been
studied for the first time, the first report about the usage of this features have
been reported on this conference just at this conference. It was a presentation
both at the workshop on collaborative software development. There was
organized by Andrew Beagle [inaudible] and there was also a paper on CSCW
conference by -- from Carnegie Mellon [inaudible] and somebody else that were
-- they performed the empirical investigation of how developers use these follow
features within GitHub. It's made -- they perform 25 interviews, and they -- and
they provide at the end lessons learned and guidelines that can be -- could be
useful for other environments that want to borrow these kind of features.
That from what I know was the first study about the -- these features. Specific on
GitHub.
Summarizing I -- here I'm including the collaborative development environments
that I presented. And looking at how the different kind of awareness are
supported. And we can see that group structural awareness and work space
awareness are supported different degrees but they are usually -- you
understand that these environments have been developed for this, for the group
structure and the work space awareness.
As the result informal awareness or presence awareness we have Rational Team
Concert and the Team Foundation Server that's supported. But as regards the
social awareness we have just the GitHub that partially support social
awareness. I am remind you begin that social awareness is the information we
know about other people. What they do and what -- how -- and what -- who they
know.
And this I say partially because it's partially done using the feature follow
developers activities in GitHub. So the question now that might -- this might be -might be something that is missing or actually also might be maybe social
awareness is not worth to be supported. And so what I'm trying to do now is
introduce some empirical studies on these past years that support our -- our
hypotheses that social awareness can be -- can benefit the developer's work.
So the first empirical study was presented at CHI on the last year by Costa,
Marcello Cataldo, Cleidson de Souza. This was -- an empirical study conducted
over three different companies, big companies and five large projects. And what
studying their repositories, doing interviews, and what they conclude that 25
percent of coordination of new coordination requirements, so people which are
asked to coordinate with somebody else because they are working at common
work items involve project members that don't often work together. That don't
know each other or they have worked with more than six month ago.
So they conclude in this paper that supporting the sharing of personal and
contextual information will increase the likelihood of successful information.
Because working with unknown people, it's common.
Another empirical study here is the controlled experiment to support the idea that
social awareness is needed. It's an experimental study performed 10 years ago
where in a laboratory setting, and the people were asked to collaborate at a
distance. And here -- and performing some task. The tasks were not specific of
software development. And what was observed that people -- being as a
distance it was affecting negatively the willingness to initially cooperate. It was a
kind of play to do. It was negatively affecting the ability to persuade the other
partner.
And there was also a tendency for dissection of that pattern and people tending
to be better of what they actually are. But this effect was decreasing because
has interactions where repeating. After -- while people were interacting, this
negative effect could be -- continue to stay or could decrease, depending on
whether the people were sent, they were living closer or not. Actually, this was
an artificial setting, so people were in the same place. But they living in two
different cities, very far.
Another group was said they were living in the same city. People living in the
same city, but they were not, but they were not working in the same place had a
feeling of being close, and this had trust building effect. Just feeling to be close,
not really be close.
The third empirical study that I am -- that we are presenting is was performed in a
global company, probably IBM. Is not said but probably it was IBM because
there was an expertise search tool was introduced in this global company. As
this spreadsheet search tool produced a list of experts that could be contacted.
And from this list of expert people I choose which one to contact. And when
studying the reasons because developers were choosing somebody in this
expert list, they were choosing on the basis of two reasons. One was the
participation of the expert to be contacted in the -- in forums or blogs because
they had the feeling that we were more likely to respond. How much they were
actively participating in social software.
And those -- another reason those are easier to guess is how they were closed
social because they directly know them or indirectly knowing and this because
can avoid the cold calls that our field -- you can feel embarrassed to contact
somebody you have never met or you don't know somebody and you can say,
you know, I know Tom. Tom told me that I could ask you something.
Finally, this is the last one, last study that support the thesis that social
awareness can be needed. This study is about how professional software
engineer were using and internal social network that was introduced in IBM. And
when interviewed, people were saying that they were used the internal social
networks to build stronger bonds with their weak ties or to reach colleagues that
they didn't know.
And so not -- the connection were being created for the -- to support the 25
percent of coordination requirements that we -- I presented before. And also
there were different motivation to establish new connection, this social networks.
Not just gathering support for project activities but also to decide to connect
professionally or even to connect personally by sharing pictures or personal
events.
So this is our studies that they've reported that provide motivation to say that
augmenting collaborative development environment with social awareness
information can be worth. And specifically our goal is to facilitate the
establishment of a on our personal connections by discovering personal interests
of developers or the competencies collecting from social media this information.
Here I show the hypothesis in a visual way. Our hypothesis is collecting
information from social media networks can work as a surrogate of the informal
communication that you exchange in a cafeteria or in pub, in a restaurant, in the
same way as remote conferencing is used to replace the physical meeting room
for formal meetings.
So I will say that this is the lack of informal communication where the informal
communication also provides information that is -- that tends to build trust is what
we -- what we want to fix.
So this is SocialTFS. SocialTFS is the tool that we have develop from -- with
using the award from the software engineering innovation award. And it is an
aggregator of developer's content collected from social media to put into the
developer's work space, specifically the developer's work space is Team
Foundation Server in the visual studio. We have developed it as an extension of
Visual Studio the GUI, and we have built a social proxy which is the aggregator.
We collect all the services from the social networks, microblogs and also collect
information from the server side of Visual Studio, the Team Foundation Server.
I will -- I present how some screenshot to show how it appears. For example
here this is Visual Studio and SocialTFS appears as a view within social studio.
Here the story that is a user story as a developer I want to share my cost with
other developers. What I do as a developer is choosing which services I want to
open. For example, this is my profile. I have opened information About Code
Plex, Facebook, LinkedIn, Twitter. And there are other that are -- I'm not open
like Yammer or StatusNet. These are professional microblogs.
And you can see here okay there is a number of posts that I've provided a
number of people that I'm following and I'm a number of people that I'm following
and I'm followed by. And actually this information is managed by on the server
side by the social proxy. It has two purposes. One is to store the user's
credentials on behalf of the user but without going the passwords. Because
fortunately now there are authorization protocols like O out that we use. And
also other than storing user's credentials, also posts are cached. This is useful to
avoid -- to provide good performance so that -- and I think it worked with well.
Here we can see now with the currents users that are active. The number of -they are summarized because this is the server side. In that main panel is this.
It's not for the developers.
This regards privacy concerns. The user has full control on totally which services
he wants to open, but also on which information want to show. For example here
if I click on one of the -- one of the servers, but in this case it was Facebook. I
can choose which information I want to provide. For example, if I want to open
the Facebook, but just to provide information about my friends, but not to provide
information about my post because I can think they are two person, domestic
that.
So this is what appear when I'm in the Visual Studio interface. And these are the
posts. The posts here appear in the view. And you can see these are the people
I'm following. They are part of the team. And this is an aggregator. So I can see
from where the posts are coming from.
So now, the question is who to follow? What are the people I'm following? And
we have two option. One is a classic option. I follow people that I decide to
follow. This up to me. Make the decision like it's usually done on a social
network. And this -- so this, the home timeline that we provide, because we have
different timelines in SocialTFS. Home timeline is the people that I have decided
to follow. Can be suggested. But basically it's something that I say yes, I want to
follow this at its static.
And second option, it's dynamic kind of follow. And it's -- it takes the inspiration
by a paper which appeared just last year and by the Cleidson de Souza and
David Redmiles. It's named the awareness network. To whom should I display
my action and whose action should I monitor that addressed the problem of the
awareness network that is the people I should be -- I should follow, I should be
concerned of.
I was looking for -- sorry.
>>: No worries. It happens. There you go.
>> Filippo Lanubile: Thank you. Sorry for the interruption. So they were arguing
in this paper that the awareness network is not something in -- the software
project is not something that is the same from the beginning to the end. But it
changes continuously. Because they say it depends on what are your
assignments and so which work item, if we talk about work item? Which work
items have been assigned to you.
And you might be -- you might be concerned about people who have some -something in common with the work items you have been assigned or that you
have commanded in some way. And here what we call the iteration timeline. I
will come back. This is something very dynamic because changes for every
iteration.
And another way to change the awareness network is to have something -- an
awareness network that change depending on the what you are watching on the
-- on Visual Studio. If you are watching a source code file or a work item. And
the awareness network might change because that -- your interest is changing.
I will present you this timeline. The first one is the static kind of timeline what we
call home timeline where the following depends on whom I decide to follow.
Here are the people in my awareness network. And they actually are the people
that end up being registered to -- being registered. And I can -- I can distinguish
between the people I'm following, the people that are following -- that I follow, the
people that are following me, and the people that are suggested. We see there
is a little recommender system that provides suggestion.
I'm currently following just Fabio Calefato here. And so in home time, right, I will
read just his posts. I was saying that we have a recommender system for this.
Very simple, actually. Because suggestions are from people who are in -- taken
from people if you are in the same projects or you are in the same TFS collection
or people that are following you in one of the social networks that have been
subscribed. Or if you are a follower of those persons. So the suggestions are
modeled and are ranked based on the number of links to those people. For
example, here -- we have here four nodes which are the users. And this is me.
And I'm on the same project of -- with Fabio and with Nicola. And I'm following --
I'm following Fabio and Giuseppe on Twitter. And I'm following -- no, Fabio and
Giuseppe are following me on Twitter. And I'm following Fabio on Twitter.
And this is a weighted motigraph. You computed the number of weights of the
links that go to the same node. And then you get the rank of the suggestion list.
Weights can be changed by the administrator on the server side. Because here
we have considered that being on the same project in the Code Plex or TFS can
be -- can be more important than being a follower in some social network. But
this can change.
And this is iteration timeline that we're following I will say and depends on work
items in which a developer is involved in the current iteration. That means that
we took the active and fixed work items and look for which work item, which
people has been involved has been assigned or have commanded and those
work items.
Or -- and this is the other way of having a dynamic awareness network which is
the interactive timeline that is depending on which tub is open in the main editor
space, I will have a different -- different people as in the timeline depending on
who have changed the source code file or who has changed or commented work
item. And if I switch between the different tubs, this changes.
And this kick because the posts are cached so I don't need to go far. Okay.
Before presenting the features. All the project is part of the Code Plex, so it's
open source. You can see both the code and also the work items because we
used -- we have -- we use the assignments of work items. This we are -- we
have some documentation that lists important -- how to install. Because there is
the installation there is the client, which is the extension of Visual Studio and
there are the server side to install the proxy.
And some -- you will find it. We have also used the code contract, something
that I -- yes, we have discovered in Paris that were presented so we used them.
And these are -- these are the people. This is small team. It's me and post-doc
Fabio Calefato and two master of -- two master of science students, Giuseppe
Santoro and Nicola Sanitate was the picture as an avatar in the tool but not in
Code Plex. And -- these are the conclusions, just to summarize.
The research question is how do we strengthen or build trust among developers
who are part of global software teams who have few or no chances at all to
meet?
And the hypothesis that the information shared on social networks if collected
can work as a surrogate of the social awareness which we gained during the
informal meetings in the real life.
And the SocialTFS does this work and aggregates it in social information into
Team Foundation Servers. It is read for demo and use.
And as future work, we think that we will evolve SocialTFS to their actions. One
is that we -- is to integrate the social information with other types of information,
which are related, for example, to work space awareness. Because that is some
-- we have not mixed yet. And we have -- but you cannot just mix. You have to
think about how to -- one kind of information could be used together with others.
And also another thing that we can do is mobile version of SocialTFS because,
for example, except for the interactive timeline, the home timeline and the
iteration timeline might be used on Windows Phone and so you can maybe while
sitting at the your desk you are not looking at the information -- social information
about your colleagues. But while you are in the Metro, you might do that. So it
can be different encompassed.
But most important that you want to -- we have a hypothesis. But we have to test
the hypothesis so that we want to measure the actual benefit of being a person
connected to other developers in the workplace.
For doing this, unfortunately it's not something that we can do with university
students. I mean, We need the large project with the real developers and that
use Microsoft tools like working with Microsoft development environment.
So this is one of the expectation I have from this talk to raise some interest about
this because these are the first place where -- that I can think about. Of course.
You can guide me about this.
I think this was the last slide here. I have the tool -- we have -- how -- time is
finished. Okay. Time is -- I have the tools. Since I have -- yes. We have some
meeting with some of you. I can continue show some -- okay. This is -- I will
show you.
Okay. These were the people. For example if I want to accept a suggestion I
can -- I can see, for example, before accepting a suggestion what kind of -- what
is doing. I can also view it for example with the skills about people. They are
taken from LinkedIn. Just because LinkedIn provides skill.
And if I decide to follow him, okay, then I have the iteration timeline. That -- as I
was saying, it changes according to the active effects of the -- and here you see
this is the interaction -- that was the iteration time. And this interaction timeline.
If I open a file, it will be filled. And if I open -- basically information is taken from
here.
And if I come back to the source code file, they switch. And if I open another -- I
mean, of course -- I mean, it does -- here are the services that I have opened.
Here about Code Plex is if I want to suggest the people that I'm in -- I won't be
part of the suggestion of being in the same projects. The properties changes
according -- okay. And that -- okay. I can also hide people if I don't want to see
-- not to match, have to show because -- have to be simple and not be
complicated. The problem is now to understand if -- of course while working if I
find this annoying I can also close it. I can reopen. Of course I can reduce it.
There's a part of the features of the IDE. It's a bigger view. I can arrange it as I
prefer.
Okay. This I have nothing else to add about the demo. And so if you have
questions.
>>: So ->>: I'm sorry. Do you want to -[applause].
>>: So [inaudible] question I have is -- I mean, you're putting this in a
development environment where presumably the person, when they're seeing
that, is doing some type of task, debugging or code or whatever. But [inaudible]
comment are actually not related to development at all, at least in your particular
context.
Do you -- do you envision that the purpose of this is to become aware, just to get
to know who you're working with, like, so you know, I've never met this guy, but I
-- looks like he likes to go to the converts for music and has a girlfriend who's
pregnant or is it more like you expect people will be writing like I fixed this major
bug in this thing today, it was really tough, so you have an idea of what they're
doing on development side of things.
>> Filippo Lanubile: Interesting questions. Currently this is about social
awareness. There are not -- no interaction with other kind of awareness, for
example, work space awareness of the ->>: Sure, yeah.
>> Filippo Lanubile: Of the group structure awareness. We can think about this,
make the different kind of awareness be dependent. For example, I was talking
about the developers, but actually in the -- in the Visual Studio are not just the
developers. There's a program. They are the tester, there is a project.
Currently, I mean, so the prospective can be broadened and currently we are
able to include all of them, of course. But the rules are not seen.
>>: Yeah.
>> Filippo Lanubile: But may be.
>>: But I could see the benefits of [inaudible] perspective. Because just knowing
what people are up to in their lives [inaudible] can create like a closer relationship
like this is a real person and it would be interesting to know if people's reaction to
what they post differs when they note that people are following them within an
IDE as well.
Like, would you post other information -- so if we're not close on Facebook, right,
I know who's looking at me and what context, so that kind of dictates to what I
write to some degree.
>> Filippo Lanubile: No, this also why we -- having done this, there are so many
things that we might add. But I think that we are at the time that we also need
the feedback on the actual usage before deciding which kind of direction or which
-- what it's worth to add.
Now, you are saying it's speculation of what I might be useful to add. What we
would like is to get this feedback from users that have tried this and saying look, I
have used this but I will -- I will need to do something more or to know also this.
>>: Yeah. So I wasn't actually necessarily saying you should add something to
it, but just like I think it would be just watch how people use it.
>> Filippo Lanubile: Yeah. Yeah.
>>: And what they are able to learn from it?
>> Filippo Lanubile: Yes. Because I can imagine as an empirical study giving to
people who want to use it and then have questions and interviews structure
interviews to gather information about some specific questions that you are
looking for about -- to discover patterns.
>> Tom Zimmermann: Let's thank Filippo again.
[applause].
>> Filippo Lanubile: Thank you.