>> Andrew Begel: All right. Well, thanks for coming, everybody. I'd like to introduce our
speaker, Jorge Aranda, who is currently a postdoc at University of Victoria working with Daniela
Damian and Peggy Storey at the same time. And he was a Ph.D. student at University of
Toronto until a year ago and has been a postdoc for one year.
And he also has the wonderful résumé of having worked for Gina Venolia right here at Microsoft
Research where he did some pretty influential work on studying some exciting things about what
software teams misremember. And he currently works in the areas of looking at collaboration
and coordination amongst the members of software teams.
So I'd like to introduce Jorge who's going to talk today about a study he did on understanding
roles in software organizations.
>> Jorge Aranda: Thank you. So hello, everyone, and thanks for coming. I'd like to start with a
picture that -- well, Andy has seen this before, probably you have, too, because it's been making
the rounds online, how different people in IT see each other.
So, I mean, this is of course in jest, but you can kind of see some of the stereotypes that people
assign to different kinds of roles in software organizations, right, how they see themselves,
always the heroes in all cases but in different ways, and how everyone else is screwing them up
in their own way.
So the study that I'm going to talk about is based on the observation that I made a while back;
that we have these terms in our literature everywhere, both in academia and in industry, we talk
about developers and designers and project or product managers, QA, et cetera.
But in the firms that I've studied and the groups that I've been to, these people apparently are
doing different things than what I often see in the literature.
So I'll give you a sample of the things. These are phrases published a year or two ago in the
academic literature. And some of them kind of adhere to what I was expecting to find from the
concept that people have on roles in software organizations, but some do not.
So, for instance, the first one someone says: Nonfunctional requirements are one of the hardest
problems analysts and designers have to deal with during software design.
And I agree that software designers have to be thinking of nonfunctional requirements all the
time; that it's just that I've never seen them doing it purposefully, right, to say now I'm going to
work on nonfunctional requirements.
Another on a similar vein: Developers identify and manage work dependencies. Again, it's
something that I guess they need to be doing in their daily work, but it's weird for me to see it
spelled out like that because I've never seen a developer discuss dependencies specifically.
Right?
A weirder one: If a software architect has to model a concern not supported by his own language
or tool. This is something that I've never seen. I've never seen a software architect modeling
concerns at all, no matter what the tool. Actually, I haven't seen anybody modeling concerns.
And just a few more. Users hardly get any support in creating process models. Okay. That's an
understanding of what a user does that does not match my own.
Our proposal provides software designers quantitative feedback they can use to meet a coverage
goal. So software designers I think in here is being used as a term to refer to software
professionals in general and how they want testing coverage, but a lot of people have associated
a label designer with something very different than, you know, establishing the quality of a piece
of code. And so on. It'll skip the last one.
The point is that in general reading the literature on software engineering, I get this feeling that
we're talking about different things; that people talk about a developer or a tester or a project
manager with a meaning very different from the one that I have and the one that I've seen.
So I said, well, I might be wrong too, right? Maybe my conception of what a software developer
is or what a tester is is very inaccurate.
So I wondered, you know, would it be a good idea to study what are the actual roles in software
development organizations and do we even have a consensus. Are there common,
well-established roles in the industry, or do they vary a lot across organizations or even within an
organization? Do roles change over time in a group, and, if so, why? What are the reasons that
people change their responsibilities? Do people in different roles use the same tools and do they
use them in a similar way? And also where do official positions fit in the picture, right? Is your
title directly related to what you're actually doing or not?
And so I started doing this study without really knowing if this was a problem that needed to be
fixed. Maybe it's not. Before starting, I hadn't heard complaints from anyone that said oh, well,
our distribution of labor is completely ambiguous and this is the influx of problems.
And what I didn't want to do was to end this study with a proposal for some kind of
standardization or certification or something like that. If you want to become a software tester,
these are the things that you should be doing. And if you want to be a project manager, these are
the things that you should be doing.
But I think there are good reasons to study roles in software organizations, mainly because they
are related in several ways to real concerns that managers and software professionals have. So
what are the right ratios of different kinds of professionals in a group, for instance. Let's say
developers and testers.
Oh, by the way, you can feel free to interrupt me anytime you want.
Often company policy is very detached from what's actually happening. How can we bring it
closer together so that what's stated in the formal policy makes sense in practice.
>>: Seems like actual dynamics. Does that ->> Jorge Aranda: Oh, but more consistent with what's actually happening in a team. So to give
an example, let's say that the policy states that requirements are written in the form of a
requirements document written by an analyst and passed on to designers and developers, and in
actual practice maybe there's -- a lot of people bypass that process.
So, you know, it's important to -- if you care a lot about having policy that's consistent to
practice, something needs to be changed there. So that's what I mean.
And also from an academic perspective, it's good to have some conceptual clarity, right? We're
talking a lot about these concepts. Almost every abstract in TSE or ICSI or FSE, you know,
mentions one kind of software professional. So it would be good at least to, you know, have a
better understanding of what are we talking about.
So the first thing I did was to go back to sociological literature and organizational science
literature to see how they conceive of roles in social groups. It's an old topic for them, right?
And the consensus seems to be that a role is -- or what we call a role is an abstraction that we use
to refer to a set of expectations placed on someone.
Okay. The keyword here is expectations. So role is kind of like at the molecule level, but the
expectations are the atoms that really constitute it.
So whatever a person's expected to do by virtue of their position, their status, their history, their
preference, all of that defines the role.
This means that roles are social constructs. There is -- a developer is whatever the relevant
social group determines that a developer should be, right?
So a researcher at Microsoft Research is whatever the group decides, perhaps unconsciously or
unintentionally, that a researcher should be.
This also means that there is no prototypical anything. There's no prototypical researcher or
developer or tester or anything because these concepts are defined at the group level.
And who places these expectations? Well, everyone. Teammates. So, for instance, you have
someone who is a developer. What does being a developer mean? Well, it means what the
teammates think that a developer should be doing, what their supervisors and maybe
subordinates think that they should be doing, people they interact with, also people they don't
interact with but with whom they have some kind of relationship.
So people who -- let's say Microsoft. You could have lots of developers who are unknown to
another division, another department, salespeople or whoever. Salespeople don't -- haven't
interacted with any developer in particular, but they are expecting something to happen in the
development group and that expectation is transferred into the actual developers.
Society in general also places some expectations. And of course the person who is enacting the
role also has some expectations on what is it I should be doing given that my social group is
giving me this role.
So there are some problems that come out of this conceptualization. The first one is that the set
of expectations of all of these people is probably not consistent or satisfiable even, right? I get
some expectations from my bosses and from my peers and from other people, and there my be no
way to make everything satisfiable.
Sociologists call this role strain; that you're being stressed by the different pulls that you have on
you. And it may lead to this function. So people are expecting you to do something, you can't
satisfy that something, and so there is a consistent mismatch between the expectations placed on
you and your behavior.
The other interesting thing is that since these expectations arise from there group that you're a
part of, that means that there's no guarantee that they match those expectations placed on
someone with a similar role somewhere else.
Now, in some professions, the roles and expectations are much more uniformly understood. So,
for instance, in the medical domain, we can talk of doctors and nurses and anesthesiologists and
surgeons. And just naming them, you can have a bunch of expectations of what is it that they
should be doing, what constitutes normal behavior and abnormal behavior from them.
But in our field I claim that the expectations are not that uniformly understood; that there's a lot
of variation. And we're going to see a few reasons why. Yes.
>>: That's what I wanted to ask before, is it seems like certain professions are more amorphous
and that's why if you can have these questions [inaudible] medical field or I think of like factory
workers [inaudible] there's not much leeway in what their roles are.
>> Jorge Aranda: Uh-huh.
>>: So have you found other professions that are more similar to software [inaudible] that you
are going to talk about?
>> Jorge Aranda: Yeah, that's a good question. I didn't specifically look for others that are
similar, but I can come up with some because I think this amorphousness -- is that a word? -comes from a number of factors.
I'm going to be going over some of those factors, but briefly, to anticipate that the main driver
for uniformity is institutionalization. Right? So the more institutionalized some role is in the
larger culture, the more that every instantiation of that role will fit that picture.
So if we have -- for instance, if we demanded that every software developer had a license or a
certification or something, you would expect to see a lot more uniformity. But I'll be going in
more detail on this.
Right. So the other wrinkle here is that we need to make a distinction between roles and
positions. A position or a title is what the business card reads. And sometimes they're the same
thing. You could be a software developer in practice, you know, you could be developing
production code, and at the same time have a business card that says software developer.
But these things don't necessarily match. And we -- so sociologists make a distinction between,
one, a position you occupy, a role you enact. Position is based on a form of policies and a role is
based on the norms of behavior in the social group. So just something to keep in mind. I'll come
back to this.
Woody Allen here is the president of some banana republic. And he has the official position of
president, but his role was slightly different, if you saw the movie.
Right. So that was a brief overview of the theoretical framework that we sketched out coming
from the sociological literature.
Then what we decided to do -- by the way, this is work that I did in collaboration with Adrian
Schröter and Daniela Damian and Peggy Storey. We decided to do a field study to test out this
theoretical framework, and I tried to modify a little bit if observations didn't match our
expectations.
So it was a case study of two organizations. And I need to tell you a little bit about both
organizations, because a lot of the examples I'm going to give in a few minutes use information
from those organizations.
So they're both small, fairly small, especially for Microsoft's numbers they're pretty small, about
50 people the first one and 25 the second one.
The 50 people one, which I'll call company A, is -- it does software for bioinformatics. It has a
development team of 17 people, mostly developers, some testers. This group is subdivided in
development teams, and they have about 30-something folks doing other kinds of stuff: sales,
field support, project management, product management, they have like five VPs.
This company is -- was -- by the time I went to spend time with them -- I spent three weeks with
them -- they were going over a rough patch. They had a downsizing two weeks before I arrived.
They lost I think about 15, 20 people. But it was the second downsizing they had gone through,
so things weren't going super for them. It was interesting for us because we could see, you
know, how responsibilities were changing as people were adapting to their new situation.
The second group, which I'll call company B -- although it's not really a company; it's a publicly
funded group -- is part of a research institute. And they're tasked with providing data and data
analysis services to scientists all over Canada and, because they do it over the Internet and the
Internet has no real boundaries, to scientists all over the world.
So they receive data from a bunch of places and they process it on site and they develop software
to help people access it, mine it, store versions of it, et cetera.
So it's a smaller group, and it's subdivided in three parts. We have the development side,
software developers, which are about eight people, if I remember correctly; they have scientists,
a bunch of scientists, who also do a little bit of development, mostly scripts; and they have
operations, an operations group, which is charged with providing data and services and ensuring
that they have a good uptime and that everyone's data is safe and so on.
All right. So I spent in total five weeks at both places. I conducted 36 interviews of about an
hour each. That comes to about 50 percent of the total staff at both sites.
And so we ended up with a bunch of stuff, a lot of examples that cannot be summarized very
easily. But we also came up with this table, which I know you won't be able to read. I'm sorry
about that. I couldn't come up with a better way to explain it.
So what is this? This is -- we got a lot of information from people when I asked them: Okay,
what is it that you should be doing here? What are you expected to do? If you go, what's going
to happen to the organization? What do you expect from other people, et cetera, right?
And so then I analyzed all that and I came up with this list of expectations that people have. And
the original list was much longer, but some of the expectations were too detailed, and so I
removed them because I was looking for something more universal to share.
And I also clustered the expectations by area. So you have the technical core. The first few
rows are technical core, which means actual development of software: fixing bugs, maintaining
legacy code, and script, scripts that -- or customizations that you need to do on the base systems.
And then in the columns in here you have a bunch of people which I mostly classify by either
positions more than by the roles. Although I made a couple of exceptions. For instance, the
second company had -- did not have official team leads, but two people were really doing more
team leading stuff, so I put them over here.
In each cluster in here, you can see a small subdivision. That means company 1 and company 2.
And each of these squares is colored by how much people expect that person to do something.
Right? So white means we don't expect you to do this, light gray means -- or gray means you're
somewhat expected to do this, and black means you're highly expected to do this.
Now, this is subjective. This comes from interview data, my interpretation of that interview
data. But I'll go back to this, because I think this could be the basis for some more objective
analysis or at least more of a self-reported analysis of expectations in software groups.
Anyway, a few patterns that I think are interesting here -- yep.
>>: Are these, those people you interviewed, perceptions of expectations of what other people
thought that they should do, or does it also combine what they thought other people should be
doing?
>> Jorge Aranda: Yeah, both. Both of them. Um-hmm. Now, mind you, this is only the people
that I talked to. If I included everyone in both organizations, the table would be twice as long.
But I had less information for the people that I didn't talk to, so I left those out. This is a subset
of all the people in both organizations.
>>: You tried to get -- are there any like [inaudible] positions that you didn't have coverage of?
>> Jorge Aranda: No. So I tried talking to people in every group. Yeah. Yeah, I did. Well, one
exception. There's a significant sales team in company 1, and I didn't talk to any of the
salespeople. And there's a sys admin at company 1 as well. I didn't talk to the sys admin. But,
other than that, I covered all the groups. Yep.
>>: One other question about when you asked them about expectations of others, were they
referring to individual people, like I expect John to be a developer or to write code as opposed to
all developers, whoever they are?
>> Jorge Aranda: Yeah. So I was looking for the first kind of response, John should be doing
this. I often got -- I often first got the second kind of response, developers should do this. And if
that was the case, then I would push them and say, okay, who in development should be doing
this? And that's what I went with. Yeah.
>>: So I noticed that you mixed QA and ops, and you also mentioned sys admins as maybe
being a subset of in some cases ops. What was the -- what was your reasoning for combining
those?
>> Jorge Aranda: Yeah. A practical reason, actually, which is that I only had one person from
ops. And looking at what that person was doing, it had a lot to do with -- I'll explain that in a
moment -- but a lot to do with quality assurance as well, so I just lumped that person together
with the others.
>>: And so really -- I know it's a small sample size, but really it sounds like there wasn't a lot of
operations and you had that specific discipline in the two companies.
>> Jorge Aranda: That's right. So the one company that had operations, an operations group,
there were four people in that group. The information you see here is from the head of that
group, which is the person I interviewed.
>>: Seems interesting that [inaudible] by the places that you get most uniformity and most
nonuniformity.
>> Jorge Aranda: Right. Exactly. So a few of the interesting patterns that I came up with, the
first one is this cluster here around the technical core and how you can spread out of working on
the technical core in two main ways. You can either spread out the way that the tech leads do by
designing system architecture, taking the toughest problems and providing some historical
perspective. You kind of become a bit of a team historian. Or the other one, the path of the team
leads, which is more towards management, reporting what the team is doing and so on. So that's
one.
Another one is you can't really see it very well in the table, but there are very few people that
have expectations placed in the technical side, in the management side, and in providing some
domain legitimacy or being domain experts.
Let's see if I can find -- so this one, for instance, this person works a lot with directors in
determining the long-term vision for the products, provides domain legitimacy to the
organization -- I'll explain what that means in a moment -- also is working coordinating product
release, writing documentation, and so on. So that person is all over the place. Those people
were very rare. But in the interviews, other people constantly refer to them as key for the
organization to keep on keep on going.
For one of those, I think it's this person, who is, again, all over the map, someone said, well, if
that person leaves, we may just, you know, close the doors and turn off the lights because there's
nothing else we can be doing here.
>>: I was -- actually I was referring to -- under maintain legacy code and the developers, the
larger organization varies from black to white [inaudible] advocate for resolutions of
user/customer problems, it's all black or gray for everybody in the support -- you know, it's
either ->> Jorge Aranda: It's true.
>>: -- [inaudible] variance.
>> Jorge Aranda: Now, I should warn you not to read too much into the actual details, because
this is only half of the organizations. Right? But you're right. And I can explain this thing here.
In that large organization, remember there's several subteams. One team was charged with the
main product that they were working on, which was kind of old and difficult to work with, and
so a lot of what they were doing was just solving bugs and working on the roadmap features that
they had.
The company also wanted to move away from that, and so they created this splinter group that
shouldn't be concerned at all with legacy code whatsoever and they had supposedly [inaudible].
So that's a reason why [inaudible].
>>: Questions were about specific people and not the role [inaudible].
>> Jorge Aranda: Yes. Right. Right. Yeah.
And just one more pattern. There were some expectations that are very weakly spread all
throughout the organization. So, for instance, writing low-level or unit test code. A lot of people
have that expectation weakly. It's like everyone says, oh, yeah, well, we should write unit tests.
We should all ensure that our code is fine. Or documentation. Yes. We're all in change of
documentation. But nobody's really charged with it. It's just everyone expects everyone else to
play nice and do something with this. Yeah.
>>: When you asked about expectations, did you ask also how significant that expectation was
for that person or for that role, whether that person was meeting their expectations?
>> Jorge Aranda: Yeah, so -- yes, I did, both. Although not always systematically, but -- so,
sorry, the first part is whether ->>: Is like what you were saying is how important is that expectation to that person, like is that a
significant part of the their job, or is that ->> Jorge Aranda: Yeah, we spent a lot of time doing this, and ->>: [inaudible] didn't seem particularly important to them.
>> Jorge Aranda: Yeah. I should also add that I also spent time observing what they were
doing. And so I interview someone and they say, oh, yeah we should unit test. And then I talk
to the QA person and I say do you have a set of unit tests to start with? No, of course not,
nobody's writing them, and that kind of thing. So we also -- as usual, I usually get conflicting
accounts, and it's a bit hard to determine where reality is.
Yeah, anyway, as you see, there are some patterns. And you can kind of classify a little bit of
what people are doing based on their expectations. But there's also a lot of role variation, so
order-solving detectives that everyone does it their own way. So just like we have that with
them, it was difficult for me to find people with the same roles in significant detail in the
organizations that I studied.
And I wanted to give you a couple of examples of that and an important exception. So in
company 1 there were three people that were doing product management stuff. Everyone called
them product managers. But -- and I talked to all three of them, and they were doing it very
differently, each of them.
So the first one was mostly doing actually product design. He was concerned with a little bit of
the architecture, talking a lot with tech leads and so on.
The second one, or a different team -- so each of them working different teams -- the second one
was mostly working on, you know, a documentation way of doing product management, trying
to specify the architecture, the APIs of the systems, writing the documentation himself and so on.
And the third one had a more external facing kind of product management. So what he did -remember, this is a bioinformatics company. He had a Ph.D. in bioinformatics and was doing a
lot of talking with clients, figuring out what they wanted and trying to bring that into the
development.
So three product managers; three very different ways of doing it.
The same thing happened with team leads. Everyone had different kind of sets of expectations
of what they should be doing as team leads. With developers, some people had to do more
documentation, some people had to do more requirements. And design, some people were just
expected to take this description and produce code that matches the specification.
The one important exception to this role variation finding was project managers at the first
company, at company 1. I talked to two of them, and I observed the third one at work. And, by
the way, there were just three. But all three were doing pretty much the exact same thing. The
variations were there but were minute.
And when I talked to them and asked why are you, you know -- I talked to them late in the study
and I was a bit surprised to see such uniformity. I asked them why is that the case. And they
said that it was intentional and part of it comes from a certification.
So all of these are project management professionals, PNPs, so they have their certification, they
have their body of knowledge and their processes. And they were actively trying to enforce their
own interpretation of what a project manager should be doing in the organization into everyone
else. So this is the way we're going to work, here are our policies, and get on with the program.
Right?
So that was an interesting exception to this role variation. Everyone else was kind of like
mucking around, but project managers were fixed on an idea of what they should be doing.
Okay. So I'll go back one slide. An interesting question for us was what makes some people in
here take some expectations, why are expectations clustered in ways that they are and not in
others. And so we came up with four kinds of factors that make certain people attract some
kinds of expectations or some expectations and not others.
The first kind of factor is institutional. This is what I was talking about earlier. This is the case
of the project managers, for instance. There is a relatively strong institution of what a project
manager should be doing, if you're trained like that. You come into an organization, you're told
you're a project manager and you know what to do. Right?
But the same thing happens to some degree with other roles. So, for instance, a tester. You
know, if you're hired into a company as a tester, even without anybody telling you anything, you
say, well, I mean, I guess I should be writing some tests, right? I should be checking the quality
of the code that people pass on to me. And when other people hear that you're a tester, they
expect certain behaviors from you.
So this comes from the larger culture from our institutions of what a software organization is
like. I have an example of this coming from company 1. For several years the company was just
chugging along as a startup, growing kind of organically, it hit about 15 to 20 people, and then
they acquired a lot of venture capital.
After acquiring venture capital, they had to change their ways. The venture capitalists expected
this. So they had to put in a CEO, a bunch of VPs. They needed a sales group, because every
self-respecting software company should have a sales group. They needed to have project
managers, a product management department, QA department, resource -- human resources
department.
So all of this structure just came into the organization externally. It's just what a company like
this was expected to be doing by the venture capitalists.
This probably was not the wisest move for them, I don't know, but it is an example of how the
larger culture affects what you're doing internally. Yes.
>>: Did either of the two companies have training programs for new hires? Like if you hire
somebody as a tester, you teach them what it is to be a tester in your organization [inaudible]?
>> Jorge Aranda: No, to a very limited extent. The only ones that have that are developers, and
the training is done by other developers. For instance, they have a week with problems to get
you started. Right? So, you know, the first thing you needed to do is to be able to build our
system. Here are the instructions.
The first day you're -- or, whatever, wherever -- whichever day you're supposed to do that, then
after that make these small modifications, see what happens, and so on. Some teams have that
better structured than others.
A second kind of factor that affects how expectations are clustered is historical factors. So, you
know, like a tree, you bend it some way; if it doesn't die, it just keeps on growing that way. Or
like, for instance, you have a patch of lawn, people start to walk over it in the same ways and
suddenly you have a path. And one day someone decides to pave that path; it becomes official.
So that's the thrust of what I'm talking about here.
The expectations that people have in a group partly come from what they used to be doing before
or other people enacting the role used to be doing before. For instance, at company 1, there was
-- when there was all this expansion, this VC-funded expansion going on, there was a guy that
they hired for the QA lead position and later moved on to become a program manager.
When the layoffs came in, they gave him a choice to either go out or become a tester, a QA
analyst. He decided to take the latter. But at the same time this guy knew a lot of what was
going on at a higher level in the organization.
He was still -- while I was there, even though he was a QA analyst, he was invited and
participated in positions that did not correspond to those of the other QAs. He was still attending
some management meetings, he still had some contacts that the company was exploiting in some
ways. In short, people were expecting from him things because of what had happened in the
past, not because of his position or institutional version of what he should be doing.
A third set of factors, structural. So if you're a slave in this line, you don't have much choice on
what you can do and the expectations put on you are pretty simple, just keep on pulling. But if
you're in some other point in the structure of the organization, you may have much more leeway
and ability to influence and to interact with others in the organization in some ways.
So an example of this in the second company I studied, in the second group, the head of
operations that I mentioned before, he had a unique position where he was the contact of the
whole group with the computing center, with the head of the software group, with the head of the
science group, with the database person, with all of the people below him.
And so that gave him more ability to do some things, but it also placed on him a lot of
expectations coming from many different ways. These expectations, just to repeat, come from
his position in the organizational structure, not from anything else.
And, finally, personal factors. Everyone brings something different to the role that they enact,
right? So it could be that you're not very good at doing what you're expected to be doing. It
could be that you're very proactive and doing far more than what is expected of you. And that
performance, that diversion of your performance, against your original expectations changes
what people expect from you.
The second company, there was this guy, a project administrator. He was -- this was really a
clerk position. He was supposedly just in charge of keeping track of the payroll and the budget
of the projects and so on. But he -- I mean, the job was not satisfying for him like that, so he
started taking on more stuff.
And the rest of the people got used to that, got used to him knowing all the terms and knowing
what was the status of the bugs and all of that and keeping track of all of it.
When he took some vacation time -- not vacation. When he took a leave to fulfill another
position in the group, the project administration role was crumbling down and everybody was -- I
guess everybody was expecting stuff to happen from the replacement of this guy that was not
happening.
By the time I was spending time at that company, they were redrafting the description of the
position so that the personal factors, in this case, the characteristics that this person brought into
the role, became official policy after a few years.
Right. So earlier I talked about the difference between roles and positions. The -- we found both
in overlap in some ways between what a person is expected to do by the group and what the
position is but also some mismatch. And we came up with an explanation for this.
Your position, your title, represents your placement in the organization structure, so it's a
structural factor. And it also defines to a large extent the institutional expectations placed on
you.
So you are a project manager. Okay. You're placed in a certain position in the organization to
satisfy the expectations of a project manager, so structure, and your title represents -- you know,
you should be managing projects, so you should be doing certain kinds of things.
But your position does not define anything -- does not have to do with the historical and the
personal factors that I also talked about. So a lot of the distinction between what your role is in
practice and what your official position is comes from the difference between what you bring to
the organization and what the formal policy of the organization is.
By the way, this is -- I don't know if you recognize him, this is from The Wire, a TV show, and
it's full of people that diverge from their positions trying to do some other role that was not
formally placed on them, and it doesn't always end well for them.
All right. So we also asked ourselves what makes a role evolve over time. And there are many
things, but we focused on two. The first one is organizational growth. As an organization
grows, sociologists tell us and our data confirms this, you have more specialization. And it is
easy to see why it is with work in practice.
So in the smallest possible company where you have just one person doing everything, that
person's role includes all the expectations of the organization. But as that person keeps hiring
other people, the expectations are more spread out, more specialized. And by the time when you
reach a larger group, you have a much heavier specialization, a much more detailed
specialization than before. So that's one of the things that drives role evolution.
The other one, I briefly alluded to it before, is the mismatch between expectations and behavior.
So either through a failure to meet the expectations placed on you or through contributing more
than what's expected from you, the other people adapt, right? They learn to expect the new
normal over time.
I gave you several examples of this already, but just one more. The head of the science group in
the second company that I was talking about was hired as a scientist just to do whatever it is that
scientists do but started to get involved in software development side of things without knowing
anything about it at the start.
By the time that I spent time with his company, he was attending all the meetings for the
development group, chipping in into what a product should be doing, how it should be behaving,
what do the user community think about it. And he became a very important voice within the
group, just thanks to his own initiative. Now people expect this from him. If he leaves, they will
miss that heavily.
One more finding and we're done. We also looked at what kinds of media people used based on
the role that they had and how they -- did they use it.
A problem for this observation was that, you know, these are small companies, completely
collocated, so a lot of the interaction was face to face. But moving beyond the face to face, what
kinds of artifacts people used to communicate, well, the broadest medium used in both
organizations was the issue tracking system. Okay.
But its use was not uniform. So developers and QA people used it a lot to have a conversation
going through the issue tracking system. But other people also interacted a lot in -- or took a lot
out of it in different leads.
So, for instance, team leads were interested in the aggregate information, what's the overall
activity going on. They get information from the issue tracking system even though they don't
put a lot in it.
Product managers use it to keep track of the general progress of particular kinds of features. The
director of software development in both places uses it to get a very high-level view of things.
He uses it more as a dashboard than anything else. Project managers use it. So, you know,
people all over use issue tracking systems.
But if you just look at who is communicating through it, you miss a lot of the people who are
listening or taking the aggregate activity out of it.
We were interested in this because of the earlier research project that Andy mentioned when he
introduced me, the one we did here with Gina Venolia. And we speculate that perhaps part of
the reason why sometimes software repositories are incomplete or misleading is that they're just
capturing one kind of role, interaction between just a few roles and missing all the other people
that are part of the conversation but in a more passive sense.
Other media, the use of other media were more localized. So, for instance, chat, IRC, was used
with some development teams but project managers didn't use it. The customer support people
used Google Docs to keep track of cases from their customers. Nobody else had access to these
documents. E-mail and phone were more prevalent for managers. Developers rarely used either,
especially phone.
Okay. So so what? So I went over our theoretical framework to make sense of roles in software
organizations. I argued that looking just at the role level may be misdirection; that we need to
look at the actual expectations placed on people and to distinguish between what a role is and
what a position is.
And I think at the very least this should give us better foundations to talk about different kinds of
software professionals.
And even though we're just beginning to do this, I think it would help us to design better studies
about organizational structure and organizational design.
For instance, remember that table I showed you earlier, we could devise a survey or some study
based on something like that, have it self-reported instead of derived from interviews to make
data collection easier, and from several people you could get a three-dimensional matrix of what
people expect from each other and what they think people expect from them and start to analyze
misunderstandings in placement of expectations within the organization.
So something like this I think could lead to better role definitions. It could help to study
bottlenecks and efficiencies in the organizational structure, what kind of expectations are just
placed on one or two people, and we lose those people, you know, the company is in trouble and
how to fix that.
Yeah. And finally some brief observations. Role-based differences in media use. So the fact
that people in different roles use different kinds of media or use them in different ways. Could
point to systematic omissions in repository mining. So that's something to pursue later on.
If in your organization you're concerned about role specialization, you're right to be concerned.
And it's important to realize that if you want a team of generalists, even though you're perhaps
having someone specialized briefly in something and then come back to the general group, the
way that Azure proponents advocate, that short-term specialization, unless you actively try to
fight it, could lead into a more hard-to-break long-term specialization [inaudible].
And finally if we just take the labels of the roles, if we just talk about project manager or
developer or product manager, that may obscure the true contributions that people bring into
the -- into their teams. If you're just a project administrator, you may not -- other people may not
realize all the other stuff that you're bringing into the project.
And I think I will end there. If there's any other questions, I will be very happy to take them.
>>: So do you have any plans or aware of any additional research? I mean, obviously, you
know, it's good to start somewhere, but varying sizes of organizations, larger organizations,
because I think that's -- it's a good start, but what's interesting to me is, as I talk to some
colleagues, or some people within Microsoft, that especially the generalization would be that
there definitely tends to be a movement -- well, I shouldn't say definitely because it's anecdotal,
but to me it [inaudible] to be a movement towards trying to generalize, or at least there's more
[inaudible] roles.
>> Jorge Aranda: Right.
>>: And it's starting to break down some of the walls.
>> Jorge Aranda: Yeah.
>>: You mentioned not just the -- the last point you made about even just the role contributions
maybe not being realized, but they also -- the potential contribution may also be constrained by
the role as well, right?
>> Jorge Aranda: That's true. Well, yeah, so to answer your question, definitely a study in a
larger organization is something that we would like to pursue.
>>: And also other domains, because, you know, [inaudible] huge and that may -- there may be
domain-specific things about roles that we don't understand yet as well.
>> Jorge Aranda: Yeah. So in the short term, in the fairly short term, if all goes as planned,
we're going to do something similar with a particular group at a very specialized -- in a very
specialized domain in a large company at General Motors. We're also pursuing other
replications of this in other sites.
What I should point out is the theoretical framework, if we got it right, it should still apply. But
it would still be interesting to see the variations in different settings.
The big one is we expect to see more specialization in larger organizations, even though you try
to fight it. Yes.
>>: Do you see your organization anywhere else [inaudible] where expectations were like
codified, like written down anywhere? Or did it really appear that people just kind of got
expectations from other people in that same role and other people in the company, just kind of
inferred what their expectations were?
>>: To a very large extent in both sites there was no formal -- no correct formal representation
of what the expectations are. The second site, remember, is publicly funded. So the position has
a description for everyone. That description often does not match what they're actually doing.
In the first site I got several people -- for instance, one of the product managers I was talking
about, the one that has a Ph.D. in bioinformatics, he says, well, I was hired in as a project
manager. I didn't know what a product manager does and nobody told me. I thought, he says,
that what a product manager should do is to, you know, be in touch with users and with the user
community and kind of tell people inside what would work and what wouldn't work.
He was frustrated and a bit confused by the amount of control he was expected to have of the
direction of the product. Like he should be saying work on this, not on that, instead of I think
maybe this is a nice avenue to pursue.
With time he learned to adapt his own behavior to the expectations that people placed on him.
And the role became more similar. His role became more similar to that of the other product
managers in terms of defining what the team is going to do. But this came from trial and error in
his case and in many others in these groups.
>>: So at the beginning you posed this question is there really a problem here. In your
interviews, did it -- did any people express like a desire to have a better understanding about their
own roles and expectations on them? Or was it like, yeah, it's kind of fuzzy, but, you know,
we're fine as -- you know, it doesn't need it?
>> Jorge Aranda: Yeah. Great question. I think -- so if I remember correctly, nobody said I
would like a better definition of what I should be doing now, although I would have liked it
before when I was figuring it out.
I think in general people like fuzziness for themselves because it's more convenient, right? I
mean, I didn't know I was supposed to be doing this, so you can't blame me, right?
So in a sense, you know, it helps you to have the fuzziness. It doesn't help you when other
people have it. And I did hear complaints that, you know ->>: [inaudible] other people have that about you?
>> Jorge Aranda: No. So what I got a lot of was stuff like, especially in the first company, these
folks should be doing this, you know, X, and they're not doing it. All the time we have to go in
and do it ourselves. That's not the way it should be. Right?
So I'm just hypothesizing here, but people, it would appear, do not like concreteness on their own
descriptions but would appreciate it on other people's descriptions.
>>: And the other thing that wasn't covered in this study is almost the opposite when you say
there's a downsizing, a very small, but let's say there's another round of VC funding, all of a
sudden, you know, company A wanted to double their size or their engineering team. And then
again, like your first point of is there a problem, well, which roles do I hire, what's the ratio of
people that I hire. And I think that's where you might get this, you know, where that fuzziness
can help, can possibly help.
>> Jorge Aranda: Yeah, that's right. Yes. Yes.
>>: Did you take these results back to the companies? And, if so, what was their reaction?
>> Jorge Aranda: I am taking them next week. I'll tell you then. So I have a presentation with
the first company on Monday.
>>: Find out if they believe you.
>> Jorge Aranda: Yeah. So informally the one -- the only person I talked this -- I talked about
this with was the director of software development. Nothing here [inaudible] him. But we'll see
about the rest of the organization. I'll report back.
>> Andrew Begel: All right. Well, let's thank our speaker.
>> Jorge Aranda: Okay. Thank you.
[applause]