Programmers a-voiding “complexity” Steinar Kristoffersen University of Oslo, Institute for Informatics, Norway Abstract. It seems taken for granted that programming is a complicated activity dealing with complex phenomena. Much of software engineering is devoted to reducing complexity. Unfortunately, there is no indication that this methodological response has been successful. This paper provides an ethnomethodologically informed ethnographic study of programmers ‘as-andwhen’ they ‘do’ programming. It is shown that programming is a conversationally oriented activity, in which complexity is inconspicuously dealt in a ‘seen-but-unnoticed’ fashion, as “claming back slack” in a plan, “transferring agency and ‘eidos’ to code” and “justifying implemented requirements”. Thus, this paper represents a first step towards re-conceptualizing programming in itself as a “co-operative system” in which ‘complexity’ is handled in a way of which the current software engineering approaches have had no awareness. On top of this, new methods for realizing co-operative systems can be devised. Keywords: Complexity, programming, ethnomethodology, ethnography, epistopics. 1. Introduction Information Systems (IS) research on topics such as project management[1], design[2], and programming[3], methodologies[4] and use[5] render the notion of complexity absolutely central. It is used for many different purposes, but most often it is either used to “scope” the work as being concerned with non-trivial issues[6], or it is used as a “strategic” approach, i.e., the problem (whatever that might be) is ostensibly resolved by reducing complexity[7]. Complexity is idiomatically everywhere in IS research, but the small study that is reported in this paper shows that it is not a conversational topic in its own right, in a community of programmers working on what would elsewhere (and rightly so) be called complex projects. Methods aiming to manage and reduce complexity are valiantly proposed as the solution to improve the somewhat dismal state-ofthe-art[8], and there is, arguably, no reason to believe that such methods are not needed. However, a “software crisis[9]” might be aggravated by methods that tend to disregard the fundamental role of the everyday world of the agents, users, designers, and managers, and the messiness and situatedness of their acting, […] In this way, they key element is neglected: human existence[10]. Does this mean that the beleaguered domains are too “complex” as such, to be managed by methods? Probably not; however, the lack of success on behalf of the methods that IS has previously suggested deal with complexity, indicates that a deeper enquiry into that notion is required. Arguably, it might be exactly because one has departed from an a priori and decontextualized understanding of complexity (from within the system sciences mainly), and used that as a template for analytical approaches to the work of programmers, that methods today are considered by many to be so problematic[10]. Ciborra’s methodological response to this problem of methods− “System design methods may be the most diffused tools […], but they work only in part (ibid., p. 13)”− is inspiring: Let us go back to the basics and encounter the world as it presents itself in our everyday experience. We rely on evidence, intuition and empathy. We listen to managers and we participate in their dealings with puzzles and riddles, and, on the other hand, we do not confer any particular relevance on works like ‘strategy’, ‘processes’, ‘data’, ‘system’ or ‘structured methodology’ (ibid. p. 25). This is the exact objective of this paper: To look closely at the ways in which issues that usually would be described as complex, are handled by programmers whilst programming. The aim is to “unpack” the concept of complexity. The rationale is that if complexity indeed is used too abstractly to describe many different concerns by IS, then one should not be surprised that a professional enterprise (such as IS) that is oriented towards resolving complexity, is not consistently successful. There seems to be, simply, something wrong with the concept of complexity. Is it a “red herring”, underspecified or much too general? 2. Using ethnomethodology This is an ethnographic study, carried out through participant observation during a sustained period of more than one year by (in average two days per week), with the author working as a programmer in a related project in the very same room as the programmers that were the subjects of the study. The documentation provided in this paper is based on transcriptions from mini-disc recordings of ordinary conversations among this community of programmers. The subjects knew about the study and its purposes. They were aware that their conversations might be recorded, but they did not know exactly when. This paper takes an ethnomethodological approach to the analysis of the data[11-14], by which it is meant that, from the outset, abstract concepts (such as complexity) and the relationships between them are not seen as “proceeding” action in such a way that they can, in a stable way, constitute a (social) order that is “given”, and that action (however voluntarily) can be seen as stipulating the behavior of members of that social order. This is also the case for “complexity”. Summarizing briefly to the policies of ethnomethodology, naturally occurring speech is indexical, which means that terms are fundamentally contextual and cannot be specified in advance, or in any “theoretical” way. However, in the exact circumstances of their use, such words are instantly useful, and operationalized by the members. This is according to Garfinkel recognizable as a practical achievement whose qualities make the production of social order possible[15]. Indexicality implicates that the only reasonable approach to learn about a concept is through empirical investigation of its usage in everyday practices, and yet the concept of complexity which IS so often tries to “deploy” as a motivation and scoping of its methods, seems to have maintained its stability and “meaning” through theoretical reflection rather than empirical inspection[16]. This paper addresses this shortcoming, and has therefore come to rely on a method that has the capability to “unpack” and put on display a social setting as it is perceived, accounted for and enacted by those involved in that setting by the bona-fide member of the community that is being studied. Ethnomethodology is explicitly not concerned with creating big theories about external “normative” structures that (may or may not) stipulate behavior; rather, it sees such “theories” as part-and-parcel of the production of social order which all members of a community is engaged in through and for the sake of “getting the job done”. Thus, the concept of complexity and its qualities are no more a phenomenon that is “enforced upon programmers” from the outside, than it is an aspect of “the account of programmers work” as and when they work. Unless programmers “practice” complexity in their work, the ethnomethodological analysis will contribute to us turning away from complexity (as an almost metaphysical construct) and instead force us to focus on the concepts that members of the community themselves are using. Second, it might seem a laborious approach to discovering the commonsensical aspects of ordinary work. This is true, of course, but it is at the same time a fundamental part of this method: I take it that a lot of the results I offer, people can see for themselves. […] And they needn’t figure that the results are wrong because they can see them[17]. It is a central methodological underpinning of this paper that if we (as members of a sufficiently-skilled community of researchers) see things (through the eyes of members that are programmers) as they are instead of as “complex” (if indeed they don’t), then the knowledge about the alternatives will be helpful in analytical terms, a well as when and if new methods are being invented. 3. Fieldwork and results The research background of this paper is a company called Branova. It is a software development and consultancy company. Their business idea is to provide software solutions for the airline industry. The core competence of Branova is integrating “ticketless” traveling and loyalty solutions with backend legacy systems, supporting industry standards and processes. The company’s employees have several years of experience in product development and software engineering from leading airline companies. Past experience among Branova’s staff as well as consultants has proven itself, and their solutions are used throughout by one midsized company. It is now marketed globally. Airline ticketing and loyalty solutions are, fundamentally, systems for creating and maintaining, distributing, marketing and selling all sorts of airline tickets in every imaginable way. It has, moreover, become a way for airline companies to reduce their operational cost and increase customer loyalty by introducing new business concepts. There has been increased competition about smaller market shares for airline companies after “9/11”; many companies have had to shut down and the survivors have been through radical transformations with regards to their product portfolio. Portfolio transformations, tweaking and redesigning their offering to the market, and changing their market profile is now the order of the day. Airline products are intractably “complex” for a number of reasons, and the continuous and rapid changes to that they undergo are another source of “complexity”. The notion of tickets as “too complex to compare” in terms of pricing is well known, and this is aggravated further by also having to consider the associated deployment of materials and crew, as well as the strategic and tactical implications that the airline company has too take into account on their side; making sure that loyal full-fare customer are not alienated by competitive pricing of tickets for new segments of travelers. The direction of the fieldwork of this setting thus became: When the system changes regularly and significantly, how do programmers in this community “work” and talk their way through such changes? If and how is “complexity” co-constructed in various ways—as intricate, obscure, unfortunate, difficult, involved, complicated, multi-layered, may-faceted, unclear, etc.? The following excerpt1 shows a discussion relating to the reorganization of text in various classes, i.e., the refactoring of code into smaller, more manageable modules in order to make these classes more maintainable: 1: Can (.) you have to make a little bit smaller [], little bit smaller classes, not so big classes, perhaps split it a little, that will make it easier 2: [Smaller] How do you think that I am going to achieve that? Ehh. (..) Yes. Ehh. It is not going to be smaller. It is still going to be 21 pages of text. Maybe that is a little much, right? It depends what you are comparing it to. It provokes a rather harshly made remark, perhaps innocently phrased, but with an emphasis on that which makes it critical. 1: [Hmm?] More? I hope that was a joke, [if it] was 10 files of 50 lines each that would be easier that one of 500 2: [Yes, but, yes but] there are some reasons for that,…, first this The tension is “defused”, rapidly, and the first speaker “backs off”. 1: [Okay, that] is okay. You don’t have to,… However, the second speaker keeps on “making excuses”. 2: It is really a flow and it should really be partitioned, but I am not allowed to do that bookinghandling and flowhandling, I need to that in advance and the saleshandling we have to do elsewhere 1 The subjects are not native English speakers. This is not why the English is not “correct”. In situ, the subjects spoke their native language, of course. The translations of the excerpts reflect the “incorrectness,” grammatically speaking, of the native conversation. The objective was to translate it into English that is correct “enough” to be understandable, whilst keeping the authenticity and original “flavour” of the exchanges. It ends up becoming an account of the architecture that is currently in place, and how the obtainable factoring is related to the overall organizational requirements of this particular part of the system. This is certainly not an “alignment” of the IT system to the requirements, since the architecture stays put; on the other hand the architecture is not random, either. Code is designed for the purpose of “being good code”, and then the justification is reframed in terms of a requirement specification afterwards. This closes this part of the argument as well. The second talker then switches topic, and starts talking about deploying the code (these are EJB (Enterprise Java Beans) that are compiled and tested locally, and then deployed to a shared application server for more extensive testing in the group). 1: (Coughing) Is it okay by you that I do deployment in half an hour’s time? 2: Yes, eh, but (sighs) then you must test this class here, then, you are on some cases now, right. 1: Yes, but I won’t be able to do that properly, it is only the EPL2 it does not matter if that is out The warning about a new deployment brings to the attention of both parties the need for further testing, which in its next turn allows them to draw a clear boundary between the work that is relevant right now with regards to the ongoing changes, and that which is not. It is a “rescoping” taking place that also, in the next instance, indicates which modules are not finished; it describes their status as “unknown”. It is, in a sense, “activating slack”, in other words, making it clear that although it was scheduled to be done already, it makes no harm that it is not. 2: The consequence of that merge there is that we don’t really what is happening Saying that “we don’t really know what happens” dissociates the code from any particular person and person’s responsibility, since it does not say that the code is not finished, or badly written, just that, in a sense, we don’t know exactly what it does. Interestingly also, and there are many examples, this is usually coupled with giving “it”, i.e., the code, agency of its own so that with no explicit disregard for the author it is legitimate to not know exactly how to solve the problems at hand, for now. 1: I gather there are not that many of us right now doing EPL-testing so it probably does not matter if,… (coughing) The following excerpt show similar patterns of talk: 1: I was busy with that production problem that we had in Malmö last week and I had to do something, (ehh), build a cache and something, so that it wouldn’t,… 2: Ok, right. 1: That is why no-show fee was a little bit shoddy, and when I go in there to look now, and there’s a bit more to do now than I though, (ehh), we never got time to sync together, and he is using a booking class per journey, for instance and I mean what is that, I mean, it is per trip, right? So 2: Did you get that reply that I sent to you? 1: You mean that “super-duper”? 2: (Ehh), (he, he), no, you , but if you do not have a fee then you cannot go inn and charge, 1: Well, I didn’t know what was necessary to get it to run, but I found out that there was no value put in there at all, and the only thing working was B’s test, that since there was nothing in the database,…, the VAT as test, so it wasn’t, and the only thing that I wanted to find out was that the flow would go through, and it did, and surely down there, in there, there were taxcodes, and suddenly it throws it “down”, and then we run that VAT, so it wasn’t finished at all, right, so it was just to see, to see if the invoicing was taking place, and the machinery working, and then to look at the results afterwards, right, and it was just nonsense, it was saying sort of “test data” here and “test data” there and nothing like, so I have to talk to W 2 Batchwise processing of booking requests. tomorrow, because it is not exactly, W was wondering if B had not spec-ed anything and I was thinking that perhaps it as W who should have spec-ed it, (tj, he, he). This is where we see again the code referred to as an “it” that does something, i.e., “it throws it ‘down’” and “it was saying”, etc, so it is in a sense being “anthropomorphized”. Moreover, the talker is claiming slack by circumscribing as important “if the invoicing was taking place, and the machinery working, and then to look at the results afterwards”, in other words, starting to work on getting the results right later. 2: And there’s actually exactly the same problem here in production, first, two things, first there’s that as always f***-all Oracle-date that he could not handle properly so he had to test on the time,[ 1: [Trunk, f*** 2: : and that means he did not find any prices, and that was the one error, (.), the price was correct but he just did not find it, just that day, and number two error was that when he didn’t find the price he did not fail, he just invoiced then nil, and so their biggest customer got a free ticket, (.) 1: [(ehh), well, well, 2: that, it was invoiced a naught 1: [(tji!), look at [yes, I saw the reply 2: [it is a disaster 1: didn’t understand was it was about, even 2: [yes, well, I saw the answer but I [yes, yes, (he, he) 1: that it should give naught [I was wondering have they put in a data 2: Yes, but that is another thing, I mean if it is naught on your data in, then it should be invoiced naught, bongo, then it should be invoiced naught, but if you do not have any data, he, he, 1: [yes] 2: so you can invoice them a thousand anyway (he, he) 1: Yes, but I got NULL, but I really don’t understand, and I didn’t understand the code here, because we did not synch, sort of, before he went on vacation, because he was sitting the whole of the previous week, right, implementing that production s***. and I was asking W and,… By the previous taker, the anthropomorphizing of code is even more obvious, because he is referring to it as a “he”, which is becoming clear when the talker in saying “he did not fail, he just invoiced then nil” clearly refers to a runtime-occurrence. There are, in other words, two “hims” in this excerpt; one is the programmer that has gone on vacation and one is the server software that invoices customers (even, alas, if “he” cannot find the correct trip). 1: Yes, but there must be some kind of exception if you cannot find the price,… 2: Well, but the way it is now, we will not, if we go to invoicing, now, we will check if there are any invoice data there and if there is no invoice data there then there is nothing to invoice (.) Again, the “we” is code, but is examined analytically and “defensibly”. It now seems to refer to us as in “the programmer and his code”. 1. N:o, but, what do then you mean then by invoice data? 2: Yes, so far we say that if you have not added any taxes, then that is not a price 1: 2: proximally, that is not invoice, sort of,… [Mmm, but 1: jupp, (snaps his fingers). [but then you have to put it out there as an error, 2: No, no, no, no, no, no 1: [Yes, because the rule says, that it should be something, so therefore it 2: [yes 1: should either be one or nothing, because the rule is saying that there should be a fee, and you cannot find a few then you must take it out 2: Yes, at least it should not be invoiced, but 1: mark it as an error and take it out [no, it should absolutely not be invoiced, you should 2: Out as an error? Out to where? 1: Just like any other error 2: There, then, did we did agree to take it down to EPL-processing, so 1: 2: to make the EPL-processing such that we find [No [Yes, we agreed 1: In EPL-processing should also put it out as error, or let it be on the queue The talk about rules and how to understand them is a particularly telling example of rules qua requirements’ specification being re-examined and re-formulated into matching with what the program is already doing. There are two possible outcomes: One is that the requirements are still open enough to allow the code to be kept. This does not mean that it is “acceptable” within an open requirement for the code to do anything. Instead, the rules have to be interpreted and formulated “within” the specifications so as to match the code. In the next instance, this interpretation of the code can be tested against the customer’s rules until they are accepted. The other is that the code definitely must be rewritten. The discussion continues. 1: Yes, for instance 2: Yes, yes, yes, yes 1: At least it should not be a done deal 2: In EPL processing this will be a no-show (2) di-di, it is a no-show. The EPL-processing has run a long time ago! 1: But how are we going to see this, then? 2: A little red blink on the screen,…, he, he 1: Whose screen??? This is a server-side service we’re talking about So, the final part of this excerpt confirms that the specification at this point were (to put it mildly) open enough to allow “incorrect” rules to be implemented within. In these small excerpts we have seen examples of “claiming” slack, anthropomorphizing the products, e.g., by attributing (offsetting or relocating) agency and transferring “eidos” to code, and justification of existing design to match the organizational requirements. There is still a lot more to be learnt from these excerpts, and the analysis is still only in its preliminary stages. Systematically though, from all the hours of observation, complexity as a term is really never used in this community of programmers. The question then becomes, what does that mean? 4. Discussion Talk is perhaps the most central element of any social activity, and it is not just the content that counts, it is the manner of talk. Social interaction includes non-work oriented talk as well, of course, but this paper has been concerned with talk as work. This paper has not in an inductive fashion made this into an extremely broad concern. Neither was it the intention to “fill” the existing concept of complexity with various interpretations. Given that the point of departure is that the definition of the concept is rather weak, that could have been rather a tautological examination of (even more) very different interpretations of “complexity. Instead, this paper has identified themes that can be described as “epistopics”[18]. Epistopics represent constitutive starting points of a program directed towards combining “ethnomethodology’s treatment of ordinary practical actions (ibid. p. 299)” with the interests of its “contents”, which can be found in the sociology of science. These pragmatical constructs are “the discursive themes that so often come up in discussions of scientific and practical reasoning (ibid. p. 299)” and in programming activities such epistopics could plausibly be planning, examining, design to meet specifications, etc. However, epistopics are not exactly the same across instances. Epistopics do not contribute to an agenda of identifying stable concepts and developing theories of the relationship between such concepts; instead they are topical “headings”, words that may bear family resemblances3 across instances; “they allow us to see them as similar [19]”, but for which the underlying practices are not necessarily governed by the same structures. “Complexity (management)” is the “mother-ofall” epistopics, of course, in IS research. This paper has identified and exemplified three explications of recurring espistopic themes in the talk within a community of “extreme” programmers: • Planning, as “Claming back slack”. • Code inspection, as “Transferring agency and ‘eidos’ to code”. • Design as “Justifying or adjusting a design to match perceived requirements”. Planning and estimating is of course at the very core of how IS deals with complexity. Finding out what to do, and when to do it in order to set and meet deadlines is, after all, the externalization of success versus all the stakeholders. This is of course not an easy task, and plans are probably often sketchy. Plans are useful resources for programmers, but they can hardly be expected to be exhaustive or entirely consistent. Plans are not the whole story, and even if they were people are not “programmable” by plans[20]. Therefore, plans have “slack”. But “slack” is not just a breathing space, a vacation, or a ‘sabbatical’ for the project members that happen to be “there”. It is “real-estate” that can be claimed, and this is an important part of the achievement that lies in making the plan work in practice. The integral role of talk in practical laboratory sessions has been pointed to elsewhere[21]. It is a tool-rich and very physical” environment and it is claimed that utterances often therefore are bound to objects and processes that are tangible in nature. The coding practices of programmers are poles apart from this. Coding involves working with abstract classes, ideas and concepts, rules and requirements that may (more often than not) even be written down anywhere. This is perhaps also the reason why code is so often given agency, by programmers talking about it as “it” or “him” doing things on its own, and not only that: Code does things in certain characteristic ways, that sometimes reflect back on the community or individual programmers in terms of its “eidos”. This is one way of making software “less soft” and thus, more manageable. Lynch challenges researchers of social settings to “follow the epistopics” and “investigate each case in accordance with a unique adequacy requirement [18]”. This is exactly what happens when programmers make their preferred solution “good enough” within the remainder of “interpretational space” that surround pertaining requirements. Either is stands its test (the rationale for making it was that it was successfully “overlaps” with the customer requirements), or the requirement is iteratively narrowed down to close that space and force the responsible programmer to redesign the code. 3 In a Wittgensteinian sense. How can we then use these findings to better get to grips with the notion of complexity? Well, in particular, IS researchers should pay attention to the noticeable absence of explicit references being made to the work of programmers, by programmers talking amongst themselves, as complex, complicated, intractable, etc. Moreover, as a mundane way of dealing with phenomena that we, as IS researchers (and programmers “externalizing” their work, as well) would describe as complex issues, the epistopics above represent at least a complement to the “big” tactics of IS methods: Generalization, modularization, decomposition, etc.; approaches that are so much at the core of most modeling methods and programming languages[22]. The world is, in a sense, generally complex, and there are of course many situations in which talking about complexity as an externalized characteristic of situations is warranted and relevant. However, for programmers as well as others, complexity may be simply an elliptical expression that can be a result of the number of elements, their interrelationship and dynamics. It may make project unfathomable, when the goals are unclear or changing. This paper does not disagree with that. However, it is not supported by the fieldwork presented here that pertaining issues are perceived, examined or resolved as if they were indeed “complex”, and consequently, perhaps, one should not be surprised when methods that aim to reduce complexity does not work all that well? 5. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] S. G. MacDonell, ”Comparative review of functional complexity assessment for effort estimation,” Software Engineering Journal, vol. 9, pp. 107-116, 1994. G. Engels and P. Lohr, ”Visual Specification of Complex Database Actions.,” presented at East/West Database Workshop, 1994. Y.-J. Joung and S. A. Smolka, ”A comprehensive study of the complexity of multiparty interaction,” J. ACM, vol. 43, pp. 75-115, 1996. H. Hahn, J. Hahn, and J. Kim, A Cognitive Engineering Study on the Development of an Object-Oriented Process Modeling Formalism, vol. 21: IEEE Computer Society, 1994. M. K. Buckland and D. Florian, ”Expertise, task complexity, and the role of intelligent information systems,” Journal of the American Society for Information Science, pp. 635-643, 1991. C. N. G. Dampney, M. Johnson, and P. Deuble, ”Taming large complex information systems,” in Complex Systems, D. Green and T. Bossomaier, Eds.: IOS Press, 1993, pp. 210-222. S. D. Eppinger, D. E. Whitney, R. P. Smith, and D. A. Gebala, Organizing the tasks in complex design projects. Cambridge, Massachusetts, United States: Springer-Verlag New York, Inc., 1991. BCS, The Challenges of Complex IT Projects: The British Computer Society, 2004. F. P. Brooks, ”No silver bullet: essence and accidents of software engineering,” Computer, vol. 20, pp. 10-19, 1987. C. Ciborra, The Labyrinths of Information - Challenging the Wisdom of Systems. Oxford: Oxford University Press, 2002. R. Bentley, J. A. Hughes, D. Randall, T. Rodden, P. Sawyer, D. Shapiro, and I. Sommerville, Ethnographicallyinformed systems design for air traffic control. Toronto, Ontario, Canada: ACM Press, 1992. A. Crabtree, D. M. Nichols, J. O'Brien, M. Rouncefield, and M. B. Twidale, ”Ethnomethodologically informed ethnography and information system design,” Journal of the American Society for Information Science, vol. 51, pp. 666-682, 2000. D. Martin and I. Sommerville, ”Patterns of cooperative interaction: Linking ethnomethodology and design,” ACM Trans. Comput.-Hum. Interact., vol. 11, pp. 59-89, 2004. M. Rouncefield, J. A. Hughes, T. Rodden, and S. Viller, Working with ”constant interruption”: CSCW and the small office. Chapel Hill, North Carolina, United States: ACM Press, 1994. H. Garfinkel, Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall, 1967. J. Kallinikos, ”The order of technology: complexity and control in a connected world,” Information and organization, vol. 15, pp. 185-202, 2005. H. Sacks, Lectures on Conversation, vol. 2. Oxford: Blackwell, 1992. M. Lynch, Scientific Practice and Ordinary Action: Ethnomethodology and Social Studies of Science. New York: Cambridge University Press, 1997. J. Tapper, ”Topics and manner of talk in undergraduate practical laboratories,” International Journal of Science Education, vol. 21, pp. 447-464, 1999. L. A. Suchman, Plans and situated actions: the problem of human-machine communication: Cambridge University Press, 1987. M. Lynch, Art and Artefact in Laboratory Science: A Study of Shop Work and Shop Talk in a Research Laboratory. London: Routledge and Kegan Paul, 1985. J. A. Anderson and C. E. Dahlke, Controlling complexity in Ada design representation. Dublin, Ireland: Cambridge University Press, 1990.