Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 Diversity and the Division of Cognitive Labor Ryan Muldoon* University of Pennyslvania Abstract In epistemology and the philosophy of science, there has been an increasing interest in the social aspects of belief acquisition. In particular, there has been a focus on the division of cognitive labor in science. This essay explores several different models of the division of cognitive labor, with particular focus on Kitcher, Strevens, Weisberg and Muldoon, and Zollman. The essay then shows how many of the benefits of the division of cognitive labor flow from leveraging agent diversity. The essay concludes by examining the benefits and burdens of diversity, particularly in the evaluative diversity that can be found in interdisicplinary science. 1. Introduction In both epistemology and the philosophy of science, there has been a growing interest in the social aspect of belief acquisition. Rather than conceiving of scientists as Robinson Crusoes, only relying on their own beliefs and talents for developing a research program, carrying it out, and justifying their conclusions, we can more accurately conceive of scientists as embedded in a larger social structure. Scientists, after all, read journals, go to conferences, establish collaborations, seek out grant money, win acclaim and prestige, and operate within a labor market. In light of this, we might wish to ask ourselves how these social aspects of the scientific enterprise influence the practice of science. Perhaps the most striking feature of the social structure of science is that it encourages a division of cognitive labor. The most obvious division appears in how scientists sort themselves into different disciplines and sub-disciplines. By and large, chemists work on chemistry, biologists work on biology, and geologists work on geology. This macro-level division of cognitive labor hides another division of cognitive labor that operates within sub-disciplines. When we consider this more fine-grained account of the division of cognitive labor, our focus turns toward how individual scientists and labs take the decisions of others into account. Perhaps most in line with traditional questions found in epistemology of science, we can investigate how scientists update their beliefs based on evidence produced by other scientists. Here we can consider how consensus might emerge in a scientific community, and ask whether or not the emergent consensus is likely to be true. While questions of consensus in science focus on the context of justification, much of the interest in the division of cognitive labor comes from the context of discovery. In particular, how does a division of cognitive labor effect scientists’ choices about what projects to pursue in the first place? This was the question that launched much of the recent interest in the division of cognitive labor. In what follows, we will examine different social features of science brought on by the division of cognitive labor. First, we will investigate Philip Kitcher’s landmark discussion of the division of cognitive labor, and Strevens’ elaborations on that model. We will ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd 118 Diversity and the Division of Cognitive Labor briefly consider some critiques of this approach. We will then turn to Weisberg and Muldoon’s epistemic landscapes model of the division of labor. We will then consider how this division of cognitive labor can fit into models of scientific consensus. Finally, we will consider the role that diversity plays in the various models of the division of cognitive labor. 2. An Economic Approach to the Division of Cognitive Labor Thomas Kuhn, in his ‘‘Objectivity, Value Judgment and Theory Choice’’ (Kuhn 1977) raised the question of whether there was a tension between individual and collective rationality in science. Kitcher (1990, 1993) picked up on this question. He envisioned a stylized scenario of some set S of N scientists, each looking to pick between different projects in the set R to uncover some significant truth (Kitcher supposes discovery of the structure of a Very Important Molecule, VIM). Each project in the set has a return function: a function that takes a number of workers as input, and outputs a probability of success. It is here that we face a tension between individual and collective rationality: if the representative scientist is a pure Hempelian, just motivated by doing the best science, then she will choose to work on the project with the most favorable return function – let’s call that P1. But this is not optimal from a collective standpoint: it would be in the community’s interest to hedge its bets by also having some scientists work on the lesslikely P2. This is just to suppose that P1(n) + P2(N – n) – Pr(both projects succeed) > P1(N). That is, distributing cognitive labor across projects would do more to increase the chance that the significant truth is discovered than if everyone worked on the most promising project. So, we arrive at a formal articulation of the tension between individual and collective rationality, as Kuhn suggested. This can manifest itself in a few ways, but fundamentally it occurs because scientists fail to divide their labor: their choices were not informed by the choices of others. However, Kitcher points out that we can do much to alleviate that tension. If we suppose that scientists, instead of being purely epistemically-minded, are ‘‘sullied’’ insofar as they care about rewards. So rather than care about pure epistemic rationality, scientists instead are economically rational: they seek out the greatest expected returns for their labor. Now, we can better align collective rationality with individual rationality. Rather than choosing a project based on its overall characteristics, scientists choose projects based on their marginal contribution: how much their contribution to the project would change the output of its return function. Scientists choose projects based on their marginal contribution because it is based on this marginal contribution that they are rewarded. Muldoon and Weisberg (2011) refer to this as a Marginal Contribution ⁄ Reward model. Strevens (2003) picked up on Kitcher’s MCR approach, and extended this model to look at a series of potential reward schemes and how they might influence project choice. Strevens argues that since scientific products are fairly unique in that they are only really valuable the first time they are produced, unlike most normal goods like chairs or cheeseburgers that are valuable each time they are produced. So, research programs must compete to provide the scientific good first, creating what he calls a benefits race. Following Kitcher, Strevens also argues that scientists themselves are competing in the rewards race. Strevens argues that the parallel structure of these two races leads us towards the Priority rule, which rewards just on actual achievement rather than effort, and rewards only the first person to achieve. The Priority rule, as Merton (1957) notes, can seem strange: if two scientists differ only in hours or minutes in achieving success, it is still the first to ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 Diversity and the Division of Cognitive Labor 119 finish that gets any credit. Strevens seeks to show that it is the Priority rule’s unique properties that promote optimal allocation of cognitive labor, and thus ensure that society maximizes its benefits from the scientific community. Because of this, we can explain why the Priority rule came to be, and justify its continued existence. To do this, Strevens relies on Kitcher’s MCR approach, examining the allocation properties of three rules: Marge, Goal, and Priority. Marge distributes resources according to marginal contribution, rewarding scientists for how much their contributions improve the success prospects of a project regardless of whether that project succeeds. Goal is a modification of Marge, in which scientists are only rewarded if their project actually succeeds. This is most compatible with the reward scheme that Kitcher suggests. Finally, Priority is a further refinement, in which rewards are only distributed to the project that finishes first. Strevens argues that risk aversion will lead scientists to increasingly favor the project most likely to succeed when we shift from Marge to Goal, since scientists are not just rewarded for their efforts, but for successfully picking the correct project. The transition from Goal to Priority offers us a similar story: now scientists are only rewarded for finishing first, and so they have reason to further bias their choices toward the most likely project. Strevens argues that this aligns the benefits race, which only cares about the first project to succeed, with the rewards race. Under this scheme, the scientific community hedges its bets, since there remain incentives to work on understaffed projects, but it skews the distribution of cognitive labor toward those projects that are most likely to provide social benefits in a timely manner. The MCR approach adopted by Kitcher and Strevens has numerous strengths: it clearly illuminates how non-epistemic motives can lead to improved epistemic outcomes, it shows how the division of cognitive labor is sensitive to the reward structure in science, and it provides us with a view of how economic reasoning can inform our philosophical understanding of the division of cognitive labor. Goldman and Shaked (1991) are even more explicit about the economic aspect of their modeling of the division of cognitive labor. In this model, scientists receive rewards when they are able to conduct experiments that lead to altering the prior probability distributions that other scientists have over the world-state W. Scientists, rather than being in a race for priority, or choosing between potential projects, are choosing between investigation and reporting their results. In this model, scientists can continually announce results, and attempt to support or rebut previous claims. So rather than a single decision like the MCR approach, scientists are continually engaging with each other, and assigning credit based on how much each scientist alters the priors of the others. At any given round, a scientist can choose between several experiments, each with three possible observational outcomes. Once one has observed the outcome, they can either stay silent, or announce that it supports one possible world-state over others. Agents can then rely on Bayesian updating to see how their priors change, and award credit. We find that experiments that are more able to discriminate between possible world-states are the experiments for which scientists can receive the most credit. Interestingly, Goldman and Shaked argue that if we compare credit-motivated agents to truth-motivated agents, credit-motivated agents do well, but there is a range of cases for which truth-motivated agents will slightly outperform them. 3. Challenges to the Economic Approach Though the economic approach has offered several significant insights to our understanding of the division of cognitive labor, it has not been without its critics. Notable among ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 120 Diversity and the Division of Cognitive Labor them has been Hands (1995, 1997), who has warned that while economics appears to be a potential resource for philosophy of science, it should be noted that economics has a number of methodological challenges that must be taken seriously prior to adopting those methods wholesale. Hands notes that much of philosophy of economics is about the failures of economic methodology, which might lead us to wonder whether those same methods will necessarily be beneficial in our efforts to better understand the practice of science. Wray (2000) points out that the invisible hand mechanism that is often invoked in economic philosophy of science, perhaps most notably by Hull (1988), is not an appropriate form of explanation, as very often the structures that help science operate were deliberately chosen, rather than being endogenously generated by the choices of scientists. Instead, we’re left with the ‘‘hidden hand’’ of our institutions guiding scientific practice. This is compatible with the MCR model, as neither Kitcher nor Strevens require that the reward schemes in science be endogenous to scientific practice. A direct critique of the MCR model is seen in Muldoon and Weisberg (2011). In it, they point out that the MCR modeling approach rests on two important idealizations: the ‘‘distribution assumption’’ and the ‘‘success function assumption’’. The distribution assumption is that each scientist knows what project every other scientist is working on. The success function assumption is that each scientist knows, prior to investigation, what the precise success function for a given research project is. Without these two assumptions, agents would be unable to calculate their expected marginal utility for working on a project. Muldoon and Weisberg weaken these idealizations in a series of agent-based models, and show that if we allow for limited knowledge of what others work on, and a distribution of beliefs about the nature of success functions, we can see dramatically different results than those predicted with the idealizations in place. This suggests that the models are not particularly robust. Given the nature of his project, this poses a larger challenge for Strevens than it does for Kitcher. 4. Epistemic Landscapes Weisberg and Muldoon (2009) developed an alternative approach to modeling the division of cognitive labor. Rather than rely on an economic methodology, they adopted something closer to an ecological approach. Scientists are imagined to be ‘‘hill-climbers’’ on an unknown ‘‘landscape.’’1 The landscape itself is interpreted as a topic of scientific inquiry. The X and Y dimensions represent potential research approaches. The Z dimension represents the epistemic significance of any findings to be had given the research approach indicated by the (X, Y) position. The landscape is broken up into ‘patches’ of unit size. At the beginning of inquiry, scientists have no knowledge of the landscape – that is, they do not know anything about the comparative significance of any research approaches. They discover this only by traversing the landscape. Weisberg and Muldoon interpret this traversal as a series of research efforts. Each patch visited is equivalent to doing an experiment, reading the existing literature as it pertains to that research approach if it exists, or communicating with others who have already tried that approach. The scientist themselves can have one of three different research strategies, which are understood as different ways of responding to the research of others. The simplest approach is simply the ‘‘control’’ research strategy. The scientists in the control group simply ignore all of the other scientists, and adopt a classic hill-climbing strategy. The strategy is simple: the scientist adopts a random heading, and moves forward one patch. If the new patch is more significant, then the scientist takes another step forward. If not, but the patch is of equivalent significance, then with a 2% probability the ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 Diversity and the Division of Cognitive Labor 121 scientist moves forward one patch with a random heading. If the patch is of lower significance, the scientist moves back her previous patch and chooses a new random heading, and starts again. With this research strategy, the scientist is sure to find local maxima of significance on the epistemic landscape. Most importantly, however, this strategy embodies scientists without a concept of the division of cognitive labor. They work on their own research, but pay no heed to what anyone else is doing. This will find them some individual success, but it is not socially optimal. A second approach that Weisberg and Muldoon consider is the ‘‘follower’’ strategy. This is strategy is designed to embody the strategy of ‘standing on the shoulder of giants’ in one’s scientific research. In each round of the model, followers consider the eight patches in their Moore neighborhood – each immediately adjacent space from their current position. If any approaches in the neighborhood have been investigated and of greater significance than the current approach, then the scientist moves towards the approach of greatest significance (in a tie, she flips a coin). If none of the already-investigated patches are of greater significance, and there are approaches that haven’t been tried, then the agents go there. Otherwise, they stop. This strategy does allow for a division of cognitive labor – followers will take the efforts of others into account in their research strategy, and take it as evidence that nearby already-explored approaches are where good prospects for future research lie. Perhaps surprisingly, a population of followers does worse than the control population. They find peaks in the landscape less often, and cover less significant ground. Followers divide their labor, but do so inefficiently. The final approach that Muldoon and Weisberg consider is the ‘‘maverick’’ strategy. Mavericks, like followers, take information about what others have done into account, but in an entirely different way. Rather than follow what others have done, mavericks take that as evidence that they should go in a different direction. In each round, they ask if their current research approach is at least as significant as the previous approach. If so, then if any patches in the scientist’s Moore neighborhood are unvisited, she picks randomly between them. If all have been investigated, then she moves to the patch of highest significance, or stops if none is higher than where she is. If her current patch is less significant than the previous patch, she returns to the previous patch and sets a new random heading. This strategy is notable for attempting to maximize cognitive diversity. Followers, by design, attempt to correlate their research efforts with what has already been done. Mavericks attempt to anti-correlate. They take what has been done as an indication that they should do something different. This strategy pays off for them remarkably well: mavericks vastly outperform both followers and controls. Even small populations of mavericks almost always find all maxima on the landscape, and have very good coverage of other significant portions of the landscape. Perhaps most interestingly, however, is what happens when Weisberg and Muldoon examine mixed populations of followers and mavericks. These mixed populations do better than just populations of mavericks, but it is not all the direct effect of mavericks. Mavericks stimulate followers to cover more fruitful territory, and serve to divide the followers up more than in a pure follower population. We see that each research strategy has a role: mavericks are very good at getting a general lay of the land, and efficiently find peaks. But followers do much of the work involved with filling in details, examining the areas around the paths blazed by the mavericks. The follower strategy, seemingly foolish by itself, becomes useful once there are a few mavericks around. If we imagine that the maverick research strategy to be more costly than the follower strategy, then this model might adequately describe some features of the real division of cognitive labor in science as it is now practiced: a few mavericks stimulating the efforts of many more followers. ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 122 Diversity and the Division of Cognitive Labor This approach affirms a key finding of Kitcher’s: that a population of Hempelian scientists who just work on the apparently-best projects, regardless of whatever anyone else is working on, do much worse than populations of scientists who consciously divide their labor by making research choices informed by the work of others. However, the paper expands on this point by demonstrating that how one responds to evidence matters a great deal. A population of followers does worse than a population of controls. But a population with mavericks will vastly outperform controls. What’s even more striking, however, is that the interaction of mavericks and followers further divides cognitive labor: mavericks break new ground, while followers do the detailed work of exploring variants on those original efforts. This diversity of responses to the work of others enables a much finer analysis of the division of labor than could be discussed in the MCR approach. 5. Models of Scientific Consensus For a scientific discipline to be a cohesive social enterprise, scientists must seek to establish some common consensus. Epistemic communities need to share some common beliefs to ground debates over their differences. Otherwise, we would just have a collection of strangers who happen to be interested in some of the same things. Biologists believe in the neo-Darwinian synthesis. Doctors believe in the germ theory of disease. Physicists agree on the speed of light. Each of these consensus beliefs started out as research programs. Through a process of evidence-gathering, publishing, and discussion, scientists slowly converged on a single answer. A prominent model of this has been the Lehrer-Wagner model of consensus formation. (Lehrer 1975; Wagner 1978, 1982, 1985; Lehrer and Wagner 1981) This is a repeatedaveraging model, which was seen in earlier work as well (French 1956; DeGroot 1974). This model represents a simplified rational reconstruction of consensus formation. As people discuss their beliefs, they update their personal beliefs based on that discussion.2 It is a gradual process, and is simply a function of the agents’ initial estimates of some value, and their epistemic respect for one another. This model assumes that the process of gathering evidence is complete: the agents have assessed all of the available information and arrived at their estimates. Hegselmann and Kraus (2006) extend this averaging model with a more explicit division of labor, and a clear anchor of truth to converge on. In their model, which utilizes both analytic proofs and agent-based simulation, they look at populations wherein only some are truth seekers, and others have varying cognitive capacities. There is a truth to be found, and the question that is explored is under what conditions will a population of scientists reliably converge on it. This breaks a number of the symmetries inherent in the repeated averaging approach, but builds on the basic principles behind the model. Surprisingly, they show that relatively few truth-seekers are needed for convergence on the truth in a wide variety of cases. Another approach to consensus modeling, perhaps closer to the actual practice of science, is one in which we continue to gather evidence until we reach consensus. Scientists continue to conduct experiments and share their results until they all come to agree about the question at hand. Zollman (2010) takes this to be one application of the division of cognitive labor. In particular, Zollman explores how diversity might facilitate ensuring that scientific consensus ultimately tracks truth, by slowing down consensus until sufficient investigation has taken place. Zollman focuses on an example taken from medicine, looking at two different treatments for peptic ulcer disease, one relying on the idea that bacteria cause the ulcers, the second relying on the idea that it is excess acid in the stomach. ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 Diversity and the Division of Cognitive Labor 123 Zollman models scientists as Bayesians who play two-armed bandits, connected by lines of communication with other scientists. Each arm of the bandit represents an investigation into a treatment method – one supposes an antibacterial approach, one an acid reduction approach. If an arm is pulled, the payoff is simply whether the particular treatment method succeeded. As with casino slot machines, the challenge is that each arm has a payoff distribution, but this distribution is unknown to the agent. Scientists, if they are connected to each other in a communication structure, can observe what payoffs the others got from pulling a particular arm. Each agent has prior beliefs about the nature of each arm’s payoff distribution. Every time the scientist pulls an arm and receives a payoff, the scientist updates their priors. Importantly, scientists can also observe anyone who they are directly connected to in a communication network, and also use that to update their priors. In every given round, scientists pull the arm that they think is superior, and update their priors based on their performance, and the performance that they observe from other scientists that they can communicate with. Zollman examines several different symmetric network structures, and demonstrates that those structures with fewer connections – less communication between scientists – are most correlated with scientists converging on the superior bandit arm. This is due to the fact that less-connected networks reduce the amount of communication among scientists. This reduced communication allows greater variety in individual scientists’ beliefs, until they all start converging on the right outcome. Too much homogeneity too soon can result in consensus, but consensus of the wrong kind. Zollman points to another way of generating this diversity. If we are stuck with highly-connected scientists, we can instead give some of them extreme priors. Those individuals who will be less responsive to new evidence from others will be able to continue to test alternative hypotheses at a rate sufficient to bring others around if they happen to be right. However, Zollman points out that if we combine these two mechanisms for generating diversity – extreme priors on a more minimal network – then diversity can prove to be too stable, and convergence cannot be achieved. 6. The Benefits and Burdens of Diversity In several of the models of the division of cognitive labor that we have seen, diversity plays an important and positive role. Kuhn discusses the risk-hedging value of diversity, which is explicitly manifested in the MCR models of Kitcher and Strevens. Weisberg and Muldoon show how diversity matters in two ways: the anti-correlation of maverick strategies promote fuller scientific exploration, while the difference in cognitive styles manifested in the information processing differences between mavericks and followers promote a division of exploration of new scientific frontiers and exploitation of existing ones amongst scientists. Zollman showed how transient diversity in beliefs, whether fostered by limited communication or stubborn scientists, can help ensure that scientific consensus tracks the truth. In all of these examples, diversity helps us. In fact, the division of cognitive labor promotes diversity in the sense that it encourages differences in agents, and as more diversity is generated, we can make a finer-grained division of labor. However, diversity can manifest itself not just in ways we process information, or what our priors are, or what project we pick. We can also find diversity in our evaluative judgments. We can disagree about what values we ought to be promoting in our epistemic endeavors. We can disagree about what the right standards are for epistemic success. D’Agostino (2009) points out that the division of cognitive labor will generate a diversity of evaluative standards as a consequence of generating the features that we have shown to ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 124 Diversity and the Division of Cognitive Labor be valuable. On this model, we get increasingly high-dimensional spaces to explore, and not necessarily any agreement on what dimensions matter the most. With this, our goals of widespread consensus may become more difficult, but the further division of cognitive labor becomes all the more important. Clusters of researchers can pick areas of this highdimensional space to explore with whatever search strategies they see as useful. If they are sufficiently diverse in their interests and abilities, they might collectively be able to effectively search the complex search space. But this will not have the character of a centralized, organized search – instead it will be a process of devolution of standards to more localized judgments about best practices and methods. If this is the path that the increasing diversity brought on by the division of cognitive labor leads us down, what are we to do? One answer is to look at those areas of philosophy of science that already focus on the massively interdisciplinary work that can be found in Big Science projects, such as radar, particle detectors, or more recently, large colliders. Galison (1997) developed the ‘‘trading zone’’ metaphor for scientific collaborations that involve multiple disciplines of science and engineering. In anthropology, a trading zone is where two or more communities meet to trade goods, even if they often lack a common language. Pidgins or creoles develop, enabling people to be able to trade effectively, even if they do not always understand everything that the other party would like to express. Similarly in science, Galison argues that scientists and engineers develop common symbols and scientific pidgins that enable information exchange. Given the need for successful coordination, new kinds of expertise can emerge: the ability to facilitate exchanges between disciplines becomes increasingly valuable in these kinds of environments. With the advent of the study of the division of cognitive labor, epistemology and the philosophy of science have become more sensitive to the social nature of scientific inquiry. Great strides have been made in understanding how our collective epistemic goals might be helped by focusing on our individual interests, and the social structures of science that can influence them. Diversity can be shown to be a benefit both in the context of discovery and the context of justification. However, there is much work left to do. Developing new mechanisms for understanding how to mitigate the burdens of increased evaluative diversity, if not find a way to leverage potential benefits, is essential as the scientific community attempts to engage in increasingly interdisciplinary work at the same time as scientists become more and more specialized. Epistemic rationality alone does not tell us how to pick a scheme of evaluation. Nor does it tell us how to negotiate with those who have different standards than us. Yet, it is essential for many of our collective epistemic goals that we find ways of navigating these challenges. The world is complex enough that it drives us to divide our cognitive labor in our attempt at understanding it. We need to be sure that we can find a way of putting our divided endeavors back together again. Short Biography Ryan Muldon is a post-doctoral fellow in the Philosophy, Politics and Economics program at the University of Pennsylvania. Previously, he was a post-doctoral fellow at the Rotman Institute of Philosophy at the University of Western Ontario. His research focuses on diversity in the social contract, norm dynamics, and the division of cognitive labor. Notes * Correspondence: Philosophy, Politics and Economics, University of Pennyslvania, 311 Claudia Cohen Hall, 249 S 36th Street, Philadelphia, PA, USA. Email: rmuldoon@sas.upenn.edu. ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000 Diversity and the Division of Cognitive Labor 125 1 Hill-climbing models are a kind of local optimization technique found in computer science. In these models, agents try and ‘‘climb uphill’’ on some landscape that they cannot see in advance. Essentially, they are like blind hikers who are looking for the top of the mountain. They climb uphill when they can, and avoid going downhill. The landscape component is a representation of whatever problem for which the modeler would like to find an optimum. 2 The core idea of the model is that we have N members of a group G that is seeking to come to consensus over something, whether it be a value of a constant, or the probability of an event, or something else that is similarly reducible to a numeric value. Each member i of the group has an assessment of each member j’s expertise on the question at hand, represented by wij. This assessment provides a measure of the respect an agent has for each of her peers. Lehrer and Wagner then use these weights to construct an NxN matrix W, representing the total respect assessments in the group. They constrain the wij’s in W such that none of them are negative, and that the values in each row sum to 1. This matrix W is then multiplied against the vector E, which is simply the vector of everyone’s estimates of the value in question. In the resulting product WE the estimates have been modified according to the respect assessments. Assuming we still lack consensus, this process is repeated k times until WkE has identical components. These components are then the consensus estimate. Works cited D’Agostino, Fred. ‘From the Organization to the Division of Cognitive Labor.’ Politics, Philosophy and Economics 8.1 (2009): 101–29. DeGroot, M. H. ‘Reaching a Consensus.’ Journal of the American Statistical Association 69.345 (1974): 118–21. French, J. R. ‘A Formal Theory of Social Power.’ Psychological Review 63.3 (1956): 181–94. Galison, P. Image and Logic: A Material Culture of Microphysics. Chicago: The University of Chicago Press, 1997. Goldman, A. and M. Shaked. ‘An Economic Model of Scientific Activity and Truth Acquisition.’ Philosophical Studies 63 (1991): 31–55. Hands, D. Wade. ‘Social Epistemology Meets the Invisible Hand: Kitcher on the Advancement of Science.’ Dialogue 34 (1995): 605–21. ——. ‘Caveat Emptor: Economics and Contemporary Philosophy of Science.’ Philosophy of Science 64 (1997): s107–16. Hegselmann, R. and U. Krause. ‘Truth and Cognitive Division of Labour: First Steps Towards a Computer Aided Social Epistemology’. Journal of Artificial Societies and Social Simulation 9(3)10 (2006). [online]. Retrieved on 29 August 2012 from: http://jasss.soc.surrey.ac.uk/9/3/10.html. Hull, David. Science as a Process: An Evolutionary Account of the Social and Conceptual Development of Science. Chicago: University of Chicago Press, 1988. Kitcher, Philip. ‘The Division of Cognitive Labor.’ Philosophy of Science 87 (1990): 5–22. ——. The Advancement of Science. New York: Oxford University Press, 1993. Kuhn, T. S. The Essential Tension. Chicago: University of Chicago Press. 1977. Lehrer, K. ‘Social Consensus and Rational Agnoiology.’ Synthese 31 (1975): 141–60. ——. and C. Wagner. Rational Consensus in Science and Society. Dordrecht: Reidel, 1981. Merton, Robert. ‘Priorities in Scientific Discovery.’ American Sociological Review 22 (1957): 639–59. Muldoon, R. and Weisberg. M. ‘Robustness and Idealization in Models of Cognitive Labor.’ Synthese 183.2 (2011): 161–74. Strevens, Michael. ‘The Role of the Priority Rule in Science.’ Journal of Philosophy 100.2 (2003): 55–79. Wagner, Carl. ‘Consensus Through Respect: A Model of Rational Group Decision-Making.’ Philosophical Studies 34 (1978): 335–49. ——. ‘Allocation, Lehrer Models, and the Consensus of Probabilities.’ Theory and Decision 14 (1982): 207–20. ——. ‘On the Formal Properties of Weighted Averaging as a Method of Aggregation.’ Synthese 25 (1985): 233–40. Weisberg, M. and R. Muldoon. ‘Epistemic Landscapes and the Division of Cognitive Labor.’ Philosophy of Science 76.2 (2009): 225–52. Wray, Brad. ‘Invisible Hands and the Success of Science.’ Philosophy of Science 67.1 (2000): 163–75. Zollman, Kevin J. S. ‘The Epistemic Benefit of Transient Diversity.’ Erkenntnis 72 (2010): 17–35. ª 2013 The Author Philosophy Compass ª 2013 Blackwell Publishing Ltd Philosophy Compass 8/2 (2013): 117–125, 10.1111/phc3.12000