From Galaxy to Universe A Multidisciplinary Meta-Analysis of Publications on Public Values from 1969 to 2012 Gjalt de Graaf (VU University Amsterdam, The Netherlands) Tina Nabatchi (Syracuse University, USA) Zeger van der Wal (VU University Amsterdam, The Netherlands) Introduction The study and discussion of public values (PVs) has generated growing interest in public administration and public management in recent years. The popularity of the topic is evidenced in several ways. There are a relatively large number of scholarly publications with public values as its central focus, with nearly 150 academic publications on the topic since 2006. The Public Values Consortium, founded in 2008, is hosting its third biennial workshop. Public values panels at prominent public administration and management conferences such as PMRC in 2011 and IRSPM in 2012 have also generated a lot of attention and drawn large audiences. In fact, the two public values tracks at IRSPM in April 2012 were the most popular at the conference. With a combined total of 72 paper submissions and 61 paper presentations, these two tracks accounted for almost ten percent of the presentations at this 48-track conference. Despite the popularity of the topic, however, there remains tremendous conceptual confusion among scholars. ‘Public values’ and its contents (whose values the concept contains) differ widely among scholars, and both parts of the concept – ‘public’ and ‘values’ – are interpreted and used in many different ways (Beck Jørgensen & Bozeman 2007; Bozeman 2007). Consequently, examples of PVs mentioned in the literature vary widely. For example, one study in public administration identified more than 100 fundamental values as being important public values (Beck Jørgensen & Bozeman 2007), whereas a broader review of the public administration literature identified over 400 different public values (van der Wal et al. 2006). Moreover, there is little (perhaps no) agreement about what PVs are, which values are ‘public’ and why, how PVs should be defined and measured, or how the many perspectives, approaches, and angles to the study of PVs should be reconciled and weighted, let alone be integrated (cf. Rutgers 2008). Part of the problem for the study of public values seems to be the different conceptual and theoretical starting points of research, leading to a ‘dialogue of the deaf’ among scholars. As de Bruijn and Dicke (2006: 718) note, “An immense landscape of theories and terminologies can be unfolded, especially when we incorporate perspectives used in institutional economics, law and public administration.” In short, to quote Gallie’s (1955) famous phrase, public values are essentially “contested concepts,” and like many other concepts, they “are not waiting for us ready-made, like heavenly bodies. There is no heaven for concepts. They must be invented, fabricated, or rather created and would be nothing without their creator’s signature” (Deleuze & Guattari 1994: 5). Despite the numerous challenges, many scholars seem to agree that “if researchers can advance, even incrementally, the study of public values beyond its current ambiguous and unbounded status, then those advances could serve many different theory developments and even practical purposes” (Beck Jørgensen & Bozeman 2007: 355). In their pursuit of greater clarity and to address these and other challenges, scholars have attempted to clarify and classify public values (e.g., Rutgers 2008), and some have even claimed to lay out the universe of public values (e.g., Beck Jørgensen & Bozeman 2007; Nabatchi 2011). However, the existing literature reviews, classifications, and conceptual maps are based on rather narrow reviews of the topic; thus, it is more accurate to say that scholars have touched upon a galaxy of public values rather than the entire universe. We agree that “There is no more important topic in public administration and policy than public values” (Beck Jørgensen & Bozeman 2007: 355) and see every reason to continue to examine the topic – to move from galaxies to the universe. However, because the concept has so many shapes and forms, and because the literature is so diffuse, it is hard to know how to move forward. Progress requires making explicit all the shapes and forms currently present in the PVs literature, unraveling existing patterns that may exist, and analyzing how PVs are studied and why scholars do so in such divergent ways. This paper is a first step in that direction. Specifically, we conducted an extensive and rigorous search of the scholarly literature on public values, identifying 399 publications (articles, book chapters, books) from over fifteen academic disciplines and dating back to 1969 that focus on or mention PVs. We then entered each publication into a database, and categorized all publications using nineteen criteria, such as the academic discipline or field of the research, the definition and conceptualization of PVs used, and the number of times the publication has been cited, among others. This paper presents the results of our initial analysis of our database. We begin with a very brief review of some of the existing literature reviews on the topic, identifying some of the problems in the public values research streams. Next, we discuss our methods for conducting the literature search and constructing our database. We then present some initial findings, concentrating, at this point, on general descriptions of the publications in the database. We conclude with a discussion about our findings and our next steps for the meta-analysis. A Brief Review of Existing Literature Reviews As noted above, PVs research currently faces a conceptual muddle. While there are many dimensions to this muddle, we only briefly touch on a few related aspects. First, the term ‘public values’ has many different meanings in research; studies that claim public values as their central focus might, in reality, be examining very different phenomena. For example, some authors discuss safeguarding PVs in a time of privatization (de Bruijn & Dicke 2006) or dominant economic individualism (Bozeman 2007), while others appeal to reconciling PVs in a time of businesslike public management philosophies (Frederickson 2005). Some, addressing public values in general, propose sets of PVs (Gregory 1999; Tait 1997) or criticize and further expand the concept by emphasizing forgotten dimensions, such as ‘the public’ and ‘democracy’ (Nabatchi 2010, 2011, 2012). Others derive sets of specific PVs (e.g., equity or lawfulness) through empirical research (Beck Jørgensen 2006; Beck Jørgensen & Rutgers 2011; van der Wal 2011; van der Wal, de Graaf, & Lasthuizen 2008). This and other research is not unified by a single definition of PVs, though many scholars use or cite similar definitions and reference overlapping literature. A second dimension of the current conceptual muddle concerns the continuing debate about whether and how to distinguish ‘public’ values from other types of values. In particular, a dominant theme in previous literature reviews on public values (Beck Jørgensen & Bozeman 2007; Rutgers 2008) or values in public administration and business administration (Schreurs 2005; van der Wal et al. 2006) pertains to examining values in the context of public and private sector organizations. Following this dichotomous perspective, some scholars suggest distinguishing between specific public sector values (public values) and specific private sector values (private values) (e.g., Jacobs 1992; van der Wal and Huberts 2008). This view stereotypically associates values such as equity, solidarity, the public interest, social goals, stability, legislation, political influence, and democratic processes with the public sector, and values such as entrepreneurship, efficiency, financial interest, competition, business risks, and the realization of corporate goals with the private sector (Osborne & Gaebler 1992; Reijniers 1994; Rosenau 2000). However, this dichotomy between public and private values is problematic. Most obviously, it is increasingly difficult to draw a clear boundary between the two sectors (Bozeman 1987; Bozeman & Bretschneider 1994; Drewry, Greve, & Tanquerel 2005). Moreover, the suggestion that PVs only belong to the public sphere and private values to the private sphere does not hold empirically. For example, de Bruijn and Dicke (2006) demonstrate that public values are not solely owned or acted upon by public officials. Instead, PVs also serve as guides for, and are protected by, private action. Thus, it is unclear whether public and private values are (empirically) different or should be (normatively) different. Third, to address this problem of labeling values as being ‘public’ or ‘private’, some scholars have attempted to categorize or classify PVs. For example, some scholars have classified public values by focusing on core values, chronological ordering, or some kind of bifurcation or dimensional distinction (see Rutgers 2008). Other classifications are derived from an analysis of the public administration and political science literatures (Beck Jørgensen & Bozeman 2007). Still other classifications are based on “hard” and “soft” values (Steenhuisen, Dicke, & de Bruijn 2009); individual, professional, organizational, legal, and public-interest values (Van Wart 1998); ethical, democratic, professional, and people values (Kernaghan 2003); political, legal, organizational, and market values (Nabatchi 2011), or on values related to administrative rationality, democratic morality, and political survival (Buchanan & Millstone 1979). However, the breadth and diversity of these classifications systems present additional conceptual problems. For example, the concept of PVs is used in reference to concrete goals such as the reliability and safety of public transport (de Bruijn & Dicke 2006; Steenhuisen 2009; Weihe 2008), to procedural- and process-related rules such as accountability and transparency (Weihe 2008), and to moral precepts of right and wrong that (should) guide public action (van der Wal 2008). But these referents are related; for example, transparency can be a concrete goal, a procedural value, and a moral precept. Moreover, in the public sphere, PVs as concrete goals, procedural rules, and moral precepts always interact. According to Walzer (1973), in the process of governance a public actor can choose a course of action that is perfectly justified on utilitarian grounds, but still leaves the actor guilty of a moral wrong. So there are moral rules of the game: the ‘means’ (or process values) specific to public governance can conflict with the ends public actors pursue. For example, “Because transparency is an obligation resting on democratic government, there is a permanent danger that, in those areas of policy requiring secrecy as a necessary condition for successful execution (notably, foreign affairs), this obligation will be violated” (Nieuwenburg 2004: 685). As a result of these and other issues, the examples of PVs mentioned in the literature vary widely. One particularly useful attempt at generating clarity is Beck Jørgensen and Bozeman’s (2007) construction of the ‘public values universe’, which contains 72 public values that are classified among seven constellations based on which aspects of public administration or public organization the value affects. This study has been used by a number of authors (Google Scholar reports that it has been cited over 120 times) in both empirical studies on public values (e.g., Andersen et al. 2012) and conceptual exercises (e.g., Nabatchi 2011). However, despite its value, and as suggested in the introduction, this article (like others) may be better considered as presenting a galaxy of public values rather than the entire universe. Specifically, the article relies on a relatively narrow review of the literature, using only publications in US, UK, and Scandinavian public administration journals from 1990-2003. Moreover, Beck Jørgensen and Bozeman (2007) simply inventory the PVs that appear in the public administration literature, and distinguish and name the different constellations of heavenly bodies – to keep with the metaphor – to which these PVs belong. Their study does not look at the meanings and usage of the PVs in context. The authors explain that taking PVs out of context allows for easier contemplation: “values are set free from partial understandings and from deadlocked, polarized debate, making it possible to construct new perceptions and judgments” (Jørgensen & Bozeman 2007: 357-358). However, they also acknowledge a disadvantage in doing so, namely that values are “robbed of their specific meaning” and “historical background is lost” (2007: 257). In contrast, our study seeks to examine the different conceptualizations and meanings of PVs in their intended context. That is, instead of asking “what PVs exist” as Beck Jørgensen and Bozeman (2007) did, we intend to ask, “what PVs are used where, when, why, and how.” Our logic is that for the study of public values to progress, we need to understand how different scholars conceptualize, interpret, use the concept of PVs, which approaches to PVs dominate different disciplines, and how and where such approaches differ and overlap. The need for such research is especially obvious when we move outside the boundaries of public administration and include other disciplines and fields such as law and economics. In essence, we are in need of a Public Values String Theory that accounts for and can be used to compare, contrast, and integrate different theories, approaches, and streams of knowledge from across disciplines – we need to move from galaxies to the universe. Before we can generate this type of understanding, however, we must begin with the basics and answer questions that have not yet been asked, let alone answered. For example, is the concept of PVs a primary focus in research or is it simply mentioned in relation to other topics and issues? What kinds of publications – journal articles, books, book chapters, etc. – exist? In what year did the concept of PVs first appear? How was it used, and in which disciplinary context? What has been the interest in PVs research over time? Do most publications on PVs indeed stem from public administration and public management scholars, or can we detect serious scholarship on PVs within fields and disciplines? Do scholars tend to use self-created definitions of public values or do they rely on others’ definitions? Do they distinguish between the concepts of public value and public values? What type of literature is being published, that is, are publications empirical, normative, or theoretical in nature? These are among the questions we seek to answer in our preliminary analysis of the PVs literature. Methodology: A Meta-Analysis of 399 Publications on Public Values Rationale Our overall purpose in conducting a meta-analysis of public values publications is to establish a genealogy of ‘public values’, that is, to study the definition of the term and the contexts in which it has been used. To begin this meta-analysis, we needed to find publications and conduct a literature review. Traditional literature reviews use generic methodological applications “to find out what is already known about the intended research topic” by identifying existing knowledge and gaps and developing a concomitant research agenda (Robinson & Reed 1998: 58). However, our (ultimate) objective in this research is to identify the overarching narrative and evolution of the PVs concept. Thus, rather than simply presenting a ‘tour’ of the literature, we (eventually) aim to map themes and develop a general narrative of how the overall concept of PVs is conceptualized and represented. Accordingly, as we move through analysis of our database on the public values literature, we will conduct a thematic, integrative literature review where we identify themes and patterns in various areas of research, and discuss those themes with general reference to the studies and contexts from which they came (Creswell 2008). We are just beginning this analysis. Here, we present some initial and preliminary findings about the nature of the literature on PVs we have found. First, however, we discuss the basics of our literature search and database construction. Sample and selection criteria To gain an overview of the multidisciplinary approach towards PVs, the first search was generated using Google Scholar. This search was conceptualized as the initial ‘broad web search’ that would be followed by cross-checking and other searches using more academically renowned systems. Essentially, Google Scholar scans content in peer-reviewed and open access journals, books, dissertations, preprint repositories, academic society papers (if available on the Web), technical reports, and other materials, thus allowing for a wide, interdisciplinary, and international search of academic literature. To conduct this search, we used the keywords “public values”1 in conjunction (i.e., by using quotation marks), included citations (i.e., not only 1 We did not include the term “public value” in our searches for two reasons. First, although many scholars do not explicitly distinguish between public values and public value (cf. Alford & O’Flynn 2009; O’Flynn 2009), we believe that these are distinct, though related concepts (cf. Nabatchi 2011, 2012; for discussions about public value, see Moore 1995 and Bennington & Moore 2011). Second, Williams and Shearer (2011) recently released a content analysis and categorization of 74 scholarly publications on public values. materials with links), and limited the dates to 1945-2012. This initial search led to 320 publications. Although Google Scholar ranks documents based on citations, hits are not listed hierarchically, that is, with the most frequently cited documents appearing first. Rather, “Google Scholar aims to rank documents the way researchers do, weighing the full text of each document, where it was published, who it was written by, as well as how often and how recently it has been cited in other scholarly literature.”2 As Burright (2006) states, however: “Its lack of authority control for basic data elements such as author names and publication titles greatly limits its ability to sustain a serious scientific and technical research audience as an exclusive source of literature. Its speedy search engine and voluminous output are tradeoffs that a researcher must consider weighing against accuracy and thoroughness in a literature search.” For this reason, additional searches were carried out using the ISI ‘Web of Knowledge’3 search engine (using “public values” in the ‘topic’ category) and ProQuest4 (using “public values” in the ‘abstract’ category). After these searches, the initial Google Scholar database was crossreferenced and any ‘missing’ articles were input. In the end, these three search engines identified 399 scholarly publications. Information for each item was entered into the database, including author(s), title, document type, and other relevant publication data. In addition to entering the items in the database, materials were downloaded or checked out of libraries for future analysis. It should be noted, however, that we do not currently have access (online or through libraries) to 67 publications. Thus, at present the database for analysis consists of 332 publications. 2 http://scholar.google.nl/intl/en/scholar/about.html http://apps.webofknowledge.com/UA_GeneralSearch_input.do?product=UA&search_mode=GeneralSearch&SID= 4BJmJ4nP6jngi9dEfnF&preferencesSaved= 4 http://www.proquest.co.uk/en-UK 3 Coding and Classification Once the articles were entered into the database, we began the process of coding. First, we opened each downloaded publication and scanned it using a keyword search for “public values.” (Books and book chapters were scanned using the ‘Google Books’ search engine.) We designated publications as either having a “focus” on public values or having a “mention” of public values. Publications that focus on PVs have the concept as their central topic of discussion or analysis (and mention the concept in their titles and/or abstracts). Publications that have a mention of PVs use the concept somewhere in the text but primarily focus on different topics and issues. Second, we asked several questions about each publication and recorded the answers in the database: 1. Do the authors use an explicit/concrete definition of public values? Yes/No 2. If no, how do the authors approach/conceptualize public values, and in what context? 3. If yes, what definition do they use? 4. Is this definition ‘self-created’ or are they using someone else’s definition? If so, who are they citing? 5. Do the authors distinguish between ‘public value’ and ‘public values’? Third, we added a category for ‘literature type’ based on classifications adapted from Williams and Shearer (2011), including: Research/Empirical – defined by the presence of an empirical ‘case study’ or analysis Theoretical Development/Debate – defined by the presence of conceptual arguments Normative Approach (generic) – defined as focusing not on what is, but what should be, with no specific or particular reference to a discipline or context Normative Approach (domain specific) – defined as focusing not on what is, but what should be, combined with a particular discipline or context Other (such as government and commission reports) Additional codes are used in the database, including, for example, document type and citation numbers, among others. Thus, all publications in the database to which we had access are coded for the following nineteen categories or criteria: 1. Author(s) 2. Title 3. Document type (e.g., journal article, book, book chapter, etc.) 4. Publication name 5. Volume and issue number 6. Year 7. Pages 8. Abstract 9. PVs focus or mention 10. Academic discipline/field 11. Distinction between public value and public values 12. Explicit definition 13. External definition (i.e., citation to other research) 14. Self definition 15. Context in which the term is used 16. Conceptualization of concept 17. Literature type 18. Number of Google Scholar citations 19. Number of Web of Science citations We are currently in the process of reviewing, discussing, and revising all of the coding for subjective categories and criteria to ensure inter-coder reliability. Preliminary Analyses As noted above, we found a total of 399 publications in our search; however, we do not have access to 67 publications or approximately 17% of the total documents identified. Thus, the following analyses are largely limited to the 332 publications to which we have access. Of these 332 accessible publications, 252 (76%) have public values as a ‘focus’ and 80 (24%) simply ‘mention’ public values. The overwhelming majority (293 or 88% percent) are journal articles. Book chapters (9) and book reviews (10) each represent about 3% of the accessible publications. Discussion papers and research reports (6), journal commentaries and editorials (5), and conference papers (5) each represent about 2% of the accessible documents. Finally, theses and dissertations (2) and other types of documents (2) each represent less than 1% of the accessible documents. Of the inaccessible publications, the majority (32 or 48%) are journal articles. Books (23) represent about 34% of the inaccessible publications, and book chapters (7) represent about 10% of the inaccessible publications. In addition, we are unable to access 3 research reports and 2 conference papers, which respectively represent 4% and 3% of the inaccessible publications. In looking at all of the publications, including those for which we have no access, the first finding of the term ‘public values’ dates back to 1969 in a book chapter titled, “The Public Values of the Private Association” (McConnell 1969). Interestingly, this publication (best situated within the scholarly domain of political science) is also the first time a distinction between public and private values is made. In the chapter, the author writes, “What I wish to do in this paper … is to look at the list of virtues attributed to the private association and then to ask what are the public values that ought reasonably to be expected from it” (p. 148). Further in his analysis, McConnell (1969: 160) asserts, The preeminent public values of the private associations … are order and stability. Perhaps to some degree the values of community, human warmth, and fellowship are also present in the private association. In the sense that order, stability, and mutual respect at a very minimal level are preconditions for liberty, this also is a value of the private association.... At the same time, however, the contribution to order and stability has come at a large cost. This cost has been paid, and is continuing to be paid in limitations on liberty, equality, and numerous other public values. The private association serves private as well as public values and it is proper that the right of men to associate should be protected. This is an aspect of individual liberty, and it is accordingly unnecessary to credit the association with virtues that are not its own. It has virtues that are real and some of these are public, but it is important to recall that these involve the payment of a price, and a large one. Since this first publication in 1969, the term PVs increasingly has made an appearance in the academic literature. Figure One shows the number of accessible publications that focus on or mention PVs between 1969 and 2012. There were only three years (1970, 1972, 1979) when this term did not appear in publications; however, it was not until the late 1990s that we see publication numbers in the double digits. Moreover, it is also interesting to note that almost 66% of the total publications on public values have been released since the year 2000. Figure One: Number of Publications that Focus On or Mention “Public Values”, 1969-2012 Although we are still in the process of revising and refining disciplinary categorizations, it is important to note that public values research is being conducted in numerous disciplines. As expected, the most publications (114) are in the field of public administration and management; however, unexpectedly, we also found significant numbers of publications in environmental studies (56) and law (51). There are also several publications in education, public health, and economics, among others. In total, we found publications on PVs within 41 academic subfields, including unexpected domains such as bioethics, environmental risk management, marketing and information studies, and librarianship, belonging to 18 recognized academic ‘disciplines’ such as law, political science, philosophy, public administration, and medicine. As noted earlier, we also coded the publications in our database in terms of whether an explicit definition is given, and if so, whether this definition is ‘self-created’ or given in reference to another publication. Shockingly, our preliminary review of the data shows that 245 (74%) of the publications do not provide an explicit definition of the concept. Of the remaining publications, 78 (24%) provide a single explicit definition, and 7 (2%) of the publications provide multiple definitions. We are currently in the process of analyzing the various definitions provided in these publications to identify common references and citations, ideas, and themes. In doing so, we hope to see patterns emerge in the overall narrative and overall evolution of PVs research (including details on ‘who cites whom’). A complicating matter in PVs research, is that many scholars do not explicitly distinguish between public values and public value (cf. Alford & O’Flynn 2009; Nabatchi 2011, 2012; O’Flynn 2009), while both concepts are only in some ways related and concern different debates and approaches (see Moore 1995, 2000 for a description of the concept ‘public value’). We therefore coded the accessible publications in terms of whether they distinguish between public values, and public value. Figure Two shows that 287 (86%) of the accessible documents have a distinct focus on public values, 35 (11%) focus on public value, and only 10 (3%) reference both concepts. This suggests that scholars are attending to the distinct meanings of these concepts. Figure Two: Distinction between Public Values and Public Value We also coded the accessible publications in terms of the literature type, using the categories defined by Williams and Shearer (2011): Research/Empirical, Theoretical Development/Debate, Normative Approach (generic), Normative Approach (domain specific), and Other. Table One shows our initial classifications of the literature type. Table One: Classification of Literature Type Research Empirical 116 Theoretical Development/Debate 95 Normative Approach (domain specific) 70 Normative Approach (generic) 46 Other 5 Total 332 35% 29% 21% 14% 2% 100% It is important to note that the Williams and Shearer (2011) classification scheme does not seem to work as well for the PVs literature as it did for the public value literature. More specifically, in attempting to classify publications, we found that few fit neatly into these existing categories, and that more often than not, publications overlapped categories. Table Two shows the overlapping classifications of the literature type. The issue of overlapping classifications is further evidence of the current conceptual muddle in public values research, and suggests that many authors are not clear about their aims, for example, whether the purpose of their publication is to develop theory or display a normative stance. We are currently in the process of refining the classification system for literature type. Table Two: Overlapping Classifications of Literature Type Research/Empirical (generic) 80 Research/Empirical & Normative Approach (domain specific) 15 Research/Empirical & Normative Approach (generic) 1 Research/Empirical & Theoretical Development/Debate 16 Research/ Empirical & Other 4 Theoretical Development/Debate 75 Theoretical Development/Debate & Research/Empirical 5 Theoretical Development/Debate & Normative Approach (generic) 15 Normative Approach (domain specific) 70 Normative Approach (generic) 46 Other/Unclassified 5 Total 332 24% 5% >1% 5% 1% 23% 2% 5% 21% 14% 2% 100% Finally, we conducted a citation search to find out which publications are cited most (the next step will be to find out ‘who cites whom’ so we can develop clusters of literature). Thus far, we have completed citation searches for 213 of the 399 publications in the database. We did a quick review of these numbers to determine which publications are most frequently cited according to Google Scholar. Four journal articles have over 300 citations, and three journal articles have between 200-299 citations. Interestingly, despite the majority of total publications coming from public administration and management, none of these publications fit squarely within that field/discipline. Instead, of the seven publications with more than 200 citations, four are from law (with 386, 304, 230, 229 citations), and one each is from health policy (310), economics (301), and environmental policy (210). Although the environmental policy article is published in the Journal of Policy Analysis and Management, its focus is on conflict in environmental planning issues. With 175 citations each, the most frequently cited articles that fall squarely within public administration and management are “Public value management: A new narrative for networked governance?” (Stoker 2006) and “Public value failure: When efficient markets may not do” (Bozeman 2002). Table Three shows the number of publications that fall within various ranges of citation numbers. Table Three: Citation Analysis of Publications # of citations # of publications Over 300 4 200-299 3 100-199 10 75-99 6 50-74 14 25-49 30 1-24 131 0 15 Total 213 In sum, our preliminary analysis of the public values literature reveals some interesting findings. First, of the 332 accessible publications, more than three-quarters have PVs as a focus, as opposed to simply mentioning the concept in relation to other topics. Second and perhaps not surprisingly given the nature of academia, over 80% of the 399 total publications we found (both accessible and inaccessible) are journal articles. Books and book chapters represent less than 10% of the total publications. Third, according to our search, the term PVs first appeared in a 1969 book chapter, which interestingly enough, articulated a distinction between public and private values in the two sectors. It is heartening (or perhaps disheartening depending on one’s view) to know that this historic debate has deep roots. Fourth, and related, since its appearance in 1969, the term PVs has received growing attention in publications, as evidenced by the steady increase in publications that focus on or mention the concept. Moreover, it is interesting to note that two-thirds of the publications we found were released since 2000. Fifth, although public administration and management are responsible for a large part of the public values research, we discovered significant publications on the topic in other disciplines such as environmental studies, law, education, public health, and economics, among others. Sixth, although this finding is extremely preliminary, we surprisingly found that nearly three-quarters of the publications do not explicitly define the term PVs – this is certainly problematic for the development of theory and research. Seventh, the literature about ‘public value’ seldom overlaps with the PVs literature. The two concepts are almost homonyms, but public value and public values are generally given clear and distinct meanings in the literature. Eight, our analysis indicates that there is wide variety in terms of the types of publications on PVs, with scholars examining, empirical, theoretical, and normative aspects of the concept. Finally, a preliminary citation analysis shows that the most frequently cited articles are in fields outside of public administration and management. Certainly more work needs to be done before we can make any strong claims about the nature of publications about PVs, and while useful, this preliminary analysis needs further development before we can being to understand the overarching narrative and evolution of the PVs concept. We discuss our next steps for this meta-analysis project in the conclusion of this paper. Conclusion We have only recently finished constructing the database of PVs publications, and only just begun very simple, preliminary analyses of the data. We have several ideas about moving forward with our meta-analysis. 1. We need to revise and refine various categorizations and classifications in the database and test for inter-coder reliability. 2. We need to more closely examine the categories of ‘focus’ and ‘mention’ to determine the varieties of foci and mentions out there. Moreover, we need to conduct this examination in light of the contexts and disciplines of the various publications. 3. We need to more closely assess the disciplinary approaches to the study of public values. Even though public administration and management are responsible for a large part of the publications, we need to examine other disciplines so that we can see what their approaches to the topic have to offer to our field. 4. We need to examine the definitions used and conduct citation analysis to identify common references, ideas, and themes, and to examine which groups of scholars cite each other and why. In doing so, we hope to see patterns emerge in the overall narrative and overall evolution of public values research. In conducting these and other analyses, we hope to go beyond simply identifying and naming approaches and perspectives; rather, we seek to distinguish, cluster, and map various ‘schools of thought.’ Doing so will (hopefully) help us develop better mutual understanding and integration among these schools. Finally, by doing this research, we hope to generate an overall narrative of the PVs research that shows its evolution over time. References Alford, J. and O'Flynn, J. 2009. Making sense of public value: Concepts, critiques and emergent meanings. International Journal of Public Adminsitration, 32, 71-91. Lotte Bøgh Andersen, Torben Beck Jørgensen, Anne Mette Kjeldsen, Lene Holm Pedersen and Vrangbaek, K. (2012). Public Values and Public Service Motivation: Conceptual and Empirical Relationships. The American Review of Public Administration published online 28 March 2012 Beck Jørgensen, T. 2006. Public values, their nature, stability and change: The case of Denmark. Public Administration Quarterly, 30(3): 365-398. Beck Jørgensen, T., & Bozeman, B. (2007). Public Values. An Inventory. Administration & Society, 39(3), 354-381. Beck Jørgensen, T. & Rutgers, M.R. 2011. Value dynamics in the public sector: A historical comparative analysis of changes in selection criteria for civil servants. Paper presented at the 2011 Public Management Research Association Annual Conference, Syracuse University, Syracuse, NY. Bennington, J. and Moore, M. (eds.) 2011. Public value: Theory and Practice. Basingstoke: MacMillan. Bozeman, B. (1987). All organizations are public: Bridging public and private organizational theories: Jossey-Bass. Bozeman, B. 2002. Public value failure: When efficient markets may not do. Public Administration Review 62(2): 145-161. Bozeman, B. (2007). Public Values and Public Interest. Washington: Georgetown University Press. Bozeman, B., & Bretschneider, S. (1994). The “publicness puzzle” in organization theory: A test of alternative explanations of differences between public and private organizations. Journal of public administration research and theory, 4(2), 197. Bruijn, H. d., & Dicke, W. (2006). Strategies for Safeguarding Public Values in Liberalized Utility Sectors. Public Administration, 84(3), 717-735. Buchanan, B. and Millstone, J. (1979). Public organizations: A value-conflict view. International Journal of Public Administration, 1(3): 261-305. Burright, Marian (2006) “Database Reviews and Reports Google Scholar -- Science & Technology”, Issues in Science and Technology Librarianship, (Accessed 1/5/12: http://www.istl.org/06-winter/databases2.html) Creswell, J. (2008). Educational research: Planning, conducting and evaluating quantitative and qualitative research (3rd ed.). Upper Saddle River, NJ: Pearson Prentice Hall. De Bruijn, H. D., & Dicke, W. (2006). Strategies for safeguarding public values in liberalized utility sectors. Public Administration, 84(3), 717-735. Deleuze, G., & Guattari, F. (1994). What is Philosophy? New York: Columbia University Press. Drewry, G., Greve, C., & Tanquerel, T. (2005). The Implications of Public Service Reform for Accountability and Audit—Recent Developments in the UK. Contracts, performance measurement and accountability in the public sector, 59. Frederickson, H. G. (2005). Public Ethics and the New Public Managerialism: An Axiomatic Theory. In H. G. Frederickson & R. K. Ghere (Eds.), Ethics in Public Management (pp. 165183). Armonk: M.E. Sharpe. Gallie, W.B. (1955). Essentially Contested Concepts. Proceedings of the Aristotelian Society 56: 167-198. Gregory, R. J. (1999). Social Capital Theory and Administrative Reform: Maintaining Ethical Probity in Public Service. Public Administration Review, 59(1), 63-76. Jacobs, J. (1992). Systems of Survival: a Dialogue on the Moral Faundations of Comerce and Politics: Random House. Kernaghan, K. (2003). Integrating values into public service: The values statement as centerpiece. Public Administration Review, 63(6): 711-719. McConnell, G. 1969.The public values of the private association. In J.R. Pincock and J.W. Chapman (eds.), Voluntary Associations, 147-160. New York: Atherton Press. Moore, M. (1995). Creating public value: Strategic management in government. Cambridge, MA: Harvard University Press. Nabatchi, T. 2010. Addressing the Citizenship and Democratic Deficits: Exploring the Potential of Deliberative Democracy for Public Administration. American Review of Public Administration, 40(4): 376-399. Nabatchi, T. 2011. Exploring the Public Values Universe: Four Frames for Understanding Values in Public Administration. Paper presented at the 2011 Public Management Research Association (PRMA) Annual Conference, Syracuse, NY. Nabatchi, T. forthcoming 2012. Putting the ‘Public’ Back in Public Values Research: Designing Public Participation to Identify and Respond to Public Values. Public Administration Review, _(_): __-__. Nieuwenburg, P. (2004). The Agony of Choice: Isaiah Berlin and the Phenomenology of Conflict. Administration & Society, 35(6), 683-700. O’Flynn, J. (2009) The Public Value Debate: Emerging Ethical Issues. Public Leadership Workshop, 26-27 November, The Australian National University, Canberra. Osborne, D., & Gaebler, T. (1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector; Reinventing government: How the entrepreneurial spirit is transforming the public sector: Addison-Wesley Publishing. Reijniers, J. (1994). Organization of public-private partnership projects:: The timely prevention of pitfalls. International Journal of Project Management, 12(3), 137-142. Robinson & Reed (1998). The A-Z of Social Research Jargon. Ashgate Publishing. Rosenau, P. V. (2000). The strengths and weaknesses of public-private policy partnerships. Public-private policy partnerships, 217-242. Rutgers, M. R. (2008). Sorting out public values? On the contingency of value classifications in public administration. Administrative Theory & Praxis, 30(1): 92-113. Schreurs, P. (2005). ‘The Value(s) of Public Administration.’ Administrative Theory & Praxis 27 (2): 301-310. Shapiro, M. (1992). Reading the Postmodern Polity. Minneapolis: University of Minnesota Press. Steenhuisen, B. (2009). Competing public values. Coping strategies in heavily regulated utility industries, Next Generation Infrastructures dissertation, Delft. Steenhuisen, B., W. Dicke, and H. de Bruijn (2009). “Soft” public values in jeopardy: Reflecting on the institutionally fragmented situation in utility sectors. International Journal of Public Administration, 32(6): 491-507. Stoker, G. 2006. Public value management: A new narrative for networked governance? American Review of Public Administration 36(1): 41-57. Tait, J. (1997). A Strong Foundation: Report of the Task Force on Public Service Values and Ethics (A Summary). Canadian Public Administration, 40(1), 1-22. Van der Wal, Z. (2011). The Content and Context of Organizational Ethics. Public Administration 89 (2): 644-660. Van der Wal, Z., L.W.J.C. Huberts, J.H.J. van den Heuvel & E.W. Kolthoff (2006). Central Values of Government and Business: differences, similarities, and conflicts. Public Administration Quarterly 30 (3): 314-364. Van der Wal, Z. & L.W.J.C. Huberts (2008). Value Solidity in Government and Business. Results of an Empirical Study on Public and Private Sector Organizational Values. American Review of Public Administration 38 (3): 264-285. Van der Wal, Z. (2008). What’s valued most? Similarities and differences between the organizational values of the public and private sector. Public Administration, 86(2), 465482. Van Wart, M. (1998). Changing public sector values. New York: Garland Publishing. Walzer, M. (1973). Political Action: The Problem of Dirty Hands. Philosophy & Public Affairs, 2(2), 160-180. Weihe, G. (2008). Public-Private Partnerships and Public-Private Value Trade-Offs. Public Money & Management, 28(3), 153-158. Williams, I. and Shearer, H. 2011. Appraising public value: Past, present, and futures. Public Administration, 89(4): 1367-1384.