Chapter 12: Science education for citizenship: introducing the discussion of socioscientific issues into the curriculum Stein Dankert Kolstø, University of Bergen, Norway Many commentators on the school science curriculum argue that this should nowadays include some opportunities for students to discuss current issues and controversies concerning the application of science and technology , of the kind which appear regularly in the news media. This chapter focuses on the impact and effects of short teaching interventions (‘consensus projects’) designed to improve students’ capability to argue and reason, using scientific information. The background and rationale for the intervention is explained, and the implementation described. The relevance of the project’s aims and the foci of the evaluation are discussed. Based on this discussion the author explores general issues concerning the evaluation of curriculum innovations with ‘citizenship’ aims of this sort. A main focus in the discussion is the role of values and personal perspectives in students’ dealing with socio-scientific controversies and consequences for evaluation of students’ learning outcome. Summary box: Chapter title: Science education for citizenship: introducing the discussion of socio-scientific issues into the curriculum (Stein Dankert Kolstø) What is the innovation? A small-scale (13 lessons) curriculum project called “Consensus projects” aiming at science for citizenship implemented in a general science course in upper secondary school. What need does it address? The importance of preparing students to deal thoughtfully with socio-scientific controversies encountered in daily life. What is its scope? Based on a teaching model proposed in the literature the Consensus project model was implemented in a science class in Norway in their compulsory science course. What evaluation data has been collected? The discussion in the article is based on the described objectives for the project and on the teacher’s evaluation report. The teacher’s report focused on students’ issue knowledge, participation in discussions and personal decision-making. What are the key findings? The students increased their issue knowledge and took actively part in discussions related to a controversial issue, including questions of the reliability and validity of knowledge claims. In evaluating the relevance of the project it was found that prerequisite knowledge needed to examine the controversial issue was under-emphasised. It was also found that evaluation of students’ dealing with socioscientific controversies needs to relate to the value aspects of personal decision-making in a more explicit manner. Dankert’s York 2 artikkel 1 Versjon pr 06.03.16 Chapter 12 Science education for citizenship: introducing the discussion of socioscientific issues into the curriculum Stein Dankert Kolstø Society today is faced with a number of controversies with a science dimension. Ability to examine information related to socio-scientific issues have been recognised as an important aim for science education (see i.e. Millar & Osborne, 1998; NRC, 1996). During the last decade there have been several curriculum initiatives aiming at bringing controversial issues and decision-making into the science classroom (Christensen & Kristensen, 1999; Eijkelhof, 1990; Fullick & Ratcliffe, 1996; Gayford, 1993; Geddis, 1991; Harms, Kross, & Bayrhuber, 1997; Kolstø, 2000a; Kortland, 1996; Lewis, Leach, Wood-Robinson, & Driver, 1997; Ratcliffe, 1996; Solomon, 1992; Waarlo, 1997). The evaluation of curriculum projects aiming at science for citizenship is a new area for science education. With an emphasis on students’ decision-making skills and relevance for dealing with the science dimension of controversies such projects poses new challenges to evaluation. The focus of science for citizenship projects might differ according to the developers’, and implementers’, perspective on controversies, on the nature of science and on science society interactions. However, if the aim is to support students’ analysis and decision-making on socio-scientific issues, not all perspectives will be of equal relevance to this goal. It is therefor important to evaluate the relevance of a project’s perspective and articulated learning goals in addition to the evaluation of students’ learning outcome. In this article I will present and discuss a Norwegian curriculum project aiming to foster students' ability to make knowledge-based decisions on socio-scientific controversies. The project focuses on examination of the science dimension of controversial issues through a teaching model called "Consensus projects" (Kolstø, 2000a). Following a description of the Consensus project model I will present an evaluation made of this curriculum initiative. This evaluation indicates the existence of several challenges for the evaluation of science for citizenship projects. As a response to these challenges I will describe a perspective on science education for citizenship which is based on Aikenhead’s (1985) concept of “thoughtful decision making” as the main goal for a science curriculum for citizenship. Aikenhead warns that a potential pitfall when trying to develop students' decision-making skills is not to include necessary prerequisite knowledge about the characteristics and limitations of science and technology. This aspect will be included in the perspective described through an emphasis on characteristics of the science dimension of controversial issues typical given media coverage. Building on the perspective outlined, I will suggest a framework describing main learning goals on which to build evaluations of science for citizenship curriculum projects. The framework enables an identification of challenges related to evaluation of such projects. It also enables a discussion of strengths and weaknesses of the evaluation made of the Consensus project. The basic claim in this article will be that the evaluation of science education for citizenship curriculum projects presupposes specification of both perspective and learning goals. Only by doing that an evaluation might gives insights in whether we have done things better regarding both learning outcome and the relevance of these. Dankert’s York 2 artikkel 2 Versjon pr 06.03.16 Consensus projects The "Consensus project" is a teaching model developed by this author and aiming at providing the students with "experiences, knowledge, skills and attitudes that empower them to deal with science, expert statements and knowledge claims with a science dimension emerging in socio-scientific issues" (Kolstø, 2000a p.651). The implementation to be reported here was a small-scale curriculum project in a lower secondary school in a suburban area on the West Coast of Norway. The science teacher implemented the teaching model in the general science course. As part of a university level course in science education the same teacher made a summative evaluation of the project on which I will base my discussion. A main goal in Consensus projects is to train the students in information-based argumentation and evaluation of knowledge claims. Through practising this it is hoped that students will appreciate to study and evaluate information prior to their decision-making, including scientific subject matter. It is also hoped that the students will understand why conflicting views exist. A final goal is to make the students understand and appreciate that disagreement and debate are legitimate and important aspects of frontier science. The teaching strategy used in the consensus project model is a combination of project work in the tradition of Dewy and Kilpatric and the Danish consensus conference model for democratic technology assessment (see Kolstø, 2000a). The controversy selected was the debate on whether Norway should allow the building of power plants using natural gas, making it more difficult for Norway to fulfil the Kyoto agreement. The class was divided into groups, which were to become "experts" on different aspects of the issues, including scientific, technological and socio-scientific dimensions. All groups were to present their findings at a "conference" in the science classroom. One group, the "lay group" had a special task and should scrutinise the other groups' presentations, and make a report presenting a consensual view on the issue, including recommended action. The groups were assigned roles (as subject matter experts, politicians, industry involved, and environmental group), allowing them to promote ideas without having to be personally convinced. The conference, lasting two school lessons, was supposed to be a kind of "summit" where all arguments from different perspectives met. The teacher chaired the conference, and had an important role in exemplifying how to ask scrutinising questions. After the conference all groups had to make a report on the information and arguments they had found and presented. Learning outcome The students filled in a questionnaire prior to and three weeks after the project. Beforehand only one half of the students knew that "gas-plants" produce electric power and they knew very few arguments. Three weeks afterwards, all the students knew what the issue was about, and knew several of the arguments involved. The most prominent finding, however, was that most students actively participated in debating the arguments put forward at the conference. From the debate and the reports from the different groups it could be seen that the students used arguments which included scientific concepts. It is not possible to state whether they understood the concepts they used, but they were able to use them in adequate ways in their arguments. Concerning personal decision making at the end of the project, fewer students had a clear opinion than beforehand. They stated there were good arguments "on both sides", which indicates that the students learned about more arguments and attained increased respect for a conflict of interests. The teacher reported that questions related to reliability or validity of knowledge claims was frequently discussed during the project's conference (even tough the value aspect of the issue and the desirability of the different alternatives were more prominent). Trough these Dankert’s York 2 artikkel 3 Versjon pr 06.03.16 discussions issues related to competence, expert disagreement, frontier science and possible interests were illuminated. There were no obvious signs of increased understanding of the nature of science based on this project. This was probably due to the lack of an explicit focus on such knowledge. Issues related to both social processes in science and science’ social relations nevertheless seem to have been debated. Also the students' competence and confidence in their own ability to formulate arguments and to state a view in public probably improved. In fact several students expressed their joy in participating at the conference both immediately afterwards and unsolicited on later occasions. The strength of the project was probably its focus on the need for both information-basis and evaluation of knowledge claims in relation to both the societal and scientific aspect of a controversy. Findings from the frontier of science were thus debated inside the classroom. The processes undertaken therefore probably exemplified critical examining of knowledge claims involved in a controversy and thus provided the students with the basic idea of what constitutes a critical attitude. The combination of expert groups and a conference probably also implied practice in the reworking (Layton, 1991) of information from different knowledge domains to articulate with complex real-world issues. The relevance of the consensus project In taking a critical look at the evaluation of the Consensus project, I will focus on the relevance of the foci for the evaluation. Thus I will not examine issues related to reliability and validity of evaluation methods used. The foci for the evaluation of the project was the students’ understanding of what the issue was all about, the students participation in discussions related to arguments, the development of personal opinions on the issue and the reliability of scientific knowledge claims. (The evaluation also focused upon the students’ learning of science concepts and on motivation, but these aspects were not judged to be relevant for this article.) The question then arises as to what extent these four foci represent science for citizenship. The evaluation of the outcome of the implementation of the Consensus project model was not based on a specified framework describing main learning goals related to science for citizenship. The implication is that even if some relevant learning seems to have taken place, it is difficult to evaluate the relative importance of the outcome in relation to the stated general goal of science for citizenship. The evaluation of the project could probably have benefited from a clarified view on the concept of science for citizenship and the relevance of different learning goals. In the following I will present my suggestions in this regard, and use this to identify strengths and weaknesses with the Consensus project. A framework for evaluation of curriculum projects The concept of science for citizenship might be defined as related to the cultural and the democratic arguments for science education for all (see i.e. Driver, Leach, Millar, & Scott, 1996; Millar, 1996; Sjøberg, 1997). The cultural argument states that science is an important part of our culture and has contributed to our worldview and our way of living. Science is therefore an important frame of reference in our culture, and thus relevant to include in compulsory education. The democratic argument is based on the fact that many issues in public and private life have a science dimension. Knowledge in science might increase the Dankert’s York 2 artikkel 4 Versjon pr 06.03.16 quality of peoples’ evaluations and decisions in such socio-scientific issues. In this article I will focus on consequences of the democratic argument for science education for citizenship. I will take as a premise that science for citizenship should be responsive to both society’s need for knowledge-based decisions and to empower the laity to analyse and act in accordance with personally held perspectives. The concept of thoughtful decision-making proposed by Aikenhead (1985) might here be seen as a concept unifying these two different aims. This concept includes aspects of decision-making related to values and ideology, in addition to emphasising the role of knowledge, evaluation and argumentation. This broad definition with inclusion of value-related aspects will, however, result in challenges when it comes to the evaluation of students’ learning outcome. To fulfil their purpose, learning goals for a science curriculum for citizenship have to be relevant in the context of socio-scientific issues. An understanding of these real-world contexts will therefor guide the development of curriculum projects for citizenship. Because of its central position I will in short state a view of important characteristics of socio-scientific issues. Socio-scientific issues A characteristic aspect of socio-scientific issues is the frequent focus on a possible risk to the environment or to human health. Socio-scientific risk issues also often evolve into controversies. The science dimension of socio-scientific issues is typically related to the potential risk. An important aspect of such risk-focused controversies is that they often contain not only one, but two central issues. One issue is the political, ethical, societal or personal question on what action to take, e.g. "Should it be legalised to put irradiated food on the marked?". The second issue relates to the science dimension. In many risk-focused controversies a scientific question related to e.g. the existence or size of the risk is not clarified or may be disputed. In the "food-irradiation issue" the scientific sub-issue could be: "Does the nutritional value of food become reduced by irradiation?". Controversies where an important scientific sub-issue is unresolved might be denoted as "double issues" (Kolstø, 2001). In double issues it will be relevant to look up results from the frontier of science when trying to argue for a specific view. Ongoing research from the frontier of science, however, is usually not emphasised in compulsory school science. Several scholars have claimed the existence of two different types of science (Bauer, 1994; Cole, 1992; Latour, 1987; Ziman, 1991). On the one hand, we have ongoing research at the frontier of science. Hypothesis and results from this research will typically be disputed within the scientific community. This science might be denoted as "frontier science" (Cole, 1992) or "science-in-the-making" (Latour, 1987). On the other hand, we have the established scientific knowledge typically found in textbooks. This science is consensual and regarded as neutral, objective and reliable within the relevant discipline. Cole denotes this science as "the core" of science and Latour denotes it as "ready-made-science". Subjective and unreliable frontier science is transformed into core science, or out of science, through different social processes which includes publication, criticism and argumentation. This description implies that the reliability of scientific knowledge claims varies. Awareness of this variation is important when examining controversies. This variation, and the fact that scientific knowledge is based on argumentation, in addition to empirical evidence, also makes it relevant for ordinary citizens to examine the reliability of scientific knowledge claims. When the scientific dimension of a double issue is disputed, it is relevant to look up results from the frontier of science when trying to argue for a specific view. The result is that tentative scientific knowledge claims are required, and actively used, by actors in Dankert’s York 2 artikkel 5 Versjon pr 06.03.16 controversies. Claims from the frontier of science are thus applied without waiting for the knowledge claims to be evaluated and judged reliable or dead-ends by a scientific community. Awareness of characteristics of knowledge claims from the frontier of science thus becomes even more important. However, it is also important to be aware that scientific knowledge is not produced in a social vacuum. Neither is the old picture true anymore where science were seen as produced at universities by free schoolars with the development of a dicipline as the main objective. Aikenhead (1994) claims that science has been undergoing a process of “socialisation” whereby "Government, industry, and the military have become the dominant patrons of scientific activity." (p.16). This “new” science has been denoted by Ravetz (1995) as “industrialised science”. The typical scientist has lost his independence, and has become an employee or a contractor. The typical scientist thus works either in industry or governmental agencies, or has to make dispositions that might give him research contracts. In industry the competition in the market might result in less openness towards methods and less emphasis on publication and open debate. Regarding risk issues, industry, governmental agencies and others might have interests connected with certain results. This could for instance be findings showing that no, or only small risks are connected to certain technologies and practices. The question then emerges as to whether scientific research, or communication of research results, can be influenced by interests. The fundamental underdetermination of theory by measurements indicates that these possibilities cannot be excluded. If we combine the use of contract-based research to provide arguments, hopefully in line with one's own interests, with the use of results from single studies in the public debate, we get a situation where important social processes in science are overthrown. This leaves more of the evaluation of reliability of results to the lay man and woman, and thus also greater demands on curriculum projects focusing on science for citizenship. This view of modern science is also echoed by studies of lay people's relation to science (Irwin, 1995; Irwin & Wynne, 1996; Layton, Jenkins, Macgill, & Davey, 1993). The view of science as "a force to be struggled against" (Irwin, 1995 p.46) sometimes found in these studies, indicates that the neutrality of science and scientists is not taken for granted anymore. Thoughtful decision-making as a learning goal The inclusion of the concept ‘thoughtful’ in relation to decision-making indicates that science for citizenship is not only about arriving at a personal decision, but also includes the quality of the decision-making process. A basic claim in this article is that decision-making on controversial socio-scientific issues is a complex task. As the descriptions in the above paragraph indicates, such issues typically involves several open questions, different kinds of scientific knowledge claims, several different actors with different interests, and knowledge from several social domains and disciplines might be relevant. In such a complex landscape it is highly desirable that students in a decision-making situation examines the issue in order to acquire a firm knowledge base for their evaluations. Decision-making involves the comparison and weighting of arguments and values involved. However, the presence of disputed questions and different actors with different interests implies that decision-making also include decisions on what information to trust. Ability to evaluate the relevance and trustworthiness of claims with a science dimension therefore becomes necessary. As in science, argumentation is a profound tool to clarify viewpoints, to examine the validity of a claim or an argument, and to communicate ones’ viewpoints to decision-makers or a Dankert’s York 2 artikkel 6 Versjon pr 06.03.16 wider public. Skills in argumentation therefor need to be included in the concept of thoughtful decision-making. The concept of thoughtful decision-making thus need to involve the following three main aspects: Knowledge-based examination of a controversy, to gain understanding of the issue and alternatives, arguments and values involved. Personal evaluations based on the outcome of the examination, and focusing on the relevance and trustworthiness of arguments (Kolstø, 2000b), evaluation of alternatives (Janis & Mann, 1977) and personally held values (Ratcliffe, 1994), and hopefully also including a personal decision. Argumentation in relation to viewpoints involved in the controversy. Although these three aspects are process-related, they necessarily involve both 1. prerequisite general knowledge, 2. issue related knowledge based on a personal examination of the controversy, 3. skills related to the performance of the examination, evaluation and argumentation, and finally 4. attitudes which underpin these processes. Each of these four dimensions of knowledge involves a challenge to the evaluation of science for citizenship projects that need to be discussed. Challenges related to evaluation of prerequisite knowledge Prerequisite knowledge relates to knowledge needed to perform an informed examination the science dimension of a controversy. As Aikenhead (1985) have warned us, if this examination is to be meaningful it has to be based on some understanding of the nature of science and science-society interactions. These learning goals are fact-oriented as long as we do not include the ability to apply this knowledge when examining controversies or in other contexts. Thus they will not represent any challenge to external evaluators of curriculum projects. They might, however, represent a challenge to teachers as they imply evaluation of partly new subject areas. In the evaluation of the Consensus project this dimension was missing. With no knowledge of students’ ideas about science and science-society interactions it is difficult to evaluate students competency in applying this knowledge in the context of the controversy. One could of course have gathered data both on aspects of students’ knowledge in, and about, science, and of the science dimension of their issue knowledge. By comparing two such sets of data it will probably be possible gain insights into the students ability to apply prerequisite knowledge in a complex real-world controversy. Challenges related to identification of issue knowledge Thoughtful decision-making obviously presupposes issue knowledge; knowledge related to the controversy to be dealt with. To gain issue knowledge the decision-maker has to examine the controversy using his or her general knowledge (for instance “prerequisite knowledge”). This examination involves a process, the examination itself, and a product, issue knowledge. This product might include knowledge of alternatives involved, main arguments, main actors and their views, interests and values, possible risks involved, and the disputed and notdisputed scientific knowledge claims involved. The outcome of the evaluation, issue knowledge, will to some extent be a consequence of the personal perspective used by the student. Thus this issue knowledge will not be a collection of “facts” agreed upon by all Dankert’s York 2 artikkel 7 Versjon pr 06.03.16 involved actors, by the student’s teacher or by an external evaluator. The evaluation of this examination therefore involves a challenge due to the subjective dimension of the students’ issue knowledge. One choice when performing an evaluation of the students examination skills could be to focus on the outcome of the examination process, neglect the subjective dimension, and compare the outcome with a standard established by the evaluators. The evaluators might for instance state that four alternative actions are involved in the controversy, and use a list of seven main arguments involved. Another choice could be to focus on the process itself, and evaluate the presence of examination activities and results of the examination. In the evaluation of the consensus project the first strategy was used through the use of a pre and post questionnaire focusing on issue knowledge. However, the questionnaire and the analysis of students’ answers did not allow for variations in the knowledge gathered due to differences in students’ values and perspectives. The second strategy was also used to some extent through the teacher’s observation of the whole-class debate, but the content, the issues knowledge gained through the examination, was not given attention here. Challenges related to evaluation of process skills An important aspect with decision-making is the personal evaluation of arguments and alternatives. These evaluations might relate both to relevance and trustworthiness of arguments and knowledge claims, and to the comparison and weighting of arguments, values and alternatives. Learning goals related to such evaluations focus on the development of skills. The evaluation of students’ skills in this regard involves a challenge, as some of the students’ evaluations will be based on values not agreed upon. There exists no neutral judgement regarding the trustworthiness of a knowledge claim or the desirability of different alternatives. People put different trust in different sources of information for instance. One option could her be to focus on presence of evaluation processes and its outcome, and neglect the content of the evaluations. Ability to present arguments in debates also involves skills. Examples here could be ability to identify and articulate own views, to use evidence to underpin a claim. A question here arises as to whether one should evaluate the quality of the arguments and the evidence put forward, or whether to focus on the presence of articulated arguments and evidence. The neglect of content implies saying that no quality differences exist between different evaluations and arguments. This is not a comfortable position. The dilemma between content and process might be “solved” through focusing on process in formal evaluations, and on content during the teaching process, leaving room for the teacher to challenge students based on quality oriented feedback. In the Consensus project the students had to write reports presenting the controversy, give an overview of arguments involved, and explain a conclusion to be drawn. This report represented an opportunity to study the quality of the students’ evaluations. The study of quality of evaluations would have presupposed the development of a framework for the analysis which defines high and low quality evaluations. The challenge remains, however, of finding quality indicators that do not favours certain perspectives and values. Concerning students’ argumentation skills the evaluation of the Consensus project focused on the presence of argumentative activities. By looking at the students’ reports, or by taping the discussion, it would have been possible to analyse the quality of the arguments put forward. To avoid an analysis biased by the evaluators perspective on the issue this analysis probably have to focus on the presence of certain quality indicators like use of evidence or warrants. Dankert’s York 2 artikkel 8 Versjon pr 06.03.16 The works of Driver, Newton and Osborne (2000), based on Toulmin’s argumentation structure, represents an interesting opportunity here. Challenges related to evaluation of underpinning attitudes If students are to perform knowledge-based examination also outside school, it has to be embedded in attitudes which leads, underpins or stimulates to this. Examples of such attitudes might be open-mindedness combined with scepticism, and curiosity and interest to understand. Evaluation of knowledge claims have to be underpinned by an attitude towards knowledge claims, or epistemology, where claims are not seen as true or false, but as supported by arguments. Other attitudes underpinning thoughtful evaluation might be willingness to consider both pros and cons. Also the evaluation dimension therefore need to involve goals related to attitudes. Concerning the goal to increase students’ability to participate in debates and argumentation we might want to develop the students’ attitudes to include a commitment to evidence and to values. Ramsden (1998) states, that attitudes “cannot be measured directly, but only inferred from words and action” (p.128). However, in relation to thoughtful decision-making it is not the attitudes per se we are interested in, but the consequences for processes of examination, evaluation and argumentation involved. It might therefore be wise to base the evaluation related to attitudes on data from the decision-making process, and focus on the consequences for the students’ evaluations and decisions more than on the attitudes themselves. Open-mindedness and willingness to consider both pros and cons might thus become studied by focusing on the presence of, and the time spent, on views and arguments not in line with the student’s own sympathies. This strategy will also make it possible, through the use of such quality indicators, to make an analysis that is responsive to the students’ own positions. In the evaluation of the Consensus project the issue of attitudes was only briefly touched upon. It was found that fewer students held a clear opinion on the controversy towards the end of the project, and that several students explained that they now saw good arguments on both sides. Together these findings were taken as an indicator for students’ attitudes regarding willingness to consider both pros and cons, and thus represent an example of the use of perspective independent quality indicators. In 1982 Biggs and Collis (1982) proposed a taxonomy for assessment purposes called Structure of Observed Learning Outcome (SOLO) based on Piagetian stage theory. The SOLO taxonomy was devised as a subject independent tool for the evaluation of learners’ response to open-ended questions. As quality indicators the SOLO taxonomy uses the student’s ways of using information. Thus is provides an interesting opportunity for assessing the quality of students’ examination, evaluation and argumentation in relation to socioscientific controversies. However, more knowledge is needed on whether students’ different values and perspectives will affect the evaluation using this taxonomy. Conclusions Above I have tried to identify challenges related to evaluation of curriculum initiatives aiming at science education for citizenship. Through a discussion based on Aikenhead’s concept of thoughtful decision-making I have identified two main challenges. The first relates to the relevance of the perspective from which the project evaluation is performed. I suggest that the Dankert’s York 2 artikkel 9 Versjon pr 06.03.16 evaluator’s perspective regarding the concept of science for citizenship need to be explicated, and need to be included in the foci for the evaluation. In evaluating this perspective its relevance for the context in focus, socio-scientific issues, need to be emphasised. My second main point relates to the role of personal held values in the decision-making process. I have claimed that values influence both the student’s examination of a controversy, evaluation of knowledge claims, arguments and alternatives, and argumentation in relation to a controversy. The challenge then becomes to find ways of evaluating the quality of the students’ decision-making without evaluator’s perspectives to bias the evaluation. One way to tacle this situation is to focus on the presence of certain activities, for instance the presence of debate and argumentation. However, it seems superficial to state that a decision is thoughtful only because an examination of the controversy, an evaluation of information involved and a debate over viewpoints were made. My suggestion here is to identify quality indicators that focus in more specified ways on the content or the quality of the processes. This strategy will not make the evaluation a value-free activity. The quality indicators will represent a value-based perspective. But the quality indicators have to be sought made so that their sensitivity to values influencing students’ decision making in socio-scientific controversies are minimised. References Aikenhead, G. S. (1985). Collective decision making in the social context of science. Science Education, 69(4), 453-475. Aikenhead, G. S. (1994). The Social Contract of Science. In J. Solomon & G. Aikenhead (Eds.), STS Education. International Perspectives on Reform (pp. 11-20). New York: Teachers College Press. Bauer, H. H. (1994). Scientific Literacy and the Myth of the Scientific Method. Urbana, Illinois: University of Illinois Press. Biggs, J. B., & Collis, K. F. (1982). Evaluating the Quality of Learning. London: Academic Press. Christensen, K. G., & Kristensen, T. (1999). Fra naturfag til politikk. Mot et paradigmeskifte i miljøundervisningen? (From Science to Politics) (Forskningsrapport fra MUVIN 2 i Norge ). Tønsberg: Statens Utdanningskontor i Vestfold. Cole, S. (1992). Making Science. Between Nature and Society. Cambridge, Massachusetts: Harvard University Press. Driver, R., Leach, J., Millar, R., & Scott, P. (1996). Young Peoples' Images of Science. Buckingham: Open University Press. Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84(3), 287-312. Eijkelhof, H. M. C. (1990). Radiation and Risk in Physics Education. Utrecht: Centre for Science and Mathematics Education. Fullick, P., & Ratcliffe, M. (Eds.). (1996). Teaching Ethical Aspects of Science. Southampton: The Bassett Press. Gayford, C. (1993). Discussion-based group work related to environmental issues in science classes with 15-year-old pupils in England. International Journal of Science Education, 15(5), 521-529. Dankert’s York 2 artikkel 10 Versjon pr 06.03.16 Geddis, A. N. (1991). Improving the quality of science classroom discourse on controversial issues. Science Education, 75(2), 169-183. Harms, U., Kross, A., & Bayrhuber, H. (1997, 2. - 6. September). Teaching gene technology to secondary school students. Paper presented at the First International Conference of the European Science Education Research Association (ESERA), Rome, Italy. Irwin, A. (1995). Citizen Science - A Study of People, Expertise and Sustainable Development. London: Routledge. Irwin, A., & Wynne, B. (Eds.). (1996). Misunderstanding Science? The Public Reconstruction of Science and Technology. Cambridge: Cambridge University Press. Janis, I. L., & Mann, L. (1977). Decision Making. New York: Free Press. Kolstø, S. D. (2000a). Consensus projects: teaching science for citizenship. International Journal of Science Education, 22(6), 645-664. Kolstø, S. D. (2000b). "To trust or not to trust, ..." Students' ways of dealing with a socio-scientific issue. International Journal of Science Education, 23(9), 877-901. Kolstø, S. D. (2001). Science Edication for Citizenship - Thoughtful Decision-Making About ScienceRelated Social Issues ( Vol. 1). Oslo: Unipub forlag. Kortland, K. (1996). Decision-making on science-related social issues: the case of garbage in physical science - a problem-posing approach. In G. Welford & J. Osborne & P. Scott (Eds.), Research in Science Education in Europe. Current Issues and Themes (pp. 115-124). London: Falmer Press. Latour, B. (1987). Science in Action: How to Follow Scientists and Engineers through Society. Milton Keynes: Open University Press. Layton, D. (1991). Science education and praxis: the relationship of school science to practical action. Studies in Science Education, 19, 43-79. Layton, D., Jenkins, E., Macgill, S., & Davey, A. (1993). Inarticulate science? Perspectives on the Public Understanding of Science and Some Implications for Science Education. Nafferton: Studies in Education Ltd. Lewis, J., Leach, J., Wood-Robinson, C., & Driver, R. (1997, 2. - 6. September). Genetic Engineering - The Limits: discussion by 15-16 year old students on the acceptable uses and limitations of genetic engineering. Paper presented at the First International Conference of the European Science Education Research Association. Millar, R. (1996). Towards a science curriculum for public understanding. School Science Review, 77(280), 7-18. Millar, R., & Osborne, J. (1998). Beyond 2000: Science Education for the Future. London: Nuffield Seminar Series: Interim Report V3. NRC. (1996). National Science Education Standards. Washington, DC: National Academy Press. Ramsden, J. M. (1998). Mission impossible?: Can anything be done about attitudes to science? International Journal of Science Education, 20(2), 125-137. Dankert’s York 2 artikkel 11 Versjon pr 06.03.16 Ratcliffe, M. (1994, 23-31 August). Decision-making about science-related social issues. Paper presented at the 7th IOSTE Symposium, Enschede, The Netherlands. Ratcliffe, M. (1996). Adolescent decision-making, by individuals and groups, about science-related social issues. In G. Welford & J. Osborne & P. Scott (Eds.), Research in Science Education in Europe (pp. 126-140). London: Falmer Press. Ravetz, J. R. (1995). Scientific Knowledge and its Social Problems ( 2nd ed.). New Brunswick, NJ: Transaction Publishers. Sjøberg, S. (1997). Scientific literacy and school science. Arguments and second thoughts. In E. Kallerud & S. Sjøberg (Eds.), Science, Technology and Citizenship. The Public Understanding of Science and Technology in Science Education and Research Policy (pp. 9-28). Oslo: Norwegian Institute for Studies in Research and Higher Education. Solomon, J. (1992). The classroom discussion of science-based social issues presented on television: knowledge, attitudes and values. International Journal of Science Education, 14(4), 431-444. Waarlo, A. J. (1997, 2-6. September). Biology students' forming and justifying of opinions on predictive genetic testing. Towards a practicable and effective teaching strategy. Paper presented at the First International Conference of the European Science Education Research Association, Rome, Italy. Ziman, J. M. (1991). Reliable Knowledge: An Exploration of the Grounds for Belief in Science. Cambridge: Cambridge University Press. Dankert’s York 2 artikkel 12 Versjon pr 06.03.16