Debating Science 2008 Survey Report, 8/17/09
Report on 2008 Debating Science Program Survey
August 17, 2009
Compiled and summarized by:
Renee Madathil, B.A.1
Blake Francis, B.A.2
David Schuldberg, Ph.D.3
Contact Information:
Department of Psychology
The University of Montana
Missoula, MT. 59812-1584
406-243-4521
rm118724@grizmail.umt.edu
blake.francis@umontana.edu
david.schuldberg@umontana.edu
1
Department of Psychology, The University of Montana
2
Department of Philosophy and The Center for Ethics, The University of Montana
3
Department of Psychology, The University of Montana
1
Debating Science 2008 Survey Report, 8/17/09
Executive Summary
This survey report comprises a summary of responses from the summer 2008 Debating
Science Program, A New Model for Ethics Education. The New Model for Ethics Education
involves an approach which guides deliberation through a logical structure designed to foster
informed and thoughtful discussion. The Debating Science Program has two connected goals: to
educate students about the scientific, ethical, and social issues involved in science debates and to
teach students to develop habits of ethical deliberation. The weeklong workshop in Montana
provided the students with the opportunity to think about and discuss scientific, ethical and social
issues through attending expert lectures on numerous topics and small group sessions based on
interest. During the semester long online course students participated in deliberation on the
“Online Deliberation Center.” The ODC guides deliberation using a logical structure by asking
students to consider the social goals, alternatives, obstacles, and side effects. This survey is
intended to evaluate the extent to which the Debating Science program has met these goals.
Prior to the summer workshop, participants expressed general agreement about
deliberation and science policy as well as the quality of public science debates. Several
respondents rated the quality of current public debates over science policy as „bad.” In
elaborating this response participants discussed the public‟s low level of science education and
the poor quality of the scientific information available to the public. Several others mentioned
the nature of science, and expressed concern over the capacity of scientists to engage the public.
Overall, the majority of respondents reported that scientists should play a role in science debates
and the vast majority envisioned being involved in public policy deliberation during their
careers. Most participants rated their preparedness to enter into public debates about science
2
Debating Science 2008 Survey Report, 8/17/09
policy with multiple stakeholders as neutral (44%) or good (35%). The majority of respondents
indicated that they were not involved at all as an active participant in public debates, while
several (42%) indicated they were somewhat involved.
After the workshop, students reported having a positive experience. Participants
mentioned the lectures, the quality of the speakers, the range of lecture topics, and the
effectiveness of group work. With respect to group work, specifically, several respondents
mentioned the benefits of working with a diverse group of students. Suggested improvements
included more time for discussion, the need for introductory science lectures, more in depth
science lectures, and better defined goals for group sessions. The workshop had a positive effect
in raising student awareness of the social and ethical complexities associated with emerging
technologies, on student confidence in participating in interdisciplinary ethical dialogue, and the
likelihood of participants getting involved in public policy deliberations in their career.
Participants rated the workshop overall as excellent or good.
Reflecting on the entire program, participants responded that online discussion and chats,
collaborative experiences, the summer workshop, instructor interaction, and deliberative
opportunities were among the strengths of the course as a whole. Descriptions of the course‟s
weaknesses were similarly varied, including content, format, goals, and participation. Some
participants reported having difficulty with the wiki, mostly having to do with format and other
barriers to participation. Overall, respondents found the online course to be a positive experience
and agreed that the course allowed them to place the scientific research discussed into the
broader social and ethical contexts. Participants responded favorably to interdisciplinary
deliberations. The majority of respondents felt comfortable exchanging ideas and opinions on the
ODC and felt it had potential as a tool for discussion for experts and graduate students, although
3
Debating Science 2008 Survey Report, 8/17/09
this measure seemed to vary depending on the course. Participants in the climate change and
nanotechnology groups provided mixed responses regarding their experience deliberating on the
ODC. Several respondents from these groups cited discomfort in online participation and lack of
deliberative discussions while online. The biotechnology group reported an overall positive
experience. Respondents from this group reported feeling comfortable participating and provided
detailed descriptions of online deliberation.
Introduction
The Debating Science program, a New Model for Ethics Education, is intended to teach
prospective scientists how to become more effective participants in public science debates.
Debating Science, conceptualized as a pilot project, completed its second year with the 2008
program. The 2008 Debating Science program consisted of a five day face-to-face workshop at
the University of Montana, Missoula during the first week of August 2008 followed by a
semester long online course during fall 2008. The face-face-workshop consisted of lectures,
group sessions, training sessions on the Online Deliberation Center, and field trips. Twenty-eight
students participated. The students were broken into three groups by interest: Climate Change,
Nanotechnology, and Biotechnology. Students were selected to participate in the groups through
a competitive application process, and were placed into groups such that each group represented
a diversity of opinions and academic interests. There were 10 students in the Biotechnology
groups, 9 in Climate Change, and 9 in Nanotechnology. A separate instructor led each group
session. All twenty-eight students attended lectures, which covered various scientific and
technical components as well as social issues of each topic. Several experts from each of the
three fields were invited to give lectures and to participate in relevant group sessions.
4
Debating Science 2008 Survey Report, 8/17/09
The online portion of the program consisted of student participation in the Online
Deliberation Center (ODC). The ODC is intended to facilitate deliberation online by allowing
students to collaboratively construct an online document and discuss their work online. During
the semester course, the students engaged with their respective group and instructor.
Communication was facilitated by email, the ODC, and live chat sessions.
Survey Methods
Internet-based surveys of the participants were conducted prior to the workshop,
immediately after it, and at the end of the following semester. For the purposes of this report,
surveys were divided by dates of administration, and information is reported separately for each.
Qualitative data were analyzed and summarized, and this information is presented first for each
survey.
Participants
Demographic information on participants was gathered at the first (pre-workshop) survey
administration. Of the 25 participants who responded to Item 27, 84% identified themselves as
“White.” Of the 25 respondents to Item 26, 56% were female, and 44% were male. Most
participants had taken previous ethics courses at varying degrees and requirement levels. The
most commonly reported career choice of the participants was academia. Tables summarizing
demographic information follow:
5
Debating Science 2008 Survey Report, 8/17/09
Table 1
27. Please indicate the racial and/or ethnic category with which you identify. Enter as many
categories as you wish.
Response Total
African American or Black
American Indian or Alaskan Native
Asian
Native Hawaiian or Other Pacific Islander
Hispanic
White
Other, please specify
Total Respondents
(skipped this question)
Response %
0
0
3
0
2
21
0
25
2
0%
0%
12%
0%
8%
84%
0%
Results
Pre-Workshop Survey (Qualitative)
Participants in this survey reported several qualities essential to deliberation about
science policy. The most frequent responses were: critical thinking, open mindedness, and
preexisting scientific knowledge. One respondent made the following remark: “To engage in
deliberations about science policy a person needs some understanding of the science involved.
They should be curious, honest, articulate, critical thinkers.” One participant noted that although
open mindedness was important, one should be firm in their stance, stating that he or she should
not be “easy to influence.”
Knowledge of scientific issues among the public was seen as poor. Several respondents
maintained that Americans were poorly educated in matters of science policies. One respondent
made the following remark: “…most of the public does not understand the language, culture, or
conventions of science.” This respondent went on to describe this in terms of the media,
6
Debating Science 2008 Survey Report, 8/17/09
claiming that they could “drive the debate by reporting on issues in ways that seem unbiased but
in fact are not.” Another respondent suggested that misunderstanding exists in both among the
public and among scientists, stating that there is a “lack of understanding regarding the nature of
science by the general public [and a] lack of understanding of the general public by scientists.”
Several individuals responded that debates about science policies are not true debates;
rather, they are an expression of opinions. One participant stated, “Science policy debates are
approached as political debates, and hence as a matter of ideology or mere opinion.” Others
talked about potential political motivations behind scientific debates. For example, one
individual stated, “People seem obligated to identify with causes based on their political,
religious, or social ideology—not on actual scientific information.” Further, other respondents
held that there is a sense of rigidity in current ideologies due to political or religious affiliations.
Most individuals listed climate change first in response to the prompt, “List two of these
social and environmental challenges, if any, in the order of their importance” (Table, 10, Item
13). There were a variety of responses for the second challenges, including energy management,
lack of access to basic resources such as water, and limited resources.
Several individuals indicated that the role of scientists in addressing these challenges is
the education of public and policy makers. For example, one individual wrote, “Education is the
key to conveying that there are issues. Scientists must foster education of the issues as well as
general education so that society may better deal with the challenges.” Another individual also
expressed a similar belief, “…There seems to be a gap between the scientific knowledge and the
beliefs in the „real world.‟ We need to figure this out and find a way to convey the necessary
information in a credible way.” Other respondents observed the “ineffective” nature of means of
communication between scientists and the general public. One respondent noted, “The way that
7
Debating Science 2008 Survey Report, 8/17/09
scientists are taught to communicate is ineffective. Even for applied science, the general scheme
is to perform experiment[s], [and then] publish technical paper[s] in a technical journal.” Other
participants held that scientists should adapt their language in order to be more “meaningful” to
the public.
In regard to career plans, most respondents indicated that they were interested in
academia. Several also stated that they would like to be involved in research, either at a
university, or at a research institution. Finally, some respondents expressed a desire to complete
a post-doc upon graduation.
Most respondents indicated that learning about proper language and terminology would
be most helpful to better prepare to participate in policy issues (Item 24). For example, one
respondent stated, “It would be especially helpful to learn about the methods, terminology, etc.
of science policy professionals.” Other respondents expressed a desire for practice, stating,
“Broader foundation in philosophical considerations, grounded in case studies and hypothetical
examples” and “Real world experience!” As discussed in earlier responses, some participants
reported a need for a more “accessible” communication style when interacting with the public.
One respondent stated, “I would like to have a better understanding of how to communicate my
own and others‟ scientific research in a way that is accessible and engaging. I am especially
concerned with how to talk about complexity and uncertainty in a way that does not detract from
the power of what we know right now.” Other respondents commented on a desire to have a
better understanding of issues in science policy. For example, one respondent stated, “…I feel
that a strong introduction to the policy issues behind scientific research requires a detailed
examination of the requisite background of the relevant science in question…I would then like to
be introduced to the specific character of the policy issues at hand including an explanation of
8
Debating Science 2008 Survey Report, 8/17/09
the political framework through which the debate has taken place (specifics on legislation
governing the research, funding, etc.).” Similarly, another respondent also expressed a desire for
“a general understanding of how it all works.”
Pre-Workshop Survey (Quantitative)
The following tables summarize the quantitative data gathered during the Pre-Workshop Survey.
This survey was administered prior to participants‟ arrival in Montana for the face-to-face
workshop. The survey inquired into both the participants‟ familiarity with the subject matter of
the workshop, but also their understanding the scientists‟ role in ethics and public deliberation
and interdisciplinary ethical dialogue.
Table 2
3. How satisfied are you with your training in graduate school to date in the
intellectual qualities you mention above?
Satisfaction Level
Total Respondents
(Skipped this
question)
Very
Satisfied
12.5%(3)
24
Satisfied
37.5%(9)
Neutral
45.83%(11)
Dissatisfied
4.17% (1)
Very
Dissatisfied
0% (0)
3
Table 3
4. How confident are you about participating in interdisicplinary ethical dialogue at
your university?
Interdisciplinary
Total Respondents
(Skipped this
question)
Extremely
confident
17% (4)
24
Somewhat
confident
46% (11)
Neither
confident
nor unsure
33% (8)
Somewhat
unsure
4% (1)
Absolutely
terrified
0% (0)
3
9
Debating Science 2008 Survey Report, 8/17/09
Table 4
5. How would you assess your current level of preparation to enter into public debates
about science policy with multiple stakeholders?
Very
Good
Preparation
Total Respondents
(skipped this
question)
4.35%(1)
23
Good
Neutral
Bad
Very Bad
34.78%(8)
43.48%(10)
17.39%(4)
0% (0)
4
Table 5
6. Scientists should be trained in research ethics (e.g. how to present data, how not to
plagiarize).
Strongly
Agree
Research ethics
Total Respondents
(skipped this
question)
84.62%(22)
26
Agree
Neutral
Disagree
Strongly
Disagree
15.38% (4)
0% (0)
0% (0)
0% (0)
1
Table 6
7. Scientists should additionally be trained in social and environmental ethics (e.g.
ethical obligations to people and to nature).
Strongly
Agree
Social and
environmental ethics
Total Respondents
(skipped this
question)
61.54% (16)
26
Agree
Neutral
Disagree
Strongly
Disagree
34.61% (9)
3.85% (1)
0% (0)
0% (0)
1
10
Debating Science 2008 Survey Report, 8/17/09
Table 7
8. How would you rate the quality of current public debates over science policy?
Quality
Total Respondents
(skipped this
question)
Very Good
Good
Neutral
Bad
Very Bad
0% (0)
11.54%(3)
30.77% (8)
50% (13)
7.69% (2)
26
1
Table 8
10. To what extent do you believe that it is good for scientists to participate in public
deliberations about directions for science and technology research?
Participation
Total Respondents
(skipped this
question)
Very Good
Good
Neutral
Bad
Very Bad
57.69%(15)
26
42.31%(11)
0% (0)
0% (0)
0% (0)
1
Table 9
11. How much influence should non-specialist citizens have over the ultimate direction of
scientific research?
Direction
Total Respondents
(skipped this
question)
Very Small
Small
Neutral
Large
Very Large
0% (0)
19.23%(5)
61.54%(16)
19.23%(5)
0% (0)
26
1
11
Debating Science 2008 Survey Report, 8/17/09
Table 10
12. Do you agree that global society faces serious social and environmental challenges?
Serious challenges
Total Respondents
(skipped this
question)
Strongly
Agree
88% (22)
25
Agree
12% (3)
Neutral
0% (0)
Strongly
Disagree
0% (0)
Disagree
0% (0)
2
Table 11
14. Solving such global challenges requires solutions that are:
Solutions
Total Respondents
(skipped this question)
Primarily
technological
8% (2)
25
2
Mostly
technological
but also political
36% (9)
Mostly political
but also
technological
68% (17)
Primarily
political
4% (1)
Table 12
16. What is your level of understanding of the term "food security"?
Food Security
Total Respondents
(skipped this
question)
Very Good
10.52% (2)
19
Good
47.36%(9)
Neutral
31.58% (6)
Bad
5.26% (1)
Very Bad
5.26% (1)
8
Table 13
17. What is your level of understanding of the problem of creating a "nano-divide"?
Nano-divide
Total Respondents
Very Good
0% (0)
17
Good
17.65%(3)
Neutral
23.53% (4)
Bad
29.41%(5)
Very Bad
35.29%(6)
12
Debating Science 2008 Survey Report, 8/17/09
(skipped this
question)
10
Table 14
18. What is your level of understanding of the phrase a "fair and effective response" to
climate change?
Climate response
Total Respondents
(skipped this
question)
Very Good
11.11% (2)
18
Good
44.44%(8)
Neutral
33.33% (6)
Bad
0% (0)
Very Bad
11.11%(2)
9
Table 15
19. What is your level of understanding of the issues raised by the "precautionary
principle"?
Very Good
Precautionary
Principle
Total Respondents
(skipped this
question)
23.08% (6)
26
Good
30.77%
(8)
Neutral
23.08% (6)
Bad
11.54%
(3)
Very Bad
11.54%
(3)
1
Table 16
20. What is your level of understanding of the term "distributive justice"?
Justice
Total Respondents
(skipped this
question)
Very Good
Good
Neutral
Bad
Very Bad
15.38% (4)
26
26.92% (7)
26.92% (7)
15.38% (4)
15.38% (4)
1
13
Debating Science 2008 Survey Report, 8/17/09
Table 17
22. Are you currently an active participant in public debates over science and technology?
Current involvement
Total Respondents
(skipped this
question)
Very involved
7.69% (2)
26
Somewhat involved
42.31% (11)
Not involved at all
50% (13)
1
Table 18
23. Do you anticipate being involved in any public policy deliberations in your chosen
career?
Expected involvement
Total Respondents
(skipped this
question)
Very Likely
42.31% (11)
26
Likely
42.31% (11)
Neutral
15.38% (4)
Unlikely
0% (0)
Very
Unlikely
0% (0)
1
Post-Workshop Survey (Qualitative)
When asked to comment on any aspect of the workshop that was particularly strong, most
of the 22 respondents mentioned the quality of the lectures. Several respondents found the
lectures to be strong in general. One participant remarked, “The whole workshop was great. I
think the lectures were very interesting and well balanced among the three areas
(nanotechnology; climate change, and biotech). The organizers did a great job finding top of the
shelf speakers who addressed the different aspects of these areas.” Another participant remarked
on the “caliber” of the speakers and added, “The addition of journalists to the mix was great
too.” Several lecturers were brought up favorably and mentioned by name; these included: Dan
Fagre, Juliet Elperin, Don Brown, Christopher Preston, Dane Scott and Andrew Light. A
participant said of Andrew Light that he “gave what I found to be the most compelling argument
14
Debating Science 2008 Survey Report, 8/17/09
for scientists to be involved in policy debates around controversial issues related to science.” A
few respondents qualified their comments regarding the lectures. One mentioned, “It would have
been nice to have a more basic intro to the science for some of the aspects.” And another noted
that although “Most of the speakers where pretty good,” there were “a few misses in my
opinion.”
Several participants made positive remarks about seminars and group work. Several
participants enjoyed the “time to interact with…peers.” For instance, one remarked, “Even
within each group there was a range of backgrounds which opened my views and thinking
beyond my research work and my concerns.” Similarly, another respondent stated, “Seminars
helped in learning deliberation.” One respondent noted the mixed views within the seminars:
“[In] the individual seminars…one got an overall view of the problems and also heard both the
scientists‟ as well as the philosopher‟s view of the issue.” Other participants indicated that peer
interaction in general, not necessarily in organized aspects of the course, was beneficial to them.
One remarked, “The team building was excellent in the biotech group.” Another participant
stated, “The best part was just meeting people who are interested in how science relates to
society and vice versa. I loved being able to just use each other as sounding boards; learning new
ideas from each other; learning how to better communicate our ideas on different issues.”
Most participants also indicated that time management was particularly strong. “Time
management was good. There was never a moment to feel bored or disengaged or tired. I
enjoyed the fact that we did not have to waste time looking for food during lunch time.” Many
respondents remarked that the days were “busy.” Several responses also stated the strength of the
extra-curricular activities, explicitly mentioning, the hike, and the PEAS farm.
15
Debating Science 2008 Survey Report, 8/17/09
When asked to comment on any aspect of the workshop that needed improvement,
several participants felt that the lectures could have been improved. Many participants mention
that lectures did not provide enough time for discussion. For example, “I…felt that there were
several lectures that would have been better if they had less time...This would have left more
time for questions; which was lacking during some presentations.” More specifically, one
participant noted, “I would have liked more time at the end of lectures for discussion. A number
of times it felt like the discussion was cut off just as it started going. I appreciated the willingness
of speakers to stay after the talk and speak informally with folks; but often it felt like that was
were the really meaty discussions happened; and it would have been nice to have more people at
least witness those conversations.”
Other respondents found the lectures less rewarding. Some felt a lack of interest in
lectures that were “outside my home group.” And some felt that the lectures were lacking in
content, for example, “I think we heard four different introductions to nanotechnology; but very
little substance or depth.” This participant also added, “It would have been more productive to
have more group meeting time and less attending of lectures outside of one‟s interest area.”
When it came to group meetings, a few participants saw room for improvement. One participant
felt that “better definition of goals and intent up front” was needed. A different respondent felt
that “some objective [goals] should be set up on [the] first day.” And another participant
suggested along similar lines, “Perhaps it would have been more productive to have more time
within the individual sections to really delve into issues.” Overall, many respondents felt that
more time was needed for discussions in both the lectures and the seminars.
When it came to the logistics of the trip, two participants mentioned that pre-arrival
planning was wanting. Several participants mentioned the living situation, specifically lack of
16
Debating Science 2008 Survey Report, 8/17/09
wireless connection in the dorm room and hard mattresses. Four participants commented on the
quality of the lunches, asking for more variety and more vegetarian options.
Descriptions of types of training would be most helpful to prepare to participate in policy
issues varied. First, many descriptions included discussion of policy, media, and
communications. Several participants discussed the need for training in matters of policy. A
participant stated, “having an understanding of factors involved in policy issues; a greater
awareness; knowing the lingo; and actual practice in deliberation helps greatly.” One respondent
thought, “More interaction with actual policy makers would be beneficial.” This particular
respondent added the importance of examining the “current political climate” in order to “couch
ethical arguments within a greater socio-political context.” Another respondent was interested in
understanding “how scientists‟ information is actually taken and used in policy.” Similarly,
respondents were interested in training with respect to “dealing with the press.” Two respondents
cited Juliet Elperin‟s mock interview exercise, “as useful in terms of dealing with the media.”
Both respondents were interested in more “hands on training.” Finally, a few respondents
mentioned training in rhetoric, or “how to express your opinions.” One sated that “Learning how
to actually communicate issues from a structured framework would be great.” Respondents also
focused on training in “writing in a more appropriate manner.”
Second, several respondents made more specific comments concerning their need for
training in terms of the workshop itself. One respondent commented on the “wide berth” of
investigation in the workshop and stated that s/he would have preferred “specific case studies
with clear outcomes.” S/he continued, “The activities planned during the week could have been
more constructive rather than revolving mostly around instruction.” Another respondent wanted
“More information on how to get involved and/or speak about issues at a local level.” And
17
Debating Science 2008 Survey Report, 8/17/09
another respondent wished “we would have had more designated time at the end of every topic
day to discuss issues with all participants—like a round table discussion or debriefing.”
Finally, two participants stated that they need training in deliberation. One stated, “I do
not feel fully prepared to debate the ethical issues related to my work. I would like to see more of
the fundamental skill sets necessary to deliberate meaningfully.” And another listed, “Learn how
to deliberate; [that] is bring to the table the very best of every participant in the deliberation.”
Post-Workshop Survey (Quantitative)
The following tables summarize the survey responses for the survey administered after
the workshop. The information presented represents both the participant‟s evaluation of the
workshop itself as well as the participant‟s confidence in participating in interdisciplinary
dialogue.
Table 19
1. What is your overall evaluation of the workshop?
Excellent
Good
Overall
59.1% (13)
40.9% (9)
Total Respondents
22
Neutral
0% (0)
Fair
0% (0)
Poor
0% (0)
Table 20
4. The workshop raised my awareness of the social and ethical complexities associated with
emerging technologies.
Strongly
Strongly
Agree
Agree
Neutral
Disagree Disagree
Complexities
50% (11)
45.45% (10)
4.55% (1)
0% (0)
0% (0)
Total Respondents
22
Table 21
5. The workshop increased my confidence about participating in interdisicplinary
dialogue.
Strongly
Agree
Agree
Neutral
Disagree
Confidence
31.82% (7)
54.55% (12) 13.64% (3) 0% (0)
ethical
Strongly
Disagree
0% (0)
18
Debating Science 2008 Survey Report, 8/17/09
Total Respondents
22
Table 22
7. As a result of this workshop, do you think it is more or less likely that you will get involved
in public policy deliberations in your chosen career?
More Likely
About the same
Less Likely
Expected Involvement
77.27% (17)
22.73% (5)
0% (0)
Total Respondents
22
End-of-Semester Survey (Qualitative)
The End-of-Semester survey covered the semester long online course as well as the
Debating Science program as a whole, including the face-to-face workshop in Missoula during
the summer. During the semester course the students were broken into groups according to
subject matter (biotechnology, nanotechnology, or climate change) and assigned to an individual
instructor. For this reason, qualitative questions that specifically deal with the online deliberation
component of the program will be analyzed separately according to subject matter
(biotechnology, nanotechnology or climate change). The general questions will be treated first.
Analysis of All Groups
Respondent‟s descriptions of what part of the course worked best for them were varied,
but several respondents mentioned the online discussions and chats. For example, “The group
chats were …good for touching base and discussing the issues we were working on in the ODC.”
Another participant remarked, “The online chats we had definitely made me more productive on
the ODC the weeks we had them.” Several respondents mentioned the ODC positively.
However, some found the ODC difficult, “I found the asynchronous communication of the wiki
not very user friendly.”
19
Debating Science 2008 Survey Report, 8/17/09
Similarly, a few participants mentioned that collaboration online was the best aspect of
the course. One respondent remarked, “The collaboration on the writing of the final pages was a
wonderful experience. It was a great learning experience to collaborate with researchers from
different disciplines.” Another respondent noted, “It was good to have collaborative teams.
Accountability seems to have been our biggest barrier; and this helped to alleviate that.” Several
participants described the face-to-face workshop in Missoula as well. “The workshop in
Missoula helped tremendously to form personal bonds; which led to a, in my eyes, very
productive deliberation. The various backgrounds of the students with regard to the topic led to a
good exchange of opinions and growth of personal views.” And another remarked, “Meeting
fellow participants in Missoula was by far the most fulfilling aspect of the course.”
Some respondents described course content and instructor interaction as working the best
in the course. One participant commented, “I…thought that the selected readings were well
chosen and that the assignments derived from the readings were directly relevant to course goals.
When it came to faculty interaction one respondent mentioned an instructor by name, “Regular
contact with Prof. Scott; his evaluations and comments was …[an] important aspect of this
course.”
When participants described the parts of the course that worked least, comments ranged
widely. However, most comments concerned the format, content, goals, and participation rather
than the technological components of the course. Several participants commented that the course
“lacked focus” or that they were unsure about the content of the course. One respondent noted,
“The lack of directed focus stunted the growth of our project and because of a lack in strong
central focus; our group‟s goals kept changing as the weeks went by without much progress.”
Another participant felt that the course “branched out too far.” One participant was concerned
20
Debating Science 2008 Survey Report, 8/17/09
with the direction of the project generally, stating, “I felt like we were mostly handed an
ambiguous challenge and told to figure it out. Without addressing some of the underlying drivers
behind why we are working on these issues; we are doomed for failure and an utterly incomplete
understanding.” This person would have preferred “a more fundamental approach” and also
commented that “It was very difficult to determine the level of detail required.” Along the same
lines, some participants felt that the content of the assigned readings detracted from the course.
One respondent remarked, “…the focus on describing the climate change mitigation wedges
often felt like busy work. It didn‟t help me learn anything about the ethical dimensions of the
climate change debate; and instead I felt like I spent long hours researching and writing about
topics that have been extensively described elsewhere.” Another respondent added, “The short
discussion question about the readings often felt like busy work and often distracted from the
course‟s goal of working on the main ODC pages.”
Several participants brought up participation as an issue. “The course took a long time to
gain momentum; as many participants did not respond to readings and assignments early on.”
Also, many respondents noted that the course lacked “equal participation.” Similarly,
participants commented on the interactions with other participants. Likewise, another participant
struggled with “Coming to a conclusion…unanimously for such a vast topic with so many people
putting their heads together.”
Some participants noted they had difficulty participating on the wiki. For example, “The
editing process for the actual ODC pages was kind of tricky. Even though we were told to edit
freely, it‟s easier said than done. It helped that we all knew each other and got along well; but it‟s
still difficult to go through someone else‟s work and make changes.” Other participants had more
difficulty with the ODC. For example, a participant noted that “Getting people to work together
21
Debating Science 2008 Survey Report, 8/17/09
in this format” was the part of the course that worked least. One commented, “It was easy to get
behind in work and when I would log on to see the changes it was difficult to compare. The more
sections that were written; the more difficult it was to make changes because you wouldn‟t want
to delete another person‟s work without good reason and modifying it became tricky.” Another
person had more technical difficulties, “…the site lacked many things. I never got emails when
people were active; so the only way to engage was by going to the site. You couldn‟t cut and past
from the wiki to word—you could only cut and paste from syntax frame. This made the site
much less user friendly.” Lastly, one participant felt “daunted” when faced with contributing, “it
was somewhat daunting to read through everyone‟s response before composing my own
response feeling like what I was going to write was lacking in sophistication or depth.”
When asked what types of training would be most helpful to feel better prepared to
participate in policy issues the majority of respondents discussed the need for ethics and policy
training, with some more focused on training in media and in science. One respondent said
“Ethics would be very important because I don‟t really think one should be out in the public
realm engaging in these issues without having developed a well thought out position on the
issue…I would hope that people‟s positions are the result of sound reasoning within their chosen
ethical framework.” This respondent went on to discuss training in policy and political theory.
Another participant suggested, “Ethics taught in graduate school for science majors would be
helpful; and not research conduct ethics as it‟s taught now.” Several other respondents pointed to
needed training in policy, political theory, science, and economics.
Respondents also noted that more training is needed in communication with media. One
noted, “I think I have a lot of room for improvement in how I interact with media and the press.”
Another participant noted the need for “practice and training to feel more confident and
22
Debating Science 2008 Survey Report, 8/17/09
comfortable participating in panel discussions [and] being interviewed.” Other respondents
discussed leading or facilitating public deliberation sessions. One participant noted, “The public
is rather…polar when it comes to discuss policies in our subject matters…The public needs to be
better prepared for deliberations and discussions.”
Several participants responded to the question pertaining to whether there is anything
helpful they would like to share. A few participants said that they had a positive experience. One
respondent stated, “I though this was a great course. The workshop was absolutely indispensable;
I think it would be very difficult and much less pleasant to try and carry out the same project
without at least some amount of face-time.” Another participant suggested, “It would have been
helpful to have some online chats with experts during the course.” At the same time a few
participants expressed concerns about the goals of the course. One stated, “…I think you need to
rethink your structural approach. It‟s a challenge to teach a generation how to think about these
problems; and I think you have failed at this. You gave us a forum to collaborate on research; but
you did not teach us anything about how or why to think about the implications.” Similarly,
another respondent remarked more specifically, “A more structured system of goals and
deadlines would have been helpful.” A couple of respondents commented that the program
needed to work on “group dynamics.” For example, “I became very disconnected from the group
after about four weeks of crawling progress due to incessant bickering about the particulars of
the project which I felt mainly came from some who kept pursuing an agenda that was not shared
by the group.” This participant went on to say, “There just came a point where I felt totally
unnecessary and only had minimal responsibilities; which lead me to become somewhat
disenchanted with the group and; by proxy; with the project.”
Analysis by Section
23
Debating Science 2008 Survey Report, 8/17/09
Six questions are analyzed according to section. Each of these questions relates to the
online class deliberations in which students were placed into sections according to their field of
interest. It is important to note there were an uneven number of respondents to each of the
questions between the groups.
Climate Change
Five participants from the Climate Change group responded to the question concerning
the effectiveness of the instructor, a follow up to Item 6 (Table 26). Two respondents answered
positively overall, stating “Becks did a great job of keeping us focused and on track” and “Becks
did a good job of keeping the project on task by checking up weekly, offering suggestions, and
focusing us when the work became inefficient or distracting.” One respondent expressed
frustration concerning the website. Another respondent said, “I just don‟t care for chatting;
especially when we are not speaking the same language.”
When asked to discuss whether their positions changed through deliberations with other
students, four of the respondents from the climate change group varied. One participant
answered overwhelmingly positively, stating, “Yes, I learned a lot from the perspectives shared
with me by other students…Whereas my initial perspective was to deprecate wedges, discussion
with fellow students convince me of their efficacy.” Another participant noted that “some of my
positions changed….I was surprised at some of the results we came up with.” And two
participants noted that their opinions did not change at all. One added, “Our deliberations were
mostly based on methodological questions, not big picture questions.”
A couple of respondents in the climate group answered that the ODC has potential as a
tool for graduate education on social and ethical issues arising from scientific research (Table
31). One respondent qualified the answer, “it needs a ton of work, but has a tone of potential.”
24
Debating Science 2008 Survey Report, 8/17/09
Similarly, other respondents pointed out specific problems, “[The ODC] is best used when it‟s
used often. However, I can‟t stand living in the meta-world of the internet from 9-5; and across
time zones. If the content were better limited, more progress and feelings of utility would result.”
Another respondent noted, “Asynchronous courses need to be properly set up.” Another
respondent expressed dissatisfaction, “I find that online interactions are difficult to sustain… I
would much rather interact with classmates in person, and feel that the course would have been
much stronger if we had more time in Montana to work on the wiki instead of sitting in lectures.”
As a follow up to Item 14 (Table 32), participants were asked whether the ODC has
potential as a tool for discussion between experts about complex social policy issues arising from
science and technology. Participants answered favorably overall, but with qualification. For
Example, “Sure, but…the discussions need to be structured and everyone needs to be speaking
the same language.” One respondent remarked, “The ODC certainly has promise, but cause must
be taken to direct the user experience better. Discussion pages are good for some things; but
much of the deliberation process should occur on the content pages...Rarely, perhaps never, did
opposing view points get fully described on the content pages.” And one respondent commented,
“Communication online will never be as strong as communication in person.”
Four respondents from the Climate change group elaborated on their response to Item 17
(Table 33), which gave the following prompt: “I felt comfortable expressing my opinions on the
ODC.” One respondent answered positively, stating, “People were interested and open to hearing
others‟ thoughts.” One respondent felt that it depended on the circumstance and offered, “Often I
felt no problem expressing my opinion about a subject. If my expertise were stretched, then I was
worried. It had more to do with the topic than the medium.” And a couple of participants
responded that they felt uncomfortable. For example, “I was not comfortable expressing my
25
Debating Science 2008 Survey Report, 8/17/09
opinions. For the most part I tried to stick to statements of fact.” Another participant felt that s/he
was unable to “fully understand certain contributions nor fully express my own.” S/he added,
“The human aspects of science, especially the ethical aspects, need to be addressed in a
humanistic manner.”
When asked for additional comments on improving the ODC, two participants responded
that they had no comment. One advocated for video conferencing. And one suggested, “Ability
to cut from ODC to word, and a strong connection between ODC and email. Also, attachments
etc. should open in separate windows!”
Biotechnology
Six participants from Biotechnology provided positive comments about the effectiveness
of the instructor (Item 6 Table 26). One respondent stated, “Dane did an excellent job with our
group, he worked very hard to keep us on track…He is very good at drawing conversations out
of participants and helping them develop and analyze their positions.” Another respondent
observed, “He was very good at encouraging our own opinions and not forcing his own.” And
yet another remarked, “The instructor did an excellent job posing important questions and
making sure that no participant‟s voice was drowned out.”
Eight participants from the biotechnology group responded to the question about whether
their position on the issues discussed in the course changed through their deliberations with the
other students. About half of the participants responded “yes” or “it did” and provided detailed
commentary. For example, “Yes; definitely. As I learned more and heard more perspectives of
my fellow deliberators I learned about newer aspects of ag biotech. To have a group of informed
and interested people to discuss a topic of this sort certainly necessitates learning more!” Several
of the respondents commented on the exchange among students. For example, “I learned a lot
26
Debating Science 2008 Survey Report, 8/17/09
from the other perspectives shared with my by other students.” One respondent discussed the
difference between consensus and deliberation:
Yes; sometimes even when we would agree on a particular response it was interesting
and profound to realize that we have different reasons for our respective responses. I
think this is important for deliberation; because it is intended to do more than come to a
consensus policy recommendation; but to reach a heightened level of understanding on
the issue. It can also be helpful to understand the motivation and rationale even when we
can‟t come to a true agreement.
Several respondents noted that their “position stayed the same,” but each gave a “nuanced”
response, maintaining, for instance, “I am much more interested in understanding the social
aspects of the GM debate,” or “I have examined and thought deeper about my positions.” One
respondent noted that although “My basic positions on biotechnology issues have remained more
or less the same…I certainly have a much more nuanced understanding and set of opinions about
more complex or grey areas.”
In elaborating on their response to Item 12 (Table 30), on the potential of the ODC for
graduate education, all eight respondents answered positivity. The strongly positive responses
ranged from observations about the important role that wiki software plays in our culture to the
benefits of using the ODC for graduate students. Benefits included, “greater level of
engagement,” and “having to write collaboratively.” One student claimed, “I think it
would…make students more informed citizens and aware of the main misunderstandings in
media and in society, and maybe even motivate them to make efforts to change misconceptions
in society.” Another respondent would have preferred face-to-face discussions, but also held that
“the wiki was a good alternative and an important one for an interdisciplinary group which is
spread across the country.” S/he went on to note some problems, “[S]ometimes there is so much
written in a short burst that very little true communication occurs as everyone is trying to submit
before a deadline or after a prolonged absence.” Similarly, another student noted that the ODC is
27
Debating Science 2008 Survey Report, 8/17/09
a “tool,” which “cannot form a course by itself, as interaction with fellow students and teachers
is an important aspect of learning.”
When asked to elaborate on Item 14 (Table 31) regarding the potential of the ODC for
discussion between experts about complex social policy issues, all but one of the seven
respondents answered positively. For example, a positive response, “In terms of a tool for
experts, I think one of the great aspects is the capacity to engage in an asynchronous discussion
that allows for geographically disparate people to participant on their own time schedule.”
However, some participants qualified their response. One respondent was concerned that experts
would use the ODC to “push their agenda” and suggested that “when experts participate in the
ODC the directors of the ODC should make sure they have a broad spectrum of experts with
varying positions to assist in deliberations…” Finally, one participant felt strongly that “for
experts who are discussing complex issues aimed at having an immediate impact on policy it is
not adequate. It could be a jumping off point, but it should be integrated into…conferences.”
When asked to elaborate about their comfort in expressing opinions on the ODC (Item
17, Table 33), the majority of the Biotech group felt comfortable. Of the five respondents, four
expressed that they were comfortable and noted that their comfort stemmed from getting to know
one another at the face-to-face workshop. For example, “It helped to know the other participants,
the weeklong workshop went a long way to breaking down communication barriers. Even when I
wrote something I thought someone else might disagree with, I could address their position and
even make a personal comment that I am...respectfully disagreeing with so-and-so, but here‟s
what I think.” Another participant remarked that [the ODC] leads to well constructed
deliberation and also accountability. One participant commented on the impersonal nature of the
28
Debating Science 2008 Survey Report, 8/17/09
wiki. “It makes it relatively easy to criticize other‟s positions and writing. It gives the impression
that communications are somewhat impersonal”
Suggestions for improving the technical aspects of the ODC included adding special
discussion pages where the “content itself can be discussed” like on Wikipedia, developing a
“tree diagram to picture how everything is linked together,” and adding a separate “track changes
page for each of the topics.” One participant suggested spending more time at the workshop
learning “the protocols and etiquette for working on this kind of a project.”
Nanotechnology
Five participants provided comments about the effectiveness of the instructor (Item 6,
Table 26). Most of the participants answered quite positively. The “instructor was an effective
organizer and guided the discussions. He was keeping track of everybody‟s work and pinpointed
difficulties and important questions.” And another noted, “He made sure that everyone‟s opinion
is being taken into consideration so that the articles [are] not biased.” One respondent claimed,
“Dr. Preston did a great job directing discussion and always asked if everyone had a clear idea of
what they would work on next.” One participant answered, “Since our group didn‟t really seem
to do much deliberating, there wasn‟t a whole lot the instructor could do to facilitate. He
encouraged and cajoled to set tasks. But there was little response in terms of discussion and any
real deliberation.”
When asked whether their position changed during deliberation with other students, the
five responses from the nanotechnology group were mixed. Two respondents replied that “yes”
their position had changed. One respondent noted that “my understanding of the central issues
that need to be addressed…has been greatly enhanced.” The other observed, “Initially I was very
against [regulation] thinking that it‟s all just empty fear…But…I have realized that ethics do
29
Debating Science 2008 Survey Report, 8/17/09
play a very important role in justifying things we discover.” One respondent noted that his/her
position changed “some,” and stated, “In certain cases I felt that communication was not very
good, since people didn‟t really understand each other‟s views.” One respondent reported feeling
“shut out,” because “engineers and scientists like to work differently than philosophers.” Finally,
one participant remarked, “it actually hardened my view in a negative direction.” This participant
held that there was “little deliberation/description” and that there was “a lack of awareness or
acceptance of significance.”
Four participants from the Nanotechnology group saw that the ODC has potential as a
tool for graduate education (Item 12, Table 30). However, a couple of participants qualified their
responses. “Though there were some serious issues that arose concerning the open-editing format
(angered colleagues etc), I think that the possibility for productive work is viable so long as the
group of writers share a common goal and a common methodological perspective.” Another
respondent thought it had potential, but focused on its technical issues including suggesting the
development of better threads and discussion pages.
When it came to the potential of the ODC as a tool for experts, two respondents noted its
potential, with qualification (Item 14, Table 31). For example, “The ease of informational access
that the wiki-page provides is an excellent source for academic debate and for the presentation of
all facets of a given subject. There are limitations…that have the potential for stunting growth.”
When asked if they felt comfortable expressing opinions in the ODC, responses were
mixed (Item 17, Table 33). One respondent noted, “I was comfortable with expressing debatable
viewpoints and was ready for discussion. Unfortunately, it did not always follow.” While another
respondent remarked that there was a “tension between the humanities and the sciences that kept
the free exchange of ideas from flowing freely.”
30
Debating Science 2008 Survey Report, 8/17/09
Two participants from the Nanotechnology group had suggestions for improving the
technical aspects of the ODC. One participant suggested “Making it easier to track changes for a
specific category.” The other participant suggested, “instead of having just one formal deadline,
the entire project should be divided into more stages consisting of deadlines for each one of
them.”
End of Semester Survey (Quantitative)
The following tables summarize the results from the end-of-semester survey. The
information presented here involves a range of issues. Several different questions involve student
evaluation of the Debating Science program as a whole, including the instructors, the face-toface workshop and the online semester course. Information gathered here also estimates the
extent to which participants were capable and comfortable of expressing thoughts and opinions
online. Students were also asked to evaluate the ODC in terms of its effectiveness for graduate
students etc. Finally, students were asked about the likelihood that they would engage in
interdisciplinary deliberations in the future.
Table 23
1. Please indicate the course in which you were enrolled.
Response Total Response Percent
Biotechnology
8
Climate Change
6
Nanotechnology
8
Total Respondents
20
(skipped this question)
3
40%
30%
30%
31
Debating Science 2008 Survey Report, 8/17/09
Table 24
2. How many hours of work do you estimate you put into this course over the semester?
0 to 15
16 to 30
31 to 45
46 to 60
61 to 75
Other, please specify
Total Respondents
(skipped this question)
Response Total
3
3
4
4
5
0
19
4
Response Percent
16%
16%
21%
21%
26%
0%
Table 25
3. Overall I found the online course to be a useful educational experience
Online
Total Respondents
(skipped this question)
Strongly Agree
42.11% (8)
19
4
Agree
47.37% (9)
Neutral
0% (0)
Disagree
5.26% (1)
Strongly
Disagree
5.26% (1)
Table 26
6. The instructor was effective at facilitating the online deliberations
Instructor
Total Respondents
(skipped this question)
Strongly Agree
52.63% (10)
19
4
Agree
31.58% (6)
Neutral
10.53% (2)
Disagree
5.26% (1)
Strongly
Disagree
0% (0)
32
Debating Science 2008 Survey Report, 8/17/09
Table 27
8. After taking this course I am better able to place the area of scientific research discussed in
the course into broader social and ethical contexts.
Context
Total Respondents
(skipped this question)
Strongly Agree
42.11% (8)
19
4
Agree
47.37% (9)
Neutral
5.26% (1)
Disagree
5.26% (1)
Strongly
Disagree
0% (0)
Table 28
9. The interdisciplinary deliberations (interacting with the other students) expanded my
understanding of the issues discussed in the course.
Deliberation
Total Respondents
(skipped this question)
Strongly
Agree
42.11% (8)
19
4
Agree
36.84% (7)
Neutral
10.53% (2)
Strongly
Disagree
0% (0)
Disagree
10.53% (2)
Table 29
11. The summer workshop prepared me well for the online portion of the course.
Workshop
Total Respondents
(skipped this question)
Strongly Agree
42.11% (8)
19
4
Agree
36.84% (7)
Neutral
15.79% (3)
Disagree
5.26% (1)
Strongly
Disagree
0% (0)
Table 30
12. The ODC (deliberative-wiki format) has potential as a tool for graduate education on social
and ethical issues arising from scientific research.
Wiki
Total Respondents
Strongly Agree
31.58% (6)
19
Agree
52.63% (10)
Neutral
10.53% (2)
Disagree
5.26% (1)
Strongly
Disagree
0% (0)
33
Debating Science 2008 Survey Report, 8/17/09
(skipped this
question)
4
Table 31
14. The ODC (deliberative-wiki format) has potential as a tool for discussion between experts
about complex social and policy issues arising from science and technology.
Expert tool
Total Respondents
(skipped this question)
Strongly
Agree
36.84% (7)
19
4
Agree
42.11% (8)
Neutral
10.53% (2)
Disagree
10.53% (2)
Strongly
Disagree
0% (0)
Table 32
16. The ODC provides a forum for freely exchanging ideas and opinions.
Free exchange
Total Respondents
(skipped this question)
Strongly Agree
31.58% (6)
19
4
Agree
63.16% (12)
Neutral
0% (0)
Disagree
5.26% (1)
Strongly
Disagree
0% (0)
Table 33
17. I felt comfortable expressing my opinions on the ODC.
Comfort level
Total Respondents
(skipped this question)
Strongly Agree
47.37% (9)
19
4
Agree
31.58% (6)
Neutral
10.53% (2)
Disagree
10.53% (2)
Strongly
Disagree
0% (0)
Table 34
21. Do you agree that scientists need formal training in ethics?
Strongly Agree
Agree
Neutral
Response Total
13
4
2
Response Percent
68%
21%
11%
34
Debating Science 2008 Survey Report, 8/17/09
Disagree
Strongly Disagree
Total Respondents
(skipped this question)
0
0
19
4
0%
0%
Table 35
22. To what extent do you believe that it is good for scientists to participate in public
deliberations about directions for science and technology research?
Participation
Total Respondents
(skipped this question)
Very Good
52.63% (10)
19
4
Good
47.37% (9)
Neutral
0% (0)
Bad
0% (0)
Very Bad
0% (0)
Table 36
23. To what extent do you currently feel equipped to participate in ethical deliberation about
science policies?
Preparation
Total Respondents
(skipped this question)
To a great extent
47.37% (9)
To some extent
52.63% (10)
To no extent at all
0% (0)
19
4
Table 37
24. Do you anticipate being involved in any public policy deliberations in your chosen career?
Expected involvement
Total Respondents
(skipped this question)
Very Likely
42.11% (8)
Likely
52.63% (10)
Neutral
5.26% (1)
Unlikely
0% (0)
Very Unlikely
0% (0)
19
4
Conclusions
Debating Science 2008 Survey Report
Participants in the 2008 Debating Science Program generally agreed about the need for
scientist involvement in public science debates. Similarly, participants agreed that the
35
Debating Science 2008 Survey Report, 8/17/09
effectiveness of public science debates were poor. Most of the participants anticipated being
involved in public debates about science, and felt that their training in the intellectual qualities of
deliberation was satisfactory or neutral.
Participants responded very positively to the summer workshop. The survey indicates that
the summer workshop was successful in raising student awareness about the ethical complexities
associated with emerging technologies and in increasing student confidence about participating
in interdisciplinary ethical dialogue. Participants indicated that they benefited from the lectures
as well as from interacting with peers, instructors, and experts. Suggestions for improvement
included making more time for discussion in lectures and group sessions, reconsidering the
content of the lectures, and changing the format. Overall, students benefited from the workshop,
and the majority rated it as excellent.
Reflections regarding the online portion of the program were considerably more varied.
Many participants reflected positively on the online chats and discussions, often citing the
benefits of interdisciplinary collaboration. However, participants also commented that the online
component was ambiguous, lacked focus, and required more structure. Also, several participants
remarked that their level of participation was affected by the format and group dynamics of the
online sessions. Many respondents remarked that it was difficult to edit each other‟s work online.
This report evaluated qualitative questions specifically pertaining to the ODC according
to group, since groups worked separately on the ODC. This revealed discrepancies between the
three groups regarding beliefs about the overall usefulness of the ODC, comfort levels on the
ODC, and the quality of deliberation on the ODC. Both the Nanotechnology group and the
Climate Change group reported mixed views on the ODC. Some respondents from these groups
had a positive experience deliberating on the ODC and felt that it had a lot of potential. Other
36
Debating Science 2008 Survey Report, 8/17/09
respondents listed several problems regarding the format and structure of the ODC, including
interpersonal dynamics, discomfort expressing opinions, and issues with the format. The
Biotechnology group, however, expressed generally positive remarks about the quality of
deliberation they experienced on the ODC and its potential as a deliberative tool. Participants in
the biotechnology group reported feeling comfortable expressing opinions on the ODC, and
editing each others work. Similarly, several members of the biotechnology group reported that
their positions had changed to some extent due to the online experience, while the
Nanotechnology and Climate Change groups reported more “hardened feelings.”
However, considering the responses of the participants to quantitative measures
collectively, reveals that the majority of the respondents agreed that the ODC provides a forum
for freely exchanging ideas and opinions and as a tool for discussing complex social and policy
issues arising from science and technology.
Comparison of Debating Science 2007 and 2008 Survey Reports
As a conclusion to the pilot course portion of the Debating Science program, the two
consecutive years of the program are compared. In keeping with the organization of the results
sections of both reports, the comparison of the two years will be compared according to each of
the three surveys. The following comments incorporate both qualitative and quantitative
elements.
In the pre-workshop survey respondents from both years of the program generally agreed
about the poor quality of public science debates. Participants felt that the public have poor
knowledge of scientific issues and for the most part emphasized the important role scientists
should play in these debates. Participants from both years of the program indicated that scientists
should receive training in ethics and expressed fairly neutral views about the quality of ethics
37
Debating Science 2008 Survey Report, 8/17/09
training received in science education. The majority of participants anticipated being involved in
public deliberation in their careers.
After the workshop, participants from both the 2007 and 2008 workshop commented
extensively on the quality of the lectures, the organization of the course, the benefit of peerinteraction, and the quality of the instructors. At the same time, respondents from both years
offered suggestions for improving the lectures and the group work. Suggestions for improving
the lectures involved more time for discussion, for both years. Yet, the 2007 participants made
several comments regarding the quality of the ethics education received in the workshop and
several respondents suggested that the workshop could have been improved with more ethics
lectures. The 2008 participants, on the other hand, suggested improvements to the science
lectures and asked for more in depth or more introductory science lectures. A few respondents
from both years commented on the need for more training in deliberation and commented that
the workshop did not provide enough opportunities for practicing deliberation. Similarly, a
couple of participants from both years noted the lack of structure and the lack of obvious goals in
the group sessions. Several participants from both years commented positively on time
management. However, several participants from 2007 commented that the time management
was quite poor and felt that the workshop was overscheduled. Time management did not appear
to be an issue in 2008.
The end-of-semester survey for the 2007 and 2008 programs gathered information about
both the program as a whole and the online portion of the course. While surveys for both years
concentrated on evaluation of the actual course, the surveys had components that were
substantially distinct. The 2007 survey focusing much more on broader issues with respect to
perceptions of the ethics and science interface. The 2008 survey focused much more on
38
Debating Science 2008 Survey Report, 8/17/09
deliberation and the ODC. Thus several aspects of the end-of-the semester survey don‟t lend
themselves to comparison. Regardless, the surveys reveal several similarities and differences as
well as evidence of improvement.
Even though the 2007 survey respondents listed the ODC as one of the best aspects of the
course, participants seemed to benefit more from the face-to-face session. Several of the 2007
survey respondents mentioned technical, organizational and participatory issues with the ODC
throughout the survey and many 2007 respondents found the ODC difficult to navigate. The
2008 participants also noted organizational and participatory issues with the ODC, but responded
much more positively to it, listing it as among the best aspect of the program and noting the
collaborative aspects of the ODC much more.
There were several notable improvements in the 2008 program when compared to the
2007 survey. Both years mentioned the lack of goals and expectations of both the program as a
whole and of the ODC. Yet, this seemed to be a much more substantial and widespread
complaint in the 2007 report; the 2008 survey contained fewer participants who were dissatisfied
with the goals and organization. Participation improved notably in 2008. The majority of 2008
respondents reported spending over 30 hours on the course during the semester, whereas less
than half reported spending over 30 hours in 2007. In addition, the lack of student participation
was a concern that many voiced in 2007. However, fewer participants in 2008 mentioned this.
The logistics of the online portion of the course improved in 2008 as well. In 2007 participants
mentioned that switching online tools midway through was disruptive. In 2008 the online portion
seemed to go more smoothly, as such frustrations were not mentioned. The addition of scheduled
chat sessions was often positively mentioned in the report and may have contributed to the
notable improvement in 2008 participants‟ experience of the ODC.
39
Debating Science 2008 Survey Report, 8/17/09
Further, the 2007 end-of-semester survey included a qualitative question asking for
suggestions to improve the summer workshop for better preparing students for the semester
course. Two prominent suggestions included: defining goals, and training on the ODC. While a
few respondents mentioned the need for improvement in defining the goals of the course, the
majority of students in 2008 either agreed or strongly agreed (79%) that the workshop prepared
them for the online portion of the course. A few 2008 respondents suggested spending more time
on the ODC during the summer workshop, but for the most part participants seemed to feel better
prepared for the online course in 2008.
Suggestions for the Design of Future Surveys
We have a number of suggestions for future evaluation efforts of similar courses. One is
the keep the wording of similar items and response choices consistent across administrations of
different version of the survey as a means of comparing responses over time. Comparison of
groups over time should also be considered, however, it should be noted that a larger sample size
is necessary for this. Also, groups should be somewhat similar in basis for comparison. Also, we
recommend developing a more research instrument to more accurately measure participant
experiences during deliberation. All in all, however, the three surveys appeared to produce very
useful information.
40
Debating Science 2008 Survey Report, 8/17/09
Appendices
1) Pre-Workshop Survey
2) Post-Workshop Survey
3) End-of-Semester Survey
41
Debating Science 2008 Survey Report, 8/17/09
Appendix 1
Pre-Workshop Survey
42
Debating Science 2008 Survey Report, 8/17/09
43
Debating Science 2008 Survey Report, 8/17/09
44
Debating Science 2008 Survey Report, 8/17/09
45
Debating Science 2008 Survey Report, 8/17/09
46
Debating Science 2008 Survey Report, 8/17/09
Appendix 2
Post-Workshop Survey
47
Debating Science 2008 Survey Report, 8/17/09
Appendix 3
End-of-Semester Survey
48
Debating Science 2008 Survey Report, 8/17/09
49
Debating Science 2008 Survey Report, 8/17/09
50
Debating Science 2008 Survey Report, 8/17/09
51