Good Computing
Ethical Integrity in the Fast-Paced World of Computing
Chuck Huff
St. Olaf College
Yes, I know Marcus Aurelius or Vauvenargues
or Chesterton has already said this, and far
better; but let’s face it -- you weren’t listening
then either.
-Don Patterson, Oct. 2005 Poetry
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Collaborators and Supporters
Student Collaborators
•Britta Anderson
•Nathan DeWall
•Erin Engelbart
•Mike Knoll
•Cassie Seningen
•Joe Stewart
•Jenny Ingebritsen
•Kristyn Aasen
•Craig Enlund
•Nicole Gilbertson
•Laura Barnard
•Lauren Cassat
Faculty Collaborators
• Simon Rogerson,
DeMontfort Univ. Leicester, UK
• William Frey, University of
Puerto Rico, Mayaguez
• Julie Legler, Statistics, St. Olaf
College
Supporters:
• National Science Foundation
• Curriculum Based Resources DUE-9980786 and DUE-9972280
• Life Stories of Moral Exemplars in Computing SES-0217298
• Centre for Computing and Social Responsibility, Demontfort University, UK
• St. Olaf College
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Overview
• Organizational climate and moral decision
making by individuals (the bad news)
• Our standard advice (the inadequate news)
• Moral exemplars in computing who rise above
this context (the good news)
• Can we create a model to teach virtues?
(spreading the good news)
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Organizational Influence on Moral Decisions
• Organizations control scope and complexity by using
“goal based rationality” (Max Weber)
• This concentrates attention and energy on the efficient
achievement of organizational goals
• As the organization concentrates on efficiency, it ignores
other values, to its own detriment.
• This irrationality increases as narrowness
of focus increases
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Individual Response to
Organizational Influence
• Milgram’s obedience study is a microcosm
of the response to organizational pressure
• The experimenter requires efficient
performance of duty.
• The individual is caught between duty
to the experimenter and duty to the
victim.
• The punch line: most people obeyed
orders
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
The Obedience Paradigm
• The experimenter
instructs a teacher to
shock a student every
time the student
makes a mistake.
• The experimenter would not take no for an answer; As
the shocks increased in intensity, the victim went from
screaming to eerie silence.
• Milgram ran many variations to determine what
influenced the teacher’s willingness to continue.
• He also observed the teacher’s attempts to cope with
the conflict.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Coping with Immoral Orders
• Two thirds of the teachers chose to continue shocking the victim
while the victim:
• Protested the pain
• Withdrew permission to continue
• Mentioned a heart condition
• Screamed to be let out
• Fell completely silent
All the teachers:
• Protested the victim’s treatment
• Tried to call off the study
• Pressed the shock buttons so as
to minimize shock
• Showed empathy for the victim
• Were worried about the victim’s
health
• Showed clear signs of distress
Many of the teachers:
• Minimized harm to the victim
• Distanced themselves from the
pain
• Narrowed their focus to efficient
performance of duty
• Blamed the victim
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Organizational Processes of Moral
Compromise
• Organizations irrationally ignore other values in favor
of efficiency.
• Ordinarily moral employees immorally ignore other
values in response.
• The technology isolates designers and decision makers
from the effects of their decisions.
• Selective recruitment, socialization, and groupthink
perpetuate this cycle.
• And it is more complicated: multiple moral agendas.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
The Bad News
(in the worst possible light)
Highly committed, well-meaning,
narrowly focused, competent
computer professionals, guided by
organizational imperatives and
isolated from the effects of their
design decisions can be induced to
commit negligent, immoral,
unethical, possibly even evil acts.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Simplistic Moral Advice
• Adopt moral values
• Have integrity
• Exercise willpower
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Simplistic Virtue Advice
• Practice the Virtues (wisdom,
justice, courage, temperance,faith,
hope, love)
• Have integrity
• Exercise willpower
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
But given what we know about:
• Organizational Irrationality
• The power of the situation to
influence us
And what we don’t know about:
• How to “practice” a virtue
• How to “strengthen” our willpower
Both these are inadequate
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
The Good News
There are professionals who operate with
moral integrity in this environment. We call
them moral exemplars.
Who are these people?
How did they become the sorts of people who
can resist organizational pressure?
First, how do we identify and study them?
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Identifying Moral Exemplars
I recruited a panel of computing ethics experts to
ground the selection of exemplars.
The panel then agreed on criteria for selection.
Prof. Simon Rogerson, DeMontfort University, UK
Prof. Don Gotterbarn, East Tennessee State University, US
Dr. Alison Adams, University of Salford, UK
Prof. Göran Collste, Linköping University, Sweden
Dr. Barbara Begier, Gdansk Polytechnic University, Poland
Prof. Barrie Thompson, University of Sunderland ,UK
Prof. Jeroen van den Hoven, Erasmus University, The
Netherlands.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Defining Moral Exemplars in Computing
The classic moral exemplars study was Colby & Damon’s (1992) study
of exemplars in social service. These were activists in human rights,
hunger, poverty, child abuse etc.
We began with their criteria, but modified them to suit the domain of
computing. The final criteria were:
• Either a) a sustained commitment to moral ideals or ethical principles
in computing, or b) sustained evidence of moral virtue in the practice
of computing.
• A disposition to make computing decisions in accord with one's moral
ideals or ethical principles.
• A willingness to risk one's self-interest for the sake of one's moral
values.
• A tendency to be inspiring to other computing professionals and
thereby to move them to moral action.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Recruiting Moral Exemplars in Computing
Each panel member nominated 3-5 potential exemplars in the UK and
in Scandinavia (Denmark, The Netherlands, Norway, Sweden). The
unanimous panel approved those selected for contact. The sample
was expanded by snowball from the interviewees.
36 in UK and 27 in Scandinavia were nominated (total, 63). 35 were
contacted. 24 interviews conducted (a 74% response).
Profile of the Sample
• 13 had significant experience in academia
• 15 had significant experience in industry
• 3 had significant experience in government
• 11 were in the final decade of career and 4 were retired
• 4 were in first decade of career
• 9 were female
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
The Interviews
Life story interviews (McAdams, 2001)
lasting 3 hours in which they told a
series of stories (from 19 to 30) about
their professional life.
• Life chapters (and overview)
• Peak experience, Nadir experience, Turning point,
Earliest memory, Childhood scene, Early career scene,
Recent career scene, One other scene
• Positive and negative influences
• Influential stories about others (four of these)
• Positive and negative futures
• Value questions (e.g. 3 most important values in system
design)
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Who Are They?
• Simon Rogerson: Founder of EthiCOMP, first Professor
of Computer Ethics
• Elizabeth France: First Data Protection Registrar in the
UK
• James Towell: Cambridge Grad, Private Consulting
• Steve Shirley: Early pioneer in business computing
• Enid Mumford: Early pioneer in socio-technical systems
• Francis Grundy: Pioneer in encouraging women in
computing
• Alan Newell: Pioneer in developing systems for the
handicapped
• Alan Cox: LINUX Pioneer,
• Jan Holvast: Pioneer privacy advocate
• Ove Ivarsen: Developer of the USER Award from LO
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Common Themes
•
•
•
•
•
Most maintained a positive focus
Most cultivated a network of social support
Most cited multiple people as positive influences
None saw themselves as morally extraordinary
All were active problem solvers; problems were often
framed as moral/social/technical
• All practiced a set of social/technical skills to create
solutions to problems
• Craftspersons focused on users, reformers focused on
systems
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Craftspersons
Designing computing technology towards
ethical ends
• Drew on pre-existing values in computing
• Focused on users or customers who had
needs
• Took the role of providers of service/product
• Viewed barriers as inert obstacles, puzzles to
be solved
• Believed they were effective in their role
Example: Prof. Alan Newell; James Towell
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Reformers
Changing social systems
• Attempted to change organizations and
their values
• Took the role of moral crusader.
• Viewed individuals as victims of injustice.
• Viewed barriers as active opposition.
• Believed in the necessity of systemic
reform.
Examples: Francis Grundy; Ove Ivarsen
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Rating the stories
• Use of technical expertise
• Designing for users or clients
Form Craftsperson
Index
• Perceiving harm to victims
• Noticing a need for reform
• Taking action for reform
Form Reformer
Index
•
•
•
•
•
Social Support
Social Antagonism
Effectiveness & Ineffectiveness
Negative and Positive Emotion
Use of social expertise
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Finding Craft & Reform Exemplars
Reform Theme Index (X=0, SD =1)
2
Elizabeth France
Francis Grundy
Ove Ivarsen
Jan Holvast
Steve Shirley
1
Simon Rogerson
0
Viiveke Fåk
Fred Eisner
Alan Cox
Andrew McGettrick
Enid Mumford
-1
Niklas Halberg
-1
James Towell
Alan Newell
0
1
Craft Theme Index (X=0, SD =1)
2
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
From Exemplars to Virtues
• There are different kinds or expressions of
moral exemplars.
• There are likely more than two kinds.
• The virtues they express are a result of a
complex mix of the technical, social, and
personal.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Pritchard’s Engineering Virtues
Based on interviews with exemplars in engineering.
Though they would say little about their own virtues,
they were vocal about what they hoped for in colleagues:
•
•
•
•
•
•
•
Responsibility
Honesty
Justice (fairness)
Articulateness
Perseverance
Loyalty
Cooperativeness
•
•
•
•
•
•
•
Creative Imagination
Habit of Documenting Work
Civic-Mindedness
Courage
Openness to Correction
Commitment to Quality
Integrity
But the list is likely to be different for different moral
ecologies, professions, & cultures. There may be no
canonical list: (What do all saints have in common?)
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
A Four Component Model of Virtue
External
Moral Ecologies
Control
Personality
Moral Skill Sets
Integration of
Morality into the
Self System
Internal
Low
Mutability
High
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Influences on the expression
of virtues (1 & 2)
Personality
• Extraversion (r = .45 with
reform, p = .026)
•
•
•
•
Conscientiousness
Neuroticism
Agreeableness
Openness to Experience (r
= .39 with craft, p = .059)
• Others?
Moral Ecology
• Complex overlapping
ecologies
• Diversity both within and
between ecologies
•
•
•
•
•
National culture
Organizational culture
Professional culture
Support networks
Other micro-climates
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Influences on the
expression of virtues (3)
Integration of Morality in the Self System
Affiliations &
Relationships
Elements of the
temporally extended and
contextually distributed
self
Motives &
Strivings
Personal
Projects
Stories & Defining
Memories
Roles, Life Tasks,
Possible Selves
Beliefs,
Attitudes, &
Values
Past Behavior
& Experience
Adapted from McGregor & Little (1998).
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Influences on the expression of virtues (4)
These skill sets undergird the work of our moral
exemplars. Both social and technical skills predict
effectiveness in our exemplar’s stories.
• Moral Imagination: projecting oneself into the
perspective of others.
• Moral Creativity: generating solutions to moral
challenges while responding to multiple
constraints.
• Reasonableness: Gathering relevant
evidence, listening to others, giving
reasons, changing plans/positions
based on reason.
• Perseverance: planning moral action
and responding to unforeseen
circumstances while keeping moral
goals intact.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Influences on the expression of virtues (4)
These skill sets include a variety of skills, supported by
intermediate and basic knowledge.
• Skills: constructing the relevant stakeholders in a
socio-technical system; data collection about
stakeholders; understanding stakeholder
perspectives; identifying value conflicts;
constructing solutions under constraint; engaging in
reasoned dialogue; revising plans under constraint
• Knowledge: domain specific (e.g. privacy, safety,
access, intellectual property); socio-technical
systems; methods of data collection and argument;
technical knowledge of constraints and
opportunities; ethical argument;ethical dissent and
whistleblowing
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
How to acquire these?
• Narvaez & Lapsley (2005) draw on the work in
expertise to suggest that practical wisdom should be
learned in structured environments that
• Reward correct solutions
• Match explicit theory and strategy with practice
and coaching
• Provide extensive, focused practice
We learn a craft by producing
[its] product; … we become
builders by building. Aristotle,
Nicomachaen Ethics, Book II
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Teaching the Virtues in the Computing Professions
Prepare for these
Moral Ecologies
External
Teach these
Control
Personality
Moral Skill Sets
Be aware of this
Integration of
Morality into the
Self System
Internal
Low
Mutability
Influence this
High
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
Good Computing
• There are practitioners of good computing.
• Good computing is not a narrow straightjacket:
Exemplars come in a variety of kinds and this
diversity may be a strength.
• There are multiple influences on the ability to
express the virtues in the computing professions.
• There are skill sets that these practitioners
employ.
• These skill sets can be learned (and taught).
• This does not guarantee good computing, but it
makes it more likely.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007
References
Aristotle (1941) Ethica Nicomachea. In R. McKeon (ed.) The basic works of Aristotle, (pp. 927 – 1112). New York: Random House.
Caprara, G.V., Barbaranelli, C., Pastorelli, C. & Perugini, M. (1994). Individual differences in the study of aggression. Aggressive
Behavior, 20, 291-303.
Carver, C. S. (2005). Impulse and Constraints: Perspectives from personality psychology, convergence with theory in other areas, and
potential for integration. Personality and Social Psychology Review, 9, 312-333.
Cawley, M.J., Martin, J.E., & Johnson, J. A. (2000). A virtues approach to personality. Personality and Individual Differences, 28, 9971013.
Colby, A., & Damon, W. (1992). Some do care: Contemporary lives of moral commitment. New York: Free Press.
Crisp, R., & Slote, M. (eds.) (1997). Virtue Ethics: Oxford readings in philosophy. Oxford, UK: Oxford Press.
Fischman, W., Solomon, B., Greenspan, D., & Gardner, H. (2004). Making good: How young people cope with moral dilemmas at work.
Cambridge, MA: Harvard Press.
Huff, C. W., & Brown, R. (2004). Integrating ethics into a computing curriculum: A case study of the Therac-25. In A. Akera & W.
Aspray (Eds.), Using history to teach computer science and related disciplines. (pp. 255-277) Washington DC: Computing
Research Association.
Huff, C. W., & Frey, W. (In press). Moral Pedagogy and Practical Ethics. Science and Engineering Ethics.
Little, B.R. (1983) Personal Projects: a rationale and method for investigation. Environment and Behavior, 15, 273-309.
Markus, H, (1977). Self-schemata and process information about the self. Journal of Personality and Social Psychology, 35, 63-78.
McAdams, D. P., Reynolds, J., Lewis, M., Patten, A. H., Bowman, P. J. (2001). When bad things turn good and good things turn bad:
Sequences of redemption and contamination in life narrative and their relation to psychosocial adaptation in midlife adults and in
students. Personality and Social Psychology Bulletin, 27(4), 474-485.
McCrae, R.R., & Costa, P.T. (1999). A five-factor theory of personality. In L.A. Pervin & O. P.Jphn (Eds.). Handbook of personality:
Theory and research. New York: Guilford.
McGregor, I., & Little, B.R. (1998). Personal projects, happiness, and meaning: On doing well and being yourself. Journal of Personality
and Social Psychology, 74, 494-512.
Milgram, S. (1983). Obedience to Authority: An Experimental View. New York: Harper/Collins.
Narvaez, D., & Lapsley, D. (2005). The psychological foundations of moral expertise. In D. Lapsley & Power, C. (Eds.), Character
Psychology and Character Education . Notre Dame, IN: University of Notre Dame Press.
Oliner, S. P., & Oliner, P.M. (1988). The altruistic personality. New York: Free Press.
Peterson, C. & Seligman, M.E.P. (2004). Character strengths and virtues: a handbook and classification. New York: Oxford Press.
Pritchard, M. (1998). Professional responsibility: Focusing on the exemplary. Science and Engineering Ethics, 4(2), 215-234.
Rest, J.R., & Narvaez D. (Eds) (1994). Moral development in the professions: psychology and applied ethics. Hillsdale (NJ): Elrbaum.
Weber, M. (1904/1930). The Protestant Ethic and the Spirit of Capitalism. Translated by Talcott Parsons. New York: Charles Scribner's
Sons.
For the Linda Hall Library of Science, Engineering & Technology, March, 2007