USC
C S E
University of Southern California
Center for Software Engineering
Barry Boehm
Fall 2015
Presented by Jim Alstad
© USC-CSSE 2015/12/02 1

USC
C S E
University of Southern California
Center for Software Engineering
• Definitions and context
– Power to do public harm or good
– ACM/IEEE Software Engineering Code of Ethics
• Principles and examples
– Rawls ’ Theory of Justice
– Relation to stakeholder win-win
– Case study: Mercy Hospital
• Integrating ethics into daily software engineering practices
– VBSE/MBASE/Win Win Spiral Model
© USC-CSSE 2015/12/02 2

USC
C S E
University of Southern California
Center for Software Engineering
•
– And with moral duty and obligation
•
•
– Professional ethics
© USC-CSSE 2015/12/02 3

USC
C S E
University of Southern California
Center for Software Engineering
• Software engineers have increasing power to do public harm or good
– Intellectual property, privacy, confidentiality, quality of work, fairness, liability, risk disclosure, conflict of interest, unauthorized access
• Professional societies have developed codes of ethics
• Hard to integrate value-based ethics into valueneutral software engineering practices
• VBSE/MBASE/Win Win Spiral enable ethics integration
© USC-CSSE 2015/12/02 4

USC
C S E
University of Southern California
Center for Software Engineering
• Intellectual Property: use without credit; use copyrighted material
• Privacy: credit, health, personal information
• Confidentiality: competitive information, political sensitivity
• Quality of work: many dimensions; see table
© USC-CSSE 2015/12/02 5

USC
C S E
University of Southern California
Center for Software Engineering
• Government agency hires company to support SW procurement
– Provides data under nondisclosure agreement
• Employee and company consultant prepare cost estimate
– Employee: “ I don ’ t see how anyone can do all this for $8M ”
• Consultant provides $8M target cost to some bidders
• Government agency angry with company for leak
– Whose fault? How could it be avoided?
© USC-CSSE 2015/12/02 6

USC
C S E
University of Southern California
Center for Software Engineering
Quality Concerns Vary by Stakeholders Role
**Critical
*Significant
0 Insignificant or indirect
Stakeholder
Classes
*
**
**
** **
*
Information
Consumers
Mission critical
**
** uncrit.
**
**
Protection
Safety
Security
Privacy
Robustness
Reliability
Availability
Survivability
Quality of Service
Performance
Accuracy, Consistency
Accessibility, ease of use; difficulty of misuse
Evolvability
Interoperability
Correctness
Cost
Schedule
Reusability
2015/12/02
**
*
**
**
**
**
**
**
**
**
**
**
**
**
**
**
**
**
*
© USC-CSSE
*
*
**
*
**
**
**
**
**
**
*
*
*
*
**
**
*
**
**
*
*
*
*
*
**
*
*
**
**
**
**
**
*
7

USC
C S E
University of Southern California
Center for Software Engineering
• Fairness: equality of opportunity/treatment; fair reward system
• Liability: accountability; parity of authority and responsibility
• Risk Disclosure: safety tests, COTS capabilities; schedule slips
• Conflict of Interest: source selection; personnel or product reviews
• Unauthorized Access: reading, copying, modifying; denial of service
© USC-CSSE 2015/12/02 8

USC
C S E
University of Southern California
Center for Software Engineering
• Enron software to schedule power outages, raise prices
– Suppose you had been asked to develop it?
• Urban fire dispatching system
– Inefficient old system caused $700M property loss
– New-system spec. includes dispatching algorithm to minimize property loss
• Any fairness issues?
© USC-CSSE 2015/12/02 9

USC
C S E
University of Southern California
Center for Software Engineering
• Your company is delivering a drug prescription fulfillment system
– Reusing software from a warehouse inventory system
• You are the quality assurance manager
– With company responsibility for certifying product safety
• The software has passed all the contracted tests
– But many off-nominal conditions untested
– Some have shown unsafe outcomes
– You feel more off-nominal testing is necessary
• Company president says if you don ’ t certify safety by delivery date, company may go out of business
– What should you do?
© USC-CSSE 2015/12/02 10

USC
C S E
University of Southern California
Center for Software Engineering
1. Products: achievable goals, realistic estimates, high quality
2. Public: safety, respect of diversity, public interest first
3. Judgment: objectivity, no bribes or conflicts of interest
4. Client and Employer: no employer-adverse interests, surface problems
5. Management: fair, ethical work rules, due process for violations
6. Profession: support profession and ethics code, don ’ t misrepresent software
7. Colleagues: credit colleagues ’ work, give colleagues a fair hearing
8. Self: improve your technical and ethical knowledge and
© USC-CSSE 11

USC
C S E
University of Southern California
Center for Software Engineering
2.01 Disclose any software-related dangers
2.02 Approve only safe, well tested software
2.03 Only sign documents in area of competence
2.04 Cooperate on matters of public concern
2.05 Produce software that respects diversity
2.06 Be fair and truthful in all matters
2.07 Always put the public ’ s interest first
2.08 Donate professional skills to good causes
2.10 Accept responsibility for your own work
© USC-CSSE 2015/12/02 12

USC
C S E
University of Southern California
Center for Software Engineering
4.01 Provide services only where competent
4.02 Ensure resources are authentically approved
4.03 Only use property as authorized by the owner
4.04 Do not use illegally obtained software
4.05 Honor confidentiality of information
4.06 Raise matters of social concern
4.07 Inform when a project becomes problematic
4.08 Accept no detrimental outside work
4.09 Represent no interests adverse to your employer
© USC-CSSE 2015/12/02 13

USC
C S E
University of Southern California
Center for Software Engineering
• Definitions and context
– Power to do public harm or good
– ACM/IEEE Software Engineering Code of Ethics
• Principles and examples
– Rawls ’ Theory of Justice
– Relation to stakeholder win-win
– Case study: Mercy Hospital
• Integrating ethics into daily software engineering practices
– VBSE/MBASE/Win Win Spiral Model
– CS 577 ethics situations
2015/12/02 © USC-CSSE 14

USC
C S E
University of Southern California
Center for Software Engineering
Following Collins et al., “ How Good Is Good Enough?
”
Comm.ACM, Jan. 1994
• Fair rules of conduct
• Principles of justice
• Participants and obligations
– Provider (developer)
– Buyer (acquirer)
– User(s)
– Penumbra (general public)
• Negotiate mutually satisfactory (win-win) agreements
2015/12/02 © USC-CSSE 15

USC
C S E
University of Southern California
Center for Software Engineering
• Fair rules of conduct
– Negotiation among interested parties
– Veil of ignorance (about what affects whom)
– Rationality
• Principles
– Least Advantaged - don ’ t increase harm to them
• Harm = probability x magnitude (~risk exposure)
– Risking harm - don ’ t risk increasing harm
• Don ’ t use “ low-threat ” software in “ high-threat ” context
– Publicity test - defensible with honor before an informed public
2015/12/02
• Use for difficult cost-benefit tradeoffs
© USC-CSSE 16

USC
C S E
University of Southern California
Center for Software Engineering
To the provider To the buyer To the user To the penumbra
¨
Make a reasonable profit
¨ reasonable use warranty
¨ informed consent about testing process and potential shortcomings
¨ clear operating instructions
¨ reasonable protections from and informative responses to use and abuse
¨ provide reasonable technical support
¨ reasonable protections against physical, emotional, and economic harm from applications
¨ open about software development process and limits of correctness
2015/12/02 © USC-CSSE 17

USC
C S E
University of Southern California
Center for Software Engineering
To the provider To the buyer To the user

negotiate in good
 provide quality faith , recognizing the importance of provider’s fair profit software appropriate to users’ needs within

learn enough about the software to make an informed decision
 facilitate adequate communication users reasonable budget constraints
 prudent introduction of automation
 informed consent to using software
To the penumbra

buy software only with reasonable safeguards for the public
 open about software capabilities and limitations
 represent user’s interests with providers
© USC-CSSE 2015/12/02 18

USC
C S E
University of Southern California
Center for Software Engineering
To the provider To the buyer To other users To the penumbra
 respect
 active
 willing
 conscientious effort ownership rights communication with the buyer cooperation in learning and to reduce any risks to the public using software
 good faith effort to learn and use software responsibly
 encourage reasonable expectations about software capabilities and limitations
 reasonable requests for computing power
© USC-CSSE 2015/12/02 19

USC
C S E
University of Southern California
Center for Software Engineering
To the provider To the buyer To other users To the penumbra
 become aware of
 advocate a
 expect only
 become aware of the capabilities and limitations of software
 advocate a suitable economic and statutory environment for quality software suitable economic and statutory environment for quality software reasonable service from users
 become aware of the capabilities and limitations of software the capabilities and limitations of software
 advocate a suitable economic and statutory environment for quality software
© USC-CSSE 2015/12/02 20

USC
C S E
University of Southern California
Center for Software Engineering
• Growing hospital
– Manual pharmacy information system reaching overload
• Spec developed for PC-based information system
– Rachel: VP, Records & Automation
– George: Chief Pharmacist
• System developed by consultants
– Hired by George
– Rachel: test procedures
– Based on mature warehouse inventory system
– Budgeted 50% more testing than other bidders
• Installation & Training discovers problems
– Helen: consultant in charge of installation & training
– Ann: skeptical nurse cross-checking computer outputs
2015/12/02 © USC-CSSE 21

USC
C S E
University of Southern California
Center for Software Engineering
• Dosage problems from data entry errors
– 10x dosage; wrong patient
• Cross-checking incomplete; not trusted by some doctors
• Heavier data-entry load
– Formalizing automated procedures
 more info. needed
– Pharmacy info > warehouse info
• Helen: Should go back to old system during cleanup
• George: - Is old system less risky?
How do we ensure cleanup will get it right?
How much will cleanup cost?
• Future practice: How to anticipate, avoid similar problems?
2015/12/02 © USC-CSSE 22

USC
C S E
University of Southern California
Center for Software Engineering
• Definitions and context
– Power to do public harm or good
– ACM/IEEE Software Engineering Code of Ethics
• Principles and examples
– Rawls ’ Theory of Justice
– Relation to stakeholder win-win
– Case study: Mercy Hospital
• Integrating ethics into daily software engineering practices
– VBSE/MBASE/Win Win Spiral Model
– CS 577 ethics situations
2015/12/02 © USC-CSSE 23

USC
C S E
University of Southern California
Center for Software Engineering
• Results chain
– Add patient safety outcome, patient stakeholder representative
– Rework-business-workflows initiative, including safety checks; add clerical-staff stakeholder
• Stakeholder Win Win
– Patient representative: safety criteria; parallel-operation phasein
– Clerical staff: prototype GUI, including safety-check support
• Business Case: includes added safety costs and benefits
• Risk Management: assess warehouse package safety, effects of workflow changes.
2015/12/02 © USC-CSSE 24

USC
C S E
University of Southern California
Center for Software Engineering
• Concurrent Engineering
– Concurrently address business workflows, GUI prototypes, COTS alternatives, feature prioritization, cost/schedule/benefits analysis, other risks
– Prepare to pass LCO, LCA, CCD, and IOC anchor point milestone reviews
• Monitoring and Control: Use Balanced
Scorecard to track progress with respect to plans; apply corrective actions as necessary
• Change as Opportunity: Look for emerging
COTS pharmacy-related fulfillment systems
2015/12/02 © USC-CSSE 25

USC
C S E
University of Southern California
Center for Software Engineering
• Software engineers have increasing power to do public harm or good
• Value-based codes of ethics are hard to integrate with value-neutral software engineering practices
• Rawls ’ Theory of Justice enables constructive approach for integrating ethics into daily software engineering practice
– Stakeholder win-win with least-advantaged system dependents as success-critical stakeholders
– Win Win Spiral Model/MBASE/VBSE provides dailypractice framework
2015/12/02 © USC-CSSE 26