Ethical Data
Looking at What It Is
and How We Use It
HarrietAnn Litwin, M.Ed., CRC
Darlene Groomes, Ph.D., CRC, LPC
Kathe Matrone, Ph.D., CRC
6th Summit Conference
Rhode Island
September 16, 2013
What do we mean by ETHICS?
• A set of principles of proper conduct.
• The rules or standards governing the conduct
of a person or the members of a profession.
The Forum Guide to Data Ethics,
National Forum on Education Statistics,
February, 2010
What is a Code of Ethics?
• Comprehensive collection of “laws”
• Documents standards that governs
expected/acceptable/moral behavior in a
group
• Written agreement among participants that
includes sanctions for unethical behavior
• Systematically arranged
• Values-driven: begins by setting out the values
of the group
Framework for today’s discussion
Using the Office of Management and Budget’s
framework including:
•Data Utility
Including data quality
•Data Objectivity
•Data Integrity
Including data security
Codes that will be used today
Among many possible sets of principles, we
have selected two:
•Commission on Rehabilitation Counselors
Certification
http://www.crccertification.com/filebin/pdf/CRCCodeOfEthics.pdf
•American Evaluation Association
http://www.eval.org/p/cm/ld/fid=51
Relationship with Organizational
Performance
• Much like ethics in counseling and guidance espouses the
relationship with client as paramount, data ethics champions
its relationship with performance excellence
• All stakeholders have an interest in the results
– Take steps to assure that evaluation results are accurately understood
and accurately communicated
– Employ evaluation results in decision-making
• Honesty in the research process and in the presentation of
results leads to accurate identification of strengths and
need in the development of strategic planning
– Data security and confidentiality leads to accurate information
– Full and accurate reporting lead to better decisions
Ethical Framework: Utility
(Forum Code of Ethics, 2010; National Center for Education Statistics, 2002)
• Usefulness of information
• Provide all relevant data and context, definitions,
documentation
• Promote understanding and accuracy of analysis
• Staying informed of stakeholder requirements/needs
• Goal to maximize usefulness, minimize cost
– Crosstalk between professional organizational codes:
• CRC Section 1: Research and Publication, 1.3: Reporting Results
• AEA Guiding Principles A – E, A: Systematic Inquiry
Ethical Framework: Utility
• Usefulness of available dissemination channels
• Provide stakeholders ability to locate information in
timely and equitable fashion
• Provide contact information on the report to facilitate
feedback and questions
– Crosstalk between professional organizational codes:
 CRC Section 1: Research and Publication, 1.4: Publications
and Presentations
 AEA Guiding Principles A – E, E: Responsibilities for General
and Public Welfare
Ethical Framework: Utility
• Mindful of Data Quality through best practices
and standards for operation:
•
•
•
•
Practical so most useful to stakeholders
Free of error and bias
Consistent, reproducible, dependable
Readily available for decision-making
– Crosstalk between professional organizational codes:
 CRC Section 1: Research and Publication, 1.3: Reporting
Results
 AEA Guiding Principles A – E, C: Integrity/Honesty
Utility: Recommended Practice
• Develop a culture of data quality
– Use best practice resources
– Update data practices to reflect changes in policy
– Train data handlers
•
•
•
•
Improvement mindedness
Privacy
Federal race and ethnicity categories
Utilization of decision support system
Utility: Recommended Practice
• Data presentation (subject to multiple reviews with comments
passed on to authors)
– Explain statistical procedures
– Summarize methodology
– Include definitions and related information to
support interpretation of data
– Add explanation to avoid misinterpretation
– Provide contact information
Ethical Framework: Objectivity
• Refers to accuracy and clarity of information
– Content of Information
– Presentation of Information
• Achieved by using reliable information sources and
appropriate techniques to prepare products.
– Crosstalk between professional organizational codes:
 CRC Section I: Research and Publications, 1.3: Reporting Results
 AEA Guiding Principles A-E, A: Systematic Inquiry
Ethical Framework: Objectivity
• Content
– Ensure accurate, reliable, and unbiased
information
– Appropriate research and statistical methods used
– Limitations, assumptions, collection methods, and
uncertainties concerning data disclosed
– Crosstalk between professional organizational codes:
 CRC Section I: Research and Publications, 1.3: Reporting Results
 AEA Standards: Accuracy
Ethical Framework: Objectivity
• Presentation
– The information is presented within a proper context
– The research design and methods are transparent
including the disclosure of data sources
– Any issues affecting the data quality are identified and
disclosed
– Crosstalk between professional organizational codes:
 CRC Section I: Research and Publications, 1.3: Reporting Results
 AEA Standards: Accuracy
Ethical Framework: Objectivity
• Questionable Practices:
– Data trimming and cooking
– Not stating the methods used
– Findings are suppressed or ignored
– Crosstalk between professional organizational codes:
 CRC Section I: Research and Publications, 1.2: Informed Consent &
Disclosure
 AEA Principles: Systematic Inquiry, Integrity, Responsibilities for
public welfare
Objectivity: Recommended Practice
• Data handlers are expected to report
information accurately and without bias, even
when the news is bad!
• Train data reporters and users to follow a
standard data preparation and presentation
methods.
– Crosstalk between professional organizational codes:
 CRC Section I: Research and Publications, 1.3: Reporting Results
 AEA Principles: Responsibilities for public welfare
Objectivity: Recommended Practice
• How you start and finish the evaluation
process need particular attention
• Disseminating the evaluation is a vital aspect
of the evaluation
• Crosstalk between professional organizational
codes:
 CRC Section I: Research and Publications, 1.3: Reporting Results
 AEA Principles: Responsibilities for public welfare
Ethical Framework: Integrity
Integrity
• security or protection of information from
unauthorized access or revision
• ensures that the information is not
compromised through corruption or
falsification
The Forum Guide to Data Ethics,
National Forum on Education Statistics,
February, 2010
NCES Canons on Integrity and Security
• Demonstrate honesty, integrity, and
professionalism at all times
• Appreciate that, while data may represent
attribute of real people, they do not describe
the whole person
• Be aware of applicable statutes, regulations,
practices, and ethical standards governing
data collection and reporting
NCES Canons on Integrity and Security
(continued)
• Report information accurately and without
bias.
• Be accountable and hold others accountable
for ethical use of data
• Treat data systems as valuable organizational
assets.
• Safeguard sensitive data to guarantee privacy
and confidentiality.
Integrity/Security: Crosstalk
– CRCC Section I: Research and Publications
 I.1 Research Responsibilities
 I.2 Informed Consent and Disclosure
 I.3 Reporting Results
 I.4 Publication and Presentation
 I.5 Confidentiality
– Section G.1 includes a reminder to report results in their cultural context.
– AEA
 Section C: Integrity/Honesty
 Section D: Respect for People
Integrity/Security: Recommended
Practice
• Respect for the individuals who are the
subject of the research
• Identify conflicts of interest
• Recognize all of the individuals who contribute
to the success of the project
• Understand that the responsibility for
accuracy and confidentiality lies with the
practitioner
Utility: Ethical Dilemma Scenario
Customer satisfaction scores were up! The
agency was ready to disseminate the
improvement in results, except:
• Management and staff knew that the survey
had been updated in design and approach
• More detailed questions included on new version
• Offered to administer in more ways than just mail
Objectivity: Ethical Dilemma Scenario
• After reading the first draft of an evaluation
report on the effectiveness of a new initiative
(highly supported by the Director), the
program manager suggested to the evaluator
the deletion of some negative findings that
the initiative was not making the impact.
Integrity: Ethical Dilemma Scenario
You conducted a study that drew a random
sample that was representative of your entire
population. In reviewing your responses, you
find that those who responded were not
representative of the entire population, but
rather favored one group disproportionately.
While you have now learned that there were
actions that might have prevented this type of
response, you are asked to set goals and
develop plans using this data.
Thank you for participating!
HarrietAnn Litwin, M.Ed., CRC
Delaware Division of Vocational Rehabilitation
(302) 761-8275
HarrietAnn.Litwin@state.de.us
Darlene Groomes, Ph.D., CRC, LPC
Oakland University
(517) 896-9807
Groomes@oakland.edu
Kathe Matrone, PhD, CRC
University of Washington
(425) 771-7436
matrok@uw.edu