Ethics and Values

Every profession has distinct ethical obligations to the public. These obligations include professional competency, integrity, honesty, confidentiality, objectivity, public safety, and fairness.

Stem cell field rocked by scam of star scientist
The star scientist at the heart of that deception –
Hwang Woo Suk -- resigned Friday from his university post, his meteoric rise to fame coming to a wrenching end.
12/24/05
Unfortunately, all too often we hear reports in the media of moral dilemmas and unethical behavior by professionals. These reports naturally receive considerable attention by the public, whose confidence in the profession is undermined with each report.

Houston, Texas
Testing Scandal in Texas Schools
NPR News
March 21, 2005 by Claudia Sanchez

• Concerns about ethical practices in assessment are not new.
• As early as 1972, the National Council on
Measurement in Education (NCME), the
Association for Measurement and Evaluation in
Guidance (AMEG), and the American
Counseling Association (ACA) developed a position paper on the responsible use of tests that was intended to ensure that tests are given, and examinees are treated, fairly and wisely
(AMEG, 1972).

• Later in the 1970s, ACA developed a statement on the responsibilities of the users of standardized tests and evaluation
• That document was revised as recently as
1989 (ACA, 1989).

• Both of these early documents recognized the need to positively influence the practices of those who use tests in ways that promote responsible use.
• These statements have been followed by the development of ethical standards by a number of other organizations having an interest, or directly involved, in testing assessment, and evaluation.

ANSI
The American National Standards Institute (ANSI) is a private, non-profit organization (501(c)3) that administers and coordinates the U.S. voluntary standardization and conformity assessment system.
The Institute's mission is to enhance both the global competitiveness of U.S. business and the U.S. quality of life by promoting and facilitating voluntary consensus standards and conformity assessment systems, and safeguarding their integrity.
Founded in 1918

JOINT COMMITTEE ON STANDARDS FOR
EDUCATIONAL EVALUATION
Created in 1975, the Joint Committee is a coalition of major professional associations concerned with the quality of evaluation. The Joint Committee is housed at The Evaluation Center, Western Michigan University.
The Joint Committee has published three sets of standards for evaluations:
1. The Personnel Evaluation Standards published in 1988
2. The Program Evaluation Standards published in 1994
3. The Student Evaluations Standards was published in 2003
The Joint Committee is accredited by the American National Standards
Institute ( ANSI ). Standards approved by ANSI become American National
Standards.

 Utility Standards
 Feasibility Standards
 Propriety Standards
 Accuracy Standards



To ensure that the evaluation will serve the information needs of intended users
U1 Stakeholder Identification Persons involved in or affected by the evaluation should be identified, so that their needs can be addressed.
U2 Evaluator Credibility The persons conducting the evaluation should be both trustworthy and competent to perform the evaluation, so that the evaluation findings achieve maximum credibility and acceptance.



U3 Information Scope and Selection
Information collected should be broadly selected to address pertinent questions about the program and be responsive to the needs and interests of clients and other specified stakeholders
U4 Values Identification The perspectives, procedures, and rationale used to interpret the findings should be carefully described, so that the bases for value judgments are clear.



U5 Report Clarity Evaluation reports should clearly describe the program being evaluated, including its context, and the purposes, procedures, and findings of the evaluation, so that essential information is provided and easily understood
U6 Report Timeliness and Dissemination
Significant interim findings and evaluation reports should be disseminated in a timely manner.

 U7 Evaluation Impact Evaluations should be planned, conducted, and reported in ways that encourage follow-through by stakeholders, so that the likelihood that the evaluation will be used is increased, and to ensure that the evaluation will serve the information needs of intended users

To ensure that an evaluation will be realistic, prudent, diplomatic, and frugal
F1 Practical Procedures The evaluation procedures should be practical, to keep disruption to a minimum while needed information is obtained.

 F2 Political Viability The evaluation should be planned and conducted with anticipation of the different positions of various interest groups, so that their cooperation may be obtained, and so that possible attempts by any of these groups to curtail evaluation operations or to bias or misapply the results can be averted or counteracted.
 F3 Cost Effectiveness The evaluation should be efficient and produce information of sufficient value, so that the resources expended can be justified

Intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results.
P1 Service Orientation Evaluations should be designed to assist organizations to address and effectively serve the needs of the full range of targeted participants.
P2 Formal Agreements Obligations of the formal parties to an evaluation (what is to be done, how, by whom, when) should be agreed to in writing, so that these parties are obligated to adhere to all conditions of the agreement or formally to renegotiate it.




P3 Rights of Human Subjects Evaluations should be designed and conducted to respect and protect the rights and welfare of human subjects.
P4 Human Interactions Evaluators should respect human dignity and worth in their interactions with other persons associated with an evaluation, so that participants are not threatened or harmed.
P5 Complete and Fair Assessment The evaluation should be complete and fair in its examination and recording of strengths and weaknesses of the program being evaluated, so that strengths can be built upon and problem areas addressed.




P6 Disclosure of Findings The formal parties to an evaluation should ensure that the full set of evaluation findings along with pertinent limitations are made accessible to the persons affected by the evaluation and any others with expressed legal rights to receive the results.
P7 Conflict of Interest Conflict of interest should be dealt with openly and honestly, so that it does not compromise the evaluation processes and results.
P8 Fiscal Responsibility The evaluator's allocation and expenditure of resources should reflect sound accountability procedures and otherwise be prudent and ethically responsible, so that expenditures are accounted for and appropriate

Intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated.
A1 Program Documentation The program being evaluated should be described and documented clearly and accurately, so that the program is clearly identified.
A2 Context Analysis The context in which the program exists should be examined in enough detail, so that its likely influences on the program can be identified.
A3 Described Purposes and Procedures The purposes and procedures of the evaluation should be monitored and described in enough detail, so that they can be identified and assessed

A4 Defensible Information Sources The sources of information used in a program evaluation should be described in enough detail, so that the adequacy of the information can be assessed.
A5 Valid Information The information-gathering procedures should be chosen or developed and then implemented so that they will assure that the interpretation arrived at is valid for the intended use.
A6 Reliable Information The information-gathering procedures should be chosen or developed and then implemented so that they will assure that the information obtained is sufficiently reliable for the intended use.

A7 Systematic Information The information collected, processed, and reported in an evaluation should be systematically reviewed, and any errors found should be corrected.
A8 Analysis of Quantitative Information Quantitative information in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered.
A9 Analysis of Qualitative Information Qualitative information in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered.

A10 Justified Conclusions The conclusions reached in an evaluation should be explicitly justified, so that stakeholders can assess them.
A11 Impartial Reporting procedures should guard against distortion caused by personal feelings and biases of any party to the evaluation, so that evaluation reports fairly reflect the evaluation findings.
A12 Meta-evaluation The evaluation itself should be formatively and summatively evaluated against these and other pertinent standards, so that its conduct is appropriately guided and, on completion, stakeholders can closely examine its strengths and weaknesses.

Simply stated, meta-evaluation is the evaluation of an evaluation, evaluation system or evaluation device to enhance the validity of the evaluation results.
Operationally, metaevaluation is also defined as the process of delineating, obtaining, and applying descriptive information and judgmental information - about the utility,
feasibility, propriety, and accuracy of an evaluation and its systematic nature, competent conduct, integrity/honesty, respectfulness, and social responsibility - to guide the evaluation and/or report its strengths and weaknesses.




Utility Standards – serve the information needs of the intended users
Feasibility Standards
– realistic, prudent, diplomatic, and frugal
Propriety Standards
– legal, ethical, and due regard for welfare of participants
 Accuracy Standards –reveal and convey information to determine the worth, value, and merit of the program evaluated

 Ratified by the AEA membership, July 2004
 Supports Four Program Evaluation Standards

1. To ensure the accuracy and credibility of the evaluative information they produce, evaluators should adhere to the highest technical standards appropriate to the methods they use.
2. Evaluators should explore with the client the shortcomings and strengths both of the various evaluation questions and the various approaches that might be used for answering those questions.
3. Evaluators should communicate their methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique their work..

Principle 2 Competence
Evaluators provide competent performance to stakeholders



1. Evaluators should possess the education, abilities, skills and experience appropriate to undertake the tasks proposed in the evaluation.
2. To ensure recognition, accurate interpretation and respect for diversity, evaluators should ensure that the members of the evaluation team collectively demonstrate cultural competence.
3. Evaluators should practice within the limits of their professional training and competence, and should decline to conduct evaluations that fall substantially outside those limits.
4. Evaluators should continually seek to maintain and improve their competencies, in order to provide the highest level of performance in their evaluations.

Principle 3 Integrity/Honesty: Evaluators must display honesty and integrity in their own behavior and in the evaluation process.




1. Evaluators should negotiate honestly with clients and relevant stakeholders concerning the costs, tasks to be undertaken, limitations of methodology, scope of results likely to be obtained, and uses of data resulting from a specific evaluation.
2. Before accepting an evaluation assignment, evaluators should disclose any roles or relationships they have that might pose a conflict of interest with their role as an evaluator.
3. Evaluators should record all changes made in the originally negotiated project plans, and the reasons why the changes were made.
4. Evaluators should be explicit about their own, their clients', and other stakeholders' interests and values concerning the conduct and outcomes of an evaluation.




5. Evaluators should not misrepresent their procedures, data or findings.
6. If evaluators determine that certain procedures or activities are likely to produce misleading evaluative information or conclusions, they have the responsibility to communicate their concerns and the reasons for them. If discussions with the client do not resolve these concerns, the evaluator should decline to conduct the evaluation.
7. Evaluators should disclose all sources of financial support for an evaluation, and the source of the request for the evaluation
.

Principle 4 Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders.
 1. Evaluators should seek a comprehensive understanding of the important contextual elements of the evaluation.
 2. Evaluators should abide by current professional ethics, standards, and regulations regarding risks, harms, and burdens that might befall those participating in the evaluation; regarding informed consent for participation in evaluation; and regarding informing participants and clients about the scope and limits of confidentiality.
 3. Because justified negative or critical conclusions from an evaluation must be explicitly stated, evaluations sometimes produce results that harm client or stakeholder interests. Under this circumstance, evaluators should seek to maximize the benefits and reduce any unnecessary harms that might occur, provided this will not compromise the integrity of the evaluation findings.

Principle 4 Respect for People (continued)



4. Knowing that evaluations may negatively affect the interests of some stakeholders, evaluators should conduct the evaluation and communicate its results in a way that clearly respects the stakeholders' dignity and self-worth.
5. Where feasible, evaluators should attempt to foster social equity in evaluation, so that those who give to the evaluation may benefit in return. For example, evaluators should seek to ensure that those who bear the burdens of contributing data and incurring any risks do so willingly. Program participants should be informed that their eligibility to receive services does not hinge on their participation in the evaluation.
6. Evaluators have the responsibility to understand and respect differences among participants, such as differences in their culture, religion, gender, disability, age, sexual orientation and ethnicity, and to account for potential implications of these differences when planning, conducting, analyzing, and reporting evaluations.

Principle 5
Responsibilities for General and Public Welfare
Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.


1. When planning and reporting evaluations, evaluators should include relevant perspectives and interests of the full range of stakeholders.
2. Evaluators should consider not only the immediate operations and outcomes of whatever is being evaluated, but also its broad assumptions, implications and potential side effects

Principle 5
(continued)
Responsibilities for General and Public Welfare


3. Freedom of information is essential in a democracy. Evaluators should allow all relevant stakeholders access to evaluative information in forms that respect people and honor promises of confidentiality. In all cases, evaluators should strive to present results clearly and simply so that clients and other stakeholders can easily understand the evaluation process and results.
4. Evaluators should maintain a balance between client needs and other needs. Evaluators necessarily have a special relationship with the client who funds or requests the evaluation. By virtue of that relationship, evaluators must strive to meet legitimate client needs whenever it is feasible and appropriate to do so. However, that relationship can also place evaluators in difficult dilemmas when client interests conflict with other interests, or when client interests conflict with the obligation of evaluators for systematic inquiry, competence, integrity, and respect for people.

Principle 5
(continued)
Responsibilities for General and Public Welfare
 5. Evaluators have obligations that encompass the public interest and good. These obligations are especially important when evaluators are supported by publicly-generated funds; but clear threats to the public good should never be ignored in any evaluation. Because the public interest and good are rarely the same as the interests of any particular group (including those of the client or funder), evaluators will usually have to go beyond analysis of particular stakeholder interests and consider the welfare of society as a whole.






1. Systematic Inquiry: Evaluators conduct systematic, databased inquiries about whatever is being evaluated.
2. Competence: Evaluators provide competent performance to stakeholders.
3. Integrity/Honesty: Evaluators ensure the honesty and integrity of the entire evaluation process.
4. Respect for People: Evaluators respect the security, dignity and self-worth of the respondents, program participants, clients, and other stakeholders with whom they interact.
5. Responsibilities for General and Public Welfare:
Evaluators articulate and take into account the diversity of interests and values that may be related to the general and public welfare.

 Four Program Evaluation Standards
Utility, Feasibility, Propriety, & Accuracy
 Five Guiding Principles for Evaluators
Systematic Inquiry, Competence, Integrity/Honesty,
Respect for People, Responsibilities for General &
Public Welfare

Program Design & Evaluation
Standards
Four Program
Evaluation Standards
Principles
Five Guiding Principles for Evaluators
SCIRR UFPA
U tility,
F easibility,
P ropriety,
A ccuracy
S
C ystematic Inquiry, ompetence,
I ntegrity/Honesty,
R espect for People,
R esponsibilities for
General & Public
Welfare

PROGRAM
DESIGN
&
EVALUATION
PROCESS
4 Standards
5 Principles
6 Steps
Define Purpose
Engage
Stakeholders
Report
Evaluation
Ensure Use &
Share Lessons
5 Guiding Principles
Describe
Program
Evaluation
Analyze Data
Justify
Conclusions 5 Guiding Principles
Focus Design,
Develop
Methodology,
Data Collection
Plan
Collect the
Data & Gather
Evidence

When conducting program evaluations, we must always remember the words of wisdom coined by a famous scientific researcher and evaluator:
“ Not everything that can be counted counts…and not everything that counts can be counted.”