January 20, 2013

1.
KEY BACKGROUND PLANNING TERMS
Accountability. Clear assignment of ownership for results or performance on a particular metric.
Accountability for a particular result or metric may be shared by units or individuals, but levels of accountability will be defined by percentages or as primary and secondary responsibilities. (3)
Strategic Goals. Goals that relate to the achievement of the vision. These tend to change as new challenges arise or a new vision is created. (3)
Strategic Plan. A document that spells out the organization’s mission, vision, values, goals, objectives, metrics/targets, and key strategies/initiatives. A strategic plan typically addresses how an organization will continue to perform its ongoing responsibilities (mission) as well as how it will address threats and opportunities for the future (vision). Having a clear strategic plan is a prerequisite to having a good balanced scorecard. (3)
Strategies. Projects, initiatives, and activities that will be used to achieve goals, objectives, and targets.
Strategies define how results will be achieved and typically include verbs or actions words. Strategies may include a broad collection of actions followed by sub-strategies. For example, a strategy for becoming profitable again may be to down-size the organization. This might include sub-strategies such as reduction in headcount, closing of non-performing offices, changing the organization structure, and cutting overhead budgets. (3)
2
Targets. A desired level of performance or range on a metric. Targets should be based on historical data, benchmarks (industry best) averages, stakeholder needs, and resource constraints such as time, money, people, and technology. A metric without a target is meaningless. Targets allow you to look at performance on a metric and determine whether or not you are doing well. (3)
Mission Statement. A mission statement defines the core purpose of the organization – why it exists.
The mission examines the “raison d’etre” for the organization beyond simply increasing shareholder wealth, and reflects employees’ motivations for engaging in the company’s work. Effective missions are inspiring, long-term in nature, and easily understood and communicated.
Vision. A powerful vision provides everyone in the organization with a shared mental framework that helps give form to the often abstract future that lies ahead. Effective visions provide a word picture of what the organization intends ultimately to become – which may be five, ten, fifteen years in the future.
This statement should not be abstract – it should contain as concrete a picture of the desired state as possible, and also provide the basis for formulating strategies and objectives.
Strategic Goals. Goals that relate to the achievement of the vision. These tend to change as new challenges arise or a new vision is created.
Goals. Statements of desired outcome or achievement that have not yet occurred. Goals are not projects or activities. They are not ongoing responsibilities of a department or unit. The best goals are
SMART: specific, measurable, achievable, realistic, and timely. (3)

Operational Goals. Goals that relate to the ongoing responsibilities of an organization or its mission.
These tend to remain the same from year to year, even though specific targets may change. For example, “maintain profitability” might be an operational goal that does not change. (3)
Objective. Objectives are similar to goals but narrower in focus. A single goal may have several objectives. Objectives typically include an outcome or achievement, a metric or measure, and a date by which the objective will be achieved. Like goals, objectives should be specific, measurable, achievable, realistic, and timely (SMART). (3)
3
Objectives often begin with an action verb (e.g., increase, reduce, achieve).
2.
PEFORMANCE MANAGEMENT SYSTEM
Performance Management System. A systematic approach for defining expectations, goals, and measures or targets; reviewing performance against goals or targets; and managing performance to achieve outcomes. An effective performance management system includes a good scorecard and processes for reviewing performance and administering positive and negative consequences for performance. An effective system ties together goals and plans with metrics/scorecards, targets/objectives, reporting systems, and human resource systems such as performance appraisal and merit pay. (3)
Performance Criteria. The categories, aspects, or dimensions of performance to be measured.
Performance Measurement. Performance measurement is the ongoing monitoring and reporting of program accomplishments, particularly progress toward pre-established goals.
Performance measures may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), or the results of those products and services (outcomes).
A “program” may be any activity, project, function, or policy that has an identifiable purpose or set of objectives.
(U.S. Government Accountability Office, May, 2011).
Critical Success Factor (CSF). Is the term for an element that is necessary for an organization or project to achieve its mission. It is a critical factor or activity required for ensuring the success of a company or an organization. Critical success factors are those few things that must go well to ensure success for a manager or an organization, and, therefore, they represent those managerial or enterprise area, that must be given special and continual attention to bring about high performance. CSFs include issues vital to an organization’s current operating activities and to its future success. Critical success factors should not be confused with success criteria; those are outcomes of a project or achievements of an organization that are needed to consider the project a success or to esteem the organization successful.
(Wikipedia, Oct. 18, 2012)

4

3.
BALANCED SCORECARD
Balanced Scorecard. A management framework, originally developed by Robert Kaplan and David
Norton, that translates an organization’s mission and strategy into a comprehensive set of performance measures. The standard scorecard measures organizational performance across four perspectives: financial, customers, internal business processes, and learning and growth.
5
Financial Perspective. One of the four standard perspectives used with the Balanced Scorecard.
Financial measures inform an organization whether strategy execution, which is detailed through measures in the other three perspectives, is leading to improved bottom line results. In public sector and not for profit applications of the Balanced Scorecard measures in the Financial perspective are often viewed as constraints within which the organization must operate.
Customer Perspective. One of the four standard perspectives used with the Balanced Scorecard.
Measures are developed based on the answer to two fundamental questions – who are our target customers and what is our value proposition in serving them? The role of the Customer perspective is often elevated in public sector and not for profit applications of the Balanced Scorecard.
Internal Process Perspective. One of the four standard perspectives used with the Balanced Scorecard.
Measures in this perspective are used to monitor the effectiveness of key processes the organization must excel at in order to continue adding value for customers, and ultimately, shareholders.
Employee Learning and Growth Perspective. One of the four standard perspectives used with the
Balanced Scorecard. Measures in this perspective are often considered “enablers” of measures appearing in the other three perspectives. Employee skills, availability of information, and organizational climate are often measured in this perspective.
Alignment. All major systems used to drive performance in an organization must be aligned. Goals link to the mission and vision, objectives are derived from goals, metrics and targets in the scorecard are defined based on goals and objectives, and strategies, projects, and initiatives are implemented to achieve goals and objectives. Every major project or initiative can be linked to a goal, objective, and performance metric. An organization is aligned when the following systems and documents are linked together: strategic plans, operational plans, balanced scorecards/metrics, individual performance plans/agreements, reporting and review processes, performance appraisals, compensation, information systems, and key reports to senior management and other stakeholders. (3)
Cascading System. A performance measurement system where measures are linked to goals and objectives in a formal strategic plan or list of organization wide priorities. High level organizational goals and measures correspond to the objectives measured by departments, divisions, and employees. (2)

4.
TYPES OF PROGRAM EVALUATION
Process (or Implementation) Evaluation. This form of evaluation assesses the extent to which a program is operating as it was intended. It typically assesses program activities conformance to statutory and regulatory requirements, program design, and professional standards or customer expectations. (4)
Outcome Evaluation. This form of evaluation assesses the extent to which a program achieves its outcome-oriented objectives. It focuses on outputs and outcomes (including unintended effects) to judge program effectiveness but may also assess program process to understand how outcomes are produced. (4)
6
Impact Evaluation. Impact evaluation is a form of outcome evaluation that assesses the net effect of a program by comparing program outcomes with an estimate of what would have happened in the absence of the program. This form of evaluation is employed when external factors are known to influence the program’s outcomes, in order to isolate the program’s contribution to achievement of its objectives. (4)
Cost-Benefit and Cost-Efficiency Evaluation. These analyses compare a program’s outputs or outcomes with the costs (resources expended) to produce them. When applied to existing programs, they are also considered a form of program evaluation. Cost-effectiveness analysis assesses the cost of meeting a single goal or objective and can be used to identify the least costly alternative for meeting that goal.
Cost-benefit analysis aims to identify all relevant costs and benefits, usually expressed in dollar terms.
(4)
5.
INPUTS, OUTPUTS, OUTCOMES, IMPACT
Inputs. The raw materials, resources, or information needed to begin and successfully complete a process. Inputs typically include equipment and tools, people, knowledge and skill, expectations and goals, directions, deadlines, procedures, rules and policies, materials and supplies, and data such as information about customer needs. (3)
Outputs. The physical products or accomplishments that result from correctly completing all steps in a process. Outputs can be tangible things such as cars and computers or reports and presentations. They can also be intangible things such as problems solved, deficiencies identified or services performed correctly. For example, an airline’s output is passengers delivered to their destination on time. A bank’s output might be a loan or new account opened. A hospital’s output might be an operation completed successfully. (3)
Outcomes. The results or accomplishments that are attributable, at least in part, to a service or program. (1)
Initial Outcome. The result of a product or service that occurs immediately or very soon after the product is delivered or the service is provided. For example, the initial outcome of a program to inform residents about the value of recycling might be the level of awareness the residents have after attending the program. (The long-term objective would be to increase recycling.) (1)

Intermediate Outcome. A measure of results that indicates progress toward desired end results but is not itself a final outcome. (1)
3.
4.
5.
Final Outcome. An end objective. The end result that is desired or anticipated. (1)
Logic Model. A visual representation displaying all inputs, outputs, and outcomes along with the links connecting each. Logic Models are used to link individual inputs to wide program or organizational goals. (2)
6.
RUBRICS
Rubrics. Are a scoring scale consisting of a set of criteria that describes what expectations are being assessed … they provide specificity concerning expectations or standards.
7.
METRICS/MEASURES
Metric/Measure. A dimension of performance or an object that can be consistently counted and graphed to assess levels and trends. There are two types of metrics: counting (for example, a dollar amount in loans or the number of problem projects) and Judgment (for example, audit scores, survey ratings 1-5). Metrics should link to goals or objectives and provide hard evidence of progress toward achievement. Gauges on a car’s dashboard function as metrics. For example, the speedometer measures miles per hours, a metric, and the tachometer measures RPMs, or revolutions per minute, another metric. Metrics must have targets in order to provide useful information for guiding performance and decision making. Metrics might also be called “key performance indicators.” (3)
Analytic. Indices or metrics on the scorecard that comprise lower level metrics addressing the same dimension of performance but using different units of measurement. For example, a health analytic might include waist size in inches, ratio of good and bad cholesterol in mg., and resting heart rate in beats per minute. A credit score is an analytic that comprises payment history, available credit, income, and ratios such as income/monthly obligations. An index and an analytic are the same thing. (3)
8.
CRITERIA/POLICY EVALUATIONS AND APPROACHES TO COST-ANALYSIS STUDIES
Criteria/Policy Evaluation.
1.
2.
effectiveness … has a value outcome been achieved? efficiency … how much effort was required to achieve a valued outcome?
6.
adequacy … to what extent does the achievement of a valued outcome resolve the problem? equity … are costs and benefits distributed equitably among different groups? responsiveness … do policy outcomes satisfy the needs, preferences or values of particular groups? appropriateness … are derived outcomes actually worthy or valuable? (Dunn, 2004, p. 358)
7

Cost Analysis – Four Major Approaches.
1.
cost-effectiveness
Which alternative yields a given level of effectiveness for the lowest cost (or the highest level of effectiveness for a given cost)?
2.
cost-benefit
Which alternative yields a given level of benefits for the lowest cost (or the highest level of benefits for a given cost)? Are the benefits of a single alternative larger than its costs?
8
3.
cost-utility
Which alternative yields a given level of utility at the lowest cost (or the highest level of utility at a given cost)?
4.
cost-feasibility
Can a single alternative be carved out within the existing budget? (Levin & McEwan, 2011, pp. 27-28)
9.
MISCELLANEOUS
Opportunity Costs. The cost of an alternative that must be forgone … in order to pursue the achievement of a certain action, that is, the benefits received by taking an alternative action.
Quality of Life Measures. Generally, community or national indicators that provide a basis for developing a comprehensive picture of overall well-being. The variables included in definitions of quality of life are diverse, complex, and wide ranging. The indicators include traditional economic measures of employment, income distribution, public safety, education, and housing, along with assessments of environmental factors, infrastructure, health, and education. (1)
Explanatory Factors. Influences that might affect an organization’s performance. Usually it is used to refer to factors outside the control of the organization that have an effect on performance data. (2)
Factors That Influence Results. Includes a variety of information about the environment and other internal and external factors that might influence an ESA performance. They can be either factors substantially outside the control of the entity, such as environmental and demographic characteristics, or factors over which the entity has some control, such as staffing levels.

1.
Government Accounting Standards Board, July 2010
2.
Government Financial Officers Association of U.S. and Canada, January 2013
3.
Pocket Guide to Performance Management, ASQ Quality Press, 2010 (Brown)
4.
U.S. Government Accountability Office, May 2011
9