Performance Audit Survey: Developing
Preliminary Findings and Audit Objectives
Handout for 07/19-20/2010
Appendices A-N
And Instructor Biography
1
Instructor Biography
Stephen L. Morgan, CIA, CGAP, CGFM, CFE, is currently the President of
Excellence in Government Accountability and Program Performance, a
company that specializes in training government auditors and managers.
Mr. Morgan is the former city auditor of Austin, Texas, who directed a full
scope audit office that conducts performance audits, fraud investigations,
and consulting engagements. Mr. Morgan initiated performance auditing in
Austin and created a training program for transitioning financial auditors into
performance auditing. Later he played a key leadership role in helping the
City of Austin evolve its performance measurement and management
system into a model for other government organizations. Before joining the
City Auditor’s Office, Mr. Morgan was an evaluator in the U.S. Government
Accountability Office’s National Productivity Group. For 20 years, Mr.
Morgan has provided training course design and delivery in performance
measurement, management, auditing.
Mr. Morgan’s Institute of Internal Auditors’ offices have included president
and governor, Austin Chapter, chair of the International Government
Relations Committee, chair of the North American Nominating Committee,
North American Director on the Global Board, and chair of the North
American Board. In January 2001 Mr. Morgan was appointed (reappointed
in 2005) by the Comptroller General of the United States to the Advisory
Council on Government Auditing Standards. He co-authored three
textbooks: Performance Auditing: A Measurement Approach (first and
second editions) and Auditor Roles in Government Performance
Measurement: A Guide to Exemplary Practices at the Local, State, and
Provincial Levels.
In May 2009, Mr. Morgan received the Victor Z. Brink Memorial Award, IIA’s
highest award for leadership and service to the global internal auditing
profession. In May 2007, Mr. Morgan accepted the National
Intergovernmental Audit Forum’s Excellence in Government Performance
and Accountability Award from the Comptroller General of the United States.
2
Also, in March 2002, Mr. Morgan became the fourth annual recipient of the
Harry Hatry Distinguished Performance Measurement Practice Award from
the American Society of Public Administration honoring his lifetime of
contributions to public service. Mr. Morgan holds a Bachelor of Arts degree in
government (with honors) from the University of Texas at Austin; he was
elected to Phi Beta Kappa. He also holds an MPA from the Lyndon B. Johnson
School of Public Affairs of the University of Texas.
Contact at egappmorgan@yahoo.com
512-244-9789
3
Appendix A
Crosscutting Performance Goals
The following crosscutting goals span and bind the four performance
accountability goals. Together they provide the basis for inspiring and
upholding public trust in government operations.
Compliance with Laws and Regulations – knowledge and wise
management of applicable laws and regulations is a basic expectation of all
government employees. Compliance involves:
1. Interpretation – establishing clear legislative and regulatory intent.
2. Responsibility and authority – defining parameters and establishing
who is responsible for what.
3. Reasonableness – applying laws and regulations in a practical and
cost/beneficial manner.
4. Documentation – providing reasonable proof that required compliance
was attained.
Safeguarding of Assets – Safeguarding controls constitute a process
designed to provide reasonable assurance of prevention or timely detection
of unauthorized acquisition, use, or disposition of an entity’s assets including
theft, exposure to elements, and waste. Safeguarding controls are meant to
provide protection for both an entity’s resources and its infrastructure. A
strong system of internal controls is the best safeguarding measure.
Some examples of specific safeguarding controls include:
1. Using pre-numbered invoices and checks for purchasing and
payments.
2. Separating the purchasing function from the payment function for
acquisition of assets.
3. Restricting access to assets (e.g., keeping spare parts in a locked
warehouse).
4. Requiring authorized signatures for purchase invoices and checks.
4
Continuous Improvement - government leaders should establish an
environment which encourages ongoing improvement in government
services through creativity, experimentation, and innovation. For example,
this environment might include:
1. Four principal management tactics.
A. Supporting persistent and passionate champions (day to day
workers) of innovation in face of opposition and low odds.
B. Management consistently stands up for and models innovation.
C. Supporting thoughtful failures from which something is learned,
and defying silly rules which impede fast action.
D. Demanding innovation through measurement and reward
systems, which apply hard number targets to what has
traditionally been conceived as a soft variable.
2. Government managers and employees should be encouraged to
develop strong communication skill (speaking, writing, listening,
thinking, negotiating, and facilitating).
3. Governments should place emphasis on technical skills, integrating
new skills and upgrading existing skills.
Reliability, Validity, and Availability of Information – the
characteristics of information needed to demonstrate results, make
decisions, measure risk and reduce uncertainty.
1. Reliability – controls which help ensure consistency, stability, and
minimize error. Examples are standard classification methods,
training, supervision, automated tests, and reconciliations.
2. Validity – information, free of error or bias, that is related to and helps
measure attainment of mission and goals. Validity includes comparing
actual results to widely accepted standards.
3. Availability – systems should be in place to gather, record, and analyze
information. Responsibility is assigned for each input, process, output,
and outcome measure.
Underlying Values – the existence of ethics, honesty, and integrity
together with the necessary knowledge, skills and abilities needed to achieve
the organization’s goals. Examples of underlying values:
1. Government leaders should establish an environment where
fundamental values are evident and practiced.
2. Programs should be established and administered equitably.
5
3. Cooperation and partnership should be a goal within all levels of
government and the private sector.
4. Managers and employees should be informed of expectations and held
accountable for their job performance though fair and effective
employee performance appraisal systems.
5. The work environment should include a commitment to safety, equal
opportunity, and individual development.
Customer and Stakeholder Satisfaction – measures of customer of
stakeholder perception of whether the inputs, processes, outputs, or
outcomes met expectations or requirements. Measurement data is collected
through surveys, focus groups, or similar techniques.
6
Appendix B
Audit Objectives Identification
(Survey Phase)
An objective is a description of what the audit is to accomplish. Subobjectives are used to define specific elements of work that should be done
to achieve the objectives.
Precise objectives provide clear direction for planning, fieldwork, and
reporting. They leave no doubt about what the auditor expects (or is
asked to find out about the issue or issues selected for audit fieldwork.
Starting an audit assignment with precise objectives can save many
hours of wading through and accumulating reams of data that
ultimately turn out to be useless to the audit and never end up in the
report. Also conducting an audit without a clear sense of direction
increases the possibility that the audit will not achieve the intended
purpose. (Raaum and Morgan, Performance Auditing: A Measurement
Approach)
Some characteristics of a good audit objective are:




Precisely worded as a question(s) or as “To determine whether---“
statement(s),
Eliminate ambiguous, abstract, or unfocused terms,
Clearly identify the subject, and
State the performance aspect under review.
The following is a list of potential performance audit objectives at a high
level reflecting only performance aspects. The objectives are derived from
the performance expectations model and do not contain the subject being
audited nor do they contain specific audit criteria.
7
1. Determine whether and how the consumption of financial resources can
be reduced (economy).
2. Determine whether and how the consumption of physical resources can
be reduced (economy).
3. Determine whether and how unit costs can be reduced (efficiency).
4. Determine whether and how productivity can be improved (efficiency).
5. Determine whether and how the quantity of outputs can be increased
(output effectiveness).
6. Determine whether and how the quality of outputs can be improved
(output effectiveness).
7. Determine whether and how the timeliness of outputs can be improved
(output effectiveness).
8. Determine whether and how the price of outputs can be reduced (output
effectiveness).
9. Determine whether the expected outcomes are or are not being
achieved (outcome effectiveness).
10.Determine why the expected outcomes are or are not being achieved
(outcome effectiveness).
11.Determine whether the entity is and will continue to be financially viable
(outcome effectiveness).
12.Determine what can be done to improve the entity’s financial viability
(outcome effectiveness).
13.Determine the cost benefit or cost effectiveness of the department,
program, or activity (outcome effectiveness).
14.Determine how to make the department, program, or activity more cost
beneficial or cost effective (outcome effectiveness).
15.Determine whether the entity is in compliance with laws and
regulations.
16.Determine how to bring the entity into compliance with laws and
regulations.
17.Determine whether organizational resources (tangible or intangible
assets) are properly safeguarded.
8
18.Determine how to better safeguard organizational resources.
19.Determine whether information is reliable (includes accuracy, timeliness,
completeness and consistency).
20.Determine whether information is valid/relevant.
21.Determine whether information is available.
22.Determine how to improve reliability, validity, and availability of
information.
23.Determine whether performance information is being used to support
decision-making.
24.Determine how to enhance the use of information to improve decisionmaking.
25.Determine whether management’s organizational values are appropriate
including integrity, honesty, and diversity.
26.Determine whether customer and stakeholder satisfaction is high
enough.
27.Determine how to increase customer and stakeholder satisfaction.
9
Appendix C
Methods/Sources for Establishing
Criteria, Condition, Cause, and Effect
Criteria








Benchmark to comparable programs
Customer expectations/demands
Program intent
Internally established targets
Compare individual comparable units within same organization
Industry or sector standards
Historical trends (benchmark to prior periods)
Optimal (may be average) performance achieved in a trend or by
comparable units
 Working time compared to actual elapsed time
 Control group
Condition
 Analyze existing performance data gathered by audited entity
 Analyze available performance data gathered by outside organization
 Develop ad hoc performance measurement system (control for variables
and/or qualify results)
Effect
 Quantify difference between criteria and condition
 Identify impacts on organization or customers of not meeting standard
10
Cause
 Identify and verify barriers and constraints to achieving standard (e.g.,
inadequate resources, external variables or acts of providence)
 Assess support systems (clear expectations, timely feedback,
empowerment, process improvement efforts) provided to staff
 Evaluate accountability systems in place for accomplishing objectives
 Evaluate qualifications and training needs of staff
 Identify evidence of critical shortages, such as chronic backlogs
 Identify evidence of clearly defined legal authority for accomplishing goals
 Assess process improvements which may be needed
 Assess whether direct cause and effect relationship exists between
program and desired outcomes
 Determine if program goals are unrealistic or unattainable
 Identify biases in the audit process (e.g., lack of random selection,
unreliability of data)
--Adapted from Performance Auditing: A Measurement Approach, Raaum and Morgan, 2001
The Institute of Internal Auditors, Altamonte Springs, FL
11
Appendix D
Finding
Descriptive
Condition
Example: The annual cost to incarcerate a prisoner was $97,800 in
2009.
Normative
Criteria and Condition
Examples:
The annual cost to incarcerate a prisoner is $97,800, compared to a
total $82,000 per year paid by comparable prisons.
The annual cost to incarcerate a prisoner was $97,800 in 2009,
although the budget appropriation authorized a net $75,000 per
prisoner.
12
Causal
Criteria,
Condition,
Cause,
Effect
Example:
The annual cost to incarcerate a prisoner was $97,800 in 2009,
although the budget appropriation authorized a net $75,000 per
prisoner, resulting in a total deficit for the period of $34.8 million. The
additional costs were caused primarily by a significant increase in labor
and benefit costs following execution of the September 2009 union
contract.
13
Appendix E
Identify findings elements and conclusion in the
following two case studies.
Case Study One: Prison Inmate Costs
Objective: to determine if the cost per year to keep a
person in the state’s prisons is satisfactory (as low as possible),
consistent with good prison care and security. If not,
determine the reasons that costs are high.
Management of the state prison system had never
conducted a study of its prison system operations and cost
structure and did not know that its costs were high.
The state’s per person prison costs are $3,000 to $3,800
higher than prison costs in two comparable states. With the
state’s current prison population, savings from this level cost
reduction would range from $90 million to $114 million a year.
The state’s costs per year to keep a person in prison are from
$3,000 to $3,800 higher than that of two states providing an equal or
slightly higher level of care and equivalent security. This is due to several
factors that could be mitigated.
Prison cost data we obtained from five States with similar
climate and inmate characteristics, showed:
—
Two with average costs of $32,000 and $31,200 per
prisoner per year and equal or slightly higher prison care
ratings.
14
—
One with an average cost of $36,000 per prisoner per year
and a higher prison care rating.
—
Two with an average cost of $34,000 per prisoner per year
and a lower prison care rating for health care and
rehabilitation.
—
All the States provided equivalent security.
The cost to keep a person in the state's prisons during 200X
averaged $35,000 a year and ranged from $27,000 to
$41,500. The average annual cost for minimum (half-way
facility), medium, and maximum security prisons was $29,400,
$36,200 and $39,100 respectively. The cost at each prison
was as follows [details would be shown]:
Four factors accounted for close to 80 percent of the cost
difference. Compared to the two comparable prison systems,
the state had a higher supervisor to security guard ratio, higher
radio communications costs per guard, a higher security guard
turnover rate (up to 40 percent) that adds training costs, and
higher administrative costs.
15
Case Study Two: Processing Applications for Home Mortgages
Objective:
Is the agency processing loan applications for family home
mortgages timely and efficiently (cost per application)? If
untimely or inefficiently processed, what is the effect on
applicants and the agency? If the effect is significant, what is
the cause and what actions are needed to improve processing
time and/or efficiency?
During the peak workload period, customers did not receive
timely service. Twenty percent of the loans were approved too
late for applicants to buy the house for which the loan was
requested. During the workload valley period, agency costs
were $1.2 million higher than necessary.
Loan application workload is seasonal and fluctuates
significantly. In 200X it varied up to 300% between the peak
June-September period and the December-February valley
period. During the peak period, employees processed about 14
loan applications a day but large backlogs developed and the
approval rate lengthened to about 30 days. During the valley
period, loans were normally approved within 3 days but
employees processed only 6 to 7 loans a day because of the
small workload.
Customers do not receive timely service in workload peaks and
the agency is inefficient in processing applications in workload
valleys which increases its operations costs. Both problems
result from management’s inattention to matching staff to
workload.
Management was not adjusting staff levels to match its
workload. Despite knowing that its workload fluctuated widely,
management employed mostly full-time personnel at a level to
16
meet its average workload with little overtime. Agency policy
provided for use of part-time staff, but field managers said that
they unsuccessfully tried to recruit competent part-time staff,
overtime dollars were limited, and laws restricted use of
compensatory time off for extra hours worked. However, we
found that the two private mortgage insurance firms employed
qualified part-time people and that their employees work 55 to
60 hours a week during peak workload periods.
Management's standards provide that: (1) loan applications be
approved or rejected within 10 working days and (2) each
employee process 12 loan applications a workday (8 hours).
These standards appear reasonable. In April and May,
employees met both standards. Also, two private insurance
firms processed family home mortgage loan applications within
10 days, including months of peak demand, and their staff
processed up to 15 applications a day.
17
ANSWERS FOR CASE STUDY ONE: PRISON INMATE COSTS
Objective: To determine if the cost per year to keep a person in the state’s
prisons is satisfactory (as low as possible), consistent with good prison care
and security. If not, determine the reasons that costs are high.
Conclusion: The state’s costs per year to keep a person in prison are from
$3,000 to $3,800 higher than that of two states providing an equal or
slightly higher level of care and equivalent security. This is due to several
factors that could be mitigated.
Criteria: Prison cost data we obtained from five states with similar climate
and inmate characteristics showed:




Two with average costs of $32,000 and $31,200 per
prisoner per year and equal or slightly higher prison care
ratings.
One with an average cost of $35,400 per prisoner per year
and a higher prison care rating.
Two with an average cost below $34,000 per prisoner per
year and a lower prison care rating fl for health care and
rehabilitation.
All the states provided equivalent security.
Condition: The cost to keep a person in the state’s prisons for the past two
years averaged $35,000 a year and ranged from $27,000 to $41,500. The
average annual cost for minimum (halfway facility), medium, and maximum
security prisons was $29,400, $36,200, and $39,100 respectively. The cost
at each prison was as follows:
[details would be shown]
18
Effect: The state’s per person prison costs are $3,000 to $3,800 higher
than prison costs in two comparable states. With the state’s current prison
population, savings from this level cost reduction would range from $90
million to $114 million a year.
Cause: Four factors accounted for close to 80 percent of the cost
difference. Compared to the two comparable prison systems, the state had a
higher supervisor to security guard ratio, higher radio communications costs
per guard, a higher security guard turnover rate (up to 40 percent) that
adds training costs, and higher administrative costs.
ANSWERS FOR CASE STUDY TWO: PROCESSING APPLICATIONS FOR
HOME MORTGAGES
Objective: Is the agency processing loan applications for
family home mortgages timely and efficiently (cost per
application)? If untimely or inefficiently processed, what is the
effect on applicants and the agency? If the effect is significant,
what is the cause and what actions are needed to improve
processing time and/or efficiency?
Conclusion: Customers do not receive timely service in
workload peaks and the agency is inefficient in processing
applications in workload valleys which increases its operations
costs. Both problems result from management’s inattention to
matching staff to workload.
Criteria: Management's standards provide that: (1) loan
applications be approved or rejected within 10 working days
and (2) each employee process 12 loan applications a workday
(8 hours). These standards appear reasonable. In April and
19
May, employees met both standards. Also, two private
insurance firms processed family home mortgage loan
applications within 10 days, including months of peak demand,
and their staff processed up to 15 applications a day.
Condition: Loan application workload is seasonal and
fluctuates significantly. In 200X it varied up to 300% between
the peak June-September period and the December-February
valley period. During the peak period, employees processed
about 14 loan applications a day but large backlogs developed
and the approval rate lengthened to about 30 days. During the
valley period, loans were normally approved within 3 days but
employees processed only 6 to 7 loans a day because of the
small workload.
Effect: During the peak workload period, customers did not
receive timely service. Twenty percent of the loans were
approved too late for applicants to buy the house for which the
loan was requested. During the workload valley period, agency
costs were $1.2 million higher than necessary.
Cause: Management was not adjusting staff levels to match
its workload. Despite knowing that its workload fluctuated
widely, management employed mostly full-time personnel at a
level to meet its average workload with little overtime. Agency
policy provided for use of part-time staff, but field managers
said that they unsuccessfully tried to recruit competent parttime staff, overtime dollars were limited, and laws restricted
use of compensatory time off for extra hours worked.
However, we found that the two private mortgage insurance
firms employed qualified part-time people and that their
employees work 55 to 60 hours a week during peak workload
periods.
20
Appendix F
Formulating Audit Objectives
The audit objective determines the subject matter and content of the audit
report.
Ask the right question
The questions or audit objectives are the most fundamental aspect of the
audit program. Posing a question incorrectly can lead an audit in the wrong
direction. How a problem is stated has implications for the kinds of data to
be collected, the sources of data, the analyses that will be necessary in
trying to answer the question, the conclusions that will be drawn, and
ultimately what the audit report will contain. How an issue is defined
influences directly how and which variables or dimensions are to be selected
and examined.
The importance of careful crafting of audit
objectives
Objectives provide direction. They:





Determine elements of finding to be developed.
Limit collection of unneeded information.
Control scope, methodology, timing, and nature of the audit work.
Increase probability that questions will be answered.
Establish the framework for the audit report.
21
Appendix F (continued)
Principles (Characteristics or Quality Factors) for
Formulating Audit Objectives
 Phrase as precisely worded questions or as “To determine whether…”
 Eliminate ambiguous, abstract, or unfocused terms.
Example
 Clearly identify the audited entity or
audit subject.
Performance Aspects Relevant to
Performance Audit Objectives
Outcome
 Include an identifiable, auditable
performance aspect. Clearly indicate
the type of performance to be audited.
Separate the objectives if more than
one element of performance is to be
reviewed.
 Identify the specific elements of
finding needed to meet the objective.







Purpose achievement
Proportion of population served
Cost/benefit
Cost recovery or profit
Readiness
Population served (coverage)
Customer satisfaction
Output
 Frame objectives that consider a
realistic scope and methodology.





 Report must answer the audit
objectives.


Quantity of units produced
Product consistency
Product accuracy, reliability
Cost to user or customer
Quality (accuracy, reliability, safety,
cleanliness, courtesy, convenience,
comfort, understandability)
Product or service quantity
Timeliness
Input



Price of resources, operating costs
Quantities of resources used
Total cash expenditures
Process


22
Productivity of resources used
Operating ratios (utilization rates,
direct to indirect costs, costs per
output)
--Adapted from Performance Auditing: A
Measurement Approach, Raaum and Morgan, 2001,
Also Consider:

Cite the criteria to be used or developed.

Cite whether conclusions are expected to be drawn.

Audit objectives may be conditional—that is, they may establish
additional objectives to be pursued, depending on the conclusions from
initial objectives.
EXERCISE: Compare the following objectives to the principles for
formulating audit objectives. What improvements are possible?
(1) Our objective was to determine the number of children without health
insurance who would have been eligible to receive SCHIP benefits, and the
amount that their noncustodial parents could potentially contribute toward
SCHIP premiums if their children had been enrolled.
(2) The objectives of our audit were to determine whether:

institutions maximize the number of inmates that complete programs
designed to prepare them for reentry into society, including occupational,
educational, psychological, and other programs; and

Eligible inmates are provided the opportunity to transition through a
Community Correction Center in preparation for reentry into society.
(3) The objectives of our audit of the National Offender DNA Sample
Program were: to assess the overall impact of the Program on the national
offender backlog; and to evaluate the adequacy of the State’s administration
of the Program and monitoring of grantee activities.
23
Appendix G
EXERCISE: Distinguish the types of audit objective by using
performance aspects contained in the Performance Expectations
Model and based on GAO’s Standards.
Audit Objective Type
OBJECTIVE
Input
Process
Economy
Efficiency
Output/
Outcome
Other
Effectiveness
One--Are Bureau of Justice
chemical dependency programs
positively associated with reduced
recidivism?
Two--Determine the causes of
reduced vehicle maintenance
employee productivity from FY 05
to FY 09.
Three--What is the rate of ontime completion of required
annual training at each field
office?
Four--Are employees satisfied
with the timeliness of grievance
resolutions?
24
Five--Are procedures for smalldollar (Under $5,000) purchases
of communications equipment
adequate to obtain the best price?
Six--What is OSHA’s total cost to
hire and train new inspectors in FY
2010 and can it be reduced?
Seven--Determine if Department
of Public Safety district offices’
average cost per driver’s license
test compares favorably with
similar states.
Eight--Are OPM’s training and
wellness programs achieving
desired changes in staff
performance and behavior?
Nine--Does actual fire
department response time meet
its FY 2009 budgeted objective?
Ten--To what extent is the
department complying with
federal safety regulations?
25
Appendix H
Examples of criteria to Prioritize Audit Objectives
Sensitivity Examples: (1) Embarrassment or (2) Extensive Media
Coverage
Significance Examples: (1) Total investment or (2) Potential loss of
human life or serious injury
Susceptibility Examples: (1) Lots of cash or (2) lack of cause/effect
relationships between IPPO (experimental government program or private
sector activity)
26
Appendix I
Development of Objectives and Sub-Objectives
DARE Example
Step 1 Report Users:
Policy Makers – City Council and AISD Board
Department Heads – HHSD and Police
Community, Citizens, Parents, Media
Step 2 Audit Issues/Preliminary Audit Objectives:
Audit Issue: DARE may not be accomplishing its mission of preventing
children from using drugs.
Rationale: Thousands of parents may be disappointed in long term results;
children need more comprehensive program that incorporates after school
intervention, and public safety resources need to be re-allocated to effective
programs or other unmet needs.
Step 3: Confirm the Audit Subject
AISD, APD, and DARE America
27
Step 4 Objective(s) Including Performance Aspects:
--Is the DARE program achieving desired outcomes promulgated by
APD/ASID and DARE America?
--Are citizens satisfied with the DARE program?
Step 5 Prioritize Audit Objectives:
Accomplishment (outcomes) is the primary objective: Is the DARE program
achieving desired outcomes promulgated by APD/ASID and DARE America?
Citizen satisfaction is important but tends to be short term and not based
on actual program results.
Step 6 Specify Which Findings Elements Are Needed:
Criteria, Condition, Effect, Cause
Step 7 Sub-Objectives:
Criteria:
What is the purpose (intent) of the DARE program?
Condition:
Did school-aged children who participated in DARE
avoid substance abuse?
What is the difference in substance abuse among
school-aged children who participated in DARE
compared to those children who did not participate?
28
Effect:
What was the impact of DARE on community health
compared to resources expended (cost/benefit)?
Cause:
Why is the DARE program not achieving desired
outcomes?
29
Appendix J
Development of Objectives and
Sub-Objectives Form
Child Care Licensing Program (CCLP)
Step 1 Report Users:
Step 2 Audit Issues/Preliminary Audit Objectives:
Audit Issue (condition/criteria):
Rationale for Selection:
Step 3: Confirm the Audit Subject:
Step 4 Objective(s) Including Performance Aspects:
30
Step 5 Prioritize Audit Objectives
Step 6 Specify Which Findings Elements Are Needed:
Step 7 Sub-Objectives:
Criteria:
Condition:
Effect:
Cause:
31
Appendix K
Case Study: Child Care Licensing Program (CCLP)
RCCLs—Residential Child Care Facilities Regulators
CPAs—Child Placement Agencies
BACKGROUND
The State’s Child Care Licensing Program (CCLP) works to safeguard the
basic health, safety, and well being of children by developing and enforcing
minimum standards for facilities that care for or place children.
It is the human services’ department’s program for licensing group daycare
homes, daycare centers, registered family homes, child placement agencies,
and publicly and privately owned residential child-care facilities. The
program is also charged with investigating complaints and serious incidents
involving daycare and residential-care facilities and, if necessary, taking
corrective action.
The number of Child Placement Agencies (CPAs) regulated by the CCLP has
tripled in the past 10 years, from 67 in 1999 primarily placing children for
adoption to 194 agencies in 2009 primarily providing foster care.
With the emphasis on placing children in smaller, more homelike settings
and the practice that the money follows the child, there has been a
substantive increase in the number of children requiring specialized care and
services within the foster care system.
In FY 04 the number of placements by CCLP and CPAs was 7522 and 1818
respectively. By FY 09 the placements by CCLP had dropped to 5331 and
CPAs placed 2584.
RCCLs (Residential Child Care Facilities Regulators), CCLP employees, are
responsible for regulating and monitoring CPAs (child care placement
32
agencies.) Child Care Placement Agencies in turn place children in foster
homes and ensure that the foster homes are meeting standards.
MOVING TO ACTIVE ENFORCEMENT
There may not be sufficient management controls in place to support
the recent philosophy change from “partner” of the child care
facilities to “protector of children”. This shift has focused agency
management on systems for regulating, monitoring, assuring
quality, managing information, and maintaining policies and
procedures.
REGULATING CHILD PLACING AGENCIES
RCCL management reported that lack of follow through to ensure foster
homes make improvements based on inspection findings has fostered
inconsistency and a lack of regulatory controls. Furthermore, the CCLP does
not have procedures for making sure that its standards are enforced in all
the regions. The Child Placing Agencies are responsible for reporting on
whether foster homes are making improvements based on inspections and
for assessing overall compliance with standards.
CPAs are responsible for ensuring that agency foster parents meet and
maintain minimum standards, but licensing inspections and investigations
conducted within the past two years indicate that agencies are failing to
carry out this responsibility. Media stories have begun surfacing on the
issues facing RCCL and the State’s ability to manage the foster care system.
RCCL staff has reported instances in which:



CPAs allow foster homes that do not meet standards to continue to
operate;
Foster parents who have been unable to meet standards under the
supervision of a CPA, will switch to another CPA with no questions
asked;
Foster children are not receiving recommended services and
treatment and some appear to be overmedicated; and
33

CPA staff corrects specific instances of non-compliance, but does
not address the underlying weaknesses that are causing the noncompliance.
STRENGHENING THE CPA’s FOSTER HOME MONITORING SYSTEM
Child placing agencies (CPAs) have not been monitoring the foster homes
they certify in a timely and efficient fashion, to ensure compliance with
standards. CCLP management admits this is a weak enforcement area and
described disagreements at several levels that have prevented the agency
from taking action until recently.
To address this lack of coverage and enforcement, CCLP management finally
obtained funding to supplement CPA monitoring with contract inspectors
hired from private firms. An inspection process was implemented two years
ago and the number of inspections completed has steadily increased.
However, during this two year period cost per inspection has also continued
to climb. Based on the most recent analysis, the cost per inspection
increased from an average of $300.00 per inspection in the first quarter of
the first year to $500.00 per inspection in the most recent quarter. To
reduce unit costs by increasing inspections, inspectors now have the
opportunity to earn annual performance bonuses based on reaching or
exceeding goals for number of homes quarterly reported as “completing
inspection.”
Despite additional investigative resources, special RCCL staff visits to
selected foster homes conducted within the past two years indicate that the
inspections may not be fully effective in making sure CPAs are performing
their duty to ensure that agency foster parents meet and maintain
compliance with minimum licensing standards. Given this situation,
concerns are not only being raised about the cost of inspections but the
quality of the inspection work being performed by the contractors. RCCL
staff members have once again attributed the problems to lack of sufficient
34
resources while contractors have recommended that IT solutions be
implemented to streamline processes and cut “bureaucratic red tape.”
FIXING THE QUALITY ASSURANCE PROCESS
The Child Care Licensing staff inputs information about their performance
(using 10 performance measures including license issuance, monitoring
visits, and investigations) into an automated system that supports the
measures. The measures are reviewed by the regional directors. However,
there is no process to ensure the information that supports the measures is
accurate.
The regulation directors are responsible for making sure that regions comply
with the quality assurance program. They have the autonomy to develop
their own quality assurance program.
The quality assurance process measures timeframes, not quality processes,
and is not standardized.
In the absence of an adequate, accurate software application to support the
program, a standardized, consistent quality assurance process with a welldefined feedback loop, becomes more critical.
FIXING THE INFORMATION SYSTEM
The centralized automated system which supports child placing, facility
licensing, and other services is antiquated. A CCLP project identified a
limitation of the current software application for the licensing program that
impedes the timely flow of sufficient program information to management
for decision making.
Program management is moving aggressively to address this problem and
has developed a comprehensive plan that has been approved by the
35
information resources department. Furthermore, this project is being
monitored by an interagency information resources over-sight committee.
FIXING THE PROCEDURES HANDBOOK
The new procedures handbook was issued months ago.
Some licensing changes from the last legislative session became effective
several months ago. However, guidelines for implementing these changes
have not been fully documented or communicated to staff.
There are 12 sets of licensing standards for various types of child care
facilities. This presents a problem for policy analysts (to continually update
the various sets as required by law), and is an enforcement and training
issue (too many sets of regulations to master).
A CCLP task force in December 2008 cited the numerous sets as a barrier to
enforcement. GAO auditors identified this problem, also, but delayed work
due to conflicting congressional priorities. The strong child care lobby has
been fighting about revised day care minimum standards. Another CCLP
task force has recently been appointed to develop recommendations to
reengineer the standards.
36
Appendix L
Development of Objectives and
Sub-Objectives Form
State Public Housing Program (SPHP)
Step 1 Report Users:
Step 2 Audit Issues/Preliminary Audit Objectives:
Audit Issue (condition/criteria):
Rationale for selection:
Step 3: Confirm the Audit Subject:
Step 4 Objective Including Performance Aspects:
37
Step 5 Prioritize Audit Objectives
Step 6 Specify Which Findings Elements Are Needed:
Step 7 Sub-Objectives:
Criteria:
Condition:
Effect:
Cause:
38
Appendix M
Case Study: State Public Housing Program (SPHP)
DHCD—Dept. of Housing and Community Development
LHAs—Local Public Housing Authorities
Chapter 23B of the State General Laws established the Department of
Housing and Community Development (DHCD). DHCD provides state and
federal funds and technical assistance to communities to help plan new
developments, encourage economic development, revitalize older areas,
improve local government management, build and manage public housing,
stimulate affordable housing through the private sector, and respond to the
needs of low-income people. Together these activities are known as the
State Public Housing Program (SPHP) with its primary focus on building and
managing public housing. For the last fiscal year, DHCD administered
approximately $688 million, of which $437 million represented federal funds.
According to DHCD, there were, as of last year, 247 local public housing
authorities (LHAs), including four regional housing authorities, that manage
49,968 units of low and moderate-income housing across the State. In
addition, 80 of these authorities operate an additional 33,507 units of
federally assisted housing units, for a total of 83,475 units under
management. The total development/replacement cost of the State housing
units under management is estimated at just over $10 billion.
The inventory of state subsidized dwelling units was as follows:
Program
Number of Units
Chapter 667 (Elderly/Disabled)
32,310
Chapter 200 (Family)
12,629
Chapter 705 (Family)
3,122
Chapters 689 & 167 (Special Needs)
1,907
Total
49,968
39
The State’s 247 LHAs, including the four regional housing authorities are
statutorily mandated, authorized, and, with the statutorily mandated
assistance from the State through DHCD, “entrusted and required to
maintain and operate units of safe, decent, and sanitary housing for the
State’s qualifying low-income individuals, families, the elderly, homeless,
disabled, and veterans who have served the State and the country.”
Funding
LHAs have three major sources of revenue: rental income, State bonds, and
State appropriations (subsidies) received from the legislature via DHDC.
Rental Income – Since the early 1970s, public housing rents have been fixed
as a percentage of tenant household income (currently a maximum of 32%)
to ensure that tenants can afford the rent. Rental income is not sufficient to
fund operating expenses.
State Bonds – Funds to make significant modernization and capital repairs as
roof replacements, heating system replacements, bathroom and kitchen
modernizations, and elevator replacements have traditionally come from
State Bond Funds. The LHAs have received fewer bond funds in recent
years.
State Subsidies – The LHAs have relied on DHCD to request from the
Legislature adequate operating subsidies to provide funding for the
difference when rents do not offset operating expenses. LHAs provide
DHCD with budgets detailing their funding needs.
In the past four fiscal years, the DHCD has in its budget guidelines required
that LHAs submit budgets with zero increases. In doing so, DHCD is not
recognizing that the actual operating costs of LHAs are increasing. As a
result of insufficient subsidies, LHAs have had to delay maintenance, reduce
staff, reduce salaries, and withstand other costs they cannot control, such as
expenses related to weather conditions, utilities, water, sewer, and trash.
DHCD expenditures for operating subsidies totaled $36.5 million and $34.8
40
million in the last two fiscal years. This is less than the $38 million
expended for operating subsidies 20 years ago when there were fewer
housing units. During the interim years, DHCD subsidy payments to LHAs
have been irregular, inadequate, and untimely.
The LHAs at the close of the last fiscal year were owed over $7.75 million
from DHCD in overdue subsidies dating back to fiscal year 2002. This
problem is exacerbated by the build-up of overdue unpaid subsidies due in
the current year. Some LHAs have been paying prior-year obligations with
future year appropriations, which is not only legally questionable but also an
unsound and unwise fiscal and economic policy.
A recent report by the Legislative Subcommittee on Public Housing stated,
“Putting off, ignoring, and not recognizing the true needs and costs to
operate, maintain, repair, and renovate the State’s housing stock will not
only increase the future costs because of inflation, but also because the
continued and extended deterioration of housing units, which will necessitate
more extensive repairs, is a disservice to the dignity of the tenants living in
public housing, and violates the mandate to provide safe, decent, and
sanitary housing.”
A recent State University “Study of Appropriate Costs for State-Funded
Public Housing” used a scientific approach to quantify the appropriate
operating cost coefficients for the several state public housing programs.
This study determined that shortfalls in state subsidies at LHAs throughout
the State have been shortchanged and under subsidized for the past four
fiscal years by $315.6 million or $78.9 million annually.
A recent media report stated that “The LHAs cannot remain solvent and
avoid bankruptcy when DHCD does not budget sufficiently to provide
adequate contributions to meet annual operating and maintenance costs and
causes delays in providing timely periodic subsidies for LHAs to pay their
bills.”
Inspection of Housing Units
DHCD’s Property Maintenance Guide, Chapter 3(F), requires that inspections
of dwelling units be conducted annually and upon each vacancy to ensure
41
that every dwelling unit conforms to minimum standards for safe, decent,
and sanitary housing as set forth in Chapter II of the State Sanitary Code.
The Legislative Subcommittee on Public Housing noted the following in its
report:
•
LHAs’ inspections noted thousands of instances of noncompliance with
Chapter II of the State Sanitary Code, including windows not working
properly, lifting and broken asbestos floor tiles, inadequate electrical
systems, cracked and broken pavement, extensive siding and roof
problems, peeling and flaking lead paint, a serious mold problem, and
other health and safety hazards.
•
A few LHAs did not conduct annual inspections of their housing units.
Condition of State Low-Income Housing
DHCD Property Maintenance Guide, Chapter 3(F), requires that LHAs report
the condition of their housing units as disclosed by inspection. The Guide
also requires that LHAs have an up-to-date written property plan. The Guide
states, in part:
The goal of good property maintenance at a public housing authority is
to serve the residents by assuring that the homes in which they live are
decent, safe and sanitary . . . every housing authority must have a
preventive plan which deals with all the elements of its physical property
and is strictly followed . . . The basic goals of an inspection program are
to improve the effectiveness and efficiency of your maintenance effort.
This will be achieved when you (LHAs) have a thorough program of
inspections, when you observe all parts of the (LHAs) physical property,
document the results of the inspections thoroughly, and convert the
findings into work orders so that the work effort can be scheduled and
organized. Inspections are the systematic observation of conditions and
42
provide the foundation for capital improvements and long range
planning, as well as a record of present maintenance needs.
The Public Housing Subcommittee report found that not all LHAs had
incorporated DHCD’s Property Maintenance Guide into their own policies and
procedures. Specifically, the report noted that a few of the LHAs it reviewed
did not have an official preventive maintenance plan to inspect, maintain,
repair, and upgrade its existing housing units.
Many of the state’s housing units are over 50 years old and, accordingly,
require more extensive renovations and repairs in order to preserve this
valuable commodity and keep the units in service to meet the growing
demand.
In its report, the Legislative Subcommittee on Public Housing concluded that
“Continued failure to provide reasonable and adequate funding at the same
level as the federal programs will only cause further deterioration and more
extensive costs to repair the state facilities in the future.”
Uninhabitable Units
DHCD requires that LHAs report all units that are placed off-line and
unavailable for occupancy. At the end of last fiscal year, DHCD reported that
there were 407 off-line units. This number is suspected as being
understated. For example, 52 LHAs (of 247) did not provide information on
their off-line units. The commonly cited reason for units being off-line was
that they were unfit for human habitation due to serious structural
deterioration and mold and would need extensive renovation in order to be
safe for occupancy.
Timeliness in Filling Vacancies
The DHCD Property Maintenance Guide requires that housing authorities
reoccupy vacant units within 21 working days. The Guide also requires that
LHAs submit quarterly vacancy reports that cite the length of time units are
43
vacant.
At the end of last fiscal year, DHCD reported that there were over 800 units
vacant for over 61 days, with most vacant for over 180 days (9 months).
Every LHA reported having a least one unit vacant for over 61 days.
The primary reason cited by LHAs for not reoccupying the units within 21
days was that the units were left in deplorable condition that required a
great deal of work and renovation. However, media in various parts of the
State have been critical of LHA procedures for making repairs, noting that
they are slow in starting and completing maintenance work.
In its report, the Subcommittee on Public Housing noted that when LHAs
take an excessive amount of time to fill vacant units with new tenants, they
lose the opportunity to earn thousands of dollars in potential rental income.
It cited as one example an LHA where five out of 13 units in one complex
were vacant from 60 days to 273 days, and eight out of 13 units in another
complex were vacant from 60 days to 395 days.
Waiting Lists
DHCD requires that LHAs report the number of applicants on the waiting list
for State low-income housing. At the end of last fiscal year, DHCD reported
that there were 81,150 applicants for State low-income housing.
In its review of selected LHAs, the Subcommittee on Public Housing fond
instances where applications for tenancy and related records (i.e., the
Waiting List Ledger, Vacancy Ledger and Master Ledger) were incomplete,
inaccurate, falsified, tampered with, improperly maintained, or recorded in
pencil.
Operating Efficiency
DHCD has not established any metrics for LHAs to use in measuring their
44
efficiency in managing and operating their housing units. An unofficial
comparison by legislative staff of administrative and maintenance cost for 50
randomly selected LHAs showed wide differences.
Customer Satisfaction
DHCD has not established a requirement for LHAs to conduct a customer
satisfaction survey. There is no record that any LHAs have conducted such a
survey.
Mission Results
DHCD has not established any metrics for assessing how well it is doing in
achieving its mission. Critics of the subsidy program claim that DHCD might
provide more low income families with housing by providing subsides for
families to obtain housing in the private sector market.
45
Appendix N: Developing Audit
Recommendations
(1) Write one audit recommendation for
Appendix E Prison Inmate Costs (page 19)
and one for Processing Applications for Home
Mortgages (page 20).
(2) Write one audit recommendation for the
Child Care Licensing Program (page 32) and
one for the State Public Housing Program
(page 39).
46