Information Technology Audit Overview
The overview includes issues/findings from several audits performed by the Office of the State
Auditor. Those audits are:
• Department of Health and Human Services/NCTracks (MMIS Replacement) Implementation - Released 5/22/2013
« Office of Information Technology Services-IT Project Budget and Schedule Variances —
Released 4/22/2013 • ' " ' • '
• UNO-General Administration Banner Hosting Services - Released 12/19/2013
• Office of the Information Technology Services - Information Technology General Controls Released 4/17/2012
« Department of Health and Human Services/Replacement MMIS Implementation - Released
1/10/2012
• E-Commerce Project Office and Statewide Portals Contract - Released 12/21/2009
Lack of Oversight:
State Chief Information Officer (SCJOJ/Enterprise Project Management Office (EMPO) - The
Enterprise Portfolio Project Management (PPM) Tool is an onliae system that can be used to monitor
IT projects for the State and is under the purview of the SCIO. The tool is used by each state agency
for recording information for the each IT project and recording data so that the progress of IT projects
can be monitored.
IT Projects with cost estimates $500,000 or more must be approved by the SCIO.
The SCIO has the authority to stop a project and the State's Budget Director has the authority to stop
funding of an IT project.
•
*
The PPM tool does not automatically inform the -SCIO of projects that have to be approved by
the SCIO before project is initiated.
o When an agency enters information in the PPM tool, the agency must click on a button
that shows aproject exceeds $500,000 and must be approved by .SCIO as opposed to
the tool recognizing this automatically based on $ entered.
••
Currently ITS allows an agency to contract with a vendor and obligate state funds after
approving initial IT cost estimate for a project,'
o ITS has not created a current standard practice for agencies'to follow for the
establishment of IT project estimates,
o No policy requires an independent entity to verify an IT project estimate is reasonably
accurate.
o
State agency managers are not required to manage IT projects so that projects meet
initial costs or schedule estimates,
o Audit revealed, after reviewing 84 projects, the final costs for'the projects were 2
times ($326 M) more than the estimated costs and the projects took 65% (389 days)
longer to complete than initial estimates.
o At the time of the audit referenced in preceding bullet, ITS personnel stated that an . . .
agency could not determine how much an IT project would cost or how long it would
take until a contract was signed with a vendor and work began.. ..culture/practice of
this sort allows for the scope, cost and deadline of IT projects to get way out of hand.
The PPM Tool relies on self-reporting by each.agency with an IT project,
o Agencies enter all information regarding their IT project into the PPM tracking tool,
but there is no independent verification by ITS,
o DHHS NCTracks system showed "green" up until the system went live despite 2
. audits that showed problems with costs over budget and the project behind schedule:
o TIMS IT project showed "green" up until the project was scraped in 2014.
o EPMO does not always have the ability to review and ensure that the information
within the PPM tool is accurate.
Data within the PPM Tool is not reliable
o DHHS NCTracks system showed "green" up until the system went live despite 2
audits that showed problems with costs over budget and the project behind schedule,
o Data for the TIMS project at the Department of Revenue showed that the initial project
estimate was $500,000 when the project was approved when, in reality, the project
estimates were $98M,
o The TIMS project showed "green" in the EPMO tool up until the project was scraped .
in 2014.
No Central Repository for Lessons Learned and no oversight to make sure the issues in
previous IT projects that made them more costly and implemented later than expected are not
repeated.
'Per DHHS, the reasons for the first vendor (ACS) being unsuccessful were:
o
«
•
Gould not use as much prograrrnning from prior projects as initially thought,
Vendor experienced high staff turnover,
• PHHS did not timely review information submitted to them in order for vendor to proceed.
Reasons 2n vendor (CSC) gave for not meeting deadlines and budget:
•
•
•
Could not use as much programming from New York state's project as first thought (expected
to use 72% but could use only 36%)
Vendor experienced high staff turnover
DHHS did not review and return to vendor, timely, information given to them by vendor
before vendor could proceed.
Because there is no central repository for problems with prior IT projects and no oversight of IT
projects (projects are "self-policed"), mistakes made in prior contracts/projects are allowed to be
repeated.
Contracting Practices - once a project is approved it is important that all contract terms are
included, the contract is written in the best interest of the state, and details of damages for
nonperformance should be clear.
•
Lack of details contained in contract
o 2nd DHHS contract with vendor (CSC) to build NC system to process Medicaid claims
- Vendor stated in presentation to DHHS that they could use 72% of programming
from the same project done for the state of New York but the 72% term never made it
to the contract. The vendor was only able to use 36% of the programming from New
York's IT project.
o Initial Contract for the building of Medicaid replacement'system - Contract terms did
not allow the state of NC to terminate the original vendor (ACS) when they failed to
meet the deadline for the design, development and installation of the system. NC
terminated the contract after having paid the vendor $5.6M. The vendor sued, won
and NC had to pay the vendor another $10M.
o Contract for Independent Validation and Verification - Federal Govt. required NC to
hire independent 3r party to independently validate and verify system capabilities and
assess risks. NC contracted with Maximus to do this but failed to require the vendor
to actually independently perform assessments regarding execution of user and
* production test cases prior to go-live. Contract stated they "may" but they did not
perform any independent testing, Maximus simply relied on data from CSC.
o Contract with CSC — NC failed to include details for dispute resolution when the
vendor failed to perform. When NC determined that CSC had let schedule slip no
details were in contract as to how damages would be determined. NC had to negotiate
the amount of damages.
o Contract with CSC — Vendor made unauthorized changes to the Medicaid
-. Replacement system and charged the NC $30M. NC had to negotiate how much of
the $30M would be paid, even though the changes were unauthorized.
o
Contracts "written to advantage of vendor
o NC does not have expertise for the writing of contracts as complex as IT contracts.
Vendors write the contracts and NC signs.
No oversight/review of the contracting process
Contracts allow vendors to get paid for incomplete projects without penalties - See items
above.
IT Project Staffing:
«
' «
•
Frequent turnover with IT staff,
NC allows for senior managers working on IT project to be working for NC state government
one day and working for the vendor the next,
NC does not have staff who can manage large IT projects, who can determine adequate needs
of a system, or who can monitor complex contracts.
Independent Validation and Verification:
Not always performed - Vendor for the IV&V for NC Tracks did not perform independent testing of
the system.
The IV &V A'endor took reports from other sources and made assessments of the system. Based on
the go-live issues it is questionable that NC and federal government received much benefit for the
dollars paid to the vendor.
Service Level Agreements (SLAs) — an agreement between an agency and a vendor. An agreement
.that establishes overall mirrmium expectations, and defines the roles and responsibilities of both
parties.
SLAs are inaccurate - found an SLA between a university and a 'data housing center and a different
university was named as the agency than the one actually using the data center.
Some SLAs lacked details - without details NC cannot hold vendor accountable,
Some SLAs were outdated.