National Science Foundation
Office of Inspector General Update
1
National Conference on College Cost Accounting
2015 Annual Conference
October 13, 2015
Overview
2
 NSF OIG overview
 Audit update
 DATA Act and IPERA
 Data analytics in the federal government
 Applications in grant oversight
Dr. Brett Baker, AIGA, NSF OIG
Inspectors General
3
 72 Federal Inspectors General (IGs)
 Promote economy and efficiency
 Prevent and detect fraud, waste, and abuse
 Audit and evaluate agency programs and activities
 Investigate allegations of wrongdoing
 Operate a fraud hotline
 Review agency compliance with legislation and regulations
 Advise agency head and Congress
Dr. Brett Baker, AIGA, NSF OIG
Inspectors General Do Not….
4
 Operate programs or decide funding
 Perform management functions
 Make agency employment decisions
 Enforce implementation of OIG recommendations
 Suspend or debar recipients
Dr. Brett Baker, AIGA, NSF OIG
Who We Are
5
Inspector General
Assistant IG for
Audit
Counsel to the IG
Financial Statement
and IT Audit
OIG Management Support
Assistant IG for
Investigations
Administrative
Investigations
Performance Audit
Civil/Criminal Investigations
External Audit
Investigative
Legal / Outreach
Audit Services and Data
Analytics
Expertise in areas of research,
grant, and contract
administration
Investigations Specialists
and Support Staff
Dr. Brett Baker, AIGA, NSF OIG
Two Sides of the House
6
Audit
 External Audit Program



 Civil and Criminal
Audits of awardee financial,
administrative and internal
controls.
Reports findings to NSF for
audit resolution (e.g., return
of funds).
 Internal Audit Program

Investigations


Cases of Fraud, Theft,
Conspiracy, Embezzlement,
False Statements, etc.
Consequences:
Restitution, Fines, Prison,
Compliance Plans.
 Administrative
Audits of NSF programs and
operations to determine
program compliance and
efficiency of operations.
Report findings to Congress, Semiannual
National Science Board, and
Reports
NSF.
to Congress


Cases of Research
Misconduct; Breach of
Confidentiality; Human
Subjects, Animal Welfare.
Consequences:
Debarment, Certifications and
Assurances, Remedial
Training, etc.
Audit Planning
7
 Work Required by Law:





Agency Financial Statement Audit (CFO Act)
Federal Information Security Management Act (FISMA)
Improper Payment Elimination and Recovery Act (IPERA)
Conference spending and government purchase card oversight
DATA Act
 Discretionary Work:





OIG Risk-based Assessments
─ Internal NSF operations
─ External award oversight
Referrals from Investigations
NSF Management Challenges
National Science Board and NSF Suggestions
Congressional Requests
Dr. Brett Baker, AIGA, NSF OIG
Distribution of Audit Work
8
Mandatory
OIG
External
Congressional
Request
OIG
Internal
NSF
Requested
Dr. Brett Baker, AIGA, NSF OIG
Greater Attention to
Analytics in Government
9
 DATA Act
 Promotes data sharing across government agencies
 Treasury data analytics center for OIGs – automated oversight
 Government-wide structured data standards for financial reporting
 USASpending data should be standardized and machine-readable

OIGs will audit data quality
 Improper Payments Elimination and Recovery Act (IPERA)
 Amends the Improper Payments Information Act of 2002
 IPERIA strengthens estimations
 Strengthens detection, prevention, and recovery efforts
 Pre-award and pre-payment checks with Do Not Pay
 Annual risk assessments of covered programs
 Published improper payment estimates with reduction targets
 Goal to reduce improper payments by $50B and recover $2B in 2 yrs
Dr. Brett Baker, AIGA, NSF OIG
Programs with Improper Payment
Estimates Over $1 Billion in FY 2014
10
$50 $45.8
$40
$30
$20
$17.7 $17.5
$10
$12.2
$5.6 $5.1
$3.0 $2.4 $1.9
$1.7
$1.5
$1.0
$0
Dollars (in billions)
Source: GAO
Dr. Brett Baker, AIGA, NSF OIG
Risk Identification for Oversight
 Why look for risk?


11
Helps ensure programs are working and funds are being used properly.
Government responsible for all activity, risk-based approaches help focus
effort on things we need to pay more attention to.
 General risks


What an institution is….
Some contract and grant awards are riskier than others
-- Smaller institutions tend to have weaker internal controls
 Activity-based risks


What an institution does….
Award actions that stand out from normal activity
-- Large drawdown on a single date – end of a fiscal year
-- Spending out remaining award fund just before and after the award
 Challenges


General risks can be more obvious
Activity-based risks are more revealing, but can be harder to see
Dr. Brett Baker, AIGA, NSF OIG
Focus on Risk
Many to the Few
12
600,000
Grant award drawdowns annually
totaling $7 billion
Each assigned a risk score
Spikes, near award expiration charges
40,000
Active awards
Each assigned a risk score
Summarize drawdown risks by award
2,000
Institutions
Each assigned a risk score
Summarize award risk and add CCR, EPLS, FAC risks
20
Audits of higher risk institutions
Each audit tests all expenditures
for all awards
with automated risk indicators
15,000,000 Transaction-level expenditures tested
Each assigned a risk score
Dr. Brett Baker, AIGA, NSF OIG
Forensic Audit Framework
13
 Set Objectives and Define Universe
 Structured brainstorming with subject matter experts
 Map out End-to-End Process

Identify systems, key processes and controls
 Obtain Transaction-Level Data
 Build Targeted Business Rules and Run Against Data

100% review with automated business rules based on risk

Award/institution anomalies and changes in patterns over time
 Examine Anomalies
Dr. Brett Baker, AIGA, NSF OIG
Risk Identification
through Automated Oversight
14
 Improved risk identification
 100% transaction review – limited statistical sampling
 Automated business rules based on risks
 Focus review on higher risks
 Key data analytics software techniques
 Join databases (need linking field)
 Summarize data (many to the few)
 Apply risk indicators using computed fields
 Develop risk profiles by institution, award-type, transaction-type
 Summarize risk into one number
 Agencies and recipients can use similar data analytics techniques
 Monitor grant spending
 Identify anomalies early
Dr. Brett Baker, AIGA, NSF OIG
Comparing Data Files
(Three Bucket Theory)
15
Vendors
Not Paid
Yet
Vendor Table
(Valid vendors)
Vendors
Paid and
In Vendor
Table
Linking field allows
for comparison
Vendors
Paid but not
In Vendor
Table
Disbursing
Transactions
Dr. Brett Baker, AIGA, NSF OIG
Normal Distribution
16
Anomalous
Activity
Normal Activity
Anomalous
Activity
Dr. Brett Baker, AIGA, NSF OIG
Building a Forensic Capability
17
 Develop organizational capability



All staff should have basic skill with data analysis tools
Forensic units perform more sophisticated analyses
Targeted reviews are more efficient in time and cost
 Phased development




Hardware and software
Access to internal and external data
Staff: system savvy, analytical, business process knowledge
Training, then immediate application to work
 Very important component is tone at the top
Dr. Brett Baker, AIGA, NSF OIG
Framework for Data Analytics
Using Government and Publicly Available Data
Payee, Contract No, CLINs, Payment Amount, Date
Data Download
Award
Systems
Payment
Systems
Disbursing
Systems
Federal Reserve
System
Data Download
Grants, Contracts
Data Download
Transaction-level Data
Data Download
Award-level Data
Contract
Invoices
Grant Pmt
Req’s
Data Download
Data Download
Data Download
Data Download
Data Analytics
Risk rank transactions
Identify anomalies for testing
Data Download
Join databases
Apply risk indicators
Commercial
Bank
GuideStar
(non-profits)
SAM
(CCR, EPLS)
Federal Audit
Clearinghouse
Master Death
File (SSA)
CPARS, FPDS
Oversight
Review by
• Auditors
• Investigators
• Agencies
Examples of systems that can be used to help understand payment transactions
Dr. Brett Baker, AIGA,18
NSF OIG
U.S. Financial Assistance Overview
19
 $600 billion in awards
 88,000 awardees and 26 Federal grant making agencies
 Project and research, block, and formula
 Outcomes are designed to promote public good
 Challenges
 Limited visibility of how Federal funds are spent by awardees
 Support for funding requests much less than for contracts
 American Recovery and Reinvestment Act (2009)
 $840 billion of assistance to stimulate the economy
 Greater accountability and transparency over spending than ever
 Opportunities to enhance oversight with less
 Automated oversight
Dr. Brett Baker, AIGA, NSF OIG
Framework for Grant Oversight
20
 Data analytics-driven, risk-based methodology to
improve oversight


Identify institutions that may not use Federal funds properly
Techniques to surface questionable expenditures
 Life cycle approach to oversight



Mapping of end-to-end process to identify controls
100% review of key financial and program information
Focus attention to award and expenditure anomalies
 Complements traditional oversight approaches


Techniques to review process and transactions are similar
Transactions of questionable activities are targeted
 Recipients and Agency Officials can use data analytics

Identify high risk activities through continuous monitoring
Dr. Brett Baker, AIGA, NSF OIG
Grants Differ From Contracts
21
GRANTS
Promote services for the
Public Good
 Merit review (competitive)
 Multiple awardees
 Award budget
 No government ownership
 Grant payments



Summary drawdowns
No invoices for claims
Expenditures not easily visible
 Salary percentages
CONTRACTS
Specified deliverables
(Goods and Services)
 Competitive process
 One awardee
 Contract Price
 Government ownership
 Contract payments



Itemized payment requests
Invoices to support claims
Detailed costs
 Salary hourly rates
Dr. Brett Baker, AIGA, NSF OIG
End to End Process for Grant Oversight
22
PRE-AWARD RISKS
ACTIVE AWARD RISKS
•Funding Over Time
•Conflict of Interest
•False Statements
•False Certifications
•Duplicate Funding
• Inflated Budgets
•Candidate
Suspended/Debarred
•Unallowable, Unallocable, Unreasonable Costs
•Inadequate Documentation
•General Ledger Differs from Draw Amount
•Burn Rate
•No /Late/Inadequate Reports
•Sub-awards, Consultants, Contracts
•Duplicate Payments
•Excess Cash on Hand/Cost transfers
•Unreported Program Income
•
AWARD END
RISKS
•No /Late Final
Reports
•Cost Transfers
•Spend-out
• Financial
Adjustments
• Unmet Cost
Share
Dr. Brett Baker, AIGA, NSF OIG
Look at
Red Flag
Areas
Use Data Analytics to identify anomalies that
are potential fraud indicators, such as:
• breaks in trends, outliers…
2
The more red flags,
the higher the risk.
3
5
Award
Notification
6
HR
System
Recipient
Project
System
1
The less red flags,
the lower the risk.
Reports
External
Grants
Portal
4
Pre-Award
Review
Proposal
System
Internal
Grants
Portal
Awards
System
23
Pay
System
Acctg
System
7
Acctg
System
8
Post Award
Monitoring
9
Award
Close-Out
Dr. Brett Baker, AIGA, NSF OIG
Data Sources
24
 Internal


Proposals: budgets, panel scores
Agency award systems, recipient reporting
 External




Excluded Parties List System (EPLS)
Central Contractor Registration (CCR/SAM)
Public tax filings
Federal Audit Clearinghouse (A-133 audits)
 Recipient financial system records





General ledger and subsidiary ledgers
Effort reporting
Property
Travel and purchase card
Subaward monitoring
Dr. Brett Baker, AIGA, NSF OIG
Identification of Higher Risk Institutions and Transactions
25
Dr. Brett Baker, AIGA, NSF OIG
Anomalous Drawdown Patterns
26
$$
Spending
Remaining
Grant funds
(before expiration)
Start up
costs
Spending
Remaining
Grant funds
(after expiration)
Drawdown
Spike
Normal drawdown pattern
Grant
Award
Grant
Expiration
26
Dr. Brett Baker, AIGA, NSF OIG
Early Drawdown
27
Dr. Brett Baker, AIGA, NSF OIG
Draw Spike
28
Dr. Brett Baker, AIGA, NSF OIG
Spend out Pattern
29
Dr. Brett Baker, AIGA, NSF OIG
Does this drawdown pattern
look okay?
30
Dr. Brett Baker, AIGA, NSF OIG
Common Audit Findings
31
Pre-Data Analytics Audits
(stat sampling projections)




Unsupported costs
Effort reporting
Effort reporting (subaward)
Pre-award charges
Data Analytics Audits
(actual transactions)




Unallowable, unallocable,
unreasonable costs
Excess salary
Indirect Costs
Equipment
Dr. Brett Baker, AIGA, NSF OIG
When should you contact OIG?
32
 Report significant administrative or financial problems
 Report allegations of wrongdoing
 Research misconduct
 Fraud / theft involving NSF funds
 Violation of regulation, directive, or policy
 1-800-428-2189 (anonymous hotline)
 703-292-7100 (business hours), or
 703-328-3932 (non-business hours)
 Online hotline through our website webform. www.nsf.gov
 Email: oig@nsf.gov
Dr. Brett Baker, AIGA, NSF OIG
OIG Outreach
33
 Presentations at conferences and seminars


For students, PIs, and administrators
OIG outreach visit can be requested
 Fact sheets and brochures, briefings, conference
presentations www.nsf.gov/oig/outreach_all.jsp
 OIG Semiannual Report
http://www.nsf.gov/oig/pubs.jsp
 Requests: oig@nsf.gov
Dr. Brett Baker, AIGA, NSF OIG
Questions?
34
Dr. Brett M. Baker
Assistant Inspector General for Audit
National Science Foundation
Office of Inspector General
Phone: 703-292-7100