DRAFT
INTOSAI
Public Debt Committee
Guidance on Using Substantive Tests in
Audits of Public Debt
Final Report
June 2006
INTOSAI’s Public Debt Committee
Page 1 of 16
Guidance on Substantive Tests
DRAFT
Public Debt Committee Chairman
Arturo González de Aragón, P.C.A.
Superior Auditor of Mexico
Members
Argentina –
Canada Gabón Korea Lithuania USA –
Mexico Portugal United Kingdom –
Russian Federation –
Sweden Zambia Yemen -
Collaborators
Chile
Jordan
Egypt
This publication has been prepared for official publication in separate English, Spanish, (________, and
________) versions by the Public Debt Committee of the International Organization of Supreme Audit
Institutions (INTOSAI).
INTOSAI’s Public Debt Committee
Page 2 of 16
Guidance on Substantive Tests
DRAFT
INDEX
INTRODUCTION
PART 1
Purposes and definitions of substantive audit procedures
PART 2
Analytical Procedures
PART 3
Substantive Audit Procedures for Derivatives
PART 4
Substantive Audit Procedures for Unrecorded Contingencies
PART 5
Sampling Procedures
INTOSAI’s Public Debt Committee
Page 3 of 16
Guidance on Substantive Tests
DRAFT
INTRODUCTION
1. Under the terms of reference laid down by the Governing Board of INTOSAI the
Public Debt Committee (PDC) was given the task of publishing guidelines and
other information for use by Supreme Audit Institutions (SAIs) to encourage the
proper reporting and sound management of public debt. In recent years the PDC
has discharged this responsibility by publishing guides on the definition,
disclosure and reporting of public debt, a guide on auditing internal controls, as
well as research studies related to financial commitments, contingencies and fiscal
risks that affect the management and audit of public debt.
2. This guide covers the following issues related to substantive tests in public debt
audits:
(1) Purposes and definitions of substantive audit procedures
(2) Analytical procedures
(3) Substantive audit procedures for derivatives
(4) Substantive audit procedures for unrecorded contingencies
(5) Audit sampling techniques
3. This guide is based on documents presented and discussed at PDC meetings in
past two years. This guide incorporates the feedback received at those meetings,
and is based mainly on two documents, namely, The Audit of Public Debt –
Application of INTOSAI Auditing Standards (1st Draft), prepared by UK National
Audit Office, and Guidance for Using Substantive Tests in Audits of Public Debt,
prepared by US Government Accountability Office.
INTOSAI’s Public Debt Committee
Page 4 of 16
Guidance on Substantive Tests
DRAFT
PART 1 – Purposes and Definitions of Substantive Audit Procedures
1. Substantive audit procedures are performed after auditors have completed the
planning and internal control steps of a public debt audit. Substantive audit
procedures help auditors to meet the INTOSAI field standard requirement:
2. “Competent, relevant and reasonable evidence should be obtained to support the
auditor’s judgment and conclusions regarding the organization, program, activity
or function under audit.
3. The audit findings, conclusions and recommendations must be based on evidence.
4. Auditors should have a sound understanding of techniques and procedures such as
inspection, observation, inquiry and confirmation, to collect audit evidence.
5. In choosing approaches and procedures, consideration should be given to the
quality of evidence.”
6. In evaluating the nature and extent of use of substantive audit procedures, auditors
consider the main objective of their audit, namely, to determine whether public
debt elements are recorded correctly with respect to accounts, amounts and
periods, authorized by law and executive regulations, and disclosed to
policymakers and the public in a timely, consistent and transparent manner.
7. In order to collect sufficient evidence to form an opinion, auditors examine public
debt transactions, such as borrowings, debt servicing, voluntary and forced debt
restructurings. Auditors also examine public debt balances, including the
amounts of main debt categories (domestic and foreign, short- and long-term,
marketable and non-marketable). In addition to examining debt balances and
borrowing transactions, auditors use analytical procedures to compare actual and
expected relations between key variables, such as the relationship between public
debt levels and interest expenditures.
8. In performing tests of debt balances and borrowings, the auditor is concerned with
overstatement or understatement of the line item in the financial statement. The
tests make use of the inherent properties of double-entry accounting systems, that
is, a test of one side of a public debt transaction simultaneously tests the other side
of the transaction (e.g. cash account). Accounting entries that are normally
examined include cash borrowings, debt rollovers, interest payments and
extraordinary debt transactions, such as advanced debt purchases at market prices,
and debt renegotiations with official and private creditors.
INTOSAI’s Public Debt Committee
Page 5 of 16
Guidance on Substantive Tests
DRAFT
9. The levels of public debt and related borrowing transactions may be described in
terms of their characteristics:
Existence – whether a liability exists at a given date. For example, the
debt securities reported in the financial statements of the government exist
at the end of the reporting period.
Rights and obligations – whether the liability pertains to the reporting
entity. For example, a government has rights and obligations associated
with the debt securities, money market instruments or derivative
instruments reporting in the government’s financial statements.
Occurrence – whether a transaction or event took place which pertains to
the borrower during the reporting period. For example, the transaction
that gave rise to a derivative, or to a profit or loss on the disposal of a debt
security, occurred during the reporting period.
Completeness – whether there are no unrecorded debt instruments or
borrowings, or undisclosed items. For example, all of the government’s
debt securities, deposits, money market instruments or derivatives are
reported in the balance sheet and disclosed in footnotes.
Valuation – the amount of assets and liabilities is recorded at an
appropriate carrying amount. For example, the values of debt securities or
derivatives reported in the financial statements were determined in
accordance with relevant legislation, regulations and applicable
accounting standards.
Measurement – a borrowing event is recorded at the proper amount and
expense is allocated to the proper period. For example, interest outlays are
properly accrued or recorded; profits or losses on sales of debt securities
are correctly calculated and attributed to the correct accounting period.
Presentation and disclosure – an item is disclosed, classified and
described in accordance with the applicable reporting framework,
including relevant legislation and applicable accounting standards and
regulations.
In order to determine if any of the above characteristics are materially
misstated, the auditor would design and apply substantive tests, but only
after performing and evaluating the results of the tests of internal controls.
Based on the level of expected overall audit assurance the auditor
determined in the planning phase of the audit, the auditor establishes the
minimum levels of substantive assurance for each level of combined audit
risk. For example, GAO’s audit risk model provides that, based on a
desired overall audit assurance of 95 percent, the following minimum
INTOSAI’s Public Debt Committee
Page 6 of 16
Guidance on Substantive Tests
DRAFT
levels of substantive assurance for each level of combined risk would be
appropriate: (a) if combined risk is low - 63%, (b) if combined risk is
moderate - 86%, and (c) if combined risk is high - 95%. In other words,
the higher the combined risk, the more substantive assurance would be
required.1
Auditors use a combination of the following substantive procedures to
obtain the evidence required to provide them with the appropriate
assurance.
Inspection – examining records, documents and tangible assets. Three
major categories of evidence are:
1) Evidence created and provided to auditors by third parties,
including such items as bank statements for cash or
custodian statements of debt security holdings.
2) Evidence created by third parties and held by the borrower,
including the results of counterparty transactions. For
example, outstanding balances on repurchase agreements,
that is, collateralized borrowings.
3) Evidence created and held by the borrower, including the
debt schedules, records and reconciliations underpinning
the financing statements.
Observation – looking at borrowing transactions and management actions
performed by debt management staff, in particular those that leave no
audit trail. For example, attending a debt auction and observing a debt
strategy committee meeting. This technique helps to ensure that all
procedures are properly adhered to and monitored.
Inquiry – seeking information from knowledgeable persons inside and
outside the debt management office. Inquiries may range from formal
written inquiries to third parties, to informal oral inquiries to persons
inside the debt management office. Responses to inquiries can provide
audito2rs with information not available previously or serve as
corroborative evidence. Key stakeholders that the auditor may consult in a
public debt audit include primary dealers and debt management office
staff in front, middle and back offices.
1
Substantive assurance is the auditor's judgment that all of the substantive tests will detect misstatements
that in total exceed materiality threshold. Substantive assurance is related to the entire audit and is directly
related to level of combined risk; it is not the same as the confidence level of an individual sample.
2
VOUCHING consists in selecting sample items from an account and going backward through the
accounting system to find the source documentation that supports the item selected. TRACING consists in
selecting sample items from basic source documents and proceeds forward through the accounting system
to find the final recording of the transactions (e.g., in the general ledger).
INTOSAI’s Public Debt Committee
Page 7 of 16
Guidance on Substantive Tests
DRAFT
Confirmation – responses to inquiries designed to corroborate
information in accounting records. For example, the auditor may seek
direct confirmation of debt amounts by communicating with creditors.
Recalculation – checking the mathematical accuracy of debt records by
footing or crossfooting or by recomputing amounts and tracing journal
postings, subsidiary ledger balances, and other details to the corresponding
general ledger accounts. For example, the auditor can recalculate
payments in the interest payable list, foot the list, and trace the total to the
general ledge interest payable amount.
10. In general, the reliability of the evidence is influenced by its source – internal or
external, and by its nature – visual, documentary, or oral. Evidence obtained and
verified directly by auditors is more reliable than that obtained by third parties.
For example, visual observation of public debt operations and computations by
auditor are more reliable than observations and computations done by third
parties. Written documents are the second most reliable form of audit evidence.
Original documents are more reliable than photocopies and facsimiles. Oral
interviews are the least reliable audit evidence. Oral interviews with third parties
can be more reliable than interviews with debt management staff, but less reliable
than written documents.
INTOSAI’s Public Debt Committee
Page 8 of 16
Guidance on Substantive Tests
DRAFT
11. The following table provides common examples of substantive procedures in
public debt audits. The procedures are grouped by type of assertion; in practice
the same procedure can be used to test several assertions.
Assertion
Existence
and
Occurrence
Rights and
Obligations
Assertion
Rights
and
Obligations
Audit Procedure
Confirmation with the holder of the debt security or fiscal agent or
custodian of debt records or trustee
Inspection of underlying debt agreements and other supporting documents,
confirmations received by creditor, in paper or electronic form, for
amounts reported
Inspection of supporting documents for subsequent realization or
settlement after the end of the reporting time period
Observation of primary auctions and underwritings
Audit Procedure
Confirmation with the holder of the debt security or fiscal agent or custodian
of debt records or trustee
Inspection of underlying debt agreements and other supporting documents,
confirmations received by creditor, in paper or electronic form, for amounts
reported
Assertion Audit Procedure
Comple- Review of all counterparty transactions. When requesting details from the
teness
counterparty, consider which part is responding, and whether this represents
all relevant aspects of its dealing with the borrower
Send zero-balance confirmations to potential debt holders or counterparties
Review primary dealers’ statements for the existence of transactions and
holdings of public debt
Use computer-aided techniques to extract aggregate trading data for
agreement with general ledger and financial statement report
Perform sampling tests of individual trades for counterparty confirmations
and after-date receipts
Review accounting records before and after the year end for unusual
transactions
Review counterparty confirmations received but not matched to transaction
records
Review unresolved reconciliation items in reports
Inspect debt agreements for embedded derivatives
Review minutes of debt committee and related papers
Perform calculation for proper accrual and recognition of debt expense
INTOSAI’s Public Debt Committee
Page 9 of 16
Guidance on Substantive Tests
DRAFT
Assertion
Valuation and
Measurement
Audit Procedure
Inspect documents to verify cash receipts in borrowing
Confirm the nominal value of debt amounts with fiscal agent or
trustee
Re-calculate mark-to-market calculations for a sample of high value
debt instruments
Check the accuracy of translation of book and market value of debt
securities denominated in foreign currencies
Use quoted market prices to verify values disclosed of debt
securities, money market instruments and derivatives
Assertion
Presentation &
Disclosure
Audit Procedure
Verify that accounting principles selected and applied are in
conformity with legislation, regulations and applicable accounting
standards, and are appropriate for the debt management office
Verify that the financial statements and related footnotes provide
sufficient disclosure that is neither too detailed nor too condensed
Verify that the financial statements provide information on matters
that may affect their use, understanding and interpretation
Verify that the financial statements reflect transactions in a manner
that present the debt levels, results of borrowings and interest
payments, and cash flows within a range of acceptable limits
Review the classification of debt securities to ensure it is in
agreement with the legislation, regulations and practices
INTOSAI’s Public Debt Committee
Page 10 of 16
Guidance on Substantive Tests
DRAFT
PART 2 – Analytical Procedures
1. Analytical procedures consist of comparing recorded account balances with the
auditor's expectations. The auditor develops an expectation or estimate of what
the recorded amount should be based on an analysis and understanding of
relationships between the recorded amounts and other data. This estimate is then
used to form a conclusion on the recorded amount. A basic premise underlying
analytical procedures is that plausible relationships among data may reasonably
be expected to continue unless conditions are known that would change the
relationship.
2. Analytical procedures generally rely on aggregate data rather than unit values,
which makes them more effective and efficient than tests of individual
transactions. Common analytical procedures involve the use of ratios, trends and
variance analysis. More sophisticated analytical procedures use econometric
analysis, including regression, simulations, stress-testing and large-scale
economic models.
3. An Illustration – Explaining the Difference between Expected and Actual Interest
Expense. Suppose the auditor estimates that the interest expense for the current
period is $80 million. The auditor obtains this estimate based on a $1 billion
public debt average balance times 8 percent, the average annual interest rate. The
materiality limit for analytical procedures is $5 million. The auditor finds that the
the actual amount of interest expense is $94.5 million. The difference - $14.5
million - exceeds the test materiality by $9.5 million. Auditors ask debt managers
and their explanation is that “we borrowed more money and interest rates are
higher than last year”. The auditor needs to corroborate this explanation. For
example, auditors can find that interest rates increased during the year and then
fell, and were computed to average 9 percent based on a monthly average instead
of 8 percent. Additionally, loan statements from lenders indicate that $100
million was borrowed and repaid during the year, and the additional borrowings
were outstanding for 6 months. Thus, the average loan balance was actually $50
million higher and the average interest rate was 1 percent higher than the figures
used in the auditor's original estimate.
4. If substantive analytical procedures are used, the auditor should perform steps a.
through l. below.
a. Determine the amount of the limit. The limit is the amount of difference
between the auditor’s expectation and the recorded amount that the auditor
will accept without investigation. The determination of the limit is a
matter of the auditor's judgment.
b. Identify a plausible, predictable relationship and develop a model to
calculate an expectation of the recorded amount. Consider the type of
INTOSAI’s Public Debt Committee
Page 11 of 16
Guidance on Substantive Tests
DRAFT
misstatements that could occur and how those misstatements would be
detected by the model.
c. Gather data for developing the expectation, and perform appropriate
procedures to establish the reliability of the data. The reliability of these
base data is subject to the auditor's judgment.
d. Develop the expectation of the recorded amount using the information
obtained during the previous steps. The preciseness of the expectation is
subject to the auditor's judgment.
e. Compare the expectation with the recorded amount, and note the
difference.
f. Obtain explanations for differences that exceed the limit, since they are
considered significant.
g. Corroborate explanations for significant differences.
h. Determine whether the explanations and corroborating evidence provide
sufficient evidence for the desired level of substantive assurance. If unable
to obtain a sufficient level of substantive assurance from analytical
procedures, perform additional procedures and consider whether the
difference represents a misstatement.
i. Consider whether the assessment of combined risk remains appropriate,
particularly in light of any misstatements identified. Revise the assessment
of combined risk, if necessary, and consider the effects on the extent of
detail tests.
j. Document the amount of any misstatements detected by substantive
analytical procedures and their estimated effects. The limit -- the amount
of the difference between the recorded amount and the expectation that
does not require explanation -- is not considered a known or likely
misstatement and is not included in the list of possible audit adjustments.
k. Conclude on the fair presentation of the recorded amount.
l. Include documentation of work performed, results, and conclusions in the
workpapers.
INTOSAI’s Public Debt Committee
Page 12 of 16
Guidance on Substantive Tests
DRAFT
PART 3 – Substantive Audit Procedures for Derivatives
1. Derivatives have become commonly used in public debt transactions. Major
industrialized countries with established capital markets routinely use
combinations of swaps and long-term debt borrowing to shorten the duration of
their debt profile, reduce their cost of borrowing, and provide liquidity to their
benchmark securities across the yield curve. Other governments find that
traditional public debt arrangements, like loans from multilateral financial
institutions, are routinely offered with derivatives features that require special
audit and management skills. Examples of the latter category include World
Bank loans with collars, different rates and currencies.
2. Common substantive audit procedures applied to derivative instruments include
the following.
Assertions: Completeness and Existence
•
•
•
•
•
•
•
•
Confirm significant terms with the holder of, or counterparty to, the
derivative
Inspect underlying agreements and other forms of supporting
documentation, in paper or electronic form
Ask holder of or counterparty to the derivative to provide details of all
derivatives and transactions with the debt management office.
Send zero-balance confirmations to potential holders or counterparties to
derivatives to test completeness
Review brokers’ statements for existence of derivative transactions and
positions held
Review counterparty confirmations received but not matched to
transactions records
Review unresolved reconciliation items
Inspect agreements, such as loan or equity agreements, for embedded
derivatives
Assertions: Valuation and Effectiveness
•
•
•
•
•
•
Assess of the reasonableness of models, variables and assumptions used to
value derivatives
Gather market prices to assess valuation that are firm and valid.
Assess the sensitivity of valuation to changes in variables and assumptions
Inspect supporting documents for settlement of the derivative transactions
after end of the reporting period
Use proprietary models or the debt management office’s internally
developed models to assess valuation when no market prices exist
Use analytical procedures to evaluate risk management policies, including
compliance with credit limits.
INTOSAI’s Public Debt Committee
Page 13 of 16
Guidance on Substantive Tests
DRAFT
•
Assess the effectiveness of hedging activities (e.g., to minimize large
gains or losses)
Assertion: Derivatives Used for Hedging
In the case of derivatives used as hedges, auditors should gather sufficient
evidence to determine effectiveness of the hedging relationship. This entails the
following procedures:
• Assess whether the derivative was designated as a hedge at the inception
of the transaction
• Determine what was the nature of the hedge
• Determine what was the risk management objective for undertaking the
hedge
• Determine what was the debt manager’s assessment of the hedge’s
effectiveness
• If the derivative was hedging a future debt transaction, determine what
was the debt manager’s assessment of the likelihood of the future event
• Assess the extent of disclosures of derivatives used as hedges, and extent
of compliance with laws and regulations that require disclosure of
derivative transactions, including notional and fair value, number and
credit quality of counterparties, value at risk, stress test results, etc.
INTOSAI’s Public Debt Committee
Page 14 of 16
Guidance on Substantive Tests
DRAFT
PART 4 – Substantive Audit Procedures for Unrecorded Contingencies
1. Large unrecorded contingencies and commitments are a major financial challenge
in both OECD and developing countries. Escalating health care costs combined
with an aging population have created an unsustainable fiscal risk in the United
States, Japan and other OECD countries. Some countries spend scarce budget
resources to subsidize government-owned enterprises (e.g. government-owned
railways). Most countries provide explicit and implicit guarantees on deposits of
financial institutions. These unrecorded contingencies and commitments are not
fully accrued and reported in the government’s financial statements.
2. Common substantive audit procedures applied to unrecorded contingencies and
commitments include the following.
•
•
•
•
•
Examine approved budget for all outstanding government guarantees,
subsidies and commitments
Review the government’s expenditures and compare actual outlays with
budget amounts
Examine long-term lease agreements for possible capital lease recognition
Examine government pension agreements for benefit vesting provisions and
funded status
Examine privatization agreements of former government-owned enterprises
for minimum revenue government guarantees
INTOSAI’s Public Debt Committee
Page 15 of 16
Guidance on Substantive Tests
DRAFT
PART 5 – Audit Sampling Techniques
1. Auditors can apply substantive audit procedures to all items that comprise a debt
account balance, to a non-representative small group of the items, or to a
representative sample of the items. All debt items would be generally selected if
their total number is small. The option to select a nonrepresentative group is
appropriate when the auditor knows enough about the population and is able to
identify the specific items of interest, usually because they are likely to be
misstated or have a high risk. In this case, the results of the application of
substantive tests and the effects of any misstatements must not be projected to the
items that were not tested.
2. Finally, a representative selection of items is appropriate when there are many
items and it is not possible to obtain sufficient assurance by examining only a
nonrepresentative group. Selecting a representative sample of debt items allows
auditors to project the results of their substantive tests on the sample to the
population. Auditors generally find that it is efficient to combine the above three
selection methods. For example, an auditor can select all debt items with a large
book amount and a statistical sample of the balance of the other debt items.
3. In the process to select a representative sample the auditor must first define the
item to be sampled. In dollar-unit sampling (DUS), each dollar of debt has an
equal chance of being selected. In classical variables estimation sampling, each
item in a stratum has an equal chance of being selected.
4. Probability-proportional-to-size (PPS) sampling or dollar-unit sampling (DUS).
This technique is used to give larger debt items proportionally more opportunity
to be selected than units with smaller debt amounts. Auditors use this technique
when they expect to find a small number of large misstatements (this is also
known as the “needles in haystack” challenge). This technique usually results in
smaller sample size and can be used in conjunction with stratified selection
techniques.
5. Stratified Selection. This technique allows auditor to group together
homogeneous debt transactions and instruments. A stratified selection allows
auditor to apply different audit techniques to each debt classification, and
improves efficiency of sample design, achieving a smaller sample size. By
stratifying the population, the auditor can give greater representation to the larger
recorded debt transactions that are small in number.
INTOSAI’s Public Debt Committee
Page 16 of 16
Guidance on Substantive Tests