Surveillance Evaluation
Assist Prof Dr. Kwankate Kanistanon, DVM, MS, PhD
What is health surveillance

“The tracking and forecasting of any health
event or health determinant through the
continuous collection of high quality data, the
integration, analysis, and interpretation of data
into surveillance products and the dissemination
of surveillance products to those who need to
know a specific public health purpose or policy
objective”
Why evaluate surveillance

To improve and support a
health surveillance
Evaluation helps answering the
questions




1. What are the success and deficiency of the
surveillance system?
2. Is the surveillance system meeting its piblic
health objective?
3. How does surveillance support and benefit
stakeholders?
4. What measures could improve performance
and productivity of the surveillance system?
What to evaluate?




1. The quality of the information that their
systems produce
2. The effectiveness in supporting the objectives
of the programs that they serve
3. The effectiveness in supporting informed
decision-making
4. The efficiency of their systems
Effectiveness



How well surveillance can produce
purposed outcome
Is the information use?
Response to the objectives of
surveillance
Efficiency


Cost, time, and labor
Resources consumed and the value of
outcomes
6 Steps for surveillance evaluation

1. Establishing the context of the surveillance

2. Developing evaluation questions

3. Designing the process for data collection and
management

4. Summarizing and presenting the findings

5. Reviewing an evaluation report

6. Following up on the use of findings
Step 1: Establishing context



Why evaluate? You must be able to identify the
reasons for evaluation
Who are stakeholders for evaluation? You must
identify the persons or organization that will benefit
from the evaluation
What is this surveillance system? You must find
information of surveillance before you evaluate it
What is this surveillance?






Purpose
Population under surveillance
What data is collected and summarized
Who uses the surveillance
What action is expected from data
analysis and interpretation
Activities of surveillance
Example of surveillance activities


Identify the disease, case, population
Develop data collection tools, active or passive
surveillance

Data collection

Data audit and analysis and Interpretation

Report writing

Dissemination of results
Step 2: Evaluation questions







Questions are designed to ask for
2.1. Acceptability
2.2. Simplicity
2.3. Flexibility
2.4. Data quality
2.5. Positive predictive value
2.6. Sensitivity
Step 2: Evaluation questions




2.7. Representativeness
2.8. Timeliness
2.9. Stability
2.10. Compliance
2.1 Acceptability

= Willingness of persons and organizations
to participate in the surveillance


Are the data collected within the normal
course of operations?
Is there a mutual understanding of
jurisdictional mandates, addressing security,
and privacy
2.2 Simplicity

The ease of operation

Appropriate technologies and labor
2.3 Flexibility

Accommodate changes in operating
conditions


Can the system respond to new conditions,
such as changes in case definition, labor,
equipment
Can the system accept, process and forward
another system’s information
2.4 Data quality


Can data describe the health event according to
the objective of the surveillance?
Is the quality of data acceptable?

Missing data

Correct conversion

Historical and current data having different meaning

Same data have more than one definition
2.5 Positive predictive value

= The percentage of cases reported to
the system that actually have the health
problem


How much the diagnosis depend on skill of
observers?
Is there a process that the diagnosis is
confirmed?
2.6 Sensitivity

= Percentage of cases with disease
detected by the surveillance

Ability to detect outbreak

in a timeframe

correct number of diseased animals
2.7 Representativeness

The surveillance can explain the health
problem by person, place, and time


Can describe population characteristics
Can describe diseased animals
characteristics
2.8 Timeliness

Appropriateness of interval between
disease occurrence and

Report to health agency

Identifying the incidence by health agency

Implementation of control measures
2.9 Stability

Stability is measured by amount of time
required to manage and disseminate
the information to decision makers
2.10 Compliance

Comply with law or policy

Ethics

Confidentiality

Security

Privacy
Step 3: Data collection/Management

Collect data for evaluation

Observation

Interview the surveillance staff

Reviewing the surveillance report
Step 4: Findings

Summarizing information gathered from
the evaluation
Step 5: Reviewing the report

Presentation and dissemination of results

How and whom should the report sent to?

What evaluation report should say?


Clarify the weak and strong point of the
surveillance
Any suggestion for surveillance?
Step 6: Follow-up

Activities after the evaluation

Allocation of resources

Creation of a policy

Technical assistance to the surveillance
How to describe the evaluation
results

1. Scoring system







1
2
3
4
5
=
=
=
=
=
Must be improved
Poor
Fair
Good
Excellent
2. Descriptive results: บรรยาย
3. Combined system: Scoring and text at
some points
Goal of evaluation

To improve and support a
health surveillance