Chapter 9: Evaluating
intelligence-led policing
Evaluation is key to crime control

Five features of a rational approach to crime control:





Adequate investment in measuring and monitoring
Open access to crime and justice information
Reliance on evidence in the development of policy
Commitment to rigorous evaluation
A flexible and eclectic approach to control
Weatherburn, D. (2004) Law and Order in Australia: Rhetoric
and Reality (Sydney: Federation Press) pp.36-38.
Basics of evaluation

Did you get what you expected?



Was the program implemented as expected?
Did it crime reduce? If so…
Compared to what?

What is the baseline ‘null hypothesis’ situation that makes
a good comparison?
Two structures to evaluation
Information structures
Criminal environment
Interpret
Analysis
Impact
Influence
Decision-maker
Two structures to evaluation
Organizational structures
Criminal environment
Interpret
Analysis
Impact
Influence
Decision-maker
Two types of evaluation

Outcome evaluation


Tells you whether a process had the desired effect, i.e. that
crime was reduced
Process evaluation

Can tell you why the outcome occurred. Examines the
operation to see why an observed outcome happened
Operation Vendas

Pilot informally assessed as successful, but Operation
Vendas did not have desired impact



New South Wales, Australia
Sought to increase the risk and speed of capture for
offenders by boosting the volume of forensic evidence
collection and reducing the time to get samples analyzed
Process evaluation



Found stated aim of crime scene examinations never
attained
Discovered a lack of forensic resources
Identified training problems
Operation Safe Streets

Philadelphia Police Department



Placed officers on permanent post at over 200 drug
corners in the city
Outcome evaluation found officers had a localized
dampening effect on crime
Giannetti (2007) reports that



Officers began to take calls away from corners
Foot patrols reverted to roving car patrols
Incentive to arrest was removed and information flow to
detectives reduced
Volume and type of anonymous drug tip
Analytical skills for evaluation

Some skills that may be required





Analytical dexterity
Proficiency in non-parametric and regression interpretation
Spatial analysis
Interrupted time series analysis
Crime mapping
‘quite soon, crime mapping will become as much an
essential tool of criminological research as statistical
analysis is at present’
Clarke, R.V. (2004) 'Technology, criminology and crime science', European
Journal on Criminal Policy and Research, 10:1, pp. 60.
Maryland Scientific Methods Scale


Scale of zero (no confidence in the findings) to five
(high confidence in the results)
A hierarchy of evaluation standards (top to bottom)





Randomized, controlled, double-blind trials
Quasi-experimental studies (experiments without
randomization)
Controlled observational studies
Observational studies without a control group
Expert opinion!
Realistic evaluation


Pawson and Tilley’s realistic evaluation or scientific
realist approach
Researchers should



Investigate the relationships between context, mechanism
and outcome
Study using more qualitative, narrative, and ethnographic
research techniques.
Understand that the key is to clarify how the choices that
people make affect the outcome of the programs under
examination
Operation Anchorage




Canberra, Australia – Australian Capital Territory
(ACT)
February 2001 to end of June 2001
Significant problem with burglary
Anchorage placed significant emphasis on



Senior leadership
Targeting of recidivist offenders through crime and
intelligence analysis
The development of joint operations across different
branches of ACT Policing
Operation Anchorage




Four teams of 10-12 investigators
6 police analysts
New targets were circulated every two weeks
Anchorage came on the heels of two relatively
unsuccessful operations, called Chronicle and Dilute
Weekly burglary frequency in the ACT
Economic and social costs of crime


In Australia, a 2003 estimate of the cost of burglary
to society found a cost in Australian dollars of
AU$2,400 per burglary




AU$2,000 per residential burglary
AU$4,500 per non-residential burglary
Total saving for Operation Anchorage
AU$7,125,600


AU$1,257,600 during Anchorage
AU$5,868,000 benefit after Anchorage
Financial benefits of Operation Anchorage
Studying recidivists



232 people arrested during Anchorage
119 had committed at least one offence before 1999
Chart their aggregate number of days in prison or on
remand…
Incarceration rates
Impact of recidivist incarceration
Measuring success in different ways

In regard to major criminal and terrorist operations…

‘Not only are we expected to anticipate the next
move, but we also have to do something about
bringing those involved to justice. This highlights one
of the fundamental differences between intelligence
that aims to warn and prevent, and investigations for
which success is measured by successful prosecution
and conviction’

Mick Keelty, Australian Federal Police Commissioner, 2004
Cost-effectiveness of the use of CIs
Reward cost only
Full cost
Cost for each arrest
£54 ($87)
£697 ($1,125)
Cost for each crime clearance
£27 ($44)
£348 ($561)
Value of property recovered to
cost ratio
£34 ($55) to 1
£2.60 ($4.20) to 1
Adapted from Dunnighan and Norris (1999)
Operation Green Ice

Operation Green Ice




DEA set up their own bank in a sting operation to tempt
drug traffickers into money laundering
Undercover agents laundered US$20 million of Colombian
drug cartel money
Led to arrest of seven of the Cali drug cartel’s top financial
managers, the seizure of more than US$50 million in assets
worldwide, and the arrest of 177 people
United Nations estimated Cali cartel profits to be
about US$30 billion a year
RCMP Disruption Attributes Tool
Core business
Attribute
The instruments /
description process central to the
criminal enterprise
High
Medium
Low
Nil
Removed the
capacity to
supply/operate
Financial
Financial capacity / status of the
group including profits / financial
assets gained through organized
crime
Removed and/or interrupted the
organization’s financial ability to
mount their large scale operations
Personnel
Individuals employed
through organized crime
Arrested and/or charged
individuals with the majority
of the knowledge, contact,
expertise, experience, and
executive influence
Interrupted
Seizure / restraint of significant
Arrested and/or charged
production and/or
proceeds of crime relative to the support personnel/skilled
distribution of supply financial scope of the organization operators with expertise,
network
knowledge and contacts
Seized commodities Seizure / restraints of minor
Arrested and/or charged
without disrupting
proceeds of crime or interruption replaceable unskilled
production and/or
of the means to launder and/or
operators / street level
distribution
legitimize proceeds of crime
operators / couriers
No commodities
seized
No profits / financial assets seized No individuals arrested
Performance anxiety


Some performance areas are so vague as to create
huge numbers of performance measures.
UK government priority areas include







reducing crime
investigating crime
promoting safety
providing assistance
citizen focus
resource use
local policing
Unintended consequences of measures










Tunnel vision
Sub-optimization
Myopia
Measure fixation
Misrepresentation
Misinterpretation
Gaming
Ossification
Demoralization
Discreditability