Data Sharing Case Study
Checkpoint Effectiveness: Models and Metrics
Jay Nunamaker & Elyse Golob
19 March 2014
Background
• Project initiated by CBP- Office of Border Patrol (OBP) to assess
the effectiveness of traffic checkpoint operations for the public
good.
– Purpose: To address recommendations noted in GAO Report No.
GAO-09-824: “Checkpoints Contribute to Border Patrol's Mission,
but More Consistent Data Collection and Performance
Measurement Could Improve Effectiveness”
• Key goals were to evaluate and address:
–
–
–
–
Checkpoint data integrity, consistency and accuracy
Measures of checkpoint impacts on local communities
Effectiveness metrics and models
Development of a managerial tool to inform resource allocation
• Overall Timeline
– Project initialization – October 2010
– Final report – December 2012
2
Data Collection
• Site visits to five different
sectors
• Meetings with OBP HQ for
feedback
• Group Collaboration sessions
and brainstorming with BP
agents to better understand
the data collection process
• Requested electronic data
sets
– e3: Apprehension Information
– CAR: Checkpoint Activity
Report
3
Data Collection
• OBP representatives
provided full support with
3 data collection
processes
– Site visits, HQ meetings,
Brainstorming sessions
• Challenges encountered in
obtaining e3 and CAR data
– Data was determined to be
Sensitive
– Time spent by DHS to
determine appropriate
solution for data handling
delayed related project
milestones
4
Explored Processes for Data Handling
• Explored:
– Use of CRADA
– Altering Cooperative agreement to allow for SBU data
• Solutions:
– OBP determined amount of SBU data was not significant
to project and did not make it available
– OBP would incorporate the SBU data in after work is
completed
– Cooperative agreement unaltered
– Unclassified data provided; No SBU data provided to
BORDERS
5
Data Acquired
• November 2011,
BORDERS received a open
source data set
• The data set was a
subsample of the data that
had been requested
– e3 data set was comprised
of apprehension data
spanning from FY2006 - May
31 of FY2011.
– Set contained only data that
did not require an NDA,
amendment to the
cooperative agreement,
6
Data Quality
• BORDERS researchers
identified multiple
challenges with this data
set
– For example: a
percentage of latitude
and longitude values for
the ‘ARREST_LATUTUDE’
and
‘ARREST_LONGITUDE’
data points were not
accurate, resulting in
arrest locations outside of
the United States
7
Observations
•
•
Time consumption spent
requesting/reviewing data received
Limited data set received
–
–
•
As a result of identifying the data quality
issues, BORDERS was able to make
recommendations to OBP on how to improve
their data collection practices
The restricted data set potentially limited the
findings and potential outcomes of this
project as data quality issues present in the
complete data set were not accessible to
researchers, and could thus not be identified
Challenges
–
–
Large amount of time attempting to acquire
data
Unclear decision process and criteria as to
what sensitive data could be sent made it
challenging for BORDERS to be proactive in
providing clarification
8