FP7-SEC-2007-217862
DETECTER
Detection Technologies, Terrorism, Ethics and Human Rights
Collaborative Project
Survey of Counter-Terrorism Data Mining and Related Programmes
D08.1
Due date of deliverable: 30 November 2009
Actual submission date: 11 December 2009
Start date of project: 1.12.2008
Duration: 36 months
Work Package number and lead: WP06 Dr. Daniel Moeckli
Author(s): Dr. Daniel Moeckli, University of Zurich; James Thurman, University of
Project co-funded by the European Commission within the Seventh Framework Programme
(2002-2006)
Dissemination
Level
PU
PP
RE
CO
Zurich
Public
Restricted to other programme participants (including the Commission Services)
Restricted to a group specified by the consortium (including the Commission
Services)
Confidential,
only for members of the consortium (including the Commission
Services)
x
Survey of Counter-Terrorism Data Mining and Related Programmes
Executive Summary
1. The survey reflects a broad definition of data mining and also includes coverage of
related programmes relating to data collection and database construction.
2. In the West, collection activities have increased dramatically in the name of
countering terrorism. In addition to data collection involving air passengers, this
survey also describes general law enforcement collection activities as well as those
specifically targeting terrorist activity.
3. Air passenger information: in the United States, data mining in this area was
proposed in order to identify terrorist suspects who might not otherwise raise
suspicions. In the European Union, too, there seems to be interest in analyzing a
passenger’s travel activities in order to identify suspicious patterns which might
indicate criminal activity.
4. Private companies and non-law enforcement databases: in the US there has been
concern about the incorporation of data from these sources into general law
enforcement data bases.
5. Data analysis programmes that have been proposed and in some cases implemented
for counter-terrorism purposes are also considered. These include not only data
mining programmes but also a discernable trend of providing tools which guide
users in their analysis and decision-making.
1. Introduction
Data mining, also known as knowledge discovery, may be defined in a number of
different ways. The Two Crows Corporation, for example, defines data mining as “a
process that uses a variety of data analysis tools to discover patterns and
relationships in data that may be used to make valid predictions.”1 Hand, Mannila,
and Smyth, on the other hand, have defined it simply as “the analysis of (often large)
observational data sets to find unsuspected relationships and to summarize the data
in novel ways that are both understandable and useful to the data owner.”2 We take
a very broad definition of the term which may be paraphrased as the use of
information technology to attempt to derive useful knowledge from (usually) very
large data sets. We adopt this broad definition at this stage in light of the privacy
and human rights concerns raised by forms of data analysis that might not meet a
narrow definition of data mining.
1
Two Crows Corp. (2005). Introduction to Data Mining and Knowledge Discovery.
http://www.twocrows.com/intro-dm.pdf.
2
Hand, D. J., Mannila, H., & Smyth, P. (2001). Principles of data mining. Cambridge, Mass.: MIT
Press, p. 1.
1
At the same time, we note that data mining and other forms of data analysis that are
being carried out or explored in the counter-terrorism context represent one stage
in a series of data-related practices, each of which presents particular issues with
respect to privacy, ethics, and human rights. In order to perform data mining, there
must be data available. Thus, data has to be collected or recorded. Collected data
may be assembled in an organized fashion to build a database (data warehousing).
One or more databases may then provide the source of data on which data mining
tasks are performed. Particularly, the desire to effectively aggregate numerous,
disparate data sources through data mining has been common in the United States.
Preparation of data forms an intermediate stage prior to mining, and tasks such as
cleaning and transforming data have significant consequences for the quality and
reliability results. Any two databases may not contain the same kinds of data and
even if they do, that data may not be organized or labelled in the same way. Thus,
such dissimilar data will have to be put into the same kind of structure to allow data
mining tasks to be carried out. This kind of preparation of data is often necessary
when data is from different databases is analyzed.
Although data mining necessarily relies on the availability of data, the collection or
warehousing of data may take place without any deliberate form of data analysis in
mind—for example data may be collected merely to provide a record for interested
parties. Yet, any set of data which is available to the data miner may theoretically be
made subject to data mining tasks. We include in this survey some databases which
have been mentioned in the context of data mining in the counter-terrorism context,
but pre-existing databases which might primarily serve completely different
purposes have also been used as data sources for counter-terrorism intelligence. It
is also worth noting that an ongoing field of study is concerned with eliminating the
need to have an organized database: The universe of information available on the
internet represents a remarkable set of data, and the development of “web mining”
tools seeks to render as much of this data as possible susceptible to analysis.
Compared to a database, free text is relatively unstructured. Thus, tools may be
necessary to structure the data for analysis—to identify dates, for example, so that
dates are put in the same category and not mixed up with other numbers, such as
distance measurements or pagination.
There are a number of basic functions which data mining may perform. We briefly
discuss a few types which appear most frequently in the programmes in this survey.
Clustering seeks to identify “natural” groupings within a set of data.3 We use the
term link analysis to refer to methods aimed at exposing relationships between
data—often in the form of social links with terrorist suspects. Pattern analysis
attempts to identify patterns of unusual or anomalous deviation among data. Such
tasks may seek to identify common characteristics among suspects. Event detection
may represent a subset of pattern analysis which has the goal of predicting or
detecting the development of a threat. We use the term search to refer to the
retrieval of items of interest including the application of filters. Search might rely
simply on Boolean operators or sophisticated algorithms such as Google’s PageRank.
3
Ibid., p. 12.
2
Lastly, simple matching seeks to identify whether a particular set of data items
matches an item on an established list. This type of function occurs most typically
where information technology is applied to determine whether a potential airline
passenger is on a terrorist watch list or other list of law enforcement interest.
In the survey below, we attempt to preface each description of a programme with a
number of attributes of potential interest. In each instance, we identify the agency
that initiated or sponsored the programme and the current status of the
programme. In the case of data analysis programmes, we attempt to give the type
of general function or functions which the programme is designed to perform,
although in many instances this is quite speculative due to the limited amount of
information available. With respect to databases or other collections of data, we try
to identify the sources from which the data is taken or made available, as well as the
entities that have access to the data.
This document represents the first deliverable under Work Package 6 of the
DETECTER project but is still very much a work in progress. The survey is exploratory
in nature and serves as an initial building block for subsequent work.4 Deliverables
D08.2 and D08.3 of Work Package 6 will provide analytic assessment of data mining
programmes and their application. This survey and subsequent versions of it will
furnish input for the analysis within those reports.
Our survey currently has a marked concentration on activities in the United States. There are
several reasons for this. By far the most prominent discussion of data mining plans and
activities for counter-terrorism purposes has been in the US, making it an obvious starting
point. Additionally, the US has long been a leader in the information technology industry,
and it would thus be no surprise that the use and development of data mining technology
and techniques may be more advanced and diffuse within US law enforcement and
intelligence communities. Moreover, the US—long noted for its massive defence spending—
probably has allocated a much larger budget to counter-terrorism research and
development than any country in Europe. A longer tradition of freedom of information
legislation may also have resulted in greater disclosure of data mining and related activities
in the US than in Europe. While for all these reasons it is easier to identify counter-terrorism
data mining activities in the US, we hope to be able to include more coverage of European
activities in future versions of this survey.
Lastly, we note that due to the nature of the topic, comprehensive information concerning
the exact nature and function of the programmes here included is generally not available.
Thus, in many instances, we are left to extrapolate and speculate based on secondary or
often cryptic primary sources.
2. Survey of Selected Programmes
4
Comments and corrections are welcome and may be submitted to the authors directly at
daniel.moeckli@rwi.uzh.ch and james.thurman@rwi.uzh.ch.
3
2.1. International
CAHORS
Agency:
North Atlantic Treaty Organisation
Data Sources: The World Wide Web, data collected by end users
Access:
Unknown
Functions:
Clustering, Search, Pattern Analysis
Status: Under development
CAHORS is a NATO project spearheaded by the French-based concern, Thales, and
supported by the French National Research Agency.5 The project aims to provide a
comprehensive platform to meet the needs of various end users throughout the
entire intelligence process from collection to decision-making. One aspect of the
project seeks to develop web mining tools for textual data on the World Wide Web
in order to identify documents of interest for anti-terrorism efforts. Part of this
endeavour would involve the automatic collection of data from open sources.6 The
project seeks to provide tools for data preparation as well as importance-ranking of
data elements based on an original model of information value.7
2.2. United States
Pre-flight and Border Collection
2.2.1. Computer Assisted Passenger Pre-Screening System II (CAPPS II)
Agency:
Transportation Security Administration
Data Entry: Transportation Security Administration & Commercial Airlines
Maintenance: Transportation Security Administration
Access:
Transportation Security Administration
Functions:
Matching, Event detection
Status: Replaced by Secure Flight Programme
CAPPS II was a proposed programme for pre-screening airline passengers in the US
that was ultimately abandoned in 2004. Following the events of Sept. 11, 2001, the
US government saw the need to improve on the CAPPS system that had been in use.
The predecessor CAPPS system relied on matching passenger names to those names
that appeared on watch lists as well as flagging those passengers or itineraries which
had certain characteristics (the “CAPPS I rules”).8 The Government Accountability
5
Thomas Delavallade & Philippe Capet, Information Evaluation as a Decision Support for CounterTerrorism, RTO-MP-IST-086, available at http://ftp.rta.nato.int/public//PubFullText/RTO/MP/RTOMP-IST-086///MP-IST-086-14.doc, p. 14-2.
6
See ibid., p. 14-3.
7
See ibid., pp. 14-5–14-6.
8
Ryan Singel (16 July 2004). Life After Death for CAPPS II? Wired,
http://www.wired.com/politics/security/news/2004/07/64240; US Government Accountability Office,
4
Office (GAO) has described these “rules” as a “set of weighted characteristics and
behaviors … that TSA has determined correlate closely with the characteristics and
behaviors of terrorists.”9 At least one report has suggested that the purchase of a
one-way flight or the submission of cash payment for flights were among the “rules”
which triggered more intense security screening.10 The CAPPS II system would
supplement passenger name records with a home telephone number, home
address, and the individual’s date of birth. The system would represent an
additional expansion beyond the initial CAPPS in that all passengers would be subject
to the screening rather than simply those passengers who check luggage, and every
airline and airport would be subject to the programme.11 Most significantly,
whereas the CAPPS system had been administered by the airline companies, CAPPS II
was to be administered by a government agency, the Transportation Security
Administration (TSA). The passenger information passed on to the TSA reportedly
would have been checked against information contained in commercial and
governmental databases.12 Based on this analysis, each passenger would be
assigned a colour code to indicate their suspected potential threat level: “Green”
indicated no threat; “yellow” represented a potential threat which meant that the
passenger should be subjected to further security checks before being allowed to
board the flight; and “red” indicated that the individual likely represented
“‘imminent threat’ to the physical safety of the people on the plane” and should be
prohibited from boarding the flight.13
The Government Accountability Office (GAO) lists seven aspects of the programme,
and notes that there was an eighth aspect which was not disclosed for security
reasons.14 In addition to the watch list matching and application of CAPPS I rules
mentioned above, the other aspects were to include the verification of passengers’
identities by checking name records against commercial databases as well as
matching passenger names against lists of international fugitives and wanted lists,
lists of participants in security credentialing programmes, and temporary watch lists
such as involving stolen passports.15 With respect to data mining, it has been
reported that the CAPPS II system would involve the application of algorithms to
assist in the screening process and/ or identify patterns among data sets within the
databases that would be available to the system.16 One data mining application that
(March 2005). Aviation Security: Secure Flight Development and Testing Under Way, but Risks Should
Be Managed as System Is Further Developed. (GAO-05-356 ), p. 10
9
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key Activities
Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks.
(GAO-09-292). Retrieved October 01, 2009, pp. 8-9.
10
Ryan Singel (16 July 2004). Life After Death for CAPPS II? Wired,
http://www.wired.com/politics/security/news/2004/07/64240.
11
Electronic Frontier Foundation, CAPPS II: Government Surveillance via Passenger Profiling,
http://w2.eff.org/Privacy/cappsii/background.php.
12
Electronic Frontier Foundation, CAPPS II: Government Surveillance via Passenger Profiling,
http://w2.eff.org/Privacy/cappsii/.
13
Ibid.
14
US Government Accountability Office, (March 2005). GAO-05-356 Aviation Security: Secure Flight
Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed.
Retrieved September 22, 2009, p. 10. and n. 9, p. 9.
15
Ibid.
16
Ryan Singel (16 July 2004). Life After Death for CAPPS II? Wired,
5
was identified by the GAO was a programme that would seek to identify unknown
terrorist suspects based on an individual’s travel or transactional patterns. 17
Presumably, this programme would rely on models of terrorist activity developed on
the basis of previous intelligence work. At least one member of Congress expressed
concern that the system would rely on information-sharing with private contractors
and fail to comply with US legal requirements pertaining to data handling.18
2.2.2. Secure Flight
Agency:
Transportation Security Administration
Data Entry: Transportation Security Administration & Commercial Airlines
Maintenance: Transportation Security Administration
Access:
Transportation Security Administration
Functions:
Matching
Status: In use
Secure Flight is the successor to the abandoned CAPPS II programme.19 According to
a GAO report, the initial plans for the programme were similar to those for CAPPS II
and would build upon technology and processes that had been developed under
CAPPS II; however, Secure Flight would only implement some of the aspects of
CAPPS II and be limited to the pre-screening of passengers en route domestically
within the US as opposed to those flying in or out of the country.20 International
flights would initially be covered by the APIS system administered by Customs and
Border Patrol. Ultimately, however, screening for international flights would also be
handled by TSA.21 As of 2005, the programme intended to check passenger name
records against an extended watch list provided by the FBI’s Terrorist Screening
Center and was also exploring whether the inclusion of commercial data would make
watch list matching more effective.22 Initial testing also involved matching against
CAPPS I rules.23 Those passengers who are matched (referred to as “selectees”)
http://www.wired.com/politics/security/news/2004/07/64240; Electronic Privacy Information Center,
Passenger Profiling, http://epic.org/privacy/airtravel/profiling.html
17
US Government Accountability Office, (March 2005). GAO-05-356 Aviation Security: Secure Flight
Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed.
Retrieved September 22, 2009, p. 10.
18
Ryan Singel (16 July 2004). Life After Death for CAPPS II? Wired,
http://www.wired.com/politics/security/news/2004/07/64240.
19
Seifert, J. W. (January 18, 2007). Data Mining and Homeland Security: An Overview. Congressional
Research Service. Retrieved June 26, 2009, p. CRS-5.
20
US Government Accountability Office (2005). GAO-05-356 Aviation Security: Secure Flight
Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed.
Retrieved September 22, 2009, p. 11.
21
Transportation Security Administration, DHS (28.10.2008). Secure Flight Program. 73 FR 64018,
64020. Retrieved October 01, 2009.
22
US Government Accountability Office (2005). GAO-05-356 Aviation Security: Secure Flight
Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed.
Retrieved September 22, 2009, p. 12.
23
US Government Accountability Office (2005). GAO-05-356 Aviation Security: Secure Flight
Development and Testing Under Way, but Risks Should Be Managed as System Is Further Developed.
Retrieved September 22, 2009, p. 23.
6
would be subjected to additional screening.24 CAPPS is separate from Secure Flight
and operations of one do not influence the operations of the other.25 With respect
to the Secure Flight matching system, the TSA has some level of discretion, according
to the GAO. This discretion manifests itself in the relative importance that is
assigned to the various items of personal information, the scoring function that is
used to determine what level of correspondence between data items results in a
“match,” and the level and manner of variations in data items (e.g. name spellings,
etc.) that are permitted to result in the correlation of data items.26
Final Rules issued in October of 2008 indicated that the TSA intended to implement
at least a two-tier system of watch list matching. Generally, the TSA would merely
perform matching against the “No Fly” and “Selectee” lists within the Terrorist
Screening Database (TSDB). When higher threat levels were present, however, the
TSA would consider running matching against a broader array of data, such as all
components of the TSDB.27
Data matching would be conducted between data provided by air carriers and that in
selected database components as mentioned immediately above. The TSA
determined during initial development of Secure Flight that effective matching
would be greatly benefitted by the addition of passengers’ gender and date of birth
to passenger name records. The final rule indicated that these data elements would
be mandatory once the Secure Flight programme had been implemented.28
Passenger data would either be submitted to the TSA electronically where air
carriers used an automatic reservation system or entered by the air carrier via a
web-based access system known as eSecure Flight.29 Once the TSA has performed
matching, it will submit boarding pass instructions to the air carrier. These
instructions will either indicate that: 1) the carrier may issue an unrestricted
boarding pass to the passenger; 2) the carrier may issue a boarding pass indicating
that the passenger has been selected for enhanced screening; or 3) the carrier may
not issue a boarding pass and must await TSA instructions after the passenger has
arrived at the airport and presented a valid identification document.30 The final rule
24
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key Activities
Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks.
GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 9.
25
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key Activities
Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks.
GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 9.
26
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key Activities
Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate Risks.
GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 8.
27
Transportation Security Administration, DHS (28.10.2008). Secure Flight Program. 73 FR 64018,
64019. Retrieved October 01, 2009.
28
Transportation Security Administration, DHS (28.10.2008). Secure Flight Program. 73 FR 64018,
64020. Retrieved October 01, 2009 (see chart at page below); see also at 64021.
29
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key
Activities Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate
Risks. GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 6.
30
Transportation Security Administration, DHS (28.10.2008). Secure Flight Program. 73 FR 64018,
64019-20. Retrieved October 01, 2009.
7
does not indicate whether only those individuals who have been “matched” by the
system will be subject to the enhanced screening. Anecdotal reports have suggested
that existing screening systems have been set up to return occasional false positives
in order to introduce an element of randomness to the system. For instance, metal
detectors have reportedly been programmed to produce a beep on certain occasions
despite the fact that the system has detected no metal.
The Secure Flight system would initially apply only to domestic flights, while
international flights would continue to fall under the Customs and Border Patrol’s
APIS programme. Ultimately, however, screening for international flights would also
be handled by TSA.31
The GAO placed 10 conditions on the development of Secure Flight which had to
have been met before Secure Flight could go into operation. One condition was the
availability of some form of redress process through which passengers who claimed
to have been falsely included on a watch list could seek resolution and ultimately
freedom from enhanced security procedures that were inappropriately imposed.
Ultimately, TSA developed a system through which these passengers would be
issued a redress number to provide a permanent reference to that individual’s
request for redress.32 In cases where there did not appear to be a false match, TSA
would refer the matter to the FBI’s Terrorist Screening Center for resolution.33
Ideally, the redress number would allow those who had been cleared of watch list
status to avoid future security complications while travelling.
2.2.3. Automated Targeting System (ATS)
Agency:
Department of Homeland Security
Data Sources: Several Customs and Border Protection databases & Passenger Name
Records
Access:
US Customs and Border Protection & DHS contractors; other federal
and Canadian agencies may have access to underlying data sets
through ATS interface
Functions:
Pattern Analysis, Rule checking
Status:
In use
ATS was a system developed to screen cargo going in and out of the US. In 2006, the
Dept. of Homeland Security announced that it would be extending the program to
screen travelers entering the US. This action has evidently already taken place as a
US citizen was able to receive passenger data pertaining to himself which was held
31
Transportation Security Administration, DHS (28.10.2008). Secure Flight Program. 73 FR 64018,
64020. Retrieved October 01, 2009.
32
See US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key
Activities Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate
Risks. GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 11.
33
US Government Accountability Office (2009). Aviation Security: TSA Has Completed Key
Activities Associated with Implementing Secure Flight, but Additional Actions Are Needed to Mitigate
Risks. GAO-09-292 (GAO-09-292). Retrieved October 01, 2009, p. 11.
8
within the ATS system on January 13, 2008 through a Freedom of Information
request. Notably, the information that was revealed in the FOI response included
data from airline booking processes such as credit card numbers, frequent flyer
numbers, and hotel reservations.34 A notice in the Federal Register stated that the
system “both collects information directly, and derives other information from
various systems.”35 The system reportedly consists of six components, one of which
represents an analytical module evidently aimed at deriving trends from system
data.36
According to a 2006 Privacy Impact Assessment, data for the ATS is pulled from the
Automated Commercial System (ACS), the Automated Export System (AES), the
Automated Commercial Environment (ACE), and the Treasury Enforcement
Communication System (TECS). Additionally, information from other federal
databases such as the National Crime Information Center is also obtained. 37 Among
this information is vehicle registration data for those individuals who enter the US
via ground transport.38 The system also receives Passenger Name Records
submitted by commercial airlines.39 Similar to the CAPPS system, ATS involves the
application of a set of rules to system data to identify shipments and travellers who
represent a security or criminal risk.40
Access evaluations and other audits are reportedly conducted on a periodic basis.41
Data Analysis Programmes and Tools
2.2.4. Total Information Awareness/ Terrorist Information Awareness (TIA)
Agency:
Data Sources:
Access:
Functions:
Status:
DARPA
Unspecified
DARPA, Project Contractors, reportedly NSA42
Link Analysis, Pattern detection, Search, Event Detection, Human
Analytic Aids
Officially terminated; believed to have been transferred to one or
more other agencies, including as the projects “Tangram” and
“Topsail”43
34
Customs and Border Patrol Travel Record, FOIA Request DIS-4-OT: FD SU,
http://philosecurity.org/wp-content/uploads/2009/09/DHS-Travel-Record.pdf.
35
71 FR 64546 (02.11.2006).
36
Department of Homeland Security (22.11.2006). Privacy Impact Assessment for the Automated
Targeting System, p. 3.
37
Ibid., p. 7.
38
Ibid., p. 6.
39
Department of Homeland Security (22.11.2006). Privacy Impact Assessment for the Automated
Targeting System, p. 2.
40
Department of Homeland Security (22.11.2006). Privacy Impact Assessment for the Automated
Targeting System, p. 3.
41
Ibid., p. 21.
42
See Harris, S. (16.06.2006). Signals and Noise, http://www.shaneharris.net/2006/06/signals-andnoise.html.
9
TIA was a programme developed by the Defense Advanced Research Projects Agency
(DARPA). The phrase “Total Information Awareness” was in use at DARPA as early as
1999,44 but TIA came to be particularly associated with the activities of the post-9/11
Information Awareness Office under the leadership of Admiral John Poindexter.
According to official sources, the programme sought to “develop a modular system
architecture using open standards that will enable a spiral development effort that
will allow the insertion of new components when they are available.”45 Several
component projects were named by Admiral Poindexter in a speech he delivered at
the DARPATech 2002 Conference:
Project Genoa: Project Genoa was under development at least as
early as 1999. It is described as being “aimed primarily at
supporting intelligence analysis.” The objective appears to have
been to assist multiple analysts to assess and interpret data
together and arrive at a decision with respect to that data.
Human Identification at Distance (HumanID): HumanID sought to
develop a system for the positive identification of individuals
based on “multi-modal biometric technologies.” The elements
that the system would incorporate included facial and other body
part recognition, gait recognition, remote iris scan, infrared and
hyper-spectral imagery, and non-image-based biometrics.
Genysis: The Genysis project aimed to develop an “ultra-largescale” database. One objective was to develop ways to gain
access to existing databases that were previously unconnected.
The project also sought to develop privacy-enhancing
technologies.
TIDES & EARS: These projects concerned IT-based linguistic analysis
tools. The objectives were reported to have been to provide
search tools for finding information in foreign languages and make
possible the conversion of speech to text.
Evidence Extraction and Link Discovery (EELD): As the name of the
project suggests, it aimed to cull information from various
classified and unclassified data sources. Reportedly it would
include extracting data from “message traffic.” Based on this
extracted data, the programme would establish links with
relevance to terrorism, such as relationships between people,
organizations, and activities. The programme would also learn
patterns that represent terrorist groups and scenarios.
War Gaming the Asymmetric Environment (WAE): This project
aimed at developing predictive indicators of terrorist attacks and
43
Harris, S. (Oct. 20, 2006). Terrorist Profiling, Version 2.0. National Journal. Retrieved June 29,
2009; Harris, S. (Feb. 23, 2006). TIA Lives On. National Journal,
http://www.nationaljournal.com/about/njweekly/stories/2006/0223nj1.htm#.
44
See Sharky, B. Total Information Awareness, DARPATech 1999 Conference,
http://www.darpa.mil/darpatech99/Presentations/Scripts/ISO/ISO_TIA_Sharkey_Script.txt.
45
Remarks as prepared for delivery by Dr. John Poindexter, Director, Information Awareness
Office of DARPA, at DARPATech 2002 Conference, Anaheim, Calif., August 2, 2002, available at
http://www.fas.org/irp/agency/dod/poindexter.html.
10
behaviour. The model relies on the past behaviour of known
terrorists to predict what activities they will take next and where
they will take them. Validation of the programme was carried out
by comparing against archival data relating to 66 attacks which
occurred over a span of 17 years.
Bio-Surveillance: The Bio-Surveillance project was aimed at
addressing biological attacks. It sought to develop a programme
that would examine data sources to provide early warning of the
release of biological agents.
Genoa II: Genoa II sought to improve upon the original Genoa
programme. One focus was on increasing the speed of analysis
and decision-making. To this end, the project sought to automate
the data collection process as well as the presentation of analyses.
The presentation process was geared to build a knowledge
repository which would facilitate re-use of previously acquired
knowledge. The project also sought to develop tools to assist
analysts in organizing information and “thinking together” in
groups. An additional goal of the project was to develop tools to
assist and speed up collaboration among individuals in different
organizations. Among the tasks that the system would assist with
were resource identification, role allocation, planning, and policy
development and enforcement.46
Due to negative public reaction to the programme, Congress suspended funding for
TIA in 2003.47 Yet, reports have surfaced that the programme has not been
terminated altogether but merely renamed and moved to one or more other
agencies. Shane Harris of the National Journal sees TIA’s continuation in a
programme known as Tangram which is sponsored by the US Air Force Materiel
Command and reportedly also under the auspices of the coordinating agency
Advanced Research and Development Activity (ARDA).48 In an information packet
provided by the federal government, Tangram is characterized as a continuation of
the “Evidence Assessment, Grouping. Linking and Evaluation” (EAGLE) programme.49
EAGLE may likely be the successor of the EELD programme following TIA’s
suspension. Tangram is further described as having the goals of improving the
scalability and performance of the most promising algorithms developed under the
EAGLE project. Additionally, Tangram seeks to transform EAGLE into a continually
46
Ibid; See also Statement by Dr. Tony Tether, Director Defense Advanced Research Projects Agency
Before the Subcommittee on Military Research and Development Committee on Armed Services
House of Representatives (26.06.2001), available at
http://armedservices.house.gov/comdocs/openingstatementsandpressreleases/107thcongress/01-0626tether.html.
47
See, e.g., Harris, S. (20.10.2006). Terrorist Profiling, Version 2.0. National Journal,
http://www.nationaljournal.com/about/njweekly/stories/2006/1020nj3.htm#; Harris, S. (23.02.2006).
TIA Lives On. National Journal,
http://www.nationaljournal.com/about/njweekly/stories/2006/0223nj1.htm#.
48
See, e.g., Harris, S. (20.10.2006). Terrorist Profiling, Version 2.0. National Journal,
http://www.nationaljournal.com/about/njweekly/stories/2006/1020nj3.htm#.
49
Advanced Research and Development Activity & AFRL (2005). TANGRAM Proposer’s Information
Packet (PIP), https://www.fbo.gov/utils/view?id=8b216e2d32c807806164266b9996b7a8.
11
operating, consolidated, more user-friendly analysis system for broader deployment
throughout the US intelligence community. Other aspects of the Tangram
programme include: the development of a theoretical model of terrorist entity and
threat detection to assist the technical aspects of the system; building upon
experiences in fraud detection systems to develop a system for terrorist threat
detection; developing a system of “suspicion scoring” that could attach a particular
threat value to individual fragments of information; developing a system to fill in
information gaps by projecting suspicion scores based on available information;
providing a method to address the problem posed by the dynamic nature of human
behaviour to the suspicion scoring project.50
2.2.5. Multi-State Anti Terrorism Information Exchange (MATRIX)
Agency:
Data Sources:
Access:
Functions:
Status:
Multi-State Consortium
Various law enforcement databases
State law enforcement personnel
Search, Link Analysis
Officially terminated
The MATRIX was a plan to build upon a digital intelligence system that has been in
use in the State of Florida. The plan intended to bring together State resources from
multiple States so that law enforcement agencies in all the participating States would
have access to the totality of the information held in those resources within a single
system.51 The databases that the system would incorporate information from were
criminal history databases, corrections information and images, sexual offender lists,
driver’s license databases, and motor vehicle registration.52 Other sources indicated
that information from commercial databases would be included as well. The ACLU,
for instance, claimed that the system would also have access to:
 Credit information
 Driver’s license photographs
 Marriage and divorce records
 Past addresses and telephone numbers
 Names and addresses of family members
 Neighbours’ addresses and telephone numbers
 Business associates
 Social security numbers and dates of birth53
50
Ibid., pp. 8-9.
See, e.g., MATRIX: Multi-State Anti TeRrorism Information Exchange, available at
http://www.aclu.org/files/FilesPDFs/matrix%20brochure.pdf; Sole Source Criteria for the Multi-State
Anti Terrorism Information Exchange (MATRIX) Project, available at
http://www.aclu.org/files/FilesPDFs/sole%20source%20criteria%20for%20matrix.pdf; Seisint,
Inc.(Sept. 29, 2003). Seisint's FACTS for the MATRIX Project, available at
http://www.aclu.org/FilesPDFs/seisint_facts_83.pdf, p. 5.
52
Seisint, Inc.( 29 Sept. 2003). Seisint's FACTS for the MATRIX Project, available at
http://www.aclu.org/FilesPDFs/seisint_facts_83.pdf, p. 6.
53
American Civil Liberties Union (2004). Data Mining Moves into the States.
http://www.aclu.org/FilesPDFs/matrix%20report.pdf, p. 2.
51
12
And a Congressional Research Service report from 2004 indicated that the MATRIX
website had identified a number of public sector information that would be available
to the system, including corporate filings, state Commercial Code filings, bankruptcy
filings, professional licenses, and property registries.54 The system in use in Florida
was reportedly able to run various search queries at a much faster rate than with
standard processing.55 In addition to search capabilities, the MATRIX system would
be able to display relationship networks, provide geographic mapping of
information, display photo montages of multiple driver’s licenses, and generate
photo lineups for witness viewing.56
MATRIX was earmarked to receive federal funding for its development, and there
was some anxiety and speculation that it might be extended to the federal
government at some point later in time.57 The project was ultimately terminated
after the programme came under increased public scrutiny and the number of States
willing to participate began to dwindle.58
2.2.6. Novel Intelligence From Massive Data (NIMD)
Agency:
Data Sources:
Access:
Functions:
Status:
US National Security Agency (NSA)
Unknown
Unknown
Human Analytic Aids
Unknown
This programme is sponsored by the US National Security agency, and aims to reveal
interesting intelligence which might not otherwise be disclosed through traditional
methods of intelligence analysis. Sources indicate that, like TIA, NIMD seeks both to
bring together information from a variety of data sources and to assist human
analysts in overcoming natural limits and failures in human cognition so that they
may recognize the significance of intelligence data and evaluate it properly.59
Reportedly, the NSA was provided access to TIA during its early development and
54
Krouse, W. J. (18.08.2004). The Multi-State Anti-Terrorism Information Exchange (MATRIX) Pilot
Project. : Congressional Research Service, p. CRS-6.
55
Seisint, Inc.( 29 Sept. 2003). Seisint's FACTS for the MATRIX Project, available at
http://www.aclu.org/FilesPDFs/seisint_facts_83.pdf, p. 8.
56
See generally, Seisint, Inc.( 29 Sept. 2003). Seisint's FACTS for the MATRIX Project, available at
http://www.aclu.org/FilesPDFs/seisint_facts_83.pdf.
57
American Civil Liberties Union (2004). Data Mining Moves into the States.
http://www.aclu.org/FilesPDFs/matrix%20report.pdf, p. 2.
58
ACLU Applauds End Of "Matrix" Program, 15.04.2005, http://www.aclu.org/technology-andliberty/aclu-applauds-end-matrix-program; see also Singel, R. (15.03.2004) Wisconsin, New York
Unplug Matrix. Wired, http://www.wired.com/politics/security/news/2004/03/62645.
59
See, e.g., MITRE Corp.(09.2002). New Research Center Focuses on IT and the Intelligence
Community. Retrieved December 09, 2009, from
http://www.mitre.org/news/digest/defense_intelligence/09_02/di_research_nnrc.html, Harris, S.
(20.01.2006). NSA spy program hinges on state-of-the-art technology. National Journal, available at
http://www.govexec.com/story_page_pf.cfm?articleid=33212.
13
appropriated tools that had been developed under TIA.60 Thus, it may be possible
that NIMD incorporates elements of TIA.
Reportedly, part of the NIMD programme consists of a programme known as Glass
Box.61 The purpose of Glass Box may consist of culling information from various
public data sources for use by NIMD, evaluating data mining algorithms developed in
other elements of the NIMD programme, providing analytic assistance, evaluating
the analytic process employed by intelligence analysts, or all of the above.62
2.2.7. Analyst Notebook I2
Agency:
Data Sources:
Access:
Functions:
Status:
Department of Homeland Security
Unknown
Unknown
Pattern Analysis, Link Analysis
In use
I2 Analyst’s Notebook is a commercially-available data mining and analysis product
from i2, Inc. which focuses on fraud detection, and criminal and anti-terrorist
intelligence. The product is designed to analyze large data sets and display analyses
such as patterns, trends, or link analysis in graphical form. The product also permits
the manual or automatic creation of tables and charts as well as briefing charts for
intelligence sharing.63 The product permits the importation of structured data from
sources such as Lexis-Nexis, Dun & Bradstreet, and the FBI’s Regional Information
Sharing Systems.64 Other i2 products that may be used in conjunction with Analyst’s
Notebook permit the importation of data from multiple sources simultaneously and
the conversion of unstructured text into structured data.65 Analyst’s Notebook also
enables geographical mapping of locations of interest via Google Earth.66 According
to a 2004 report from the GAO, the product has been used by the Dept. of Homeland
Security to “[c]orrelate[ ] events and people to specific information.”67
2.2.8. Secure Collaborative Operational Prototype Environment (SCOPE)
Agency:
Department of Justice/ FBI
Data Sources: Unspecified
60
Harris, S. (16.06.2006). Signals and Noise, http://www.shaneharris.net/2006/06/signals-andnoise.htm.
61
Dillard, W.P. III. (17.06.2005). NSA searches for novel intel answers in the Glass Box. Government
Computer News, http://gcn.com/articles/2005/06/17/nsa-searches-for-novel-intel-answers-in-the-glassbox.aspx?sc_lang=en.
62
See ibid.
63
See http://www.i2inc.com/products/analysts_notebook/#capabilities.
64
Ibid.
65
See ibid. See Introduction for an explanation of “structured” versus “unstructured” data.
66
http://www.i2inc.com/products/analysts_notebook/#new.
67
US General Accounting Office. (May 2004). GAO-04-548 Data Mining: Federal Efforts Cover a
Wide Range of Uses, p. 44.
14
Access:
Functions:
Status:
FBI
Search, possibly Link or Relationship Analysis
In use
SCOPE provides data processing functionality for the Investigative Data Warehouse.
It represents a single interface through which FBI agents may conduct searches
across multiple data sources in order “to uncover terrorist and criminal activities and
relationships.”68 SCOPE can handle both structured and unstructured textual data. 69
It was listed as operational in 2004.70
2.2.9. Insight Smart Discovery
Agency:
Data Sources:
Access:
Functions:
Status:
Defense Intelligence Agency
Unspecified public sector data
Unknown
Data Preparation
Under development in 2004
The short description of Insight Smart Discovery provided in the GAO’s 2004 report
suggests that it is a programme designed to prepare unstructured data for data
mining processes and provide visual analysis of data in terms of charts and
diagrams.71
2.2.10. Verity K2 Enterprise
Agency:
Data Sources:
Access:
Functions:
Status:
Defense Intelligence Agency
Unspecified private and public sector data
Unknown
Unknown
Listed as operational in 2004
Verity K2 Enterprise is a data mining programme aimed at identifying terrorist
suspects, including among US citizens.72
2.2.11. PATHFINDER
Agency:
Data Sources:
Access:
Functions:
Defense Intelligence Agency
Unspecified private and public sector data
Unknown
Unknown
68
US General Accounting Office. (May 2004). GAO-04-548 Data Mining: Federal Efforts Cover a
Wide Range of Uses, p. 47.
69
Ibid.
70
Ibid.
71
See ibid., p. 30.
72
Ibid.
15
Status:
Listed as operational in 2004
The GAO Report provides virtually nothing in terms of specifics with respect to this
data mining programme. Information as to whether the programme would utilize
private sector data is also contradictory.73
2.2.12. Autonomy
Agency:
Data Sources:
Access:
Functions:
Status:
Defense Intelligence Agency
Unspecified public sector data
Unknown
Search
Listed as operational in 2004
Autonomy is search engine designed to perform searches on textual data.74
2.2.13. Counterintelligence Automated Investigative Management System
(CI-AIMS)
Agency:
Data Sources:
Access:
Functions:
Status:
Department of Energy
Unspecified public sector data
Unknown
Potentially Pattern Analysis and/or Event Detection
Listed as operational in 2004
CI-AIMS is described as a system for tracking cases related to individuals or countries
that represent a threat to US energy infrastructure. The “purpose” of the system is
listed as detecting criminal activities or patterns, suggesting that it contains data
mining tools which perform either pattern or event detection or both.75
2.2.14. Autonomy
Agency:
Data Sources:
Access:
Functions:
Status:
Department of Energy
Unspecified public sector databases
Unknown
Potentially Pattern Analysis and/or Event Detection
Under development in 2004
The Autonomy programme sought to detect threats to US Dept. of Energy assets.
Data sources were referred to as “intelligence-related” databases although the 2004
GAO report indicates that other agency data would not be utilized.76
73
Ibid.
Ibid.
75
See ibid., p. 40.
76
Ibid.
74
16
2.2.15. Counterintelligence Analytical Research Data System (CARDS)
Agency:
Data Sources:
Access:
Functions:
Status:
Department of Energy
DoE briefing and debriefing reports
Unknown
Pattern Analysis and/or Event Detection
Listed as operational in 2004
CARDS is a system designed to analyse reports filed with respect to briefing and
debriefing Department of Energy (DoE) personnel who travel to foreign countries or
interact with foreign visitors to DoE facilities. The aim of analysis is to detect
potential threats to DoE assets.77
2.2.16. BioSense
Agency:
Data Sources:
Access:
Functions:
Status:
Center for Disease Control
Unspecified public and private sector data
Unknown
Event Detection
Listed as operational in 2004
A system designed to detect bioterrorist threats.78
2.2.17. Foreign Terrorist Tracking Task Force Activity
Agency:
Data Sources:
Access:
Functions:
Status:
Federal Bureau of Investigation
Unspecified public sector data and DHS and FBI data
Unknown
Event Detection
Listed as operational in 2004
This data mining programme is designed to identify instances of unlawful entry into
the United States and support deportation actions and prosecutions against foreign
nationals in the US.79
2.2.18. NETLEADS
Agency:
Department of Homeland Security, Immigration and Customs
Enforcement, Customs and Border Patrol
77
Ibid.
Ibid., p. 41.
79
Ibid., p. 47.
78
17
Data Sources: Multiple DHS databases and commercial databases
Access:
Department of Homeland Security, Immigration and Customs
Enforcement, Customs and Border Patrol (various levels of access)
Functions:
Search, Pattern Analysis
Status:
In operation
NETLEADS represents a suite of software tools that provides search capabilities over
multiple databases as well as trend and pattern analysis of data. The tools are used
by the Dept. of Homeland Security, Immigration and Customs Enforcement (ICE), and
Customs and Border Control.80 The software accesses data from internal databases
that are maintained by the Dept. of Homeland Security as well as from public
sources such as geographical location data and news feeds.81
Raw data going in to the system is evidently organized automatically in indexed
records.82 It is these records that are available to the search and analysis tools. The
search capabilities of the tool suite permit search of both unstructured and
structured data. The software also includes graphical analysis tools which have been
referred to as link analysis and trend analysis.83 Link analysis depicts relationships
and connections between and among both individuals and organizations. There is
also a “Timeline Analysis” feature which permits the comparison of different link
analysis graphs representing connections existing at different points in time. The
trend analysis feature permits users to look for trends across immigration cases.84
In 2006, it was reported that the ICE was seeking to establish appropriate
agreements to permit information sharing with other state and federal agencies. 85
The DHS has indicated that users are provided with data handling training prior to
receiving access and that the ability to audit the activities of users is in place. 86 The
system automatically generates audit logs which are examined daily for evidence of
anomalous activity.87
2.2.19. ICE Pattern Analysis and Information Collection System (ICEPIC)
Agency:
Immigration and Customs Enforcement
Data Sources: Multiple DHS, Department of State, Department of Justice, and Social
Security Administration databases
Access:
Immigration and Customs Enforcement
Functions:
Link Analysis, Search
80
DHS Privacy Office. (6 July 2006). Data Mining Report: Report to Congress on the Impact of Data
Mining Technologies on Privacy and Civil Liberties. : Dept. of Homeland Security (DHS Privacy
Office Response to House Report 108-774), p. 22.
81
Ibid., p. 23.
82
See ibid., p. 23.
83
Ibid., p. 22.
84
Ibid.
85
Ibid., pp. 22-23.
86
Ibid., pp. 23, 24.
87
Ibid., p. 24.
18
Status:
Deployment date 2006
ICEPIC is a programme implemented by the ICE which is specifically designed to
assist in counter-terrorism efforts. The program makes use of IBM’s Non-Obvious
Relationship Technology (NORA) to draw out connections between individuals and
organizations which may have used or been known under different names at
different times.88 The program assists analysts in generating leads for further
investigation. It also offers users search capabilities over multiple databases using a
simple search query.89 In 2006, it was reported that ICEPIC’s search functionalities
did not access commercial databases.90 The primary source for data consists of
databases maintained by the Dept. of Homeland Security, but databases maintained
by the Department of State, the Department of Justice, and the Social Security
Administration are also available. One issue identified with the programme was that
there was no way to identify whether source data had been modified.91 This fact
could be particularly significant in light of the fact that NORA required local
replication and storage of data.92 If future tasks relied on data stored from previous
tasks, results might not reflect the most up-to-date information and data identified
as erroneous might persist in the system.
2.2.20. Intelligence and Information Fusion (I2F)
Agency:
Data Sources:
Access:
Functions:
Status:
Office of Intelligence and Analysis
Multiple DHS and commercial databases
Office of Intelligence and Analysis
Link Analysis, Entity Resolution, Geospatial and Temporal Analysis
Under development in 2006
The I2F programme was listed as under development in 2006. Its aim was to provide
an interface for viewing, searching and analyzing multiple data sources. It was
planned that the program would consist of commercially available “off-the-shelf”
software products rather than a specially designed system. The program potentially
envisioned the capability to derive “unpredicated patterns, relationships and rules”
from data sources. Data sources would include both commercial as well as
government databases.93
88
DHS Privacy Office. (6 July 2006). Data Mining Report: Report to Congress on the Impact of Data
Mining Technologies on Privacy and Civil Liberties. : Dept. of Homeland Security (DHS Privacy
Office Response to House Report 108-774), p. 24.
89
Ibid., p. 24.
90
Ibid., p. 25.
91
Ibid., p. 26.
92
Ibid., p. 25.
93
DHS Privacy Office. (6 July 2006). Data Mining Report: Report to Congress on the Impact of Data
Mining Technologies on Privacy and Civil Liberties. : Dept. of Homeland Security (DHS Privacy
Office Response to House Report 108-774), p. 26.
19
2.2.21. ProActive Intelligence (PAINT)
Agency:
Data Sources:
Access:
Functions:
Status:
Office of Intelligence and Analysis
Multiple DHS and commercial databases
Office of Intelligence and Analysis
Link Analysis, Entity Resolution, Geospatial and Temporal Analysis
Under development in 2006
According to the 2008 ODNI Report, this program was aimed at studying “the
dynamics of complex intelligence targets . . . by examining causal relationships that
are indicative of nefarious activity.”94
2.2.22. Knowledge Discovery and Dissemination
Agency:
Data Sources:
Access:
Functions:
Status:
Intelligence Advanced Research Projects Activity
Multiple undisclosed intelligence databases
Unspecified
Network Tomography, Predictive Analysis, Hypothesis Generation and
Validation
Under development in 2008
This project was part of the Incisive Analysis portfolio of IARPA. It sought to develop
tools for accessing and utilizing data in numerous databases which were maintained
by separate agencies and offices within the US intelligence community. According to
a report from the Office of the Director of National Intelligence, the project did not
initially perform data mining; however, there Office expressed some possibility that
the tools developed under the project might be used for data mining purposes at
some later date. These tools the Office characterized as “network tomography,
predictive analysis, and hypothesis generation and validation tools.”95 Network
tomography, according to the report, is being used to identify patterns of deceptive
behaviour.96 The term “network tomography” is often used in the context of
evaluating computer networks—in particular the internet.97 It is unclear whether
the term is used in this sense or whether the networks of interest in this context
refer to networks of persons.
2.2.23. Video Analysis and Content Extraction (VACE)
Agency:
Intelligence Advanced Research Projects Activity
Data Sources: Unspecified, testing on foreign public footage and NIST TRECVID
material
Access:
Unspecified
94
Office of the Director of National Intelligence (15.02.2008). Data Mining Report, p. 5.
Office of the Director of National Intelligence (15.02.2008). Data Mining Report, p. 3.
96
Ibid.
97
See, e.g., “Network Tomography,” Wikipedia, http://en.wikipedia.org/wiki/Network_tomography.
95
20
Functions:
Status:
Object & Event Detection, Video Mining
Under development in 2008
VACE is a project aimed at developing tools for the automated evaluation of video
materials for matter of intelligence significance. The primary function of the
software is to permit subject-based queries for searching databases of video
content. However, the ODNI identified two aspects of the project which might
involve some form of pattern-based data mining. One aspect involves computer
vision and machine learning functions such as “(a) object detection, tracking, event
detection and understanding, (b) scene classification, recognition, and modeling, (c)
intelligent content services such as indexing, video browsing, summarization,
content browsing, video mining, and change detection.”98 The other aspect is the
application of these techniques to pattern-based issues such as would be involved in
the automated evaluation of surveillance footage from CCTV systems and the
identification of specific events within video content such as news footage. 99
According to the ODNI Report, the project uses footage collected lawfully from
public places outside of the US as well as footage from the NIST TRECVID project.100
2.2.24. Rapid Knowledge Formulation
Agency:
DARPA
Data Sources: Unspecified, testing on foreign public footage and NIST TRECVID
material
Access:
Unspecified
Functions:
Search, Human Analytic Aid
Status:
Under development in 2001
The Rapid Knowledge Formulation project was aimed at developing methods for
conducting quick database searches, build massive “knowledge bases” within a
relatively short time frame, and “draw inferences for key information.”101 The
project also strove to enable users to construct formal theories without referring to
the use of formal logic. Some of the things which the program evidently sought to
address were the identification of terrorist sleepers and weapons of mass
destruction capabilities.102
2.2.25. Analysis, Dissemination, Visualization, Insight and Semantic
Enhancement (ADVISE)
Agency:
Department of Homeland Security
Data Sources: Unknown
98
Office of the Director of National Intelligence (15.02.2008). Data Mining Report, p. 4.
Ibid.
100
Ibid., p. 8.
101
Statement by Dr. Tony Tether, Defense Advanced Research Projects Agency, Subcommittee on
Military Research and Development Committee on Armed Services House of Representatives, 26 June
2001, p. 23.
102
Ibid.
99
21
Access:
Functions:
Status:
Intended for numerous DHS bodies
Link Analysis
Terminated
ADVISE was a DHS research project which aimed to mine data from multiple
databases and provide results in the form of visual analysis.103 Descriptions of the
project suggest that data mining technology would include link and relationship
analysis tools.104 The program also sought to provide suspicious activity alerts.105
One DHS official was quoted in 2004 as stated that the program would be able to
import one billion pieces of structured data per hour and one million pieces of
information from unstructured text per hour.106 The program was discontinued in
2007. The revelation that testing was conducted on real data and that results had
been used as input for at least one intelligence report may have played a role in this
decision in addition to criticism that required procedural steps had failed to have
been taken in association with the project.107 DHS officials, however, also cited the
program’s high maintenance cost as well as the availability of less expensive
commercial off-the-shelf products which could also perform the same or similar
tasks.108
2.2.26. Able Danger
Agency:
Data Sources:
Access:
Functions:
Status:
Department of the Army
Unknown
Army Land Information Warfare Agency
Link Analysis
Terminated
Able Danger was a project under the auspices of the Army’s Land Information
Warfare Agency carried out between 1999-2000. The project had reportedly been
requested by the Special Operations Command to assist in counter-terrorism
efforts.109 The Defense Dept. described the project as a test project to assess the
application of certain analytic methods and technology on large amounts of data.
The project reportedly used link analysis to uncover non-obvious relationships
between individuals and relied on data from both classified and public data sources
amounting to 2.5 terabytes.110 The data and results were reportedly destroyed after
103
DHS, O.I. G. (2007). ADVISE Could Support Intelligence Analysis More Effectively. OIG-07-56
(OIG-07-56). Retrieved October 08, 2009, p. 4.
104
DHS, O.I. G. (2007). ADVISE Could Support Intelligence Analysis More Effectively. OIG-07-56
(OIG-07-56). Retrieved October 08, 2009, p. 4.
105
DHS, O.I. G. (2007). ADVISE Could Support Intelligence Analysis More Effectively. OIG-07-56
(OIG-07-56). Retrieved October 08, 2009, p. 5.
106
Sniffen, M.J. (05.09.2007). DHS Ends Criticized Data-Mining Program. Washington Post.
107
See ibid.; see generally, DHS, O.I. G. (2007). ADVISE Could Support Intelligence Analysis More
Effectively. OIG-07-56 (OIG-07-56). Retrieved October 08, 2009.
108
Sniffen, M.J. (05.09.2007). DHS Ends Criticized Data-Mining Program. Washington Post.
109
Seifert, J. W. (03.04.2008). Data Mining and Homeland Security: An Overview. Congressional
Research Service. p. CRS-18.
110
Ibid.
22
conclusion of the project in accordance with US Army regulations since it included
data on US individuals.111
Collection & Warehousing Activities
2.2.27. Threat and Local Observation Notice (TALON)
Agency:
US Department of Defense
Data Entry: All branches of military112
Maintenance: Counterintelligence Field Activity/ US Northern Command
Access:
Defense Intelligence Agency, Joint Intelligence Task Force-Combating
Terrorism, Northern Command
Sharing:
Believed to be limited sharing with local law enforcement
Status: Officially terminated, although a system of reporting still exists
The Threat and Local Observation Notice Program was an operation initiated by the
US Department of Defense (DoD) in an effort to create a central database of
suspicious activity reports associated with DoD activities and installations. TALON
was initially developed as a report format for the Air Force Office of Special
Investigations in 2001.113 This practice was then adopted throughout the entire DoD
on May 2, 2003.114 The input for the reports was supplied by civilians, military
personnel, and law enforcement.115 It was widely acknowledged that the
information that went into TALON reports was “non-validated, may or may not be
related to an actual threat, and by its very nature may be fragmented and
incomplete.”116 According to the 2003 memorandum, it was envisioned that the
following categories of information would be collected: “(1) non-specific threats to
DoD interests; (2) suspected surveillance of DoD facilities and personnel; (3)
elicitation, attempts, suspicious questioning or other suspected intelligence
collection activities focused on DoD interests; (4) tests of security; (5) unusual
repetitive activity; (6) bomb threats; and (7) any other suspicious activity and
incidents reasonably believed to be related to terrorist activity directed against DoD
personnel, property, and activities within the United States.”117
The DoD’s Counterintelligence Field Activity (CIFA) unit incorporated TALON reports
onto a database known as Cornerstone.118 Full access was to be provided to the
Ibid., pp. CRS-18 – CRS-19.
Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice
(TALON) Report Program (Report No. 07-1 NTEL-09), p. 7.
113
Ibid., p. 1.
114
Ibid.
115
Ibid., pp. 25-26.
116
Deputy Secretary of Defense, Memorandum for Secretaries of the Military Departments et al.
(02.05.2003), reprinted in Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local
Observation Notice (TALON) Report Program (Report No. 07-1 NTEL-09), p. 30.
117
Ibid., p. 31.
118
Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice
(TALON) Report Program (Report No. 07-1 NTEL-09), pp. 1, 5.
111
112
23
Defense Intelligence Agency, Joint Intelligence Task Force-Combating Terrorism.119
Another branch of the Defense Department, the US Northern Command, also held
TALON reports in its Joint Protection Enterprise Network (JPEN).120
The TALON system came under increased scrutiny after news reports surfaced that
the database contained numerous reports on anti-war demonstrators and pacifist
organizations.121 The Office of the Inspector General of the Defense Department
was subsequently prompted by members of Congress to perform an audit of the
system.122 At the time of the audit in December 2005, there were about 13,000
reports in the Cornerstone database.123 The audit determined that initially only ITpersonnel were capable of deleting reports from Cornerstone.124 Thus, those
individuals who might be charged with evaluating the content of the reports and
ensuring compliance with relevant laws and regulations were unable to delete
reports, nor, evidently, were IT-personnel routinely instructed to delete any reports.
Between 2 December 2005 and 18 January 2006, however, CIFA began evaluating
the contents of the database and deleted 1,131 TALON reports which were
determined not to fall within the categories provided in the 2003 memorandum or
which did not meet DoD regulations on document retention.125 According to the
Inspector General’s report, these TALON entries “pertained to criminal activity such
as Be On the Look Out (BOLO) reports; resolved activity with no DoD threat or
foreign terrorist link, such as innocent photography by tourists or private citizens;
bomb threats; and other activity not related to potential international terrorists.”126
Examining the 1,131 reports that had been deleted, the Office of the Inspector
General determined that 263 of those related to protests and demonstrations. Of
those 263 reports, the Inspector General found that 157 identified some “action or
event that took place” of which 75 involved “criminal actions . . . that resulted in
arrests, required court appearances, violence, destruction, and required police
intervention.”127 The Inspector General took these numbers as an indication that
“creating TALON reports to inform local commanders of protests and
demonstrations planned for their vicinity appears to be justified”.128 Personal
information contained in reports concerning protests or demonstrations included
“names of individuals and organizations, phone numbers, addresses, e-mail
addresses and websites associated with the protestors.”129 It is unclear whether
TALON reports were shared with local law enforcement, but at least one report
suggests that they were used to alert local law enforcement of potential threats.130
119
Ibid., p. 1.
Ibid., p. 2.
121
See, e.g., Walter Pincus (2005, December 15). Pentagon Will Review Database on US Citizens.
Washington Post, A01.
122
Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice
(TALON) Report Program (Report No. 07-1 NTEL-09), p. i.
123
Ibid., p. 26.
124
Ibid., p. 5.
125
Ibid., p. 10.
126
Ibid.
127
Ibid.
128
Ibid.
129
Ibid., p. 25.
130
See Robert Block & Jay Solomon (2006, April 27). Pentagon Steps Up Intelligence Efforts Inside
120
24
As for TALON reports held in the Northern Command’s JPEN, the Inspector General
discovered that all TALON reports had been deleted from the JPEN in November
2005 and the maintenance of the entire system was terminated in June 2006.131
Thus, although the the Office of the Inspector General was able to determine that
the CFIA had failed to comply with Department of Defense retention regulations
which required that information pertaining to non-DoD US persons and
organizations be deleted within 90 days unless retention was required by law or
authorized by the Secretary of Defense, it was unable to determine if the Northern
Command had complied with respect to the TALON reports it maintained until
November 2005.132
Significant changes were introduced in 2006, probably due to the increased
attention that the system came under both from within and without the Department
of Defense. The Under Secretary of Defense for Intelligence issued a memorandum
on 2 February 2006 which shifted the function of the reporting system. The memo
indicated that TALON would no longer be considered primarily a “law enforcement”
database, but rather a “counterintelligence” database.133 This meant that different
DoD regulations would be implicated with respect to the handling of data concerning
US persons. On 30 March 2006, a memorandum from the Deputy Secretary of
Defense stipulated that TALON “should be used only to report information regarding
possible international terrorist activity”.134 By April 2006, changes had also been
made to the technical aspects of the Cornerstone system. As a result, a select group
of CIFA analysts was provided with the ability to edit US person information located
anywhere within any TALON report. Additionally, the system provided alerts on any
reports which required review within 90 days of entry and also implemented a
tracking mechanism for edits to US person information.135 The CIFA also introduced
a process of review before reports were entered into the Cornerstone database. As
a result, the CIFA began to reject a substantial number of newly incoming reports
that had been filed.136
On 21 August 2007, it was announced that the TALON system would be
discontinued. The DoD press release indicated that there were plans to introduce a
US Borders. Wall Street Journal, A1, A14, available at
http://www.umaryland.edu/healthsecurity/related/1%20Pentagon%20Steps%20Up%20Intelligence%20
Efforts%20Inside%20USpdf.
131
Ibid., p. 8.
132
Ibid., p. 5; see also DoD Directive 5200.27, Acquisition of Information Concerning Persons and
Organizations not Affiliated with the Department of Defense, 07.01.1980.
133
See, Under Secretary of Defense, Memorandum for Director, Counterintelligence Field Activity, 2
Feb. 2006, available at http://www.gwu.edu/~nsarchiv/NSAEBB/NSAEBB230/11.pdf; see also Dept.
of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice (TALON)
Report Program (Report No. 07-1 NTEL-09, p. 9.
134
Deputy Secretary of Defense, Memorandum for Secretaries of the Military Departments et al., 30
Mar. 2006, reprinted in Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local
Observation Notice (TALON) Report Program (Report No. 07-1 NTEL-09, p. 32.
135
Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice
(TALON) Report Program (Report No. 07-1 NTEL-09, p. 9.
136
Dept. of Defense, Inspector General. (19.06.2007). The Threat and Local Observation Notice
(TALON) Report Program (Report No. 07-1 NTEL-09, p. 9; and in particular n. 8.
25
better system for threat reporting and that existing TALON reports would be sent to
the FBI for incorporation in its Guardian database.137 In news reports, it was
suggested that TALON was being shutting down because “‘the analytical value had
declined’” rather than due to public complaints.138
2.2.28. TIDE (Datamart)
Agency:
Data Entry:
Department of Homeland Security
Department of Homeland Security, National Counterterrorism Center,
although entries may be suggested by “federal agencies”139
Maintenance: Department of Homeland Security, National Counterterrorism Center
Access:
Unknown
Sharing:
Some entries shared with FBI for addition to no-fly lists
Status: In operation
TIDE represents a central database containing the names of individuals who have
been connected with terrorist activity. The database, however, does not include
“purely domestic terrorism information.”140 The database is maintained by an office
known as the Terrorist Identities Group under the auspices of the National
Counterterrorism Center (DHS). A subset of the data is submitted to the FBI to be
added to the national watchlists such as the no-fly list.141 Among the activities that
have been identified as triggering inclusion in the database are: committing
international terrorist activity; preparing or planning international terrorist activity;
gathering information on potential targets for international terrorist activity;
soliciting funds or other things of value for international terrorist activity or for a
terrorist organization; soliciting membership in an international terrorist
organization; providing material support to international terrorists or to advance
international terrorist activity; being a member or representative of an international
terrorist organization.142 In January, 2009, the database contained approximately
564,000 entries which corresponded to about 500,000 individuals. US persons
(including legal permanent residents) were believed to make up about 5% of the
database at that time.143
2.2.29. FBI Intelligence Community Data Marts
137
Department of Defense.( 21.08.2007). DoD to Implement Interim Threat Reporting Procedures.
Retrieved November 18, 2009, from
http://www.defenselink.mil/releases/release.aspx?releaseid=11251, also available at
http://www.gwu.edu/~nsarchiv/NSAEBB/NSAEBB230/18.pdf.
138
Associated Press (2007, Aug. 21). Pentagon to shut down controversial database. MSNBC,
http://www.msnbc.msn.com/id/20375361/.
139
National Counterterrorism Center. (2009). Terrorist Identities Datamart Environment (TIDE).
Retrieved September 03, 2009, from http://www.nctc.gov/docs/Tide_Fact_Sheet.pdf.
140
Ibid.
141
Ibid.
142
Ibid.
143
Ibid.
26
Agency:
Federal Bureau of Investigation
Data Entry: Unspecified
Maintenance: Federal Bureau of Investigation
Access:
Federal Bureau of Investigation analysts & other members of
intelligence community
Sharing:
Unspecified intelligence agencies
Status: Under development
These data marts were planned as a means to isolate certain portions of FBI
databases for sharing with other intelligence agencies.144
2.2.30. Investigative Data Warehouse (IDW)
Agency:
Federal Bureau of Investigation
Data Entry: Various
Maintenance: Federal Bureau of Investigation & Private Contractors
Access:
Federal Bureau of Investigation analysts & contractors (different levels
of access for different personnel)
Sharing:
Unknown
Status: In operation
The Investigative Data Warehouse represents a platform that provides access to
numerous databases through a single interface—essentially a virtual repository.145
The platform incorporates search and analysis tools. In 2003, one goal of the
development of the IDW was to enable mining of the data within the various data
sources to which the platform had access.146 In 2005, the Chief Information Officer
of the FBI indicated that the system was able to read data from “more than 47
sources of counterterrorism data, including information from FBI files, other
government agency data, and open source news feeds, that were previously
available only through separate, stove-piped systems.”147 Among the data sources
that were available to the system in 2004, according to the Electronic Frontier
Foundation, were the FBI’s digital case management system—the Automated Case
System (ACS)—copies of certain messages and documents circulated between the
FBI and various other federal agencies, a database of individuals the FBI associated
with violent gang or terrorist activity which included biographical information and
photos, structured data derived from a set of online newspapers from around the
world, the names of individuals on the TSA’s “selectee” and no-fly lists, names,
aliases, and biographical information associated with individuals on the FBI’s
144
Ibid., p. 48.
Chiliad. (2006). Chiliad Success Story: Federal Bureau of Investigation. Retrieved November 23,
2009, from http://www.chiliad.com/docs/ChiliadCaseStudy_FBI.pdf, p. 2.
146
Federal Bureau of Investigation. (26.03.2004). Description of the IDW Project., available at
http://www.eff.org/files/filenode/foia_idw/20080408_idw01-Project-Description.pdf.
147
Zalmai Azmi (2005, January 26). Re: DRAFT AUDIT REPORT -THE FEDERAL BUREAU OF
INVESTIGATION'S MANAGEMENT OF THE TRILOGY INFORMATION TECHNOLOGY
MODERNIZATION PROJECT (Letter), reprinted in: Office of the Inspector General, Audit Report
No. 05-07, Appendix 7.
145
27
Terrorist Screening Database, databases of the names of individuals who were the
subject of an FBI investigation as well as those individuals referred to in an FBI case
file, several databases containing scanned documents from FBI terrorist files, files
related to terrorist financing including the Financial Crimes Enforcement Network
(FinCen) Databases, and databases containing biographical data supplied by foreign
financial institutions on individuals suspected of having connections with terrorist
financing, the State Department’s list of lost and stolen passports as well as
documents from passport fraud investigations.148
With regard to the analytic features of the system, a 2007 report indicated that the
IDW made use of commercial off-the-shelf products as well as open source
applications and scripting languages.149 The reference to scripting languages
suggests that in-house IT specialists may be able to introduce custom features on an
ad hoc basis.150 Initial plans for IDW used images from i2’s Analyst Notebook to
illustrate the analytic tools that IDW would incorporate,151 suggesting that Analyst
Notebook might be one off-the-shelf product that is being used to provide data
mining functions for the IDW. Reports also indicate that IDW includes automatic
notifications to alert interested parties when new information of relevance to their
case needs becomes available.152 Flexible search functions were designed to
alleviate issues arising from misspellings, alternative spellings, and various
nomenclatures for rendering dates.153 The FBI cited remarkable reductions in
processing time resulting from the implementation of the system. One official for
instance suggested that the time to complete certain tasks had been reduced from
32,000 hours to ½ hour.154
2.3. European Developments
2.3.1. Creation of European Terrorist Profiles
On 18 November 2002, the Article 36 Committee of the European Union submitted a
draft Council Decision which would establish terrorist profiles to be used in European
148
Electronic Frontier Foundation (April 2009). Report on the Investigative Data Warehouse,
http://www.eff.org/issues/foia/investigative-data-warehouse-report; see also Federal Bureau of
Investigation (22.04.2004). Investigative Data Warehouse Integration System (IDW-I) System Security
Plan, Version 0.6, available at https://www.eff.org/files/filenode/foia_idw/20080508_idw01.pdf.
149
Office of the Inspector General. (August 2007). Audit of the Department of Justice Information
Technology Studies, Plans, and Evaluations (Audit Report 07-39), Appendix 6, available at
http://www.justice.gov/oig/reports/plus/a0739/app6.htm.
150
Scripting languages represent computer code which runs on top of another application. Java, for
instance, is a common scripting language that can be used to execute commands within a browser
environment such as Microsoft’s Internet Explorer or Mozilla’s Firefox.
151
See Federal Bureau of Investigation (14.04.2004). The FBI’s Counterterrorism Program Since
September 2001, available at http://www.fbi.gov/publications/commission/9-11commissionrep.pdf,
pp.55-56.
152
Ibid., p. 54; Chiliad. (2006). Chiliad Success Story: Federal Bureau of Investigation. Retrieved
November 23, 2009, from http://www.chiliad.com/docs/ChiliadCaseStudy_FBI.pdf, p. 3.
153
Chiliad. (2006). Chiliad Success Story: Federal Bureau of Investigation. Retrieved November 23,
2009, from http://www.chiliad.com/docs/ChiliadCaseStudy_FBI.pdf, p. 3
154
Ibid.
28
counter-terrorism efforts. The document foresaw that the Member States would
exchange information among themselves and with Europol and cooperate to
develop profiles.155 The Committee defined the creation of terrorist profiles as
involving “putting together a set of physical, psychological or behavioural variables,
which have been identified, as typical of persons involved in terrorist activities and
which may have some predictive value in that respect.”156 It also listed suggested
variables which Europol and the Member States might consider including in the
terrorist profiles. Those variables were:
 nationality
 travel document
 method and means of travel
 age
 sex
 physical distinguishing features (e.g. battle scars)
 education
 choice of cover identity
 use of techniques to prevent discovery or counter questioning
 places of stay
 methods of communication
 place of birth
 psycho-sociological features
 family situation
 expertise in advanced technologies
 skills at using non-conventional weapons (CBRN)
 attendance at training courses in paramilitary, flying and other specialist
techniques157
The profiles would not be set in stone but rather be subject to change whenever
terrorists changed their operational methods.158
This proposal came under criticism from the European Parliament’s Committee on
Civil Liberties, Justice and Home Affairs.159 In 2008, the Committee issued a draft
report on profiling which included a number of recommendations for the Council,
including that “the collection of data and use of profiling techniques in respect of
persons not suspected of a specific crime or threat must be subject to a particularly
strict “necessity” and “proportionality” test” and that “reliance by private or public
bodies on computers to take decisions on individuals without human assessment
should only be allowed exceptionally under strict safeguards”.160
Article 36 Committee, “I/A Item Note” to COREPR/ Council, 11858/3/02 REV 3 (Annex),
18.11.2002, p. 3.
156
Ibid., p. 5.
157
Ibid., p. 7.
158
Ibid., p. 6.
159
See, e.g., Sarah Ludford, WRITTEN QUESTION P-3694/03 reprinted in: DRAFT REPLY TO
WRITTEN QUESTION P-3694/03 put by Sarah LUDFORD on 5 December 2003, 7846/04, p. 2.
160
European Parliament, Committee on Civil Liberties, Justice and Home Affairs, DRAFT REPORT
with a proposal for a European Parliament recommendation to the Council on the problem of profiling,
notably on the basis of ethnicity and race, in counterterrorism, law enforcement, immigration, customs
and border control, 2008/2020(INI), 12.02.2008, p. 10.
155
29
2.3.2. European Passenger Name Records System
The European Council has expressed interest in developing a European passenger
name records (PNR) system for law enforcement purposes and is working toward
drafting a Framework Decision on the matter.161 In describing the utility of such
systems, the language of the Council is reminiscent of reports on data mining
programmes in the US. They state that PNR data, “gives access to specific
information about offenders' behaviour, such as the itineraries for and frequency of
their journeys, the circumstances in which their plane tickets are bought (travel
agency, means of payment, credit card details, group purchases, etc.) and other
matters connected with the trip (hotel reservation, car hire etc). It makes it possible
to detect offences because of suspicious behaviour, to find those suspected of
crimes, to reveal links between a person and a known criminal, or links between a
person and a particular criminal case.”162 There has been resistance, however, to
the idea of having a centralized system,163 and there is thus some likelihood that an
EU PNR system would be distributed. Additionally, the Council has acknowledged
the need to provide independent oversight and implement auditing capabilities.164
Issues concerning the appropriateness of the inclusion of sensitive data, the proper
period of retention of PNR data, and the scope of exchange of PNR data have also
been raised.165
2.3.3. European Security Research
The Seventh Framework Programme is primarily concerned with the development of
security technologies. Of the current FP7 projects, only one explicitly mentions data
mining in its project description. The ODYSSEY project aims to develop systems to
analyse ballistics data and provide alerts to assist law enforcement in addressing
organized crime.166 The project will reportedly not make use of any personal data,
but does contemplate the demonstration of “how migration to other data sources
can take place.”167 Additionally, three of the current FP7 Security programmes are
concerned with detecting “abnormal” or threatening behaviour: INDECT,168
161
See European Council, Proposal for a Council Framework Decision on the use of Passenger Name
Records (PNR) for law enforcement purposes - Report on thematic work carried out from July to
November 2008, 15319/0, 28.11.2008.
162
Ibid., p. 7.
163
Ibid., p. 9.
164
See ibid., pp. 15-16.
165
See ibid., p. 17.
166
See European Commission, Towards a more secure society and increased industrial
competitiveness, available at ftp://ftp.cordis.europa.eu/pub/fp7/security/docs/towards-a-moresecure_en.pdf, p. 62.
167
Strategic pan-European ballistics intelligence platform for combating organised crime and terrorism
(ODYSSEY),
http://cordis.europa.eu/fetch?CALLER=FP7_PROJ_EN&ACTION=D&DOC=12&CAT=PROJ&QUE
RY=01257560c1da:b844:4f90acbf&RCN=89324.
168
indect Homepage, http://www.indect-project.eu/.
30
SAMURAI,169 and ADABTS.170 As seen in several US-based programmes, this kind of
event detection may rely on data mining algorithms.
2.4. Germany
2.4.1. Terrorist Rasterfahndung
Agency:
Data Sources:
Access:
Functions:
Status:
Federal Criminal Police Office (Bundeskriminalamt)
Various public and private databases
Unknown
Search
Terminated
The system of methods known as the Rasterfahndung was reportedly first used in
connection with investigations concerning the Rote Armee Faktion in the 1970s.171
The method applied in that instance consisted of determining a set of characteristics
which were believed to match the persons sought. These characteristics were then
applied to search through public or private databases to filter out all individuals
except those who fulfilled the set of characteristics.172 The original implementation
ultimately revealed only one RAF conspirator, who was taken into custody as a
result, as well as another individual who was reportedly engaged in drug-dealing.173
Both Germany and Austria passed national laws which enabled the use of data
processing methods on personal data for criminal law enforcement purposes.174 The
law in Germany limited the use to law enforcement efforts targeting illegal drug
trade and organised crime.175 Reportedly, however, more far-reaching enabling laws
and regulations were introduced at the local level of the German Länder.176
Following the September 11 attacks, the Bundeskriminalamt organized a nation-wide
implementation of a Rasterfahndung which sought to turn up the names of males
between the ages of 18 and 40 who were from certain Islamic states and were either
169
Suspicious and abnormal behaviour monitoring using a network of cameras & sensors for situation
awareness enhancement,
http://cordis.europa.eu/fetch?CALLER=FP7_SECURITY_PROJ_EN&ACTION=D&DOC=29&CAT=
PROJ&QUERY=012572cd757e:70ce:7aa94bb4&RCN=89343.
170
Automatic Detection of Abnormal Behaviour and Threats in crowded Spaces,
http://cordis.europa.eu/fetch?CALLER=FP7_SECURITY_PROJ_EN&ACTION=D&DOC=3&CAT=
PROJ&QUERY=012572cd757e:70ce:7aa94bb4&RCN=91158.
171
1 BvR 518/02, para. 3.
172
See, e.g., (2007.18.09). Rasterfahndung: nur bedingt effektiv. heise online,
http://www.heise.de/newsticker/Rasterfahndung-Nur-bedingt-effektiv--/meldung/96183.
173
“Rasterfahndung”, Wikipedia, http://de.wikipedia.org/wiki/Rasterfahndung (citing Die Position der
RAF hat sich verbessert, Der Spiegel 37/1986 (08.09.1986), pp. 38-61; see also Kett-Straub, G.
Rasterfahndung fällt durch das Raster des Grundgesetzes, ZIS 9/2006, p. 2, available at
http://www.zis-online.com/dat/artikel/2006_9_69.pdf.
174
1 BvR 518/02, para. 4, “Rasterfahndung”, Wikipedia, http://de.wikipedia.org/wiki/Rasterfahndung.
175
1 BvR 518/02, para. 4.
176
Ibid., para. 6.
31
current or former students. The aim was to uncover “sleepers” who were somehow
involved in terrorist activity or planning.177 The data processing was carried out by
the local police agencies with the results offered up to the federal
Bundeskriminalamt.178 Ultimately, upwards of around 300,000 individuals were
singled out,179 although reportedly the action did not result in any prosecutions of
terrorists.
This use of a Rasterfahndung became the subject of a controversy before the
German Constitutional Court. The decision of the Court, rendered in 2006, found
that the implementation of the Rasterfahndung violated the subjects’ rights to
informational self-determination180—a relatively novel concept which has been
derived as an implicit right stemming from other explicit basic rights provided in the
German Constitution (Grundgesetz). The Court held that such an action could only
be justified in the face of a tangible danger to high-ranking legal interests. A general
sense of a heightened threat level in the wake of 9/11 was insufficient to justify the
use of the methods.181
2.4.2. Case of Andrej Holm
Agency:
Bundeskriminalamt
Data Sources:World Wide Web
Access:
Unknown
Functions:
Search
Status:
Ended
According to news reports, the Bundeskriminalamt conducted ordinary web searches
in an investigation concerning a group known as the “militante gruppe” (“militant
group”) and was lead to link the urban sociologist Andrej Holm to the group.182
Allegedly, a search for selected key words such as “Gentrification” and
“Prekarisierung”183 turned up links to the writings of Prof. Holm.184 One news
correspondent suggested that the similarity in the use of words in writings by
members of the militante gruppe and Prof. Holm was thus the basis of initial
suspicion.185 This suspicion then allegedly led to nearly a year-long period of
177
Ibid., para. 7.
Ibid., para. 9.
179
Ibid.
180
Ibid., para. 66.
181
Ibid., paras. 160 et seq.
182
(22.08.2007). Kommissar Google jagt Terroristen, Tageszeitung,
http://www.taz.de/index.php?id=start&art=3471&id=deutschland-artikel&cHash=5218eee73a.
183
Describes the process through which the number of secure, quality, and legal jobs are pushed out by
less secure and even illegal work. “Prekarisierung”, Wikipedia,
http://de.wikipedia.org/wiki/Prekarisierung.
184
(22.08.2007). Kommissar Google jagt Terroristen, Tageszeitung,
http://www.taz.de/index.php?id=start&art=3471&id=deutschland-artikel&cHash=5218eee73a.
185
Robert Siegel (21.08.2007). Professor's Research Results in Terrorism Charges, NPR All Things
Considered, 2007 WLNR 16296717.
178
32
surveillance.186 Within this period, Holm was observed meeting with at least one
member of the militante gruppe on more than one occasion,187 and suspicion grew
when it was observed that Holm did not bring his mobile phone with him to one of
these meetings—a sign the authorities took to mean that he was attempting to
avoid detection.188 Holm was subsequently arrested but the arrest order was
ultimately lifted by the Bundesgerichtshof, Germany’s highest court, since there was
insufficient evidence to suggest that Holm was a member of the group.189
3. Programmes of Potential Future Interest
The following programmes may warrant inclusion in future drafts of this survey:
3.1. REVEAL (US)
3.2. SCION (US)
3.3. National Security Branch Analysis Center (US)
3.4. Guardian (US)
3.5. Eurodac (EU)
3.6. Schengen Information System II (EU)
3.7. Europol Information System (EU)
3.8. Visa Information System (EU)
3.9. EDVIGE/ EDVIPR (FR)
3.10.
CHRISTINA (FR)
3.11.
Project Rich Picture (UK)
3.12.
National Public Order Intelligence Unit Database (UK)
186
(22.08.2007). Kommissar Google jagt Terroristen, Tageszeitung,
http://www.taz.de/index.php?id=start&art=3471&id=deutschland-artikel&cHash=5218eee73a.
187
Compare (22.08.2007). Kommissar Google jagt Terroristen, Tageszeitung,
http://www.taz.de/index.php?id=start&art=3471&id=deutschland-artikel&cHash=5218eee73a and
Robert Siegel (21.08.2007). Professor's Research Results in Terrorism Charges, NPR All Things
Considered, 2007 WLNR 16296717.
188
Robert Siegel (21.08.2007). Professor's Research Results in Terrorism Charges, NPR All Things
Considered, 2007 WLNR 16296717.
189
See (24.10.2007). Haftbefehl gegen Berliner Soziologen aufgehoben, 154/2007 (Press Release),
available at http://juris.bundesgerichtshof.de/cgibin/rechtsprechung/document.py?Gericht=bgh&Art=pm&Datum=2007&Sort=3&nr=41477&pos=0&a
nz=154.
33