Anomaly Detection
brief review of my prospectus
Ziba Rostamian
CS590 – Winter 2008
What I am planning to accomplish




Study Learning Finite Automaton. Focusing of
CSSR algorithm.
Choose an application of desire and test the
performance of the CSSR algorithm. (Once I
implement the algorithm I can try it for different
application and find out where it performs
better).
Study CSSR and its extensions and use it for
detecting anomaly of moving object.
Apply some modification in to the algorithm (it
depends on how I proceed).
Why this is academically interesting

Finite automaton inference has several "real
world" applications.




Electrical engineering
DFA’s have been proposed as a model of players.
Model the problem of robot trying to learn its
envirounment.
The application of PFAs (Probabilistic Finite
automaton), of which Hidden Markov Models
(HMMs) are special case, are much more
extensive.

Speech recognition and handwriting recognition

recognizing patterns in biological sequences such a
DNA and proteins
Anomaly Detection

What are anomalies/outliers?


Variants of Anomaly/Outlier Detection Problems



The set of data points that are considerably different
than the remainder of the data
Given a database D, find all the data points x  D
with anomaly scores greater than some threshold t
Given a database D, find all the data points x  D
having the top-n largest anomaly scores f(x)
Applications:

Credit card fraud detection, telecommunication fraud
detection, network intrusion detection, fault detection
Importance of Anomaly Detection
Ozone Depletion History

In 1985 three researchers (Farman, Gardinar
and Shanklin) were puzzled by data gathered
by the British Antarctic Survey showing that
ozone levels for Antarctica had dropped 10%
below normal levels

Why did the Nimbus 7 satellite, which had
instruments aboard for recording ozone levels,
not record similarly low ozone concentrations?

NASA discovered that the spring-time ''ozone
hole'' had been covered up by a computerprogram desinged to discard sudden, large
drops in ozone concentrations as ''errors''.
Anomaly detection in moving object
Example:
There are a large number of massive vessels sailing near
American coasts. It’s unrealistic to manually trace such a
enormous number of moving objects and identify the
suspicious ones. Therefore, it’s highly desirable to
develop automated tools that can evaluate the behavior
of all maritime vessels and flag the suspicious ones.
This will allow human agent to focus their monitoring more
efficiently and accurantely.
Mechanisms for Anomaly detection

Classification, which relies on training data set.



Normal
Outliers
Clustering, which performs automated grouping
without using training set.
Anticipated Challenges

Tracking moving object can generate an
enormous amount of complex data.


Example: the time and the location of a vessel might
be recorded every few seconds, and non-spatial
information such a vessel’s weight, speed, shape and
color may be included in this recording
There exists substantial complexities of
possible abnormal behavior.