--
--
Computer Emergency Response - An International Problem †
R ichard D. Pethia
K enneth R . van W yk
C om puter Em ergency Response Team / Coordination Center
Software Engineering Institute
Carnegie Mellon University
4500 Fifth Avenue
P ittsburgh, PA 15213-3890
U.S.A .
A BSTR ACT
Com puter security incidents during the past few years have illustrated that
u nau thorized com puter activity does not obey traditional boundaries ( e.g.,
national, network, com puter architecture) . Instead, su ch activity frequently
crosses these boundaries not just once, but several tim es per incident [Stoll89].
International cooperation am ong
e ffective m ean s of dealing with
com puter user com m unity. This
and su ggests m ethods by which
can work together internationally
com puter security response groups can be an
com puter security issu es faced today by the
paper addresses the need for su ch cooperation
individual com puter security response groups
to cope with com puter security incidents.
1. Backgroun d
T he increasing use an d dependence on interconnected local, regional, an d wide area networks,
while bringing im portan t new capabilities, also brings new vulnerabilities. Widely publicized
e vents su ch as the Novem ber 1988, Internet Worm , which affected thousands of system s on
the international research network Internet, or the October 1989, WANK worm , which affecte d
h undreds of system s on NASA’s Space Physics an d Analysis ( SPAN) network are unusu al,
although dram atic, events. There are m an y m ore events su ch as intrusions, exploitation of vuln erabilities, an d discovery of new vulnerabilities that occur with m uch greater frequency an d
require effective m ethods of response. Several exam ples are listed below.
1 .1. Incidents
From August 1986 until late 1987, staff m em bers at Lawrence Berkeley Laboratory worked with
investigators to trace the paths of a com puter intruder; the trail eventually lead them to a
K G B-funded intruder operating out of Han nover, G erm an y [Stoll88]. The investigation was
often ham pered by a lack of cooperation am ong "bureau cratic organ izations" [Stoll88]. On the
o ther han d, "cooperation between system m an agers, com m unications technicians, an d network
operators was excellent" [Stoll88]. Still, it was only when the investigators in both countries go t
involved that the intruder was apprehended [Stoll89]. It is worthwhile to note that the breakins in this case utilized the sam e attack m ethods over an d over ( su ch as repeatedly guessing
com m on an d system defau lt usernam e/ password com binations, exploiting well known security
holes which had not yet been fixed by the system adm inistrators, etc.) ; through diligent ,
m ethodical application of these m ethods, the intruders were su ccessful at entering dozens of
hhhhhhhhhhhhhhhhhh
‡ Sponsored by the U.S. D epartm ent of D efense
--
--
-2com puter system s [Stoll89].
I n Novem ber 1988, a rogue worm program entered the Internet an d cau sed widespread system
failures [Spafford88]. The worm , written by Cornell University graduate student Robert Tap p an Morris, Jr. [Markoff90a], exploited lax password policies as well as two software im plem entation errors in specific versions of UNIX, the predom inan t operating system on Internet com p uters.
More recently, three Australian com puter intruders were arrested by Australian Federal police
"after a two-year investigation that included cooperation with United States au thorities"
[ Markoff90b]. Again , the intruders exploited known vulnerabilities to gain unau thorized entry
onto system s [D an ca90].
I n October 1989, a worm program called Worm s Again st Nuclear Killers ( WANK) infected a
National Aeronau tics an d Space Adm inistration ( NASA) network [Alexan der89]. The worm
p rogram spread to m an y com puters in different countries by using system vulnerabilities that
"sh ould have been closed m onths ago" following a sim ilar incident in D ecem ber 1988 [Alexander89].
In an other, albeit dom estic, case, two com puter intruders were arrested an d charged with illegal
use of com puter system s at Pennsylvan ia State University. The intrusions took place on a com p uter system at the University of Chicago. University of Chicago officials contacted CERT/ CC,
which then contacted adm inistrators at Penn State. Eventually, through the cooperation of the
adm inistrators an d investigators, the two Penn State students were charged [G raf90].
These cases all illustrate the need for cooperation am ong com puter security response groups.
1 .2. System Vulnerabilities
Another situation in which cooperation across m ultiple organ izations becom es essential is in
dissem ination of system vulnerability alerts ( an d, m ore im portan tly, their solutions) . As sys t em intruders su ccessfully gain access to system s which have weak passwords or system s where
known security vulnerabilities have not been closed, they often sh are inform ation on vulnerab ilities in these system s with others. Likewise, as intruders discover new vulnerabilities in particular operating system or other software packages, inform ation on the vulnerabilities is quickly
com m unicated through various bulletin boards an d other electronic forum s.
As a resu lt, m an y large com m unities of system users quickly becom e vulnerable. Traditional
m ethods of dealing with vulnerability inform ation, including closely protecting inform ation on
the existence of the vulnerability, are not effective once intruders have learned of system
weaknesses. In these cases, su pplying password guideline an d security vulnerability inform ation
to system adm inistrators is crucial in raising security levels an d deterring attacks.
T he Com puter Em ergency Response Team Coordination Center ( CERT/ CC) ( see Section 2.1)
frequently distributes CERT Advisories that, am ong other things, inform the public of vulnerab ilities, fixes, an d active m ethods of attack.
2. Emergency Res ponse Groups
2 .1. Introduction to CERT
Shortly after the Internet worm of Novem ber 1988 [Spafford88], the D efense Advan ced
Research Projects Agency ( D ARPA) started the Com puter Em ergency Response Team
( CERT) , whose Coordination Center ( CERT/ CC) is located at Carnegie Mellon University’s
Software Engineering Institute ( SEI) [Scherlis88]. "The CERT is a com m unity group intended
t o facilitate com m unity response to com puter security events involving Internet hosts" [D enning90]. CERT consists of hundreds of highly qualified volunteers throughout the com puter
com m unity, as well as the staff of the CERT/ CC an d of the other em ergency response groups
in the CERT-System ( see Section 2.2 for details) . The CERT/ CC serves as a focal point fo r
r esponse to Internet com puter security problem s [D enning90]. Since it would be im possible for
an y one response group to address the needs of all constituencies‡, the need for m ultiple CERT
hhhhhhhhhhhhhhhhhh
‡ The term "constituency" is used here to defin e a group with som e com m on needs.
--
--
-3groups exists. ( This issu e, too, is covered in m ore detail in Section 2.2. )
C ERT groups m ust have su fficient in-house technical expertise to han dle a reasonable portion
of day to day security incidents, leaving the volunteer contacts for situations which requir e
additional expertise. However, becau se em ergency response involves addressing m ore than just
technical issu es, CERT m em bersh ip includes not only technical experts, but site m an agers,
security officers, industry representatives, an d governm ent officials [D enning90].
On e of the essential characteristics of a CERT group is being available to its constituency on a
2 4 hour per day basis. There m ust be a well publicized central point of contact which is available continuously. This sh ould include, at a m inim um , a "hotline" telephone which is con stan tly m an ned, an d an electronic m ailbox which is m onitored during business hours. The
CERT/ CC hotline num ber is ( 412) 268-7090, an d its electronic m ailbox address is
cert@cert.sei.cm u.edu, on the Internet.
It is critical that a CERT group build an d m ain tain a collection of contacts, both within the
group’s constituency an d externally [D alton90]. The contact inform ation sh ould include othe r
C ERT groups, system vendors, law enforcem ent, network operation centers, technical experts,
site adm inistrators, etc. Building the contact inform ation is an on-going process in which cont acts are developed an d m ain tain ed over tim e. Each contact m ust be aware of its responsibilities an d/ or expectations in the em ergency response process [D alton90].
I n addition to the contact inform ation, a CERT group sh ould m ain tain an inform ation repository which will be drawn upon in future incidents. The inform ation in the repository will
include contact inform ation ( as detailed above) , system vulnerability details, security incident
reports, electronic m ail archives, an d other relevan t inform ation [D enning90]. D ue to the
n ature of this inform ation, the security on the system on which it resides m ust be beyond
reproach . CERT/ CC m ain tain s its inform ation database on an off-line system , which is no t
accessible via network connections.
As system vulnerabilities ( an d their fixes) , break-in warning inform ation, an d other relevan t
inform ation becom es available, CERT groups sh ould issu e advisories to m em bers of their constituency [D enning90]. Past CERT/ CC advisories have included vulnerability notification
( along with appropriate solutions) , warnings of widespread break-ins an d sym ptom s thereof ,
and secure system adm inistration su ggestions. The entire collection of CERT/ CC advisories
are m ain tain ed on-line an d are accessible to CERT/ CC constituents.
2 .1.1. Example CERT Incident H andling Procedures
As an ongoing process, CERT/ CC has developed an d is continuing to im prove upon its even t
h an dling procedures. Naturally, the procedures are different for each distinct type of event
( e.g., system break-in, vulnerability report, worm ) . This section presents an overview of som e
o f these procedures.
When CERT/ CC receives a report of a system break-in, it first works together with the affected
system adm inistrator( s) in determ ining how the intruder gain ed access to the system . This is
generally in the form of offering guidan ce on what sort of signs the adm inistrator sh ould look
for to determ ine m ean s of access. Next, CERT/ CC offers assistan ce in repairing the exploite d
h ole( s) , as well as other com m only known vulnerabilities. Exam ining system s for backdoors or
trojan horses that have been plan ted by the intruder is an especially im portan t activity. If the
b reak-in cam e from other sites, or if the intruder broke into other system s from the current
system , CERT/ CC notifies other affected site adm inistrators ( from tim e to tim e, the adm inist rator will already have contacted other affected sites; in su ch a case, CERT/ CC requests to be
kept up to date with the relevan t flow of inform ation between the sites) . In som e cases, othe r
affected, or potentially affected, sites are not Internet sites. In these cases, com m unication
across traditional "territorial" boundaries is especially im portan t. It is im portan t to note that ,
when contacting sites, CERT/ CC always m ain tain s the confidentiality of the affected sites
unless the sites specify otherwise.
--
--
-4A s system vulnerabilities are reported to CERT/ CC, they are first au thenticated an d then
reported to the affected vendor( s) . CERT/ CC offers guidan ce to the vendor com m unity b y
r eporting the m agn itude of the threat ( e.g., whether the hole is being actively exploited,
whether the hole is known to a widespread au dience, whether the hole can be exploited from a
r em ote system or requires existing system access in order to be exploited) . CERT/ CC also
offers technical input, if desired by the vendors. The vendor com m unity will generally respon d
with a fix, a workaround, or a reference to sam e for the problem . In m an y cases, the
CERT/ CC has received advan ced versions of fixes from vendors an d has received the vendor’s
authorization to release the fix to selected m em bers of the technical com m unity for review an d
com m ent. This technical review process sh ows prom ise of im proving the quality of correction s
t o vulnerabilities.
D epending upon the situation, CERT/ CC then drafts an advisory for review by the vendors,
the CERT-System , an d/ or technical affiliates. When the draft advisory is m utually accepted, it
is distributed electronically to CERT/ CC’s constituency, the Internet research com m unity. For
this, CERT/ CC operates a CERT Advisory m ailing list, in addition to a Usenet newsgroup ,
com p.security.an nounce [Qu arterm an 90]. ( See Appendix 1 for an exam ple CERT/ CC
advisory.)
2 .2. CERT-System
As m entioned in Section 2.1, no single em ergency response group can be expected to address
the needs of every portion of the com puting world, due to the diversities an d scale of all of the
v arious com puting environm ents [D enning90]. Individual com m unities each have their own
distinct policies, rules, regulations, procedures, an d culture. Methods effective in one com m unity ( e.g., the Internet research com m unity) would not likely su cceed in other com m unities
that have significan tly different cultures ( e.g., the m ilitary com m unity or the ban king com m un ity) .
In addition, im plem entation platform s ( operating system s, networking software an d protocols)
vary widely. A single CERT group would not likely be su ccessful in dealing with technica l
d iversity, or at least could not do so econom ically.
The "CERT-System " m odel, therefore, includes m ultiple, cooperating individual CERT organ izations. Each individual CERT group in the CERT-System focuses on a particular constituency.
Each constituency in the m odel can be defin ed by either user or technology boundaries. The
u ser constituencies consist of groups with com m on networks, needs, an d/ or policies, while the
technology constituencies are groups with com m on com puting architectures [D enning90]. An
e xam ple of a user constituency is the Internet research com m unity, which is m ade up of organ izations in academ ia, governm ent, m ilitary, as well as com m ercial groups. These groups are
b ound together by being m em bers of the Internet network. An exam ple technology constituency is the IBM m ain fram e com m unity, which is bound by a com m on com puter architecture.
T he CERT/ CC group addresses both a user constituency ( Internet research com m unity) an d a
technology constituency ( UNIX-based workstations an d m ain fram es, which is the prim ary type
o f system on the Internet) .
The CERT m odel lends itself well to network groups su ch as the Internet research com m unity,
as well as corporate [Fedeli90], governm ent, m ilitary, etc., groups.
I n tim es of crisis, m an y CERT groups can be active with a technology coordination center
an alyzing problem s an d coordinating the search for solutions an d with user constituency coordin ation centers gathering inform ation an d inform ing their constituents as appropriate.
In less troubled tim es, the CERTs work together to build effective com m unication m echan ism s,
share inform ation on effective com puter security tools an d techniques, an d conduct proactive
cam paign s aim ed at increasing the awareness of com puter security issu es an d im proving th e
security of operational system s.
The CERT-System m odel has been widely accepted an d eleven groups funded by U.S. governm ent agencies an d several private firm s now participate in the system . Interest in participating
--
--
-5h as been expressed by several other organ izations an d steps are being taken to m ore form ally
structure the CERT-System . This structure, including a charter an d by-laws that are bein g
r eviewed an d approved by existing CERT-System m em bers as this paper is being written, will
provide a fram ework to enable wider participation.
3 . Conclus ions
It has been sh own that the CERT concept can be an effective m ean s of responding to com puter
security-related incidents [G raf90]. In incidents prior to the existence of CERT, system
adm inistrators were frequently at a loss for outside assistan ce when han dling security incidents
[Stoll88, Stoll89]. It has also been sh own that com puter system security incidents do not obey
n etwork, national, or architectural boundaries [Stoll88] an d that the intruders frequently exploit
lax security procedures ( due, perhaps, to a lack of specific knowledge on the adm inistrators’
p art) [Stoll88, D an ca90, Alexan der89].
Effective com puter security incident response requires com m unication an d coordination across
m ultiple com m unities. While m an y incidents occur becau se software design or im plem entation
d eficiencies are exploited, resolution of the incidents requires m ore than a technical solution.
Com m unication of threat an d vulnerability inform ation across com puting com m unities is essent ial to resolving specific incidents an d im proving the security of operational system s.
A well form ed CERT-System will raise security awareness an d knowledge am ong site adm inis t rators as well as give the adm inistrators sources of assistan ce in tim es of com puter em ergencies. By drawin g on the experiences of individual CERT groups, the knowledge level of the
C ERT-System as a whole will grow, enabling all m em bers to m ore effectively an d efficiently
deal with com puter security incidents as they arise.
1 . Example CERT/ CC Advisory
CA-90:02
CERT Advisory
March 19, 1990
I nternet Intruder Warning
------------------------------------------------------------------------------------- T here have been a num ber of m edia reports stem m ing from a March 19 New York Tim es article entitled ’Com puter System Intruder Plucks Passwords an d Avoids D etection.’ The article
r eferred to a program that attem pts to get into com puters around the Internet.
At this point, the Com puter Em ergency Response Team Coordination Center ( CERT/ CC) doe s
n ot have hard evidence that there is su ch a program . What we have seen are several persistent
attem pts on system s using known security vulnerabilities. All of these vulnerabilities have
b een previously reported. Som e national news agencies have referred to a ’virus’ on the Internet; the inform ation we have now indicates that this is NOT true. What we have seen an d can
confirm is an intruder m aking persistent attem pts to get into Internet system s.
It is possible that a program m ay be discovered. However, all the techniques used in these
attem pts have also been used, in the past, by intruders probing system s m an ually.
As of the m orning of March 19, we know of several system s that have been broken into an d
several dozen m ore attem pts m ade on Thursday an d Friday, March 15 an d 16.
System s adm inistrators sh ould be aware that m an y system s around the Internet m ay have these
v ulnerabilities, an d intruders know how to exploit them . To avoid security breach es in the
future, we recom m end that all system adm inistrators check for the kinds of problem s noted in
t his m essage.
The rest of this advisory describes problem s with system configu rations that we have seen
intruders using. In particular, the intruders attem pted to exploit problem s in Berkeley BSD
d erived UNIX system s an d have attacked D EC VM S system s. In the advisory below, points 1
through 12 deal with Unix, points 13 an d 14 deal with the VM S attacks.
--
--
-6If you have questions about a particular problem , please get in touch with your vendor .
T he CERT m akes copies of past advisories available via an onym ous FTP ( see the end of this
m essage) . Adm inistrators m ay wish to review these as well.
W e’ve had reports of intruders attem pting to exploit the following areas:
1) Use TFTP ( Trivial File Tran sfer Protocol) to steal password files.
To test your system for this vulnerability, connect to your system using TFTP an d try ’get
/ etc/ m otd’. If you can do this, an yone else can get your password file as well. To avoid this
p roblem , disable tftpd.
In conjunction with this, encourage your users to choose passwords that are difficult to guess
( e.g. words that are not contain ed in an y dictionary of words of an y lan guage; no proper nouns,
including nam es of "fam ous" real or im agin ary characters; no acronym s that are com m on to
com puter professionals; no sim ple variations of first or last nam es, etc.) Furtherm ore, inform
your users not to leave an y clear text usernam e/ password inform ation in files on an y system .
If an intruder can get a password file, he/ sh e will usu ally take it to an other m ach ine an d ru n
p assword guessing program s on it. These program s involve large dictionary searches an d run
quickly even on slow m ach ines. The experience of m an y sites is that m ost system s that do no t
p ut an y controls on the types of passwords used probably have at least one password that can be
guessed.
2 ) Exploit accounts without passwords or known passwords ( accounts with vendor su pplied
defau lt passwords are favorites) . Also uses fin ger to get account nam es an d then tries sim ple
p asswords.
Scan your password file for extra UID 0 accounts, accounts with no password, or new entries
in the password file. Always chan ge vendor su pplied defau lt passwords when you install new
system software.
3) Exploit holes in sendm ail .
Make su re you are running the latest sendm ail from your vendor. BSD 5.61 fixes all known
holes that the intruder is using.
4 ) Exploit bugs in old versions of FTP; exploit m is-configu red
an onym ous FTP
M ake su re you are running the m ost recent version of FTP which is the Berkeley version
4.163 of Nov. 8 1988. Check with your vendor for inform ation on configu ration upgrades.
A lso check your an onym ous FTP configu ration. It is im portan t to follow the instructions provided with the operating system to properly configu re the files available through an onym ous ftp
( e.g., file perm issions, ownersh ip, group, etc.) . Note especially that you sh ould not use your
system ’s stan dard password file as the password file for FTP.
5 ) Exploit the fin gerd hole used by the Morris Internet worm .
Make su re you’re running a recent version of fin ger. Num erous Berkeley BSD derived ver sions of UNIX were vulnerable.
Som e other things to check for:
6 ) Check user’s .rhosts files an d the / etc/ hosts.equiv files for system s outside your dom ain .
Make su re all hosts in these files are au thorized an d that the files are not world-writable.
7 ) Exam ine all the files that are run by cron an d at. We’ve seen intruders leave back doors in
files run from cron or su bm itted to at. These techniques can let the intruder back on the sys t em even after you’ve kicked him / her off. Also, verify that all files/ program s referenced
( directly or indirectly) by the cron an d at jobs, an d the job files them selves, are not world writable.
8) If your m ach ine su pports uucp, check the L.cm ds file to see if they’ve added extra com m an ds an d that it is owned by root ( not by uucp!) an d world-readable. Also, the L.sys file
should not be world-readable or world-writable.
--
--
-79) Exam ine the / usr/ lib/ aliases ( m ail alias) file for unau thorized entries. Som e alias file s
include an alias nam ed ’uudecode’; if this alias exists on your system , an d you are not explicitly
using it, then it sh ould be rem oved.
1 0) Look for hidden files ( files that start with a period an d are norm ally not sh own by ls) with
odd nam es an d/ or setuid capabilities, as these can be used to "hide" inform ation or privilege d
( setuid root) program s, including / bin/ sh . Nam es su ch as ’.. ’ ( dot dot space space) , ’...’, an d
.xx have been used, as have ordinary looking nam es su ch as ’.m ail’. Places to look includ e
e specially / tm p, / usr/ tm p, an d hidden directories ( frequently within users’ hom e directories) .
11) Check the integrity of critical system program s su ch as su , login, an d telnet. Use a known ,
good copy of the program , su ch as the original distribution m edia an d com pare it with the program you are running.
1 2) Older versions of system s often have security vulnerabilities that are well known to
intruders. On e of the best defenses against problem s is to upgrade to the latest version of you r
v endor’s system .
VM S SYSTEM ATTACKS:
1 3) The intruder exploits system defau lt passwords that have not been chan ged since installation. Make su re to chan ge all defau lt passwords when the software is installed. The intruder
also guesses sim ple user passwords. See point 1 above for su ggestions on choosing good passwords.
1 4) If the intruder gets into a system , often the program s loginout.exe an d sh ow.exe are
m odified. Check these program s against the files found in your distribution m edia.
I f you believe that your system has been com prom ised, contact CERT via telephone or em ail.
J. Pau l Holbrook
C om puter Em ergency Response Team ( CERT)
Software Engineering Institute
C arnegie Mellon University
Pittsburgh, PA 15213-3890
I nternet: cert@cert.sei.cm u.edu
Telephone: 412-268-7090 24-hour hotline: CERT personnel an swer
7:30a.m .-6:00p.m . EST, on call for em ergencies
other hours.
Past advisories an d other inform ation are available for an onym ous ftp from cert.sei.cm u.edu
( 128.237.253.5) .
R eferences
D alton90.
D alton, J., ‘‘Building a Constituency - An On going Process,’’ Proceedings, Computer Emergency R esponse Team W orkshop, 1990.
D an ca90 .
D an ca, R., ‘‘Officials Confirm Latest Attem pt to Invade Internet,’’ Federal Computer
W eek , vol. 4, no. 12, 1990.
D enning90.
D enning, P., Computers Under Attack, ACM Press, 1990.
Fedeli90.
Fedeli, A., ‘‘Form ing an d Man agin g a Response Team ,’’ Proceedings, Computer Emergency
R esponse Team W orkshop, 1990.
G raf90 .
G raf, J., ‘‘2 Charged With Illegal Com puter Use,’’ Centre Daily Times, February 17, 1990.
--
--
-8Alexan der89.
Alexan der, M., Johnson, M., ‘‘Worm Eats Holes in NASA’s D ecnet,’’ Computer W orld,
October 23, 1989.
M arkoff90a.
Markoff, J., ‘‘3 Arrests Show G lobal Threat to Com puters, ’’ New Y ork Times, April 4,
1990.
M arkoff90b.
Markoff, J., ‘‘Student Says His Error Crippled Com puters, ’’ New Y ork Times, Jan uary 19,
1990.
Quarterm an 90.
Qu arterm an , J., The M atrix: Computer Networks and Conferencing Systems W orldwide, D igital Press, 1990.
Scherlis88 .
Scherlis, W., ‘‘D ARPA Establishes Com puter Em ergency Response Team ,’’ DA R PA
Press R elease, D ecem ber 6, 1988.
Spafford88.
Spafford, E., ‘‘The Internet Worm Program : An Analysis,’’ Technical Report, Purdue
University D epartm ent of Com puter Sciences, 1988.
Stoll88 .
Stoll, C., ‘‘Stalking the Wily Hacker,’’ Communications of the ACM , vol. 31, no. 5, 1988.
Stoll89.
Stoll, C., The Cuckoo’s Egg - Tracking a Spy Through the M aze of Computer Espionage,
D oubleday, 1989.
--
--