An Introduction to Computer System
Security
CS-502 Operating Systems
(Notes taken from guest CS-502 lecture by
Bob Strong, April 4, 2005)
CS502 Spring 2006
Computer Security 1
Outline
• A few principles about security
• The relationship between OS services and
their implementation, and system and data
security
• The importance of the human side of the
problem
• A little fun with cryptography
CS502 Spring 2006
Computer Security 2
Reading Material
• Tanenbaum, Chapter 9
– Security and threats
– Viruses
• How to write and detect!
– Protection – implementation of security
• Silbershatz, Chapters 14-15
– Protection
– Security
CS502 Spring 2006
Computer Security 3
Part 1: OS Issues
• Why “Security”? …and Security
principles
• Authorization and the Access Control
Policy
• User Authentication
CS502 Spring 2006
Computer Security 4
Fundamental Goals for Computer Security
• Data Confidentiality
• Data Integrity
• System Availability
• Related issue – Privacy
• A legal and ethic question
• Implemented by Confidentiality goal
CS502 Spring 2006
Computer Security 5
In Business Terms
Asset
Threat
Cost
Risk
CS502 Spring 2006
Computer Security 6
Threat Analysis
• What are we trying to protect? (and why?)
• What are the vulnerabilities of those assets?
• Who might exploit a vulnerability?
– Either on purpose or by accident
• How can we prevent a specific threat?
• How much is it worth to us to prevent it?
• How much will it cost to prevent it?
CS502 Spring 2006
Computer Security 7
Threat Analysis (2)
• An Example: OCTAVE
– Operationally Critical Threat And Vulnerability
Evaluation
• http://www.cert.org/octave
• CERT (Computer Emergency Response Team)
– OCTAVE is a business-focused process
• Example:– How much effort should I spend
protecting my company’s web site?
CS502 Spring 2006
Computer Security 8
The Core Technical Problem
• Controlling access to machine and data
resources
• Controlling the way access rights are
passed from holder to holder
• person to person
• program to program
• Preventing maliciousness and errors from
subverting the controls
CS502 Spring 2006
Computer Security 9
Access Rights
• In general case, need triplet for every
possible combination of right, protected
asset, and potential user
 (U, A, R)
• …and some entity must be responsible for
checking and enforcing any limitation...
• The 3-D matrix is hard to manage…We
need a simpler approach!
CS502 Spring 2006
Computer Security 10
Textbook 2D Matrix
file1
file 2
file 3
device
domain
User/Domain 1 r
rx
rwx
–
enter
User/Domain 2 r
x
rx
rwx
–
User/Domain 3 rw
…
–
–
–
–
• Columns are access control lists (ACLs)
• Associated with each object
• Rows are capabilities
• Associated with each user or each domain
CS502 Spring 2006
Computer Security 11
What about efficiency?
• At run-time…
– What does the OS know about the user?
– What does the OS know about the resources?
• What is the cost of checking and enforcing?
– Access to the data
– Cost of searching for a match
CS502 Spring 2006
Computer Security 12
ACLs vs. Capabilities
• Focus on resources – the Access Control List
– Good if resources greatly outnumber users
– Can be implemented so that minimal information
caching is needed
– Good when the user who creates a resource has
authority over it
• Focus on users – the Capability System
– Good if users greatly outnumber resources
– Lots of information caching is needed
– Good when a system manager has control over all
resources
CS502 Spring 2006
Computer Security 13
We need both …
• ACLs for files and other proliferating resources
• Capabilities for major system functions
• The common OSs offer BOTH
– Linux emphasizes an ACL model
• provides good control over files and resources that are file-like
– Win200x emphasizes Capabilities
• provides good control over access to system functions (e.g.
creating a new user, or doing a system backup…)
CS502 Spring 2006
Computer Security 14
…and good management, too!
• What do we need to know to set up a new
user or to change their rights?
• …to set up a new resource or to change the
rights of its users?
• …Who has the right to set/change access
rights?
• No OS allows you to implement all the
possible policies easily.
CS502 Spring 2006
Computer Security 15
Enforcing Access Control
• User level privileges must always be less than OS
privileges!
– For example, a user should not be allowed to grab
exclusive control of a critical device
– or write to OS memory space
• …and the user cannot be allowed to raise his
privilege level!
• The OS must enforce it…and the user must not be
able to bypass the controls
• In most modern operating systems, the code which
manages the resource enforces the policy
CS502 Spring 2006
Computer Security 16
Requirements: System Call Code
• No user can interrupt it while it is running
• No user can feed it data to make it
– violate access control policies
– stop serving other users
• No user can replace or alter any system call
code
• No user can add functionality to the OS!
• Data must NEVER be treated as code!
CS502 Spring 2006
Computer Security 17
“Yeah, but …”
• No user can interrupt it while it is running
• Windows, Linux routinely interrupt system calls
• No user can feed it data to make it
– violate access control policies
– stop serving other users
• No user can replace or alter any system call code
• Except your average virus
• No user can add functionality to the OS!
• Except dynamically loaded device drivers
• Data must NEVER be treated as code!
• “One man’s code is another man’s data” A. Perlis
CS502 Spring 2006
Computer Security 18
Saltzer-Schroeder Guidelines
•
•
•
•
•
System design should be public
Default should be no access
Check current authority – no caching!
Least privilege possible for processes
Protection mechanism should be
– Simple, uniform, built into lowest layers of system
• Psychologically acceptable
• KISS!
CS502 Spring 2006
Computer Security 19
Authentication
• How does the machine know who it is
talking to???
• Who do I SAY that I am?
• How can I verify that?
• Something I know (that nobody else should know)
• Something I have (that nobody else should have)
• Something I am (that nobody else should be…)
CS502 Spring 2006
Computer Security 20
Threats against authentication
I want to pretend to be you:
• I can steal your password
– the sticky on your monitor or the list in your desk
drawer
– by monitoring your communications or looking over
your shoulder
• I can guess your password (particularly useful if I
can also guess your user name)
• I can get between you and the system you are
talking to
CS502 Spring 2006
Computer Security 21
The Trouble with Passwords
•
•
•
•
•
They are given away
They are too easy to guess
They are used too often
There are too many of them
They are used in too many places
CS502 Spring 2006
Computer Security 22
Some ways around the problem
• Better passwords
– longer
– larger character set
– more random in nature/encrypted
• Used less often
– changed frequently, one system per password
– challenge/response – use only once (p. 599)
CS502 Spring 2006
Computer Security 23
The Challenge/Response Protocol
Art
Mary
Hello, I’m Art
Decrypt This {R}P
R
Hello Art! How can I help you?
CS502 Spring 2006
Computer Security 24
Threat: Steal passwords from the system
• Don’t keep them in an obvious place
• Encrypt them so that the version the system
sees is not the same as the one the user
enters or the version on the wire.
CS502 Spring 2006
Computer Security 25
Too many passwords to remember?
• Third-party authentication
– Get someone to vouch for you
• The basics: “This guy says you know him..”
“Yes, I trust him, so you should too..”
• Kerberos – Certificate-based authentication
within a trust community
CS502 Spring 2006
Computer Security 26
What is in a certificate?
•
•
•
•
•
•
Who issued it
When was it issued
For what purpose was it issued
For what time frame is it valid
(possibly other application-specific data)
A “signature” that proves it has not been
forged
CS502 Spring 2006
Computer Security 27
Systems and Networks Are Not Different
• Same basic rules about
code behavior apply
• Same authentication
rules apply
• The same security
principles apply
CS502 Spring 2006
• Same Coding Rules
Apply To:
– An application
– Code which manages
incoming messages
– Code which imposes
access controls on a
network
– ...
Computer Security 28
The Principles
• Understand what you are trying to protect
• Understand the threat(s) you are trying to
protect against
– Also, costs and risks
• Be prepared to establish trust by telling
people how you do it
• Assume that the bad guys are at least as
clever as you are!
CS502 Spring 2006
Computer Security 29
Break
CS502 Spring 2006
Computer Security 30
Part 2: Human and Application Issues
• What are we defending against…and why?
• Weaknesses and Responses
– Technical
– Human
• Things to think about...
CS502 Spring 2006
Computer Security 31
Reading Assignment
• Ken Thompson, “Reflections on Trusting
Trust,” Communications of ACM, vol.27,
#8, August 1984, pp. 761-763.
– The very first virus.
CS502 Spring 2006
Computer Security 32
How do these attacks work?
• Messages that attack mail readers or
browsers
• Denial of service attacks against a web
server
• Password crackers
• Viruses, Trojan Horses, other “malware”
CS502 Spring 2006
Computer Security 33
The concept of a “Vulnerability”
• Buffer overflow
• Protocol/bandwidth interactions
– Protocol elements which do no work
• “execute this” messages
– The special case of “mobile agents”
• Human user vulnerabilities
– eMail worms
– Phishing
CS502 Spring 2006
Computer Security 34
Another Principle
• There is a never-ending war going on
between the “black hats” and the rest of us.
• For every asset, there is at least one
vulnerability
• For every protective measure we add,
“they” will find another vulnerability
CS502 Spring 2006
Computer Security 35
Yet Another Principle
• There is no such thing as a bullet-proof
barrier
• Every level of the system and network
deserves an independent threat evaluation
and appropriate protection
• Only a multi-layered approach has a chance
of success!
CS502 Spring 2006
Computer Security 36
Actual Losses:
• Approximately 70% are due to user error
• More than half of the remainder are caused
by insiders
• “Social Engineering” accounts for more loss
than technical attacks.
CS502 Spring 2006
Computer Security 37
What is “Social Engineering”?
“Hello. This is Dr. Burnett of the cardiology
department at the Conquest Hospital in
Hastings. Your patient, Sam Simmonds has
just been admitted here unconscious. He has
an unusual ventricular arrhythmia. Can you
tell me if there is anything relevant in his
record?”
CS502 Spring 2006
Computer Security 38
Social Engineering (2)
From: 3dksobinsky@zoom-internet.net
Sent: Sunday, March 26, 2006 8:10 AM
To: rmstronger@charter.net
Subject: Re: Approved
Please read the attached file.
CS502 Spring 2006
Computer Security 39
Computerworld 3/19/04
• 10 cents on risk assessment
• 15 cents on policy
• 40 cents on awareness and employee
education
• 20 cents on technology
• 15 cents on process, process, process
Patrick McBride
CS502 Spring 2006
Computer Security 40
What is a “Security Policy?”
• What rights MAY a user have?
– Define the maximum!
• What rights can a user pass on?
• How can a user acquire additional rights?
• Linux/Unix:
CS502 Spring 2006
-rwxr-xr-- /foo
-rw--w---- /bar
Computer Security 41
Policy Models (1)
A “Policy Model” is a framework for creating
a specific policy for a specific organization
• Linux/Unix
–
–
–
–
Users, groups, everybody
“owner” (or “…”) controls grant of rights
Rights based on UID, GID – Focus on files
Process has rights of parent
• can change GID or drop rights
CS502 Spring 2006
Computer Security 42
Policy Models (2)
• Win200X
– Users and groups
– Groups may be members of groups
– Rights are the combined rights of all groups of
which the user is a direct or indirect member
– Administrator controls everything
• can grant any right
– The default is strong control over admin
functions and little control over files
CS502 Spring 2006
Computer Security 43
Policy Models (3)
• Typical Business
– Managers can (usually) grant rights to their staff
– Information is visible to people above in the
organization
– Managers do not have authority to grant access
downward for some classes of information
– Overall control is maintained by restricting access to
applications rather than to data
– databases have their own distinct access controls
CS502 Spring 2006
Computer Security 44
Policy Models (4)
• The Military Mind
– Access rights are granted only by a higher
authority
– Access is broken into two models
• need-to-know (usually organizational with upward
visibility)
• item-by-item (classification may occur in advance of
creation or after)
– Creator may be denied access to own work
– Some weird anomalies
CS502 Spring 2006
Computer Security 45
Policy Models (5)
• The BMA (British Medical Assoc.) model (1995)
– Each medical record has an access control list
– Access may be granted to a new clinician by the subject
or the primary clinician
– Patient must be notified of all ACL changes, and may
revoke access
– Deletions are not allowed
– All access must be logged and auditable
– Information may be aggregated from A into B only if
ACL(A) is a superset of ACL(B)
• Reference
Anderson, Ross, “An Update on the BMA Security Policy,”
1996. (.pdf)
CS502 Spring 2006
Computer Security 46
Policy Models (6)
• The HIPAA model (1998)
– The patient controls the right to access
“personally identifiable health information”
– Access is granted to any clinician or facility
staff participating in the care of the patient
– Patient must be notified of all breaches
– Deletions are not allowed
– All access must be logged and auditable
– Privileges may be revoked
CS502 Spring 2006
Computer Security 47
More Principles
• Think about Assets, Threats and
Vulnerabilities FIRST
• Find an appropriate (and minimally
complex) Policy Model
• Match your OS capabilities to the policy
model as best you can
• Train staff to recognize social engineering!
• Train staff to make a habit out of the policy!
CS502 Spring 2006
Computer Security 48
Break
CS502 Spring 2006
Computer Security 49
Part 3: Fun with Cryptography
• What is cryptography about?
• General Principles of Cryptography
• Basic Protocols
– Single-key cryptography
– Public-key cryptography
• An example...
CS502 Spring 2006
Computer Security 50
Principles
• Cryptography is about the exchange of
messages
• The key to success is that all parties to an
exchange trust that the system will both
protect them from threats and accurately
convey their message
• TRUST is essential
CS502 Spring 2006
Computer Security 51
Therefore
• Algorithms must be public and verifiable
• We need to be able to estimate the risk of
compromise
• The solution must practical for its users, and
impractical for an attacker to break
CS502 Spring 2006
Computer Security 52
Guidelines
• Cryptography is always based on algorithms
which are orders of magnitude easier to
compute in the forward (normal) direction
than in the reverse (attack) direction.
• The attacker’s problem is never harder than
trying all possible keys
• The more material the attacker has the
easier his task
CS502 Spring 2006
Computer Security 53
Example
• What is
314159265358979  314159265358979?
vs.
• What are prime factors of
3912571506419387090594828508241?
CS502 Spring 2006
Computer Security 54
Time marches on…
• We must assume that there will be
improvements in computational power,
mathematics and algorithms.
– Messages which hang around get less secure
with time!
• Increases in computing power help the good
guys and HURT the bad guys for new and
short-lived messages
CS502 Spring 2006
Computer Security 55
Caveat
• We cannot mathematically PROVE that the
inverse operations are really as hard as they
seem to be…It is all relative…
The Fundamental Tenet of Cryptography:
If lots of smart people have failed to solve a
problem, it won’t be solved (soon)
CS502 Spring 2006
Computer Security 56
Secret key cryptography
K
T
Cleartext
CS502 Spring 2006
f (T,K)
K
C
Cyphertext
g (C,K)
Computer Security 57
T
Cleartext
Secret Key Methods
• DES (56 bit key)
• IDEA (128 bit key)
• http://www.mediacrypt.com/community/index.asp
• Triple DES (three 56 bit keys)
• AES
– From NIST, 2000
– choice of key sizes up to 256 bits and more
– Commercial implementations available
CS502 Spring 2006
Computer Security 58
Diffie – Hellman
Alice
Agree on p,g
choose random A
Bob
choose random B
TA = gA mod p
TB = gB mod p
compute (TB)A
compute (TA)B
Shared secret key is gAB mod p
CS502 Spring 2006
Computer Security 59
D–H Problems
• Not in itself an encryption method – we
must still do a secret key encryption
• Subject to a “man in the middle” attack
(Alice thinks she is talking to Bob, but
actually Trudy is intercepting all of the
messages and substitution her own)
CS502 Spring 2006
Computer Security 60
RSA Public key cryptography
Key #1
T
Cleartext
f ()
Key #2
C
Cyphertext
f ()
T
Cleartext
Key #1 can be either a Public Key or a Private Key.
Key #2 is then the corresponding Private Key or Public Key.
CS502 Spring 2006
Computer Security 61
RSA Public Key Cryptography
• Rivest, Shamir and Adelman (1978)
• I can send messages that only you can read
• I can verify that you and only you could
have sent a message
• I can use a trusted authority to distribute my
public key
– The trusted authority is for your benefit!
CS502 Spring 2006
Computer Security 62
RSA Details
• We will use the same operation to encrypt
and decrypt
• To encrypt, we will use “e” as a key, to
decrypt we will use “d” as a key
• e and d are inverses with respect to the
chosen algorithm
CS502 Spring 2006
Computer Security 63
RSA Details
• Choose n as the product of two large primes
– Finding the factors of a large number is
mathematically hard (difficult)
– Finding primes is also hard
• Choose e to be a (fairly small) prime and
compute d from e and the factors of n
• THROW AWAY THE FACTORS OF n!
• Publish two numbers, e (public key) and n
CS502 Spring 2006
Computer Security 64
RSA Details
• Encryption: Cyphertext = (Cleartext)e mod n
• Decryption: Cleartext = (Cyphertext)d mod n
• Typical d will be on the order of 500 to 700 bits
• The cost of the algorithm is between 1 and 2 
the size of n,
– Each operation is a giant shift and add (multiply by a
power of 2)
CS502 Spring 2006
Computer Security 65
RSA Problems
• It is much more costly than typical secretkey methods 
– Use RSA to hide (i.e., encrypt) a secret key,
– Encrypt the message with the secret key and
append/prefix the encrypted key
• Requires a “Public Key Infrastructure” for
effective key generation and distribution
– Chain of trust thing again!
CS502 Spring 2006
Computer Security 66
Message Digests (aka Digital Signatures)
• A message digest is a non-reversable
algorithm which reduces a message to a
fixed-length “summary”
• The summary has the property that a change
to the original will produce a new summary
• The probability that the new summary is the
same as the old should be 1/(size of digest)
• Tanenbaum, p. 590 (§9.2.4)
CS502 Spring 2006
Computer Security 67
Message Digests (2)
• There are several good (but possibly no
perfect) message digest algorithms
• MD5 is probably the most common one in
use – 128 bit digest
• has known weaknesses
• SHA-1 – 160 bit digest (current best choice)
• [Another product of NIST]
CS502 Spring 2006
Computer Security 68
Resources
• Network World Security Newsletter
– http://www.nwsubscribe.com
– Practical advice, not a virus alert newsletter. Especially good for
the links to other security resources at the bottom of each article
• CERT Coordination Center at CMU
– http://www.cert.org
• News about system threats, including viruses and other
problems. Source for OCTAVE papers and process
• Norton AntiVirus Site (Symantec)
– http://securityresponse.symantec.com/avcenter/
• McAfee Security (Network Associates)
– http://us.mcafee.com/virusinfo/
CS502 Spring 2006
Computer Security 70
Textbooks
Network Security: C. Kaufman, R. Perlman, M. Speciner,
Prentice Hall (2002)
– A practical but rigorous presentation of network security issues and
techniques with emphasis on cryptographic solutions
Security Engineering: R. Anderson, Wiley (2001)
– Focused on learning from past mistakes in security system design.
– Excellent discussion of policies and policy models.
– See author’s web site (www.ross-anderson.com) if you are
interested in current research.
CS502 Spring 2006
Computer Security 71
Other Books
Real World Linux Security: R. Toxen, Prentice Hall (2003)
– An excellent read. Lists hundreds of vulnerabilities and what to do
about them. Valuable for non Linux users too.
Windows 2003 Security Bible: B. Rampling, Wiley (2003)
– Good example of a how-to book. Specific to WIN2003
The Art of Deception: K. Mitnick, Wiley (2002)
– Mitnick is one of the most famous social engineers.
– Must-read for those involved in broad security planning, and fun
for everyone.
CS502 Spring 2006
Computer Security 72