SCSC 555 Computer Security
Chapter 10 Malicious software
Part B
Index





Social Engineering
Malware Payload
Countermeasures of malware
DDoS
Buffer overflow
Social Engineering

Tricking user to assist in the comprise of their
own systems or personal information

Spam e-mail



Most spam is sent by botnets using compromised user
systems
Advertising, scams, carrier of malware (attachment), or
phishing attack
Trojan horse program

A useful, or apparently useful program or utility
containing hidden code that performs some unwanted or
harmful funciton
Social Engineering

Trojan horse program
 E.g. claim to be antivirus scanner, security update
actual carrying payload such as spyware
 Three models of Trojan horses



Continuing to perform the function of the original
program and additionally performing a separate
malicious activity
… but modifying the function to perform malicious
activity
Performing a malicious function that completely replaces
the function of the original program
Malware Payload

System corruption




Data destruction
Ransomware, e.g. Gpcode Trojan
Real-world damage: cause damage to physical
equipment
Logic bomb: is set to “explode” when certain
conditions are met
Malware Payload

Attack agent



Bot: malware subverts the computational and
network resources of infected system for use by
the attacker
The uses of bots: DDoS attack, Spamming …
Botnet: the collection of bots often is capable of
acting in a coordinated manner.
Malware Payload

Information theft

Keyloggers and Spyware

What is a keylogger? (next slide)



Countermeasure to keylogger
Spyware
Phishing and Identity theft


URL of fake Web site controlled by the attacker
Spear-phishing: an email claiming to be from a trusted
sources, the recipients are carefully researched by the
attacker  greatly increases the likehood of the
recipient responding as desired by the attacker
Malware Payload

Backdoor and Rootkit


Backdoor (trapdoor) is a secret entry point into a
program without going through the usual security
access procedures
Rootkit is a set of programs installed on a system
to maintain covert access to that system with root
privileges, while hiding evidence of its presence


Make many changes to a system to hide its existence
Difficult to detect
Keyloggers

Keyloggers are used to capture
keystrokes on a computer



Software


Hardware
Software
Behaves like Trojan programs
Hardware



Easy to install
Goes between the keyboard and the
CPU
KeyKatcher and KeyGhost
Countermeasures of malware

Ensure all systems are current


Set appropriate access controls on the
applications and data


All patches applied
to reduce the number of files that any user can
access
Training the users to against social
engineering attack
Countermeasures of malware

Technical mechanism to mitigate threat




Detection
Identification
Removal
Requirements for effective malware
countermeasures

Generality, Timeliness, Resiliency, minimal DOS
costs, transparency, global and local coverage
Countermeasures of malware

Four generations of antivirus software





1st generation
2nd generation
3rd generation
4th generation
More sophisticated antivirus approaches
Host-based behavior-blocking
 Perimeter scanning approaches
(Reading page 323 – 327)

Distributed Denial-of-Service (DDoS)
Attacks



DDoS attack on a host from multiple servers or
workstations
Network could be flooded with billions of requests
 Loss of bandwidth
 Degradation or loss of speed
Often participants (zombies) are not aware they are part
of the attack
 Thousands zombies are controlled by the attacker via
Trojan programs
DDoS Tools and Countermeasures
DDoS countermeasures:
• Security patches from software vendors
• Antivirus software
• Firewalls: Ingress (inbound) and egress (outbound) filtering
(details next …)
Buffer Overflow Attacks

A vulnerability in poorly written code
 does not check predefined size of input field

Goal of buffer overflow attack:
 Fill overflow buffer with executable code
 OS executes this code, elevates attacker’s permission



Administrator
Owner of running application
To stop software exploits
 Train your programmer in developing applications with security in
mind
 Stay appraised of latest security patches provided by software
vendors
Buffer Overflow Exploits

Buffer Overflow Exploits is the Most common cause of
Internet attacks
Over 50% of advisories published by CERT (computer
security incident report team) are caused by various
buffer overflows

Morris worm (1988): overflow in fingerd
Infected 10% of the existing Internet
CodeRed (2001): overflow in MS-IIS server
300,000 machines infected in 14 hours
SQL Slammer (2003): overflow in MS-SQL server
75,000 machines infected in 10 minutes


Memory Buffers

Buffer is a data storage area inside computer
memory (stack or heap)



Intended to hold pre-defined amount of data
 If more data is stuffed into it, it spills into adjacent
memory
If executable code is supplied as “data”, victim’s machine
may be fooled into executing it
 Code will self-propagate or give attacker control over
machine
Attack can exploit any memory operation

Pointer assignment, format strings, memory allocation
and de-allocation, function pointers, calls to library
routines via offset tables
Stack Buffers

Suppose Web server contains this function
void func(char *str) {
char buf[126];
strcpy(buf,str);
}

Allocate local buffer
(126 bytes reserved on stack)
Copy argument into local buffer
When this function is invoked, a new frame with
local variables is pushed onto the stack
Stack grows this way
Top of
stack
Frame of the
calling function
buf
Local variables
sfp
ret
addr
str
Pointer to Execute
Arguments
previous
code at
frame this address
after func()
finishes
What If Buffer is Overstuffed?

Memory pointed to by str is copied onto stack…
void func(char *str) {
char buf[126];
strcpy does NOT check whether the string
strcpy(buf,str); at *str contains fewer than 126 characters
}

If a string longer than 126 bytes is copied into
buffer, it will overwrite adjacent stack locations
Top of
stack
Frame of the
calling function
buf
overflow
This will be
interpreted
as return address!
str
Executing Attack Code

Suppose buffer contains attacker supplied string
 For example, *str contains a string received from the network as
input to some network service daemon
Top of
stack
Frame of the
calling function
code
Attacker puts actual
instructions into his input string, e.g.,
binary code of execve(“/bin/sh”)

ret
str
In the overflow, a pointer back
into the buffer appears in
the location where the system
expects to find return address
When function exits, code in the buffer will be
executed, giving attacker a shell
 The attacker gets a root shell if the victim program is SUID root
Some Issues on Buffer Overflow

Executable attack code is stored on stack, inside the buffer
containing attacker’s string
 Stack memory is supposed to contain only data, but…

Overflow portion of the buffer must contain correct address of
attack code in the RET position
 The value in the RET position must point to the beginning
of attack code in the buffer
 Otherwise application will crash with segmentation
violation
 Attacker must know or correctly guess in which stack
position his buffer will be when the function is called
The Cause : No Range Checking

strcpy does not check input size
 strcpy(buf, str) simply copies memory contents into buf
starting from *str until “\0” is encountered
 Ignoring the size of area allocated to buf

Many C library functions are unsafe
 strcpy(char *dest, const char *src)
 strcat(char *dest, const char *src)
 gets(char *s)
 scanf(const char *format, …)
 printf(const char *format, …)
Examples of Common Buffer
Overflow Attacks
Examples of Common Buffer
Overflow Attacks