ECS 235A Computer Security Course Scribe for 29 th Sept’11
Prepared by: Mohammad Rezaur Rahman
Buffer overrun and its prevention
Buffer overrun
Introduction




Historically most widespread of vulnerabilities
Instance of internet worm
Java and Python are buffer overflow safe but C is not because in C there is no way to perform
bounds checking
Buffer overrun is a programming error
Attack exploiting buffer overflow
int main ()
{
f();
}
void f()
{
Char buf[8];
gets(buf);
}
The above program reads a string from the terminal and giving more than 7 characters as input will result
in buffer overflow. When program like this is loaded into the memory the main memory is segmented
stack, heap, global variables and the text area which stores executable instruction. Now, to exploit buffer
overflow the attacker uses different approaches like:


Attacker places his own payload on the stack
Or return-into-libc exploit
Goals of Attacker





The attacker will overwrite the return address
No legitimate instruction
Not valid address which will result in segmentation fault
Exploit the corner case which will result in causing the program crash
Malicious code executed
Types of Programming errors
Trapped error
 Behavior is specified
 Program terminate immediately


And detected and handled immediately by hardware
Example: division by 0 error
Untrapped error
 Not specified
 Not handled by hardware
 More dangerous than trapped errors
Buffer overflow is untrapped



May continue running
Or may stop running since we don’t know what the attacker is running there
More vulnerable
Type safety



No program in a language may cause untrapped error.
Java is type safe language
Java does most of the type safety checking before it runs the program
Approach to prevent buffer overrun
Use a canary to check that return address has not been tampered with. Attacker has to overwrite canary
before overwriting. This approach is called Stack-Guard.
Placement of Canary in the stack of memory layout
FFF…F
Retf
Canary
Canary
buf[8]
000…0
What value of canary




Must make sure attacker doesn’t know the canary value
Has to be randomized and unpredictable
It should be global variable so that the variable is not on the stack
o Otherwise if the canary is on the stack and the attacker can not only overflow the buffer
but also can overflow the local canary
How big is the canary?
o Depends on security parameter
Stack-Guard



Would like to have the source code to protect from buffer overflow
Include it in the compilation process
No need to modify every source program to embed it
Where compiler should put the code for placing the canary and for checking the canary?
 Placing need to be in the callee
o So that it resides bellow the return address
 Checking should be done in the callee
o So that it’s not too late and have to check the canary before the function returns
Advantage of Stack-Guard
 Linkage of a Stack-Guard protected library with a non-Stack-Guard protected application or vice
versa works without failure
 Though it might need some initialization function but it is solved easily and the linkage works
Disadvantage
 We may not have the source code or the group recompilation may not be feasible
Alternative approach if we don’t have the source code


Making the stack non-executable: page writable or executable (W^X)
But it can also be circumvented by return-to-libc attack
Another defense

Change the relevant position of the return address with the buffer
o Suppose, placing the return address bellow the buffer: but it doesn’t work when buffer is
sent as a parameter
Another approach: Make the address of stack frame unpredictable

But not bulletproof:
o Attacker can use trampoline involving registers
o Or the use of trampoline without involving the registers
Detect buffer overflow


Static analysis
Dynamic/runtime analysis