UNIT-I
Subject Name : System Security
Subject Code : TCS-591
Control hijacking attacks
Attacker wants:
• Take control over target machine (e.g. web server)
• Execute arbitrary code on target by hijacking application control
flow
• Example: Buffer overflow attacks, Integer overflow attacks
Buffer overflow attack
• Buffer overflow occurs when a program or process attempts to
write more data to a fixed length block of memory (a buffer), than
the buffer is allocated to hold.
• By sending carefully crafted input to an application, an attacker
can cause the application to execute arbitrary code, possibly taking
control over the machine.
Memory layout
• In order to understand how buffer overflows work, we need to understand what happens
in memory when a program runs.
• In this example we’re using a C program in Linux.
• When a program is run by the operating system (OS), the executable will be held in
memory in a very specific way.
• The operating system will effectively call the main method of the code as a function,
which then starts the flow for the rest of the program.
Memory layout
Memory layout
• The top of the memory is the kernel area, which contains the command-line parameters
that are passed to the program and the environment variables.
• The bottom area of the memory is called text and contains the actual code, the compiled
machine instructions, of the program. It is a read- only area, because these should not be
allowed to be changed.
• Above the text is the data, where uninitialized and initialized variables are stored.
• On top of the data area, is the heap. This is a big area of memory where large objects are
allocated (like images, files, etc.)
Memory layout
• Below the kernel is the stack. This holds the local variables for
each of the functions. When a new function is called, these are
pushed on the end of the stack.
• Note that the heap grows up (from low to higher memory) when
more memory is required by an application and the stack grows
downwards (from high to lower memory).
Memory layout
• Since the stack grows downward, every item pushed on top of the stack, will make it
grow towards the low memory address area.
• Consider the case where a program calls a function, a piece of code that does something
and returns where it was before.
• When the call is made, the parameters that are passed to the function, are pushed on top
of the stack.
• With the parameters on the stack, the code of the function will then jump to somewhere
else in memory and do something with these parameters.
Memory layout
Piece of C-code for overflow
• The program calls a function which allocates some memory onto
the stack, copies a string from the command line into it and outputs
the string with a welcome message.
• argc(argument count) stores the number of the arguments passed
to the main function and argv(argument vector) stores the array of
the one-dimensional array of strings.a
Piece of C-code for overflow
• The main-method is the entry point for the program and the first thing it does, is to call
the function (func), passing the first command-line parameter to it (i.e. argv[1]).
• Then, the function should know where to return when the function exits, so the address of
the next instruction is pushed onto the stack as the return address.
• The EBP (or extended base pointer) is then pushed onto the stack. This pointer is used to
refer to parameters and local variables.
• Then, a buffer of 100 bytes long is allocated in the stack, followed by a call to the
stringcopy function (strcpy) which will copy the name-parameter into the buffer.
• After this the contents of the buffer are output together with the welcome message.
How to smash (attack)
• Inject a malicious code at a specific address in the target machine.
• Give application a very long string which also contains the address of the malicious code.
• The string length, being much larger than the space allocated in the heap (buffer size
declaration) causes the heap to overflow into the stack and overwrites the return address.
• Overwrite the return address with the address where you have injected the malicious
code.
• The return address now points to the beginning of the malicious code.
• Then system will start the execution of the malicious code (because from here it has to
start the execution).
Integer overflows attack
• Integer overflow, an arithmetic operation results in an integer
(whole number) that is too large for the integer type meant to store
it that further results in a buffer overflow.
Integer overflows attack
• Similar to other overflow attacks.
• Integer overflows and other integer manipulation vulnerabilities
frequently result in buffer overflows.
• An integer overflow occurs when an arithmetic operation (i.e.,
addition) results in a number that is too large to be stored in the
space allocated for it.
• Code checks the incoming data is no larger than the destination
buffer, adding two integers together.
• Imagine that the first variable is 0x98 and the second variable is
0xEEEEEEEC, and then output is bizarre as following (something
unusual happens) .152+ 4008636140=4008636292
• This is conceptually overflowing memory all the way.
• Addition result goes out of the memory bound. Hence, results in
overflow condition which further causes integer overflow attack.
• Rest of the steps are similar to buffer overflow attacks as
previously discussed i.e., overwriting of Return address.
References
● Textbook: Security in Computing, 5th Edition by C. P. Pfleeger, S. L. Pfleeger, J.
Margulies
● Preventing Buffer Overflows by Paladion. Information available at :
https://www.paladion.net/blogs/preventing-buffer-overflows
● fsu.edu
● Basic Integer Overflow. Information available at: https://www.cs.utexas.edu/
● ~shmat/courses/cs361s/blexim.txt
