AddressSanitizer - A Tool for Programmers to Detect Memory Access Errors
Memory access errors are the most common software errors that often cause program crashes. The AddressSanitizer tool, developed by Google engineers in 2012, has become the first choice of C/C++ programmers for its wide coverage, high efficiency, and low overhead. Here is a brief introduction to its principle and usage.
Tool Overview
The C/C++ language allows programmers to have low-level control over memory, and this direct memory management has made it possible to write efficient application software. However, this has also made memory access errors, including buffer overflows, accesses to freed memory, and memory leaks, a serious problem that must be coped with in program design and implementation. While there are tools and software that provide the ability to detect such errors, their operational efficiency, and functional coverage are often less than ideal.
In 2012, Google engineer Konstantin Serebryany and team members released an open-source memory access error detector for C/C++ programs called AddressSanitizer1. AddressSanitizer (ASan) applies new memory allocation, mapping, and code stubbing techniques to detect almost all memory access errors efficiently. Using the SPEC 2006 benchmark analysis package, ASan runs with an average slowdown of less than 2 and memory consumption of about 2.4 times. In comparison, another well-known detection tool Valgrind has an average slowdown of 20, which makes it almost impossible to put into practice.
The following table summarizes the types of memory access errors that ASan can detect for C/C++ programs:
Error Type | Abbreviation | Notes |
---|---|---|
heap use after free | UAF | Access freed memory (dangling pointer dereference) |
heap buffer overflow | Heap OOB | Dynamic allocated memory out-of-bound read/write |
heap memory leak | HML | Dynamic allocated memory not freed after use |
global buffer overflow | Global OOB | Global object out-of-bound read/write |
stack use after scope | UAS | Local object out-of-scope access |
stack use after return | UAR | Local object out-of-scope access after return |
stack buffer overflow | Stack OOB | Local object out-of-bound read/write |
ASan itself cannot detect heap memory leaks. But when ASan is integrated into the compiler, as it replaces the memory allocation/free functions, the original leak detection feature of the compiler tool is consolidated with ASan. So, adding the ASan option to the compilation command line also turns on the leak detection feature by default.
This covers all common memory access errors except for "uninitialized memory reads" (UMR). ASan detects them with a false positive rate of 0, which is quite impressive. In addition, ASan detects several C++-specific memory access errors such as
- Initialization Order Fiasco: When two static objects are defined in different source files and the constructor of one object calls the method of the other object, a program crash will occur if the former compilation unit is initialized first.
- Container Overflow: Given libc++/libstdc++ container, access [container.end(), container.begin() + container.capacity())], which crosses the [container.begin(), container.end()] range but still within the dynamically allocated memory area.
- Delete Mismatch: For the array object created by
new foo[n]
, should not calldelete foo
for deletion, usedelete [] foo
instead.
ASan's high reliability and performance have made it the preferred choice of compiler and IDE developers since its introduction. Today ASan is integrated into all four major compilation toolsets:
Compiler/IDE | First Support Version | OS | Platform |
---|---|---|---|
Clang/LLVM2 | 3.1 | Unix-like | Cross-platform |
GCC | 4.8 | Unix-like | Cross-platform |
Xcode | 7.0 | Mac OS X | Apple products |
MSVC | 16.9 | Windows | IA-32, x86-64 and ARM |
ASan's developers first used the Chromium open-source browser for routine testing and found more than 300 memory access errors over 10 months. After integration into mainstream compilation tools, it reported long-hidden bugs in numerous popular open-source software, such as Mozilla Firefox, Perl, Vim, PHP, and MySQL. Interestingly, ASan also identified some memory access errors in the LLVM and GCC compilers' code. Now, many software companies have added ASan run to their mandatory quality control processes.
Working Principle
The USENIX conference paper 3, published by Serebryany in 2012, comprehensively describes the design principles, algorithmic ideas, and programming implementation of ASan. In terms of the overall structure, ASan consists of two parts.
- Compiler instrumentation - modifies the code to verify the shadow memory state at each memory access and creates poisoned red zones at the edges of global and stack objects to detect overflows or underflows.
- Runtime library replacement - replaces
malloc/free
and its related functions to create poisoned red zones at the edge of dynamically allocated heap memory regions, delay the reuse of memory regions after release, and generate error reports.
Here shadow memory, compiler instrumentation, and memory allocation function replacement are all previously available techniques, so how has ASan innovatively applied them for efficient error detection? Let's take a look at the details.
Shadow Memory
Many inspection tools use separated shadow memory to record metadata about program memory, and then apply instrumentation to check the shadow memory during memory accesses to confirm that reads and writes are safe. The difference is that ASan uses a more efficient direct mapping shadow memory.
The designers of ASan noted that typically the malloc
function returns a memory address that is at least 8-byte aligned. For example, a request for 20 bytes of memory would divide 24 bytes of memory, with the last 3 bits of the actual return pointer being all zeros. in addition, any aligned 8-byte sequence would only have 9 different states: the first \(k\,(0\leq k \leq 8)\) bytes are accessible, and the last \(8-k\) are not. From this, they came up with a more compact shadow memory mapping and usage scheme:
- Reserve one-eighth of the virtual address space for shadow memory
- Directly map application memory to shadow memory using a formula that divides by 8 plus an offset
- 32-bit application:
Shadow = (Mem >> 3) + 0x20000000;
- 64-bit application:
Shadow = (Mem >> 3) + 0x7fff8000;
- 32-bit application:
- Each byte of shadow memory records one of the 9 states of the corresponding 8-byte memory block
- 0 means all 8 bytes are addressable
- Any negative value indicates that the entire 8-byte word is unaddressable (poisoned )
- k (1 ≤ k ≤ 7) means that the first k bytes are addressable
The following figure shows the address space layout and mapping relationship of ASan. Pay attention to the Bad area in the middle, which is the address segment after the shadow memory itself is mapped. Because shadow memory is not visible to the application, ASan uses a page protection mechanism to make it inaccessible.
Compiler Instrumentation
Once the shadow memory design is determined, the implementation of compiler instrumentation to detect dynamic memory access errors is easy. For memory accesses of 8 bytes, the shadow memory bytes are checked by inserting instructions before the original read/write code, and an error is reported if they are not zero. For memory accesses of less than 8 bytes, the instrumentation is a bit more complicated, where the shadow memory byte values are compared with the last three bits of the read/write address. This situation is also known as the "slow path" and the sample code is as follows.
1 | // Check the cases where we access first k bytes of the qword |
For global and stack (local) objects, ASan has designed different instrumentation to detect their out-of-bounds access errors. The red zone around a global object is added by the compiler at compile time and its address is passed to the runtime library at application startup, where the runtime library function then poisons the red zone and writes down the address needed in error reporting. The stack object is created at function call time, and accordingly, its red zone is created and poisoned at runtime. In addition, because the stack object is deleted when the function returns, the instrumentation code must also zero out the shadow memory it is mapped to.
In practice, the ASan compiler instrumentation process is placed at the end of the compiler optimization pipeline so that instrumentation only applies to the remaining memory access instructions after variable and loop optimization. In the latest GCC distribution, the ASan compiler stubbing code is located in two files in the gcc subdirectory gcc/asan.[ch]
.
Runtime Library Replacement
The runtime library needs to include code to manage shadow memory. The address segment to which shadow memory itself is mapped is to be initialized at application startup to disable access to shadow memory by other parts of the program. The runtime library replaces the old memory allocation and free functions and also adds some error reporting functions such as __asan_report_load8
.
The newly replaced memory allocation function malloc
will allocate additional storage as a red zone before and after the requested memory block and set the red zone to be non-addressable. This is called the poisoning process. In practice, because the memory allocator maintains a list of available memory corresponding to different object sizes, if the list of a certain object is empty, the OS will allocate a large set of memory blocks and their red zones at once. As a result, the red zones of the preceding and following memory blocks will be connected, as shown in the following figure, where \(n\) memory blocks require only \(n+1\) red zones to be allocated.
The new free
function needs to poison the entire storage area and place it in a quarantine queue after the memory is freed. This prevents the memory region from being allocated any time soon. Otherwise, if the memory region is reused immediately, there is no way to detect incorrect accesses to the recently freed memory. The size of the quarantine queue determines how long the memory region is in quarantine, and the larger it is the better its capability of detecting UAF errors!
By default, both the malloc
and free
functions log their call stacks to provide more detailed information in the error reports. The call stack for malloc
is kept in the red zone to the left of the allocated memory, so a large red zone can retain more call stack frames. The call stack for free
is stored at the beginning of the allocated memory region itself.
Integrated into the GCC compiler, the source code for the ASan runtime library replacement is located in the libsanitizer subdirectory libsanitizer/asan/*
, and the resulting runtime library is compiled as libasan.so
.
Application Examples
ASan is very easy to use. The following is an example of an Ubuntu Linux 20.4 + GCC 9.3.0 system running on an x86_64 virtual machine to demonstrate the ability to detect various memory access errors.
Test Cases
As shown below, the test program writes seven functions, each introducing a different error type. The function names are cross-referenced with the error types one by one:
1 | /* |
The test program calls the getopt
library function to support a single-letter command line option that allows the user to select the type of error to be tested. The command line option usage information is as follows.
1 | $ ./asan-test |
The GCC compile command for the test program is simple, just add two compile options
-fsanitize=address
: activates the ASan tool-g
: enable debugging and keep debugging information
OOB Test
For Heap OOB error, the run result is
1 | $ ./asan-test -b |
Referring to the heap-buffer-overflow
function implementation, you can see that it requests 40 bytes of memory to hold 10 32-bit integers. However, on the return of the function, the code overruns to read the data after the allocated memory. As the above run log shows, the program detects a Heap OOB error and aborts immediately. ASan reports the name of the source file and line number asan-test.c:34
where the error occurred, and also accurately lists the original allocation function call stack for dynamically allocated memory. The "SUMMARY" section of the report also prints the shadow memory data corresponding to the address in question (observe the lines marked by =>
). The address to be read is 0x604000000038, whose mapped shadow memory address 0x0c087fff8007 holds the negative value 0xfa (poisoned and not addressable). Because of this, ASan reports an error and aborts the program.
The Stack OOB test case is shown below. ASan reports an out-of-bounds read error for a local object. Since the local variables are located in the stack space, the starting line number asan-test.c:37
of the function stack_buffr_overflow
is listed. Unlike the Heap OOB report, the shadow memory poisoning values for the front and back redzone of the local variable are different, with the previous Stack left redzone
being 0xf1 and the later Stack right redzone
being 0xf3. Using different poisoning values (both negative after 0x80) helps to quickly distinguish between the different error types.
1 | $ ./asan-test -s |
The following Global OOB test result also clearly shows the error line asan-test.c:16
, the global variable name ga
and its definition code location asan-test.c:13:5
, and you can also see that the global object has a red zone poisoning value of 0xf9.
1 | $ ./asan-test -o |
Note that in this example, the global array int ga[10] = {1};
is initialized, what happens if it is uninitialized? Change the code slightly
1 | int ga[10]; |
Surprisingly, ASan does not report the obvious Global OOB error here. Why?
The reason has to do with the way GCC treats global variables. The compiler treats functions and initialized variables as Strong symbols, while uninitialized variables are Weak symbols by default. Since the definition of weak symbols may vary from source file to source file, the size of the space required is unknown. The compiler cannot allocate space for weak symbols in the BSS segment, so it uses the COMMON block mechanism so that all weak symbols share a COMMON memory region, thus ASan cannot insert the red zone. During the linking process, after the linker reads all the input target files, it can determine the size of the weak symbols and allocate space for them in the BSS segment of the final output file.
Fortunately, GCC's -fno-common
option turns off the COMMON block mechanism, allowing the compiler to add all uninitialized global variables directly to the BSS segment of the target file, also allowing ASan to work properly. This option also disables the linker from merging weak symbols, so the linker reports an error directly when it finds a compiled unit with duplicate global variables defined in the target file.
This is confirmed by a real test. Modify the GCC command line for the previous code segment
1 | gcc asan-test.c -o asan-test -fsanitize=address -fno-common -g |
then compile, link, and run. ASan successfully reported the Global OOB error.
UAF Test
The following is a running record of UAF error detection. Not only is the information about the code that went wrong reported here, but also the call stack of the original allocation and free functions of the dynamic memory is given. The log shows that the memory was allocated by asan-test.c:25
, freed at asan-test.c:27
, and yet read at asan-test.c:28
. The shadow memory data printed later indicates that the data filled is negative 0xfd, which is also the result of the poisoning of the memory after it is freed.
1 | $ ./asan-test -f |
HML Test
The results of the memory leak test are as follows. Unlike the other test cases, ABORTING
is not printed at the end of the output record. This is because, by default, ASan only generates a memory leak report when the program terminates (process ends). If you want to check for leaks on the fly, you can call ASan's library function __lsan_do_recoverable_leak_check
, whose definition is located in the header file sanitizer/lsan_interface.h
.
1 | $ ./asan-test -l |
UAS Test
See the stack_use_after_scope
function code, where the memory unit holding the local variable c
is written outside of its scope. The test log accurately reports the line number line 54
where the variable is defined and the location of the incorrect writing code asan-test.c:57
:
1 | ./asan-test -p |
UAR Test
The UAR test has its peculiarities. Because the stack memory of a function is reused immediately after it returns, to detect local object access errors after return, a "pseudo-stack" of dynamic memory allocation must be set up, for details check the relevant Wiki page of ASan4. Since this algorithm change has some performance impact, ASan does not detect UAR errors by default. If you really need to, you can set the environment variable ASAN_OPTIONS
to detect_stack_use_after_return=1
before running. The corresponding test logs are as follows.
1 | $ export ASAN_OPTIONS=detect_stack_use_after_return=1 |
ASan supports many other compiler flags and runtime environment variable options to control and tune the functionality and scope of the tests. For those interested please refer to the ASan flags Wiki page5.
A zip archive of the complete test program is available for download here: asan-test.c.gz
Serebryany, K.; Bruening, D.; Potapenko, A.; Vyukov, D. "AddressSanitizer: a fast address sanity checker". In USENIX ATC, 2012↩︎