Defense Layers

Modern Mitigations

No single defense stops every exploit. Together — stacked — they raise the cost of every step in the chain.

The previous lectures show how memory corruption gives attackers code execution. Mitigations are the runtime, compiler, and OS countermeasures that turn each exploitation step into a research problem. They rarely fix the underlying bug — they make the bug harder to weaponize.

The pattern is always the same: an exploit needs a reliable address, a writable location for shellcode, or an unprotected control-flow edge. Each mitigation removes one of those affordances. Attackers respond with new techniques (ROP defeated DEP, infoleaks defeated ASLR), and defenses evolve to close the new gaps. This is the arms race, and it never settles.

Layer 1 / Compiler

Stack Canaries

A random "canary" value is placed between local buffers and the saved return address at every function prologue. The epilogue checks it. If a buffer overflow rewrites the return address, it also rewrote the canary — the check fails and the process aborts before the corrupted return is used.

Also calledStackGuard, SSP, /GS StopsStraight-line stack buffer overflows Bypassed byCanary leaks via format-string bugs, structured-exception-handler hijack, brute force on forking servers Flag-fstack-protector-strong
Layer 2 / OS + Hardware

DEP / NX / W^X

Memory pages are marked either writable or executable, never both. Stack, heap, and data segments lose their exec bit; only the code segment keeps it. Shellcode written into a buffer can no longer be jumped to — the CPU faults on instruction fetch from a non-executable page.

Also calledNX bit, XD bit, Write XOR Execute StopsClassic "inject shellcode and return into it" Bypassed byROP / JOP / ret2libc — reuse existing executable code instead of injecting new ShippedLinux 2.6+, Windows XP SP2+, macOS, iOS, Android
Layer 3 / OS Loader

ASLR / PIE

Address Space Layout Randomization picks a different base address every time a program loads its stack, heap, libraries, and (with PIE) the main binary. Hard-coded addresses in shellcode — the address of system(), of a buffer, of a ROP gadget — no longer work. The exploit must first leak an address.

Also calledPIE, KASLR (for the kernel) StopsExploits with hard-coded addresses Bypassed byInformation-disclosure bugs that leak a pointer, partial overwrites (low byte), brute force on 32-bit systems Entropy~28 bits on Linux x86_64 (mmap base)
Layer 4 / Compiler + Hardware

Control-Flow Integrity (CFI)

Indirect calls and returns are restricted to a set of legitimate targets determined at compile time. Even with full memory corruption, the CPU refuses to call a function whose signature doesn't match the call site, or to return to an address that isn't on a verified list.

Also calledCFG (Windows), CET (Intel), BTI/PAC (ARM) StopsROP, JOP, virtual-call hijacks Bypassed byData-only attacks, call-target collisions, JIT-spraying around the policy Flag-fsanitize=cfi (Clang), /guard:cf (MSVC)
Layer 5 / Allocator

Heap Hardening

Modern allocators reject double-frees, verify metadata, separate metadata from user data, and place freed blocks on a quarantine before reuse. Hardened allocators (PartitionAlloc, jemalloc, mimalloc, scudo) make heap exploitation noticeably harder.

StopsNaive double-frees, heap-metadata smashing, simple use-after-free reuse Bypassed byType-aware heap grooming, cross-cache attacks, sufficient determinism in size-class behavior Used byChrome (PartitionAlloc), Android (Scudo), FreeBSD (jemalloc)
Layer 6 / Language

Memory-Safe Languages

The most effective mitigation is to remove the bug class entirely. Rust, Swift, Go, C#, Java, Python, and JavaScript provide memory safety as a language guarantee: no use-after-free, no buffer overflow, no double-free, no uninitialized read. Microsoft, Google, and the NSA have all publicly recommended migration from C/C++ for new security-sensitive code.

StopsRoughly 70% of historical Microsoft and Chromium CVEs CostsSteeper learning curve (Rust); GC pauses; FFI boundaries still unsafe In productionAndroid Bluetooth, Linux kernel drivers, Windows kernel components, Firefox CSS engine
ASLR Demo — same binary, four runs
Stack Canary Demo — overflow detection in the function epilogue
Pick a scenario above.
Attack → Mitigation Matrix
Exploit TechniqueCanaryDEP/NXASLR/PIECFISafe Lang
Inject + jump shellcodestopsstopspartialstopsstops
ret2libc / ROPpartialnostops (without leak)stopsstops
Use-after-free + vtablenonopartialstopsstops
Format-string read (%s)nonononostops
Format-string write (%n)partialnopartialstopsstops
Race / TOCTOUnonononopartial
Data-only attacksnonononostops

Defense in depth, in practice

No single mitigation is sufficient. Stack canaries catch the simplest overflows but miss heap corruption. DEP kills shellcode injection but lets attackers reuse libc with ROP. ASLR hides addresses until an info-leak gives them up. CFI constrains where control can transfer but doesn't prevent data-only attacks that corrupt application logic without touching code pointers.

What works is the stack: every mitigation makes the next step in a chain expensive. To exploit a modern target an attacker typically needs (1) a corruption primitive, (2) an info-leak to defeat ASLR, (3) ROP gadgets to defeat DEP, (4) a control-flow transfer the CFI policy permits, and (5) a stable heap layout to survive allocator hardening. Each requirement is research; each failure resets the chain.

The most durable defense, however, is upstream: do not write the bug. Memory-safe languages eliminate entire CWE families at compile time. Where C and C++ are mandatory (kernels, performance-critical paths, legacy code), the discipline is: bounded primitives only (strncpy_s, snprintf, std::span), smart pointers for ownership, sanitizers in CI (ASan, UBSan, MSan), fuzzing as a first-class testing input, and aggressive compiler hardening flags by default.

The mitigations on this page are the safety net under the trapeze. They catch what you missed. They are not a substitute for the discipline of writing software that doesn't fall.