The Debugger Architecture: Four Components, One Mediator

▶ Watch (01:47)

A native debugger sits above four components: the debugger itself, the running process (code plus data), the binary on disk, and the source files. None of those four talk to each other directly. Every read, every write, every signal goes through the operating system. The OS controls hardware access, and that mediation is what makes debugger operations possible.

The key asymmetry is that the binary carries debug information and the process does not. The debugger’s job is to translate machine-level state (addresses, registers, raw bytes) into the source-level view a developer expects.

How Software Breakpoints Actually Work

▶ Watch (06:09)

Setting a breakpoint on line 149 of petcat.c starts with the DWARF line table, a custom bytecode format the parser must interpret. See the DWARF line table lookup for line 149 of petcat.c (08:26) for how the dump looks. That table maps source line 149 to machine address 0x1886.

The debugger saves the byte at 0x1886, overwrites it with 0xCC (INT3), and resumes the process. When the CPU hits INT3, the OS signals the process to stop and the debugger intercepts. On resume: restore the original byte, move the instruction pointer back one byte, single-step over the real instruction via ptrace, re-insert 0xCC, then continue. Software breakpoints are unlimited in number but fire only on execution, not reads or writes.

Watchpoints and Hardware Debug Registers

▶ Watch (14:37)

x86 provides exactly four hardware debug registers: DR0 through DR3. Each holds a memory address. A fifth register, DR7, holds enable and condition bits per register, controlling whether the break fires on execution, read, write, or both, and over how many bytes. When a condition triggers, a status register records which DR fired.

That is what makes watchpoints useful for memory corruption bugs. Set a watchpoint on a variable, and the next write stops the process immediately, pointing at the offending instruction. The limit: four hardware breakpoints total, no exceptions.

Stack Unwinding and Call Frame Information

▶ Watch (21:07)

Reconstructing the call chain means walking from the current frame back to start and restoring register state at each level. Frame pointers help locate return addresses but cannot recover clobbered registers. DWARF call frame information (CFI) handles that: a compiler-generated bytecode table that maps every instruction pointer range to rules for restoring each register.

“And it’s horrifying.” — Sy Brand

For each frame, CFI gives expressions like “CFA = stack pointer plus 8, return address at CFA minus 8.” The rules chain all the way up to start. Some registers may be gone, but CFI provides the best reconstruction possible.

Calling a Function Inside a Running Process

▶ Watch (35:01)

The call must run inside the target process, not a copy. LLDB calls out to Clang for full expression evaluation.

“Unfortunately, this is kind of right.” — Sy Brand

A simpler path uses the System V ABI: look up the function’s DWARF entry and its mangled Itanium C++ name, locate the target object via variable lookup, load the pointer into RDI and the integer argument into RSI, point the instruction pointer at the function start, continue, then read the return value from RAX. The same DWARF lookup and ptrace register writes used throughout the rest of the debugger make it work.

Q&A

Once you replace the INT3 byte with the original and single-step, how do you re-insert the breakpoint before continuing? Brand explained that ptrace’s single-step mode returns control to the debugger after exactly one instruction, at which point the debugger re-inserts 0xCC before calling continue. ▶ 13:48

Is call frame information stored per-instruction or only at return points? CFI is encoded as bytecode that produces rules for ranges of addresses, not individual instructions; the rule applies until the next entry says otherwise. ▶ 28:46

Notable Quotes

And it’s horrifying. Sy Brand · ▶ 8:04

Unfortunately, this is kind of right. Sy Brand · ▶ 36:26

Thank you. This is how debuggers work. Sy Brand · ▶ 0:12

Key Takeaways

  • Software breakpoints overwrite one byte with 0xCC (INT3), trap the CPU, and restore the original byte before resuming.
  • x86 provides exactly four hardware debug registers, enabling read/write watchpoints that software breakpoints cannot do.
  • DWARF call frame information is Turing-complete bytecode that tells the debugger how to restore every register for every frame.