Debugging and Tracing on Real Hardware

On a microcontroller there is no operating system to print a stack trace, no core dump, and often no screen. When something goes wrong you get a frozen LED or a device that resets every few seconds. The engineers who are fast at this job are not luckier; they know exactly what their debug probe can see, they have logging that costs microseconds instead of milliseconds, and they follow a method instead of guessing. This lesson covers the debug port and probes, OpenOCD and gdb for embedded targets, trace output over SWO and RTT, how to design a logging system, the end-to-end workflow from a crash record to the faulting line, and the ways the debugger itself changes the behaviour you are trying to observe.

The debug port

Every Cortex-M has a Debug Access Port (DAP) built into the silicon. A probe talks to it over one of two wire protocols:

• SWD (Serial Wire Debug): two signals, SWDIO (bidirectional data) and SWCLK (clock), plus ground and usually NRST. Arm-specific, the default for Cortex-M. On the STM32F407, SWDIO is PA13 and SWCLK is PA14.
• JTAG: four or five signals (TCK, TMS, TDI, TDO, optional nTRST). Older, industry-standard, can daisy-chain several chips on one connector, and supports boundary scan for board testing. On the F407 JTAG shares PA13/PA14 and adds PA15 (TDI), PB3 (TDO) and PB4 (nTRST).

An optional third SWD-era signal, SWO (Serial Wire Output), carries trace data out of the chip. On the F407 it is PB3, the same pin as JTAG’s TDO, which is why you cannot use SWO with JTAG.

Caution

If your firmware reconfigures PA13 or PA14 as GPIO, the probe loses the chip the moment that code runs. Recover by holding the board in reset while the probe connects (“connect under reset”), or by booting from system memory with BOOT0 high. Treat the SWD pins as reserved unless you are desperate for pins.

Probes

Probe	Where you meet it	Notes
ST-Link	Built into Discovery and Nucleo boards	Works with OpenOCD, STM32CubeIDE, pyOCD; SWO up to about 2 MHz on V2
SEGGER J-Link	The professional standard	Fast, unlimited flash breakpoints, native RTT, its own GDB server
CMSIS-DAP / DAPLink	Many vendor boards, cheap open designs	Standard USB protocol, supported by OpenOCD and pyOCD

What the probe can do

Through the DAP, the probe has its own path onto the chip’s bus, independent of the CPU:

• Read and write memory while the core runs. This is how variable watches, peripheral views, and RTT work without stopping your program. It costs bus cycles but no CPU instructions.
• Halt, step, and resume the core, and read or write core registers while halted.
• Hardware breakpoints via the Flash Patch and Breakpoint unit (FPB). The Cortex-M4 has 6 instruction address comparators. Code in flash cannot hold a software breakpoint instruction without reprogramming flash, so every breakpoint you set in flash uses one of these six.
• Software breakpoints (a BKPT instruction written over your code) work only in RAM.
• Flash breakpoints: J-Link can reprogram a flash sector to insert BKPT instructions, giving “unlimited” breakpoints at the cost of flash wear and speed. OpenOCD does not by default.
• Watchpoints via the DWT unit: the M4 has 4 comparators that stop the core when an address is read, written, or both. These are the most underused debugging tool in firmware.

When gdb says “Cannot insert breakpoint” or “Could not insert hardware breakpoints”, you have run out of the six comparators. Delete some.

OpenOCD and gdb

OpenOCD talks to the probe and exposes a gdb server. arm-none-eabi-gdb talks to OpenOCD. Start OpenOCD with the board configuration:

openocd -f board/stm32f4discovery.cfg

Info : clock speed 2000 kHz
Info : STLINK V2J37S0 (API v2) VID:PID 0483:3748
Info : Target voltage: 2.892857
Info : [stm32f4x.cpu] Cortex-M4 r0p1 processor detected
Info : [stm32f4x.cpu] target has 6 breakpoints, 4 watchpoints
Info : Listening on port 3333 for gdb connections

The “6 breakpoints, 4 watchpoints” line is the FPB and DWT you just read about. OpenOCD also listens on port 4444 for a telnet command console. Then, in another terminal:

$ arm-none-eabi-gdb build/app.elf
(gdb) target extended-remote :3333
(gdb) monitor reset halt
target halted due to debug-request, current mode: Thread
xPSR: 0x01000000 pc: 0x080002a0 msp: 0x20020000
(gdb) load
Loading section .isr_vector, size 0x188 lma 0x8000000
Loading section .text, size 0x1a4c lma 0x8000188
Loading section .data, size 0x10 lma 0x8001bd4
Start address 0x080002a0, load size 7140
(gdb) break main
Breakpoint 1 at 0x80003b4: file src/main.c, line 41.
(gdb) continue
Breakpoint 1, main () at src/main.c:41
41      clock_init();

monitor passes the rest of the line to OpenOCD, so monitor reset halt resets the chip and stops it at the first instruction, with pc at the reset vector and msp at the initial stack value. load writes every loadable section of the ELF into flash, which is why gdb needs the ELF and not the .bin: the ELF knows the addresses and has the debug information.

The commands you will use every day:

Command	What it does
break file.c:57, break func	Breakpoint (in flash this uses a hardware comparator)
hbreak func	Explicitly request a hardware breakpoint
watch var / rwatch var / awatch var	Stop when var is written / read / either (DWT)
watch -l ptr->field	Watch the address of the expression, not the expression
info registers, p $pc, p/x $sp	Core registers
x/8xw 0x08000000	Examine 8 words in hex at an address
p/x *(uint32_t *)0x40020C14	Read GPIOD ODR (if uint32_t is in the debug info; otherwise use unsigned int)
set var counter = 0	Change memory
bt, frame 2, info locals	Call stack and the locals of a frame
finish, next, step, stepi	Run to return, step over, step into, one instruction
display/i $pc	Show the current instruction after every step
monitor mdw 0x40020C14	OpenOCD reads a word directly, bypassing gdb’s caches
monitor mww 0x40020C18 0x1000	OpenOCD writes a word (here: BSRR, set PD12)
monitor reset run	Reset and let it run

Peripheral registers deserve care. gdb may cache memory reads, and some registers change by themselves; monitor mdw always goes straight to the hardware. Reading a register can also have side effects, which is the subject of the heisenbug section below.

Helpers: .gdbinit and Python

Typing the same five commands at the start of every session gets old. Put them in a .gdbinit in the project directory:

.gdbinit

target extended-remote :3333
monitor reset halt
load
source tools/fault.py
break HardFault_Handler

For security, gdb refuses to run a project-local .gdbinit unless you allow it, for example with add-auto-load-safe-path /path/to/project in your home directory’s ~/.gdbinit. gdb will tell you exactly which line to add when it refuses.

gdb embeds Python, which makes it easy to write commands that know about your hardware. This one decodes the fault status registers from Watchdogs, faults, and recovering from them the moment you hit HardFault_Handler:

tools/fault.py

import gdb

CFSR_BITS = {
    0: "IACCVIOL", 1: "DACCVIOL", 3: "MUNSTKERR", 4: "MSTKERR", 5: "MLSPERR",
    7: "MMARVALID", 8: "IBUSERR", 9: "PRECISERR", 10: "IMPRECISERR",
    11: "UNSTKERR", 12: "STKERR", 13: "LSPERR", 15: "BFARVALID",
    16: "UNDEFINSTR", 17: "INVSTATE", 18: "INVPC", 19: "NOCP",
    24: "UNALIGNED", 25: "DIVBYZERO",
}


def read_u32(addr):
    return int(gdb.parse_and_eval(f"*(unsigned int *){addr:#x}")) & 0xFFFFFFFF


class Fault(gdb.Command):
    """Decode the Cortex-M fault status registers."""

    def __init__(self):
        super().__init__("fault", gdb.COMMAND_USER)

    def invoke(self, arg, from_tty):
        cfsr, hfsr = read_u32(0xE000ED28), read_u32(0xE000ED2C)
        mmfar, bfar = read_u32(0xE000ED34), read_u32(0xE000ED38)
        names = [n for bit, n in CFSR_BITS.items() if cfsr & (1 << bit)]
        print(f"CFSR  = {cfsr:#010x}  {' '.join(names) or '-'}")
        print(f"HFSR  = {hfsr:#010x}  {'FORCED' if hfsr & (1 << 30) else ''}")
        if cfsr & (1 << 7):
            print(f"MMFAR = {mmfar:#010x}")
        if cfsr & (1 << 15):
            print(f"BFAR  = {bfar:#010x}")


Fault()

The addresses are the SCB fault registers (CFSR at 0xE000ED28, HFSR at 0xE000ED2C, MMFAR at 0xE000ED34, BFAR at 0xE000ED38). The fault address registers are printed only when their VALID bit says the content means something. Type fault at the gdb prompt to run it.

VS Code and Cortex-Debug

The Cortex-Debug extension wraps the same OpenOCD and gdb in a graphical front end. You set it up in Setting up your toolchain; the two settings that matter most here are the SVD file and the run-to point:

.vscode/launch.json

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug (OpenOCD)",
      "type": "cortex-debug",
      "request": "launch",
      "servertype": "openocd",
      "cwd": "${workspaceFolder}",
      "executable": "build/app.elf",
      "configFiles": ["board/stm32f4discovery.cfg"],
      "svdFile": "STM32F407.svd",
      "runToEntryPoint": "main"
    }
  ]
}

The SVD file (System View Description, published by the vendor) describes every peripheral register and bit field. With it, Cortex-Debug shows a peripheral tree where you can expand GPIOD and see MODER12 = Output instead of a hex word. It is the fastest way to check that your configuration actually landed in the hardware.

Getting text out of the chip

You will want a printf. There are four ways to get one, with very different costs.

Semihosting

Semihosting lets the target use the host’s I/O through the debugger. Your code puts an operation number in r0, a pointer to arguments in r1, and executes BKPT 0xAB. The core halts, the debugger sees the special breakpoint, performs the operation on the host (print a string, open a file, read the clock), writes the result back, and resumes the core.

To use it: link with --specs=rdimon.specs instead of nosys.specs, call initialise_monitor_handles() at the start of main, and enable it in OpenOCD with monitor arm semihosting enable. Then printf output appears in the OpenOCD console.

It is convenient and dreadfully slow: every call halts the core and round-trips over USB, so each printf costs milliseconds during which nothing runs, interrupts included. Worse, a BKPT with no debugger attached is a debug event the core cannot handle, which escalates to a HardFault. A build with semihosting crashes the moment it runs without the probe. Use it for quick host-assisted experiments; never let it near a release build.

ITM over SWO

The Cortex-M3/M4 ITM (Instrumentation Trace Macrocell) has 32 stimulus ports. Writing a byte or word to a stimulus port register sends it out through the TPIU (Trace Port Interface Unit) onto the SWO pin as a serial bit stream the probe captures. The CPU cost is a single store, plus a short wait if the small trace FIFO is full. It does not halt the core and needs no UART.

CMSIS provides ITM_SendChar(). The probe software usually configures the TPIU for you, but configuring it in firmware makes it independent of the tool:

swo.c

#include <stdint.h>

#include "stm32f4xx.h"

void swo_init(uint32_t cpu_hz, uint32_t swo_hz)
{
    CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;   /* power the trace blocks   */
    DBGMCU->CR |= DBGMCU_CR_TRACE_IOEN;               /* async trace: PB3 is SWO  */
    TPI->SPPR = 2u;                                   /* NRZ (UART-like) encoding */
    TPI->ACPR = cpu_hz / swo_hz - 1u;                 /* SWO bit-rate prescaler   */
    TPI->FFCR = 0x100u;                               /* formatter off            */
    ITM->LAR = 0xC5ACCE55u;                           /* unlock ITM registers     */
    ITM->TCR = ITM_TCR_ITMENA_Msk | ITM_TCR_SYNCENA_Msk | ITM_TCR_TSENA_Msk |
               (1u << 16);                            /* trace bus ID 1           */
    ITM->TER = 1u;                                    /* enable stimulus port 0   */
}

int _write(int fd, const char *buf, int len)
{
    (void)fd;
    for (int i = 0; i < len; i++) {
        ITM_SendChar((uint32_t)(uint8_t)buf[i]);
    }
    return len;
}

TRCENA in DEMCR powers the DWT, ITM and TPIU; it is the same bit you set for DWT->CYCCNT in Clocks, reset, and timing. On the F4 the trace clock is the CPU clock, so swo_init(168000000u, 2000000u) gives a prescaler of 83 and a 2 MHz SWO stream, the most an ST-Link V2 accepts. The probe side must be told the same frequency; a mismatch produces garbage, exactly like a wrong UART baud rate. TSENA enables local timestamps, so the host can tag each message with the cycle at which it was emitted, which is something no UART log can do. _write is the newlib retarget hook from The C standard library on a microcontroller, so printf now goes to SWO. ITM_SendChar checks that ITM and port 0 are enabled and returns immediately if not, so a build that never ran swo_init does not hang.

To receive it, use OpenOCD’s TPIU/ITM support (its trace commands changed between versions, so check the manual for yours), Cortex-Debug’s SWO configuration, STM32CubeProgrammer’s SWV viewer, or J-Link SWO Viewer. Check that SWO is actually wired from PB3 to your probe; on some boards it is routed through a solder bridge or not at all. The DWT can also emit periodic PC samples and exception entry/exit events through the same path, which is the foundation of non-intrusive profiling.

SEGGER RTT

RTT (Real-Time Transfer) uses no pins at all. The firmware keeps a control block in RAM with a few ring buffers (“up” buffers to the host, “down” buffers from it), identified by the string "SEGGER RTT". Writing a log message is a memcpy into a ring buffer, typically a microsecond or two. The probe finds the control block by scanning RAM, then polls the buffers using background memory access while the core runs.

The SEGGER RTT source is free to use; you add SEGGER_RTT.c to your build and call SEGGER_RTT_WriteString(0, "boot\n") or SEGGER_RTT_printf(0, "x=%d\n", x). J-Link supports it natively, and OpenOCD can serve it as a TCP port:

(gdb) monitor rtt setup 0x20000000 0x20000 "SEGGER RTT"
(gdb) monitor rtt start
(gdb) monitor rtt server start 9090 0

Then nc localhost 9090 (or any TCP terminal) shows channel 0. If nobody reads the buffer, RTT by default drops data when the buffer is full rather than blocking, so it is safe to leave in a build that runs without a probe.

Choosing

Method	Pins	Cost per short message	Needs a probe	Usable in the field
UART printf (polled)	TX	about 1 ms at 115200 baud for 10 characters	No	Yes
UART with DMA and a ring buffer	TX	a few microseconds	No	Yes
Semihosting	None	milliseconds, core halted	Yes, or it crashes	No
ITM / SWO	SWO	a few microseconds	Yes (to read)	No
RTT	None	a microsecond or two	Yes (to read)	No

In development, RTT or SWO for high-rate debug output. In the field, a UART or a log stored in flash, because there is no probe.

Designing a logging system

printf scattered through the code is not a logging system. A real one has:

• Levels (error, warning, info, debug), so you can turn noise down without deleting lines.
• Module tags, so you can tell the UART driver’s messages from the sensor’s.
• Compile-time filtering, so debug messages cost zero flash and zero cycles in a release build.
• Deferred formatting, so logging from time-critical code costs a few stores, and the expensive formatting happens later in the idle loop.

log.h

#ifndef LOG_H
#define LOG_H

#include <stdint.h>

enum { LOG_ERR = 0, LOG_WRN = 1, LOG_INF = 2, LOG_DBG = 3 };

#ifndef LOG_LEVEL
#define LOG_LEVEL LOG_INF            /* override per build: -DLOG_LEVEL=LOG_DBG */
#endif
#ifndef LOG_MODULE
#define LOG_MODULE "app"             /* define before including, per source file */
#endif

void log_push(uint8_t level, const char *fmt, uint32_t a0, uint32_t a1);
void log_drain(void);                /* call from the idle loop */

#define LOG_AT(lvl, fmt, a0, a1)                                              \
    do {                                                                      \
        if ((lvl) <= LOG_LEVEL) {                                             \
            log_push((lvl), LOG_MODULE ": " fmt "\n", (uint32_t)(a0),         \
                     (uint32_t)(a1));                                         \
        }                                                                     \
    } while (0)

#define LOG_E(fmt, a0, a1) LOG_AT(LOG_ERR, fmt, a0, a1)
#define LOG_W(fmt, a0, a1) LOG_AT(LOG_WRN, fmt, a0, a1)
#define LOG_I(fmt, a0, a1) LOG_AT(LOG_INF, fmt, a0, a1)
#define LOG_D(fmt, a0, a1) LOG_AT(LOG_DBG, fmt, a0, a1)

#endif

LOG_MODULE ": " fmt "\n" uses string-literal concatenation, so the tag costs nothing at run time; it is part of the same string in .rodata. The if ((lvl) <= LOG_LEVEL) compares two constants. When the level is filtered out, the compiler removes the call and the format string is never referenced, so neither ends up in flash. That is compile-time filtering with no #ifdef at each call site.

log.c

#include <stdint.h>
#include <stdio.h>

#include "log.h"
#include "stm32f4xx.h"

typedef struct {
    uint32_t    cycles;
    const char *fmt;
    uint32_t    a0, a1;
} log_rec_t;

#define LOG_DEPTH 64u                        /* power of two */
static log_rec_t ring[LOG_DEPTH];
static volatile uint32_t head, tail;
static volatile uint32_t dropped;

void log_push(uint8_t level, const char *fmt, uint32_t a0, uint32_t a1)
{
    (void)level;
    uint32_t primask = __get_PRIMASK();      /* callable from ISRs and main */
    __disable_irq();
    if (head - tail < LOG_DEPTH) {
        log_rec_t *r = &ring[head & (LOG_DEPTH - 1u)];
        r->cycles = DWT->CYCCNT;
        r->fmt = fmt;
        r->a0 = a0;
        r->a1 = a1;
        head++;
    } else {
        dropped++;                           /* never block, count instead */
    }
    __set_PRIMASK(primask);
}

void log_drain(void)
{
    while (tail != head) {
        const log_rec_t *r = &ring[tail & (LOG_DEPTH - 1u)];
        printf("[%10lu] ", (unsigned long)r->cycles);
        printf(r->fmt, (unsigned long)r->a0, (unsigned long)r->a1);
        tail++;
    }
}

Logging is now about a dozen instructions with interrupts briefly masked, safe from any ISR, and the slow printf runs in log_drain from the idle loop. Three rules come with deferred formatting:

• Arguments are copied as numbers, so format strings must use %lu, %lx or %ld to match the unsigned long they receive. The compiler cannot check deferred formats, so be consistent.
• Never pass a pointer to a temporary string (%s with a stack buffer). By the time log_drain runs, the buffer is gone. Only string literals are safe.
• Count drops. A full ring discards and increments dropped; print the count so you know the log is incomplete.

Take the idea one step further and you get binary logging: instead of formatting on the target, send the raw record (the address of the format string plus the arguments) over UART or RTT, and let a host tool look the string up in the ELF file. The format strings can then live in a section that is never loaded into flash at all, a log record shrinks to about 12 bytes, and logging becomes cheap enough to leave on in production. Rust’s defmt, the trice tool, and Pigweed’s tokenizer all work this way. For RTOS-level behaviour (which task ran when, how long each ISR took), tools like Percepio Tracealyzer and SEGGER SystemView record kernel events through RTT or a RAM buffer and draw timelines.

The fault analysis workflow

Your device has crashed in the field and, thanks to the crash record from Watchdogs, faults, and recovering from them, it reports on the next boot:

CRASH: pc=0x08000a3c lr=0x08000a1f cfsr=0x00008200 bfar=0x6c6c6548 r3=0x6c6c6548

Work it end to end.

1. Decode the status. CFSR = 0x00008200 sets bits 15 (BFARVALID) and 9 (PRECISERR): a precise bus fault, and BFAR holds the exact address that was accessed. 0x6C6C6548 is in the external memory region where nothing is mapped on this board, so the bus returned an error.

2. Find the line. addr2line maps an address to a source line using the ELF’s debug information. Give it the faulting pc and the return address in lr (which tells you the caller):

arm-none-eabi-addr2line -e build/app.elf -f -i -p 0x08000a3c 0x08000a1e

sensor_read at /home/me/fw/src/sensor.c:57
app_sample at /home/me/fw/src/app.c:112

-f prints the function name, -i unwinds inlined functions, -p prints one readable line per address. For lr, clear the Thumb bit (and, for exactness, subtract 2 to land on the call instruction rather than after it). The ELF must be exactly the one that was flashed, which is why release builds must archive their ELF files.

3. Read the instruction. The disassembly around pc shows what the CPU was doing:

arm-none-eabi-objdump -d -S --start-address=0x08000a30 --stop-address=0x08000a44 build/app.elf

    return dev->bus->read(dev->addr, reg);
 8000a3a:  6843        ldr  r3, [r0, #4]
 8000a3c:  681b        ldr  r3, [r3, #0]
 8000a3e:  7800        ldrb  r0, [r0, #0]
 8000a40:  4798        blx  r3

At 0x08000a3a the code loaded dev->bus into r3; at 0x08000a3c it tried to load bus->read through it, and r3 held 0x6C6C6548. So dev->bus was corrupt.

4. Recognise the value. 0x6C6C6548 stored little-endian is the bytes 48 65 6C 6C: ASCII "Hell". Something wrote text containing "Hello" over the pointer. That smells like a string copy that overflowed into the structure next to it in RAM. Values that decode to ASCII, 0xA5A5A5A5 (stack paint), 0xFFFFFFFF (erased flash), or 0xDEADBEEF (a fill pattern) are clues; learn to spot them.

5. Catch the writer. Reproduce on the bench, then put a watchpoint on the corrupted field:

(gdb) watch -l sensor0.bus
Hardware watchpoint 2: -location sensor0.bus
(gdb) continue
Hardware watchpoint 2: -location sensor0.bus
Old value = (const bus_ops_t *) 0x8001c40 <i2c_bus_ops>
New value = (const bus_ops_t *) 0x6c6c6548
0x08001234 in memcpy ()
(gdb) bt
#0  0x08001234 in memcpy ()
#1  0x08000c12 in shell_set_name (arg=0x200004c1 "Greetings & Hello") at src/shell.c:88

-l watches the address, so the watchpoint survives leaving the scope where you typed it. The DWT stops the core on the exact store that changed the value, and the backtrace shows the culprit: a shell command copying an unbounded string into an 8-byte name buffer that sits just before sensor0 in .bss. Bytes 12 to 15 of "Greetings & Hello" are "Hell", and they land exactly on sensor0.bus. The map file confirms the neighbours. Fix the copy with a length check, then prove the fix by sending the same command again and seeing the crash record stay empty.

That sequence, crash record to pc, addr2line, disassembly, register values, a hypothesis, and a watchpoint to confirm it, resolves the majority of memory corruption bugs in under an hour.

More techniques

GPIO toggles and a scope. To measure how long something takes, or whether an ISR runs when you think it does, set a pin at the start and clear it at the end with single BSRR writes (each costs a couple of cycles). A scope or logic analyzer then shows duration, period, and jitter with nanosecond resolution and zero effect on timing, which no log can match. Use one pin per ISR and you can see pre-emption directly.

Logic analyzers. A cheap 8-channel analyzer with sigrok/PulseView (or a Saleae with its own software) decodes UART, SPI, I2C and CAN into bytes. When a sensor does not answer, the analyzer settles in seconds whether the problem is on your side (wrong address, wrong SPI mode, no clock) or on the device side (no ACK). Look at the bus before you look at the code.

printf debugging done right. It is a legitimate tool when you respect its costs: log state transitions rather than every loop iteration, rate-limit anything periodic, never call a blocking printf from an ISR, and prefer deferred logging. Remember that adding a print changes timing, which can make a race disappear.

Debugging optimised code. At -O2, gdb shows <optimized out> for variables, lines execute out of order, and stepping jumps around. Options, in order of preference:

• Build with -Og, which optimises without destroying debuggability, for day-to-day work.
• Temporarily make one variable volatile so the compiler keeps it in memory where gdb can see it.
• Mark one function __attribute__((optimize("O0"))) to debug it unoptimised while the rest stays at -O2. GCC documents this attribute as a debugging aid, not for production code.
• Learn to read the disassembly with display/i $pc and stepi, and inspect registers directly. Bugs that only appear at -O2 (usually undefined behaviour, see Undefined behavior, volatile, and what the optimizer assumes) must be debugged at -O2.

Heisenbugs: when the debugger changes the bug

A heisenbug disappears or changes when you try to observe it. On microcontrollers the debugger itself is a common cause.

• Reading a register can have side effects. Reading USART SR then DR clears RXNE and ORE. If the peripheral view in your IDE refreshes USART2 while you are stepping, it consumes received bytes and your driver never sees them. Close live register views of peripherals you are debugging, or know which registers clear on read.
• Halting stops the CPU, not the world. Timers keep counting, the UART keeps receiving (and overruns), DMA keeps transferring, and the independent watchdog keeps counting down and resets the chip a second after you hit a breakpoint. The STM32 provides freeze bits so selected peripherals stop when the core halts:

#include "stm32f4xx.h"

void debug_freeze_init(void)
{
    DBGMCU->APB1FZ |= DBGMCU_APB1_FZ_DBG_IWDG_STOP |   /* watchdog pauses at breakpoints */
                      DBGMCU_APB1_FZ_DBG_TIM2_STOP;    /* so does TIM2 */
    DBGMCU->CR |= DBGMCU_CR_DBG_SLEEP | DBGMCU_CR_DBG_STOP | DBGMCU_CR_DBG_STANDBY;
}

• Low-power modes behave differently. With DBG_SLEEP, DBG_STOP and DBG_STANDBY set, the debugger keeps clocks running in low-power modes so the connection survives, which means the current you measure with the probe attached is not the real figure. The DBGMCU registers are reset only by power-on, not by a system reset, so a bit set by an earlier debug session persists until you power-cycle the board. Power-cycle before measuring current.
• Timing changes. A breakpoint in an ISR stops the core for seconds; a protocol peer times out, a motor control loop goes unstable. Use watchpoints with conditions, trace, or GPIO toggles for timing-sensitive code.
• Memory layout changes. A debug build at -O0 puts variables in different places than a release build, so an overflow corrupts a different neighbour and the symptom moves.

A systematic method

When the bug is not obvious after ten minutes, stop poking and follow a method:

1. Reproduce. Find a reliable way to trigger it. A bug you can trigger in one minute is half solved; an intermittent bug needs a stress loop, a script, or a fault-injection test until it becomes reproducible.
2. Isolate. Shrink the problem: disable modules, bisect commits with git bisect, swap hardware, move the test to the host. Each step halves the search space.
3. Hypothesise. State a specific, falsifiable cause: “the ISR fires twice because the flag is cleared at the end of the handler.”
4. Instrument. Design an observation that distinguishes your hypothesis from the alternatives: a watchpoint, a GPIO toggle, a counter, a trace. Predict the outcome before you run it.
5. Fix the root cause, not the symptom. A delay_ms(10) that makes the problem go away is a symptom fix that will return.
6. Prove. Show the original reproduction no longer fails, add a regression test where possible, and check that you understand why the fix works. If you cannot explain it, you have not found the bug.

Write down what you tried. Debugging notes turn a three-day hunt into a post-mortem the next engineer can learn from.

Common mistakes

• Using semihosting in a build that runs without a probe. The first printf hits BKPT and HardFaults. Use RTT, SWO, or a UART instead, and keep rdimon.specs out of release builds.
• Running out of hardware breakpoints. In flash every breakpoint uses one of 6 FPB comparators. Delete old breakpoints, or use a J-Link with flash breakpoints.
• Watchdog resets at every breakpoint. Set DBG_IWDG_STOP in DBGMCU->APB1FZ in debug builds.
• A live peripheral view that eats data. The IDE reads DR and clears flags. Close the view or avoid read-sensitive registers.
• Wrong ELF for addr2line. A rebuilt ELF has different addresses. Archive the exact ELF of every released build.
• Measuring sleep current with the debugger attached. The probe keeps clocks alive. Disconnect and power-cycle before measuring.
• SWO shows garbage. The probe’s SWO frequency does not match TPI->ACPR, or the CPU clock is not what you assumed. Check both, exactly like a UART baud mismatch.

Exercises

1. Start OpenOCD and gdb on your board (or Renode with its gdb server), set a breakpoint on main, and use x/2xw 0x08000000 to read the initial stack pointer and reset vector. Confirm they match p/x $msp and p/x $pc right after monitor reset halt.
2. Set a watchpoint with watch -l on a global counter that an ISR increments, run, and read the backtrace when it triggers. Then use rwatch on the same variable and find the code that reads it. Count how many watchpoints you can set before OpenOCD refuses.
3. Add swo_init and the _write retarget to your blinky, print a message every second, and receive it with your tool of choice. Then deliberately set the probe’s SWO frequency to half the correct value and describe what you see.
4. Implement log.h and log.c, log from both a SysTick handler and main, and measure the cost of one LOG_I call with DWT->CYCCNT (expect a few dozen cycles). Build once with -DLOG_LEVEL=LOG_ERR and use arm-none-eabi-size and arm-none-eabi-strings on the ELF to confirm the info and debug format strings are gone.
5. Hard: write a function that corrupts a global pointer through an overflowing strcpy, crash the board through it, decode the crash with your HardFault handler’s output, addr2line, objdump, and the fault gdb command, then find the writer with a watchpoint. Write up each step as in the workflow section.

Check yourself

Question 1:You set a seventh breakpoint in flash code on an STM32F407 through OpenOCD and gdb reports it cannot insert it. Why?

Flash cannot be read by the debugger while the core runs The Cortex-M4 FPB has 6 instruction comparators, and every breakpoint in flash needs one OpenOCD limits breakpoints to protect flash endurance Breakpoints only work in RAM

Question 2:A firmware build works perfectly with the debugger attached and HardFaults immediately when powered on its own. It prints with printf and links with --specs=rdimon.specs. What is happening?

The clock configuration depends on the debugger Semihosting executes BKPT, which without a debugger escalates to a HardFault printf requires the debugger to allocate heap memory The vector table is only valid when the debugger loads it

Question 3:While you step through your UART receive code, the driver never sees any received bytes, although a logic analyzer shows them on the wire. The IDE shows a live view of the USART2 registers. What is the most likely cause?

Stepping disables the UART clock The live view reads SR and DR, which clears RXNE and consumes the bytes before your code reads them The logic analyzer loads the RX line Breakpoints mask all interrupts permanently

Question 4:A crash record shows bfar=0x6c6c6548 and CFSR with PRECISERR and BFARVALID set. What is the best next step after finding the faulting line?

Increase the stack size and hope Rebuild at -O0 so the bug goes away Notice that the value is ASCII text, reproduce, and set a watchpoint on the corrupted pointer to catch the code that writes it Replace the pointer with a global variable

Question 5:Why does the deferred logging design forbid %s with a pointer to a local buffer?

%s is not supported by newlib-nano Strings are too long to fit in a log record The string is formatted later in the idle loop, after the local buffer has gone out of scope and been overwritten %s would make logging non-reentrant

Key takeaways

• The debug probe accesses memory over its own bus path, so it can read and write RAM and peripherals while the core runs; the M4 gives you 6 hardware breakpoints and 4 data watchpoints.
• Data watchpoints (watch -l) find memory corruption faster than any other technique: they stop on the exact store.
• Semihosting halts the core on every call and crashes without a probe; use SWO or RTT for development output and a UART or flash log in the field.
• A good logging system has levels, module tags, compile-time filtering, and deferred formatting, and it counts what it drops.
• Fault analysis is mechanical: decode CFSR, addr2line the pc and lr, read the disassembly and registers, recognise the bad value, catch the writer.
• The debugger changes the system: register reads have side effects, halting does not stop peripherals or the watchdog, and debug bits keep clocks alive in low-power modes.

Next: Power management, where you make the device spend most of its life asleep and measure what that actually buys you.