Concurrency, Atomics, and Memory Ordering

You do not need an RTOS to have concurrency bugs. The moment an interrupt handler and main share a variable, you have two flows of execution touching the same memory, and every assumption about “this happens before that” is up for negotiation with the compiler and the CPU. This lesson is about that negotiation: which memory accesses are atomic on Cortex-M, why volatile does not make shared data safe, what C11 atomics and memory orders actually guarantee, how LDREX and STREX build atomic operations in hardware, how to write a lock-free ring buffer that is correct rather than lucky, when each barrier instruction is required, and how to make races show up on the bench instead of in the field.

ISRs and main are concurrent

A data race is two accesses to the same memory location, at least one a write, not ordered by any synchronization. In C11 a data race on a non-atomic object is undefined behavior, which means the compiler may assume it never happens. On a microcontroller the most common race is between an ISR and main:

#include <stdint.h>
#include "stm32f4xx.h"

volatile uint32_t events;               /* counted by main AND by the ISR */

void TIM2_IRQHandler(void)
{
    TIM2->SR = ~TIM_SR_UIF;             /* clear the update flag (rc_w0: write 0 clears) */
    events++;
}

void on_button(void)                    /* called from main */
{
    events++;
}

volatile makes the compiler perform every access, so events++ in on_button compiles to three instructions:

    ldr     r2, =events        @ address of events
    ldr     r3, [r2]           @ 1. load N
    adds    r3, r3, #1         @ 2. compute N+1   <-- TIM2 interrupt here: memory becomes N+1
    str     r3, [r2]           @ 3. store N+1, overwriting the ISR's increment

If the timer interrupt lands between the load and the store, the ISR’s increment is lost: both increments happened, but the counter went up by one. The window is two instructions wide, a few nanoseconds at 168 MHz, so with a 1 kHz timer it might fail once in a hundred thousand button presses. That is exactly what makes it dangerous. It passes every test you run and fails in the field.

Torn reads and writes

A torn access is one that the hardware performs as several smaller accesses, so a concurrent writer can be observed half-done.

64-bit values

A Cortex-M4 has 32-bit loads and stores. A uint64_t is read with two loads (or one LDRD, which the architecture does not guarantee to be atomic either):

volatile uint64_t uptime_us;            /* incremented by a timer ISR */

uint64_t now = uptime_us;               /* two 32-bit loads */

Suppose the value is 0x00000000_FFFFFFFF and the ISR increments it to 0x00000001_00000000 between the two loads. Reading low then high gives 0x00000001_FFFFFFFF, about 2^32 microseconds in the future. Reading high then low gives 0x00000000_00000000, a jump into the past. Either way, a timestamp is off by 71 minutes, once every 71 minutes.

Two correct fixes. Mask interrupts for the read:

uint64_t uptime_read(void)
{
    const uint32_t primask = __get_PRIMASK();
    __disable_irq();
    const uint64_t v = uptime_us;
    __set_PRIMASK(primask);
    return v;
}

Or, if the ISR maintains two 32-bit halves (lo++; if (lo == 0) hi++;), read without masking by retrying until the high word is stable:

extern volatile uint32_t uptime_hi, uptime_lo;

uint64_t uptime_read_lockfree(void)
{
    uint32_t hi, lo;
    do {
        hi = uptime_hi;
        lo = uptime_lo;
    } while (hi != uptime_hi);          /* low word wrapped while we read: try again */
    return ((uint64_t)hi << 32) | lo;
}

If the low word wraps anywhere during the read, the high word changes and the loop repeats. This works because all three reads are volatile, so the compiler keeps them in program order, and because a single core observes its own ISR’s writes in order.

Structs, bitfields and packed data

A struct copy is many accesses. If an ISR updates accel.x, accel.y, accel.z and main copies the struct, main can get x from the new sample and y and z from the old one. A bitfield write is a read-modify-write of the whole containing word, so flags.ready = 1 in main can erase flags.error = 1 written by an ISR in between. A member of a packed struct at an odd offset is accessed with byte loads or an unaligned access that the bus splits into several transfers; neither is atomic.

What is atomic on Cortex-M

Access	Atomic with respect to interrupts on M3/M4?	Why
Aligned 8, 16 or 32-bit load or store (LDRB, LDRH, LDR, STRB, STRH, STR)	yes	single-copy atomic: one bus transfer
64-bit uint64_t or double load or store	no	two 32-bit transfers
Unaligned 16 or 32-bit access	no	split into several transfers
x++, x += n, x |= mask	no	separate load, modify, store
Bitfield assignment	no	read-modify-write of the containing word
Struct copy or memcpy	no	many transfers
LDM, STM, PUSH, POP	no	can be interrupted between registers and continued or restarted

The rule to memorize: a single aligned load or store of 32 bits or less is atomic; everything else is not. A volatile uint32_t flag written by an ISR and read by main cannot be torn. A counter incremented on both sides can lose updates. Anything larger than a word, or any read-modify-write, needs either an atomic operation or a critical section.

volatile is not enough

volatile has one precise meaning, covered in Undefined behavior, volatile, and what the optimizer assumes: every volatile access in the source is performed, and volatile accesses are not reordered relative to each other. It says nothing about non-volatile accesses around them. Consider the most common ISR-to-main handoff:

#include <stdbool.h>
#include <stdint.h>

static uint8_t       frame[32];          /* plain memory, filled by the ISR */
static volatile bool frame_ready;

void USART2_IRQHandler(void)             /* producer */
{
    /* ... store received bytes into frame[0..31] ... */
    frame_ready = true;
}

void poll_frames(void)                   /* consumer, called from main */
{
    if (frame_ready) {
        process(frame);
        frame_ready = false;
    }
}

Three things are wrong, and volatile fixes none of them:

1. The compiler may sink the frame stores below the flag store in the ISR. They are non-volatile, so their order relative to the volatile store is not constrained. main could see frame_ready == true while the last bytes have not been written yet.
2. The compiler may hoist frame loads above the flag check in poll_frames, especially if process is inlined. It is allowed to assume no other thread writes frame (that would be a data race), so loading early is legal.
3. The protocol races. After main sees the flag, the ISR can start writing the next frame into the same buffer while main is still processing it.

On a Cortex-M4 at -O2, problems 1 and 2 happen rarely in practice, which is the worst kind of bug: correct today, broken after a compiler upgrade or when someone marks a function static inline. The fix is to make the flag an atomic with ordering that covers the surrounding memory:

#include <stdatomic.h>
#include <stdbool.h>
#include <stdint.h>

static uint8_t     frame[32];
static atomic_bool frame_ready;          /* zero-initialized: false */

void USART2_IRQHandler(void)             /* producer */
{
    if (!atomic_load_explicit(&frame_ready, memory_order_acquire)) {
        /* ... slot is free: store received bytes into frame[] ... */
        atomic_store_explicit(&frame_ready, true, memory_order_release);
    }
    /* else: main still owns the frame; count an overrun */
}

void poll_frames(void)                   /* consumer */
{
    if (atomic_load_explicit(&frame_ready, memory_order_acquire)) {
        process(frame);                  /* sees every byte the ISR wrote */
        atomic_store_explicit(&frame_ready, false, memory_order_release);
    }
}

The flag now transfers ownership of the buffer: whoever sees it in their state owns frame. The release store guarantees that all writes before it (the frame bytes, or main’s reads of them) are complete before the flag changes; the acquire load guarantees that nothing after it is done before the flag is read. The next sections explain those guarantees.

C11 atomics

C11 added _Atomic types and the <stdatomic.h> header. The operations you will use:

#include <stdatomic.h>
#include <stdbool.h>
#include <stdint.h>

static _Atomic uint32_t rx_bytes;        /* or atomic_uint_least32_t */
static atomic_flag      tx_busy = ATOMIC_FLAG_INIT;

void counters_example(void)
{
    atomic_fetch_add_explicit(&rx_bytes, 1u, memory_order_relaxed);   /* indivisible ++ */
    const uint32_t n = atomic_load_explicit(&rx_bytes, memory_order_relaxed);
    atomic_store_explicit(&rx_bytes, 0u, memory_order_relaxed);
    (void)n;
}

/* Record the maximum latency seen, from several ISRs, without locks. */
void stat_update_max(_Atomic uint32_t *max, uint32_t v)
{
    uint32_t cur = atomic_load_explicit(max, memory_order_relaxed);
    while (v > cur &&
           !atomic_compare_exchange_weak_explicit(max, &cur, v,
                                                  memory_order_relaxed,
                                                  memory_order_relaxed)) {
        /* cur now holds the value someone else stored; loop re-checks v > cur */
    }
}

/* Try-lock that never spins: safe to call from an ISR. */
bool tx_try_start(void)
{
    if (atomic_flag_test_and_set_explicit(&tx_busy, memory_order_acquire)) {
        return false;                    /* already busy: caller retries later */
    }
    /* ... start the transfer; completion calls atomic_flag_clear_explicit(&tx_busy, memory_order_release) */
    return true;
}

• atomic_load and atomic_store read and write indivisibly with a chosen ordering. Without the _explicit suffix the ordering is memory_order_seq_cst.
• atomic_fetch_add (and _sub, _or, _and, _xor) perform the whole read-modify-write as one indivisible operation and return the old value. This fixes the lost-update counter.
• atomic_compare_exchange_weak(obj, &expected, desired) stores desired only if *obj == expected, and otherwise loads the current value into expected. It is the building block for any lock-free update. The weak form may fail spuriously (on Arm, when an interrupt hits between the two halves of the operation), so it belongs in a loop; the strong form retries internally.
• atomic_flag is the one type guaranteed lock-free everywhere: a test-and-set bit.

Never build a spinning lock from atomic_flag between an ISR and main on one core. If main holds the lock and the ISR spins waiting for it, main can never run to release it: the system hangs. Between ISRs and thread code, use try-locks that give up, or critical sections.

What does the compiler emit? For atomic_fetch_add(&counter, 1) with the default sequentially consistent ordering, GCC for Cortex-M4 produces approximately:

    dmb     ish
1:  ldrex   r3, [r2]           @ load and mark exclusive
    adds    r3, r3, #1
    strex   r1, r3, [r2]       @ store only if still exclusive; r1 = 0 on success
    cmp     r1, #0
    bne     1b                 @ interrupted: retry
    dmb     ish

With memory_order_relaxed the two dmb instructions disappear. A relaxed atomic_load or atomic_store of a 32-bit value is a plain ldr or str, because aligned word accesses are already atomic.

Check lock-freedom with atomic_is_lock_free(&obj) or the ATOMIC_INT_LOCK_FREE family of macros (on arm-none-eabi, uint32_t is unsigned long, so check ATOMIC_LONG_LOCK_FREE). Objects up to 32 bits are lock-free on M3/M4. A _Atomic uint64_t is not: ARMv7-M has no 64-bit exclusive instructions, so GCC emits calls to library helpers such as __atomic_load_8, which many bare-metal setups do not provide (you get a link error) or provide with a lock. Treat 64-bit shared data with a critical section.

Memory orders on a single-core MCU

Two different things can reorder your memory accesses:

• The compiler, at any optimization level above -O0. It moves, merges and eliminates loads and stores as long as a single-threaded program could not tell the difference. This is the main enemy on a microcontroller.
• The hardware: write buffers, out-of-order execution, caches, and other bus masters (DMA, a second core). A single-core Cortex-M4 always observes its own accesses in program order, including from its own ISRs, so for ISR-versus-main sharing the hardware is not a source of reordering. It matters when DMA or a second core reads or writes the same memory, and on the Cortex-M7, whose caches and memory system are far more aggressive.

C11 memory orders constrain both at once:

Order	Guarantee	Typical use
memory_order_relaxed	the operation itself is atomic; no ordering of other accesses	counters, statistics, flags that guard nothing
memory_order_release (stores)	no earlier read or write may be moved after this store	publishing data: “the buffer is ready”
memory_order_acquire (loads)	no later read or write may be moved before this load	consuming data: “is the buffer ready?”
memory_order_acq_rel	both, for read-modify-write operations	lock-free exchanges
memory_order_seq_cst	acquire and release plus one total order of all seq_cst operations	the default; the safest when unsure

The pairing rule: if an acquire load reads the value written by a release store, everything the writer did before the release is visible to the reader after the acquire. That is the happens-before edge that made the frame handoff correct.

On ARMv7-M, GCC implements acquire and release with a dmb after the load or before the store. For ISR-versus-main on one core the dmb is strictly unnecessary (only the compiler ordering matters), and it costs a few cycles. C11 has an exact tool for “two flows on the same processor”: atomic_signal_fence(memory_order_release) and its acquire counterpart are compiler-only fences that emit no instructions. They were designed for a thread and its signal handler, which is precisely an ISR and main. Use them only when you have measured that the dmb matters and the code will never run with DMA, caches or a second core involved; the ordinary acquire and release operations are correct everywhere and cost almost nothing.

How LDREX and STREX work

Cortex-M3, M4, M7 and M33 build atomic read-modify-write operations from a pair of instructions and a small piece of hardware called the local exclusive monitor.

• LDREX Rt, [Rn] loads a word and sets the monitor to the exclusive state for that address: “this core has an exclusive access in progress”.
• STREX Rd, Rt, [Rn] stores Rt only if the monitor is still exclusive. It writes 0 to Rd on success, or skips the store and writes 1 on failure. Either way the monitor returns to the open state.

What moves the monitor back to open before the STREX? Another STREX, a CLREX instruction, and, crucially, exception entry and exception return, which on ARMv7-M clear the local monitor. So if any interrupt is taken between LDREX and STREX, even one that never touches the variable, the STREX fails and the loop retries with a fresh load. An implementation may also clear the monitor for other reasons, which is why the code must always be written as a retry loop, never as a single attempt.

Walk through the lost-update race with exclusives:

 main: LDREX r3,[events]  -> 5, monitor exclusive
 main: ADDS  r3,#1        -> 6
       ---- TIM2 interrupt: exception entry clears the monitor ----
 ISR : LDREX -> 5, ADDS -> 6, STREX succeeds: events = 6
       ---- exception return clears the monitor again ----
 main: STREX r1,r3,[events] -> monitor open: store NOT done, r1 = 1
 main: BNE retry -> LDREX -> 6, ADDS -> 7, STREX succeeds: events = 7   (correct)

No interrupts were disabled, no lock was taken, and nothing can be lost. The CMSIS intrinsics expose the instructions directly if you need them:

#include <stdint.h>
#include "stm32f4xx.h"      /* __LDREXW, __STREXW */

static inline uint32_t atomic_inc_u32(volatile uint32_t *p)
{
    uint32_t v;
    do {
        v = __LDREXW(p) + 1u;
    } while (__STREXW(v, p) != 0u);    /* 0 = stored; 1 = interrupted, retry */
    return v;
}

Rules for exclusive sequences: keep the code between LDREX and STREX short and free of other memory accesses and function calls, and never nest them. The C11 atomics generate correct loops for you; prefer them.

The local monitor sees only this core’s activity. It does not notice a DMA controller writing the location, so exclusives give no atomicity against DMA. On multi-core MCUs, memory shared between cores needs a global monitor in the interconnect, which the chip vendor must provide for that memory region.

The Cortex-M0 and M0+ (ARMv6-M) have no exclusive instructions at all. On those cores GCC turns atomic_fetch_add into calls to helper functions such as __atomic_fetch_add_4, and the only way to implement them on a single core is to disable interrupts:

#include <stdint.h>
#include "cmsis_compiler.h"   /* __get_PRIMASK, __disable_irq, __set_PRIMASK on a Cortex-M0 build */

unsigned int __atomic_fetch_add_4(volatile void *ptr, unsigned int val, int memorder)
{
    (void)memorder;
    volatile unsigned int *p = ptr;
    const uint32_t primask = __get_PRIMASK();
    __disable_irq();
    const unsigned int old = *p;
    *p = old + val;
    __set_PRIMASK(primask);
    return old;
}

A lock-free SPSC ring buffer

The ring buffer from UART: serial communication works because it has a single producer and a single consumer (SPSC): the ISR only writes head, main only writes tail. Here it is written so that its correctness does not depend on luck or on a particular compiler.

spsc.h

#ifndef SPSC_H
#define SPSC_H

#include <stdatomic.h>
#include <stdbool.h>
#include <stdint.h>

#define SPSC_SIZE 256u                   /* must be a power of two */
_Static_assert((SPSC_SIZE & (SPSC_SIZE - 1u)) == 0u, "SPSC_SIZE must be a power of two");

typedef struct {
    uint8_t          buf[SPSC_SIZE];
    _Atomic uint32_t head;               /* written only by the producer */
    _Atomic uint32_t tail;               /* written only by the consumer */
} spsc_t;                                /* a static instance is zero-initialized: empty */

bool spsc_push(spsc_t *q, uint8_t byte);    /* producer only */
bool spsc_pop(spsc_t *q, uint8_t *out);     /* consumer only */

#endif

spsc.c

#include "spsc.h"

bool spsc_push(spsc_t *q, uint8_t byte)
{
    const uint32_t head = atomic_load_explicit(&q->head, memory_order_relaxed);  /* own index */
    const uint32_t tail = atomic_load_explicit(&q->tail, memory_order_acquire);  /* other's index */
    if (head - tail == SPSC_SIZE) {
        return false;                                       /* full */
    }
    q->buf[head & (SPSC_SIZE - 1u)] = byte;                 /* write the slot... */
    atomic_store_explicit(&q->head, head + 1u, memory_order_release);   /* ...then publish it */
    return true;
}

bool spsc_pop(spsc_t *q, uint8_t *out)
{
    const uint32_t tail = atomic_load_explicit(&q->tail, memory_order_relaxed);  /* own index */
    const uint32_t head = atomic_load_explicit(&q->head, memory_order_acquire);  /* other's index */
    if (head == tail) {
        return false;                                       /* empty */
    }
    *out = q->buf[tail & (SPSC_SIZE - 1u)];                 /* read the slot... */
    atomic_store_explicit(&q->tail, tail + 1u, memory_order_release);   /* ...then free it */
    return true;
}

Why each ordering is what it is:

• Indices are free-running. head and tail count every byte ever pushed and popped, and are masked only when indexing. head - tail is the fill level even after both wrap past 2^32, because unsigned subtraction is modular and SPSC_SIZE divides 2^32. This also lets all 256 slots be used, with no “one empty slot” trick.
• Each side reads its own index relaxed. Only the producer writes head, so the producer always knows its value; no ordering is needed.
• Producer: write slot, then release-store head. The release guarantees the byte is in memory before the new head is visible. Paired with the consumer’s acquire load of head, the consumer is guaranteed to read the byte the producer wrote, never stale data.
• Consumer: read slot, then release-store tail. This half is easy to forget. The release guarantees the consumer has finished reading the slot before tail says it is free. Paired with the producer’s acquire load of tail, the producer cannot overwrite a slot that is still being read. With relaxed ordering here, the compiler (or a weakly ordered CPU) could let the producer’s next write land before the consumer’s read of the same slot.

Usage is exactly the UART receive path: spsc_push(&uart_rx, (uint8_t)USART2->DR) in the ISR and spsc_pop(&uart_rx, &b) in the main loop. For bulk transfers, compute the available count once, copy a block with memcpy, and publish the new index with a single release store.

Because this code uses only standard C11, you can stress it on your PC with real threads, where the hardware really does run both sides at the same time:

test_spsc.c

#include <inttypes.h>
#include <pthread.h>
#include <stdio.h>
#include "spsc.h"

#define N 20000000u

static spsc_t q;

static void *producer(void *arg)
{
    (void)arg;
    for (uint32_t i = 0; i < N;) {
        if (spsc_push(&q, (uint8_t)i)) {
            i++;
        }
    }
    return NULL;
}

int main(void)
{
    pthread_t t;
    pthread_create(&t, NULL, producer, NULL);
    uint32_t errors = 0;
    for (uint32_t i = 0; i < N;) {
        uint8_t b;
        if (spsc_pop(&q, &b)) {
            if (b != (uint8_t)i) {
                errors++;
            }
            i++;
        }
    }
    pthread_join(t, NULL);
    printf("received %" PRIu32 " bytes, %" PRIu32 " errors\n", N, errors);
    return errors != 0u;
}

gcc -std=c11 -O2 -Wall -Wextra -Wpedantic -pthread spsc.c test_spsc.c -o test_spsc && ./test_spsc
gcc -std=c11 -O1 -g -fsanitize=thread -pthread spsc.c test_spsc.c -o test_spsc_tsan && ./test_spsc_tsan

received 20000000 bytes, 0 errors

ThreadSanitizer understands C11 atomics and reports any unordered access to shared memory. Replace the atomics with plain volatile uint32_t indices and it flags the race immediately, even on runs where the byte check happens to pass.

Barriers: compiler, DMB, DSB, ISB

Sometimes you need ordering without an atomic variable. There are four tools, from weakest to strongest:

• Compiler barrier: __asm volatile ("" ::: "memory"), also available as CMSIS __COMPILER_BARRIER(). Emits no instruction. The "memory" clobber tells the compiler that memory may have changed, so it must not cache values in registers across the barrier or move loads and stores over it. The CMSIS intrinsics __disable_irq, __enable_irq, __set_PRIMASK, __DMB, __DSB and __ISB all include this clobber, so they are compiler barriers too.
• DMB (data memory barrier): memory accesses before it are observed before memory accesses after it, by every observer including DMA and other cores. It does not wait for them to finish.
• DSB (data synchronization barrier): stronger; execution stalls until every memory access before it has completed.
• ISB (instruction synchronization barrier): flushes the pipeline so later instructions are fetched and executed with any new processor state in effect.

When each is required:

Situation	Barrier	Reason
Before WFI or WFE	DSB	outstanding writes (clearing a flag, arming a wake-up source) must complete before the core sleeps
Clearing a peripheral interrupt flag at the very end of an ISR	DSB, or read the register back	the write can still be in flight through the bus bridge at exception return, so the ISR fires again
After writing VTOR, before an exception that must use the new table	DSB	the table base must be updated before the next vector fetch
After changing CONTROL (stack pointer select, privilege)	ISB	following instructions must run with the new state
After enabling the FPU in CPACR	DSB then ISB	before the first floating-point instruction
After enabling or reprogramming the MPU	DSB then ISB	later accesses and instruction fetches must use the new regions
Buffer filled by the CPU, then DMA started	DMB (or DSB) before the write that enables the DMA	the DMA controller is another bus master and must see the data first
DMA complete, CPU about to read the buffer	DMB after seeing completion, plus cache invalidation on M7	the CPU must not read the buffer before completion is observed
NVIC_DisableIRQ that must take effect before the next line	DSB then ISB	recent CMSIS versions include these in __NVIC_DisableIRQ
Just after enabling a peripheral clock in RCC	DSB or a dummy read-back	ST errata: the first access can be lost

On the Cortex-M3 and M4, the simple pipeline means several of these cases happen to work without the barrier, and Arm’s application note on memory barriers says so. They are still architecturally required, they are mandatory on the Cortex-M7, and code gets ported. Write them.

Disabling interrupts properly

When a critical section is the right tool (it is for anything larger than a word, or anything an atomic operation cannot express), write it so it composes:

#include <stdint.h>
#include "stm32f4xx.h"

static inline uint32_t irq_lock(void)
{
    const uint32_t primask = __get_PRIMASK();
    __disable_irq();
    return primask;
}

static inline void irq_unlock(uint32_t primask)
{
    __set_PRIMASK(primask);           /* restores the caller's state, enabled or not */
}

/* Mask only priorities numerically >= prio; more urgent interrupts keep running. */
static inline uint32_t irq_lock_below(uint32_t prio)
{
    const uint32_t old = __get_BASEPRI();
    __set_BASEPRI_MAX(prio << (8u - __NVIC_PRIO_BITS));   /* only ever raises the mask */
    return old;
}

static inline void irq_unlock_below(uint32_t old)
{
    __set_BASEPRI(old);
}

The save and restore is the point. A helper that does __disable_irq(); ... __enable_irq(); re-enables interrupts when it returns, even if its caller had them disabled, silently opening the caller’s critical section halfway through. With irq_lock and irq_unlock, nested sections are correct automatically.

BASEPRI masks only interrupts at or below a priority, leaving more urgent ones running. That is what RTOS kernels use, and it is the right choice when a motor-control or safety ISR must never be delayed. Those unmasked ISRs must then never touch the protected data. __set_BASEPRI_MAX writes only if the new value raises the mask, so nested BASEPRI sections cannot accidentally lower it. The Cortex-M0 has no BASEPRI; it has only PRIMASK.

Every critical section adds its length to the worst-case latency of every interrupt it masks. Give them a budget and measure it with the DWT cycle counter. A concrete example: a UART at 921600 baud delivers a byte every 10.9 µs, and the STM32F4 USART holds one received byte, so a critical section longer than about 10 µs (roughly 1800 cycles at 168 MHz) can cause an overrun. Keep critical sections to tens of cycles: no loops of unbounded length, no calls into code you have not measured, never a printf.

Things that are not concurrency tools

• -fno-strict-aliasing changes what the compiler may assume about pointers of different types aliasing the same memory, as Undefined behavior, volatile, and what the optimizer assumes explains. It does not make any access atomic, does not order anything, and does not stop the compiler from keeping a shared variable in a register. If adding it “fixes” a race, the race is still there.
• Making everything volatile. It prevents the compiler from caching values, but it does not make read-modify-writes atomic, does not order non-volatile accesses, and makes all the code slower.
• Compiling at -O0. It hides the compiler-reordering class of bugs and leaves the lost-update and torn-read classes exactly as they were.

Double-checked initialization

A function that lazily initializes a shared object on first use is a classic place for a subtle race:

static bool       ready;          /* BROKEN */
static sensor_ctx ctx;

sensor_ctx *ctx_get(void)
{
    if (!ready) {                 /* unlocked read: races with the write below */
        lock();
        if (!ready) {
            ctx_init(&ctx);
            ready = true;         /* may become visible before ctx's fields do */
        }
        unlock();
    }
    return &ctx;
}

The fast path reads ready without the lock, so a second task can see ready == true and use ctx before the stores made by ctx_init are visible, because nothing orders those stores before the store to ready. The correct version makes the flag atomic, with release on publication and acquire on the fast path:

static atomic_bool ready;
static sensor_ctx  ctx;

sensor_ctx *ctx_get(void)
{
    if (!atomic_load_explicit(&ready, memory_order_acquire)) {
        lock();                                       /* a mutex between tasks */
        if (!atomic_load_explicit(&ready, memory_order_relaxed)) {
            ctx_init(&ctx);
            atomic_store_explicit(&ready, true, memory_order_release);
        }
        unlock();
    }
    return &ctx;
}

The second load can be relaxed because the lock already orders it. In firmware the better answer is usually not to initialize lazily at all: initialize everything in main before enabling the interrupts or starting the scheduler that could race. Never call a function like this from an ISR, because the ISR cannot wait for a lock held by the code it interrupted.

Flushing out races: test with interrupts fast

Races hide because their windows are a few instructions wide and interrupts are rare. Make interrupts common. Configure a timer to interrupt at a very high rate and make its ISR perform the operation under suspicion, then hammer the same operation from main and check an invariant:

#include <inttypes.h>
#include <stdint.h>
#include <stdio.h>
#include "stm32f4xx.h"

#define ITERATIONS 1000000u

static volatile uint32_t shared;         /* the variable under test */
static volatile uint32_t isr_count;

void TIM2_IRQHandler(void)
{
    TIM2->SR = ~TIM_SR_UIF;
    shared++;                            /* the operation under test */
    isr_count++;
}

void race_test(void)
{
    RCC->APB1ENR |= RCC_APB1ENR_TIM2EN;
    (void)RCC->APB1ENR;
    TIM2->PSC  = 0u;
    TIM2->ARR  = 159u;                   /* 16 MHz timer clock after reset: 100 kHz interrupts */
    TIM2->DIER = TIM_DIER_UIE;
    NVIC_SetPriority(TIM2_IRQn, 5u);
    NVIC_EnableIRQ(TIM2_IRQn);
    TIM2->CR1  = TIM_CR1_CEN;

    for (uint32_t i = 0; i < ITERATIONS; i++) {
        shared++;                        /* racy read-modify-write from thread context */
    }

    TIM2->CR1 = 0u;
    NVIC_DisableIRQ(TIM2_IRQn);
    const uint32_t expected = ITERATIONS + isr_count;
    printf("expected %" PRIu32 ", got %" PRIu32 ", lost %" PRIu32 "\n",
           expected, shared, expected - shared);
}

expected 1056230, got 1037114, lost 19116

Your numbers will vary. In this tight loop roughly one interrupt in three lands between the load and the store, so with plain shared++ thousands of updates are lost. Replace the thread-side increment with atomic_fetch_add_explicit on an _Atomic uint32_t, or wrap it in irq_lock and irq_unlock, and it drops to zero on every run. Two refinements make the method stronger. Vary the timer period from the ISR (for example TIM2->ARR = 100u + (lfsr() & 0x3Fu);) so interrupts land on every instruction of the loop rather than in a fixed pattern. And run the real application with this extra high-rate interrupt enabled during soak tests: if a race exists anywhere the interrupt can reach, it will show up in minutes instead of months.

Tip

A race test that passes proves little; a race test that fails proves a lot. Before trusting a fix, make sure the test fails reliably on the broken version. If it does not, your interrupt is not hitting the window and the test is not testing anything.

Common mistakes

• x++ on a variable shared with an ISR. Updates are lost when the interrupt lands mid-operation; use atomic_fetch_add or a critical section.
• Reading a uint64_t or a struct written by an ISR. The value tears; copy it with interrupts masked or use a retry protocol.
• volatile flag guarding plain data. The compiler may reorder the data around the flag; make the flag atomic with release on store and acquire on load.
• Relaxed ordering on the consumer’s index update. The producer can overwrite a slot still being read; publish tail with release and read it with acquire.
• __disable_irq(); ... __enable_irq(); in a helper. It re-enables interrupts inside the caller’s critical section; save and restore PRIMASK.
• Spinning on a lock in an ISR. The holder it interrupted can never release it; use try-locks or critical sections between ISRs and thread code.
• Missing DSB before WFI or after clearing an interrupt flag at the end of an ISR. The core sleeps too early or the ISR re-enters; add the barrier.
• Expecting exclusives to protect against DMA. The local monitor does not see other bus masters; hand buffers over with ownership flags and barriers instead.

Exercises

1. Run the race test on the board or in Renode with the plain shared++, then with atomic_fetch_add_explicit and with irq_lock. Record the lost count for each over five runs. Then set ARR to 15999 (1 kHz) and see how the lost count scales with the interrupt rate. Estimate how often the plain version would fail if the increment ran once per button press instead of in a tight loop.
2. Compile spsc.c for the target with -O2 and read the disassembly with arm-none-eabi-objdump -d. Find the dmb instructions and match each to the acquire or release operation that produced it. Change every ordering to memory_order_relaxed, recompile, and diff the output.
3. Build and run the host test_spsc normally and under ThreadSanitizer. Then make head and tail plain volatile uint32_t with ordinary loads and stores and run the TSan build again. Copy the first race report and explain which two accesses it names.
4. Write uptime_read_lockfree and a TIM2 overflow ISR that maintains uptime_hi and uptime_lo with a very small ARR, so the low word wraps often. Show that a naive two-read version returns non-monotonic values within seconds while the retry version never does.
5. Write a single-slot mailbox (mailbox_post from an ISR, mailbox_take from main) for a 16-byte struct using one atomic_bool and acquire/release. Argue in writing, step by step, why neither side can ever see a half-written struct, then try to break it with the high-rate interrupt method.

Check yourself

Question 1:On a Cortex-M4, which shared access is safe without atomics or a critical section?

An ISR and main both doing count++ on a volatile uint32_t main reading a volatile uint64_t timestamp written by an ISR main reading an aligned volatile uint32_t that only the ISR writes main setting one bitfield member while the ISR sets another in the same word

Question 2:An ISR fills buf[] (plain memory) and then sets volatile bool ready = true;. What can the compiler legally do?

Nothing; volatile orders all memory accesses around it Move some buf[] stores after the store to ready, because volatile only orders volatile accesses relative to each other Remove the store to ready Make the stores to buf[] atomic

Question 3:Between LDREX and STREX in main, an interrupt fires whose handler never touches the variable. What happens to the STREX?

It succeeds, because the ISR did not write the address It fails and returns 1, because exception entry and return clear the local monitor; the loop retries It faults with a UsageFault It stores a corrupted value

Question 4:In the SPSC ring buffer, the consumer publishes tail with memory_order_relaxed instead of release. What can go wrong?

The consumer can read stale data from the slot The producer can overwrite a slot before the consumer has finished reading it The buffer can report full when it is empty Nothing; only the producer needs release ordering

Question 5:A helper used inside other critical sections does __disable_irq(); update(); __enable_irq();. What is the bug?

__disable_irq does not mask SysTick When called with interrupts already disabled, it re-enables them on return, ending the caller's critical section early __enable_irq must be followed by ISB or it has no effect Critical sections cannot be nested on Cortex-M

Question 6:Adding -fno-strict-aliasing makes an intermittent ISR data-sharing bug disappear. What should you conclude?

The bug was a strict aliasing violation and is fixed The race is probably still there; the flag merely changed code generation so the window moved or shrank -fno-strict-aliasing makes shared variables atomic The compiler had a bug that the flag works around

Key takeaways

• An ISR and main are concurrent; a data race on a non-atomic object is undefined behavior and the compiler exploits it.
• Only single aligned accesses of 32 bits or less are atomic on Cortex-M; read-modify-writes, 64-bit values, structs, bitfields and unaligned accesses are not.
• volatile forces accesses to happen but orders only volatile accesses relative to each other; use C11 atomics with release on publish and acquire on consume.
• LDREX/STREX make lock-free read-modify-writes on M3/M4/M7 because exceptions clear the exclusive monitor; M0 has no exclusives and must mask interrupts; DMA is invisible to the monitor.
• A correct SPSC ring uses free-running indices, relaxed loads of its own index, acquire loads of the other side’s index, and release stores after touching the slot.
• DSB before WFI and after end-of-ISR flag clears, ISB after CONTROL changes, DMB around DMA handoff; the "memory" clobber is the compiler-only barrier.
• Save and restore PRIMASK or BASEPRI in critical sections, budget their length, and flush out races with a deliberately fast interrupt.

Next: Memory protection, stack safety, and robustness, where the MPU turns silent memory corruption into immediate, diagnosable faults.