Firmware Architecture at Scale

A 2,000-line firmware written by one person can live in its author’s head. A 200,000-line firmware written by twelve people over six years, shipping on four boards and two MCU families, cannot. At that scale the problems stop being “how do I drive this peripheral” and become “why does a change to the logger break the motor controller”, “which bootloader versions can run this app”, and “who decided the radio task could block for 30 ms”. This lesson is about the structures that keep a large firmware codebase understandable: enforced layering, module contracts, portability layers, multi-core communication, system-level timing, budgets, variants, versioning, review, documentation, and the reference architecture of a connected product.

Layering, enforced

Driver Design and Hardware Abstraction introduced the layers and the dependency rule: code may depend only on layers below it.

+-------------------------------------------------------------+
|  app/        product logic, state machines, UI               |
+-------------------------------------------------------------+
|  services/   protocol, storage, logging, update, power mgmt |
+-------------------------------------------------------------+
|  drivers/    sensor, radio, display, flash chip drivers      |
+-------------------------------------------------------------+
|  hal/        uart, spi, i2c, gpio, timer: MCU-specific       |
+-------------------------------------------------------------+
|  platform/   startup, linker scripts, clocks, register hdrs  |
+-------------------------------------------------------------+
   board/ (pin maps, which instance is what) wires layers together
   lib/   (CRC, ring buffers, COBS) is usable from any layer

Every team agrees with this diagram on day one. By year two, someone in hal/uart.c has added #include "app/state.h" to check whether the device is in factory mode, and the layering is fiction. A rule that is not enforced by a machine does not exist. Here are three ways to enforce it.

One library per layer, with explicit visibility

In CMake, make each layer a library target and declare its dependencies. Include paths then follow the dependency graph automatically:

CMakeLists.txt

add_library(platform STATIC platform/startup.c platform/clock.c)
target_include_directories(platform PUBLIC platform/include)

add_library(hal STATIC hal/uart.c hal/spi.c hal/i2c.c hal/gpio.c)
target_include_directories(hal PUBLIC hal/include PRIVATE hal/src)
target_link_libraries(hal PUBLIC platform)

add_library(drivers STATIC drivers/tmp_sensor.c drivers/spi_nor.c)
target_include_directories(drivers PUBLIC drivers/include)
target_link_libraries(drivers PUBLIC hal lib)

add_library(services STATIC services/proto.c services/storage.c)
target_include_directories(services PUBLIC services/include)
target_link_libraries(services PUBLIC drivers lib)

add_executable(firmware app/main.c app/app_fsm.c board/board_disco.c)
target_link_libraries(firmware PRIVATE services)

hal cannot include anything from app/ because app/’s headers are not on its include path: the attempt fails to compile with “file not found”. PRIVATE include directories hide a layer’s internal headers from the layers above. The structure of the build is now the architecture diagram.

A dependency checker in CI

Not every build system makes this easy, and a library-per-layer split does not catch includes written with relative paths (#include "../../app/state.h"). A short script closes the gap:

tools/check_layers.py

import re
import sys
from pathlib import Path

LAYERS = ["platform", "hal", "drivers", "services", "app"]   # low to high
SHARED = {"lib", "board"}
INCLUDE = re.compile(r'^\s*#\s*include\s+"([^"]+)"')

def layer_of(path: Path):
    for part in path.parts:
        if part in LAYERS or part in SHARED:
            return part
    return None

def main(root: Path) -> int:
    headers = {p.name: p for p in root.rglob("*.h")}
    errors = 0
    for src in list(root.rglob("*.c")) + list(root.rglob("*.h")):
        src_layer = layer_of(src.relative_to(root))
        if src_layer not in LAYERS:
            continue
        for n, line in enumerate(src.read_text(errors="replace").splitlines(), 1):
            m = INCLUDE.match(line)
            if not m:
                continue
            target = headers.get(Path(m.group(1)).name)
            if target is None:
                continue
            dst_layer = layer_of(target.relative_to(root))
            if dst_layer in LAYERS and LAYERS.index(dst_layer) > LAYERS.index(src_layer):
                print(f"{src}:{n}: {src_layer} must not include {dst_layer} ({m.group(1)})")
                errors += 1
    return 1 if errors else 0

if __name__ == "__main__":
    sys.exit(main(Path(sys.argv[1] if len(sys.argv) > 1 else ".")))

src/hal/uart.c:14: hal must not include app (app/state.h)

The script maps every included header to the layer directory it lives in and fails if any file includes a header from a higher layer. It runs in a second, and it turns an architectural opinion into a failing check on the offending pull request. Tools such as include-what-you-use and Doxygen’s include graphs help you see the existing graph before you start enforcing it.

What to do when a lower layer needs something from above

The hal/uart.c author had a real need: behave differently in factory mode. The fix is never to break the rule; it is to invert the dependency. The lower layer exposes a configuration option or a callback, and the upper layer sets it:

• Pass the behaviour in as configuration at init (uart_cfg.allow_break_detect = true).
• Register a callback (uart_set_break_handler(cb, ctx)), so the HAL calls up without knowing who it calls.
• Publish an event and let the app decide.

Module boundaries and contracts

A module is a unit with one responsibility, a public header, and private everything else. Its contract is what callers may rely on. Write it in the header, because that is where callers look:

services/include/storage.h

/**
 * Storage service: persistent key-value settings and an append-only event log.
 *
 * Threading: all functions must be called from the storage task only,
 *            except storage_log_post(), which is safe from any task or ISR.
 * Blocking:  storage_set() may block up to 40 ms (sector erase on compaction).
 * Errors:    returns ERR_NO_MEM when the log partition is full and compaction
 *            cannot free space; the caller decides whether to drop or retry.
 * Lifetime:  storage_init() must succeed before any other call.
 */

The four headings (threading, blocking, errors, lifetime) are the questions that bite in firmware. A contract that says “may block up to 40 ms” tells the radio team immediately that they cannot call it from their 10 ms-deadline task.

Versioned internal APIs

When modules are owned by different teams, or ship separately (a bootloader and an application, two cores, a shared library in ROM), the interface between them needs a version. The usual mechanism is a struct of function pointers with a version field, checked at bind time:

boot_api.h

#include <stdint.h>

#define BOOT_API_MAGIC   0xB007A410u
#define BOOT_API_MAJOR   2u
#define BOOT_API_MINOR   1u

typedef struct {
    uint32_t magic;
    uint16_t major;           /* incompatible changes bump this */
    uint16_t minor;           /* new functions appended bump this */
    uint32_t (*get_boot_count)(void);
    int      (*request_update)(uint32_t slot);
    /* minor 1: */
    int      (*get_reset_reason)(uint32_t *out);
} boot_api_t;

The application finds the table at a fixed address in the bootloader’s flash and checks it before use:

app_boot.c

#include <stddef.h>
#include "boot_api.h"

#define BOOT_API_ADDR 0x08000200u

const boot_api_t *boot_api_get(void)
{
    const boot_api_t *api = (const boot_api_t *)BOOT_API_ADDR;
    if (api->magic != BOOT_API_MAGIC || api->major != BOOT_API_MAJOR) {
        return NULL;                      /* incompatible: run without it */
    }
    return api;
}

int boot_reset_reason(uint32_t *out)
{
    const boot_api_t *api = boot_api_get();
    if (api == NULL || api->minor < 1u || api->get_reset_reason == NULL) {
        return -1;                        /* older bootloader: feature absent */
    }
    return api->get_reset_reason(out);
}

The rules are the semantic versioning rules applied to a binary interface. Adding a function at the end bumps the minor version, and callers check minor before using it. Changing a signature, reordering members, or removing a function bumps the major version, and an app built for major 2 refuses major 3. Never reorder or remove members within a major version: the app was compiled with fixed offsets.

Portability across MCU families

Products outlive MCUs. Supply shortages, cost-downs, and new features force ports, and the cost of a port is decided years earlier by how well the MCU-specific code was contained.

The containment boundary is the port layer: the HAL plus platform code, behind interfaces that speak in product terms (uart_write, timer_start_us, flash_erase_sector) rather than vendor terms. A port to a new family then means writing a new hal/ and platform/ and leaving drivers/, services/, and app/ untouched.

In practice, things leak. Watch for these:

Leak	Example	Containment
Vendor types in interfaces	HAL_StatusTypeDef returned by a driver	Translate to err_t at the HAL boundary
Interrupt numbers and priorities	NVIC_SetPriority(USART2_IRQn, 5) in a service	Priorities assigned in the board/platform layer from a table
Memory layout assumptions	DMA buffers placed in CCM RAM, which some parts lack or cannot DMA into	Linker-section macros (DMA_BUFFER) defined per platform
Flash geometry	Code that assumes 128 KB sectors	Query geometry from the flash driver
Clock assumptions	delay loops calibrated for 168 MHz	All timing from a timer service, never from cycle counting in upper layers
Endianness and alignment	Casting byte buffers to uint32_t *	Shift-based packing everywhere; Cortex-M parts are little-endian, but your protocol peers may not be
Compiler extensions	__attribute__ scattered everywhere	Wrap in a compiler.h, as C Standards, Compiler Extensions, and Portable Code shows

A useful test: compile all layers above the HAL for the host as part of the unit-test build. Anything that fails to compile on x86 has leaked MCU specifics upward.

Multi-core MCUs

More MCUs now have two cores: the STM32H7 dual-core parts pair a Cortex-M7 with a Cortex-M4, the nRF5340 has an application core and a network core, the RP2040 has two Cortex-M0+ cores, and most ESP32 variants have two cores. The usual split is by function: one core runs the radio stack or hard real-time control, the other runs the application.

Inter-processor communication

The cores share some RAM and need a protocol to use it safely. The building blocks:

• Shared memory with ownership rules. Each buffer has exactly one owner at a time. Ownership is transferred explicitly, and only the owner reads or writes the contents.
• Hardware semaphores. The STM32H7’s HSEM peripheral provides lock registers that only one core can take at a time, with an interrupt to the other core on release. The RP2040 has hardware spinlocks for the same purpose.
• Mailboxes and inter-core interrupts. One core signals the other with a dedicated interrupt (for example a hardware FIFO between cores on the RP2040, or the HSEM release interrupt) to say “there is a message”.
• OpenAMP and RPMsg. A standard framework for message passing between cores (and between Linux and an MCU core) built on shared-memory ring buffers (virtio vrings) and inter-core notifications. It gives you named endpoints and a socket-like API, at a cost of a few tens of KB of code.

Here is the core idea in its simplest form: a single-producer, single-consumer message ring in shared RAM, one per direction.

ipc_ring.c

#include <stdbool.h>
#include <stdint.h>
#include <string.h>

#define IPC_SLOTS     16u               /* power of two */
#define IPC_MSG_BYTES 28u

typedef struct {
    uint16_t type;
    uint16_t len;
    uint8_t  data[IPC_MSG_BYTES];
} ipc_msg_t;

typedef struct {
    volatile uint32_t head;             /* written only by the producer core */
    volatile uint32_t tail;             /* written only by the consumer core */
    ipc_msg_t slot[IPC_SLOTS];
} ipc_ring_t;

#define DMB() __asm volatile ("dmb" ::: "memory")

void ipc_notify_remote(void);           /* platform: inter-core interrupt */

bool ipc_send(ipc_ring_t *r, const ipc_msg_t *m)
{
    uint32_t head = r->head;
    if (head - r->tail == IPC_SLOTS) {
        return false;                   /* full */
    }
    memcpy(&r->slot[head % IPC_SLOTS], m, sizeof *m);
    DMB();                              /* message visible before the index */
    r->head = head + 1u;
    ipc_notify_remote();
    return true;
}

bool ipc_receive(ipc_ring_t *r, ipc_msg_t *out)
{
    uint32_t tail = r->tail;
    if (r->head == tail) {
        return false;                   /* empty */
    }
    DMB();                              /* read index before the message */
    memcpy(out, &r->slot[tail % IPC_SLOTS], sizeof *out);
    DMB();                              /* finish reading before freeing slot */
    r->tail = tail + 1u;
    return true;
}

Each index is written by exactly one core, so no lock is needed. The indices run freely and wrap at 2^32; head - tail is the fill level even across the wrap because the slot count is a power of two. The barriers do the job that Concurrency, Atomics, and Memory Ordering explains for ISRs, but here they are essential at the hardware level too: the two cores have independent bus masters and write buffers, so without DMB the other core could see the new head before the message bytes. The ring itself must be placed in a RAM region both cores can access, at the same address in both cores’ memory maps, through a shared linker section.

Cache coherency between cores

On a Cortex-M4 there are no data caches and the code above is complete. On a Cortex-M7 with its data cache enabled, it is not: the M7 may hold the message in its cache while the M4 reads stale RAM, or read a stale cached head forever. Two fixes exist:

• Make the shared region non-cacheable with an MPU region (or shareable, where the hardware supports coherency). This is simple and the usual choice for small control structures.
• Maintain the cache manually: clean (write back) the message lines before publishing the index, and invalidate them before reading. This keeps bulk data fast but every buffer must be aligned to and sized in 32-byte cache lines, or cleaning one buffer can corrupt its neighbour.

The same issue appears between an M7 and DMA, as DMA noted. The rule is identical: any memory shared with another bus master is either non-cacheable or explicitly maintained.

Asynchronous design at system level

At small scale, “keep ISRs short” and “do not block” are coding rules. At large scale they become system properties you have to design and measure.

Everything is an event. Interrupts, timer expiries, received messages, and user input all become events delivered to tasks or active objects, as State Machines and Event-Driven Firmware described. A module that waits in a loop for something is a module that hides latency from everyone else.

Every activity has a time budget. Write down, for each task or event handler, its period or trigger, its deadline, and its worst-case execution time (WCET):

Activity	Trigger	Deadline	Measured WCET	Priority
Motor control loop	TIM1 update, 10 kHz	100 µs	38 µs	ISR, priority 1
Radio RX handling	Radio IRQ	2 ms (next packet)	420 µs	Task 5
Sensor sampling	100 ms timer	100 ms	1.1 ms	Task 4
Storage writes	Queue	1 s	45 ms (sector erase)	Task 2
Logging	Queue	Best effort	3 ms	Task 1

WCET reasoning for firmware is usually measurement-based plus margin, not formal analysis: measure the worst case you can provoke (all branches taken, maximum message sizes, caches cold on an M7), then add a margin of 20 to 50% depending on how much of the code path you are sure you exercised. Static WCET analysis tools exist and are used in avionics and automotive work, but they need detailed processor models and annotated loops.

Measure jitter, do not guess it. The DWT cycle counter from Clocks, Reset, and Timing gives cycle-accurate timestamps:

jitter.c

#include <stdint.h>

#define DEMCR      (*(volatile uint32_t *)0xE000EDFCu)
#define DWT_CTRL   (*(volatile uint32_t *)0xE0001000u)
#define DWT_CYCCNT (*(volatile uint32_t *)0xE0001004u)

static uint32_t last, min_period = UINT32_MAX, max_period;

void jitter_init(void)
{
    DEMCR    |= (1u << 24);      /* TRCENA: enable DWT */
    DWT_CYCCNT = 0;
    DWT_CTRL |= 1u;              /* CYCCNTENA */
}

/* Call at the start of the periodic handler being measured. */
void jitter_sample(void)
{
    uint32_t now = DWT_CYCCNT;
    uint32_t period = now - last;      /* wraps correctly every ~25 s at 168 MHz */
    last = now;
    if (period < min_period) min_period = period;
    if (period > max_period) max_period = period;
}

For a 10 kHz loop at 168 MHz the ideal period is 16,800 cycles. If after an hour of operation under load max_period - min_period is 2,400 cycles, the loop’s jitter is about 14 µs peak to peak, and you can compare that against what the control algorithm tolerates. The first sample after init is garbage (it measures from zero) and should be discarded. A GPIO toggle on a scope gives the same information with zero code size and shows the distribution, which is often more revealing than min and max.

The resource budget

A resource budget is a document (a table in the repository is enough) that allocates every finite resource and is kept current:

Resource	Total	Budget	Current	Owner
Flash (app slot)	448 KB	400 KB (rest for growth)	312 KB	Arch lead
RAM	128 KB	112 KB (16 KB margin)	87 KB	Arch lead
CPU load (average)	100%	60%	34%	Each task owner
Interrupt latency, motor ISR	n/a	2 µs	1.1 µs	Motor team
Average current, sleep mode	n/a	40 µA	33 µA	Power owner

Budgets that are only measured before release are always blown before release. Track them in CI. Flash and RAM are trivial to check from the ELF:

tools/check_budget.sh

#!/bin/sh
set -eu
ELF="$1"
FLASH_BUDGET=409600       # 400 KB
RAM_BUDGET=114688         # 112 KB

# Berkeley format: text data bss dec hex filename
set -- $(arm-none-eabi-size -B "$ELF" | tail -n 1)
TEXT=$1; DATA=$2; BSS=$3
FLASH=$((TEXT + DATA))    # .data initial values live in flash too
RAM=$((DATA + BSS))

echo "flash: $FLASH / $FLASH_BUDGET bytes, ram: $RAM / $RAM_BUDGET bytes"
[ "$FLASH" -le "$FLASH_BUDGET" ] || { echo "FLASH BUDGET EXCEEDED"; exit 1; }
[ "$RAM"   -le "$RAM_BUDGET" ]   || { echo "RAM BUDGET EXCEEDED"; exit 1; }

Counting .data in both flash and RAM is the detail people miss: initialized data occupies flash for its initial values and RAM at runtime. RAM here does not include stacks if your linker script reserves them separately; budget them explicitly. Post the numbers as a comment on every pull request so growth is visible when it happens, not three months later. CPU load and latency are harder to automate, but a nightly HIL job that runs a fixed workload and reads the RTOS’s run-time statistics catches regressions.

Product variants

A successful product becomes a family: a cheaper SKU without the display, a pro version with an extra sensor, a board revision with a different flash chip, a regional variant with a different radio band. The goal is one codebase, many builds, never a copy of the repository per variant.

The structure that works:

• Board files describe hardware: pin maps, which peripheral instance is which, which chips are fitted. One per board revision (board_disco_f407.c, board_rev_c.c).
• Feature flags describe product capabilities: CONFIG_FEATURE_DISPLAY, CONFIG_FEATURE_LTE. Always defined to 0 or 1.
• A variant definition binds a board to a feature set, as a CMake preset or a Kconfig defconfig.
• Runtime detection where hardware differs subtly: a board ID read from resistor straps on GPIOs, or a flash chip identified by its JEDEC ID, so one binary can support several board revisions.

CMakePresets.json (excerpt)

{
  "configurePresets": [
    { "name": "sensor-basic", "inherits": "arm-release",
      "cacheVariables": { "BOARD": "rev_c", "FEATURE_DISPLAY": "OFF", "FEATURE_LTE": "OFF" } },
    { "name": "sensor-pro", "inherits": "arm-release",
      "cacheVariables": { "BOARD": "rev_c", "FEATURE_DISPLAY": "ON",  "FEATURE_LTE": "ON" } }
  ]
}

CI builds every variant on every change. A variant that is only built at release time is broken at release time. Keep the number of variants honest: each one multiplies test effort, and a feature flag that has been 1 in every shipping variant for two years should be deleted along with its #if.

Versioning and compatibility

Firmware does not run alone. It runs under a bootloader, talks to a host app or cloud service over a protocol, and may coexist with firmware on another core or a co-processor. Each of those interfaces needs its own version, and the product needs a compatibility matrix.

Use semantic versioning for the firmware image: MAJOR.MINOR.PATCH. Bump PATCH for fixes, MINOR for backwards-compatible features, MAJOR when something a user, host tool, or bootloader depends on breaks. Embed the version plus the git hash and build ID, as Build Systems, Toolchain Flags, and CI shows, so every crash report identifies the exact build.

Version the interfaces separately from the image:

Firmware	Bootloader API	Protocol version	Config schema	Minimum host app
1.4.x	2.0	3	5	2.1
1.5.0	2.1 (uses reset reason if present)	3, 4	6 (migrates 5)	2.1
2.0.0	2.1 required	4 only	6	3.0

Each row answers questions support will ask: “can this device in the field, with bootloader 2.0, take firmware 2.0.0?” (no: it requires bootloader API 2.1). Protocol and config schema versions let a device negotiate with older host apps and migrate stored settings forward. The rule that saves the most pain: firmware should accept at least the previous protocol version and migrate at least the previous config schema, so updates do not have to be lockstep across the whole system.

Code review for firmware

Generic code review checks naming and structure. Firmware review must also check the things that cause field failures. A reviewer reads every change asking:

• Concurrency. Is any variable shared between an ISR and a task, or two tasks? Is access atomic or protected? Is the critical section short?
• Blocking. Can this code wait? For how long, worst case? Is it called from a context with a deadline, or from an ISR?
• Bounds. Every index and length that comes from outside the function: checked before use? Size arithmetic that can overflow?
• Error paths. Every return value checked? On failure, is every acquired resource released? Does the error reach someone who can act?
• Register writes. Correct register and bit for this part (checked against the reference manual, not memory)? Read-modify-write on a register an ISR also writes? Reserved bits preserved? Write-1-to-clear flags cleared correctly?
• Timing. Timeouts on every hardware wait? Wraparound-safe time comparisons? Any assumption about clock frequency?
• Resources. Stack depth of new call chains, new static buffers, flash growth.

Keep changes small enough to review properly: 200 to 400 lines is a practical upper limit for a careful review. Safety, Standards, and Coding Guidelines turns this into a full checklist.

Documentation that stays true

Documentation decays at the rate the code changes. Keep only what earns its maintenance cost, and keep it next to the code:

• An architecture document (5 to 15 pages): layers, tasks and priorities, the budget table, memory map, the main data flows, and the threading rules. Updated when any of those change, as part of the same pull request.
• Module READMEs or header comments stating each module’s contract: responsibility, threading, blocking, errors, lifetime.
• Register-level comments that cite the source: /* RM0090 28.6.4: BRR = fck / baud for OVER8=0 */. A comment pointing at the reference manual section lets the next person verify the code.
• Architecture Decision Records (ADRs): short documents recording one significant decision, why it was made, and what was rejected.

docs/adr/0007-littlefs-for-logs.md

# ADR 0007: Use LittleFS on external SPI NOR for event logs

Status: Accepted (2026-03-14)

Context: Logs must survive power loss, hold 30 days at 1 record/s, and be
exportable as files over USB. Internal flash sectors are 128 KB with 10k cycles.

Decision: LittleFS on the 4 MB SPI NOR, logs as daily files, rotated at 30 days.

Rejected: FatFs (not power-fail safe); raw circular log (cannot export files).

Consequences: +18 KB flash, +2 KB RAM. Mount time measured at 90 ms on a full
device; boot sequence must tolerate this.

Two years later, when someone proposes switching to FatFs, the ADR answers why that was rejected and what changed would justify revisiting it.

Technical debt and refactoring with tests

Every firmware codebase accumulates debt: the driver written in a hurry for a trade show, the #if for a board revision that no longer exists, the global variable that “will be fixed later”. Debt is not a moral failure; unmanaged debt is. Keep a list of known debt items with their cost (what they slow down or risk) and pay them down continuously, a little in every sprint, rather than waiting for a rewrite that never gets scheduled.

Refactoring firmware safely follows a fixed order. First, pin the current behaviour with tests: characterization tests that record what the code does today, even where that behaviour is odd. Then make small changes, each one leaving the tests green and the firmware shippable. Re-run HIL tests for anything touching drivers or timing. Never mix a refactor and a behaviour change in one pull request; when something breaks, you need to know which one did it.

A reference architecture: connected sensor

Here is a complete architecture for a typical connected sensor product: a battery-powered device that samples environmental sensors, stores data, and reports to a gateway over a sub-GHz radio module on SPI, with firmware updates over the same link.

 +------------------------------- app/ ----------------------------------+
 |  app_fsm: INIT -> SAMPLING -> REPORTING -> SLEEPING -> UPDATING        |
 +----------------------------------------------------------------------+
        | commands/events (queues)                 ^ events
 +------+---------------- services/ ---------------+---------------------+
 | proto (frames,  | storage (config KV, | update (slot B   | power     |
 | acks, versions) |  event log)         |  writer, verify) | (sleep)   |
 +-----------------+---------------------+------------------+-----------+
        |                     |                    |
 +------+---- drivers/ -------+--------------------+---------------------+
 | radio (SPI + DIO IRQ) | env_sensor (regdev) | spi_nor | battery (ADC)  |
 +-----------------------+---------------------+---------+----------------+
 | hal/: spi1, i2c1, exti, adc1, tim, rtc, flash, iwdg                    |
 +-----------------------------------------------------------------------+
 | platform/: startup, linker scripts (boot / slot A / slot B), clocks    |
 +-----------------------------------------------------------------------+
 Tasks: radio (prio 5), app (4), storage (2), idle with WFI (0)

A packet’s journey, inbound

Follow a “set sample period” command from the gateway through the system:

1. Radio IRQ. The radio module finishes receiving a packet and raises its DIO pin. The EXTI ISR (priority 6 of 16) does one thing: notifies the radio task (a task notification from ISR), then returns. Time in ISR: under 2 µs.
2. Radio driver. The radio task wakes, reads the packet length and payload from the module over SPI (DMA, about 300 µs for 64 bytes at 2 MHz), and passes the bytes to proto_rx().
3. Protocol service. proto_rx() checks the frame: version, length against buffer size, CRC-16, sequence number (duplicate detection). It decodes the payload into a command_t and pushes it onto the app’s command queue. It queues an acknowledgement frame for the radio task to send. Malformed frames are counted and dropped, never asserted on.
4. Application. The app task pops the command, validates the range (100 ms to 1 hour), updates the active config, and asks storage to persist it by posting to the storage task’s queue. It does not wait for the write.
5. Storage. The storage task appends the new config record to the key-value store; if that triggers compaction, the sector erase takes 40 ms, which matters to nobody because no other task is waiting on it.

And back, outbound

1. Application builds a report (latest samples, battery voltage, health counters) at the next reporting interval and posts it to the protocol service.
2. Protocol encodes it, adds sequence number and CRC, and hands the frame to the radio task’s TX queue.
3. Radio task writes the frame to the module over SPI and starts transmission, then waits (blocked, not spinning) for the TX-done interrupt or a timeout. If no acknowledgement arrives within the window, protocol retries up to three times with back-off, then reports the failure as an event to the app, which logs it and keeps the data for the next report.
4. Power sees all queues empty and no pending timers before the next sample, and enters Stop mode until the RTC wake-up or a radio interrupt.

Every arrow in this walkthrough is a queue or an event, every wait has a timeout, every layer depends only downward, and every timing claim (2 µs, 300 µs, 40 ms) belongs in the budget table and is measured, not assumed.

Common mistakes

• Layering rules that exist only in a diagram. Fix: one build target per layer with explicit dependencies, plus an include checker in CI.
• A lower layer including an upper-layer header to check some mode. Fix: invert the dependency with configuration, a callback, or an event.
• Binary interfaces (bootloader API, shared ROM functions, IPC messages) without a version. Fix: magic plus major/minor in every shared table, checked at bind time.
• Multi-core shared memory without barriers, or cached on an M7. Fix: DMB between data and index updates, and non-cacheable or explicitly maintained shared regions.
• Resource budgets checked only before release. Fix: flash and RAM checks in every CI build, CPU and latency in nightly HIL.
• A branch or repository copy per product variant. Fix: board files, feature flags, and presets in one codebase, all variants built in CI.
• Refactoring and changing behaviour in the same pull request. Fix: characterization tests first, then refactor, then change behaviour separately.
• Architecture documents that describe the system as it was two years ago. Fix: update docs in the same pull request as the change, and keep decisions in ADRs.

Exercises

1. Run tools/check_layers.py on a project from an earlier part, after arranging its files into platform/, hal/, drivers/, services/, and app/. Fix every violation it reports by inverting the dependency, and describe in one sentence how you fixed each one.
2. Write the resource budget table for your capstone or largest project so far. Fill in current flash and RAM from arm-none-eabi-size, then add check_budget.sh to your build and set the budget 1 KB below the current usage to watch it fail.
3. Use jitter_init() and jitter_sample() on a 1 kHz timer interrupt. Measure min and max period with only that interrupt running, then add a second interrupt at the same priority that busy-waits 50 µs every 7 ms. Explain the new max period in terms of the interrupt model, then fix it with priorities and remeasure.
4. Implement the IPC ring on the host with two threads standing in for two cores (use C11 atomic_thread_fence(memory_order_release) and memory_order_acquire in place of DMB). Run a producer sending 10 million sequence-numbered messages and a consumer checking that none are lost or reordered. Then remove the fences and see if your machine shows a failure (x86 may not; explain why).
5. Write an ADR for a real decision in one of your projects (RTOS or not, which flash layout, which protocol). Include at least two rejected alternatives with reasons and the measurable consequences of the decision.

Check yourself

Question 1:In hal/uart.c, a developer needs to suppress break detection while the product is in factory mode. Which change respects the dependency rule?

#include "app/app_state.h" and check app_state_is_factory() Add a field to the UART config struct (or a setter) that the app sets when it enters factory mode Make the app's state variable extern so the HAL can read it Copy the factory-mode flag into a global in lib/

Question 2:An application built for bootloader API major 2, minor 1 boots on a device whose bootloader reports major 2, minor 0. What should the application do?

Refuse to run Call every function in the table, since the major version matches Run, but not call functions added in minor 1, treating that feature as absent Overwrite the bootloader with a newer one automatically

Question 3:Core A writes a message into a shared ring slot and then increments head. Core B occasionally reads stale message bytes even though it saw the new head. Both are Cortex-M4 cores without caches. What is missing?

volatile on the message slots A larger ring A DMB on core A between writing the message and updating head, and on core B between reading head and reading the message Disabling interrupts on core B

Question 4:arm-none-eabi-size -B reports text 300000, data 4000, bss 60000. How much flash and RAM does the image use (excluding separately reserved stack)?

Flash 300000, RAM 60000 Flash 364000, RAM 64000 Flash 304000, RAM 64000 Flash 300000, RAM 64000

Question 5:A 10 kHz control loop at 168 MHz measures min_period 16,700 and max_period 19,900 cycles. What does this tell you?

The timer is misconfigured for 9 kHz The loop start is being delayed by up to about 19 µs at times, most likely by another interrupt at equal or higher priority or a critical section The DWT counter has wrapped Nothing; DWT cycles are not related to time

Key takeaways

• Layering only exists if a machine enforces it: one build target per layer and an include checker in CI.
• Module contracts state threading, blocking, errors, and lifetime, in the header where callers look.
• Binary interfaces between separately built images carry magic and major/minor versions, and callers check them.
• Multi-core shared memory needs single-writer ownership, barriers between data and index, and non-cacheable or maintained regions on cached cores.
• Time budgets, WCET with margin, and measured jitter turn “keep it fast” into a verifiable system property.
• Flash, RAM, CPU, latency, and power budgets are tracked continuously in CI, not checked at release.
• One codebase serves all variants through board files, feature flags, and presets, and every variant builds on every change.

Next: Safety, Standards, and Coding Guidelines, where these practices meet MISRA, CERT, and the functional safety standards.