Skip to content

Part 4 · Professional Firmware26 min read

Embedded Linux and Where Bare Metal Ends

Everything in this course so far has run on a microcontroller: one address space, no MMU, your code in control of every cycle. Many products eventually outgrow that. They need a touchscreen UI, a full network stack with TLS and a web server, a camera, a database, or containerized updates, and the answer is a microprocessor (MPU) running Linux. This lesson explains when that move is right and what it costs, how a Linux system boots, how the device tree and driver model replace your register headers, how the concepts you know (MMIO, interrupts, DMA) appear in the kernel, how to split work between Linux and a real-time MCU, and how to drive a GPIO and an I2C sensor from Linux userspace in C.

An MCU like the STM32F407 has 1 MB of flash and 192 KB of RAM on chip. An MPU like a Cortex-A7 or Cortex-A53 has an MMU, external DDR memory measured in hundreds of megabytes, external flash or eMMC measured in gigabytes, and clock speeds around 1 GHz. That class of hardware runs Linux, and Linux brings capabilities that would take years to build on bare metal.

NeedMCU realityLinux reality
NetworkinglwIP, a few sockets, TLS strains RAMFull TCP/IP, Wi-Fi drivers, VPNs, firewalls, many simultaneous TLS connections
FilesystemsLittleFS on a few MBext4, UBIFS, SquashFS, overlay filesystems on GB of storage
User interfaceSmall displays, hand-rolled or LVGL graphicsGPU-accelerated graphics, Qt, Wayland, web-based UIs, video
SecuritySecure boot plus careful CProcess isolation, users and permissions, mandatory access control, maintained crypto libraries
Software reuseLibraries ported by handPython, databases, package ecosystems, thousands of drivers
MemoryHundreds of KBHundreds of MB to GB

The costs are just as real:

  • Boot time. An MCU runs main() within milliseconds of power-up. A typical embedded Linux system takes 5 to 30 seconds to reach its application; a carefully optimized one gets under 2 seconds, with significant effort.
  • Power. An MPU with DDR draws hundreds of milliwatts to watts when running, and deep sleep with DDR in self-refresh is milliwatts, not microwatts. Coin-cell products stay on MCUs.
  • Determinism. Linux is a general-purpose OS. Even with real-time patches, worst-case latency is tens of microseconds, against well under a microsecond for an MCU interrupt.
  • Complexity. Millions of lines of code you did not write, a build system that builds an entire distribution, and a security patching obligation for the life of the product.
  • Bill of materials. The MPU, external DDR, eMMC or NAND, and a multi-rail power management IC cost several times what an MCU and its passives cost, and the PCB often needs more layers to route DDR.

The usual decision rule: if the product needs a rich UI, heavy networking, large storage, or third-party software ecosystems, use Linux. If it needs microamp sleep currents, instant-on, hard real-time control, or the lowest possible cost, use an MCU. Many products need both, which is why heterogeneous designs are common.

On an MCU, reset loads the stack pointer and reset vector from flash and your code runs. On an MPU, booting is a relay race, because at power-up the DDR memory is not even initialized:

Power on
|
v
Boot ROM (in the SoC, immutable)
- reads boot pins / fuses to pick a boot device (eMMC, SD, SPI NOR, USB)
- loads the first stage into on-chip SRAM (tens to hundreds of KB)
- optionally verifies its signature (root of the secure boot chain)
|
v
First stage: SPL (U-Boot's secondary program loader) or TF-A BL2
- initializes clocks, PMIC, and DDR controller
- loads the full bootloader into DDR
|
v
U-Boot (full bootloader)
- loads kernel image, device tree blob, and optional initramfs
- selects A/B slot, passes the kernel command line
|
v
Linux kernel
- decompresses, sets up MMU, probes drivers listed in the device tree
- mounts the root filesystem
|
v
init (systemd, BusyBox init, or your own)
- starts services, then your application

This is the same chain-of-trust structure as Firmware Security described for MCUs (ROM, then bootloader, then application), with more stages. Secure boot on Linux verifies each stage’s signature with the previous stage’s key, ending with a signed kernel and a verified root filesystem (dm-verity is the common mechanism).

On the STM32F407 your code knew that USART2 lives at 0x40004400 because a header said so. That works because one firmware image targets one board. Linux supports thousands of boards with one kernel binary, so hardware description is moved out of the code into data: the device tree.

A device tree source file (.dts) describes the board: which peripherals exist, at which addresses, with which interrupts, clocks, and pins, and which external chips are attached to which buses. It is compiled to a binary blob (.dtb) that the bootloader passes to the kernel.

board.dts (excerpt)
&i2c1 {
status = "okay";
clock-frequency = <100000>;
pinctrl-names = "default";
pinctrl-0 = <&i2c1_pins>;
temp_sensor: temperature@48 {
compatible = "ti,tmp102";
reg = <0x48>;
};
};
leds {
compatible = "gpio-leds";
status_led {
label = "status";
gpios = <&gpioa 13 GPIO_ACTIVE_HIGH>;
default-state = "off";
};
};

Each node has a compatible string. At boot, the kernel matches compatible strings against its drivers and calls the matching driver’s probe function with the node’s resources (register address, interrupt number, bus address). The &i2c1 reference modifies a node defined in the SoC’s .dtsi include file, where the I2C controller’s register base and interrupt are already described; the board file only enables it, sets the clock rate, and adds the sensor at address 0x48.

Compare to what you have been doing:

Bare metal (this course)Linux
#define USART2_BASE 0x40004400u in a headerreg = <0x40004400 0x400>; in the SoC .dtsi
NVIC_EnableIRQ(38)interrupts = <...>; in the node; the driver requests it
Board pin map in board.cpinctrl nodes
Which sensors are fitted: #if or board fileChild nodes under the bus; drivers probe only what is listed
Rebuild firmware to change hardware descriptionChange the .dtb (or apply an overlay) without rebuilding the kernel

Device tree overlays let a base board description be patched at boot, which is how add-on boards and hardware variants are handled without a separate kernel per SKU.

On Linux, your application cannot write to 0x40004400. It runs in userspace, in a virtual address space enforced by the MMU, without privilege to touch hardware. Hardware access goes through the kernel, either via a full kernel driver or via a generic interface the kernel exposes to userspace.

InterfaceWhat it isUse when
Kernel driver (hwmon, iio, input, net)Full driver in the kernel, exposes a standard interface (/sys/class/hwmon/..., /dev/input/event0, a network interface)A driver already exists, or the device needs interrupts and DMA with tight timing
/dev/gpiochipNCharacter device for GPIO lines, used through ioctls or libgpiodToggling outputs, reading inputs, waiting for edge events from userspace
spidev (/dev/spidevB.C)Raw SPI transfers from userspacePrototyping, simple SPI devices without a kernel driver
i2c-dev (/dev/i2c-N)Raw I2C transfers from userspacePrototyping, simple register-style devices
UIOMaps a device’s registers into a userspace process, delivers interrupts via read()Custom FPGA logic or peripherals where a small kernel stub plus a userspace driver is simpler

Userspace drivers are easier to write and debug (a crash kills a process, not the system), can use any library, and do not have to follow kernel coding and licensing rules. Kernel drivers are required when you need interrupt latency in microseconds, DMA, integration with a kernel subsystem (so standard tools work), or access from many processes. A common path is to prototype in userspace and move to a kernel driver if the product needs it.

You will sometimes see code that opens /dev/mem and mmaps physical register addresses. It works, and it is useful for a quick experiment, but it bypasses every protection, races with any kernel driver using the same hardware, and requires root. Do not ship it.

How familiar concepts appear in the kernel

Section titled “How familiar concepts appear in the kernel”

If you write a kernel driver, everything from Parts 2 and 3 is still there, under different names:

ConceptBare metalLinux kernel
Register accessvolatile pointer dereferencedevm_platform_ioremap_resource() to map, then readl() / writel() (which include the right barriers)
Interrupt handlervoid USART2_IRQHandler(void) in the vector tabledevm_request_irq() or devm_request_threaded_irq(): a short top half plus a threaded bottom half
Deferring work from an ISRSet a flag, post to a queueThreaded IRQ, workqueue, tasklet
Critical section with an ISRPRIMASK save, disable, restorespin_lock_irqsave() / spin_unlock_irqrestore()
Mutex between tasksRTOS mutexmutex_lock() (may sleep; never in interrupt context)
DMA bufferStatic array in DMA-reachable RAM, cache care on M7dma_alloc_coherent(), or streaming mappings with dma_map_single() that handle cache maintenance
Delaydelay_ms() (bad), timer servicemsleep(), usleep_range() in process context; udelay() only for tiny busy waits
Init and cleanupinit() / deinit(), goto cleanupprobe() / remove(), devm_* managed resources freed automatically, goto cleanup
Error codesProject err_tNegative errno values (-EIO, -ETIMEDOUT, -ENODEV)

The rules you learned carry over exactly: keep interrupt handlers short, never sleep in interrupt context, protect shared data, mind the caches around DMA, and release resources in reverse order. The kernel enforces some of them (sleeping in atomic context produces a loud warning) and has tools to find others (lockdep for lock ordering problems, KASAN for memory errors).

Building the system: Yocto, Buildroot, or Debian

Section titled “Building the system: Yocto, Buildroot, or Debian”

An embedded Linux product needs a bootloader, a kernel, a device tree, a root filesystem with libraries and your application, and an update mechanism. Three common ways to produce them:

  • Buildroot. A set of Makefiles and Kconfig menus that build a cross toolchain, kernel, bootloader, and a small root filesystem from source. Simple to learn, fast to rebuild, produces small images (tens of MB). No package manager on the target: the image is built as a whole. Good for single-purpose devices and small teams.
  • Yocto Project (OpenEmbedded). A powerful, layered build framework where recipes describe how to build each component and layers add BSPs, features, and products. Steep learning curve and long first builds (hours), but excellent for product families, long maintenance, license compliance reports, and SBOM generation. Most silicon vendors ship their BSPs as Yocto layers.
  • Debian or Ubuntu based images. Start from a binary distribution for your architecture and add your packages. Fastest to get going and familiar to developers, but images are large, boot is slower, and controlling exactly what ships (and patching it for years) is harder. Common for gateways and prototypes on boards such as the Raspberry Pi.

Whichever you choose, plan updates from day one: an A/B root filesystem scheme with a tool such as RAUC, SWUpdate, or Mender, signed images, and rollback on failed boot. The bootloader concepts from Bootloaders and Firmware Update apply unchanged.

“Can Linux do real time?” depends on the deadline. The mainline kernel with the PREEMPT_RT configuration (merged into mainline Linux in 2024, after many years as an external patch set) makes almost all kernel code preemptible, turns interrupt handlers into schedulable threads, and replaces most spinlocks with priority-inheriting mutexes. On a tuned system with PREEMPT_RT, worst-case scheduling latency for a high-priority real-time thread is typically in the tens of microseconds; on a stock kernel under load it can reach milliseconds.

That is good enough for many control loops at 1 kHz. It is not good enough for motor commutation at 20 kHz, for microsecond-accurate pulse generation, or for anything where a certification body wants a provable bound. Measure your own system with cyclictest under realistic load (network traffic, storage I/O, GPU activity), and treat the worst number you see after hours of testing as optimistic.

When the deadline is tighter than Linux can promise, put that function on a microcontroller.

Many products combine both worlds. Two common forms:

  • Separate chips. An MPU running Linux talks to an STM32 over UART, SPI, USB, or CAN. The MCU handles motor control, safety monitoring, sensor sampling, and low-power standby; Linux handles UI, networking, and storage.
  • One SoC with both core types. Parts such as the STM32MP1 (Cortex-A7 plus Cortex-M4) and the NXP i.MX 8M family (Cortex-A53 plus Cortex-M4 or M7) put both on one die. Linux’s remoteproc framework loads and starts the MCU core’s firmware, and RPMsg provides message channels between them over shared memory, appearing on the Linux side as a character device or a TTY.

Split responsibilities by asking what must keep working when Linux is slow, rebooting, or being updated:

Put on the MCUPut on Linux
Hard real-time loops (motor control, power conversion)User interface, web server, cloud connectivity
Safety monitoring and safe-state controlData storage, databases, logs
Wake-up sources and power sequencing (the MCU often turns Linux on and off)Firmware update orchestration, including updating the MCU
Fast sensor sampling with precise timingHeavy computation, analytics, machine learning

The link between them is a protocol you design with all the rules from Communication Stacks: framing, CRC, version negotiation (the two sides are updated separately), sequence numbers, and a watchdog on each side so one can detect and recover the other.

Worked example: GPIO and I2C from userspace

Section titled “Worked example: GPIO and I2C from userspace”

Here is a complete userspace program that blinks an LED on a GPIO line and reads the temperature sensor from Testing Embedded Software (a register-style I2C device at 0x48, register 0x00, 16-bit big-endian, 1/128 °C per count) through i2c-dev. It uses only the kernel’s userspace APIs, so it needs no libraries beyond libc.

Since Linux 4.8, GPIOs are accessed through /dev/gpiochipN character devices; the old /sys/class/gpio interface is deprecated. The program requests a line as an output with the version 2 ioctl interface (Linux 5.10 and later), which returns a new file descriptor that owns the line:

gpio_line.c
#define _POSIX_C_SOURCE 200809L
#include <errno.h>
#include <fcntl.h>
#include <linux/gpio.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <unistd.h>
/* Returns a line fd that owns the GPIO, or -1. */
int gpio_request_output(const char *chip, unsigned int offset, const char *consumer)
{
int chip_fd = open(chip, O_RDONLY | O_CLOEXEC);
if (chip_fd < 0) {
perror("open gpiochip");
return -1;
}
struct gpio_v2_line_request req;
memset(&req, 0, sizeof req);
req.offsets[0] = offset;
req.num_lines = 1;
req.config.flags = GPIO_V2_LINE_FLAG_OUTPUT;
strncpy(req.consumer, consumer, sizeof req.consumer - 1);
int rc = ioctl(chip_fd, GPIO_V2_GET_LINE_IOCTL, &req);
int saved = errno;
close(chip_fd); /* the line fd stays valid */
if (rc < 0) {
errno = saved;
perror("GPIO_V2_GET_LINE_IOCTL");
return -1;
}
return req.fd;
}
int gpio_set(int line_fd, int value)
{
struct gpio_v2_line_values v;
memset(&v, 0, sizeof v);
v.mask = 1; /* bit 0 = first line in the request */
v.bits = value ? 1 : 0;
return ioctl(line_fd, GPIO_V2_LINE_SET_VALUES_IOCTL, &v);
}

The design is the opaque-handle pattern from Design Patterns for Embedded C, enforced by the kernel: the line belongs to whichever file descriptor requested it, another process asking for the same line gets EBUSY, and when the process exits (or crashes) the kernel releases the line automatically. The consumer string shows up in gpioinfo output, so you can see who holds which line. Bits in gpio_v2_line_values refer to positions in the request’s offsets array, not to line numbers, which lets one request control several lines atomically.

The i2c-dev driver exposes each I2C bus as /dev/i2c-N. For register-style devices, the I2C_RDWR ioctl performs a write of the register pointer followed by a read, joined by a repeated start, in one call: exactly the transaction the F4 I2C peripheral needed a page of register sequencing to perform in SPI and I2C.

i2c_temp.c
#define _POSIX_C_SOURCE 200809L
#include <fcntl.h>
#include <linux/i2c.h>
#include <linux/i2c-dev.h>
#include <stdint.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <unistd.h>
int i2c_read_reg(int bus_fd, uint16_t addr, uint8_t reg, uint8_t *buf, uint16_t len)
{
struct i2c_msg msgs[2] = {
{ .addr = addr, .flags = 0, .len = 1, .buf = &reg },
{ .addr = addr, .flags = I2C_M_RD, .len = len, .buf = buf },
};
struct i2c_rdwr_ioctl_data xfer = { .msgs = msgs, .nmsgs = 2 };
return (ioctl(bus_fd, I2C_RDWR, &xfer) == 2) ? 0 : -1;
}
int temp_read_centi(int bus_fd, int32_t *out)
{
uint8_t raw[2];
if (i2c_read_reg(bus_fd, 0x48, 0x00, raw, sizeof raw) != 0) {
return -1;
}
int32_t counts = (int32_t)(((uint32_t)raw[0] << 8) | raw[1]);
if (counts >= 0x8000) {
counts -= 0x10000;
}
*out = (counts * 100) / 128;
return 0;
}

I2C_RDWR returns the number of messages transferred, so anything other than 2 is a failure (a NAK from the device shows up as -1 with errno set to EREMOTEIO or ENXIO depending on the controller driver). The conversion code is identical to the MCU driver: the C you wrote for the STM32 runs unchanged, only the bus access function differs. That is the payoff of the bus abstraction from Driver Design and Hardware Abstraction.

main.c
#define _POSIX_C_SOURCE 200809L
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <time.h>
#include <unistd.h>
int gpio_request_output(const char *chip, unsigned int offset, const char *consumer);
int gpio_set(int line_fd, int value);
int temp_read_centi(int bus_fd, int32_t *out);
int main(void)
{
int led = gpio_request_output("/dev/gpiochip0", 17, "fwpath-demo");
int bus = open("/dev/i2c-1", O_RDWR | O_CLOEXEC);
if (led < 0 || bus < 0) {
fprintf(stderr, "setup failed\n");
return 1;
}
const struct timespec half_second = { .tv_sec = 0, .tv_nsec = 500000000L };
for (int i = 0; i < 10; i++) {
int32_t centi;
gpio_set(led, i & 1);
if (temp_read_centi(bus, &centi) == 0) {
int32_t whole = centi / 100;
int32_t frac = (centi < 0 ? -centi : centi) % 100;
printf("led=%d temp=%s%d.%02d C\n", i & 1,
(centi < 0 && whole == 0) ? "-" : "",
(int)whole, (int)frac);
} else {
printf("led=%d temp=read error\n", i & 1);
}
nanosleep(&half_second, NULL);
}
gpio_set(led, 0);
close(led);
close(bus);
return 0;
}
Terminal window
aarch64-linux-gnu-gcc -std=c11 -Wall -Wextra -O2 main.c gpio_line.c i2c_temp.c -o demo
scp demo root@device:/tmp/ && ssh root@device /tmp/demo
led=0 temp=23.43 C
led=1 temp=23.43 C
led=0 temp=23.50 C
led=1 temp=23.50 C
...

Cross-compile with the toolchain for your board’s architecture (aarch64-linux-gnu-gcc for 64-bit Arm, arm-linux-gnueabihf-gcc for 32-bit), or with the SDK your Yocto or Buildroot build produced, which matches the target’s C library exactly. Line 17 on gpiochip0 and bus 1 are examples; use gpioinfo and i2cdetect -l from libgpiod-tools and i2c-tools to find the right ones on your board. The temperature print handles readings between -1.00 and 0.00 °C, where integer division gives a whole part of zero and the sign would otherwise be lost. Note that printf here is perfectly fine: this is a process with a real C library and a terminal, not an ISR on a microcontroller.

One caution about mixing approaches. If the device tree binds a kernel driver to the sensor (the ti,tmp102 node above does), the kernel owns that device and the reading appears in /sys/class/hwmon/hwmonN/temp1_input in millidegrees. Accessing the same address through i2c-dev at the same time races with the driver. Use the kernel interface when a driver exists, and raw i2c-dev when it does not.

To run the program as a service that starts at boot and restarts on failure, a systemd unit is a few lines:

/etc/systemd/system/fwpath-demo.service
[Unit]
Description=Firmware Path demo sensor service
After=multi-user.target
[Service]
ExecStart=/usr/bin/fwpath-demo
Restart=on-failure
RestartSec=2
[Install]
WantedBy=multi-user.target

The Restart=on-failure line is the Linux equivalent of the watchdog-and-reset pattern for one process; systemd can also feed a hardware watchdog so a hung system resets.

Almost everything in this course transfers directly:

  • C and memory. Pointers, layout, undefined behavior, and the discipline around them are the same in kernel and userspace code.
  • Concurrency. Races, atomics, memory ordering, and locking apply to threads and to kernel code, with more cores and real hardware reordering.
  • Protocols and data. Framing, CRCs, versioning, serialization, power-fail-safe storage.
  • Debugging method. Reproduce, isolate, instrument, prove. On Linux you add gdb on processes, strace for system calls, perf for profiling, dmesg and ftrace for the kernel.
  • Hardware literacy. Reading datasheets, scopes and logic analyzers, understanding what a bus transaction should look like. Linux driver work needs this as much as MCU work.

What you need to learn next, in roughly this order: Linux userspace systems programming (processes, file descriptors, poll/epoll, signals, threads with POSIX APIs); systemd and how a Linux system is put together; the networking stack from the application side; one build system (Buildroot first, then Yocto); device tree syntax and bindings; and finally kernel modules and the driver model, starting with a simple platform driver and the devm_* APIs.

  • Choosing Linux for a product that must sleep at microamps or respond in microseconds. Fix: an MCU, or an MCU co-processor that owns those functions.
  • Driving hardware through /dev/mem in production. Fix: an existing kernel driver, a userspace interface (gpiochip, spidev, i2c-dev, UIO), or a real kernel driver.
  • Using the deprecated /sys/class/gpio interface in new code. Fix: the /dev/gpiochipN character device directly or through libgpiod.
  • Accessing a device through i2c-dev while a kernel driver is bound to it. Fix: use the driver’s interface, or remove the node from the device tree.
  • Assuming PREEMPT_RT gives MCU-like guarantees. Fix: measure worst-case latency with cyclictest under load and keep tight loops on an MCU.
  • Building the image without a plan for updates and security patches. Fix: A/B rootfs with a maintained update tool and a patching process for the product’s lifetime.
  • Linking a userspace program against the host’s C library instead of the target’s. Fix: use the SDK or sysroot generated by your Buildroot or Yocto build.
  1. On any Linux machine, list the GPIO chips and lines with gpioinfo (or on a PC, ls /sys/bus/i2c/devices and i2cdetect -l). Explain what each column means and which consumers currently hold lines.
  2. On a Raspberry Pi or similar board, wire an LED and build the GPIO part of the example. Then start a second copy of the program while the first runs and record the error. Kill the first copy with kill -9 and show that the line is released without any cleanup code running.
  3. Connect any register-style I2C sensor, find its address with i2cdetect -y 1, and adapt temp_read_centi() to its register map. Compare the bytes on the bus with a logic analyzer against the STM32 version from SPI and I2C; the transactions should be identical.
  4. Install rt-tests and run cyclictest -m -p 90 -i 1000 -l 100000 on an idle system, then again while running stress-ng --cpu 4 --io 2 and a large network transfer. Record the maximum latencies and decide whether a 1 kHz control loop and a 20 kHz loop could run there.
  5. Design the split for a product with a touchscreen, Wi-Fi cloud connectivity, a brushless motor, and a door interlock that must stop the motor within 10 ms. Assign each function to an MPU or an MCU, define the messages on the link between them with versioning and acknowledgements, and describe what the system does while Linux reboots after an update.

Question 1:A battery product must wake once per minute, take a reading, and sleep at under 5 µA, for two years. Which platform fits?

Question 2:On a Linux board, what does the device tree replace from your bare-metal code?

Question 3:A userspace program uses I2C_RDWR with two messages (write register pointer, then read). What happens on the bus?

Question 4:A control loop must run at 20 kHz with under 5 µs jitter. The product already has a Linux MPU. What is the sound design?

Question 5:A process holding a GPIO line through GPIO_V2_GET_LINE_IOCTL crashes. What happens to the line?

  • Move to an MPU with Linux for rich UI, heavy networking, large storage, and software ecosystems; stay on an MCU for microamp sleep, instant-on, hard real time, and lowest cost.
  • An MPU boots through a chain: ROM, first stage that initializes DDR, U-Boot, kernel, init, application, with signatures checked at each step for secure boot.
  • The device tree describes hardware as data, replacing register headers and board files, and the kernel probes drivers by matching compatible strings.
  • Userspace reaches hardware through kernel drivers or generic interfaces (/dev/gpiochipN, spidev, i2c-dev, UIO), never through /dev/mem in production.
  • MMIO, interrupts, deferral, locking, DMA, and cleanup all exist in the kernel under new names, with the same rules.
  • PREEMPT_RT brings worst-case latency to tens of microseconds; anything tighter goes on an MCU, often in a heterogeneous MPU plus MCU design.

Next: Production, Provisioning, and the Field, where firmware leaves the lab and meets factories, fleets, and years of support.