rhdl_dsig_scalable_core/

engine.rs

//! The sustained-signing RTL kernel: eight slices, one certificate unit, HBM.
//!
//! (Eight in the default build; [`SLICES`] is 2, 4 or 8, chosen by the
//! `DSIG_SLICES` build variable. The diagrams below draw the 8-slice case.)
//!
//! # The problem
//!
//! Everything else in this crate is a block: a key factory, a signer, a
//! certificate unit. A card needs *one* top-level RTL kernel that Vitis can
//! link: a box with a host control port and some HBM ports. This module is
//! that box. It answers three questions:
//!
//! 1. **How does the host talk to it?** Through a register file on
//!    AXI4-Lite that follows the Vitis `ap_ctrl_hs` convention ("handshake"
//!    control: the host sets `ap_start`, the kernel reports `ap_idle`,
//!    `ap_done`, `ap_ready`). That lets ordinary XRT host code start the
//!    kernel and pass arguments without a custom driver.
//! 2. **How do messages get in and frames get out?** One HBM read master
//!    ([`crate::msg_reader`]) for messages and one HBM write master per slice
//!    ([`crate::hbm_writer`]) for frames.
//! 3. **How do we measure it honestly?** With a free-running cycle counter
//!    that is latched at start and again when the last frame is acknowledged
//!    by memory.
//!
//! # The pieces and how data flows
//!
//! ```text
//!            AXI-Lite (host)                         HBM
//!                 │                                   ▲  ▲ … ▲
//!   ┌─────────────▼───────────────────────────────────┼──┼───┼──┐
//!   │ registers ── config/start ──┐            writer0 … writer7 │
//!   │    ▲ counters, ticks        ▼               ▲          ▲   │
//!   │    │              MessageReader ──msg n──▶ slice (n mod 8) │
//!   │    │                   ▲ HBM                │ roots / certs │
//!   │    └────────────── CertUnit ◀───────────────┘               │
//!   └──────────────────────────────────────────────────────────────┘
//! ```
//!
//! Follow one message through the kernel:
//!
//! 1. The host writes the arguments (addresses, counts, namespace, secrets)
//!    into the registers, then sets `ap_start`.
//! 2. [`MessageReader`] fetches messages from HBM and deals message `n` to
//!    slice `n mod SLICES`.
//! 3. Each [`Slice`] has been making one-time keys since start. It builds its
//!    subtree root for each batch and hands the root to the [`CertUnit`],
//!    which joins all slice roots into the batch root and signs it with
//!    Ed25519. The certificate comes back to every slice.
//! 4. The slice signs the message with a fresh key and streams the
//!    26-beat frame to its own [`HbmWriter`], which writes it to HBM.
//!
//! Every signal between a slice and a shared block passes through registers
//! ([`crate::relay`]): handshakes through relay pipes, the rest through
//! two-register delay lines. The slices sit in their own SLRs, up to two die
//! crossings from the shared blocks next to the HBM, and build v9 showed
//! that an unregistered link across that distance cannot run at 205 MHz.
//! 5. When the writers' acknowledged-frame counts add up to `count`, the
//!    kernel stops, latches `DONE_TICK` and raises `ap_done`.
//!
//! # The run lifecycle
//!
//! ```text
//!   reset ──▶ IDLE ──host writes args──▶ IDLE ──AP_CTRL bit0=1──▶ RUNNING
//!              │ (args writable,                                    │
//!              │  no read path for secrets)          frames_total == count
//!              │                                                    ▼
//!              └────────── only a reset leaves ◀──────────────── DONE
//!                                                   (start_tick, done_tick
//!                                                    latched; ap_done set)
//! ```
//!
//! `running` and `done` in [`EngineControl`] encode the states: IDLE is
//! neither, RUNNING is `running`, DONE is `done`. A start request is ignored
//! unless both are clear, and argument writes are ignored unless both are
//! clear, so nothing about a run can change after it starts.
//!
//! # How a run is measured
//!
//! The host programs the configuration registers, then writes `CTRL.start`.
//! The kernel latches the free-running tick counter at start, makes every key
//! from scratch, and latches the counter again when the last frame's HBM write
//! response arrives. `(done_tick − start_tick)` cycles at the clock frequency
//! the host measures through the same counter
//! (reading the low word snapshots the high word) is
//! the denominator of the reported rate. The numerator is the number of frames
//! the host reads back and verifies.
//!
//! One run per kernel reset: the kernel never restarts on its own, and the host
//! must allocate a fresh namespace (epoch, batch range, master seed) for every
//! run, because keys made but not spent are still burned.
//!
//! # Register map (32-bit words)
//!
//! The control block follows the Vitis `ap_ctrl_hs` convention, so the host
//! drives the kernel with ordinary `xrt::kernel` / `xrt::run` calls and
//! arguments; every argument below is declared in `kernel.xml` at its offset.
//!
//! | Offset | Name | Access |
//! |---|---|---|
//! | `0x000` | `AP_CTRL`: bit 0 `ap_start`, 1 `ap_done` (clear on read), 2 `ap_idle`, 3 `ap_ready` | RW |
//! | `0x004` | `GIE` global interrupt enable | RW |
//! | `0x008` | `IER` bit 0 = done interrupt | RW |
//! | `0x00c` | `ISR` bit 0 = done (write 1 to toggle) | RW |
//! | `0x010` | arg `messages`: message buffer address (u64) | W |
//! | `0x018 + 8s` | arg `frames{s}`: slice `s` frame buffer address (u64) | W |
//! | `0x058` | arg `count`: messages to sign (u32) | W |
//! | `0x05c` | arg `batches`: batches to make (u32) | W |
//! | `0x060` | arg `epoch` (u64) | W |
//! | `0x068` | arg `first_batch` (u64) | W |
//! | `0x070 + 4w` | args `cv0`..`cv7`: derive-key prefix CV (secret) | W |
//! | `0x090 + 4w` | args `seed0`..`seed7`: Ed25519 seed (secret) | W |
//! | `0x200` | `MAGIC` = `0x33475344` ("DSG3") | R |
//! | `0x204` | `ABI` = [`ABI_VERSION`] | R |
//! | `0x208` | `SHAPE`: slices in bits 0..8, table mask in bits 8..24 | R |
//! | `0x20c` | `FAULT` bits: 0 writer, 1 reader, 2 cert, 3 slice | R |
//! | `0x210`/`0x214` | `TICKS` lo (snapshots hi) / hi | R |
//! | `0x218`/`0x21c` | `START_TICK` lo/hi | R |
//! | `0x220`/`0x224` | `DONE_TICK` lo/hi | R |
//! | `0x228` | `CERTIFIED` batches | R |
//! | `0x22c` | `DEALT` messages | R |
//! | `0x230`/`0x234` | `CERT_BUSY` cycles lo/hi | R |
//! | `0x240 + 4s` | `FRAMES[s]` acknowledged per slice | R |
//! | `0x260 + 4s` | `KEYS[s]` started per slice | R |
//! | `0x280 + 4w` | `PUBLIC_KEY` words (8 × u32) | R |
//!
//! Arguments are accepted only while the kernel is idle and have no read
//! path, so secrets cannot be read back. A completed kernel stays done until
//! the next reset: one run per reset, each with a fresh namespace.
//!
//! # Why the rules are so strict
//!
//! A W-OTS+ one-time key that signs two different messages leaks enough of
//! its chains to forge. So the kernel is built so that a key can never be
//! reused, even by accident:
//!
//! * A one-time key is never reused after admission, timeout, reset or
//!   uncertain launch. The kernel has no "resume": after a reset it starts
//!   from nothing, and the host must give it a fresh namespace (epoch, batch
//!   range, master seed). [`crate::lease`] is the host-side guard that never
//!   hands out the same run directory, namespace or batch range twice.
//! * Secrets are write-only. `cv0..cv7` (the derive-key prefix chaining
//!   value) and `seed0..seed7` (the Ed25519 seed) have no read path in the
//!   register file. The master seed itself is never stored on the card; the
//!   host passes only the derived prefix CV (see
//!   [`crate::slice_model::prefix_cv`]).
//! * Arguments freeze at start. Writes to argument offsets are dropped while
//!   running or done.
//!
//! # Neighbours
//!
//! | Child | Module | Role here |
//! |---|---|---|
//! | `reader` | [`crate::msg_reader`] | HBM messages → per-slice buffers |
//! | `slices` | [`crate::slice`] | key factory, cache, subtree, signer |
//! | `cert` | [`crate::cert_unit`] | Merkle top + Ed25519 certificate per batch |
//! | `writers` | [`crate::hbm_writer`] | frames → HBM, one port per slice |
//! | `beats` | [`crate::relay`] | relay pipes: frame beats flow at up to one per clock |
//! | `messages`, `certificates` | [`crate::relay`] | toggle links: at most one message per hash round trip and one certificate per 128 signatures need no relay stations |
//! | `to_slice`, `from_slice` | [`crate::relay`] | delay lines for configuration, roots, credit and status |
//! | (types) | [`crate::axi4`] | the AXI4 / AXI4-Lite signal structs |
//!
//! The Verilog for this kernel is produced by the `dsig_engine_export`
//! binary, and the host program that drives it is `dsig_engine_host`.

use crate::axi4::{ReadRequest, ReadResponse, WriteRequest, WriteResponse};
use crate::cert_unit::{CertConfig, CertInput, CertUnit, SLICES};
use crate::ed25519::ScalarWords;
use crate::hbm_writer::{HbmWriter, WriterInput};
use crate::msg_reader::{MessageReader, ReaderInput};
use crate::relay::{Delay, PipeIn, RelayPipe, ToggleLink};
use crate::slice_merkle::SubtreeRoot;
use crate::slice_signer::{Certificate, FrameBeat, Message};
use crate::slice::{Slice, SliceInput};
use crate::slice_factory::FactoryConfig;
use crate::axi4::{AxiLiteIn, AxiLiteOut};
use rhdl::core::ReplicatedSynchronous;
use rhdl::prelude::*;
use rhdl_fpga::core::dff::DFF;

/// `MAGIC` register value ("DSG3").
///
/// Read at `0x200`. The bytes `0x44 0x53 0x47 0x33` are ASCII "DSG3" in
/// little-endian order. The card runner
/// (`fpga/u280/engine/host/dsig_engine_runner.cpp`) checks it before anything
/// else, so it never drives a different kernel with this register map.
pub const MAGIC: b32 = b32(0x3347_5344);
/// Register-map version.
///
/// Read at `0x204`. It changes whenever the register map changes, so host
/// code can tell which layout a kernel image has; a moved register could, for
/// example, turn a count into an address.
pub const ABI_VERSION: b32 = b32(0x0003_0001);

/// Kernel inputs: every signal that enters the kernel from outside.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct EngineInput {
    /// Host control channel (AXI4-Lite into the register file).
    pub axi: AxiLiteIn,
    /// Message read channel (the HBM port of the message reader).
    pub messages: ReadResponse,
    /// Frame write channels, one per slice (each slice's own HBM port).
    pub frames: [WriteResponse; SLICES],
}

/// Kernel outputs: every signal that leaves the kernel.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct EngineOutput {
    /// Host control channel (register reads, write acknowledgements and the
    /// interrupt line).
    pub axi: AxiLiteOut,
    /// Message read channel.
    pub messages: ReadRequest,
    /// Frame write channels, one per slice.
    pub frames: [WriteRequest; SLICES],
}

/// Host-written configuration: the kernel's arguments.
///
/// Written over AXI4-Lite before start and frozen while running. Together,
/// `prefix_cv`, `epoch` and the batch range `first_batch .. first_batch +
/// batches` decide every key the run makes, which is why the host must never
/// repeat them (see the module notes).
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct Registers {
    /// Derive-key prefix CV (8 × 32 bits, secret, write-only): the BLAKE3
    /// chaining value from which every one-time key of this signer is
    /// derived. Args `cv0`..`cv7` at `0x070 + 4w`.
    pub prefix_cv: [b32; 8],
    /// Epoch (64 bits): part of the run's namespace; mixed into every key
    /// and into every certificate envelope.
    pub epoch: b64,
    /// First batch id (64 bits): the number of the run's first 128-key
    /// batch. Batch ids are part of the namespace.
    pub first_batch: b64,
    /// Batches to make (32 bits). Each batch is 128 keys.
    pub batches: b32,
    /// Messages to sign (32 bits). The run is done when this many frames
    /// have been acknowledged by HBM.
    pub count: b32,
    /// Message base address (64-bit HBM byte address, 4 KiB aligned).
    pub msg_base: b64,
    /// Frame base address per slice (64-bit HBM byte address, 2 KiB aligned).
    pub out_base: [b64; SLICES],
    /// Ed25519 seed words (32-bit halves; 8 × 32 = 256 bits, secret,
    /// write-only): the certificate signing key. Args `seed0`..`seed7` at
    /// `0x090 + 4w`.
    pub seed: [b32; 8],
}

/// Resettable control state.
///
/// This one register holds the configuration, the run state machine, the
/// timing counters and the AXI4-Lite handshake latches. All of it resets to
/// zero (see `Default for Engine`), which is the IDLE state with no
/// arguments.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct EngineControl {
    /// Configuration (the host-written arguments).
    pub regs: Registers,
    /// Running: set by `ap_start`, cleared when the last frame is acknowledged.
    pub running: bool,
    /// Finished (sticky until reset). Blocks a second start and any argument
    /// write, enforcing one run per reset.
    pub done: bool,
    /// `ap_done` not yet read by the host. `ap_ctrl_hs` makes `ap_done` clear
    /// on read, so this bit is cleared when `AP_CTRL` is read.
    pub done_unread: bool,
    /// Global interrupt enable (`GIE`, bit 0 of `0x004`).
    pub gie: bool,
    /// Interrupt enable (done) (`IER`, bit 0 of `0x008`).
    pub ier: bool,
    /// Interrupt status (done) (`ISR`, bit 0 of `0x00c`). Set when the run
    /// finishes; the host toggles it by writing 1.
    pub isr: bool,
    /// Free-running cycle counter (64 bits). Counts every clock from reset;
    /// the host uses it both to time the run and, by sampling it twice
    /// against its own wall clock, to measure the clock frequency.
    pub ticks: b64,
    /// Tick at start (64 bits): the value of `ticks` on the cycle the start
    /// write was handled.
    pub start_tick: b64,
    /// Tick when the last frame was acknowledged (64 bits).
    pub done_tick: b64,
    /// High half captured when `TICKS` lo is read. A 64-bit counter is read
    /// as two 32-bit words; capturing the high half at the moment the low
    /// half is read makes the pair consistent even if the low half wraps in
    /// between.
    pub ticks_hi: b32,
    /// Latched write address (12 bits). AXI4-Lite may deliver the address
    /// (AW) and the data (W) on different cycles, so each is held until both
    /// have arrived.
    pub aw_addr: b12,
    /// A write address is latched.
    pub aw_have: bool,
    /// Latched write data (32 bits).
    pub w_data: b32,
    /// Write data is latched.
    pub w_have: bool,
    /// Write response pending (B `valid`); held until the host's `b_ready`.
    pub b_valid: bool,
    /// Read data pending (R `valid`); held until the host's `r_ready`.
    pub r_valid: bool,
    /// Read data (32 bits), held stable while `r_valid` waits.
    pub r_data: b32,
}

/// The kernel.
///
/// `TABLE_MASK` selects which AES rounds of each Haraka chain-step lane use
/// T-table ROMs instead of LUT logic (bit `r` = round `r`; see
/// [`crate::chain_step::TableChainStepLane`]). It is passed down to every
/// slice and reported in the `SHAPE` register.
#[derive(Clone, Debug, Synchronous, SynchronousDQ)]
pub struct Engine<const TABLE_MASK: usize> {
    /// Register file, run state, tick counters ([`EngineControl`]).
    control: DFF<EngineControl>,
    /// The single HBM message reader and dealer.
    reader: MessageReader,
    /// The shared Merkle-top and Ed25519 certificate unit.
    cert: CertUnit,
    /// `SLICES` identical slices: RHDL lowers one and instantiates it.
    slices: ReplicatedSynchronous<Slice<TABLE_MASK>, SLICES>,
    /// One HBM frame writer per slice.
    writers: [HbmWriter; SLICES],
    // The links below come last, so the older children keep their netlist
    // names (`c0`..`c4`); the floorplan finds these as `c5`..`c9`.
    /// Frame beats, signer → writer (`c5/c<s>`).
    beats: [RelayPipe<FrameBeat>; SLICES],
    /// Messages, reader → slice (`c6/c<s>`): the signer takes at most one
    /// per message-hash round trip, so a [`ToggleLink`] keeps up.
    messages: [ToggleLink<Message>; SLICES],
    /// Certificates, cert unit → slice (`c7/c<s>`): rare and 1,345 bits
    /// wide, so a [`ToggleLink`] (no per-bit logic) rather than relays.
    certificates: [ToggleLink<Certificate>; SLICES],
    /// Configuration, `run` and root credit, engine → slice (`c8/c<s>`).
    to_slice: [Delay<SliceControl>; SLICES],
    /// Root pulses, status and fault, slice → engine (`c9/c<s>`).
    from_slice: [Delay<SliceReport>; SLICES],
}

/// What the engine sends a slice that tolerates latency: run configuration
/// (static during a run), `run`, and the root credit (a late credit only
/// makes the slice wait longer).
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct SliceControl {
    /// Key-factory configuration, including the slice's number.
    pub config: FactoryConfig,
    /// Allow key starts.
    pub run: bool,
    /// Batches whose roots the cert unit has taken.
    pub roots_popped: b32,
}

/// What a slice reports that tolerates latency: the one-clock subtree-root
/// pulse (the cert unit queues roots, and credit covers the delay), the
/// status counters the host reads, and the fault bit.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct SliceReport {
    /// Subtree root pulse for the cert unit.
    pub root: SubtreeRoot,
    /// Real keys started (`KEYS[s]`).
    pub keys_started: b32,
    /// Internal fault.
    pub fault: bool,
}

/// The power-on kernel: IDLE, every argument zero, every child reset.
impl<const TABLE_MASK: usize> Default for Engine<TABLE_MASK> {
    fn default() -> Self {
        Self {
            // Reset value of the control register: IDLE, no arguments, all
            // counters and handshake latches clear.
            control: DFF::new(EngineControl {
                regs: Registers {
                    prefix_cv: [b32(0); 8],
                    epoch: b64(0),
                    first_batch: b64(0),
                    batches: b32(0),
                    count: b32(0),
                    msg_base: b64(0),
                    out_base: [b64(0); SLICES],
                    seed: [b32(0); 8],
                },
                running: false,
                done: false,
                done_unread: false,
                gie: false,
                ier: false,
                isr: false,
                ticks: b64(0),
                start_tick: b64(0),
                done_tick: b64(0),
                ticks_hi: b32(0),
                aw_addr: b12(0),
                aw_have: false,
                w_data: b32(0),
                w_have: false,
                b_valid: false,
                r_valid: false,
                r_data: b32(0),
            }),
            reader: MessageReader::default(),
            cert: CertUnit::default(),
            // One slice template, replicated SLICES times by RHDL.
            slices: ReplicatedSynchronous::new(Slice::default()),
            writers: core::array::from_fn(|_| HbmWriter::default()),
            beats: core::array::from_fn(|_| RelayPipe::default()),
            messages: core::array::from_fn(|_| ToggleLink::default()),
            certificates: core::array::from_fn(|_| ToggleLink::default()),
            to_slice: core::array::from_fn(|_| Delay::default()),
            from_slice: core::array::from_fn(|_| Delay::default()),
        }
    }
}

/// Binds the kernel's input and output types to [`engine_kernel`].
impl<const TABLE_MASK: usize> SynchronousIO for Engine<TABLE_MASK> {
    type I = EngineInput;
    type O = EngineOutput;
    type Kernel = engine_kernel<TABLE_MASK>;
}

/// Replace the low or high half of a 64-bit register.
///
/// AXI4-Lite moves 32 bits at a time, so a 64-bit argument (an address, the
/// epoch, the first batch id) arrives as two writes: the low word at offset
/// `X` and the high word at `X + 4`.
///
/// * Inputs: `current`, the register's present 64-bit value; `value`, the
///   32-bit word just written; `high`, true for the `X + 4` half.
/// * Output: `current` with that half replaced and the other half kept.
/// * Timing: pure combinational logic (a mask and an OR), no state.
#[kernel]
pub fn set_half_kernel(current: b64, value: b32, high: bool) -> b64 {
    // Keep the low half, place `value` in bits 32..64; or keep the high half
    // and place `value` in bits 0..32.
    if high {
        (current & b64(0xffff_ffff)) | (value.resize::<64>() << 32)
    } else {
        (current & b64(0xffff_ffff_0000_0000)) | value.resize::<64>()
    }
}

/// The kernel's wiring and register file.
///
/// Every clock this function:
///
/// 1. advances the tick counter;
/// 2. sums progress from the writers and ORs fault flags from every child,
///    and finishes the run when all `count` frames are acknowledged;
/// 3. serves at most one AXI4-Lite register write and one register read;
/// 4. wires the children together: reader → slices, slices ↔ certificate
///    unit, slices → writers (every slice link through [`crate::relay`]
///    registers), and the HBM ports to the outside.
///
/// * Inputs: `clock_reset`, the [`EngineInput`] pins, and `q`, the registered
///   outputs of every child (`q.control`, `q.reader`, `q.cert`, `q.slices[s]`,
///   `q.writers[s]`).
/// * Outputs: the [`EngineOutput`] pins and `EngineD`, the next control value
///   and every child's inputs for this clock.
/// * Timing: in RHDL, `d` carries each child's inputs and `q` its outputs.
///   The AXI4-Lite outputs come only from the registered control state, and
///   the HBM outputs are the children's outputs passed through. A register
///   write takes effect on the clock after both its address and data are
///   latched; a register read answers on the clock after the address.
#[allow(clippy::too_many_lines, clippy::needless_range_loop)]
#[kernel]
pub fn engine_kernel<const TABLE_MASK: usize>(
    clock_reset: ClockReset,
    input: EngineInput,
    q: EngineQ<TABLE_MASK>,
) -> (EngineOutput, EngineD<TABLE_MASK>) {
    // Not used by this level's logic (see `let _ = resetting` at the end);
    // every child register receives the same reset directly.
    let resetting = clock_reset.reset.any();
    // `c` becomes the next control value; `regs` is the current configuration.
    let mut c = q.control;
    let regs = q.control.regs;
    // The free-running tick counter: +1 every clock, never stopped.
    c.ticks = q.control.ticks + b64(1);

    // ------------------------------------------------ progress and faults
    // Frames acknowledged by HBM across all slices; the run's finish line.
    let mut frames_total = b32(0);
    // Any writer got a non-OKAY write response.
    let mut writer_error = false;
    // Any slice reported an internal fault.
    let mut slice_fault = false;
    for s in 0..SLICES {
        frames_total = frames_total + q.writers[s].frames_written;
        writer_error = writer_error || q.writers[s].error;
        slice_fault = slice_fault || q.from_slice[s].fault;
    }
    // The `FAULT` register: bit 0 writer, 1 reader, 2 cert, 3 slice.
    let fault_bits: b32 = (if writer_error { b32(1) } else { b32(0) })
        | (if q.reader.error { b32(2) } else { b32(0) })
        | (if q.cert.fault { b32(4) } else { b32(0) })
        | (if slice_fault { b32(8) } else { b32(0) });
    // RUNNING → DONE: every requested frame is in HBM. Latch the end time and
    // raise `ap_done` and the done interrupt status.
    if q.control.running && frames_total == regs.count {
        c.running = false;
        c.done = true;
        c.done_unread = true;
        c.isr = true;
        c.done_tick = q.control.ticks;
    }

    // ------------------------------------------------ AXI-Lite write
    // Latch AW and W independently (they may arrive on different clocks);
    // `aw_ready`/`w_ready` are low while a value is held.
    let axi = input.axi;
    if axi.aw_valid && !q.control.aw_have {
        c.aw_addr = axi.aw_addr;
        c.aw_have = true;
    }
    if axi.w_valid && !q.control.w_have {
        c.w_data = axi.w_data;
        c.w_have = true;
    }
    // The host took the write response.
    if q.control.b_valid && axi.b_ready {
        c.b_valid = false;
    }
    // Both halves are here and the previous response is gone: perform the
    // write, free the latches and raise the response.
    if q.control.aw_have && q.control.w_have && !q.control.b_valid {
        // Register offset and the 32-bit value written to it.
        let a = q.control.aw_addr;
        let v = q.control.w_data;
        c.aw_have = false;
        c.w_have = false;
        c.b_valid = true;
        // AP_CTRL bit 0 (`ap_start`): IDLE → RUNNING, only once per reset.
        if a == b12(0x000) && (v & b32(1)) == b32(1) && !q.control.running && !q.control.done {
            c.running = true;
            c.start_tick = q.control.ticks;
        }
        if a == b12(0x004) {
            c.gie = (v & b32(1)) == b32(1);
        }
        if a == b12(0x008) {
            c.ier = (v & b32(1)) == b32(1);
        }
        // ISR is "write 1 to toggle", the `ap_ctrl_hs` convention.
        if a == b12(0x00c) && (v & b32(1)) == b32(1) {
            c.isr = !q.control.isr;
        }
        // Arguments: accepted only in IDLE, so a run's namespace, secrets and
        // buffers cannot change once it has started.
        if !q.control.running && !q.control.done {
            if a == b12(0x010) || a == b12(0x014) {
                c.regs.msg_base = set_half_kernel(regs.msg_base, v, a == b12(0x014));
            }
            for s in 0..SLICES {
                if a == b12((0x018 + 8 * s) as u128) || a == b12((0x01c + 8 * s) as u128) {
                    c.regs.out_base[s] =
                        set_half_kernel(regs.out_base[s], v, a == b12((0x01c + 8 * s) as u128));
                }
            }
            if a == b12(0x058) {
                c.regs.count = v;
            }
            if a == b12(0x05c) {
                c.regs.batches = v;
            }
            if a == b12(0x060) || a == b12(0x064) {
                c.regs.epoch = set_half_kernel(regs.epoch, v, a == b12(0x064));
            }
            if a == b12(0x068) || a == b12(0x06c) {
                c.regs.first_batch = set_half_kernel(regs.first_batch, v, a == b12(0x06c));
            }
            // The two secrets, one 32-bit word per offset. There is no read
            // case for these offsets below: they are write-only.
            for w in 0..8 {
                if a == b12((0x070 + 4 * w) as u128) {
                    c.regs.prefix_cv[w] = v;
                }
                if a == b12((0x090 + 4 * w) as u128) {
                    c.regs.seed[w] = v;
                }
            }
        }
    }

    // ------------------------------------------------ AXI-Lite read
    // The host took the read data.
    if q.control.r_valid && axi.r_ready {
        c.r_valid = false;
    }
    // Accept a new read address only when no answer is waiting, so `r_data`
    // stays stable until the host takes it.
    let ar_ready = !q.control.r_valid;
    if axi.ar_valid && ar_ready {
        // Offset being read, and the word to return (0 for unmapped offsets
        // and for the write-only arguments and secrets).
        let a = axi.ar_addr;
        let mut value = b32(0);
        // AP_CTRL: bit 0 ap_start (reads back as "running"), 1 ap_done,
        // 2 ap_idle, 3 ap_ready. Reading it clears ap_done/ap_ready.
        if a == b12(0x000) {
            let idle = !q.control.running;
            value = (if q.control.running { b32(1) } else { b32(0) })
                | (if q.control.done_unread {
                    b32(2)
                } else {
                    b32(0)
                })
                | (if idle { b32(4) } else { b32(0) })
                | (if q.control.done_unread {
                    b32(8)
                } else {
                    b32(0)
                });
            c.done_unread = false;
        }
        if a == b12(0x004) {
            value = if q.control.gie { b32(1) } else { b32(0) };
        }
        if a == b12(0x008) {
            value = if q.control.ier { b32(1) } else { b32(0) };
        }
        if a == b12(0x00c) {
            value = if q.control.isr { b32(1) } else { b32(0) };
        }
        if a == b12(0x200) {
            value = MAGIC;
        }
        if a == b12(0x204) {
            value = ABI_VERSION;
        }
        // SHAPE: what this image was compiled with, so the host can size its
        // buffers and argument list (slices in bits 0..8, TABLE_MASK above).
        if a == b12(0x208) {
            value = b32((SLICES | (TABLE_MASK << 8)) as u128);
        }
        if a == b12(0x20c) {
            value = fault_bits;
        }
        // TICKS lo: also snapshot the high half, so the following read of
        // 0x214 returns the matching upper word.
        if a == b12(0x210) {
            value = q.control.ticks.resize();
            c.ticks_hi = (q.control.ticks >> 32).resize();
        }
        if a == b12(0x214) {
            value = q.control.ticks_hi;
        }
        if a == b12(0x218) {
            value = q.control.start_tick.resize();
        }
        if a == b12(0x21c) {
            value = (q.control.start_tick >> 32).resize();
        }
        if a == b12(0x220) {
            value = q.control.done_tick.resize();
        }
        if a == b12(0x224) {
            value = (q.control.done_tick >> 32).resize();
        }
        if a == b12(0x228) {
            value = q.cert.certified;
        }
        if a == b12(0x22c) {
            value = q.reader.dealt;
        }
        if a == b12(0x230) {
            value = q.cert.busy_cycles.resize();
        }
        if a == b12(0x234) {
            value = (q.cert.busy_cycles >> 32).resize();
        }
        // Per-slice progress: frames acknowledged and keys started.
        for s in 0..SLICES {
            if a == b12((0x240 + 4 * s) as u128) {
                value = q.writers[s].frames_written;
            }
            if a == b12((0x260 + 4 * s) as u128) {
                value = q.from_slice[s].keys_started;
            }
        }
        // The Ed25519 public key the certificate unit derived from the seed:
        // four 64-bit words, returned as eight 32-bit halves (low half first).
        for w in 0..4 {
            if a == b12((0x280 + 8 * w) as u128) {
                value = q.cert.public_key.words[w].resize();
            }
            if a == b12((0x284 + 8 * w) as u128) {
                value = (q.cert.public_key.words[w] >> 32).resize();
            }
        }
        // Register the answer; it is presented next clock and held until taken.
        c.r_valid = true;
        c.r_data = value;
    }

    // ------------------------------------------------ datapath wiring
    // The one "go" signal every child sees.
    let running = q.control.running;
    // Ed25519 seed as the cert unit wants it: four 64-bit words built from
    // the eight 32-bit register words (even index = low half).
    let mut seed_words = [b64(0); 4];
    for w in 0..4 {
        seed_words[w] =
            regs.seed[2 * w].resize::<64>() | (regs.seed[2 * w + 1].resize::<64>() << 32);
    }
    // Every slice link goes through registers (`crate::relay`): the slices
    // sit up to two die crossings away from the shared blocks.
    //
    // Gather per-slice values for the shared children: subtree roots (out
    // of each slice's delay line) go to the cert unit, and the cert unit and
    // the reader see "taken" when the first station of the slice's relay
    // pipe takes what they offer. Arrays are first filled from slice 0
    // because RHDL needs a typed initial value.
    let mut roots = [q.from_slice[0].root; SLICES];
    let mut cert_taken = [false; SLICES];
    let mut message_taken = [false; SLICES];
    for s in 0..SLICES {
        roots[s] = q.from_slice[s].root;
        cert_taken[s] = q.cert.certificates[s].valid && q.certificates[s].ready;
        message_taken[s] = q.reader.messages[s].valid && q.messages[s].ready;
    }
    // Delay-line inputs toward each slice: the shared run configuration
    // plus its own slice number (which decides which keys of each batch the
    // slice makes, so no two slices make the same key), `run`, and the root
    // credit. Built as SLICES copies of slice 0's, then patched per slice.
    let mut to_slice = [SliceControl {
        config: FactoryConfig {
            prefix_cv: regs.prefix_cv,
            epoch: regs.epoch,
            first_batch: regs.first_batch,
            batches: regs.batches,
            slice: b3(0),
        },
        run: running,
        roots_popped: q.cert.roots_popped,
    }; SLICES];
    // Each slice's inputs, all from link registers: its delayed control,
    // its message and certificate out of their relay pipes, and its beat
    // pipe's `ready`. A relay pipe's `valid` is copied into the item's own
    // `valid` field.
    let mut slices = [SliceInput {
        config: q.to_slice[0].config,
        run: q.to_slice[0].run,
        message: q.messages[0].data,
        certificate: q.certificates[0].data,
        beat_ready: q.beats[0].ready,
        roots_popped: q.to_slice[0].roots_popped,
    }; SLICES];
    // Each writer: its frame base, its slice's beat out of the beat pipe,
    // and its own HBM port's responses.
    let mut writers = [WriterInput {
        base: regs.out_base[0],
        beat: q.beats[0].data,
        memory: input.frames[0],
    }; SLICES];
    // Relay-pipe inputs: what the sender offers, and whether the receiver
    // took the pipe's output this clock.
    let mut beats = [PipeIn::<FrameBeat> {
        data: q.slices[0].beat,
        valid: q.slices[0].beat.valid,
        taken: q.writers[0].beat_ready,
    }; SLICES];
    let mut messages = [PipeIn::<Message> {
        data: q.reader.messages[0],
        valid: q.reader.messages[0].valid,
        taken: q.slices[0].message_taken,
    }; SLICES];
    let mut certificates = [PipeIn::<Certificate> {
        data: q.cert.certificates[0],
        valid: q.cert.certificates[0].valid,
        taken: q.slices[0].certificate_taken,
    }; SLICES];
    // Delay-line inputs from each slice.
    let mut from_slice = [SliceReport {
        root: q.slices[0].root,
        keys_started: q.slices[0].keys_started,
        fault: q.slices[0].fault,
    }; SLICES];
    for s in 0..SLICES {
        to_slice[s].config.slice = b3(s as u128);
        slices[s] = SliceInput {
            config: q.to_slice[s].config,
            run: q.to_slice[s].run,
            message: q.messages[s].data,
            certificate: q.certificates[s].data,
            beat_ready: q.beats[s].ready,
            roots_popped: q.to_slice[s].roots_popped,
        };
        slices[s].message.valid = q.messages[s].valid;
        slices[s].certificate.valid = q.certificates[s].valid;
        writers[s] = WriterInput {
            base: regs.out_base[s],
            beat: q.beats[s].data,
            memory: input.frames[s],
        };
        writers[s].beat.valid = q.beats[s].valid;
        beats[s] = PipeIn::<FrameBeat> {
            data: q.slices[s].beat,
            valid: q.slices[s].beat.valid,
            taken: q.writers[s].beat_ready,
        };
        messages[s] = PipeIn::<Message> {
            data: q.reader.messages[s],
            valid: q.reader.messages[s].valid,
            taken: q.slices[s].message_taken,
        };
        certificates[s] = PipeIn::<Certificate> {
            data: q.cert.certificates[s],
            valid: q.cert.certificates[s].valid,
            taken: q.slices[s].certificate_taken,
        };
        from_slice[s] = SliceReport {
            root: q.slices[s].root,
            keys_started: q.slices[s].keys_started,
            fault: q.slices[s].fault,
        };
    }
    // The writers' AXI4 write requests become the kernel's HBM output pins.
    let mut frames = [q.writers[0].memory; SLICES];
    for s in 0..SLICES {
        frames[s] = q.writers[s].memory;
    }
    // Next state: control register plus the inputs of every child.
    let d = EngineD::<TABLE_MASK> {
        control: c,
        reader: ReaderInput {
            base: regs.msg_base,
            count: regs.count,
            run: running,
            memory: input.messages,
            taken: message_taken,
        },
        cert: CertInput {
            config: CertConfig {
                epoch: regs.epoch,
                first_batch: regs.first_batch,
                key_seed: ScalarWords { words: seed_words },
                run: running,
            },
            roots,
            taken: cert_taken,
        },
        slices,
        writers,
        beats,
        messages,
        certificates,
        to_slice,
        from_slice,
    };
    let _ = resetting;
    (
        EngineOutput {
            // All AXI4-Lite outputs come straight from registers. Responses
            // are always OKAY; unmapped reads return 0 and unmapped writes are
            // ignored. The interrupt is a level: GIE and IER and ISR.
            axi: AxiLiteOut {
                aw_ready: !q.control.aw_have,
                w_ready: !q.control.w_have,
                b_valid: q.control.b_valid,
                b_resp: b2(0),
                ar_ready,
                r_valid: q.control.r_valid,
                r_data: q.control.r_data,
                r_resp: b2(0),
                interrupt: q.control.gie && q.control.ier && q.control.isr,
            },
            messages: q.reader.memory,
            frames,
        },
        d,
    )
}