rhdl_dsig_scalable_core/

slice_factory.rs

//! A slice's key factory: one complete one-time key every 68 cycles.
//!
//! This is the hardware form of [`slice_model::model_key`](crate::slice_model::model_key):
//! it issues exactly the requests listed there, in the order fixed by the
//! [`slice_schedule`](crate::slice_schedule), and writes every chain node to
//! the slice's cache.
//!
//! # The story so far
//!
//! DSig signs a message with a *one-time key*: a W-OTS+ key with Winternitz
//! parameter `w = 4`. Such a key is 68 hash chains (64 message digits plus 4
//! checksum digits). Each chain starts at an 18-byte secret and is hashed
//! three times, so it has four positions, 0 to 3. Position 3 is the public
//! chain end. When we sign, each message digit tells us which position of its
//! chain to reveal. Making keys ahead of time, and storing all four positions,
//! turns signing into "look up 68 nodes and copy them out".
//!
//! A one-time key must never sign twice. So every key gets a fresh number
//! (`batch`, `index`), and its secrets are derived from that number. 128 keys
//! form a *batch*; the hash of each key's chain ends is a leaf of a Merkle
//! tree, and one Ed25519 signature certifies the batch root. A finished DSig
//! signature is 1,642 bytes.
//!
//! This module is the slice's key maker. Its job, every 68 clock cycles at the
//! 205 MHz design clock, is:
//!
//! 1. derive the key's secret seed from `(epoch, batch, index)` (BLAKE3);
//! 2. derive two nonces from the seed: the public chain salt `pk_nonce` and the
//!    private message nonce;
//! 3. expand the seed into 68 × 18 secret bytes (a BLAKE3 *XOF*, an
//!    "extendable output", read as 20 blocks of 64 bytes);
//! 4. push the 68 chain secrets through the chain engine, one per cycle, and
//!    write all four positions of each chain into the cache;
//! 5. hash `pk_nonce` and the 68 chain ends into the key's `pk_hash` (the
//!    *endpoint commitment*, 20 BLAKE3 blocks plus one parent);
//! 6. report the finished key ([`KeyDone`]) so the subtree unit can use
//!    `pk_hash` as a Merkle leaf.
//!
//! # The reasoning behind the shape
//!
//! Step 4 sets the pace: the chain engine ([`crate::slice_chain`]) accepts one
//! chain per cycle, and a key has 68 chains, so a new key must start every 68
//! cycles. Steps 1, 2, 3 and 5 need 44 BLAKE3 compressions per key
//! (1 seed + 2 nonces + 20 XOF + 20 commitment blocks + 1 parent), and two
//! more slots per period are lent to outsiders (see below): 46 in all. A
//! dedicated hash core per
//! step would cost far too many LUTs, so they all share **one** pipelined
//! BLAKE3 lane that accepts one compression per cycle. 46 of every 68 cycles
//! are therefore busy; the rest are idle.
//!
//! Sharing creates two problems:
//!
//! * **Who uses the lane in this cycle?** Several keys are in flight at once
//!   (a key's work spans several 68-cycle periods). Instead of an arbiter, we
//!   use a fixed *modulo schedule* ([`crate::slice_schedule`]): every op of a
//!   key has a fixed offset from that key's start, and no two offsets are
//!   equal modulo 68. So the cycle number modulo 68 (the *residue*) alone says
//!   which op runs. The table lookup is all the control logic there is.
//! * **Where does each in-flight key keep its half-done state?** Each key in
//!   flight owns a *context*: one entry in a small ring of RAMs indexed by the
//!   key's number modulo [`CONTEXTS`]. The seed, nonces and BLAKE3 chaining
//!   values of a key live in its context until the key is done.
//!
//! An op's *lag* is how many 68-cycle periods after its key's start it runs
//! (negative for ops that run before chain 0). From the current period and an
//! op's lag, the factory knows which key, and thus which context, the op
//! belongs to: `context = (period − lag) mod CONTEXTS`.
//!
//! # Picture
//!
//! ```text
//!               residue r = cycle mod 68 ──▶ timetable ──▶ (op, key context)
//!                                                │
//!   prefix_cv, epoch, batch ──┐                  ▼
//!   context RAMs (seed, CVs) ─┼──▶ request mux ──▶ BLAKE3 lane (28 stages, II=1)
//!   commitment words ─────────┘                          │ result + tag
//!         ▲                                              ▼
//!         │                       write back into the key's context RAMs
//!   packer ◀── chain ends ◀── chain engine ◀── XOF window ◀── XOF RAM
//!                                │
//!                                └──▶ cache writes (all four positions)
//! ```
//!
//! Read the picture from the top. A counter gives the residue; the timetable
//! ([`SLOTS`]) turns it into an op and a lag; the lag picks the key context.
//! The request mux builds that op's 64-byte BLAKE3 block from the run
//! configuration and the context RAMs, and the lane hashes it. 28 cycles
//! later the result comes out with its *tag* (a 32-bit label that says which
//! op and which context it belongs to) and is written back into that
//! context. XOF results go to a separate XOF RAM. From there, a two-block
//! *window* feeds 18-byte secrets into the chain engine, one per cycle. Chain
//! nodes go to the cache, and chain ends go through the packer into the
//! commitment RAM. Later commitment ops read them back as BLAKE3 blocks.
//!
//! # Where the data lives
//!
//! | Storage | Width × depth | Written by | Read by |
//! |---|---|---|---|
//! | `contexts.seed` | 8 words × [`CONTEXTS`] | `Seed` result | nonce and XOF requests |
//! | `contexts.pkn_feed` | 16 bytes × [`CONTEXTS`] | `PkNonce` result | chain-salt latch (residue 66/67) |
//! | `contexts.pkn_meta` | 16 bytes × [`CONTEXTS`] | `PkNonce` result | [`KeyDone`] metadata |
//! | `contexts.nonce` | 16 bytes × [`CONTEXTS`] | `Nonce` result | [`KeyDone`] metadata |
//! | `contexts.cv0` / `cv1` | 8 words × [`CONTEXTS`] | chunk-0 / chunk-1 results | next block of that chunk, parent |
//! | `xof_ram` | 16 words × 64 | `Xof` results | chain window |
//! | `commit_ram` | 64 bytes × (32 × [`CONTEXTS`]) | packer | chunk requests |
//!
//! `pk_nonce` is stored twice because two different readers need it at
//! different addresses in the same cycle, and each RAM has one read port.
//!
//! # Cycle contract
//!
//! Key `k` is decided (real or bubble) [`KEY_LEAD`] periods before its chain
//! 0 enters the chain engine at residue 0 of period `k`. Each op of key `k`
//! issues at cycle `68k + offset`, the offsets coming from
//! [`SCHEDULE`]. Every op is two registered steps:
//!
//! 1. **decode** (cycle `y − 1`): look up residue `y mod 68`, compute the key's
//!    context index, present context-RAM read addresses, register the op;
//! 2. **issue** (cycle `y`): assemble the request from the RAM outputs and
//!    feed the lane.
//!
//! Results leave the lane 28 cycles after issue, are registered once, and are
//! written into the key's context RAM. They are readable by an op issuing
//! [`use_delay`](crate::slice_schedule::Schedule::use_delay) cycles after the
//! producer, which the schedule guarantees.
//!
//! ```text
//!   cycle:   y−1        y          y+1 … y+27     y+28          y+29
//!            decode     issue      lane stages    lane output   result register
//!            residue    request ─▶ BLAKE3 lane ─▶ (q.lane)   ─▶ (q.data.result)
//!            lookup,    enters                                  ─▶ context-RAM write
//!            RAM addr   lane
//!
//!   An op that consumes this result issues use_delay = 28 + 4 = 32 cycles
//!   after y, once the written value can be read back.
//! ```
//!
//! # How a key moves through periods
//!
//! One *period* is 68 cycles. Key `k`'s chain 0 enters the chain engine at
//! residue 0 of period `k`. Its preparation ops (seed, nonces, XOF) run in
//! earlier periods, its commitment ops in later ones. Meanwhile other keys
//! do their own ops in the same periods, on other residues:
//!
//! ```text
//!   period:      k−KEY_LEAD   …      k−1           k              k+1    …
//!   key k:       decided      seed, nonces, XOF    chains 0..67   commitment, parent → KeyDone
//!   key k+1:                  decided  …           seed, XOF …    chains 0..67  …
//! ```
//!
//! The ring of [`CONTEXTS`] contexts must be larger than the number of
//! periods a key spans, so a context is never reused while its key still has
//! ops pending; compile-time assertions below check this.
//!
//! The Merkle and message slots carry requests from outside (the subtree unit
//! and the signer). A request offered on [`SliceFactoryInput::merkle`] or
//! [`SliceFactoryInput::message`] is taken at that slot's residue, signalled
//! by `merkle_taken` or `message_taken`, and its 16 result words return on
//! [`SliceFactoryOutput::slot_result`] with the caller's tag.
//!
//! # Safety rules carried by this block
//!
//! * Each real key gets a fresh `(batch, index)` from a monotonic counter; a
//!   key index is never issued twice within a run, and reset ends the run.
//! * Bubble keys issue nothing: their slots are idle.
//! * The secret prefix CV is an input, never stored beyond the run's config.
//!
//! # Neighbours
//!
//! * [`crate::slice_schedule`] computes the timetable this block follows.
//! * [`crate::slice_chain`] is the chain engine fed one chain per cycle.
//! * [`crate::chain_lane`] picks the chain hash (Haraka-512 by default) and
//!   with it the chain engine's latency, which shapes [`END_DELAY`],
//!   [`PACK_HISTORY`] and [`CTX_BITS`].
//! * [`crate::packer`] turns 18-byte chain ends into 64-byte commitment words.
//! * [`crate::blake3_core`] supplies the shared BLAKE3 lane.
//! * [`crate::slice_model`] is the software model the tests compare against.
//! * [`crate::slice`] wires this factory to the cache, subtree unit and signer.

use crate::blake3_core::{
    CHUNK_END, CHUNK_START, CompressionInput, DERIVE_KEY_MATERIAL, IV, InlineCompressionPipeline,
    PARENT, PIPELINE_STAGES, ROOT, SINGLE_BLOCK_ROOT,
};
use crate::packer::{COMMITMENT_PREFIX, ElementPacker, PackerInput};
use crate::slice_chain::{SLICE_CHAIN_LATENCY, SliceChainEngine, SliceChainInput};
use crate::slice_model::{NONCE_PREFIX, PK_NONCE_PREFIX};
use crate::slice_schedule::{
    CHUNK0_BLOCKS, KEY_PERIOD, OPS, Op, Schedule, XOF_BLOCKS, first_chain_of_xof, op_at, solve,
};
use crate::{CHAINS, ELEMENT_BYTES, KEY_ONE, KEY_ZERO, KEYS_PER_SLICE, KeyIndex, SUBTREE_LEVELS};
use rhdl::prelude::*;
use rhdl_fpga::core::{
    dff::DFF,
    ram::synchronous::{In as RamIn, SyncBRAM, Write as RamWrite},
};

/// Cycles from a chain's issue until its commitment word is readable.
///
/// The chain engine takes [`SLICE_CHAIN_LATENCY`] cycles (three chain-hash
/// lanes in series); two more cycles cover the registers on the way from the
/// engine output through the packer into the commitment RAM. Haraka: 3 × 20 + 2 = 62. BLAKE3: 3 × 28 + 2 = 86.
/// SHA-512: 3 × 82 + 2 = 248. The schedule solver needs this number to know
/// when a commitment block's chain ends are all present.
pub const END_DELAY: i32 = SLICE_CHAIN_LATENCY as i32 + 2;
/// Earlier keys' `pk_nonce`s kept for the commitment prefix.
///
/// A key's chain ends leave the chain engine [`END_DELAY`] cycles after its
/// chains start. With Haraka (62 cycles) that is still the key's own period;
/// with SHA-512 chains (248 cycles) three periods later, when newer keys are
/// already being fed. The packer picks the nonce by the key's age.
///
/// The formula counts how many whole periods the chain engine spans, plus
/// one. Haraka: 62 / 68 + 1 = 1. BLAKE3: 86 / 68 + 1 = 2. SHA-512:
/// 248 / 68 + 1 = 4. Each entry is one 16-byte register, so this is cheap.
pub const PACK_HISTORY: usize = END_DELAY as usize / KEY_PERIOD + 1;
/// The timetable this factory implements.
///
/// Computed at compile time by [`solve`] from the BLAKE3 lane latency
/// ([`PIPELINE_STAGES`] = 28) and [`END_DELAY`]. The hardware never runs the
/// solver; it only reads the resulting tables ([`SLOTS`], [`WINDOW_LOADS`]).
pub const SCHEDULE: Schedule = solve(PIPELINE_STAGES as i32, END_DELAY);
/// Bits of a key-context index (and of an op lag). The SHA-512 chain lanes
/// (82 cycles each) stretch one key over more 68-cycle periods, so that profile
/// needs 32 contexts; Haraka and BLAKE3 need 16.
///
/// A *context* is the slot of per-key state (seed, nonces, chaining values,
/// commitment words) that a key owns while it is in flight. More contexts
/// cost deeper (but still small) RAMs, so we use the fewest that fit.
pub const CTX_BITS: usize = if cfg!(dsig_chain = "sha512") { 5 } else { 4 };
/// Key contexts held at once; must exceed the span of op lags.
///
/// Key number `k` uses context `k mod CONTEXTS`. If a key's ops spanned
/// `CONTEXTS` periods or more, a newer key would overwrite its state.
pub const CONTEXTS: usize = 1 << CTX_BITS;
/// The last key number of a slice's batch share.
const LAST_KEY: KeyIndex = bits((KEYS_PER_SLICE - 1) as u128);
/// A key-context index (wraps modulo [`CONTEXTS`]).
///
/// [`CTX_BITS`] wide. Because the arithmetic wraps, "period minus lag" gives
/// the right context even across the counter's wrap-around.
pub type Ctx = Bits<CTX_BITS>;
/// Commitment-RAM address: context and word.
///
/// [`CTX_BITS`] + 5 bits: `ctx << 5 | word`. The low 5 bits select one of 32
/// word slots per context (20 are used: 16 for chunk 0, 4 for chunk 1).
pub type CommitAddress = Bits<{ CTX_BITS + 5 }>;
/// `1` as a [`Ctx`].
///
/// RHDL kernels cannot convert integers on the fly, so commonly used values
/// are prepared as typed constants.
pub const CTX_ONE: Ctx = bits(1);
/// [`KEY_LEAD`] as a [`Ctx`].
pub const CTX_LEAD: Ctx = bits(KEY_LEAD as u128);
/// [`LAG_BIAS`] as a [`Ctx`].
pub const CTX_BIAS: Ctx = bits(LAG_BIAS as u128);
/// Every context index as a [`Ctx`] (kernels compare against these).
///
/// `CTX_INDEX[i] == i`. A kernel that needs "the ring entry at a run-time
/// index" loops over all `i` and compares, which synthesizes to a plain
/// multiplexer ([`ring_entry_kernel`]).
pub const CTX_INDEX: [Ctx; CONTEXTS] = ctx_index();

/// Builds [`CTX_INDEX`] at compile time: entry `k` holds the value `k`.
const fn ctx_index() -> [Ctx; CONTEXTS] {
    // Start from zeros, then write each index.
    let mut out = [bits(0); CONTEXTS];
    let mut k = 0;
    while k < CONTEXTS {
        out[k] = bits(k as u128);
        k += 1;
    }
    out
}
/// Bits of a cache slot number.
///
/// 9 bits name 512 slots ([`CACHE_SLOTS`]); a slot holds one whole key.
pub const SLOT_BITS: usize = 9;
/// One-time keys the slice cache can hold (ring size): 32 batches of 16.
///
/// Why 32 batches: a key cannot be spent until its batch certificate returns.
/// The Ed25519 core reaches one signature per ~758 cycles only with ~32
/// requests in flight, each taking ~25,000 cycles, so roughly 25 batches per
/// slice wait in the cache at the full rate. See [`sustained`](crate::sustained), section 4a.
pub const CACHE_SLOTS: usize = 1 << SLOT_BITS;
/// Rows of each cache bank: keys packed densely, `CHAINS` rows per key.
///
/// 512 × 68 = 34,816 rows. One row holds one chain position (18 bytes).
pub const CACHE_ROWS: usize = CACHE_SLOTS * CHAINS;
/// Bits of a cache-bank row address.
///
/// 16 bits reach 65,536 rows, enough for [`CACHE_ROWS`] = 34,816.
pub const CACHE_ADDRESS_BITS: usize = 16;

/// Dense cache row of `(slot, chain)`: `slot × 68 + chain`.
///
/// `68 = 64 + 4`, so this is two shifts and two adds. A power-of-two layout
/// (`slot << 7 | chain`) would waste 60 of every 128 rows and need 1,024
/// UltraRAMs for 512 keys per slice, more than the U280's 960.
///
/// Inputs: `slot` (9 bits, 0..512) and `chain` (7 bits, 0..68). Output: the
/// 16-bit row address. Purely combinational: the result is ready in the same
/// cycle.
#[kernel]
pub fn cache_address_kernel(slot: b9, chain: b7) -> b16 {
    // Widen the slot first so the shifts do not lose high bits.
    let s: b16 = slot.resize();
    // slot × 64 + slot × 4 + chain = slot × 68 + chain.
    (s << 6) + (s << 2) + chain.resize::<16>()
}

/// Op codes in the residue table.
///
/// Each residue of the 68-cycle timetable runs one kind of op. These small
/// integers (4 bits in hardware, see [`Slot::kind`]) name the kinds. They are
/// `usize` because RHDL kernels accept `usize` constants and turn them into
/// bit patterns where they are compared.
pub mod code {
    /// No op this cycle.
    pub const IDLE: usize = 0;
    /// Seed derivation.
    pub const SEED: usize = 1;
    /// Public chain salt.
    pub const PK_NONCE: usize = 2;
    /// Private message nonce.
    pub const NONCE: usize = 3;
    /// XOF block (argument = block).
    pub const XOF: usize = 4;
    /// Commitment chunk-0 block (argument = block).
    pub const CHUNK0: usize = 5;
    /// Commitment chunk-1 block (argument = block within chunk).
    pub const CHUNK1: usize = 6;
    /// Commitment parent, yielding `pk_hash`.
    pub const PARENT: usize = 7;
    /// External Merkle-subtree request.
    pub const MERKLE: usize = 8;
    /// External message-digit request.
    pub const MESSAGE: usize = 9;
}

/// The *lag* of an op: how many whole 68-cycle periods after its key's
/// chain-0 cycle `T` it issues.
///
/// `offset` is the op's issue cycle relative to `T` (from [`SCHEDULE`]). A
/// negative offset gives a negative lag (the op runs in an earlier period).
/// `div_euclid` rounds toward minus infinity, so offset −1 has lag −1, not 0.
const fn lag_of(offset: i32) -> i32 {
    offset.div_euclid(KEY_PERIOD as i32)
}

/// Smallest and largest op lag (period of an op relative to its key's period).
///
/// The difference tells how many periods one key spans, which bounds how many
/// contexts must exist ([`CONTEXTS`]) and when a key must be decided
/// ([`KEY_LEAD`]).
const fn lag_range() -> (i32, i32) {
    // Start with impossible extremes; every op tightens them.
    let mut low = i32::MAX;
    let mut high = i32::MIN;
    let mut index = 0;
    while index < OPS {
        // Lag of op number `index` in the timetable.
        let lag = lag_of(SCHEDULE.offsets[index]);
        if lag < low {
            low = lag;
        }
        if lag > high {
            high = lag;
        }
        index += 1;
    }
    (low, high)
}

/// Bias added to lags so they fit an unsigned [`CTX_BITS`]-bit field.
///
/// Lags can be negative (preparation ops run before chain 0), but a hardware
/// field is unsigned. The timetable stores `lag + LAG_BIAS`, and the decode
/// step adds [`CTX_BIAS`] back in, so the bias cancels:
/// `period + BIAS − (lag + BIAS) = period − lag`. A compile-time assertion
/// below checks that every biased lag fits.
pub const LAG_BIAS: i32 = 4;
/// Periods between deciding key `k` and its chain 0: one before its first op.
///
/// A key must be decided (real key or bubble) before its earliest op decodes,
/// because decode reads the key's [`KeyEntry::real`] flag. The earliest op has
/// lag `lag_range().0` (negative); deciding one period earlier than that is
/// `1 − lag_range().0` periods before chain 0.
pub const KEY_LEAD: i32 = 1 - lag_range().0;

/// [`LAG_BIAS`] as a `usize`, usable inside RHDL kernels.
pub const LAG_BIAS_U: usize = LAG_BIAS as usize;
/// [`KEY_LEAD`] as a `usize`, usable inside RHDL kernels.
pub const KEY_LEAD_U: usize = KEY_LEAD as usize;

// RHDL kernels accept only RHDL-typed or `usize` constants, so every BLAKE3
// constant the request mux needs is converted here, once.
/// Converts the BLAKE3 initial vector [`IV`] (`u32` words) into RHDL `b32`
/// words, at compile time.
const fn iv_bits() -> [b32; 8] {
    let mut out = [b32(0); 8];
    let mut w = 0;
    while w < 8 {
        out[w] = b32(IV[w] as u128);
        w += 1;
    }
    out
}
/// BLAKE3 IV words.
///
/// The chaining value a BLAKE3 hash starts from. Used for every request that
/// is the first block of a hash (nonces, XOF, first commitment blocks, parent,
/// Merkle, message). The seed request starts from the secret prefix CV instead.
pub const IV_BITS: [b32; 8] = iv_bits();
/// Flags of the seed request.
///
/// BLAKE3 "derive key" mode: the 20-byte material block is the whole (last,
/// root) block of a hash keyed by the secret prefix CV.
pub const FLAGS_SEED: b32 = b32((CHUNK_END | ROOT | DERIVE_KEY_MATERIAL) as u128);
/// Flags of a complete single-block root hash.
///
/// The default for a request: one block that is both the start and the end of
/// its chunk and the root of the hash.
pub const FLAGS_SINGLE: b32 = b32(SINGLE_BLOCK_ROOT as u128);
/// Flag marking a chunk's first block.
pub const FLAG_START: b32 = b32(CHUNK_START as u128);
/// Flag marking a chunk's last block.
pub const FLAG_END: b32 = b32(CHUNK_END as u128);
/// Flags of the commitment parent.
///
/// The commitment is 1,240 bytes, so it spans two BLAKE3 chunks; a parent
/// node marked root joins their chaining values into `pk_hash`.
pub const FLAGS_PARENT: b32 = b32((PARENT | ROOT) as u128);
/// `CHUNK_END | ROOT`: profile 2's message tail block.
pub const FLAGS_END_ROOT: b32 = b32((CHUNK_END | ROOT) as u128);
/// Profile 2 (`DSIG_MESSAGE_BYTES=64`): the message digest is two blocks.
pub const TWO_BLOCK_MESSAGE: bool = crate::MESSAGE_BYTES == 64;
/// `pk_nonce` derivation prefix word.
///
/// Placed in word 0 of the nonce request so the two nonces derived from the
/// same seed differ (domain separation).
pub const PK_NONCE_WORD: b32 = b32(PK_NONCE_PREFIX as u128);
/// Message-nonce derivation prefix word.
pub const NONCE_WORD: b32 = b32(NONCE_PREFIX as u128);

// Every biased lag fits the unsigned CTX_BITS-bit field of `Slot::lag`.
const _: () = assert!(lag_range().0 + LAG_BIAS >= 0 && lag_range().1 + LAG_BIAS < CONTEXTS as i32);
// From its decision to its last op, a key spans fewer than CONTEXTS periods,
// so no newer key can reuse its context while it still has work pending.
const _: () = assert!(lag_range().1 - lag_range().0 + KEY_LEAD < CONTEXTS as i32);

/// Maps a schedule [`Op`] to its `(code, argument)` pair for the hardware
/// table. The argument is the block number for XOF and chunk ops, else 0.
const fn code_of(op: Op) -> (usize, usize) {
    match op {
        Op::Seed => (code::SEED, 0),
        Op::PkNonce => (code::PK_NONCE, 0),
        Op::Nonce => (code::NONCE, 0),
        Op::Xof(n) => (code::XOF, n as usize),
        Op::Chunk0(n) => (code::CHUNK0, n as usize),
        Op::Chunk1(n) => (code::CHUNK1, n as usize),
        Op::Parent => (code::PARENT, 0),
        Op::Merkle => (code::MERKLE, 0),
        Op::Message => (code::MESSAGE, 0),
        // Profile 2's tail residue: the same slot code, argument 1.
        Op::MessageTail => (code::MESSAGE, 1),
    }
}

/// One timetable row in RHDL types.
///
/// The decode step reads one `Slot` per cycle, chosen by the residue. It says
/// *what* to do (`kind`, `arg`) and *for which key* (`lag`, turned into a
/// context by subtracting it from the current period).
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct Slot {
    /// [`code`] of the op at this residue.
    ///
    /// 4 bits: values 0 ([`code::IDLE`]) to 9 ([`code::MESSAGE`]).
    pub kind: b4,
    /// Op argument (block number).
    ///
    /// 5 bits: XOF block 0..20, chunk-0 block 0..16 or chunk-1 block 0..4.
    pub arg: b5,
    /// Lag of the op plus [`LAG_BIAS`].
    ///
    /// [`CTX_BITS`] wide, so it can be subtracted directly from a [`Ctx`].
    pub lag: Ctx,
}

/// Builds [`SLOTS`]: for each residue `r`, the op the schedule places there
/// (or an idle row), converted into RHDL types. Runs at compile time.
const fn slot_table() -> [Slot; KEY_PERIOD] {
    // Every residue starts idle (kind 0).
    let mut table = [Slot {
        kind: b4(0),
        arg: b5(0),
        lag: bits(0),
    }; KEY_PERIOD];
    let mut r = 0;
    while r < KEY_PERIOD {
        // `index` is the op number the solver placed at residue `r`, if any.
        if let Some(index) = SCHEDULE.residue[r] {
            let (kind, arg) = code_of(op_at(index as usize));
            // Store the lag biased, so it is never negative.
            let lag = lag_of(SCHEDULE.offsets[index as usize]) + LAG_BIAS;
            table[r] = Slot {
                kind: b4(kind as u128),
                arg: b5(arg as u128),
                lag: bits(lag as u128),
            };
        }
        r += 1;
    }
    table
}

/// The residue table as RHDL constants.
///
/// 68 rows, one per residue. In hardware this is a small ROM (a constant
/// lookup), selected by the residue counter.
pub const SLOTS: [Slot; KEY_PERIOD] = slot_table();

/// XOF window load: which block (if any) to *address* at each residue.
///
/// Block `t` is loaded into the window at chain `first_chain(t) − 1`, so its
/// RAM read is addressed one cycle earlier. Block 0 is addressed at residue 66
/// of the previous period (for the next key).
///
/// Why a window: chain `i` needs XOF bytes `18i..18i+18`, which may straddle
/// two 64-byte blocks. Keeping the current and the next block in two
/// registers (indexed by block parity) lets every chain pick its 18 bytes
/// with a fixed multiplexer ([`pick_secret_kernel`]), and the XOF RAM needs
/// only one read per block.
///
/// ```text
///   residue:  first_chain(t) − 2    first_chain(t) − 1     first_chain(t)
///             address xof_ram       RAM data → window[t%2]  chain reads window
/// ```
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct WindowLoad {
    /// A block read is addressed this residue.
    pub active: bool,
    /// The block number.
    ///
    /// 5 bits: 0..[`XOF_BLOCKS`] (20).
    pub block: b5,
    /// The block belongs to the key that starts next period.
    ///
    /// Only block 0 has this set: its read happens two cycles before chain 0,
    /// which is at the end of the previous period.
    pub next_key: bool,
}

/// Builds [`WINDOW_LOADS`] at compile time: for each XOF block, mark the
/// residue at which its RAM read is addressed.
const fn window_table() -> [WindowLoad; KEY_PERIOD] {
    // All residues start with no load.
    let mut table = [WindowLoad {
        active: false,
        block: b5(0),
        next_key: false,
    }; KEY_PERIOD];
    let mut t = 0;
    while t < XOF_BLOCKS {
        // Address two cycles before the first chain that needs block `t`:
        // one cycle for the RAM read, one for the window register.
        let address = first_chain_of_xof(t) - 2;
        // A negative cycle falls in the previous period (block 0 only).
        let next_key = address < 0;
        table[address.rem_euclid(KEY_PERIOD as i32) as usize] = WindowLoad {
            active: true,
            block: b5(t as u128),
            next_key,
        };
        t += 1;
    }
    table
}

/// Window-load table as RHDL constants.
///
/// 68 rows, indexed by the residue of the current cycle.
pub const WINDOW_LOADS: [WindowLoad; KEY_PERIOD] = window_table();

/// Where chain `i`'s 18 secret bytes sit in the two-block XOF window.
///
/// Chain `i` starts at XOF byte `18i`. That byte lies in block `18i / 64` at
/// offset `18i mod 64`. Because 18 is even, every offset is even, which halves
/// the multiplexer in [`pick_secret_kernel`].
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct ChainPick {
    /// Parity of the block holding the chain's first byte.
    ///
    /// The window stores even blocks in `window[0]` and odd ones in
    /// `window[1]`; this says which register is the "low" (earlier) block.
    pub low_parity: bool,
    /// Byte offset of the first byte within that block (even, 0..64).
    ///
    /// 6 bits.
    pub offset: b6,
}

/// Builds [`CHAIN_PICKS`] at compile time: entry `i` locates chain `i`'s
/// secret. The table has [`KEY_PERIOD`] entries because it is indexed by
/// residue, and residue `i` feeds chain `i`.
const fn pick_table() -> [ChainPick; KEY_PERIOD] {
    let mut table = [ChainPick {
        low_parity: false,
        offset: b6(0),
    }; KEY_PERIOD];
    let mut i = 0;
    while i < CHAINS {
        // First XOF byte of chain `i`.
        let start = ELEMENT_BYTES * i;
        table[i] = ChainPick {
            low_parity: (start / 64) % 2 == 1,
            offset: b6((start % 64) as u128),
        };
        i += 1;
    }
    table
}

/// Chain-pick table as RHDL constants.
///
/// Indexed by residue; only the first [`CHAINS`] (= all 68) entries are
/// meaningful.
pub const CHAIN_PICKS: [ChainPick; KEY_PERIOD] = pick_table();

/// Run configuration; hold stable while `run` is set.
///
/// Written by the host before a run. Together, `epoch`, the batch id and the
/// key index make every key's number unique, and the seed is derived from
/// that number. This is how the design guarantees fresh secrets per key.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct FactoryConfig {
    /// Secret derive-key prefix CV ([`slice_model::prefix_cv`](crate::slice_model::prefix_cv)).
    ///
    /// Eight 32-bit words: the BLAKE3 chaining value after hashing the secret
    /// key-derivation key. The seed request starts from it, so seeds cannot be
    /// predicted without the secret.
    pub prefix_cv: [b32; 8],
    /// Durably allocated epoch.
    ///
    /// 64 bits. The host persists it before use, so a restart never repeats an
    /// epoch, and therefore never repeats a key number.
    pub epoch: b64,
    /// First batch id of this run.
    ///
    /// 64 bits. Batch `b` of the run has id `first_batch + b`.
    pub first_batch: b64,
    /// Batches to make (0 = no limit).
    ///
    /// 32 bits. When [`FactoryControl::next_batch`] reaches it, no new key is
    /// admitted.
    pub batches: b32,
    /// This slice's number; key index within a batch is
    /// `KEYS_PER_SLICE·slice + j`.
    ///
    /// 3 bits: up to 8 slices.
    pub slice: b3,
}

/// A request for one of the two externally fed slots.
///
/// The subtree unit and the signer also need single-block BLAKE3 hashes. The
/// timetable reserves one residue per period for each of them, so they can
/// borrow the factory's lane instead of building their own.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct SlotRequest {
    /// Hold true until taken.
    ///
    /// The request is taken only at its slot's residue, so the caller may wait
    /// up to one period (68 cycles).
    pub valid: bool,
    /// The 64-byte block as words (single-block root hash, IV key).
    pub block: [b32; 16],
    /// Profile 2's message digest only (`false` otherwise): this is the
    /// tail block (48 bytes, `CHUNK_END | ROOT`, chained from [`Self::cv`]);
    /// when `false` a profile-2 message block is the first one
    /// (`CHUNK_START`, not a root, its CV returned).
    pub tail: bool,
    /// The chaining value of a tail block.
    pub cv: [b32; 8],
    /// Returned with the result.
    ///
    /// 16 bits chosen by the caller, so it can match results to requests.
    pub tag: b16,
}

/// Factory inputs.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct SliceFactoryInput {
    /// Configuration, stable while running.
    pub config: FactoryConfig,
    /// Allow new keys to be started.
    ///
    /// Sampled once per period, at residue 0. Clearing it turns later keys
    /// into bubbles; keys already admitted finish normally.
    pub run: bool,
    /// The cache has room for one more key.
    ///
    /// Also sampled at residue 0. Without room, the key becomes a bubble
    /// instead of overwriting a cached key that may not be spent yet.
    pub space: bool,
    /// Merkle-subtree request.
    pub merkle: SlotRequest,
    /// Message-digit request.
    pub message: SlotRequest,
}

/// Four positions of one chain for the cache.
///
/// One write per cycle while a real key's chains leave the chain engine:
/// 68 writes per key, one per chain.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct CacheWrite {
    /// A write this cycle.
    pub valid: bool,
    /// Cache slot of the key.
    ///
    /// 9 bits (0..[`CACHE_SLOTS`]).
    pub slot: b9,
    /// Chain number.
    ///
    /// 7 bits (0..68).
    pub chain: b7,
    /// Positions 0..=3.
    ///
    /// Four 18-byte nodes: the secret, the two intermediate nodes and the
    /// public chain end. All four are stored so signing needs no hashing.
    pub nodes: [[b8; ELEMENT_BYTES]; 4],
}

/// One finished key's public and private metadata.
///
/// Emitted once per real key, the cycle after its parent result leaves the
/// BLAKE3 lane. By then all 68 chains are already in the cache.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct KeyDone {
    /// A key finished this cycle.
    pub valid: bool,
    /// Cache slot holding its nodes.
    pub slot: b9,
    /// Batch number relative to `first_batch`.
    pub batch: b32,
    /// Position `j` of the key within this slice's share of the batch
    /// (`0..KEYS_PER_SLICE`).
    pub j: KeyIndex,
    /// Endpoint commitment (Merkle leaf).
    ///
    /// 32 bytes: BLAKE3 of `pk_nonce ‖ end₀ ‖ … ‖ end₆₇`.
    pub pk_hash: [b8; 32],
    /// Public chain salt.
    ///
    /// 16 bytes. A verifier needs it to recompute the chains, so it goes into
    /// the signature.
    pub pk_nonce: [b8; 16],
    /// Private message nonce.
    ///
    /// 16 bytes, kept by the signer until the key is used.
    pub nonce: [b8; 16],
}

/// Result of a Merkle or message slot request.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct SlotResult {
    /// A result this cycle.
    pub valid: bool,
    /// True for a message result, false for Merkle.
    pub message: bool,
    /// All sixteen output words.
    ///
    /// The full 512-bit compression output; callers keep what they need.
    pub words: [b32; 16],
    /// The request's tag.
    pub tag: b16,
}

/// Factory outputs.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct SliceFactoryOutput {
    /// Chain nodes for the cache.
    pub cache_write: CacheWrite,
    /// Finished key metadata.
    pub key_done: KeyDone,
    /// Merkle/message results.
    pub slot_result: SlotResult,
    /// The Merkle request was taken this cycle.
    ///
    /// The caller drops or replaces its request on the next cycle.
    pub merkle_taken: bool,
    /// The message request was taken this cycle.
    pub message_taken: bool,
    /// Real keys started so far.
    ///
    /// A 32-bit counter for status and tests; bubbles are not counted.
    pub keys_started: b32,
}

/// Per-key bookkeeping, one entry per context.
///
/// Written at admission (residue 0, [`KEY_LEAD`] periods before chain 0) and
/// read by every op of the key. Small enough to keep in registers, so any
/// stage can read any entry in the same cycle.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct KeyEntry {
    /// The key is real (not a bubble).
    ///
    /// A *bubble* is a period in which no key is made; its ops do nothing.
    pub real: bool,
    /// Cache slot.
    pub slot: b9,
    /// Batch relative to the run's first batch.
    pub batch: b32,
    /// Position within this slice's share of the batch.
    pub j: KeyIndex,
}

/// A decoded op waiting to issue next cycle.
///
/// The output of the decode step (cycle `y − 1`), registered so that the
/// issue step (cycle `y`) sees it together with the context-RAM outputs.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct Decoded {
    /// [`code`] value.
    pub kind: b4,
    /// Op argument.
    pub arg: b5,
    /// Key context.
    pub ctx: Ctx,
    /// The op should issue (real key or valid external request).
    pub live: bool,
}

/// Resettable control state.
///
/// Everything here returns to zero on reset. That kills all in-flight work:
/// the ring forgets every key, so their remaining ops become idle. Keys cut
/// off this way are never used. Reset ends the run; the host starts any new
/// run with a fresh batch range, so no key number repeats.
#[derive(Clone, Copy, Debug, PartialEq, Digital, Default)]
pub struct FactoryControl {
    /// Residue of the *current* cycle.
    ///
    /// 7 bits, counts 0..=67 and wraps.
    pub residue: b7,
    /// Low bits of the current period.
    ///
    /// Increments when the residue wraps. Only the low [`CTX_BITS`] bits are
    /// kept, which is exactly the context index of the key whose chains
    /// run in this period.
    pub period: Ctx,
    /// Key contexts.
    ///
    /// One [`KeyEntry`] per context, indexed by key number mod [`CONTEXTS`].
    pub ring: [KeyEntry; CONTEXTS],
    /// Next cache slot to hand out.
    ///
    /// Wraps after 511; the `space` input stops admission before a slot still
    /// in use would be handed out again.
    pub next_slot: b9,
    /// Next `j` within the slice's share of a batch.
    pub next_j: KeyIndex,
    /// Batch currently being filled, relative.
    pub next_batch: b32,
    /// Real keys started.
    pub started: b32,
    /// Op decoded last cycle.
    pub decoded: Decoded,
    /// A window load was addressed last cycle (load the data now).
    pub window_pending: bool,
    /// Parity of the pending window block.
    pub window_parity: bool,
    /// Latch the next key's `pk_nonce` from the RAM output now.
    pub latch_pending: bool,
    /// Result register holds a valid result.
    pub result_valid: bool,
    /// Commitment packer context for the key whose ends are streaming.
    pub pack_ctx: Ctx,
    /// Next commitment word index.
    ///
    /// 5 bits, 0..20, reset to 0 after the packer's last word of a key.
    pub pack_word: b5,
}

/// Resetless datapath state.
///
/// Wide data registers without reset: reset logic on hundreds of bits costs
/// area, and these values are only used when a matching valid flag in
/// [`FactoryControl`] says so.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct FactoryData {
    /// Two-block XOF window, indexed by block parity.
    ///
    /// `window[0]` holds the latest even block, `window[1]` the latest odd
    /// block, 64 bytes each.
    pub window: [[b8; 64]; 2],
    /// Current key's `pk_nonce` for chain salting and the commitment prefix.
    pub pk_nonce: [b8; 16],
    /// The [`PACK_HISTORY`] keys before it, newest first.
    ///
    /// Needed because chain ends of a key can emerge after newer keys have
    /// started feeding (see [`PACK_HISTORY`]).
    pub older_nonces: [[b8; 16]; PACK_HISTORY],
    /// Registered lane result.
    ///
    /// Sixteen 32-bit words: the full BLAKE3 compression output.
    pub result: [b32; 16],
    /// Tag of the registered result.
    ///
    /// Layout from [`lane_tag_kernel`]: kind in bits 0..4, argument in bits
    /// 4..9, context from bit 9, external tag in bits 16..32.
    pub result_tag: b32,
}

/// The six small per-key context RAMs, grouped into one child.
///
/// RHDL's `Synchronous` derive stores child state in a tuple, and Rust only
/// compares tuples of up to twelve elements, so a parent can have at most
/// eleven direct children. Grouping these LUT RAMs keeps the factory legal.
///
/// Each RAM has [`CONTEXTS`] entries, one per key context, a registered read
/// (data appears the cycle after the address) and one write port.
#[derive(Clone, Debug, Synchronous, SynchronousDQ)]
pub struct ContextRams {
    /// The key's 32-byte one-time seed, as eight words.
    seed: SyncBRAM<[b32; 8], CTX_BITS>,
    /// `pk_nonce` copy for the chain-salt latch.
    pkn_feed: SyncBRAM<[b8; 16], CTX_BITS>,
    /// `pk_nonce` copy for the finished-key metadata.
    pkn_meta: SyncBRAM<[b8; 16], CTX_BITS>,
    /// The private message nonce.
    nonce: SyncBRAM<[b8; 16], CTX_BITS>,
    /// Running chaining value of commitment chunk 0.
    cv0: SyncBRAM<[b32; 8], CTX_BITS>,
    /// Running chaining value of commitment chunk 1.
    cv1: SyncBRAM<[b32; 8], CTX_BITS>,
}

/// Six empty RAMs; their contents are don't-care until written.
impl Default for ContextRams {
    /// Builds the six RAMs with default (unloaded) contents.
    fn default() -> Self {
        Self {
            seed: SyncBRAM::default(),
            pkn_feed: SyncBRAM::default(),
            pkn_meta: SyncBRAM::default(),
            nonce: SyncBRAM::default(),
            cv0: SyncBRAM::default(),
            cv1: SyncBRAM::default(),
        }
    }
}

/// Ports of [`ContextRams`], one simple-dual-port interface each.
///
/// "Simple dual port" means one read address and one independent write port
/// per RAM, both usable in the same cycle.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct ContextRamsIn {
    /// One-time seed, written by `Seed`, read by nonce and XOF ops.
    pub seed: RamIn<[b32; 8], CTX_BITS>,
    /// `pk_nonce` copy read when latching the next key's salt.
    pub pkn_feed: RamIn<[b8; 16], CTX_BITS>,
    /// `pk_nonce` copy read when a key finishes.
    pub pkn_meta: RamIn<[b8; 16], CTX_BITS>,
    /// Private nonce, read when a key finishes.
    pub nonce: RamIn<[b8; 16], CTX_BITS>,
    /// Chunk-0 chaining value.
    pub cv0: RamIn<[b32; 8], CTX_BITS>,
    /// Chunk-1 chaining value.
    pub cv1: RamIn<[b32; 8], CTX_BITS>,
}

/// Registered read data of [`ContextRams`].
///
/// Each field is the entry addressed on the *previous* cycle.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct ContextRamsOut {
    /// Seed words.
    pub seed: [b32; 8],
    /// `pk_nonce` for the salt latch.
    pub pkn_feed: [b8; 16],
    /// `pk_nonce` for key metadata.
    pub pkn_meta: [b8; 16],
    /// Private nonce.
    pub nonce: [b8; 16],
    /// Chunk-0 chaining value.
    pub cv0: [b32; 8],
    /// Chunk-1 chaining value.
    pub cv1: [b32; 8],
}

/// Connects [`ContextRams`] to its port types and wiring kernel.
impl SynchronousIO for ContextRams {
    /// All six RAMs' read addresses and write ports.
    type I = ContextRamsIn;
    /// All six RAMs' registered read data.
    type O = ContextRamsOut;
    /// The pass-through wiring.
    type Kernel = context_rams_kernel;
}

/// Pure wiring between the grouped RAM ports and their children.
///
/// Inputs go straight to the child RAMs (`D`), and the RAMs' read data goes
/// straight out (`Q` to output). No logic and no extra cycle: this block only
/// exists to group six children under one name.
#[kernel]
pub fn context_rams_kernel(
    _clock_reset: ClockReset,
    input: ContextRamsIn,
    q: ContextRamsQ,
) -> (ContextRamsOut, ContextRamsD) {
    (
        ContextRamsOut {
            seed: q.seed,
            pkn_feed: q.pkn_feed,
            pkn_meta: q.pkn_meta,
            nonce: q.nonce,
            cv0: q.cv0,
            cv1: q.cv1,
        },
        ContextRamsD {
            seed: input.seed,
            pkn_feed: input.pkn_feed,
            pkn_meta: input.pkn_meta,
            nonce: input.nonce,
            cv0: input.cv0,
            cv1: input.cv1,
        },
    )
}

/// The factory: one BLAKE3 lane, one chain engine and their bookkeeping.
///
/// `TABLE_MASK` is the chain-lane area knob, passed through to
/// [`SliceChainEngine`]: it chooses which AES rounds of the Haraka lane use
/// table ROMs. See [`crate::chain_step`].
#[derive(Clone, Debug, Synchronous, SynchronousDQ)]
pub struct SliceFactory<const TABLE_MASK: usize> {
    /// The shared BLAKE3 compression pipeline: one request per cycle,
    /// [`PIPELINE_STAGES`] (28) cycles latency.
    lane: InlineCompressionPipeline,
    /// The chain engine: one chain in per cycle, four positions out
    /// [`SLICE_CHAIN_LATENCY`] cycles later.
    chains: SliceChainEngine<TABLE_MASK>,
    /// Packs `pk_nonce` and 68 chain ends (18 bytes each) into 64-byte
    /// commitment words.
    packer: ElementPacker<COMMITMENT_PREFIX>,
    /// The six per-key context RAMs.
    contexts: ContextRams,
    /// XOF blocks, 16 words each. 64 entries addressed `parity << 5 | block`:
    /// two keys' worth (the key feeding chains and the next one), selected by
    /// the low bit of the key's context.
    xof_ram: SyncBRAM<[b32; 16], 6>,
    /// Commitment words, 64 bytes each, addressed `ctx << 5 | word`
    /// ([`CommitAddress`]).
    commit_ram: SyncBRAM<[b8; 64], { CTX_BITS + 5 }>,
    /// Counters, the key ring and pipeline flags; cleared by reset.
    control: DFF<FactoryControl>,
    /// Wide data registers; not reset.
    data: DFF<FactoryData>,
}

/// A factory at power-up: control state zeroed, data registers don't-care.
impl<const TABLE_MASK: usize> Default for SliceFactory<TABLE_MASK> {
    /// Builds every child in its default state.
    fn default() -> Self {
        Self {
            lane: InlineCompressionPipeline,
            chains: SliceChainEngine::default(),
            packer: ElementPacker::default(),
            contexts: ContextRams::default(),
            xof_ram: SyncBRAM::default(),
            commit_ram: SyncBRAM::default(),
            control: DFF::new(FactoryControl::default()),
            data: DFF::resetless(),
        }
    }
}

/// Connects [`SliceFactory`] to its port types and wiring kernel.
impl<const TABLE_MASK: usize> SynchronousIO for SliceFactory<TABLE_MASK> {
    /// Run control, configuration and the two external slot requests.
    type I = SliceFactoryInput;
    /// Cache writes, finished keys and slot results.
    type O = SliceFactoryOutput;
    /// The factory's wiring, [`slice_factory_kernel`].
    type Kernel = slice_factory_kernel<TABLE_MASK>;
}

/// Little-endian bytes to words.
///
/// Input: 64 bytes. Output: 16 words of 32 bits, word `w` made of bytes
/// `4w..4w+4` with byte `4w` as the lowest. BLAKE3 reads its message block
/// this way. Pure wiring: no logic gates, no cycles.
#[allow(clippy::needless_range_loop)]
#[kernel]
pub fn words_from_bytes_kernel(bytes: [b8; 64]) -> [b32; 16] {
    let mut words = [b32(0); 16];
    for w in 0..16 {
        // Byte 4w is the least significant byte of word w.
        words[w] = bytes[4 * w].resize::<32>()
            | (bytes[4 * w + 1].resize::<32>() << 8)
            | (bytes[4 * w + 2].resize::<32>() << 16)
            | (bytes[4 * w + 3].resize::<32>() << 24);
    }
    words
}

/// Words to little-endian bytes.
///
/// The inverse of [`words_from_bytes_kernel`]: 16 words in, 64 bytes out.
/// Used to turn BLAKE3 outputs into the byte strings the protocol defines.
#[allow(clippy::needless_range_loop)]
#[kernel]
pub fn bytes_from_words_kernel(words: [b32; 16]) -> [b8; 64] {
    let mut bytes = [b8(0); 64];
    for w in 0..16 {
        bytes[4 * w] = words[w].resize();
        bytes[4 * w + 1] = (words[w] >> 8).resize();
        bytes[4 * w + 2] = (words[w] >> 16).resize();
        bytes[4 * w + 3] = (words[w] >> 24).resize();
    }
    bytes
}

/// Pick a chain's 18 secret bytes from the two-block window.
///
/// Inputs: `low` (the block holding the secret's first byte), `high` (the
/// block after it) and `offset` (even byte offset 0..62 within `low`).
/// Output: the 18 bytes starting at `offset` in `low ‖ high`. Combinational.
///
/// Why written as an OR over 32 cases: RHDL turns a dynamic array index into
/// whole-array copies, which is very large. Comparing `offset` against each
/// of the 32 possible even values with constant indices instead gives a
/// 32-way select per output byte (see the RHDL notes in [`crate::packer`]).
#[allow(clippy::needless_range_loop)]
#[kernel]
pub fn pick_secret_kernel(low: [b8; 64], high: [b8; 64], offset: b6) -> [b8; ELEMENT_BYTES] {
    // The two blocks side by side, so a secret that crosses the block
    // boundary is one contiguous run of bytes.
    let mut joined = [b8(0); 128];
    for byte in 0..64 {
        joined[byte] = low[byte];
        joined[64 + byte] = high[byte];
    }
    let mut secret = [b8(0); ELEMENT_BYTES];
    for k in 0..ELEMENT_BYTES {
        // Byte k of the secret: exactly one case below matches, the others
        // contribute zero.
        let mut value = b8(0);
        for f in 0..32 {
            // Case f: the secret starts at byte 2f.
            if offset == b6((2 * f) as u128) {
                value = value | joined[2 * f + k];
            }
        }
        secret[k] = value;
    }
    secret
}

/// Pack (op, ctx) into a lane tag; slot requests keep their 16-bit tag above.
///
/// The BLAKE3 lane carries a 32-bit tag beside each request and returns it
/// with the result. 28 cycles later, the tag alone tells the result path what
/// the result is and where to write it.
///
/// ```text
///   bit:  31            16 15   9+CTX_BITS  9 8     4 3    0
///         [ external tag ] [ 0 ] [   ctx    ] [  arg  ] [kind]
/// ```
#[kernel]
pub fn lane_tag_kernel(kind: b4, arg: b5, ctx: Ctx, external: b16) -> b32 {
    kind.resize::<32>()
        | (arg.resize::<32>() << 4)
        | (ctx.resize::<32>() << 9)
        | (external.resize::<32>() << 16)
}

/// Read one ring entry by dynamic context index.
///
/// Inputs: the whole ring and a context index. Output: `ring[ctx]`. Written
/// as a compare-and-select loop for the same reason as
/// [`pick_secret_kernel`]: a [`CONTEXTS`]-way multiplexer instead of array
/// copies. Combinational.
#[kernel]
pub fn ring_entry_kernel(ring: [KeyEntry; CONTEXTS], ctx: Ctx) -> KeyEntry {
    let mut entry = KeyEntry::default();
    for index in 0..CONTEXTS {
        // Exactly one index matches.
        if ctx == CTX_INDEX[index] {
            entry = ring[index];
        }
    }
    entry
}

/// The factory's wiring.
///
/// Like every RHDL kernel, this runs once per clock: it reads the current
/// register and child outputs (`q`) and the inputs, and returns this cycle's
/// outputs and the next values for every register and child input (`d`).
///
/// It does six jobs each cycle, in the sections marked below:
///
/// | Section | What happens this cycle |
/// |---|---|
/// | time | advance the residue (0..=67) and period counters |
/// | key admission | at residue 0, decide key `period + KEY_LEAD`: real or bubble |
/// | decode (y − 1) | look up next cycle's op, find its context, address the RAMs |
/// | issue (y) | build the BLAKE3 request of the op decoded last cycle |
/// | result path | register the lane's output; route it to a RAM or an output |
/// | chain feed | feed chain `residue` of the key whose period this is |
/// | chain results → cache | write four positions per chain; pack chain ends |
/// | RAM ports | collect every RAM's read address and write port |
///
/// Inputs: [`SliceFactoryInput`]. Outputs: [`SliceFactoryOutput`]; the
/// cache write appears [`SLICE_CHAIN_LATENCY`] cycles after its chain was
/// fed, [`KeyDone`] one cycle after the key's parent result leaves the lane.
#[allow(clippy::too_many_lines, clippy::needless_range_loop)]
#[kernel]
pub fn slice_factory_kernel<const TABLE_MASK: usize>(
    clock_reset: ClockReset,
    input: SliceFactoryInput,
    q: SliceFactoryQ<TABLE_MASK>,
) -> (SliceFactoryOutput, SliceFactoryD<TABLE_MASK>) {
    // True during reset: every valid flag below is forced low.
    let resetting = clock_reset.reset.any();
    // Short name for the run configuration.
    let cfg = input.config;
    // Next-state copies: start from the current state and change fields.
    let mut control = q.control;
    let mut data = q.data;

    // ---------------------------------------------------------------- time
    // Residue of this cycle (0..=67) and of the next one.
    let residue = q.control.residue;
    let next_residue = if residue == b7((KEY_PERIOD - 1) as u128) {
        b7(0)
    } else {
        residue + b7(1)
    };
    // Period of the *next* cycle (decode looks one cycle ahead).
    let next_period = if residue == b7((KEY_PERIOD - 1) as u128) {
        q.control.period + CTX_ONE
    } else {
        q.control.period
    };
    control.residue = next_residue;
    control.period = next_period;

    // ------------------------------------------------------- key admission
    // At residue 0 of period p, decide key p + KEY_LEAD.
    if residue == b7(0) {
        // Context of the key being decided (key number mod CONTEXTS).
        let decided: Ctx = q.control.period + CTX_LEAD;
        // The run asked for a fixed number of batches and they are all made.
        let limit_reached = cfg.batches != b32(0) && q.control.next_batch >= cfg.batches;
        // No key starts until every UltraRAM S-box is loaded (see
        // `aes_table::SboxLoader`); instantly true with no UltraRAM boxes.
        let real = input.run && input.space && !limit_reached && q.chains.ready;
        // A bubble gets the default entry (real = false).
        let mut entry = KeyEntry::default();
        if real {
            // A real key takes the next slot and the next fresh index. These
            // counters only move forward, so no index is handed out twice.
            entry = KeyEntry {
                real: true,
                slot: q.control.next_slot,
                batch: q.control.next_batch,
                j: q.control.next_j,
            };
            control.next_slot = q.control.next_slot + b9(1);
            control.started = q.control.started + b32(1);
            control.next_j = q.control.next_j + KEY_ONE;
            // After the slice's last key of a batch, move to the next batch.
            if q.control.next_j == LAST_KEY {
                control.next_j = KEY_ZERO;
                control.next_batch = q.control.next_batch + b32(1);
            }
        }
        // Store the entry in the decided key's context.
        for index in 0..CONTEXTS {
            if decided == CTX_INDEX[index] {
                control.ring[index] = entry;
            }
        }
    }

    // ----------------------------------------------------- decode (y − 1)
    // The op that issues next cycle: the table row of next cycle's residue.
    let mut slot = Slot::default();
    for r in 0..KEY_PERIOD {
        if next_residue == b7(r as u128) {
            slot = SLOTS[r];
        }
    }
    // Context of the key this op belongs to: (period − lag) mod CONTEXTS.
    // (The table's lag carries LAG_BIAS, which CTX_BIAS cancels.)
    let ctx: Ctx = next_period + CTX_BIAS - slot.lag;
    // Bookkeeping of that key: is it real?
    let owner = ring_entry_kernel(q.control.ring, ctx);
    // External slots run only when their caller has a request waiting;
    // key ops run only for real keys. Idle residues never run.
    let is_merkle = slot.kind == b4(code::MERKLE as u128);
    let is_message = slot.kind == b4(code::MESSAGE as u128);
    let live = if is_merkle {
        input.merkle.valid
    } else if is_message {
        // Each message residue takes only its own kind of request: argument
        // 0 first blocks, argument 1 (profile 2's tail residue) tails. A
        // first block taken at the tail residue would burn a second key in
        // the same period while its tail had nowhere to go.
        input.message.valid && (input.message.tail == (slot.arg == b5(1)))
    } else {
        slot.kind != b4(code::IDLE as u128) && owner.real
    };
    control.decoded = Decoded {
        kind: slot.kind,
        arg: slot.arg,
        ctx,
        live,
    };
    // Context-RAM read addresses for the op issuing next cycle.
    // Commitment word to read: chunk-0 block n is word n, chunk-1 block n
    // is word 16 + n.
    let word_index: b5 = if slot.kind == b4(code::CHUNK1 as u128) {
        slot.arg + b5(CHUNK0_BLOCKS as u128)
    } else {
        slot.arg
    };
    let ctx_wide: CommitAddress = ctx.resize();
    let word_wide: CommitAddress = word_index.resize();
    // Address = ctx << 5 | word (see `CommitAddress`).
    let commit_address: CommitAddress = (ctx_wide << 5) | word_wide;

    // ------------------------------------------------------- issue (y)
    // The op decoded last cycle; the RAM data it asked for is ready now.
    let op = q.control.decoded;
    let issuing = op.live && !resetting;
    // Its key's bookkeeping (batch and j for the seed request).
    let entry = ring_entry_kernel(q.control.ring, op.ctx);
    // The key's seed, read from its context.
    let seed = q.contexts.seed;
    // Default request: a single-block root hash starting from the IV, with a
    // full 64-byte block of zeros. Each op kind below changes what it needs.
    let mut request = CompressionInput {
        cv: IV_BITS,
        block: [b32(0); 16],
        counter_low: b32(0),
        counter_high: b32(0),
        block_len: b32(64),
        flags: FLAGS_SINGLE,
        tag: lane_tag_kernel(op.kind, op.arg, op.ctx, b16(0)),
        valid: issuing,
    };
    if op.kind == b4(code::SEED as u128) {
        // Seed: derive-key hash, keyed by the secret prefix CV, of the
        // 20-byte material epoch ‖ batch id ‖ key index (little-endian).
        // Key index within the batch = slice << SUBTREE_LEVELS | j.
        let index: b32 =
            (cfg.slice.resize::<32>() << (SUBTREE_LEVELS as u128)) | entry.j.resize::<32>();
        // Absolute batch id.
        let batch_id = cfg.first_batch + entry.batch.resize::<64>();
        request.cv = cfg.prefix_cv;
        request.block[0] = cfg.epoch.resize();
        request.block[1] = (cfg.epoch >> 32).resize();
        request.block[2] = batch_id.resize();
        request.block[3] = (batch_id >> 32).resize();
        request.block[4] = index;
        request.block_len = b32(20);
        request.flags = FLAGS_SEED;
    } else if op.kind == b4(code::PK_NONCE as u128) || op.kind == b4(code::NONCE as u128) {
        // Nonces: hash of a 4-byte prefix word and the 32-byte seed (36 bytes).
        // The prefix word separates the public and the private nonce.
        request.block[0] = if op.kind == b4(code::PK_NONCE as u128) {
            PK_NONCE_WORD
        } else {
            NONCE_WORD
        };
        for w in 0..8 {
            request.block[1 + w] = seed[w];
        }
        request.block_len = b32(36);
    } else if op.kind == b4(code::XOF as u128) {
        // XOF block n: hash of the 32-byte seed with the BLAKE3 block counter
        // set to n, so each n gives 64 new bytes of the stream.
        for w in 0..8 {
            request.block[w] = seed[w];
        }
        request.counter_low = op.arg.resize();
        request.block_len = b32(32);
    } else if op.kind == b4(code::CHUNK0 as u128) || op.kind == b4(code::CHUNK1 as u128) {
        // Commitment blocks. Chunk 0 has blocks 0..16, chunk 1 blocks 0..4.
        let chunk1 = op.kind == b4(code::CHUNK1 as u128);
        // First block of a chunk starts from the IV; later blocks continue
        // from the chunk's chaining value stored in the context.
        let first = op.arg == b5(0);
        let last = if chunk1 {
            op.arg == b5(3)
        } else {
            op.arg == b5((CHUNK0_BLOCKS - 1) as u128)
        };
        if !first {
            request.cv = if chunk1 {
                q.contexts.cv1
            } else {
                q.contexts.cv0
            };
        }
        // The 64-byte commitment word, read from the commitment RAM.
        request.block = words_from_bytes_kernel(q.commit_ram);
        // BLAKE3's chunk counter: 0 for chunk 0, 1 for chunk 1.
        request.counter_low = if chunk1 { b32(1) } else { b32(0) };
        // The commitment is 1,240 = 19 × 64 + 24 bytes: the last block is short.
        request.block_len = if chunk1 && last { b32(24) } else { b32(64) };
        // Not a root here: the parent below is the root.
        let mut flags = b32(0);
        if first {
            flags = flags | FLAG_START;
        }
        if last {
            flags = flags | FLAG_END;
        }
        request.flags = flags;
    } else if op.kind == b4(code::PARENT as u128) {
        // Parent: the two chunk chaining values side by side form the block.
        for w in 0..8 {
            request.block[w] = q.contexts.cv0[w];
            request.block[8 + w] = q.contexts.cv1[w];
        }
        request.flags = FLAGS_PARENT;
    } else if op.kind == b4(code::MERKLE as u128) {
        // External slots: the caller supplies the whole block and its tag.
        request.block = input.merkle.block;
        request.tag = lane_tag_kernel(op.kind, op.arg, op.ctx, input.merkle.tag);
    } else if op.kind == b4(code::MESSAGE as u128) {
        request.block = input.message.block;
        request.tag = lane_tag_kernel(op.kind, op.arg, op.ctx, input.message.tag);
        if TWO_BLOCK_MESSAGE {
            if input.message.tail {
                request.cv = input.message.cv;
                request.block_len = b32(48);
                request.flags = FLAGS_END_ROOT;
            } else {
                request.flags = FLAG_START;
            }
        }
    }
    // Tell the callers their request entered the lane this cycle.
    let merkle_taken = issuing && op.kind == b4(code::MERKLE as u128);
    let message_taken = issuing && op.kind == b4(code::MESSAGE as u128);

    // ------------------------------------------------------ result path
    // Lane output → result register (this cycle) → RAM writes (next cycle).
    control.result_valid = q.lane.valid && !resetting;
    data.result = q.lane.words;
    data.result_tag = q.lane.tag;
    // Unpack the registered result's tag: what op, which argument, which key.
    let r_kind: b4 = q.data.result_tag.resize();
    let r_ctx: Ctx = (q.data.result_tag >> 9).resize();
    let r_arg: b5 = (q.data.result_tag >> 4).resize();
    let r_valid = q.control.result_valid;
    let result = q.data.result;
    let result_bytes = bytes_from_words_kernel(result);
    // Nonces are the first 16 bytes of their hash output.
    let mut first16 = [b8(0); 16];
    for byte in 0..16 {
        first16[byte] = result_bytes[byte];
    }
    // Seeds and chaining values are the first 8 words.
    let mut cv8 = [b32(0); 8];
    for w in 0..8 {
        cv8[w] = result[w];
    }
    // Meta reads for a parent result are addressed when it leaves the lane.
    // (So `pkn_meta` and `nonce` output that key's values next cycle, when
    // the parent result sits in the result register.)
    let lane_ctx: Ctx = (q.lane.tag >> 9).resize();
    // Bookkeeping of the result's key.
    let r_entry = ring_entry_kernel(q.control.ring, r_ctx);
    // pk_hash is the first 32 bytes of the parent output.
    let mut pk_hash = [b8(0); 32];
    for byte in 0..32 {
        pk_hash[byte] = result_bytes[byte];
    }
    // A parent result means the key is complete.
    let key_done = KeyDone {
        valid: r_valid && r_kind == b4(code::PARENT as u128),
        slot: r_entry.slot,
        batch: r_entry.batch,
        j: r_entry.j,
        pk_hash,
        pk_nonce: q.contexts.pkn_meta,
        nonce: q.contexts.nonce,
    };
    // Merkle and message results go back to their callers, with the
    // caller's 16-bit tag from the top half of the lane tag.
    let slot_result = SlotResult {
        valid: r_valid
            && (r_kind == b4(code::MERKLE as u128) || r_kind == b4(code::MESSAGE as u128)),
        message: r_kind == b4(code::MESSAGE as u128),
        words: result,
        tag: (q.data.result_tag >> 16).resize(),
    };

    // ---------------------------------------------------- chain feed (T+i)
    // Window loads: address at residue first_chain(t) − 2, data next cycle.
    // This residue's row of the window-load table.
    let mut load = WindowLoad::default();
    for r in 0..KEY_PERIOD {
        if residue == b7(r as u128) {
            load = WINDOW_LOADS[r];
        }
    }
    // Which key the block belongs to: this period's key, or (block 0) the
    // next period's key.
    let feed_period: Ctx = if load.next_key {
        q.control.period + CTX_ONE
    } else {
        q.control.period
    };
    // XOF parity storage: (key parity, block).
    let xof_read: b6 = (feed_period.resize::<6>() & b6(1)) << 5 | load.block.resize::<6>();
    // Remember to capture the RAM data next cycle, and into which half.
    control.window_pending = load.active;
    control.window_parity = (load.block & b5(1)) == b5(1);
    // The read addressed last cycle has arrived: store it in the window.
    if q.control.window_pending {
        let block = bytes_from_words_kernel(q.xof_ram);
        if q.control.window_parity {
            data.window[1] = block;
        } else {
            data.window[0] = block;
        }
    }
    // pk_nonce of the next key: address at residue 66, latch at 67.
    control.latch_pending = residue == b7((KEY_PERIOD - 2) as u128);
    // At residue 67: the current pk_nonce becomes history (shift by one,
    // oldest dropped), and the next key's pk_nonce becomes current, ready
    // for its chain 0 at residue 0.
    if q.control.latch_pending {
        data.older_nonces[0] = q.data.pk_nonce;
        for k in 1..PACK_HISTORY {
            data.older_nonces[k] = q.data.older_nonces[k - 1];
        }
        data.pk_nonce = q.contexts.pkn_feed;
    }
    // Where this cycle's chain secret sits in the window.
    let mut pick = ChainPick::default();
    for r in 0..KEY_PERIOD {
        if residue == b7(r as u128) {
            pick = CHAIN_PICKS[r];
        }
    }
    // Order the two window halves as (block with the first byte, next block).
    let (low, high) = if pick.low_parity {
        (q.data.window[1], q.data.window[0])
    } else {
        (q.data.window[0], q.data.window[1])
    };
    // The key whose chains run in this period is the key numbered `period`.
    let feed_ctx = q.control.period;
    let feed_entry = ring_entry_kernel(q.control.ring, feed_ctx);
    // At residue i we feed chain i (KEY_PERIOD = CHAINS = 68).
    let chain_index: b7 = residue;
    // Bubbles feed nothing.
    let feeding = feed_entry.real && residue < b7(CHAINS as u128) && !resetting;
    // Chain tag: chain in bits 0..7, cache slot in bits 7..16, context from
    // bit 16. The engine returns it unchanged with the finished chain.
    let chain_tag: b32 = chain_index.resize::<32>()
        | (feed_entry.slot.resize::<32>() << 7)
        | (feed_ctx.resize::<32>() << 16);

    // --------------------------------------------- chain results → cache
    // A finished chain from the engine, and its unpacked tag.
    let out = q.chains;
    let out_chain: b7 = out.tag.resize();
    let out_slot: b9 = (out.tag >> 7).resize();
    let out_ctx: Ctx = (out.tag >> 16).resize();
    let cache_write = CacheWrite {
        valid: out.valid,
        slot: out_slot,
        chain: out_chain,
        nodes: out.nodes,
    };
    // Chain 0 of a key starts a new commitment stream; remember its context
    // so the packer's words land in that key's part of the commitment RAM.
    let starting = out.valid && out_chain == b7(0);
    if starting {
        control.pack_ctx = out_ctx;
    }
    // The emerging key's pk_nonce: current, or `age` keys back.
    // age 0 = this period's key (current pk_nonce); age k + 1 =
    // older_nonces[k]. With a long chain hash, ends emerge periods late.
    let age: Ctx = q.control.period - out_ctx;
    let mut pack_nonce = q.data.pk_nonce;
    for k in 0..PACK_HISTORY {
        if age == CTX_INDEX[k + 1] {
            pack_nonce = q.data.older_nonces[k];
        }
    }
    // The packer's prefix: pk_nonce in the first 16 bytes, rest unused.
    let mut prefix = [b8(0); 64];
    for byte in 0..16 {
        prefix[byte] = pack_nonce[byte];
    }
    // Packer words → commitment RAM at (ctx, word).
    let packed = q.packer;
    let pack_ctx_wide: CommitAddress = q.control.pack_ctx.resize();
    let pack_word_wide: CommitAddress = q.control.pack_word.resize();
    let pack_address: CommitAddress = (pack_ctx_wide << 5) | pack_word_wide;
    // Advance the word index; wrap to 0 after the key's last word.
    if packed.valid {
        control.pack_word = if packed.last {
            b5(0)
        } else {
            q.control.pack_word + b5(1)
        };
    }

    // --------------------------------------------------------- RAM ports
    // Each result kind is written to its own RAM, at its key's context.
    let write_seed = r_valid && r_kind == b4(code::SEED as u128);
    let write_pkn = r_valid && r_kind == b4(code::PK_NONCE as u128);
    let write_nonce = r_valid && r_kind == b4(code::NONCE as u128);
    let write_xof = r_valid && r_kind == b4(code::XOF as u128);
    let write_cv0 = r_valid && r_kind == b4(code::CHUNK0 as u128);
    let write_cv1 = r_valid && r_kind == b4(code::CHUNK1 as u128);
    // XOF RAM address: key parity in bit 5, block number in bits 0..5.
    let xof_write: b6 = (r_ctx.resize::<6>() & b6(1)) << 5 | r_arg.resize::<6>();

    // Next-cycle inputs of every child and register.
    let d = SliceFactoryD::<TABLE_MASK> {
        // The request built in the issue section.
        lane: request,
        // This cycle's chain: its secret from the window, the key's salt.
        chains: SliceChainInput {
            secret: pick_secret_kernel(low, high, pick.offset),
            pk_nonce: q.data.pk_nonce,
            index: chain_index,
            tag: chain_tag,
            valid: feeding,
        },
        // Only the chain end (position 3) goes into the commitment.
        packer: PackerInput {
            prefix,
            start: starting,
            node: out.nodes[3],
            valid: out.valid,
        },
        // Reads use the decoded op's context (`ctx`) except where noted;
        // writes use the result's context (`r_ctx`).
        contexts: ContextRamsIn {
            seed: RamIn::<[b32; 8], CTX_BITS> {
                read_addr: ctx,
                write: RamWrite::<[b32; 8], CTX_BITS> {
                    addr: r_ctx,
                    value: cv8,
                    enable: write_seed,
                },
            },
            // Always reads the next key's pk_nonce, for the latch at 67.
            pkn_feed: RamIn::<[b8; 16], CTX_BITS> {
                read_addr: q.control.period + CTX_ONE,
                write: RamWrite::<[b8; 16], CTX_BITS> {
                    addr: r_ctx,
                    value: first16,
                    enable: write_pkn,
                },
            },
            // Reads follow the lane output, for the KeyDone metadata.
            pkn_meta: RamIn::<[b8; 16], CTX_BITS> {
                read_addr: lane_ctx,
                write: RamWrite::<[b8; 16], CTX_BITS> {
                    addr: r_ctx,
                    value: first16,
                    enable: write_pkn,
                },
            },
            nonce: RamIn::<[b8; 16], CTX_BITS> {
                read_addr: lane_ctx,
                write: RamWrite::<[b8; 16], CTX_BITS> {
                    addr: r_ctx,
                    value: first16,
                    enable: write_nonce,
                },
            },
            cv0: RamIn::<[b32; 8], CTX_BITS> {
                read_addr: ctx,
                write: RamWrite::<[b32; 8], CTX_BITS> {
                    addr: r_ctx,
                    value: cv8,
                    enable: write_cv0,
                },
            },
            cv1: RamIn::<[b32; 8], CTX_BITS> {
                read_addr: ctx,
                write: RamWrite::<[b32; 8], CTX_BITS> {
                    addr: r_ctx,
                    value: cv8,
                    enable: write_cv1,
                },
            },
        },
        // XOF RAM: window reads, XOF result writes.
        xof_ram: RamIn::<[b32; 16], 6> {
            read_addr: xof_read,
            write: RamWrite::<[b32; 16], 6> {
                addr: xof_write,
                value: result,
                enable: write_xof,
            },
        },
        // Commitment RAM: chunk-op reads, packer writes.
        commit_ram: RamIn::<[b8; 64], { CTX_BITS + 5 }> {
            read_addr: commit_address,
            write: RamWrite::<[b8; 64], { CTX_BITS + 5 }> {
                addr: pack_address,
                value: packed.word,
                enable: packed.valid,
            },
        },
        control,
        data,
    };
    (
        SliceFactoryOutput {
            cache_write,
            key_done,
            slot_result,
            merkle_taken,
            message_taken,
            keys_started: q.control.started,
        },
        d,
    )
}