rhdl_dsig_scalable_core/

haraka.rs

//! Haraka-512 v2 and the 144-bit DSig chain-compression primitive.
//!
//! # Why this module exists
//!
//! DSig signs with W-OTS+ (a Winternitz one-time signature, here with `w = 4`).
//! Its secret key is 68 short *chains*. Each chain starts at an 18-byte
//! (144-bit) secret node, and every step along the chain hashes the current
//! node once. A signature needs 204 of these chain steps, so the chain hash is
//! the single most-repeated operation in the whole signer.
//!
//! The `DSIG_FPGA_W4_HARAKA_BLAKE3_V1` profile uses **Haraka-512 v2** for that
//! hash. Haraka is built entirely from AES rounds, which suits hardware: an AES
//! round is a few table lookups and XORs, and it pipelines well. This module is
//! the *meaning* of Haraka in this crate. It holds:
//!
//! - a plain-Rust reference ([`haraka512`], [`dsig_chain`]) that the tests and
//!   the host use as an oracle;
//! - the constants (round keys, the word mix, the truncation) shared by every
//!   hardware variant;
//! - small RHDL kernels for each piece of an AES round;
//! - a general 20-stage pipeline ([`Haraka512Pipeline`]) that hashes any block.
//!
//! The signer itself uses a specialised copy in [`crate::chain_step`], which
//! reuses the kernels and constants here but folds away the constant bytes of
//! a chain block. The S-box memories live in [`crate::aes_table`].
//! [`crate::chain_lane`] picks which chain hash is compiled in.
//!
//! # Haraka-512 in one picture
//!
//! The 64-byte state is four 16-byte *AES lanes*. (Here "lane" means a 16-byte
//! quarter of the state. Elsewhere in the crate a "lane" is a whole hardware
//! chain-step pipeline; the context makes clear which.) One Haraka round is two
//! AES rounds on every AES lane, then a fixed shuffle of 32-bit words across
//! lanes ([`MIX_WORDS`]). There are five Haraka rounds:
//!
//! ```text
//!   input (64 B) ─┬───────────────────────────────────────────────────────┐
//!                 ▼                                                       │
//!   AES lane 0  [AES][AES]┐                                               │
//!   AES lane 1  [AES][AES]├─ MIX ─▶ ... repeat: 5 Haraka rounds in all     │
//!   AES lane 2  [AES][AES]│        = 5 × 2 AES rounds × 4 lanes            │
//!   AES lane 3  [AES][AES]┘        = 40 AES rounds                         │
//!                 │                                                       │
//!                 ▼                                                       ▼
//!   final state ── pick 32 bytes (DIGEST_BYTES) ──▶ XOR ◀── same 32 input bytes
//!                                                    │       ("feed-forward")
//!                                                    ▼
//!                                          32-byte Haraka-512 digest
//!                                                    │
//!                                          first 18 bytes = next DSig chain node
//! ```
//!
//! Each `[AES]` box is one AES round (`AESENC`), drawn in detail in
//! [`crate::aes_table`]: SubBytes → ShiftRows → MixColumns → AddRoundKey.
//! The feed-forward (XOR the input back in) is what turns the reversible AES
//! permutation into a one-way compression function.
//!
//! # From the picture to a pipeline
//!
//! Each AES round is cut into two registered halves: the S-box lookups on one
//! side, the XOR network of MixColumns and the key on the other. That way an
//! S-box can be a synchronous ROM whose own output register is the stage
//! register (see [`crate::aes_table`]). Ten AES rounds give 20 stages
//! ([`HARAKA_PIPELINE_LATENCY`]):
//!
//! ```text
//!   stage:   0          1                2          3                     ...  19
//!          SubBytes   MixColumns       SubBytes   MixColumns
//!          +ShiftRows +AddRoundKey     +ShiftRows +AddRoundKey + MIX
//!          (round 0)  (round 0)        (round 1)  (round 1)
//!            reg        reg              reg        reg                        reg
//! ```
//!
//! Even stages are [`sub_shift512_kernel`]; odd stages are
//! [`mix_key512_kernel`], and every fourth stage (3, 7, 11, 15, 19) also
//! applies the word mix. A new block can enter every clock (initiation
//! interval, "II", of 1), and its digest leaves 20 clocks later.
//!
//! # Reference, pipeline and provenance
//!
//! [`haraka512`](crate::haraka::haraka512) is a byte-oriented, allocation-free reference. The reference
//! uses field arithmetic independently of the hardware S-box. It is an oracle,
//! not a side-channel-hardened software signing implementation. The RHDL
//! [`Haraka512Pipeline`](crate::haraka::Haraka512Pipeline) has twenty registered stages: each AES round is split
//! between SubBytes/ShiftRows and MixColumns/AddRoundKey. Haraka's inter-lane
//! permutation is wiring following every second AES round. A separate valid
//! pipeline resets ownership; resetless state and tags are never meaningful
//! while `valid` is false. No throughput, clock, or resource figure here is a
//! synthesis or hardware measurement.
//!
//! Constants, lane permutations, and the official known-answer vector follow
//! [kste/haraka](https://github.com/kste/haraka), Haraka v2. DSig's
//! `mid_haraka512` selects exactly the first eighteen bytes of the standard
//! 32-byte output, **not** the first eighteen bytes of the full 64-byte state.
//! The salt layout follows DSig commit
//! `1647bfbb12c42a249d68ce4909f81c74baab69bc`
//! (`dsig/src/util.hpp`, `third-party/src/haraka/haraka.h`).
//!
//! Derived Haraka constants/permutation attribution: Copyright (c) 2016 kste.
//! The MIT License (MIT): Permission is hereby granted, free of charge, to any
//! person obtaining a copy of this software and associated documentation files
//! (the "Software"), to deal in the Software without restriction, including
//! without limitation the rights to use, copy, modify, merge, publish,
//! distribute, sublicense, and/or sell copies of the Software, and to permit
//! persons to whom the Software is furnished to do so, subject to the following
//! conditions: The above copyright notice and this permission notice shall be
//! included in all copies or substantial portions of the Software. THE SOFTWARE
//! IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
//! INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR
//! A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
//! COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
//! WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
//! IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

use rhdl::prelude::*;
use rhdl_fpga::core::dff::DFF;

/// AES rounds per Haraka-512 invocation (two rounds per mixing layer).
///
/// Five Haraka rounds × two AES rounds = 10. Each of these rounds runs on all
/// four 16-byte AES lanes at once, so one hash is 40 AES-round evaluations.
pub const AES_ROUNDS: usize = 10;
/// Registered transfers per complete hash; there is no input backpressure.
///
/// Two stages per AES round (SubBytes/ShiftRows, then MixColumns/AddRoundKey),
/// so 20 clock cycles from input to digest.
pub const HARAKA_PIPELINE_LATENCY: usize = 2 * AES_ROUNDS;

/// Haraka v2 constants as integers whose little-endian bytes match AES-NI.
///
/// Upstream `_mm_set_epi32(a, b, c, d)` is represented as the integer
/// `0xaaaabbbbccccdddd`: the first byte in memory is the low byte of `d`.
///
/// 40 entries of 128 bits: entry `4·round + lane` is the AddRoundKey value for
/// AES lane `lane` in AES round `round`. Haraka uses fixed public constants
/// instead of a secret key; they only break the symmetry between rounds and
/// lanes.
pub const ROUND_CONSTANTS: [u128; 40] = [
    0x0684704c_e620c00a_b2c5fef0_75817b9d,
    0x8b66b4e1_88f3a06b_640f6ba4_2f08f717,
    0x3402de2d_53f28498_cf029d60_9f029114,
    0x0ed6eae6_2e7b4f08_bbf3bcaf_fd5b4f79,
    0xcbcfb0cb_4872448b_79eecd1c_be397044,
    0x7eeacdee_6e9032b7_8d5335ed_2b8a057b,
    0x67c28f43_5e2e7cd0_e2412761_da4fef1b,
    0x2924d9b0_afcacc07_675ffde2_1fc70b3b,
    0xab4d63f1_e6867fe9_ecdb8fca_b9d465ee,
    0x1c30bf84_d4b7cd64_5b2a404f_ad037e33,
    0xb2cc0bb9_941723bf_69028b2e_8df69800,
    0xfa0478a6_de6f5572_4aaa9ec8_5c9d2d8a,
    0xdfb49f2b_6b772a12_0efa4f2e_29129fd4,
    0x1ea10344_f449a236_32d611ae_bb6a12ee,
    0xaf044988_4b050084_5f9600c9_9ca8eca6,
    0x21025ed8_9d199c4f_78a2c7e3_27e593ec,
    0xbf3aaaf8_a759c9b7_b9282ecd_82d40173,
    0x6260700d_6186b017_37f2efd9_10307d6b,
    0x5aca45c2_21300443_81c29153_f6fc9ac6,
    0x9223973c_226b68bb_2caf92e8_36d1943a,
    0xd3bf9238_225886eb_6cbab958_e51071b4,
    0xdb863ce5_aef0c677_933dfddd_24e1128d,
    0xbb606268_ffeba09c_83e48de3_cb2212b1,
    0x734bd3dc_e2e4d19c_2db91a4e_c72bf77d,
    0x43bb47c3_61301b43_4b1415c4_2cb3924e,
    0xdba775a8_e707eff6_03b231dd_16eb6899,
    0x6df3614b_3c755977_8e5e2302_7eca472c,
    0xcda75a17_d6de7d77_6d1be5b9_b88617f9,
    0xec6b43f0_6ba8e9aa_9d6c069d_a946ee5d,
    0xcb1e6950_f957332b_a2531159_3bf327c1,
    0x2cee0c75_00da619c_e4ed0353_600ed0d9,
    0xf0b1a5a1_96e90cab_80bbbabc_63a4a350,
    0xae3db102_5e962988_ab0dde30_938dca39,
    0x17bb8f38_d554a40b_8814f3a8_2e75b442,
    0x34bb8a5b_5f427fd7_aeb6b779_360a16f6,
    0x26f65241_cbe55438_43ce5918_ffbaafde,
    0x4ce99a54_b9f3026a_a2ca9cf7_839ec978,
    0xae51a51a_1bdff7be_40c06e28_22901235,
    0xa0c1613c_ba7ed22b_c173bc0f_48a659cf,
    0x756acc03_02288288_4ad6bdfd_e9c59da1,
];

/// Source-word positions for the Haraka-512 32-bit inter-lane permutation.
///
/// The state is 16 words of 4 bytes. After the mix, word `w` holds what was
/// word `MIX_WORDS[w]`. This is the only thing that moves data between AES
/// lanes, and it is pure wiring in hardware (no logic at all).
pub const MIX_WORDS: [usize; 16] = [3, 11, 7, 15, 8, 0, 12, 4, 9, 1, 13, 5, 2, 10, 6, 14];
/// Source-byte positions of the standard 256-bit feed-forward/truncation.
///
/// Digest byte `i` is `final[DIGEST_BYTES[i]] ⊕ input[DIGEST_BYTES[i]]`. The
/// picked bytes are the upper half of AES lanes 0 and 1 (bytes 8..16 and
/// 24..32) and the lower half of lanes 2 and 3 (bytes 32..40 and 48..56).
/// DSig keeps the first 18 of them, which come from only five of the sixteen
/// 32-bit words; [`crate::chain_step`] uses that to skip unobserved S-boxes.
pub const DIGEST_BYTES: [usize; 32] = [
    8, 9, 10, 11, 12, 13, 14, 15, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39,
    48, 49, 50, 51, 52, 53, 54, 55,
];

/// Re-pack [`ROUND_CONSTANTS`] as RHDL bytes, one 64-byte row per AES round.
///
/// Row `round` holds the four lanes' 16-byte keys side by side, matching the
/// 64-byte state layout, so a stage can XOR it in byte by byte.
#[allow(clippy::cast_lossless)] // From<u8> is not available in a const context.
const fn rhdl_round_keys() -> [[b8; 64]; AES_ROUNDS] {
    // result[round][16·lane + byte]: the finished table.
    let mut result = [[b8(0); 64]; AES_ROUNDS];
    let mut round = 0;
    while round < AES_ROUNDS {
        let mut lane = 0;
        while lane < 4 {
            // This lane's 128-bit key as 16 bytes in AES-NI memory order.
            let bytes = ROUND_CONSTANTS[4 * round + lane].to_le_bytes();
            let mut byte = 0;
            while byte < 16 {
                result[round][16 * lane + byte] = b8(bytes[byte] as u128);
                byte += 1;
            }
            lane += 1;
        }
        round += 1;
    }
    result
}

/// Four compile-time AES round keys per registered linear stage.
///
/// `ROUND_KEYS_RHDL[r]` is the 64-byte AddRoundKey value of AES round `r`.
/// Being constants, the key XORs become inverters or nothing at all in
/// synthesis.
pub const ROUND_KEYS_RHDL: [[b8; 64]; AES_ROUNDS] = rhdl_round_keys();

/// Multiply two bytes in the AES field GF(2^8) (compile-time reference).
///
/// GF(2^8) is AES's byte arithmetic: add is XOR, and multiply is
/// shift-and-add reduced by `x^8 + x^4 + x^3 + x + 1` (`0x1b`). This is the
/// slow, obviously-correct form, used only to build reference tables.
const fn gf_multiply(mut left: u8, mut right: u8) -> u8 {
    // Running product (XOR is addition in GF(2^8)).
    let mut product = 0;
    // Which bit of `right` is being processed (8 in all).
    let mut bit = 0;
    while bit < 8 {
        // If this bit of `right` is set, add the current multiple of `left`.
        if right & 1 != 0 {
            product ^= left;
        }
        // Multiply `left` by x (i.e. 2), reducing when the top bit falls off.
        left = (left << 1) ^ if left & 0x80 == 0 { 0 } else { 0x1b };
        right >>= 1;
        bit += 1;
    }
    product
}

/// Independent reference AES S-box via inversion in GF(2^8) and affine map.
///
/// The AES S-box is `S(x) = A(x^254) ⊕ 0x63`, where `x^254` is the field
/// inverse of `x` (and 0 maps to 0) and `A` is a fixed bit-rotation mix.
/// Computing it from the definition, rather than copying a table, gives the
/// tests an oracle that does not share a source with [`aes_sbox_kernel`].
#[must_use]
pub const fn aes_sbox(value: u8) -> u8 {
    // Square-and-multiply for value^254: `inverse` accumulates the result,
    // `power` walks value^1, value^2, value^4, ..., `exponent` is what's left.
    let mut inverse = 1;
    let mut power = value;
    let mut exponent = 254;
    while exponent != 0 {
        if exponent & 1 != 0 {
            inverse = gf_multiply(inverse, power);
        }
        power = gf_multiply(power, power);
        exponent >>= 1;
    }
    // The AES affine map: XOR of the byte with four left rotations, plus 0x63.
    inverse
        ^ inverse.rotate_left(1)
        ^ inverse.rotate_left(2)
        ^ inverse.rotate_left(3)
        ^ inverse.rotate_left(4)
        ^ 0x63
}

/// Evaluate [`aes_sbox`] for all 256 inputs, at compile time.
const fn sbox_table() -> [u8; 256] {
    // table[x] = S(x).
    let mut table = [0; 256];
    let mut x = 0;
    while x < 256 {
        table[x] = aes_sbox(x as u8);
        x += 1;
    }
    table
}

/// [`aes_sbox`] evaluated once for every input at compile time.
///
/// The field-arithmetic definition stays the independent oracle; this table
/// only makes host-side verification of millions of signatures fast.
pub const SBOX: [u8; 256] = sbox_table();

/// One reference AESENC round, including MixColumns and the supplied key.
///
/// `state` and `key` are 16 bytes in AES column-major order: byte
/// `4·column + row`. This is the software meaning of one `[AES]` box in the
/// module picture; the hardware splits it into [`aes_sub_shift_kernel`] and
/// [`aes_mix_key_kernel`].
#[must_use]
pub fn aes_round(state: &[u8; 16], key: &[u8; 16]) -> [u8; 16] {
    // SubBytes and ShiftRows together: output (column, row) is the S-box of
    // input (column + row mod 4, row).
    let mut shifted = [0; 16];
    for column in 0..4 {
        for row in 0..4 {
            shifted[4 * column + row] = SBOX[usize::from(state[4 * ((column + row) % 4) + row])];
        }
    }
    // MixColumns and AddRoundKey: each output byte is
    // 2·s[row] ⊕ 3·s[row+1] ⊕ s[row+2] ⊕ s[row+3] ⊕ key, within its column.
    let mut result = [0; 16];
    for column in 0..4 {
        for row in 0..4 {
            // Index of the column's first byte.
            let base = 4 * column;
            result[base + row] = gf_multiply(shifted[base + row], 2)
                ^ gf_multiply(shifted[base + (row + 1) % 4], 3)
                ^ shifted[base + (row + 2) % 4]
                ^ shifted[base + (row + 3) % 4]
                ^ key[base + row];
        }
    }
    result
}

/// Pure Rust Haraka-512 v2 with fixed standard constants and 32-byte output.
///
/// This fixed-input-length compression function is not a general-purpose
/// variable-length hash and must not be extended by ad-hoc concatenation.
///
/// Input: exactly 64 bytes. Output: the standard 32-byte digest. This is the
/// oracle that every hardware pipeline is compared against.
#[must_use]
pub fn haraka512(input: &[u8; 64]) -> [u8; 32] {
    // The 64-byte working state, four AES lanes of 16 bytes.
    let mut state = *input;
    for round in 0..AES_ROUNDS {
        // One AES round on each of the four lanes, with that lane's key.
        for lane in 0..4 {
            // A copy of this lane's 16 bytes.
            let mut block = [0; 16];
            block.copy_from_slice(&state[16 * lane..16 * (lane + 1)]);
            // The lane after one AES round.
            let next = aes_round(&block, &ROUND_CONSTANTS[4 * round + lane].to_le_bytes());
            state[16 * lane..16 * (lane + 1)].copy_from_slice(&next);
        }
        // After every second AES round (rounds 1, 3, 5, 7, 9): the word mix.
        if round % 2 == 1 {
            // Snapshot, because the permutation reads and writes the state.
            let before = state;
            for word in 0..16 {
                state[4 * word..4 * word + 4]
                    .copy_from_slice(&before[4 * MIX_WORDS[word]..4 * MIX_WORDS[word] + 4]);
            }
        }
    }
    // Truncate to 32 bytes and XOR the same input bytes back in (feed-forward).
    core::array::from_fn(|byte| state[DIGEST_BYTES[byte]] ^ input[DIGEST_BYTES[byte]])
}

/// DSig's 144-bit Haraka output, exactly the prefix of [`haraka512`].
///
/// 18 bytes, the size of one DSig chain node (`ELEMENT_BYTES` in the crate
/// root).
#[must_use]
pub fn mid_haraka512(input: &[u8; 64]) -> [u8; 18] {
    // The full 32-byte digest; keep bytes 0..18.
    let digest = haraka512(input);
    core::array::from_fn(|byte| digest[byte])
}

/// Builds DSig's 64-byte chain block with wrapping low-32-bit nonce salt.
///
/// Layout: nonce (16 bytes), secret (18 bytes), zero padding (30 bytes).
/// Only nonce bytes 0..4 change: their little-endian value adds
/// `index + 68 * depth` modulo 2^32. There is no carry into nonce byte four.
/// This low-level oracle deliberately accepts all indices/depths; the WOTS
/// protocol must separately enforce `index < 68` and its legal chain depth.
///
/// ```text
///  byte  0    4          16              34                          64
///        ├────┼──────────┼───────────────┼───────────────────────────┤
///        │salt│ nonce    │ secret (node) │  30 zero bytes            │
///        │ed  │ 4..16    │ 18 bytes      │                           │
///        └────┴──────────┴───────────────┴───────────────────────────┘
///   bytes 0..4 = (nonce[0..4] as u32 LE) + index + 68·depth   (mod 2^32)
/// ```
///
/// Why salt at all: adding `index + 68·depth` makes every (chain, step) pair
/// hash a block with a different prefix, so no two hash calls in one key
/// share an input shape. `index` is the chain number (0..68) and `depth` is
/// how far along the chain the step is.
#[must_use]
#[allow(clippy::cast_possible_truncation)] // Protocol explicitly reduces modulo 2^32.
pub fn dsig_chain_block(
    secret: &[u8; 18],
    nonce: &[u8; 16],
    index: usize,
    depth: usize,
) -> [u8; 64] {
    // Start from all zeros; bytes 34..64 stay zero.
    let mut block = [0; 64];
    block[..16].copy_from_slice(nonce);
    // The nonce's low 32 bits, little-endian.
    let low = u32::from_le_bytes([nonce[0], nonce[1], nonce[2], nonce[3]]);
    // The salt for this chain and depth, wrapping at 2^32.
    let offset = (index as u32).wrapping_add(68_u32.wrapping_mul(depth as u32));
    // Overwrite bytes 0..4 with the salted value (no carry into byte 4).
    block[..4].copy_from_slice(&low.wrapping_add(offset).to_le_bytes());
    block[16..34].copy_from_slice(secret);
    block
}

/// Applies one DSig chain step at the given index and depth (Haraka-512 in
/// the V1 profile; see [`chain_lane`](crate::chain_lane)).
///
/// Input: the current 18-byte node, the 16-byte key nonce, the chain index
/// and the depth. Output: the next 18-byte node. This is the oracle that
/// `tests/chain_step.rs` compares the hardware lane against.
#[must_use]
pub fn dsig_chain(secret: &[u8; 18], nonce: &[u8; 16], index: usize, depth: usize) -> [u8; 18] {
    // The compiled-in chain hash (Haraka-512 unless a `chain-*` feature says otherwise).
    crate::chain_lane::software_chain(&dsig_chain_block(secret, nonce, index, depth))
}

/// RHDL byte packing for the exact [`dsig_chain_block`] salt domain.
///
/// The index and depth are reduced modulo 2^32 by their input types; the
/// caller must enforce WOTS protocol bounds independently. This function only
/// packs the block and does not perform a hash or allocate a one-time key.
///
/// Inputs: the 18-byte node, the 16-byte nonce, and 32-bit `index` and
/// `depth`. Output: the 64-byte chain block. Combinational: one 32-bit
/// add-and-multiply-by-constant, the rest is wiring.
#[allow(clippy::needless_range_loop)] // Static byte wiring.
#[kernel]
pub fn dsig_chain_block_kernel(
    secret: [b8; 18],
    nonce: [b8; 16],
    index: b32,
    depth: b32,
) -> [b8; 64] {
    // Zero block; bytes 34..64 are never written and stay zero.
    let mut block = [b8(0); 64];
    for byte in 0..16 {
        block[byte] = nonce[byte];
    }
    // Assemble the nonce's low 32 bits, little-endian (byte 0 is least
    // significant). RHDL needs explicit widening before each shift.
    let low: b32 = nonce[0].resize()
        | (nonce[1].resize::<32>() << 8)
        | (nonce[2].resize::<32>() << 16)
        | (nonce[3].resize::<32>() << 24);
    // Add the salt. `b32` arithmetic wraps at 2^32, matching the reference.
    let salted = low + index + b32(68) * depth;
    // Split the salted word back into bytes 0..4.
    block[0] = salted.resize();
    block[1] = (salted >> 8).resize();
    block[2] = (salted >> 16).resize();
    block[3] = (salted >> 24).resize();
    // The 18-byte node goes into bytes 16..34.
    for byte in 0..18 {
        block[16 + byte] = secret[byte];
    }
    block
}

/// Exhaustive combinational AES S-box for synthesis as fixed Boolean logic.
///
/// Input and output are one byte. A 256-way `match` is the most direct way
/// to tell RHDL "this is a fixed table"; synthesis turns it into LUT logic.
/// The chain-step lane uses ROMs from [`crate::aes_table`] instead, so the
/// memory type can be chosen per S-box; this kernel serves the general
/// [`Haraka512Pipeline`] and the small equivalence kernels.
#[allow(clippy::too_many_lines)] // The complete permutation has 256 cases.
#[kernel]
pub fn aes_sbox_kernel(value: b8) -> b8 {
    // One arm per input byte, values from the AES standard (FIPS-197).
    match value {
        Bits::<8>(0x00) => b8(0x63),
        Bits::<8>(0x01) => b8(0x7c),
        Bits::<8>(0x02) => b8(0x77),
        Bits::<8>(0x03) => b8(0x7b),
        Bits::<8>(0x04) => b8(0xf2),
        Bits::<8>(0x05) => b8(0x6b),
        Bits::<8>(0x06) => b8(0x6f),
        Bits::<8>(0x07) => b8(0xc5),
        Bits::<8>(0x08) => b8(0x30),
        Bits::<8>(0x09) => b8(0x01),
        Bits::<8>(0x0a) => b8(0x67),
        Bits::<8>(0x0b) => b8(0x2b),
        Bits::<8>(0x0c) => b8(0xfe),
        Bits::<8>(0x0d) => b8(0xd7),
        Bits::<8>(0x0e) => b8(0xab),
        Bits::<8>(0x0f) => b8(0x76),
        Bits::<8>(0x10) => b8(0xca),
        Bits::<8>(0x11) => b8(0x82),
        Bits::<8>(0x12) => b8(0xc9),
        Bits::<8>(0x13) => b8(0x7d),
        Bits::<8>(0x14) => b8(0xfa),
        Bits::<8>(0x15) => b8(0x59),
        Bits::<8>(0x16) => b8(0x47),
        Bits::<8>(0x17) => b8(0xf0),
        Bits::<8>(0x18) => b8(0xad),
        Bits::<8>(0x19) => b8(0xd4),
        Bits::<8>(0x1a) => b8(0xa2),
        Bits::<8>(0x1b) => b8(0xaf),
        Bits::<8>(0x1c) => b8(0x9c),
        Bits::<8>(0x1d) => b8(0xa4),
        Bits::<8>(0x1e) => b8(0x72),
        Bits::<8>(0x1f) => b8(0xc0),
        Bits::<8>(0x20) => b8(0xb7),
        Bits::<8>(0x21) => b8(0xfd),
        Bits::<8>(0x22) => b8(0x93),
        Bits::<8>(0x23) => b8(0x26),
        Bits::<8>(0x24) => b8(0x36),
        Bits::<8>(0x25) => b8(0x3f),
        Bits::<8>(0x26) => b8(0xf7),
        Bits::<8>(0x27) => b8(0xcc),
        Bits::<8>(0x28) => b8(0x34),
        Bits::<8>(0x29) => b8(0xa5),
        Bits::<8>(0x2a) => b8(0xe5),
        Bits::<8>(0x2b) => b8(0xf1),
        Bits::<8>(0x2c) => b8(0x71),
        Bits::<8>(0x2d) => b8(0xd8),
        Bits::<8>(0x2e) => b8(0x31),
        Bits::<8>(0x2f) => b8(0x15),
        Bits::<8>(0x30) => b8(0x04),
        Bits::<8>(0x31) => b8(0xc7),
        Bits::<8>(0x32) => b8(0x23),
        Bits::<8>(0x33) => b8(0xc3),
        Bits::<8>(0x34) => b8(0x18),
        Bits::<8>(0x35) => b8(0x96),
        Bits::<8>(0x36) => b8(0x05),
        Bits::<8>(0x37) => b8(0x9a),
        Bits::<8>(0x38) => b8(0x07),
        Bits::<8>(0x39) => b8(0x12),
        Bits::<8>(0x3a) => b8(0x80),
        Bits::<8>(0x3b) => b8(0xe2),
        Bits::<8>(0x3c) => b8(0xeb),
        Bits::<8>(0x3d) => b8(0x27),
        Bits::<8>(0x3e) => b8(0xb2),
        Bits::<8>(0x3f) => b8(0x75),
        Bits::<8>(0x40) => b8(0x09),
        Bits::<8>(0x41) => b8(0x83),
        Bits::<8>(0x42) => b8(0x2c),
        Bits::<8>(0x43) => b8(0x1a),
        Bits::<8>(0x44) => b8(0x1b),
        Bits::<8>(0x45) => b8(0x6e),
        Bits::<8>(0x46) => b8(0x5a),
        Bits::<8>(0x47) => b8(0xa0),
        Bits::<8>(0x48) => b8(0x52),
        Bits::<8>(0x49) => b8(0x3b),
        Bits::<8>(0x4a) => b8(0xd6),
        Bits::<8>(0x4b) => b8(0xb3),
        Bits::<8>(0x4c) => b8(0x29),
        Bits::<8>(0x4d) => b8(0xe3),
        Bits::<8>(0x4e) => b8(0x2f),
        Bits::<8>(0x4f) => b8(0x84),
        Bits::<8>(0x50) => b8(0x53),
        Bits::<8>(0x51) => b8(0xd1),
        Bits::<8>(0x52) => b8(0x00),
        Bits::<8>(0x53) => b8(0xed),
        Bits::<8>(0x54) => b8(0x20),
        Bits::<8>(0x55) => b8(0xfc),
        Bits::<8>(0x56) => b8(0xb1),
        Bits::<8>(0x57) => b8(0x5b),
        Bits::<8>(0x58) => b8(0x6a),
        Bits::<8>(0x59) => b8(0xcb),
        Bits::<8>(0x5a) => b8(0xbe),
        Bits::<8>(0x5b) => b8(0x39),
        Bits::<8>(0x5c) => b8(0x4a),
        Bits::<8>(0x5d) => b8(0x4c),
        Bits::<8>(0x5e) => b8(0x58),
        Bits::<8>(0x5f) => b8(0xcf),
        Bits::<8>(0x60) => b8(0xd0),
        Bits::<8>(0x61) => b8(0xef),
        Bits::<8>(0x62) => b8(0xaa),
        Bits::<8>(0x63) => b8(0xfb),
        Bits::<8>(0x64) => b8(0x43),
        Bits::<8>(0x65) => b8(0x4d),
        Bits::<8>(0x66) => b8(0x33),
        Bits::<8>(0x67) => b8(0x85),
        Bits::<8>(0x68) => b8(0x45),
        Bits::<8>(0x69) => b8(0xf9),
        Bits::<8>(0x6a) => b8(0x02),
        Bits::<8>(0x6b) => b8(0x7f),
        Bits::<8>(0x6c) => b8(0x50),
        Bits::<8>(0x6d) => b8(0x3c),
        Bits::<8>(0x6e) => b8(0x9f),
        Bits::<8>(0x6f) => b8(0xa8),
        Bits::<8>(0x70) => b8(0x51),
        Bits::<8>(0x71) => b8(0xa3),
        Bits::<8>(0x72) => b8(0x40),
        Bits::<8>(0x73) => b8(0x8f),
        Bits::<8>(0x74) => b8(0x92),
        Bits::<8>(0x75) => b8(0x9d),
        Bits::<8>(0x76) => b8(0x38),
        Bits::<8>(0x77) => b8(0xf5),
        Bits::<8>(0x78) => b8(0xbc),
        Bits::<8>(0x79) => b8(0xb6),
        Bits::<8>(0x7a) => b8(0xda),
        Bits::<8>(0x7b) => b8(0x21),
        Bits::<8>(0x7c) => b8(0x10),
        Bits::<8>(0x7d) => b8(0xff),
        Bits::<8>(0x7e) => b8(0xf3),
        Bits::<8>(0x7f) => b8(0xd2),
        Bits::<8>(0x80) => b8(0xcd),
        Bits::<8>(0x81) => b8(0x0c),
        Bits::<8>(0x82) => b8(0x13),
        Bits::<8>(0x83) => b8(0xec),
        Bits::<8>(0x84) => b8(0x5f),
        Bits::<8>(0x85) => b8(0x97),
        Bits::<8>(0x86) => b8(0x44),
        Bits::<8>(0x87) => b8(0x17),
        Bits::<8>(0x88) => b8(0xc4),
        Bits::<8>(0x89) => b8(0xa7),
        Bits::<8>(0x8a) => b8(0x7e),
        Bits::<8>(0x8b) => b8(0x3d),
        Bits::<8>(0x8c) => b8(0x64),
        Bits::<8>(0x8d) => b8(0x5d),
        Bits::<8>(0x8e) => b8(0x19),
        Bits::<8>(0x8f) => b8(0x73),
        Bits::<8>(0x90) => b8(0x60),
        Bits::<8>(0x91) => b8(0x81),
        Bits::<8>(0x92) => b8(0x4f),
        Bits::<8>(0x93) => b8(0xdc),
        Bits::<8>(0x94) => b8(0x22),
        Bits::<8>(0x95) => b8(0x2a),
        Bits::<8>(0x96) => b8(0x90),
        Bits::<8>(0x97) => b8(0x88),
        Bits::<8>(0x98) => b8(0x46),
        Bits::<8>(0x99) => b8(0xee),
        Bits::<8>(0x9a) => b8(0xb8),
        Bits::<8>(0x9b) => b8(0x14),
        Bits::<8>(0x9c) => b8(0xde),
        Bits::<8>(0x9d) => b8(0x5e),
        Bits::<8>(0x9e) => b8(0x0b),
        Bits::<8>(0x9f) => b8(0xdb),
        Bits::<8>(0xa0) => b8(0xe0),
        Bits::<8>(0xa1) => b8(0x32),
        Bits::<8>(0xa2) => b8(0x3a),
        Bits::<8>(0xa3) => b8(0x0a),
        Bits::<8>(0xa4) => b8(0x49),
        Bits::<8>(0xa5) => b8(0x06),
        Bits::<8>(0xa6) => b8(0x24),
        Bits::<8>(0xa7) => b8(0x5c),
        Bits::<8>(0xa8) => b8(0xc2),
        Bits::<8>(0xa9) => b8(0xd3),
        Bits::<8>(0xaa) => b8(0xac),
        Bits::<8>(0xab) => b8(0x62),
        Bits::<8>(0xac) => b8(0x91),
        Bits::<8>(0xad) => b8(0x95),
        Bits::<8>(0xae) => b8(0xe4),
        Bits::<8>(0xaf) => b8(0x79),
        Bits::<8>(0xb0) => b8(0xe7),
        Bits::<8>(0xb1) => b8(0xc8),
        Bits::<8>(0xb2) => b8(0x37),
        Bits::<8>(0xb3) => b8(0x6d),
        Bits::<8>(0xb4) => b8(0x8d),
        Bits::<8>(0xb5) => b8(0xd5),
        Bits::<8>(0xb6) => b8(0x4e),
        Bits::<8>(0xb7) => b8(0xa9),
        Bits::<8>(0xb8) => b8(0x6c),
        Bits::<8>(0xb9) => b8(0x56),
        Bits::<8>(0xba) => b8(0xf4),
        Bits::<8>(0xbb) => b8(0xea),
        Bits::<8>(0xbc) => b8(0x65),
        Bits::<8>(0xbd) => b8(0x7a),
        Bits::<8>(0xbe) => b8(0xae),
        Bits::<8>(0xbf) => b8(0x08),
        Bits::<8>(0xc0) => b8(0xba),
        Bits::<8>(0xc1) => b8(0x78),
        Bits::<8>(0xc2) => b8(0x25),
        Bits::<8>(0xc3) => b8(0x2e),
        Bits::<8>(0xc4) => b8(0x1c),
        Bits::<8>(0xc5) => b8(0xa6),
        Bits::<8>(0xc6) => b8(0xb4),
        Bits::<8>(0xc7) => b8(0xc6),
        Bits::<8>(0xc8) => b8(0xe8),
        Bits::<8>(0xc9) => b8(0xdd),
        Bits::<8>(0xca) => b8(0x74),
        Bits::<8>(0xcb) => b8(0x1f),
        Bits::<8>(0xcc) => b8(0x4b),
        Bits::<8>(0xcd) => b8(0xbd),
        Bits::<8>(0xce) => b8(0x8b),
        Bits::<8>(0xcf) => b8(0x8a),
        Bits::<8>(0xd0) => b8(0x70),
        Bits::<8>(0xd1) => b8(0x3e),
        Bits::<8>(0xd2) => b8(0xb5),
        Bits::<8>(0xd3) => b8(0x66),
        Bits::<8>(0xd4) => b8(0x48),
        Bits::<8>(0xd5) => b8(0x03),
        Bits::<8>(0xd6) => b8(0xf6),
        Bits::<8>(0xd7) => b8(0x0e),
        Bits::<8>(0xd8) => b8(0x61),
        Bits::<8>(0xd9) => b8(0x35),
        Bits::<8>(0xda) => b8(0x57),
        Bits::<8>(0xdb) => b8(0xb9),
        Bits::<8>(0xdc) => b8(0x86),
        Bits::<8>(0xdd) => b8(0xc1),
        Bits::<8>(0xde) => b8(0x1d),
        Bits::<8>(0xdf) => b8(0x9e),
        Bits::<8>(0xe0) => b8(0xe1),
        Bits::<8>(0xe1) => b8(0xf8),
        Bits::<8>(0xe2) => b8(0x98),
        Bits::<8>(0xe3) => b8(0x11),
        Bits::<8>(0xe4) => b8(0x69),
        Bits::<8>(0xe5) => b8(0xd9),
        Bits::<8>(0xe6) => b8(0x8e),
        Bits::<8>(0xe7) => b8(0x94),
        Bits::<8>(0xe8) => b8(0x9b),
        Bits::<8>(0xe9) => b8(0x1e),
        Bits::<8>(0xea) => b8(0x87),
        Bits::<8>(0xeb) => b8(0xe9),
        Bits::<8>(0xec) => b8(0xce),
        Bits::<8>(0xed) => b8(0x55),
        Bits::<8>(0xee) => b8(0x28),
        Bits::<8>(0xef) => b8(0xdf),
        Bits::<8>(0xf0) => b8(0x8c),
        Bits::<8>(0xf1) => b8(0xa1),
        Bits::<8>(0xf2) => b8(0x89),
        Bits::<8>(0xf3) => b8(0x0d),
        Bits::<8>(0xf4) => b8(0xbf),
        Bits::<8>(0xf5) => b8(0xe6),
        Bits::<8>(0xf6) => b8(0x42),
        Bits::<8>(0xf7) => b8(0x68),
        Bits::<8>(0xf8) => b8(0x41),
        Bits::<8>(0xf9) => b8(0x99),
        Bits::<8>(0xfa) => b8(0x2d),
        Bits::<8>(0xfb) => b8(0x0f),
        Bits::<8>(0xfc) => b8(0xb0),
        Bits::<8>(0xfd) => b8(0x54),
        Bits::<8>(0xfe) => b8(0xbb),
        // The only remaining input is 0xff, whose S-box value is 0x16.
        _ => b8(0x16),
    }
}

/// AES field multiplication by x, reduced modulo x^8 + x^4 + x^3 + x + 1.
///
/// "Multiply by x" is "multiply by 2" in GF(2^8): shift left one bit, and if
/// bit 7 was set, XOR in `0x1b`. In hardware that is wiring plus three XOR
/// gates, which is why MixColumns is cheap compared with an S-box.
#[kernel]
pub fn aes_xtime_kernel(value: b8) -> b8 {
    // Shift left, then conditionally reduce by the AES polynomial.
    (value << 1)
        ^ if (value & b8(0x80)) != b8(0) {
            b8(0x1b)
        } else {
            b8(0)
        }
}

/// First half of an AESENC round, with conventional AES column-major bytes.
///
/// SubBytes then ShiftRows on one 16-byte AES lane: 16 S-boxes, and wiring.
/// Combinational; the pipeline puts a register after it.
#[allow(clippy::needless_range_loop)] // Static loops describe replicated hardware.
#[kernel]
pub fn aes_sub_shift_kernel(state: [b8; 16]) -> [b8; 16] {
    // Every byte is overwritten below; starting from `state` just gives the
    // array a value.
    let mut result = state;
    for column in 0..4 {
        for row in 0..4 {
            // ShiftRows source: same row, column (column + row) mod 4.
            result[4 * column + row] = aes_sbox_kernel(state[4 * ((column + row) & 3) + row]);
        }
    }
    result
}

/// Second half of AESENC: MixColumns followed by AddRoundKey.
///
/// Uses the XOR-saving form of MixColumns:
/// `out[r] = s[r] ⊕ t ⊕ 2·(s[r] ⊕ s[r+1]) ⊕ key[r]`, with `t` the XOR of
/// the whole column. It equals the textbook `2·s[r] ⊕ 3·s[r+1] ⊕ s[r+2] ⊕
/// s[r+3]` but shares `t` across the four rows. Combinational, XOR only.
#[allow(clippy::needless_range_loop)] // Static loops describe replicated hardware.
#[kernel]
pub fn aes_mix_key_kernel(state: [b8; 16], key: [b8; 16]) -> [b8; 16] {
    // Placeholder value; every byte is overwritten.
    let mut result = state;
    for column in 0..4 {
        // Index of the column's first byte.
        let base = 4 * column;
        // XOR of all four bytes of the column, shared by its four outputs.
        let total = state[base] ^ state[base + 1] ^ state[base + 2] ^ state[base + 3];
        for row in 0..4 {
            result[base + row] = state[base + row]
                ^ total
                ^ aes_xtime_kernel(state[base + row] ^ state[base + ((row + 1) & 3)])
                ^ key[base + row];
        }
    }
    result
}

/// One complete AESENC round for small-stage lowering/equivalence gates.
///
/// Both halves back to back, with no register between them. Not used in the
/// pipelines; it exists so a single AES round can be lowered and checked
/// against [`aes_round`] cheaply.
#[kernel]
pub fn aes_round_kernel(state: [b8; 16], key: [b8; 16]) -> [b8; 16] {
    aes_mix_key_kernel(aes_sub_shift_kernel(state), key)
}

/// Pure wiring of the sixteen 32-bit Haraka lane words.
///
/// Applies [`MIX_WORDS`] to a 64-byte state. No gates: synthesis just
/// reconnects wires.
#[allow(clippy::needless_range_loop)] // Compile-time permutation.
#[kernel]
pub fn mix512_kernel(state: [b8; 64]) -> [b8; 64] {
    // Placeholder value; every byte is overwritten.
    let mut result = state;
    for word in 0..16 {
        for byte in 0..4 {
            // Output word `word` takes the bytes of input word MIX_WORDS[word].
            result[4 * word + byte] = state[4 * MIX_WORDS[word] + byte];
        }
    }
    result
}

/// Parallel nonlinear half-round for all four AES lanes.
///
/// `SubBytes ∘ ShiftRows` on the full 64-byte state: 64 S-boxes. This is
/// the function every [`SboxStage`](crate::aes_table::SboxStage) computes,
/// and the fast simulation model of that stage calls it directly.
#[allow(clippy::needless_range_loop)] // Static four-lane replication.
#[kernel]
pub fn sub_shift512_kernel(state: [b8; 64]) -> [b8; 64] {
    // Placeholder value; every byte is overwritten.
    let mut result = state;
    for lane in 0..4 {
        // Cut out AES lane `lane` (16 bytes).
        let mut block = [b8(0); 16];
        for byte in 0..16 {
            block[byte] = state[16 * lane + byte];
        }
        // Run the half-round on it and paste it back.
        let next = aes_sub_shift_kernel(block);
        for byte in 0..16 {
            result[16 * lane + byte] = next[byte];
        }
    }
    result
}

/// Parallel linear half-round with optional post-round inter-lane wiring.
///
/// `MixColumns ∘ AddRoundKey` on all four AES lanes, using `keys` (a row of
/// [`ROUND_KEYS_RHDL`]). When `mix` is true, the [`MIX_WORDS`] shuffle
/// follows. In the pipelines `mix` is always a compile-time constant, so the
/// final `if` costs nothing.
#[allow(clippy::needless_range_loop)] // Static four-lane replication.
#[kernel]
pub fn mix_key512_kernel(state: [b8; 64], keys: [b8; 64], mix: bool) -> [b8; 64] {
    // Placeholder value; every byte is overwritten.
    let mut result = state;
    for lane in 0..4 {
        // This AES lane's 16 state bytes and its 16 key bytes.
        let mut block = [b8(0); 16];
        let mut key = [b8(0); 16];
        for byte in 0..16 {
            block[byte] = state[16 * lane + byte];
            key[byte] = keys[16 * lane + byte];
        }
        let next = aes_mix_key_kernel(block, key);
        for byte in 0..16 {
            result[16 * lane + byte] = next[byte];
        }
    }
    // After every second AES round, mix words across lanes.
    if mix { mix512_kernel(result) } else { result }
}

/// One input token; every asserted valid bit is accepted on the next edge.
///
/// A *token* is one unit of work moving down the pipeline: here, one block
/// to hash plus its bookkeeping. The pipeline has no "ready" signal (no
/// backpressure), so the sender must never offer more than one per clock.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct HarakaInput {
    /// Exactly 64 input bytes in conventional memory order.
    pub block: [b8; 64],
    /// Opaque caller context identifier, preserved without modification.
    pub tag: b32,
    /// This token owns a result; false inserts a bubble.
    pub valid: bool,
}

/// The idle input: an all-zero block with `valid` false (a bubble).
impl Default for HarakaInput {
    /// A bubble: nothing is launched.
    fn default() -> Self {
        Self {
            block: [b8(0); 64],
            tag: b32(0),
            valid: false,
        }
    }
}

/// Final digest token; all fields except `valid` are unspecified when invalid.
///
/// Appears exactly [`HARAKA_PIPELINE_LATENCY`] clocks after its input.
#[derive(Clone, Copy, Debug, PartialEq, Digital)]
pub struct HarakaOutput {
    /// Standard 32-byte Haraka v2 result; take bytes 0..18 for DSig chains.
    pub digest: [b8; 32],
    /// Caller tag aligned with the digest.
    pub tag: b32,
    /// A result is available and has not been invalidated by reset.
    pub valid: bool,
}

/// Prefix of the twenty-stage Haraka pipeline, for bounded validation.
///
/// `STAGES` must be between one and twenty inclusive. Only the full twenty
/// stages implement Haraka-512. Shorter instances expose the truncated,
/// feed-forwarded intermediate state for focused equivalence tests; they must
/// never be used as a cryptographic hash. Every specialization accepts one
/// token per cycle without backpressure and delays validity by `STAGES`
/// registered transfers. Reset flushes every in-flight token.
///
/// Four parallel shift registers, all `STAGES` deep, move in lock step:
///
/// ```text
///   input ─▶ [stage 0] ─▶ [stage 1] ─▶ ... ─▶ [stage STAGES-1] ─▶ digest
///   state:    sub/shift    mix/key             (computed)
///   feed_forward, tag, valid: copied unchanged from stage to stage
/// ```
///
/// Only `valid` has a reset value. The data registers do not, which keeps
/// them cheap (no reset fan-out) and is safe because nothing reads data
/// whose `valid` is false.
#[derive(Clone, Debug, Synchronous, SynchronousDQ)]
pub struct HarakaPipeline<const STAGES: usize> {
    /// The 64-byte working state of each stage, after that stage's half-round.
    state: [DFF<[b8; 64]>; STAGES],
    /// The 32 input bytes the final XOR needs, carried alongside the state so
    /// they arrive at the end at the same time as their token.
    feed_forward: [DFF<[b8; 32]>; STAGES],
    /// The caller's 32-bit tag, delayed to match.
    tag: [DFF<b32>; STAGES],
    /// "This stage holds a real token." The only registers that reset.
    valid: [DFF<bool>; STAGES],
}

/// Build a pipeline of `STAGES` stages, checking that `STAGES` is legal.
impl<const STAGES: usize> Default for HarakaPipeline<STAGES> {
    /// Data registers start as "don't care"; every valid bit starts false.
    ///
    /// # Panics
    /// Panics if `STAGES` is outside `1..=20`.
    fn default() -> Self {
        assert!((1..=HARAKA_PIPELINE_LATENCY).contains(&STAGES));
        Self {
            state: core::array::from_fn(|_| DFF::resetless()),
            feed_forward: core::array::from_fn(|_| DFF::resetless()),
            tag: core::array::from_fn(|_| DFF::resetless()),
            valid: core::array::from_fn(|_| DFF::new(false)),
        }
    }
}

/// Ports of [`HarakaPipeline`]: one block in, one digest out per clock.
impl<const STAGES: usize> SynchronousIO for HarakaPipeline<STAGES> {
    /// A block, a tag and a valid bit.
    type I = HarakaInput;
    /// A digest, the same tag and a valid bit, `STAGES` clocks later.
    type O = HarakaOutput;
    /// The stage wiring below.
    type Kernel = haraka_pipeline_kernel<STAGES>;
}

/// Complete Haraka-512 v2 pipeline with twenty-cycle registered latency.
pub type Haraka512Pipeline = HarakaPipeline<HARAKA_PIPELINE_LATENCY>;

/// Connectivity of the resetless datapath and resettable ownership pipeline.
///
/// Inputs: `clock_reset`, the new token `input`, and `q` (every register's
/// current value). Outputs: the token leaving the last stage, and `d` (every
/// register's next value). Timing: a token entering at clock `t` leaves at
/// clock `t + STAGES`. Stage 0 computes on `input` directly; stage `s > 0`
/// computes on the register of stage `s - 1`.
#[allow(clippy::needless_range_loop)] // Compile-time pipeline elaboration.
#[kernel]
pub fn haraka_pipeline_kernel<const STAGES: usize>(
    clock_reset: ClockReset,
    input: HarakaInput,
    q: HarakaPipelineQ<STAGES>,
) -> (HarakaOutput, HarakaPipelineD<STAGES>) {
    // True while reset is asserted; used to hide the output during reset.
    let resetting = clock_reset.reset.any();
    // Start with "every register keeps its value"; each stage is set below.
    let mut d = HarakaPipelineD::<STAGES> {
        state: q.state,
        feed_forward: q.feed_forward,
        tag: q.tag,
        valid: q.valid,
    };
    // Stage 0: round 0's SubBytes/ShiftRows on the incoming block.
    d.state[0] = sub_shift512_kernel(input.block);
    // Capture the 32 input bytes the final feed-forward XOR will need.
    for byte in 0..32 {
        d.feed_forward[0][byte] = input.block[DIGEST_BYTES[byte]];
    }
    d.tag[0] = input.tag;
    d.valid[0] = input.valid;
    // Stages 1..STAGES: alternate the two half-rounds.
    for stage in 1..STAGES {
        if stage & 1 == 0 {
            // Even stage: SubBytes/ShiftRows of AES round stage/2.
            d.state[stage] = sub_shift512_kernel(q.state[stage - 1]);
        } else {
            // Odd stage: MixColumns + key of AES round stage/2 (`>> 1`),
            // plus the word mix after every second AES round (stage 3, 7, ...).
            d.state[stage] = mix_key512_kernel(
                q.state[stage - 1],
                ROUND_KEYS_RHDL[stage >> 1],
                stage & 3 == 3,
            );
        }
        // Side-band data just moves one stage along.
        d.feed_forward[stage] = q.feed_forward[stage - 1];
        d.tag[stage] = q.tag[stage - 1];
        d.valid[stage] = q.valid[stage - 1];
    }
    // Output: truncate the last stage's state and XOR in the saved input bytes.
    let mut digest = [b8(0); 32];
    for byte in 0..32 {
        digest[byte] = q.state[STAGES - 1][DIGEST_BYTES[byte]] ^ q.feed_forward[STAGES - 1][byte];
    }
    // `valid` is forced low during reset so no stale token escapes.
    (
        HarakaOutput {
            digest,
            tag: q.tag[STAGES - 1],
            valid: q.valid[STAGES - 1] && !resetting,
        },
        d,
    )
}

/// Generates the complete twenty-stage pipeline's RHDL-authored Verilog.
///
/// # Errors
/// Returns an RHDL lowering/rendering/syntax-check error. Successful rendering
/// alone is not synthesis, timing closure, equivalence, or hardware evidence.
///
/// This is how the RHDL source becomes Verilog for Vivado: RHDL "lowers" the
/// kernels to its intermediate form and prints Verilog modules.
pub fn haraka512_verilog() -> Result<String, RHDLError> {
    haraka_pipeline_verilog::<HARAKA_PIPELINE_LATENCY>()
}

/// Generates a registered prefix for focused lowering and RTL tests.
///
/// # Panics
/// Panics if `STAGES` is outside 1..=20.
///
/// # Errors
/// Returns any RHDL lowering/rendering/syntax-check error.
pub fn haraka_pipeline_verilog<const STAGES: usize>() -> Result<String, RHDLError> {
    // Build the circuit, describe it under a fixed top-level module name,
    // lower it to HDL and print the modules.
    Ok(HarakaPipeline::<STAGES>::default()
        .descriptor("haraka512_pipeline".into())?
        .hdl()?
        .modules
        .pretty())
}