Module sustained 

The full design walk-through: budgets, slices, caches and decisions.

The sustained engine: as many signatures per second as 420,000 LUTs allow

Status (October 2026): running on the card. The 4-slice engine (build v13) signs 12,038,905 signatures per second on a verified 1,572,864-signature burst and routes at 205 MHz in 388,542 kernel LUT. This page is the design walk-through written while the engine was built (2026-10-14/15); its budgets and estimates are kept as they were, and sections that were later measured say so. The current measured numbers and the builds that produced them are in How the speed adds up and What is proven.

The finite capture (the DSig repository’s u280_capture module) proves the signing front end on 16 keys that the host prepared in advance. A sustained signer must also make every key itself, certify every batch with Ed25519 and write every complete signature to memory, all at the same rate, indefinitely.

The rules this engine is built under (set by the project owner):

Rule	Value
Kernel clock	205 MHz
LUT budget	420,000 Vivado “CLB LUTs” for the kernel, a hard cap (raised from 350,000 on 2026-10-14 so DSig and Ed25519 share one budget for the comparison)
Language	RHDL only: no imported Verilog, no black boxes, no hand-written HDL. Generated Verilog is an output for synthesis, never an input.
Evidence	Only measured on-card throughput counts

1. Start from the arithmetic

Every number below comes from the frozen profile constants in the crate root (CHAINS = 68, POSITIONS = 4, BATCH_KEYS = 128) and the reference. Per signature:

Work per signature	Amount
Haraka chain steps	68 chains × 3 steps = 204
AES S-box evaluations (folded, see chain_step)	204 × 494 = 100,776
BLAKE3 compressions	≈ 47
Ed25519 signatures	1 per 128
Complete signature bytes	1,642

A slice (section 2) turns out one key every 68 cycles, so at 205 MHz it signs at most 205 M / 68 ≈ 3.01 M signatures/s. The engine’s ceiling is that times the number of slices.

What a slice costs, measured by Vivado 2023.2 out-of-context place-and-route on the U280 part (2026-10-14; module estimates, not whole-design acceptance):

Block	CLB LUTs	Registers	Other	Max clock alone
One Haraka chain-step lane (494 LUT S-boxes)	23,902	7,995		~570 MHz
One BLAKE3 lane	16,422	18,716		~325 MHz
One whole slice (3 lanes + BLAKE3 + control)	116,989	59,091	72 URAM, 38 BRAM	see §4b

So the S-boxes dominate: about 48 LUTs per S-box once MixColumns and pipeline logic are included. With 420,000 LUTs, two slices fit as they are; four slices need each slice about 30% smaller, which is what the area knobs in section 4 are for.

2. One slice is one complete signer

The key observation: a batch has 128 keys, and 128 = 2 × 64 = 4 × 32 = 8 × 16. Split the engine into identical slices. Each slice makes its share of every batch, keeps those keys in its own cache, signs with them, and writes its signatures through its own HBM port. Slices share only the small batch-level work: the top Merkle levels and the Ed25519 certificate.

How many slices to build is chosen at compile time with the DSIG_SLICES build variable (see SLICES):

DSIG_SLICES	Slices	Keys per slice per batch	Subtree levels	Top levels	Ceiling @ 205 MHz
2	2	64	6	1	6.0 M/s
4	4	32	5	2	12.1 M/s
8 (default)	8	16	4	3	24.1 M/s

DSIG_SLICES=4 cargo test   # build/test the 4-slice engine

The picture shows the 8-slice build:

                         ┌──────────────── one slice (×8) ────────────────┐
 master seed, epoch ───▶ │ seed/nonce/XOF ─▶ chain engine ─▶ key cache     │
                         │   (BLAKE3)         3 Haraka lanes   (URAM ring)  │
                         │        ▲               │   chain ends     │       │
                         │        │               ▼                  │ nodes │
                         │   BLAKE3 lane ◀─ endpoint commitment       │       │
                         │   (shared by       + 16-leaf subtree       ▼       │
                         │    all phases)            │          signer ──────┼─▶ HBM port s
 messages (HBM) ───────▶ │                           │      (digits, select, │   (frames)
                         │                           │       frame)          │
                         └───────────────────────────┼───────────▲───────────┘
                                                     ▼           │ certificate,
                                     ┌── shared ───────────┐     │ top proof
                                     │ Merkle top (3 levels)├─────┘
                                     │ Ed25519 batch signer │
                                     └──────────────────────┘

Why this shape works:

• Rates balance inside a slice. A chain engine finishes one chain per cycle, so one key every 68 cycles. The signer reads one cached node per cycle, so it also uses one key every 68 cycles. Neither side waits for the other in steady state.
• Slices do not talk to each other. Each has its own memory and output port, so the eight can be placed in different regions of the chip (the U280 has three SLRs, separate silicon dies) without long wires between them.
• The shared part is tiny. With 8 slices one batch completes every 1,088 cycles (16 keys × 68 cycles), so the top of the Merkle tree needs 7 compressions per 1,088 cycles and Ed25519 needs one signature per 1,088 cycles: 188,000 per second at 205 MHz. The Ed25519 core measured ~229,000 per second at 205 MHz (section 4a). Fewer slices need proportionally fewer.

The ceiling is slices × f / 68: at 205 MHz, 6.0M, 12.1M or 24.1M signatures/s for 2, 4 or 8 slices. How close a build gets is measured (section 6), never assumed.

3. Follow one signature through a slice

1. Seed. The BLAKE3 lane derives the one-time seed from the master seed, signer identity, epoch, batch and key index (two compressions, reference::PreparedBatch). The seed then gives the two nonces (one compression each) and 20 XOF blocks holding the 68 chain secrets.
2. Chains. Each secret enters the chain engine, which runs three ChainStepLanes in series. Sixty cycles later, all four positions of that chain are written to the cache: one write per position bank per cycle.
3. Public key. Chain ends stream into the endpoint commitment, a 1,240-byte BLAKE3 hash (21 compressions). Its result pk_hash is the Merkle leaf.
4. Subtree. After 16 leaves, the slice hashes its 16-leaf subtree (15 compressions) and sends the subtree root to the shared Merkle top.
5. Certificate. The Merkle top combines eight subtree roots into the batch root. The Ed25519 core signs the 64-byte DSIGFP01 envelope (certificate). The certificate and the top three proof levels are broadcast to all eight slices. Only now are the batch’s keys eligible.
6. Sign. A message arrives. The slice burns its next eligible key, hashes the message to 68 base-4 digits (one compression), then reads one node per cycle from the cache bank that holds each digit’s position.
7. Frame. The signer emits the exact 1,642-byte wire signature (DsigSignature::to_bytes) padded to a 1,664-byte frame (26 × 64-byte beats). The padding carries the message index so the host can match frames to messages.

A key is burned at step 6 before any output. If the frame is lost, the key is still gone; if anything is uncertain, the whole namespace is retired. That is the one-time rule from the repository contributor rules (AGENTS.md).

4. Where the S-boxes live: area knobs

Each knob is a compile-time choice, so a smaller build is the same RTL with fewer copies, not a different design.

Knob	Values	Effect
SLICES (DSIG_SLICES)	2, 4, 8	Throughput and area scale together (section 2).
TABLE_MASK	bit per round 3–8 (in rounds 7 and 8, 16 of the 64 positions never reach the output; their ROMs have no loads and synthesis removes them)	Rounds computed by block-RAM T-table lookups instead of LUT S-boxes (TableChainStepLane).
sbox-bram (Cargo feature)	on / off	Every AES S-box ROM becomes one BRAM18 instead of ~32 LUTs, same latency (see sbox_placement, which also places S-boxes per lane with DSIG_SBOX_PLACEMENT). Needs block RAM left over once the caches are in UltraRAM.
BLAKE3 adders in DSPs	planned	Vivado USE_DSP on the BLAKE3 lane’s 32-bit adders, set in Tcl. No RTL change.

The budget at 420,000 LUTs

With the measured ~117k LUTs per slice (section 1):

Build	Slices	Engine extras (cert unit, Ed25519, HBM I/O)	Total	Fits?
slices-2, all-LUT S-boxes	~234k	~55k (Ed25519 estimated 29–37k)	~290k	yes
slices-4, all-LUT S-boxes	~468k	~55k	~520k	no: needs ~20% smaller slices (the sbox-bram knob alone may do it)

Levers for the 4-slice build, largest first:

1. T-tables in block RAM (TABLE_MASK). An S-box becomes a lookup in a 256 × 32 ROM holding T0, which also absorbs most of MixColumns. Each DualTableRom is one true dual-port BRAM18 serving two lookups, so a table round costs 32 BRAM18s (fewer in rounds 7 and 8). The budget is the kernel region’s block RAM, not the device’s: 3,552 BRAM18 sites, of which a slices-4 kernel’s caches and FIFOs use about 590. Build v4 learned both facts the hard way: with one single-port ROM per lookup (64 BRAM18 per round) it needed 4,033 sites, synthesis spilled the rest of the ROMs into 271k LUTs of distributed RAM, and placement refused the design. With dual-port tables, rounds 3–8 cost 176 BRAM18 per lane, 2,112 for 12 lanes (~76 % of the region with the caches).
2. BLAKE3 adders in DSPs: about 7k LUTs per lane, for ~224 DSPs each.
3. Packer as fixed wiring. Element positions are fixed by the frame layout, so a general byte shifter is unnecessary: a few k LUTs per slice.

4b. Timing: the key-cache read

The first Vivado run of a whole slice missed timing badly (−2.0 ns at 3.333 ns). The culprit was a read from the 32,768-row key cache: a cascade of seven UltraRAMs feeding the signer’s logic directly, 5.3 ns in total. Since then, CacheBank registers each RAM output READ_PIPE = 2 more times before its mux, where Vivado folds those registers into the UltraRAMs’ own output and cascade registers. The signer tracks reads with a small token line (NodeRead) of the same length, so each node arrives with its digit, frame slot and message index.

4a. Waiting for certificates: why each slice caches 512 keys

A key cannot be spent until its batch’s Ed25519 certificate exists, and the Ed25519 core is slow per request but fast in bulk. The numbers below were measured by the Ed25519 project on its DsigSignCore<1, 1, 1, 0>, in native simulation (2026-10-14, after its mixed-radix program change):

Quantity	Cycles
Key load (once per run)	21,509
One request alone, in → out	22,240
Throughput with 32 requests in flight	~895 per signature (~229k/s at 205 MHz)

With 8 slices a batch of 128 keys is finished every 1,088 cycles. The engine therefore needs about 23,000 / 1,088 ≈ 21 batches inside the core at once (fewer with fewer slices):

time ──▶   batch b    b+1   b+2  …  b+20        (one every ~1,088 cycles)
           ├─ request ──── 25–28k cycles in the Ed25519 core ──── result ─┤
cache:     keys of b … b+20 all wait in their slices until their result

Three pieces make that possible:

1. cert_unit issues a request per batch as soon as the Merkle top is done, tags it with the batch number, and keeps up to IN_FLIGHT = 32 outstanding (the core’s own credit limit). Results may come back in any order; distribution to the slices is strictly in order.
2. Each slice caches CACHE_SLOTS = 512 keys. They are packed densely at row slot × 68 + chain, 72 UltraRAMs per slice (Vivado-confirmed), built from two RHDL SyncBRAMs per bank (cache_bank). With 128-row slots eight slices would need 1,024 UltraRAMs, more than the U280’s 960.
3. A slice starts a key only while fewer than ADMIT_LIMIT = 512 − KEYS_PER_SLICE are unspent. The one batch of slack guarantees that the subtree ring never overwrites a node that a still-unspent key needs as a Merkle sibling.

At ~895 cycles per signature, one core supports ~229k batches/s at 205 MHz, i.e. ~29M signatures/s, above every slice count’s ceiling. The Ed25519 profile 2,1,1,0 would roughly halve that cost for about 470 DSPs if ever needed.

5. What the host does

The host never computes a signature during a measurement. It:

1. durably reserves a fresh (epoch, batch range) namespace (lease) and a fresh master seed;
2. writes N 16-byte messages into HBM;
3. loads the Ed25519 seed and namespace through the private control path;
4. starts the kernel and reads the tick counter before and after (the DSig repository’s u280_capture module explains the coherent tick snapshot);
5. reads every frame back and verifies every signature against the trusted Ed25519 public key with the reference verifier, checking that no key index repeats.

6. How a number becomes a result

The reported throughput is

            verified signatures
rate = ───────────────────────────────────────────────
       kernel cycles (start → last write acknowledged) / measured clock

The run starts with an empty cache. The denominator therefore includes cold start, all key generation, all certification and every HBM write. A finite run of N = 4 million signatures takes about 0.67 s at 6M/s, which spans more than 30,000 batches; that is the continuous figure. Any shorter burst, or any run that starts from a pre-filled cache, is reported separately and labeled as such.

The kernel also exports occupancy counters (signer idle because no eligible key, BLAKE3 lane busy, HBM write stalls). When the rate falls short, these counters show which floor in section 1 was hit.

7. Everything is RHDL

The whole kernel, Ed25519 core included, is written in Rust/RHDL and lowered to Verilog in one pass by dsig-engine-export. Nothing enters the build as Verilog: no imported cores, no black boxes, no hand-written HDL. That keeps every block readable, testable in native simulation and documented next to its code.

The few things that look like hardware primitives are RHDL’s own (rhdl_fpga components):

Need	RHDL part	Where
Registers	DFF	everywhere
RAM, FIFO buffers	SyncBRAM (registered read, read-before-write)	signer, subtree, factory, cert unit
ROM (S-box, T-table)	SyncBRAM::new(contents), never written	aes_table
Non-power-of-two RAM	two SyncBRAMs	cache_bank

Which physical memory Vivado uses (LUT, block RAM or UltraRAM) is a synthesis constraint, not part of the RTL. The exporter’s closure check proves the bundle contains only modules defined by this lowering.

The Ed25519 core. The Ed25519 project’s DsigSignCore<1, 1, 1, 0> (one fixed-base machine, one SHA-512 ring, one scalar ring; ports CachedSignIn / CachedSignOut) will be vendored as a sealed RHDL source snapshot and instantiated directly. Until that snapshot arrives, the vendored compact RHDL core stands in: same ports and handshake (ready = request accepted this cycle; valid = one-cycle pulse carrying signature and tag), slower.

8. Code map

Piece	Module	Verified by
Folded chain step (494 S-boxes)	chain_step	tests/chain_step.rs vs dsig_chain
S-box stage / T-table ROM	aes_table	unit tests vs reference AES
Three-depth chain engine	slice_chain	tests/slice_chain.rs vs reference
68-cycle BLAKE3 timetable	slice_schedule	tests/slice_schedule.rs re-checks every constraint
Per-op request model	slice_model	equals PreparedBatch (keys, digits, root)
Key factory	slice_factory	tests/slice_factory.rs bit-exact vs model
16-leaf subtree	slice_merkle	tests/slice_merkle.rs
Signer and frames	slice_signer	tests/slice_signer.rs vs to_bytes()
Whole slice	slice	tests/slice.rs keys-from-scratch to frames
Merkle top + Ed25519	cert_unit	tests/cert_unit.rs verifies under Dalek
HBM I/O	msg_reader, hbm_writer	tests/engine.rs
Kernel + registers	engine	tests/engine.rs (ignored by default: slow)
Bundle / XO / host	dsig-engine-export, fpga/u280/engine/, dsig-engine-host	card run

9. Build order

Each step is validated before the next starts. A newcomer can pick any unchecked item whose prerequisites are done.

1. ☑ Folded lane, T-table lane, chain engine, schedule, model, factory, subtree, packer, signer, slice, pipelined certificate unit: native tests pass, bit-exact against the reference (bounded gates, 2026-10-14).

2. ☑ Whole engine (8 slices) signs a batch end to end in native simulation; all 128 frames verify (2026-10-14, before the all-RHDL conversion).

3. ☑ Vivado out-of-context: lane, BLAKE3 lane and whole slice measured (section 1). ☐ Re-run after the cache-read pipeline fix (§4b).

4. ☑ All-RHDL conversion (section 7). ☐ Vendor the Ed25519 RHDL snapshot.

5. ◐ SLICES knob validated at 2 (card) and 8 (simulation); 4 is next.

6. ☑ slices-2 bundle → XO → v++ link at 205 MHz → routed acceptance (build v3, 2026-10-15: kernel 296,868 CLB LUT, WNS +0.016 ns, WHS +0.006 ns).

7. ☑ Card run: dsig-engine-host prepare → dsig_engine_runner → dsig-engine-host verify. Only its rate_lower is a measured result. Build v3, verified finite bursts (one cold start each):

   Signatures	Cycles	Cycles/signature	rate_lower
   1,024	78,704	76.9 (cold start dominates)	2,667,174/s
   65,536	2,272,135	34.7	5,912,813/s
   786,432	26,782,541	34.06	6,019,465/s

   34 cycles per signature is the slices-2 ceiling (KEY_PERIOD 68 per slice, two slices), so the card runs at its design rate. 786,432 is the largest burst that fits: each slice’s frames live in four 256 MiB HBM banks, and HBM[0] also holds the messages.

8. ◐ slices-4 with T-tables (DSIG_TABLE_MASK=0x1f8, rounds 3–8), then the same ladder.

   • Build v4 (two single-port ROMs per DualTableRom, plus 60 BRAM S-boxes per lane) failed placement: it needed 4,033 BRAM18 sites in the kernel region, which has 3,552 (see lever 1 above).
   • Build v5 (one true dual-port ROM per DualTableRom, no BRAM S-boxes; estimated ~2,700 BRAM18 sites) is the current attempt. Its routed report is the measurement; until then the slice size is an estimate.

9. ☐ BLAKE3 adders in DSPs, if a larger slice count needs the LUTs.