Crate rhdl_ubft 

rhdl_ubft: ubft_new’s consensus kernel, rewritten in RHDL

Building for Wolverine’s U280

The full consensus implementation does not yet have an established, accepted end-to-end U280 image flow. A successful Cargo build or RTL export is only a prerequisite; it does not establish routed timing or card behavior. Do not substitute crates/dsig/tools/wolverine-ubft.sh: that script builds the isolated DSig signature-service probe, not this consensus kernel.

As recorded in PROJECT_COORDINATION.md on October 24, 2026, production export for source d806f5bd passed, but the subsequent child synthesis timed out without an accepted checkpoint. A later receiver fixture hit an Icarus parser input-buffer overflow. The ledger also recorded less than the 30 GiB free disk required for the planned full link. These are dated observations, not a live Wolverine inventory or a claim that the current source was built.

Continue from the pinned source, configuration and evidence under evidence/rhdl-ubft/card-bringup-20261024/: resolve the lowering/parser and child-synthesis gates, confirm disk capacity, then package/link with Vivado/Vitis 2023.2 and the installed U280 platform. Require routed acceptance and host verification before programming or claiming performance. Preserve the existing evidence and match the host configuration to the exported kernel.

Run commands from the workspace root. Before compilation or RTL generation, record ownership in PROJECT_COORDINATION.md and /home/sahasta/Development/FPGA_COORDINATION.md, and use the applicable nonblocking local-heavy.lock; crates/ed25519/tools/bounded.sh wraps local Cargo commands with that lock and resource limits. Reserve Wolverine builds in both ledgers too, then use crates/dsig/tools/with-u280-lock.sh to hold u280.lock. Do not bypass a busy lock.

ubft_new (commit a7537bd) runs Byzantine fault-tolerant consensus on an Alveo U280. Each card is one replica. Replicas talk by RDMA-writing into each other’s HBM over a RoCE stack. A key-value store sends every PUT through consensus before it touches its hash table. ubft_new’s kernels are Vitis HLS C++. rhdl_ubft is its consensus kernel (ubft) in RHDL: the same ports, the related HBM layout and protocol, with targeted repairs and mechanics rebuilt for hardware. Certified Byzantine recovery and full integration remain open.

In numbers. Not measured yet. The comparison against ubft_new’s own RTL (HLS cosimulation, Vivado route and the card) is the next chapter of this page to be written. Until then this page states no speedup.

Contents

1. What ubft_new does
2. Why it is slow, and where it is wrong
3. The rewrite
4. Choosing a build
5. What is proven
6. Where to go next

Chapter 1: What ubft_new does

A slot is one consensus decision: sequence number k, value m. The leader proposes, and every replica votes by writing a word into the same mailbox address of every peer’s HBM (layout). Replicas poll their own HBM to count votes.

• Fast path: prepare, then two rounds of votes (will_cert, will_commit). Each round needs a quorum of 2f + 1, where uBFT tolerates f = (n − 1) / 2 faults among n replicas.
• Slow path, after a timeout: every replica signs the slot’s summary with Ed25519, sends its 64-byte share to all, and verifies shares until it holds 2f + 1. Then one round of commit votes.
• View change, after another timeout: the leader rotates, and the new leader re-proposes the highest-numbered prepare anyone saw.

Chapter 2: Why it is slow, and where it is wrong

Slow. The HLS kernel is one loop that does one state’s work per iteration, and every mailbox word it reads is its own HBM transaction. Its Ed25519 calls are inline: a signature takes 3.07M cycles and a verification 10.4M (C/RTL cosimulation, measured by DSig), and the loop stops for each. That signer is also the routed design’s worst path. The kernel clock misses 300 MHz by 2.866 ns, and the linker selects about 161 MHz. That is a timing-derived setting, not measured runtime frequency.

Wrong. Reading the code, and running it as a C-simulated cluster, shows a protocol that cannot reach a decision as written:

• a fast-path vote is sent to one queue pair, then the state moves on;
• the leader never votes, so 2f + 1 out of 2f + 1 is unreachable;
• the slow path never broadcasts the flag that tells peers its share is there;
• sends go to queue pairs id·N + j, but the RoCE stack programs N − 1 connections at (N − 1)·id + rank;
• k = slot + 1 repeats, and nothing clears old votes;
• shares go out as eight one-word packets;
• the signed summary XORs its fields, and the library hashes only one byte of it.

protocol lists each fix next to the line it changes.

Chapter 3: The rewrite

The kernel is small engines around one controller (replica):

Engine	Job	Replaces
control	ubft_new’s register map, plus counters the host reads after a run	HLS s_axilite
fetch	one burst per poll, any alignment	one HBM read per word
broadcast	a write to every peer, one packet each, nothing dropped	rdma_write* loops
writer	the replica’s own copy of what it sent	(missing)
share	a peer’s share from HBM, realigned, into the verifier	16 word reads
setup + crypto	keys into a signature service on streams	inline xf Ed25519
protocol	the four machines	ubft_machines.cpp

Chapter 4: Choosing a build

Choice	How	Values
Signature scheme	Cargo feature, then RHDL_UBFT_SCHEME if both are on	ed25519 (default), dsig
Most replicas	RHDL_UBFT_NODES	3..=16 (ubft_new: 12)
Window	RHDL_UBFT_WINDOW	1..=8 (ubft_new: 3)

The run’s cluster size is node_num, at most the build’s. The Tuner writes supported values to build-config.toml; apply them with apply_ubft_plan. Saving TOML alone does not change a build.

Chapter 5: What is proven

Nothing on hardware yet. In cycle simulation (feature sim: three real RHDL replicas, models of HBM, RoCE and the host, and a stand-in signature scheme that is not secure):

Run	Result
healthy cluster, 5 proposals	all decide on the fast path, in order, and agree; every write one framed packet to a programmed queue pair
one replica’s fast votes dropped	all decide on the slow path (2 shares verified each)
equivocating leader	no conflicting decisions, but nothing decided: views drift apart and an all-n view change never completes (a known failure, an ignored test)

The replica lowers to Verilog (export_rhdl_ubft). Its view change is a value-binding prototype, not uBFT’s certified recovery; the plan to bring that back is docs/rhdl_ubft-recovery-design.md.

Where to go next

You want to…	Read
see the protocol and its fixes	protocol
see what crosses the wire	layout, wire
add a signature scheme	ports::CryptoIn, ports::CryptoScheme, crypto
plug into the Vitis link	ports

Re-exports

pub use replica::RhdlUbftEd25519;

Modules

broadcast

Sending: one RDMA write to every replica, each a single packet.

build_config

Build variables (build.rs).

card

The one-card cluster: replicas, the on-chip fabric, the service (DSig). The one-card cluster (D2, DSig): N rhdl_ubft replicas, the [fabric] that stands in for RoCE between them, and their signature service, as one card kernel, so a whole consensus slot can be timed on hardware without a second card or the RoCE IP.

cert

Certificates (W4; docs/rhdl_ubft-view-change.md): q = f + 1 signature shares from distinct replicas over one exact key (role, view, instance, digest).

checkpoint

Certified checkpoints and state transfer (W5; docs/rhdl_ubft-checkpoints.md).

consensus

Consensus over CTB, the normal case (W4 step 2; docs/rhdl_ubft-view-change.md). The view change is added on top.

control

The host’s view: ubft_new’s register map, plus what rhdl_ubft measured.

crypto

Signature schemes for the slow path, one module each, all behind the same ports (crate::ports::CryptoIn / crate::ports::CryptoOut).

ctb

Consistent tail broadcast (CTB): recovery layer 3, workstream W3 (docs/rhdl_ubft-ctb.md).

fetch

Reading mailboxes: one burst instead of one read per word.

kv

The application side of uBFT, authored in RHDL.

layout

Where ubft_new keeps its mailboxes in HBM, and what crosses the wire.

ports

The kernel’s ports: the same as ubft_new’s ubft kernel, so the Vitis link (config.cfg) connects rhdl_ubft where ubft_new’s kernel was.

protocol

The consensus controller: ubft_new’s four machines, fixed and faster.

register

Single-writer registers: recovery layer 1 (docs/rhdl_ubft-recovery-design.md).

register_reader

Reading a replicated register: recovery layer 2, the reader (docs/rhdl_ubft-recovery-design.md, docs/rhdl_ubft-host-generation.md).

replica

One rhdl_ubft replica: the kernel that takes the place of ubft_new’s.

rpc

Bounded RPC components preserving explicit request and replica identities.

setup

Starting the signature service: keys in, then go.

share

Streaming a peer’s share out of HBM, realigned, into the verify request.

view_change

The view change (W4 steps 3–4; docs/rhdl_ubft-view-change.md): uBFT Algorithm 5 with every NEWVIEW check upstream leaves out.

wire

The words on rhdl_ubft’s streams, bit for bit as ubft_new has them.

writer

Writing one word into this replica’s own HBM.

Macros

define_replica

Define a replica type $name<N, G> for signature scheme $crypto with $sb share beats.

Type Aliases

SelectedReplica

The replica selected at build time, including its actual service and shape.