Module broadcast 

Sending: one RDMA write to every replica, each a single packet.

Problem. ubft_new has two broadcast styles, and both lose messages. The fast path’s bcast_vote_rr sends to one queue pair, then the state machine moves on, so a vote reaches one replica. The slow path loops over the queue pairs but skips any destination whose stream is full that cycle. Its 64-byte certify share is eight 64-bit beats, each marked last, so the RoCE stack sees eight packets, not one.

Idea. Broadcaster sends the same write to the replica’s N − 1 peers, on local queue pairs qbase .. qbase + N − 1 (qbase = (N − 1)·id; peer j is at rank j below id and j − 1 above, as the RoCE stack programs them), and waits for each meta and data beat to be accepted. Nothing is dropped. A write is one packet: its data beats are contiguous, and tlast is set on the last beat only, so a 64-byte Ed25519 share is one beat and a 26-beat DSig share is one packet. The controller records the same words in its own HBM (crate::writer); there is no connection to self.

Result. Every peer gets every vote, and the replica counts its own. That is what lets the fast path reach its 2f + 1 quorum at all.

Structs

BroadcastIn

Inputs of Broadcaster.

BroadcastOut

Outputs of Broadcaster.

BroadcastReq

A broadcast request.

BroadcastState

The broadcaster’s registers.

Broadcaster

Sends one write to every replica (see the module docs).

Functions

broadcast_kernel

The broadcaster’s logic.

idle_bcast_req

No broadcast.

keep_bytes

tkeep for len bytes from byte 0.

write_meta

The RoCE request word for a write of len bytes to raddr on queue pair qpn (op 1, laddr 0: data on the stream).