rhdl_dsig_scalable_core/

lease.rs

//! Durable namespace reservation before deriving or loading one-time keys.
//!
//! A lease is a permanent, exclusive-create record keyed by trusted signing
//! identity, epoch and batch number. Both the file and containing directory are
//! synchronized before a lease is returned. A crashed/abandoned reservation is
//! burned, never recovered or recycled. No private key material is stored.
//!
//! # The problem
//!
//! DSig signs every message with a *one-time* key: a W-OTS+ key that is safe
//! to use for exactly one message. Signing two different messages with the
//! same one-time key reveals enough hash-chain nodes for a forger to sign
//! other messages. So the most important rule of the whole project is: **a
//! one-time key is never reused**, not after it was admitted, not after a
//! timeout, not after a reset and not after a launch whose outcome is
//! uncertain.
//!
//! The keys are not stored anywhere. They are *derived*: the hardware (and the
//! software reference in [`crate::reference`]) computes every key of a batch
//! from the master seed, the epoch and the batch number. So the triple
//! `(signer identity, epoch, batch number)` is the real name of 128 one-time
//! keys ([`crate::BATCH_KEYS`]). If two runs ever derive the same triple from
//! the same seed, they derive the same keys, and reuse has happened, even if
//! each run on its own was perfectly correct.
//!
//! # The decision
//!
//! Before any keys are derived or loaded, the host *leases* the batch numbers
//! by creating a small text file on disk. The file is the proof that those
//! names have been used, and it is never deleted:
//!
//! * The file is created with `create_new` (the operating system's exclusive
//!   create). If it already exists the call fails, so two processes racing for
//!   the same batch cannot both win.
//! * The file *and* its directory are `fsync`ed (forced to stable storage)
//!   before the lease is handed out. A power cut after that point cannot make
//!   the reservation disappear.
//! * There is no "release", "resume" or "reopen". A crashed or abandoned
//!   reservation is *burned*: its batch numbers are simply lost. Losing some
//!   numbers out of a 64-bit space costs nothing; reusing one could cost
//!   everything.
//! * No private key material is ever written. The record holds only the
//!   public identity, the epoch and the batch numbers. The master seed is
//!   never stored.
//!
//! # Two kinds of lease
//!
//! | Type | Covers | Durable writes | Epoch half |
//! |---|---|---|---|
//! | [`BatchLease`] (via [`BatchLease::reserve`]) | one batch (128 keys) | one record per batch | low half: bit 63 clear, nonzero |
//! | [`RangeLease`] | [`RANGE_BATCHES`] = 2^20 batches (2^27 keys) | one record per range | high half: bit 63 ([`RANGE_EPOCH_BIT`]) set |
//!
//! A record per batch is simple but means one synchronous disk write per 128
//! keys, far too slow for a card that makes keys continuously. A range lease
//! burns a whole fixed interval of batch numbers with one durable write, then
//! hands the batches out one by one from memory. Because single leases and
//! range leases live in different halves of the epoch space, a single lease
//! can never overlap a range.
//!
//! # Lifecycle
//!
//! ```text
//!  single batch                          fixed range
//!  ────────────                          ───────────
//!  BatchLease::reserve(dir, id, e, b)    RangeLease::reserve(dir, id, E|bit63, r)
//!        │ create_new + write record           │ create_new + write record
//!        │ fsync file, fsync directory         │ fsync file, fsync directory
//!        ▼                                     ▼
//!  ┌───────────────┐                    ┌──────────────────────────────┐
//!  │  BatchLease   │                    │ RangeLease  next = r × 2^20  │
//!  │ (not Clone)   │                    │             end  = next+2^20 │
//!  └──────┬────────┘                    └──────┬───────────────────────┘
//!         │                                    │ take_next(): next += 1
//!         │◀───────────────────────────────────┘ (memory only, no fsync)
//!         │ prepare(master_seed, signing key)
//!         │   identity mismatch → error, lease consumed (burned)
//!         ▼
//!   PreparedBatch (keys derived; the local seed copy is zeroized)
//!
//!  Every path ends in "burned": drop, crash, error, depletion or use.
//!  The record file stays on disk forever; nothing reads it back to resume.
//! ```
//!
//! # Where it fits
//!
//! The host program `dsig_engine_host` (`src/bin/dsig_engine_host.rs`) calls
//! [`RangeLease::reserve`] in its `prepare` step *before* it draws the master
//! seed, so a crash at any later point burns the range instead of leaving it
//! reusable. The card ([`crate::engine`]) then derives every key of that run
//! from the loaded parameters. For software users, [`BatchLease::prepare`]
//! wraps [`PreparedBatch::new`] so a batch is burned on disk before its keys
//! exist in memory.
//!
//! # What it does not protect against
//!
//! This protects cooperating processes on a trusted local filesystem, not a
//! malicious administrator, filesystem rollback, restored snapshots or cloned
//! machines. Keep the directory persistent and backed by storage with correct
//! `fsync` semantics. Reusing a master seed after losing this journal is unsafe;
//! use new independent entropy in that case. Lease files must not be deleted.

use std::{
    fs::{self, OpenOptions},
    io::{self, Write},
    path::{Path, PathBuf},
};

use ed25519_dalek::SigningKey;
use zeroize::Zeroize;

use crate::reference::PreparedBatch;

/// Epoch high bit reserved for fixed-range allocation; single leases reject it.
///
/// Bit 63 of the 64-bit epoch splits the epoch space into two halves: epochs
/// with this bit clear belong to [`BatchLease::reserve`], epochs with it set
/// belong to [`RangeLease::reserve`]. The split exists so a single-batch
/// lease and a range lease can never name the same batch, even though their
/// record files have different names and so would not collide on disk.
pub const RANGE_EPOCH_BIT: u64 = 1 << 63;
/// One permanent range contains this many 128-key batches.
///
/// 2^20 = 1,048,576 batches, so 2^27 one-time keys per range record. Range
/// number `r` always covers batch IDs `r × 2^20` up to (not including)
/// `(r + 1) × 2^20`. The mapping is fixed, so no "next free batch" counter
/// has to be updated on disk, and there is no such counter to corrupt.
pub const RANGE_BATCHES: u64 = 1 << 20;

/// A non-cloneable allocation capability, returned only after durable creation.
///
/// Holding a `BatchLease` means "this batch number is burned on disk and is
/// mine alone". It deliberately does not derive `Clone` or `Copy`: the only
/// thing you can do with it is spend it once, in [`Self::prepare`]. Dropping
/// it without preparing simply wastes the batch number, which is safe.
pub struct BatchLease {
    /// The signer's Ed25519 public (verifying) key, 32 bytes. It is part of
    /// the lease's name, and [`Self::prepare`] checks the signing key against it.
    signer: [u8; 32],
    /// The epoch: a 64-bit namespace number chosen by the operator. Low half
    /// (bit 63 clear) for single leases, high half for batches from a range.
    epoch: u64,
    /// The batch number inside the epoch; one batch is 128 one-time keys.
    batch_id: u64,
    /// Path of the on-disk record that burned this batch. For a batch taken
    /// from a [`RangeLease`], this is the range's shared record.
    record: PathBuf,
}

impl BatchLease {
    /// Atomically reserve a fresh batch in an existing, trusted journal directory.
    ///
    /// # Errors
    /// Duplicate IDs, a zero or range-mode epoch, missing/unwritable directories
    /// or durability failures return an error. A record left behind by any
    /// failure stays burned. Range allocation uses a disjoint epoch namespace,
    /// preventing a single lease from overlapping any [`RangeLease`].
    ///
    /// Arguments: `directory` is the journal (it must already exist and be
    /// persistent); `signer` is the 32-byte Ed25519 public key; `epoch` and
    /// `batch_id` name the batch. On success the record
    /// `dsig-v1-<signer hex>-<epoch hex>-<batch hex>.lease` exists on stable
    /// storage and will stay there forever.
    pub fn reserve(
        directory: &Path,
        signer: [u8; 32],
        epoch: u64,
        batch_id: u64,
    ) -> io::Result<Self> {
        // Epoch 0 is rejected, and so is the high half: that half belongs to
        // range leases, and keeping the halves apart is what stops overlap.
        if epoch == 0 || epoch & RANGE_EPOCH_BIT != 0 {
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                "single leases require a nonzero low-half epoch",
            ));
        }
        // The signer's public key as 64 lowercase hex characters, used in the
        // file name so different signers never share a record.
        let identity: String = signer.iter().map(|byte| format!("{byte:02x}")).collect();
        // Fixed-width hex fields make the name unique per (signer, epoch, batch).
        let record = directory.join(format!(
            "dsig-v1-{identity}-{epoch:016x}-{batch_id:016x}.lease"
        ));
        // `create_new` is the exclusive create: it fails with AlreadyExists if
        // any process (this one, an earlier crashed one, a racing one) has
        // ever created this record. This one call is the whole race guard.
        let mut file = OpenOptions::new()
            .write(true)
            .create_new(true)
            .open(&record)?;
        // Human-readable public contents only: profile, identity, epoch, batch.
        // If this write fails, the empty record still exists and stays burned.
        writeln!(
            file,
            "DSIG_FPGA_W4_HARAKA_BLAKE3_V1\nsigner={identity}\nepoch={epoch}\nbatch={batch_id}\nstate=permanently-reserved"
        )?;
        // Flush the file's contents to stable storage...
        file.sync_all()?;
        // ...and the directory, so the new directory entry (the name) survives
        // a power cut too. Only after both does the caller get a lease.
        fs::File::open(directory)?.sync_all()?;
        Ok(Self {
            signer,
            epoch,
            batch_id,
            record,
        })
    }

    /// Permanent public record path, possibly covering an entire fixed range.
    /// Dropping this permit never removes or reopens its record.
    #[must_use]
    pub fn record(&self) -> &Path {
        &self.record
    }

    /// Consume the lease before preparing keys. Identity mismatches burn it too.
    ///
    /// # Errors
    /// Returns an error if the supplied signing key is not the reserved identity.
    ///
    /// Takes `self` by value, so the lease is gone after this call whether it
    /// succeeds or fails: a batch can be prepared at most once. `master_seed`
    /// is the 32-byte secret from which the batch's 128 one-time keys are
    /// derived (see [`PreparedBatch::new`]); this function wipes its own copy
    /// before returning and never writes it anywhere.
    pub fn prepare(
        self,
        mut master_seed: [u8; 32],
        signer: &SigningKey,
    ) -> io::Result<PreparedBatch> {
        // The lease names one public key; refuse to derive keys certified by
        // any other key. The seed copy is wiped on this error path too.
        if signer.verifying_key().to_bytes() != self.signer {
            master_seed.zeroize();
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                "lease signer mismatch",
            ));
        }
        // Derive and certify the batch's keys for exactly the leased
        // (epoch, batch) pair; the caller cannot pick different numbers here.
        let prepared = PreparedBatch::new(master_seed, self.epoch, self.batch_id, signer);
        // Overwrite this stack copy of the seed with zeros before it goes out
        // of scope, so the secret does not linger in memory.
        master_seed.zeroize();
        Ok(prepared)
    }
}

/// Persist-ahead allocation of a fixed, disjoint range of batch identifiers.
///
/// Each record burns 2^20 batches (2^27 one-time keys) before any permit is
/// returned. At 20 million keys/s this amortizes durability to roughly one
/// reservation every 6.7 seconds rather than one per 128 keys. This is capacity
/// arithmetic, not a measured allocator or signing rate.
///
/// Range epochs have their high bit set; single-batch leases reject those
/// epochs. Each range number maps to exactly one fixed-width interval, so
/// exclusive file creation prevents races without an in-place journal or a
/// crash-sensitive high-water-mark update. A partial record is still burned.
/// There is deliberately no resume/reopen method: a crash or drop discards all
/// unused batch IDs. Do not delete records or reuse them after snapshot rollback.
///
/// Like [`BatchLease`], a `RangeLease` is not `Clone`: only one live object
/// can hand out batches from a range, so no two callers get the same ID.
pub struct RangeLease {
    /// The signer's 32-byte Ed25519 public key; copied into every
    /// [`BatchLease`] this range hands out.
    signer: [u8; 32],
    /// The range epoch; always has [`RANGE_EPOCH_BIT`] set.
    epoch: u64,
    /// The next batch ID to hand out. Starts at `range_number × 2^20` and only
    /// ever increases; it lives in memory only and is never saved.
    next: u64,
    /// One past the last batch ID of the range (`start + RANGE_BATCHES`).
    end: u64,
    /// Path of the single record that burned the whole interval.
    record: PathBuf,
}

impl RangeLease {
    /// Durably reserve the fixed interval belonging to `range_number`.
    ///
    /// # Errors
    /// Rejects non-range epochs, identifier overflow, an existing record, and
    /// filesystem/durability errors. Failures never authorize any batch permit.
    ///
    /// Arguments: `directory` is the persistent journal; `signer` the 32-byte
    /// Ed25519 public key; `epoch` a high-half epoch; `range_number` picks the
    /// interval `range_number × 2^20 .. (range_number + 1) × 2^20`.
    pub fn reserve(
        directory: &Path,
        signer: [u8; 32],
        epoch: u64,
        range_number: u64,
    ) -> io::Result<Self> {
        // Ranges must use the high half of the epoch space (the mirror image
        // of the check in BatchLease::reserve), so the two never overlap.
        if epoch & RANGE_EPOCH_BIT == 0 {
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                "range leases require a high-half epoch",
            ));
        }
        // First batch ID of the range. Checked multiply: a huge range number
        // must be an error, not a silent wrap-around onto low batch IDs.
        let start = range_number
            .checked_mul(RANGE_BATCHES)
            .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "range start overflow"))?;
        // One past the last batch ID; also checked, for the topmost range.
        let end = start
            .checked_add(RANGE_BATCHES)
            .ok_or_else(|| io::Error::new(io::ErrorKind::InvalidInput, "range end overflow"))?;
        // Public key as 64 hex characters, for the record's file name.
        let identity: String = signer.iter().map(|byte| format!("{byte:02x}")).collect();
        // The word "range" in the name keeps range records apart from
        // single-batch records in the same directory.
        let record = directory.join(format!(
            "dsig-v1-{identity}-{epoch:016x}-range-{range_number:016x}.lease"
        ));
        // Exclusive create: exactly one caller ever wins a given range.
        let mut file = OpenOptions::new()
            .write(true)
            .create_new(true)
            .open(&record)?;
        // Public description of the burned interval; no secrets. A failure
        // here leaves a partial record that still burns the range.
        writeln!(
            file,
            "DSIG_FPGA_W4_HARAKA_BLAKE3_V1\nsigner={identity}\nepoch={epoch}\nrange={range_number}\nstart={start}\nend_exclusive={end}\nstate=permanently-reserved"
        )?;
        // Make the contents durable, then the directory entry, before any
        // batch of the range can be handed out.
        file.sync_all()?;
        fs::File::open(directory)?.sync_all()?;
        Ok(Self {
            signer,
            epoch,
            next: start,
            end,
            record,
        })
    }

    /// The durable reservation record that burned this whole interval.
    #[must_use]
    pub fn record(&self) -> &Path {
        &self.record
    }

    /// Unallocated batch permits in this live capability, never on-disk recovery.
    #[must_use]
    pub const fn remaining(&self) -> u64 {
        self.end - self.next
    }

    /// Burn the next batch ID before handing out its non-cloneable capability.
    ///
    /// No filesystem synchronization is needed here: the entire interval was
    /// durably burned by [`Self::reserve`]. Dropping a returned permit also burns
    /// that batch. A depleted range returns `None` and cannot be recycled.
    pub fn take_next(&mut self) -> Option<BatchLease> {
        // Depleted: the range is used up for good; reserve a new range number.
        if self.next == self.end {
            return None;
        }
        // Hand out the current ID and advance first, so the same ID can never
        // be returned twice by this object.
        let batch_id = self.next;
        self.next += 1;
        Some(BatchLease {
            signer: self.signer,
            epoch: self.epoch,
            batch_id,
            record: self.record.clone(),
        })
    }
}

/// Tests of the burn-forever rules: duplicates fail, drops do not free a
/// batch, races have one winner and ranges never overlap single leases.
#[cfg(test)]
mod tests {
    use super::*;
    use std::sync::atomic::{AtomicU64, Ordering};
    /// Counter that makes each test's scratch directory name unique within
    /// one test process (tests run in parallel threads).
    static NEXT: AtomicU64 = AtomicU64::new(0);

    /// A fresh, empty directory under the system temp dir, named by process
    /// ID and [`NEXT`], used as a throwaway journal by one test.
    fn temporary_directory() -> PathBuf {
        let path = std::env::temp_dir().join(format!(
            "dsig-lease-test-{}-{}",
            std::process::id(),
            NEXT.fetch_add(1, Ordering::Relaxed)
        ));
        fs::create_dir(&path).unwrap();
        path
    }

    /// Dropping a lease (or restarting the process) must not free its batch:
    /// a second reserve of the same triple fails with `AlreadyExists`, while
    /// a different batch or epoch still succeeds and epoch 0 is rejected.
    #[test]
    fn dropping_or_restarting_cannot_reallocate_a_durable_lease() {
        let directory = temporary_directory();
        let signer = SigningKey::from_bytes(&[0x19; 32])
            .verifying_key()
            .to_bytes();
        let lease = BatchLease::reserve(&directory, signer, 1, 4).unwrap();
        assert!(lease.record().is_file());
        drop(lease);
        assert_eq!(
            BatchLease::reserve(&directory, signer, 1, 4)
                .err()
                .unwrap()
                .kind(),
            io::ErrorKind::AlreadyExists
        );
        assert!(BatchLease::reserve(&directory, signer, 1, 5).is_ok());
        assert!(BatchLease::reserve(&directory, signer, 2, 4).is_ok());
        assert!(BatchLease::reserve(&directory, signer, 0, 4).is_err());
        // Only this test's uniquely-created directory is removed. Real journals
        // are permanent and expose no cleanup operation in the API.
        fs::remove_dir_all(directory).unwrap();
    }

    /// Four threads race for the same single batch; exactly one may win.
    #[test]
    fn simultaneous_reservations_have_exactly_one_winner() {
        let directory = temporary_directory();
        let handles: Vec<_> = (0..4)
            .map(|_| {
                let path = directory.clone();
                std::thread::spawn(move || BatchLease::reserve(&path, [0x42; 32], 10, 3).is_ok())
            })
            .collect();
        assert_eq!(
            handles
                .into_iter()
                .map(|h| usize::from(h.join().unwrap()))
                .sum::<usize>(),
            1
        );
        fs::remove_dir_all(directory).unwrap();
    }

    /// Range rules: wrong epoch halves are rejected both ways, overflowing
    /// range numbers are rejected, ranges 0 and 1 hand out disjoint IDs, a
    /// dropped range cannot be re-reserved, depletion returns `None`, and
    /// even an empty (torn) record burns its range.
    #[test]
    fn ranges_are_disjoint_permanent_and_separate_from_single_leases() {
        let directory = temporary_directory();
        let signer = [0x29; 32];
        let epoch = RANGE_EPOCH_BIT | 3;
        assert!(BatchLease::reserve(&directory, signer, epoch, 0).is_err());
        assert!(RangeLease::reserve(&directory, signer, 3, 0).is_err());
        assert!(RangeLease::reserve(&directory, signer, epoch, u64::MAX).is_err());
        assert!(RangeLease::reserve(&directory, signer, epoch, u64::MAX / RANGE_BATCHES).is_err());
        let mut first = RangeLease::reserve(&directory, signer, epoch, 0).unwrap();
        let mut second = RangeLease::reserve(&directory, signer, epoch, 1).unwrap();
        assert_eq!(first.remaining(), RANGE_BATCHES);
        let zero = first.take_next().unwrap();
        let one = first.take_next().unwrap();
        let next_range = second.take_next().unwrap();
        assert_eq!(
            (zero.batch_id, one.batch_id, next_range.batch_id),
            (0, 1, RANGE_BATCHES)
        );
        assert_eq!(first.remaining(), RANGE_BATCHES - 2);
        drop(first);
        assert_eq!(
            RangeLease::reserve(&directory, signer, epoch, 0)
                .err()
                .unwrap()
                .kind(),
            io::ErrorKind::AlreadyExists
        );
        assert_eq!(
            RangeLease::reserve(&directory, signer, epoch, 1)
                .err()
                .unwrap()
                .kind(),
            io::ErrorKind::AlreadyExists
        );
        // Exercise the final allocation without looping over a million permits.
        second.next = second.end - 1;
        assert_eq!(second.take_next().unwrap().batch_id, 2 * RANGE_BATCHES - 1);
        assert_eq!(second.remaining(), 0);
        assert!(second.take_next().is_none());
        // Even a torn/empty record from a failed reservation is never resumed.
        let identity: String = signer.iter().map(|byte| format!("{byte:02x}")).collect();
        let torn = directory.join(format!(
            "dsig-v1-{identity}-{epoch:016x}-range-0000000000000002.lease"
        ));
        OpenOptions::new()
            .write(true)
            .create_new(true)
            .open(torn)
            .unwrap();
        assert_eq!(
            RangeLease::reserve(&directory, signer, epoch, 2)
                .err()
                .unwrap()
                .kind(),
            io::ErrorKind::AlreadyExists
        );
        fs::remove_dir_all(directory).unwrap();
    }

    /// Four threads race for the same range number; exactly one may win.
    #[test]
    fn concurrent_range_claims_have_one_winner() {
        let directory = temporary_directory();
        let handles: Vec<_> = (0..4)
            .map(|_| {
                let path = directory.clone();
                std::thread::spawn(move || {
                    RangeLease::reserve(&path, [0x24; 32], RANGE_EPOCH_BIT, 2).is_ok()
                })
            })
            .collect();
        assert_eq!(
            handles
                .into_iter()
                .map(|h| usize::from(h.join().unwrap()))
                .sum::<usize>(),
            1
        );
        fs::remove_dir_all(directory).unwrap();
    }
}