Skip to content

blackbox_format.hpp

The SHULIB BLACKBOX on-disk format, v1 — the binary record SdSink writes and BlackboxReader reads.

This header declares 6 types (55 members), 12 free functions, and 8 constants.

Extracted from include/shulib/diag/blackbox_format.hpp — this page is that header's documentation, reformatted, so it cannot disagree with the code. Prose about how to think about the API lives in the user guide; worked recipes live in the cookbook; this page is the complete, mechanical list of what exists.

Contents

kMagic

inline constexpr char kMagic[4] = {'S', 'H', 'B', 'B'}

The four magic bytes every blackbox file starts with ("SHulib BlackBox").

constant, declared at include/shulib/diag/blackbox_format.hpp:70.

kFormatVersion

inline constexpr std::uint16_t kFormatVersion = 1

On-disk format version. BUMP THIS whenever any layout below changes — a reader refuses a version it was not built for rather than misreading it (header note).

constant, declared at include/shulib/diag/blackbox_format.hpp:74.

kHeaderBytes

inline constexpr std::size_t kHeaderBytes = 256

Size of the fixed file header, in bytes (v1). Fixed width so a reader can seek past it without parsing, and generous enough to hold full provenance.

constant, declared at include/shulib/diag/blackbox_format.hpp:78.

kFrameHeaderBytes

inline constexpr std::size_t kFrameHeaderBytes = 4

Size of the per-frame prefix: {u8 type, u8 reserved, u16 payloadBytes}.

constant, declared at include/shulib/diag/blackbox_format.hpp:81.

kTickPayloadBytes

inline constexpr std::size_t kTickPayloadBytes = 428

Payload size of one Tick frame (v1). Pinned by the golden test; the encoder asserts it wrote exactly this many bytes.

constant, declared at include/shulib/diag/blackbox_format.hpp:85.

kSummaryPayloadBytes

inline constexpr std::size_t kSummaryPayloadBytes = 168

Payload size of one Summary frame (v1).

constant, declared at include/shulib/diag/blackbox_format.hpp:88.

kTriagePayloadBytes

inline constexpr std::size_t kTriagePayloadBytes = 24 + kTickPayloadBytes

Payload size of one Triage frame (v1): the D-7 triage fields PLUS the complete record of the tick the fault fired on (header note on dump ordering in sd_sink.hpp).

constant, declared at include/shulib/diag/blackbox_format.hpp:92.

kEndPayloadBytes

inline constexpr std::size_t kEndPayloadBytes = 28

Payload size of one End frame (v1) — the graceful-end stamp.

constant, declared at include/shulib/diag/blackbox_format.hpp:95.

enum class FrameType

enum class FrameType : std::uint8_t

What a frame carries. WIRE-STABLE: explicit values, append-only — an unknown type is skipped by length, never guessed at.

enum class, declared at include/shulib/diag/blackbox_format.hpp:99.

FrameType::Tick

Tick = 1

one DebugRecord (kTickPayloadBytes)

enumerator, declared at include/shulib/diag/blackbox_format.hpp:100.

FrameType::Summary

Summary = 2

one RunSummary (kSummaryPayloadBytes)

enumerator, declared at include/shulib/diag/blackbox_format.hpp:101.

FrameType::Triage

Triage = 3

the D-7 fault triage block + the fault tick's own record

enumerator, declared at include/shulib/diag/blackbox_format.hpp:102.

FrameType::End

End = 4

the graceful-end stamp: counts, brownout latch, end time

enumerator, declared at include/shulib/diag/blackbox_format.hpp:103.

struct TriageInfo

struct TriageInfo

The D-7 triage block, as data: which fault, when, on which tick, and how many preceding ticks follow it in the file. The record of the fault tick itself travels in the same frame (see sd_sink.hpp's dump-ordering rule).

struct, declared at include/shulib/diag/blackbox_format.hpp:109.

TriageInfo::fault

FaultCode fault = FaultCode::None

the fault that triggered the dump

field, declared at include/shulib/diag/blackbox_format.hpp:110.

TriageInfo::brownout

bool brownout = false

the latched brownout marker at dump time

field, declared at include/shulib/diag/blackbox_format.hpp:111.

TriageInfo::tickIndex

std::uint32_t tickIndex = 0

how many records the sink had seen when it fired

field, declared at include/shulib/diag/blackbox_format.hpp:112.

TriageInfo::faultTime

double faultTime = 0.0

the fault tick's t, seconds since the run epoch

field, declared at include/shulib/diag/blackbox_format.hpp:113.

TriageInfo::precedingTicks

std::uint32_t precedingTicks = 0

Tick frames that follow, oldest first (0 when streaming)

field, declared at include/shulib/diag/blackbox_format.hpp:114.

struct EndInfo

struct EndInfo

The end frame: what the sink knows about its own run when it closes cleanly. A file WITHOUT this frame ended abruptly — that absence is the truncation signal a reader can act on.

struct, declared at include/shulib/diag/blackbox_format.hpp:120.

EndInfo::tickFrames

std::uint32_t tickFrames = 0

Tick frames staged over the run

field, declared at include/shulib/diag/blackbox_format.hpp:121.

EndInfo::droppedFrames

std::uint32_t droppedFrames = 0

frames dropped for want of buffer (byte budget)

field, declared at include/shulib/diag/blackbox_format.hpp:122.

EndInfo::bytesBefore

std::uint32_t bytesBefore = 0

Bytes of this file that PRECEDE this frame — i.e. the frame's own offset. A reader can verify it against where it actually found the frame, which is how a file that was appended to, interleaved, or spliced gives itself away. (It is NOT "bytes the device confirmed": at close() the bulk of a caller-paced run is still staged and goes out in the same write as this frame, so that figure would read 0 for the most common run of all.)

field, declared at include/shulib/diag/blackbox_format.hpp:129.

EndInfo::messagesSeen

std::uint32_t messagesSeen = 0

log() lines handed to the sink and NOT carried (header note)

field, declared at include/shulib/diag/blackbox_format.hpp:130.

EndInfo::brownout

bool brownout = false

the latched brownout marker

field, declared at include/shulib/diag/blackbox_format.hpp:131.

EndInfo::deviceFailed

bool deviceFailed = false

a write() or flush() reported failure during the run

field, declared at include/shulib/diag/blackbox_format.hpp:132.

EndInfo::endTime

double endTime = 0.0

clock time at close, seconds since the run epoch

field, declared at include/shulib/diag/blackbox_format.hpp:133.

struct BlackboxHeader

struct BlackboxHeader

A decoded file header. Value type with bounded storage, like RunSummary: a decoded header must never hold views into a buffer the caller may free.

struct, declared at include/shulib/diag/blackbox_format.hpp:138.

BlackboxHeader::formatVersion

std::uint16_t formatVersion = 0

as read from the file

field, declared at include/shulib/diag/blackbox_format.hpp:139.

BlackboxHeader::headerBytes

std::uint16_t headerBytes = 0

self-declared header size (lets a reader seek)

field, declared at include/shulib/diag/blackbox_format.hpp:140.

BlackboxHeader::tickRecordBytes

std::uint16_t tickRecordBytes = 0

self-declared Tick payload size (cross-checked)

field, declared at include/shulib/diag/blackbox_format.hpp:141.

BlackboxHeader::flags

std::uint16_t flags = 0

reserved, 0 in v1

field, declared at include/shulib/diag/blackbox_format.hpp:142.

BlackboxHeader::epochSeconds

double epochSeconds = 0.0

the injected clock's reading when the file opened

field, declared at include/shulib/diag/blackbox_format.hpp:143.

BlackboxHeader::ringCapacity

std::uint32_t ringCapacity = 0

flight-recorder ring size the writer was configured with

field, declared at include/shulib/diag/blackbox_format.hpp:144.

BlackboxHeader::byteBudget

std::uint32_t byteBudget = 0

RAM byte budget the writer was configured with

field, declared at include/shulib/diag/blackbox_format.hpp:145.

BlackboxHeader::buildHash

[[nodiscard]] std::string_view buildHash() const noexcept

The git build hash the run was built from. EMPTY means MISSING — render it loudly and never invent a plausible value (§18.5, build_info.hpp).

function, declared at include/shulib/diag/blackbox_format.hpp:149.

BlackboxHeader::routineId

[[nodiscard]] std::string_view routineId() const noexcept

The routine id the run was started with (may be empty).

function, declared at include/shulib/diag/blackbox_format.hpp:151.

BlackboxHeader::alliance

[[nodiscard]] std::string_view alliance() const noexcept

Alliance as free text ("red"/"blue"/"skills"); may be empty.

function, declared at include/shulib/diag/blackbox_format.hpp:153.

BlackboxHeader::side

[[nodiscard]] std::string_view side() const noexcept

Side as free text ("left"/"right"); may be empty.

function, declared at include/shulib/diag/blackbox_format.hpp:155.

BlackboxHeader::portMap

[[nodiscard]] std::string_view portMap() const noexcept

The caller-authored port map; may be empty.

function, declared at include/shulib/diag/blackbox_format.hpp:157.

BlackboxHeader::buildHash_

char buildHash_[48] = ""

Storage for buildHash() — written by the decoder, NUL-terminated.

field, declared at include/shulib/diag/blackbox_format.hpp:160.

BlackboxHeader::routineId_

char routineId_[32] = ""

Storage for routineId().

field, declared at include/shulib/diag/blackbox_format.hpp:162.

BlackboxHeader::alliance_

char alliance_[16] = ""

Storage for alliance().

field, declared at include/shulib/diag/blackbox_format.hpp:164.

BlackboxHeader::side_

char side_[16] = ""

Storage for side().

field, declared at include/shulib/diag/blackbox_format.hpp:166.

BlackboxHeader::portMap_

char portMap_[96] = ""

Storage for portMap().

field, declared at include/shulib/diag/blackbox_format.hpp:168.

class ByteWriter

class ByteWriter

Little-endian byte writer with a hard end: a write that would not fit writes NOTHING and latches overflow, so an undersized buffer can never corrupt neighbouring memory and can never half-write a field. Callers check ok().

class, declared at include/shulib/diag/blackbox_format.hpp:174.

ByteWriter::ByteWriter

explicit ByteWriter(std::span<std::byte> out) noexcept

Write into out, starting at offset 0.

function, declared at include/shulib/diag/blackbox_format.hpp:177.

ByteWriter::u8

void u8(std::uint8_t v) noexcept

Append one unsigned byte.

function, declared at include/shulib/diag/blackbox_format.hpp:180.

ByteWriter::boolean

void boolean(bool v) noexcept

Append a bool as 0x00 / 0x01.

function, declared at include/shulib/diag/blackbox_format.hpp:187.

ByteWriter::u16

void u16(std::uint16_t v) noexcept

Append a 16-bit unsigned value, little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:189.

ByteWriter::u32

void u32(std::uint32_t v) noexcept

Append a 32-bit unsigned value, little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:197.

ByteWriter::i32

void i32(std::int32_t v) noexcept

Append a 32-bit signed value as two's complement, little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:206.

ByteWriter::f64

void f64(double v) noexcept

Append an IEEE-754 binary64 value, little-endian (bit pattern preserved, so a NaN or an infinity survives the trip exactly as it was recorded).

function, declared at include/shulib/diag/blackbox_format.hpp:209.

ByteWriter::text

void text(std::string_view s, std::size_t fieldBytes) noexcept

Append fieldBytes of text: s truncated to fit, NUL-padded to the full width. Fixed width by design — a variable-length string would make every later offset depend on run-time content.

function, declared at include/shulib/diag/blackbox_format.hpp:222.

ByteWriter::zeros

void zeros(std::size_t n) noexcept

Append n zero bytes (reserved space).

function, declared at include/shulib/diag/blackbox_format.hpp:232.

ByteWriter::offset

[[nodiscard]] std::size_t offset() const noexcept

How many bytes have been appended.

function, declared at include/shulib/diag/blackbox_format.hpp:241.

ByteWriter::ok

[[nodiscard]] bool ok() const noexcept

False once any append did not fit (nothing was written for that append).

function, declared at include/shulib/diag/blackbox_format.hpp:243.

class ByteReader

class ByteReader

Little-endian byte reader with a hard end: a read past the end yields zero and latches exhaustion, so a truncated or corrupt file can never read out of bounds and can never half-read a field. Callers check ok().

class, declared at include/shulib/diag/blackbox_format.hpp:262.

ByteReader::ByteReader

explicit ByteReader(std::span<const std::byte> in) noexcept

Read from in, starting at offset 0.

function, declared at include/shulib/diag/blackbox_format.hpp:265.

ByteReader::u8

[[nodiscard]] std::uint8_t u8() noexcept

Read one unsigned byte (0 past the end).

function, declared at include/shulib/diag/blackbox_format.hpp:268.

ByteReader::boolean

[[nodiscard]] bool boolean() noexcept

Read a bool: any nonzero byte is true.

function, declared at include/shulib/diag/blackbox_format.hpp:275.

ByteReader::u16

[[nodiscard]] std::uint16_t u16() noexcept

Read a 16-bit unsigned value, little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:277.

ByteReader::u32

[[nodiscard]] std::uint32_t u32() noexcept

Read a 32-bit unsigned value, little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:286.

ByteReader::i32

[[nodiscard]] std::int32_t i32() noexcept

Read a 32-bit signed value (two's complement), little-endian.

function, declared at include/shulib/diag/blackbox_format.hpp:297.

ByteReader::f64

[[nodiscard]] double f64() noexcept

Read an IEEE-754 binary64 value, little-endian (bit pattern preserved).

function, declared at include/shulib/diag/blackbox_format.hpp:299.

ByteReader::text

void text(char* dst, std::size_t dstBytes, std::size_t fieldBytes) noexcept

Read fieldBytes of NUL-padded text into dst (capacity dstBytes, always NUL-terminated). Bytes beyond the destination are consumed and discarded, so the cursor stays aligned no matter how the caller sized its storage.

function, declared at include/shulib/diag/blackbox_format.hpp:314.

ByteReader::skip

void skip(std::size_t n) noexcept

Skip n bytes (reserved space).

function, declared at include/shulib/diag/blackbox_format.hpp:327.

ByteReader::offset

[[nodiscard]] std::size_t offset() const noexcept

How many bytes have been consumed.

function, declared at include/shulib/diag/blackbox_format.hpp:333.

ByteReader::ok

[[nodiscard]] bool ok() const noexcept

False once any read ran past the end.

function, declared at include/shulib/diag/blackbox_format.hpp:335.

encodeHeader

[[nodiscard]] inline std::size_t encodeHeader(std::span<std::byte> out, const SessionInfo& info, double epochSeconds, std::uint32_t ringCapacity, std::uint32_t byteBudget) noexcept

Encode the 256-byte file header into out. Returns the bytes written (0 if out is too small). Provenance strings are copied in, truncated to their field widths — an EMPTY build hash stays empty, because MISSING must stay loud all the way to disk.

free function, declared at include/shulib/diag/blackbox_format.hpp:365.

decodeHeader

[[nodiscard]] inline bool decodeHeader(std::span<const std::byte> in, BlackboxHeader& out) noexcept

Decode a file header. Returns false if in is shorter than the header or the magic does not match; the VERSION is decoded but NOT judged here — BlackboxReader owns the refusal policy, and a caller inspecting a rejected file still wants to see what version it claims to be.

free function, declared at include/shulib/diag/blackbox_format.hpp:395.

encodeTick

[[nodiscard]] inline std::size_t encodeTick(std::span<std::byte> out, const DebugRecord& r) noexcept

Encode one DebugRecord. Returns the bytes written, or 0 if out was too small or the layout did not come out to exactly kTickPayloadBytes (a loud, testable failure rather than a silently short record).

free function, declared at include/shulib/diag/blackbox_format.hpp:440.

safeAngle

[[nodiscard]] inline math::Angle safeAngle(double radians, bool& corrupt) noexcept

Rebuild an Angle from a decoded radian value WITHOUT trusting the file: a corrupt or truncated blackbox can contain any bit pattern, and math::Angle's factory rejects non-finite input by precondition. A decoder that throws on a corrupt file is a decoder you cannot use on the file you most need to read, so a non-finite heading decodes to zero and corrupt is raised for the caller to see.

free function, declared at include/shulib/diag/blackbox_format.hpp:500.

decodeTick

[[nodiscard]] inline bool decodeTick(std::span<const std::byte> in, DebugRecord& r, bool& corrupt) noexcept

Decode one DebugRecord. Returns false if the payload is not exactly kTickPayloadBytes. corrupt is set (never cleared) when a field could not be represented — today: a non-finite heading, which decodes to zero (safeAngle).

free function, declared at include/shulib/diag/blackbox_format.hpp:511.

encodeSummary

[[nodiscard]] inline std::size_t encodeSummary(std::span<std::byte> out, const RunSummary& s, std::uint32_t blackboxDropped) noexcept

Encode one RunSummary. blackboxDropped is the SINK's own drop count, passed in rather than read from the summary so the file always carries the writer's live figure even when the caller assembled the summary before the last drop. Returns the bytes written, or 0 on a layout/space failure.

free function, declared at include/shulib/diag/blackbox_format.hpp:592.

decodeSummary

[[nodiscard]] inline bool decodeSummary(std::span<const std::byte> in, RunSummary& s, std::uint32_t& blackboxDropped) noexcept

Decode one RunSummary; blackboxDropped receives the sink's own drop count. Returns false if the payload is not exactly kSummaryPayloadBytes.

free function, declared at include/shulib/diag/blackbox_format.hpp:623.

encodeTriage

[[nodiscard]] inline std::size_t encodeTriage(std::span<std::byte> out, const TriageInfo& info, const DebugRecord& faultTick) noexcept

Encode the D-7 triage block plus the complete record of the tick the fault fired on. Returns the bytes written, or 0 on a layout/space failure.

free function, declared at include/shulib/diag/blackbox_format.hpp:663.

decodeTriage

[[nodiscard]] inline bool decodeTriage(std::span<const std::byte> in, TriageInfo& info, DebugRecord& faultTick, bool& corrupt) noexcept

Decode a triage frame and the fault tick's record. Returns false if the payload is not exactly kTriagePayloadBytes.

free function, declared at include/shulib/diag/blackbox_format.hpp:688.

encodeEnd

[[nodiscard]] inline std::size_t encodeEnd(std::span<std::byte> out, const EndInfo& e) noexcept

Encode the graceful-end stamp. Its PRESENCE is the signal that the run closed cleanly; its absence is how a reader knows a file was cut short. Returns the bytes written, or 0 on a layout/space failure.

free function, declared at include/shulib/diag/blackbox_format.hpp:711.

decodeEnd

[[nodiscard]] inline bool decodeEnd(std::span<const std::byte> in, EndInfo& e) noexcept

Decode the graceful-end stamp. Returns false if the payload is not exactly kEndPayloadBytes.

free function, declared at include/shulib/diag/blackbox_format.hpp:729.

encodeFrameHeader

[[nodiscard]] inline std::size_t encodeFrameHeader(std::span<std::byte> out, FrameType type, std::uint16_t payloadBytes) noexcept

Write a frame prefix {type, reserved, payloadBytes} into out. Returns the bytes written (kFrameHeaderBytes) or 0 if it did not fit.

free function, declared at include/shulib/diag/blackbox_format.hpp:747.

Design commentary, from the header

The header opens with the reasoning behind these shapes. It is reproduced here in full because a reference that only lists signatures teaches nobody why.

The header’s own reasoning — 52 lines, click to expand 
 The SHULIB BLACKBOX on-disk format, v1 — the binary record SdSink writes and
 BlackboxReader reads (master plan §18; diagnostics-plan D-6/D-7; WS13, chunk E1).

 ── Why a format document lives in code ─────────────────────────────────────────────
 A file outlives the program that wrote it. Six months from now the only thing that
 can open a run from a competition day is a decoder that agrees with this layout, so
 the layout is written down ONCE, here, with its version, and the encoder and the
 decoder are both held to it BY A BYTE-EXACT GOLDEN TEST — not by their agreement
 with each other. Two sides that share one bug agree perfectly; that is precisely the
 failure a round trip cannot see, which is why test/blackbox_format_test.cpp pins the
 literal bytes of the header and of one full tick record.

 ── The shape ───────────────────────────────────────────────────────────────────────
   [ 256-byte file header ][ frame ][ frame ][ frame ] …
 Every frame is  { u8 type, u8 reserved, u16 payloadBytes }  followed by payloadBytes
 of payload. A reader that meets a frame type it does not know SKIPS it by its length
 and counts it — that is what makes a v1 reader survive a v1.1 writer that appended a
 new frame kind, without ever GUESSING at content it does not understand.

 ── Versioned from byte zero, and refusal over misreading ───────────────────────────
 The header carries kFormatVersion. A decoder that mis-reads an old file is worse
 than one that refuses it: a wrong number read confidently sends the 2am investigation
 somewhere false, while a refusal sends it to the git history. So the reader refuses
 any version it was not built for, and ALSO cross-checks the header's self-declared
 record width against its own — that catches the specific human error of changing the
 layout and forgetting to bump the version.

 ── Fixed width, and no narrowing ANYWHERE ──────────────────────────────────────────
 Every field is fixed width and little-endian, written byte by byte (never a struct
 memcpy: padding and ABI are not a file format). Every floating-point field is IEEE-754
 binary64, including the per-wheel arrays and the tick-phase slots, where binary32
 would have saved ~35% of the file. That trade was made deliberately: the chunk's
 central promise is that a decoded record equals the encoded one FIELD BY FIELD, and
 with narrowing that sentence would have quietly become "equals after a documented
 rounding" — a weaker claim that also makes every future comparison of two runs
 depend on rounding behaviour. Bytes are cheap on an SD card; a fuzzy record is not.

 ── What v1 does NOT carry, said plainly ────────────────────────────────────────────
 The log() message channel. Variable-length text in a fixed-width record format is a
 different problem with different trade-offs (bounded-vs-truncated strings, per-line
 framing), and the terminal (A1) already owns text. The blackbox carries the per-tick
 RECORD, the run SUMMARY, and the fault TRIAGE block. SdSink counts the messages it
 was handed and writes that count into the end frame, so the omission is visible in
 the file rather than silent — a reader can always see that N lines existed elsewhere.

 ── What H1 (F9, the SHUL/2 wire) inherits ──────────────────────────────────────────
 This is a PERSISTENCE contract the moment a file exists, but it is deliberately NOT
 the wire: SHUL/2 is streamed, sequenced and versioned on its own terms. What H1
 should reuse is the FIELD ORDER of encodeTick() (it follows debug_record.hpp's own
 declaration order, so a schema append lands at the end on both) and the refuse-don't-
 misread rule. Nothing here freezes anything: F9 is H1's, and E1 freezes nothing.

Last updated 2026-08-15 — published from the main release branch, commit fa545ff. This site tracks releases, so between them it can lag the repository — that lag is deliberate, and it is what keeps the development log off the public site. For the current state of the code, see the repository.