Building Append-Only Hash-Chained Audit Logs

This guide is one component of the broader Audit Log Schema Design for Claims Systems guidance: it implements the tamper-evident mechanism that makes an append-only ledger evidentiary, showing exactly how each record binds to the one before it with sha256 and how a verifier detects any alteration after the fact.

An append-only store is not, by itself, tamper-evident. A file you only ever append to still permits someone with write access to rewrite an earlier line, re-serialize the file, and leave no trace — the “append-only” property is a convention, not a proof. For a claims audit trail that must stand up in a market-conduct examination or a coverage dispute, that gap is fatal: an examiner has no cryptographic reason to believe the sequence they are reading is the sequence that actually occurred.

A hash chain closes the gap. Each record stores the sha256 of the canonical serialization of the previous record’s hash concatenated with its own payload. Because every record’s hash depends on its predecessor’s, altering any historical record changes its hash, which invalidates the prev_hash stored in the next record, which changes that record’s hash, and so the break propagates all the way to the tail. A verifier that walks the chain and recomputes each hash detects the earliest point of tampering exactly. The entire scheme rests on one fragile assumption — that the canonical serialization of a record is byte-for-byte identical every time it is computed. Get canonicalization wrong and the chain reports breaks on untouched records; get it right and a single altered cent is detectable years later.

This pattern needs only the Python standard library — hashlib, json, and dataclasses — plus structlog for the audit lines. Pin it so JSON separator and coercion behavior cannot drift between the writer and a verifier that runs years later:

Copy
python -m venv .venv && source .venv/bin/activate
pip install "structlog==24.*"
# hashlib, json, dataclasses, decimal are all stdlib

The upstream state this assumes: events already arrive as the immutable envelope defined in the parent guide, with quantitative fields normalized to Decimal at ingestion. A float reaching the serializer is the single most common cause of a phantom chain break, so it must be excluded before this stage.

Every byte fed to the hash must be reproducible. json.dumps with sorted keys and no insignificant whitespace fixes field order and spacing, but two traps remain: Decimal is not JSON-serializable by default, and if you let it fall back to float you reintroduce platform-dependent reprs. Render Decimal (and any other non-native type) to a stable string in a default hook, and pin the separators explicitly.

Copy
from __future__ import annotations

import json
from decimal import Decimal
from typing import Any


def _stable_default(value: Any) -> str:
    """Render non-JSON-native types to a deterministic string.

    Decimal → its canonical string ('6550.00', never 6550.0 as float),
    everything else that reaches here is a hard error, not a silent str().
    """
    if isinstance(value, Decimal):
        return format(value, "f")  # fixed-point, preserves trailing cents
    raise TypeError(f"non-canonicalizable type in audit payload: {type(value)!r}")


def canonical_bytes(payload: dict[str, Any]) -> bytes:
    """Deterministic UTF-8 encoding of a payload for hashing.

    Sorted keys and tight separators guarantee byte-identical output for
    logically-equal payloads regardless of construction order.
    """
    return json.dumps(
        payload,
        sort_keys=True,
        separators=(",", ":"),
        ensure_ascii=False,
        default=_stable_default,
    ).encode("utf-8")

Each record is an immutable dataclass carrying its sequence number, the payload, and the prev_hash of its predecessor. The record’s own hash is sha256 over the previous hash concatenated with the canonical payload bytes — the concatenation is what binds the record to its position, not just its content.

Copy
from dataclasses import dataclass, field
from datetime import datetime, timezone
import hashlib
from typing import Any, Final

import structlog

log = structlog.get_logger("audit.chain")

GENESIS_HASH: Final[str] = "0" * 64


@dataclass(frozen=True, slots=True)
class ChainedRecord:
    seq: int
    occurred_at: str                 # UTC ISO 8601
    prev_hash: str
    payload: dict[str, Any]

    def compute_hash(self) -> str:
        material = self.prev_hash.encode("ascii") + canonical_bytes(self.payload)
        return hashlib.sha256(material).hexdigest()


def append_record(
    tail_hash: str, seq: int, payload: dict[str, Any]
) -> tuple[ChainedRecord, str]:
    """Seal a new record against the current tail and return it with its hash."""
    record = ChainedRecord(
        seq=seq,
        occurred_at=datetime.now(timezone.utc).isoformat(),
        prev_hash=tail_hash,
        payload=payload,
    )
    record_hash = record.compute_hash()
    log.info("audit.sealed", seq=seq, prev=tail_hash[:12], record_hash=record_hash[:12])
    return record, record_hash

Verification recomputes each record’s hash from its stored payload and checks two things per step: that the record’s prev_hash equals the previous record’s computed hash, and that its own recomputed hash matches what storage claims. The first mismatch is the tamper point. The verifier returns it explicitly rather than a bare boolean so an examiner learns where the chain broke.

Copy
from dataclasses import dataclass
from typing import Iterable, Optional


@dataclass(frozen=True, slots=True)
class VerifyResult:
    ok: bool
    broken_at_seq: Optional[int] = None
    reason: str = ""


def verify_chain(records: Iterable[ChainedRecord]) -> VerifyResult:
    """Walk records in order; return the first point the chain fails, if any."""
    expected_prev = GENESIS_HASH
    for rec in records:
        if rec.prev_hash != expected_prev:
            return VerifyResult(False, rec.seq, "prev_hash does not match preceding record")
        recomputed = rec.compute_hash()
        # In a real store you also compare against the persisted record_hash;
        # here the recomputed value becomes the link the next record must cite.
        expected_prev = recomputed
    return VerifyResult(True)

The decisive test is that tampering is detected, at the right place. Build a short chain, mutate one record’s payload, and assert the verifier flags exactly that sequence number. A second test proves a clean chain verifies, guarding against a verifier that rejects everything.

Copy
from decimal import Decimal


def _build_chain() -> list[ChainedRecord]:
    tail = GENESIS_HASH
    records: list[ChainedRecord] = []
    payloads = [
        {"action": "record.validated", "passed": True},
        {"action": "deductible.applied", "threshold": Decimal("6550.00"),
         "verdict": "CLEARS"},
        {"action": "referral.raised", "score": Decimal("0.87")},
    ]
    for seq, p in enumerate(payloads):
        rec, tail = append_record(tail, seq, p)
        records.append(rec)
    return records


def test_clean_chain_verifies() -> None:
    assert verify_chain(_build_chain()).ok is True


def test_tamper_is_detected_at_source() -> None:
    records = _build_chain()
    # Forge the settled deductible on record 1 without re-linking record 2.
    tampered = ChainedRecord(
        seq=records[1].seq,
        occurred_at=records[1].occurred_at,
        prev_hash=records[1].prev_hash,
        payload={**records[1].payload, "threshold": Decimal("0.00")},
    )
    records[1] = tampered
    result = verify_chain(records)
    assert result.ok is False
    assert result.broken_at_seq == 2   # record 2's back-link no longer matches


def test_decimal_canonicalization_is_stable() -> None:
    a = canonical_bytes({"threshold": Decimal("6550.00")})
    b = canonical_bytes({"threshold": Decimal("6550.00")})
    assert a == b
    assert b"6550.00" in a          # trailing cents preserved, never 6550.0

Run the suite with python -m pytest -q. The tamper test is the one that matters: because record 1’s forged payload changes its hash, the prev_hash stored in record 2 no longer matches, and the verifier pinpoints seq 2 as the first break — proof that no downstream edit can conceal an upstream one.

A hash-chained ledger is what turns “we log everything” into evidence a regulator will accept. Under NAIC market-conduct examination standards, an insurer must demonstrate not merely that a decision was recorded but that the record is the original and unaltered. The chain provides exactly that assurance: any post-hoc modification to a settled figure, a coverage verdict, or a fraud referral is detectable and localizable, so the ledger’s integrity is a property that can be proven rather than asserted. Seal the terminal hash of each retention partition into a separate register before the partition is archived, and the entire history remains verifiable even after older partitions are moved to cold storage — satisfying market-conduct traceability without keeping the full chain hot indefinitely. Because records reference subjects by token rather than embedding personal data, the chain stays verifiable without turning the immutable ledger into a store of regulated PII.

• Phantom breaks on untouched records. Symptom: verification fails on records no one edited. Cause: non-deterministic serialization — a float leaked in, or two producers ordered keys differently. Fix: route every hash through canonical_bytes with sort_keys=True and the Decimal-to-string default hook; forbid float in payloads upstream.
• Chain verifies but should not. Symptom: a known-altered record passes. Cause: the verifier compared stored hashes to each other instead of recomputing from the payload. Fix: always recompute compute_hash() from the record’s own payload and feed that forward as the expected prev_hash.
• Genesis mismatch. Symptom: the very first record fails verification. Cause: the writer seeded prev_hash with an empty string or None while the verifier expects the 64-zero sentinel. Fix: standardize on a single GENESIS_HASH constant shared by writer and verifier.
• Trailing-cent loss. Symptom: Decimal("6550.00") and Decimal("6550.0") hash differently after a round-trip. Cause: a serializer normalized the scale. Fix: render with format(value, "f") and never pass currency through float or str(Decimal(...).normalize()).
• Retention delete breaks history. Symptom: verification fails after records expire. Cause: a row-level delete inside the chain. Fix: partition by retention window, seal each partition’s terminal hash into a register, and expire whole partitions rather than individual records.

• Audit Log Schema Design for Claims Systems — the parent guide defining the envelope and writer this chain secures
• Data Boundary Enforcement — where tokenized subject references resolve so the chain carries no raw PII
• Policy Schema Design — the versioned contracts and PII classification the payloads are drawn from
• SIU Referral Orchestration — a producer whose referral decisions become chained, tamper-evident records
• Core Architecture & Compliance Mapping — the platform this audit ledger underpins