SIU Referral Orchestration for Suspicious Claims

SIU referral orchestration is the governed bridge between fraud detection and a Special Investigations Unit: it converts fused fraud signals into auditable, deduplicated referrals without ever stalling or denying a claim on suspicion alone. This guide sits under the Fraud Pattern Detection reference and consumes the scores produced by its sibling detectors — it is the stage where machine suspicion becomes a human, regulated decision.

Every fraud program eventually produces scores. The statistical anomaly scoring models flag claim amounts that sit far outside their peer distribution, duplicate claim detection surfaces the same loss submitted twice under different first-notice-of-loss records, and entity network analysis exposes the repair shop wired to forty unrelated claimants. What almost no early fraud program has is a disciplined answer to the next question: given these signals, what happens to the claim, and who is allowed to know?

Left unmanaged, the referral step is where a technically sound fraud program becomes a legal liability. Three failure patterns recur across carriers. First, suspicion leaks into the settlement path: an adjuster sees a high fraud score, quietly slows the claim, and the carrier has now committed an unfair-claims-settlement violation because a statutory clock kept running while the claim sat. Second, referrals duplicate and thrash: the same claim crosses the threshold on Monday from the anomaly model, again on Tuesday from the duplicate detector, and the SIU queue fills with three copies of one case, each with a different rationale. Third, nothing closes the loop: investigators substantiate or clear cases, but those outcomes never flow back to recalibrate the models, so the detectors keep firing on the same false-positive shapes forever.

Orchestration solves all three by making the referral a first-class, stateful object with an explicit lifecycle, hard transition guards, and a fail-open contract with the claims pipeline. The claim continues down its normal path — coverage validation, severity scoring, adjuster assignment — regardless of what the SIU does. The referral is a parallel investigative track, never a gate on payment.

Model a referral as a finite state machine, not a boolean flag. A claim that “looks fraudulent” moves through a sequence of states, each of which is owned by a different actor and carries different regulatory obligations. The states are: candidate (a detector or the scoring layer proposes a referral), referred (the orchestrator has packaged it and handed it to the SIU), accepted or declined (an investigator triages the packet and either opens a case or bounces it back), and finally substantiated or cleared (the investigation reaches a documented conclusion). Only the two terminal states carry a fraud determination; everything before them is a suspicion under review, and the claim is paid or denied on its own merits meanwhile.

The value of the machine is that illegal or careless transitions become impossible to express. There is no code path from candidate straight to a denial. There is no way to reach substantiated without passing through referred and accepted, which means there is always an investigator and a case file behind any fraud determination. And because every transition is a single guarded method, every transition is a natural audit point.

This orchestrator targets Python 3.10+ and leans on the standard library plus structlog for audit-grade structured logging. Pin the versions so transition-log formatting and enum semantics cannot drift:

Copy
python -m venv .venv && source .venv/bin/activate
pip install "structlog==24.*" "pydantic==2.*"

Two upstream contracts must be in place before this stage runs. The fraud detectors must emit scores on a normalized 0.0–1.0 scale with a stable detector identifier, and the persistence layer must expose an append-only audit sink — ideally the same hash-chained ledger described in Audit Log Schema Design. The orchestrator writes but never mutates: a referral row’s history is the source of truth for any later market-conduct examination.

The orchestrator’s job is to hold a referral’s state and expose exactly the transitions the lifecycle permits. Each transition is a method with an explicit guard; an illegal transition raises rather than silently no-ops. Model the states as an enum, the allowed edges as a frozen mapping, and the referral itself as a mutable aggregate whose only entry points are the guarded methods.

Copy
from __future__ import annotations

from dataclasses import dataclass, field
from datetime import datetime, timezone
from decimal import Decimal
from enum import Enum
from typing import Callable

import structlog

log = structlog.get_logger("siu.orchestrator")


class ReferralState(str, Enum):
    CANDIDATE = "candidate"
    REFERRED = "referred"
    ACCEPTED = "accepted"
    DECLINED = "declined"
    SUBSTANTIATED = "substantiated"
    CLEARED = "cleared"


# The only legal edges. Anything not listed here is unreachable by construction.
_TRANSITIONS: dict[ReferralState, frozenset[ReferralState]] = {
    ReferralState.CANDIDATE: frozenset({ReferralState.REFERRED}),
    ReferralState.REFERRED: frozenset({ReferralState.ACCEPTED, ReferralState.DECLINED}),
    ReferralState.ACCEPTED: frozenset({ReferralState.SUBSTANTIATED, ReferralState.CLEARED}),
    ReferralState.DECLINED: frozenset(),          # terminal: SIU bounced triage
    ReferralState.SUBSTANTIATED: frozenset(),     # terminal
    ReferralState.CLEARED: frozenset(),           # terminal
}

_TERMINAL = frozenset({
    ReferralState.DECLINED,
    ReferralState.SUBSTANTIATED,
    ReferralState.CLEARED,
})


class IllegalTransition(Exception):
    """Raised when a caller attempts an edge the lifecycle does not allow."""

    def __init__(self, frm: ReferralState, to: ReferralState) -> None:
        super().__init__(f"illegal referral transition {frm.value} -> {to.value}")
        self.frm, self.to = frm, to


@dataclass(frozen=True, slots=True)
class ReasonCode:
    """A single machine-readable justification for the referral."""
    detector: str          # e.g. "entity_network"
    code: str              # e.g. "SHARED_REPAIR_SHOP"
    score: Decimal         # normalized 0..1 contribution
    detail: str            # human-readable evidence pointer


@dataclass(frozen=True, slots=True)
class TransitionEvent:
    frm: ReferralState
    to: ReferralState
    actor: str             # "orchestrator" | "siu:jjones" | "system"
    at: datetime
    note: str


@dataclass(slots=True)
class Referral:
    referral_id: str
    claim_id: str
    dedupe_key: str
    reason_codes: tuple[ReasonCode, ...]
    state: ReferralState = ReferralState.CANDIDATE
    history: list[TransitionEvent] = field(default_factory=list)

    def _transition(self, to: ReferralState, *, actor: str, note: str) -> None:
        if to not in _TRANSITIONS[self.state]:
            raise IllegalTransition(self.state, to)
        event = TransitionEvent(self.state, to, actor, _now(), note)
        self.history.append(event)
        log.info(
            "siu.referral.transition",
            referral_id=self.referral_id,
            claim_id=self.claim_id,
            frm=self.state.value,
            to=to.value,
            actor=actor,
            note=note,
        )
        self.state = to

    @property
    def is_terminal(self) -> bool:
        return self.state in _TERMINAL


def _now() -> datetime:
    return datetime.now(timezone.utc)

The guarded methods live on the orchestrator rather than the aggregate, because each transition is also where side effects — packaging, audit emission, bureau reporting — belong. Crucially, the constructor of a referral is the only place that touches the claims pipeline, and it touches it in exactly one way: it does nothing to it. Payment routing is never a parameter here.

Copy
class ReferralAudit:
    """Append-only sink. Real implementations chain hashes; see audit-log-schema-design."""

    def record(self, referral: Referral, event: TransitionEvent) -> None:
        raise NotImplementedError


class SIUOrchestrator:
    def __init__(self, audit: ReferralAudit, *, open_referrals: "OpenReferralIndex") -> None:
        self._audit = audit
        self._open = open_referrals

    def refer(self, referral: Referral, *, packager: "CasePackager") -> Referral | None:
        """Move CANDIDATE -> REFERRED, unless an open duplicate already exists."""
        existing = self._open.find(referral.dedupe_key)
        if existing is not None:
            # Merge new reason codes into the live referral instead of forking a case.
            existing.reason_codes = _merge_codes(existing.reason_codes, referral.reason_codes)
            log.info("siu.referral.deduped", kept=existing.referral_id,
                     dropped=referral.referral_id, claim_id=referral.claim_id)
            return None
        packager.attach(referral)                     # evidence + explanation packet
        referral._transition(ReferralState.REFERRED, actor="orchestrator",
                             note="packaged and queued to SIU")
        self._open.add(referral)
        self._audit.record(referral, referral.history[-1])
        return referral

    def accept(self, referral: Referral, *, investigator: str) -> None:
        referral._transition(ReferralState.ACCEPTED, actor=f"siu:{investigator}",
                             note="case opened")
        self._audit.record(referral, referral.history[-1])

    def decline(self, referral: Referral, *, investigator: str, reason: str) -> None:
        referral._transition(ReferralState.DECLINED, actor=f"siu:{investigator}",
                             note=reason)
        self._open.remove(referral.dedupe_key)
        self._audit.record(referral, referral.history[-1])

    def resolve(self, referral: Referral, *, investigator: str,
                substantiated: bool, findings: str) -> None:
        target = ReferralState.SUBSTANTIATED if substantiated else ReferralState.CLEARED
        referral._transition(target, actor=f"siu:{investigator}", note=findings)
        self._open.remove(referral.dedupe_key)
        self._audit.record(referral, referral.history[-1])


def _merge_codes(a: tuple[ReasonCode, ...],
                 b: tuple[ReasonCode, ...]) -> tuple[ReasonCode, ...]:
    seen = {(c.detector, c.code): c for c in a}
    for c in b:
        seen.setdefault((c.detector, c.code), c)
    return tuple(seen.values())

The refer method carries the deduplication logic, and it is deliberately the busiest method in the class. When a second detector fires on a claim that already has a live referral, the orchestrator does not create a new case — it merges the new reason codes into the existing referral so the investigator sees one enriched packet instead of three thin ones. The entity network analysis hit and the anomaly-score hit end up as two reason codes on one referral, which is exactly how an investigator wants to read them.

A referral is only as useful as the packet the investigator opens. The packager assembles a self-contained case file: the claim snapshot as of referral time, the reason codes with their evidence pointers, and a plain-language explanation of why the system referred. That explanation is not decoration — under many state fraud statutes the carrier must be able to articulate the “reasonable belief” that justified the referral, and a bag of raw model scores does not meet that bar.

Copy
@dataclass(slots=True)
class CasePacket:
    referral_id: str
    claim_snapshot: dict[str, object]
    reason_codes: tuple[ReasonCode, ...]
    narrative: str
    aggregate_score: Decimal


class CasePackager:
    def __init__(self, snapshot_reader: Callable[[str], dict[str, object]]) -> None:
        self._read = snapshot_reader

    def attach(self, referral: Referral) -> CasePacket:
        snapshot = self._read(referral.claim_id)
        agg = sum((c.score for c in referral.reason_codes), Decimal("0"))
        packet = CasePacket(
            referral_id=referral.referral_id,
            claim_snapshot=snapshot,
            reason_codes=referral.reason_codes,
            narrative=self._explain(referral.reason_codes),
            aggregate_score=agg,
        )
        log.info("siu.packet.built", referral_id=referral.referral_id,
                 codes=len(referral.reason_codes), aggregate=str(agg))
        return packet

    @staticmethod
    def _explain(codes: tuple[ReasonCode, ...]) -> str:
        lines = [f"- {c.code} ({c.detector}, score {c.score}): {c.detail}" for c in codes]
        return "Referred on the following converging indicators:\n" + "\n".join(lines)

The two levers that matter most — where to set the referral threshold and how much score churn to tolerate — belong in configuration, not code, and they are jurisdiction-aware. A percentage-of-claims referral target that is reasonable in one state may collide with anti-discrimination review in another, so the scoring layer reads its thresholds per jurisdiction. That threshold logic, including hysteresis and expected-value gating, is involved enough to warrant its own treatment in Scoring Claims for SIU Referral Thresholds; the orchestrator here simply consumes the boolean decision that layer produces.

Keep three settings environment-driven: the dedupe window (how long a closed referral suppresses a re-referral of the same claim), the carrier-specific reason-code allow-list (some carriers cannot act on certain protected-class-correlated signals at all), and the SIU capacity ceiling that caps daily referrals so a model drift event cannot flood the unit. When capacity is hit, excess candidates are held, not dropped — they stay in candidate state with an audit note and re-enter the queue the next cycle, so nothing is silently lost.

Referral orchestration is where fraud engineering meets three hard regulatory surfaces, and each maps to a concrete design constraint.

Fair-claims-settlement timelines. Every state’s unfair-claims-practices act sets clocks — acknowledge within N days, decide within M days. A referral must never pause those clocks. The fail-open design is not a nicety; it is the mechanism that keeps the carrier compliant while an investigation runs. If the investigation needs more time than the statute allows, the correct action is a documented, statute-compliant extension notice to the insured, issued by the claims side — never a silent hold driven by the fraud score.

Anti-discrimination. Referral rates are audited for disparate impact. The reason-code allow-list and the jurisdiction-aware thresholds exist so that the basis of every referral is inspectable and defensible. Because each referral carries its reason codes into the audit ledger, a compliance officer can query “what fraction of referrals in this state rest solely on indicator X” and act on the answer. This is also why the narrative is mandatory: an unexplained referral cannot be defended.

Mandatory fraud-bureau reporting. Most states require carriers to report substantiated fraud to a state fraud bureau or the NAIC, often on a prescribed form and timeline. That obligation attaches to exactly one transition — the move into substantiated — which is why bureau reporting hooks there and nowhere else. The specific form, recipient, and deadline vary by state; sourcing those rules is the job of the State Regulation Mapping reference, and the orchestrator looks them up rather than hardcoding them.

Every transition event is written to the append-only ledger with the actor, the timestamp, and the note, so the full life of a referral — including a decline or a clear — is reconstructable years later. A cleared referral is as important to retain as a substantiated one: it is the evidence that the carrier investigated fairly and released the claim.

The terminal states are labels. A substantiated referral is a confirmed positive; a cleared referral is a confirmed false positive; a declined referral is a weaker signal that the SIU did not consider the packet worth opening. Emit all three back to the detectors so the models learn. This closes the loop that most fraud programs leave open, and it is what keeps the false-positive rate from ossifying.

Copy
@dataclass(frozen=True, slots=True)
class CalibrationLabel:
    claim_id: str
    reason_codes: tuple[ReasonCode, ...]
    outcome: ReferralState        # SUBSTANTIATED | CLEARED | DECLINED
    labeled_at: datetime


def emit_label(referral: Referral, sink: Callable[[CalibrationLabel], None]) -> None:
    if not referral.is_terminal:
        raise IllegalTransition(referral.state, referral.state)
    sink(CalibrationLabel(referral.claim_id, referral.reason_codes,
                          referral.state, _now()))
    log.info("siu.calibration.labeled", claim_id=referral.claim_id,
             outcome=referral.state.value)

Feeding these labels back is what lets the anomaly and duplicate detectors sharpen over time; it is the same discipline that Dynamic Threshold Tuning applies to severity routing, applied here to fraud scoring.

• Referral silently blocks a payment. Symptom: claims with open referrals show elevated cycle time. Cause: a downstream consumer read the referral state and gated settlement on it. Fix: the referral store must not be readable from the payment path; expose referral state only to the SIU and compliance surfaces, and assert the fail-open contract in an integration test that pays a claim with a live referral attached.
• Duplicate cases for one claim. Symptom: the SIU queue shows three packets for the same claim from three detectors. Cause: the dedupe key is too specific (it includes the detector id). Fix: key deduplication on the claim identity and loss event, not on the signal that triggered it, so all detectors collapse into one enriched referral.
• Substantiated case never reported to the bureau. Symptom: an examiner finds confirmed fraud with no filed report. Cause: the reporting hook was wired to case closure generally instead of the substantiated transition specifically. Fix: attach bureau reporting to the resolve(substantiated=True) edge and make the report emission part of the same audited transition, so an unreported substantiation is impossible.
• Referral storm after a model deploy. Symptom: referrals spike 20x overnight. Cause: a recalibrated detector shifted its score distribution and the threshold did not move with it. Fix: enforce the SIU capacity ceiling (candidates queue rather than flood), alert on referral-rate deltas, and treat the spike as a calibration incident feeding back through the label channel, not as a real fraud wave.
• Cleared claim re-referred immediately. Symptom: the same claim bounces back into candidate the day after it was cleared. Cause: no dedupe window on terminal referrals. Fix: suppress re-referral of a recently cleared claim for the configured window unless materially new evidence (a new reason code) arrives.

The one part of this pipeline deliberately factored out above is the decision itself: given a bag of detector scores, do we cross the line into a referral, and can we defend where that line sits? Scoring Claims for SIU Referral Thresholds works through combining detector scores into a single referral decision with jurisdiction-aware thresholds, hysteresis to stop borderline claims from thrashing across the line, and expected-value gating that weighs investigation cost against likely recovery before a candidate is ever raised. Read it next — it produces the boolean this orchestrator consumes.

• Fraud Pattern Detection — the parent reference this orchestration stage belongs to
• Scoring Claims for SIU Referral Thresholds — the threshold decision that raises the candidates this machine governs
• Entity Network Analysis — a primary source of the converging signals a referral packages
• Statistical Anomaly Scoring — supplies the outlier scores that become reason codes
• Duplicate Claim Detection — the detector whose hits most often merge into an existing referral
• State Regulation Mapping — where per-state fraud-bureau reporting and settlement-timeline rules are sourced