Scoring Claims for SIU Referral Thresholds

This page is a focused implementation guide under SIU Referral Orchestration: it shows how to combine several detector scores into a single, defensible “refer or not” decision using jurisdiction-aware thresholds, hysteresis to stop borderline claims from thrashing, and an expected-value gate that weighs investigation cost against likely recovery.

The orchestrator that governs the referral lifecycle consumes one boolean per claim: refer, or do not. Producing that boolean well is deceptively hard. A claim carries scores from several independent detectors — an outlier claim amount at 0.81, a fuzzy duplicate match at 0.44, an entity-network shared-provider hit at 0.67 — and the naive answer, if max(scores) > 0.8, throws away most of the signal and ignores where the threshold should sit.

Three concrete defects follow from a naive threshold. First, a claim whose fused score oscillates around the line (0.79 today, 0.81 tomorrow as one detector re-scores) gets referred, un-referred, and re-referred, thrashing the SIU queue and the audit log. Second, a single global cutoff ignores that referral rates are audited per jurisdiction for disparate impact, so a threshold that is defensible in one state produces a compliance finding in another. Third, referring a claim whose maximum plausible recovery is 900 dollars costs more to investigate than it can ever return, yet a pure-score threshold refers it anyway. This page fixes all three: a weighted fusion, per-jurisdiction thresholds with a hysteresis band, and an expected-value gate that runs before any candidate is raised.

This pattern targets Python 3.10+ and uses only the standard library plus structlog. All monetary reasoning uses Decimal; all scores are normalized floats in the closed interval 0.0–1.0.

Copy
python -m venv .venv && source .venv/bin/activate
pip install "structlog==24.*"

The scoring layer runs after every detector has emitted a normalized score for the claim and after the claim’s jurisdiction has been resolved from the risk address. It must also read the claim’s prior referral decision (from the last cycle) to apply hysteresis, so the store that holds referral candidates must expose the last fused decision keyed by claim id.

Combine the per-detector scores into one fused score with a weighted sum, normalized by the weights actually present so a missing detector does not silently deflate the result. Weights are configuration, not constants, because the relative trust in each detector shifts as the models are recalibrated from SIU feedback.

Copy
from __future__ import annotations

from dataclasses import dataclass
from decimal import Decimal

import structlog

log = structlog.get_logger("siu.scoring")


@dataclass(frozen=True, slots=True)
class DetectorScore:
    detector: str          # "anomaly" | "duplicate" | "network" | "velocity"
    score: float           # normalized 0.0..1.0
    detail: str            # evidence pointer for the reason code


def fuse_scores(scores: tuple[DetectorScore, ...],
                weights: dict[str, float]) -> float:
    """Weighted mean over the detectors that actually reported."""
    if not scores:
        return 0.0
    num = 0.0
    den = 0.0
    for s in scores:
        if not 0.0 <= s.score <= 1.0:
            raise ValueError(f"{s.detector} score {s.score} out of [0,1]")
        w = weights.get(s.detector, 0.0)
        num += w * s.score
        den += w
    fused = num / den if den else 0.0
    log.info("siu.scoring.fused", fused=round(fused, 4),
             detectors=[s.detector for s in scores])
    return fused

A single cutoff is what causes thrashing. Replace it with two thresholds per jurisdiction: an upper bound to enter the referral zone and a lower bound to exit it. Between the bounds the claim keeps its previous decision. The gap is the hysteresis band, and it is what stops a claim scoring 0.79/0.81 on alternate cycles from flapping.

Copy
@dataclass(frozen=True, slots=True)
class ReferralThresholds:
    jurisdiction: str
    enter: float           # cross above this to become a candidate
    exit: float            # fall below this to drop out
    # invariant: exit < enter, giving a non-empty hysteresis band


def decide_with_hysteresis(fused: float, prior_referred: bool,
                           t: ReferralThresholds) -> bool:
    if t.exit >= t.enter:
        raise ValueError(f"{t.jurisdiction}: exit {t.exit} must be < enter {t.enter}")
    if fused >= t.enter:
        decision = True
    elif fused < t.exit:
        decision = False
    else:
        decision = prior_referred          # inside the band: hold steady
    log.info("siu.scoring.hysteresis", jurisdiction=t.jurisdiction,
             fused=round(fused, 4), enter=t.enter, exit=t.exit,
             prior=prior_referred, decision=decision)
    return decision

A claim can be genuinely suspicious yet not worth investigating: if the recoverable indemnity is smaller than the loaded cost of an investigation, referring it burns SIU capacity for no return. Compute expected value as the fraud probability times the recoverable amount, minus the investigation cost, and require it to be positive. Keep the money in Decimal.

Copy
@dataclass(frozen=True, slots=True)
class ExpectedValueInputs:
    fraud_probability: Decimal     # calibrated 0..1 from the fused score
    recoverable: Decimal           # indemnity at stake, dollars
    investigation_cost: Decimal    # loaded SIU cost per case, dollars


def expected_value(ev: ExpectedValueInputs) -> Decimal:
    return (ev.fraud_probability * ev.recoverable) - ev.investigation_cost


def should_refer(fused: float, prior_referred: bool,
                 thresholds: ReferralThresholds,
                 ev: ExpectedValueInputs) -> bool:
    if not decide_with_hysteresis(fused, prior_referred, thresholds):
        return False
    value = expected_value(ev)
    refer = value > Decimal("0")
    log.info("siu.scoring.ev_gate", expected_value=str(value), refer=refer)
    return refer

When the decision is to refer, turn the contributing detector scores into the reason codes the orchestrator packages. Only detectors that materially contributed — above a floor — become reason codes, so the investigator’s packet is not padded with near-zero noise.

Copy
@dataclass(frozen=True, slots=True)
class ReasonCode:
    detector: str
    code: str
    score: Decimal
    detail: str


_CODE_BY_DETECTOR = {
    "anomaly": "OUTLIER_AMOUNT",
    "duplicate": "DUPLICATE_FNOL",
    "network": "SHARED_ENTITY",
    "velocity": "RAPID_FIRE",
}


def assemble_reason_codes(scores: tuple[DetectorScore, ...],
                          floor: float = 0.35) -> tuple[ReasonCode, ...]:
    codes = tuple(
        ReasonCode(s.detector, _CODE_BY_DETECTOR.get(s.detector, "UNMAPPED"),
                   Decimal(str(round(s.score, 4))), s.detail)
        for s in scores if s.score >= floor
    )
    log.info("siu.scoring.reason_codes", count=len(codes))
    return codes

Prove the three defect classes are closed with explicit tests: hysteresis holds a mid-band claim to its prior decision, the expected-value gate blocks a small-dollar claim, and fusion ignores a missing detector’s weight rather than deflating.

Copy
from decimal import Decimal


def test_hysteresis_holds_in_band() -> None:
    t = ReferralThresholds("CA", enter=0.80, exit=0.70)
    # 0.75 is inside the band: decision must equal the prior decision.
    assert decide_with_hysteresis(0.75, prior_referred=True, t=t) is True
    assert decide_with_hysteresis(0.75, prior_referred=False, t=t) is False


def test_hysteresis_crosses_cleanly() -> None:
    t = ReferralThresholds("CA", enter=0.80, exit=0.70)
    assert decide_with_hysteresis(0.81, prior_referred=False, t=t) is True
    assert decide_with_hysteresis(0.69, prior_referred=True, t=t) is False


def test_ev_gate_blocks_small_dollar_claim() -> None:
    t = ReferralThresholds("CA", enter=0.80, exit=0.70)
    ev = ExpectedValueInputs(fraud_probability=Decimal("0.9"),
                             recoverable=Decimal("900"),
                             investigation_cost=Decimal("1200"))
    # Strongly suspicious but not worth investigating: 0.9*900 - 1200 < 0.
    assert should_refer(0.95, False, t, ev) is False


def test_fusion_ignores_absent_weight() -> None:
    scores = (DetectorScore("anomaly", 0.8, "amt"),
              DetectorScore("velocity", 0.2, "rate"))
    # weight only for anomaly → fused equals the anomaly score exactly.
    assert fuse_scores(scores, {"anomaly": 1.0}) == 0.8

Run the suite with python -m pytest -q. The hysteresis tests are the decisive ones: they assert that identical inputs (0.75) yield opposite decisions purely from the prior state, which is exactly the non-thrashing behavior a single cutoff cannot express.

The reason this decision is engineered as pure functions over the fused score, the jurisdiction thresholds, the prior decision, and the expected-value inputs is that every referral — and every non-referral — becomes reproducible from an audit row alone. Persist the fused score, the ReferralThresholds in force, the prior decision, the expected-value inputs, and the assembled reason codes before the candidate is handed to the SIU referral orchestration layer. Because thresholds are per-jurisdiction and captured in the record, a compliance officer auditing referral rates for disparate impact can reconstruct exactly why a given state’s claims cross the line where they do — the same per-state discipline described in State Regulation Mapping. The expected-value inputs matter here too: they document that a claim was not referred for a cost reason rather than an arbitrary one, which is the defensible answer when an examiner asks why a suspicious small-dollar claim was released.

• Claims flap in and out of referral. Symptom: the same claim is referred, cleared, and re-referred on consecutive cycles. Cause: the enter and exit thresholds are equal, collapsing the hysteresis band to a single cutoff. Fix: enforce exit < enter as an invariant (the code raises if violated) and widen the band until a re-scoring detector’s normal jitter no longer crosses both bounds.
• Referral rate spikes in one state only. Symptom: one jurisdiction’s referral rate jumps after a model deploy. Cause: a global threshold was applied instead of the per-jurisdiction set, so a distribution shift hit every state uniformly. Fix: key ReferralThresholds by jurisdiction and calibrate each independently against that state’s claim history.
• Small-dollar claims flood the SIU. Symptom: investigators close many referrals as not-worth-pursuing. Cause: the expected-value gate is disabled or the investigation cost is understated. Fix: set the loaded per-case cost realistically and require positive expected value before raising a candidate; suspicion alone is not sufficient.
• A strong single signal never triggers a referral. Symptom: a claim with one detector at 0.95 and the rest near zero scores below the enter threshold. Cause: weighted-mean fusion dilutes a lone strong signal. Fix: add a per-detector override so any single detector above a high confidence floor short-circuits to candidate, and record which rule fired in the audit row.
• Fused score deflates when a detector is offline. Symptom: fused scores drop across the board during a detector outage. Cause: the missing detector’s weight is still in the denominator. Fix: normalize only over detectors that actually reported, as fuse_scores does, and alert on the missing detector rather than absorbing it as a lower score.

• SIU Referral Orchestration — the parent guide whose referral machine consumes this decision
• Fraud Pattern Detection — the reference domain these detectors and thresholds belong to
• Statistical Anomaly Scoring — source of the outlier-amount score fused here
• Duplicate Claim Detection — source of the duplicate-FNOL score fused here
• Adjuster Assignment Algorithms — the parallel routing decision that runs while a referral is pending
• State Regulation Mapping — where per-jurisdiction referral thresholds are sourced and defended