Automated Severity Scoring Models for Claims Triage

Automated severity scoring models translate raw First Notice of Loss (FNOL) telemetry into quantified risk signals that drive every routing decision downstream. For InsurTech developers, claims analysts, and compliance officers, this component is where a noisy intake event either becomes a defensible, reproducible severity tier — or a black-box number that no examiner can reconstruct. Unlike an experimental notebook, a production severity engine runs inside tight latency windows, handles malformed payloads without corrupting state, and must explain itself well enough to satisfy a state insurance commissioner during a market-conduct exam.

This component is one stage of the broader Claims Triage & Routing Engines control plane; treat the patterns here as the scoring node that sits between coverage gating and resource assignment, consuming validated claim events and emitting the severity tier that sets the boundary between the straight-through and senior-adjuster pathways.

At pilot volume, calling a trained model on a clean payload and returning its probability looks like it works. At production volume — tens of thousands of daily loss events across multiple jurisdictions, spiking an order of magnitude during a catastrophe window — three failure classes emerge that a naive model.predict() call cannot survive.

The first is silent feature corruption. A telematics feed starts sending estimated_damage_amount in cents instead of dollars, or a legacy core system emits a null prior_claims_count that gets coerced to zero. The model returns a confident, plausible-looking score on garbage input, and a total-loss claim routes straight through to automated settlement. Without strict schema enforcement and range constraints at the boundary, bad data is indistinguishable from good data by the time it reaches inference.

The second is untraceable decisions. When a regulator asks why claim CLM-00481923 was scored 0.91 and pulled out of the standard queue, an engine that logged only the final float cannot answer. Reproducing the decision requires the exact input payload, the feature transformations applied, the model version, and any business rules that modified the raw score — all captured at decision time, not reconstructed after the fact.

The third is calibration drift under load. A model trained on last year’s loss distribution slowly decalibrates as weather patterns, repair costs, and fraud tactics shift. Scores creep upward, the share of claims hitting the senior pathway climbs, loss adjustment expense follows, and nothing throws an error. Drift is silent by definition; it must be measured continuously or it is not managed at all.

A disciplined scoring layer treats inference as a stateful, auditable component with explicit validation boundaries, isolated business rules, and an immutable decision trail — not a black-box utility bolted onto the FNOL handler.

This component targets Python 3.10+ for modern type-hint syntax and structural pattern matching. Pin the modeling and validation libraries explicitly so a minor version bump cannot silently change calibration or validation semantics:

Copy
python -m venv .venv && source .venv/bin/activate
pip install "pydantic==2.*" "numpy==1.26.*" "scikit-learn==1.4.*" "joblib==1.3.*"

Infrastructure dependencies:

• A message broker (Kafka with idempotent producers, or SQS FIFO) delivering canonical FNOL events so scoring consumes from the same validated stream the rest of the triage engine uses, never directly from raw intake.
• Object or model storage holding versioned, immutable model artifacts keyed by version string, so any historical score can be re-derived against the exact estimator that produced it.
• An append-only audit store — the same governance backbone described in the Core Architecture & Compliance Mapping domain — into which every decision record is written before the routing action is published.

Establish two environment-driven thresholds before writing any scoring logic: the senior-pathway boundary (SEVERITY_SENIOR_THRESHOLD, default 0.85) and the uncertainty-band half-width that diverts ambiguous scores to manual review (SEVERITY_UNCERTAINTY_BAND, default 0.05). These are tuned per line of business, never hard-coded into the inference call.

The central design decision is to keep the probabilistic model and the deterministic business rules in separate stages. The model contributes a calibrated base score; explicit, version-controlled rules then apply coverage boundaries, jurisdictional caps, and policy exclusions after inference. This separation is not stylistic — it is what makes the engine auditable. A regulator can review the rule layer as plain decision-as-code without needing to interpret a gradient-boosted estimator, and applying modifiers post-inference prevents training-data leakage that would corrupt the model’s calibration.

Data flows through four isolation boundaries. Validated FNOL events enter the ingestion gate, which rejects anything that fails the schema contract. Survivors pass to feature extraction, a stateless, idempotent transform that is versioned independently of the model so features can evolve without retraining. Extracted vectors reach the inference stage, which loads a pinned model artifact and produces a raw probability. Finally the rule-modifier stage applies deterministic adjustments and clamps the result into the valid band. Every stage emits a structured audit fragment keyed by claim_id, and only the rule-modifier stage may produce the final severity tier that feeds the Adjuster Assignment Algorithms engine.

Because severity is consumed by both Coverage Validation Rules upstream and Dynamic Threshold Tuning downstream, the scoring stage must treat its output as an immutable contract: a severity score, a confidence interval, the model version, and the list of rules applied. Changing that shape is a breaking change for the whole routing tree.

The reliability of any severity model is bounded by its ingestion layer. Claims and policy data arrive from telematics APIs, adjuster mobile applications, third-party repair networks, and legacy core systems. Malformed payloads must be quarantined rather than silently coerced, so the boundary uses strict Pydantic validation with explicit range constraints before any feature touches the model.

Copy
import logging
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field, ValidationError, field_validator

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
)
logger = logging.getLogger("insurtech.severity.ingest")


class FNOLPayload(BaseModel):
    claim_id: str = Field(pattern=r"^CLM-\d{8,12}$")
    policy_number: str
    loss_date: str
    incident_type: str
    estimated_damage_amount: Optional[float] = Field(None, ge=0.0)
    vehicle_make: Optional[str] = None
    policy_state: str = Field(min_length=2, max_length=2)
    prior_claims_count: int = Field(ge=0)

    @field_validator("loss_date")
    @classmethod
    def parse_iso_date(cls, v: str) -> str:
        datetime.fromisoformat(v)  # raises ValueError on malformed dates
        return v


def ingest_fnol_payload(raw_data: dict) -> dict:
    """Validate a raw FNOL event; raise on structural failure so the caller
    can route the message to the dead-letter queue rather than scoring garbage."""
    try:
        validated = FNOLPayload.model_validate(raw_data)
        return validated.model_dump(mode="json")
    except ValidationError as e:
        logger.error("Schema validation failed", extra={"errors": e.errors()})
        raise ValueError("Invalid FNOL payload structure") from e

This validation boundary guarantees that downstream scoring functions receive only type-safe, range-constrained inputs. A claim that fails here never reaches inference; it routes to the dead-letter queue for forensic replay, exactly as a fatal structural anomaly does elsewhere in the triage engine.

Once validated, raw payloads are transformed into model-ready features. Production feature engineering must be stateless, idempotent, and version-controlled for regulatory audits. Common transformations include temporal decay weighting, categorical encoding, and missing-value imputation using policy-level defaults rather than dataset-wide means — imputing against a global mean leaks aggregate information into individual claims and is hard to defend in an exam.

Copy
import numpy as np
from datetime import datetime, timezone
from typing import Any, Dict

FEATURE_VERSION = "features_v2.1"


def extract_severity_features(payload: Dict[str, Any]) -> Dict[str, float]:
    """Pure, versioned transform from a validated payload to a feature map.
    No I/O, no global state — identical input always yields identical output."""
    features: Dict[str, float] = {}

    # Temporal decay: older incidents typically carry lower immediate severity
    loss_dt = datetime.fromisoformat(payload["loss_date"])
    if loss_dt.tzinfo is None:
        loss_dt = loss_dt.replace(tzinfo=timezone.utc)
    days_since_loss = (datetime.now(timezone.utc) - loss_dt).days
    features["temporal_decay_weight"] = float(max(0.1, np.exp(-days_since_loss / 30.0)))

    # Prior claims, capped to limit outlier distortion
    features["prior_claims_normalized"] = min(payload["prior_claims_count"], 5) / 5.0

    # Log-transformed damage estimate; fall back to a policy-level baseline
    raw_damage = payload["estimated_damage_amount"]
    features["log_damage"] = float(np.log1p(raw_damage if raw_damage is not None else 2500.0))

    # Incident-type base severity
    incident_map = {"collision": 1.0, "comprehensive": 0.7, "liability": 0.9}
    features["incident_severity_base"] = incident_map.get(
        payload["incident_type"].lower(), 0.5
    )

    return features

Pinning a FEATURE_VERSION alongside the model version lets you evolve transformations without retraining, while still reconstructing exactly which feature logic produced any historical score.

The inference stage loads a pre-calibrated estimator and produces a raw probability; deterministic modifiers then apply jurisdictional caps and clamp the value into the valid band. Isotonic or Platt calibration on the underlying classifier matters here — an uncalibrated gradient-boosted score is a ranking, not a probability, and routing thresholds expressed as probabilities will behave unpredictably against it.

Copy
import os
import joblib
from typing import Any, Dict

# Pinned, immutable artifact loaded once at process start
SEVERITY_MODEL = joblib.load(os.environ["SEVERITY_MODEL_PATH"])
MODEL_VERSION = "severity_v3.1.2"
CAPPED_STATES = {"CA", "NY", "FL"}  # high-litigation jurisdictions, externalized


def compute_severity_score(features: Dict[str, float], policy_state: str) -> Dict[str, Any]:
    feature_vector = [[
        features["temporal_decay_weight"],
        features["prior_claims_normalized"],
        features["log_damage"],
        features["incident_severity_base"],
    ]]

    raw_score = float(SEVERITY_MODEL.predict_proba(feature_vector)[0][1])
    rules_applied: list[str] = []

    # Regulatory routing cap for select high-litigation states, applied
    # AFTER inference so it never contaminates model calibration.
    if policy_state in CAPPED_STATES:
        if raw_score > 0.85:
            rules_applied.append("state_cap_enforcement")
        raw_score = min(raw_score, 0.85)

    final_score = max(0.01, min(0.99, raw_score))
    band = float(os.environ.get("SEVERITY_UNCERTAINTY_BAND", "0.05"))

    return {
        "severity_score": round(final_score, 4),
        "confidence_interval": (
            round(max(0.0, final_score - band), 4),
            round(min(1.0, final_score + band), 4),
        ),
        "feature_version": FEATURE_VERSION,
        "model_version": MODEL_VERSION,
        "rules_applied": rules_applied,
    }

Modifiers are applied post-inference for two reasons: it keeps the model calibration intact, and it makes the rule layer reviewable in isolation during dispute resolution.

Severity scores are routing signals, not end products. A thin, deterministic selector maps the scored claim to a queue, deferring to the configured senior-pathway threshold and a fraud-hold rule for repeat claimants.

Copy
import os


def determine_routing_action(severity_score: float, prior_claims: int) -> str:
    senior_threshold = float(os.environ.get("SEVERITY_SENIOR_THRESHOLD", "0.85"))
    if severity_score >= senior_threshold:
        return "high_severity_specialist_queue"
    if prior_claims >= 3:
        return "fraud_investigation_hold"
    if severity_score >= 0.60:
        return "standard_adjuster_pool"
    return "automated_stp_processing"

When this action feeds the Adjuster Assignment Algorithms engine, the severity tier becomes one input to capacity-aware dispatch across appropriately licensed specialists.

Threshold calibration is the highest-leverage tuning surface in a severity pipeline, and a single global threshold is wrong for every line of business. An auto-glass book may justify a 0.88 senior boundary, while a commercial-liability book needs 0.78 because the tail risk of under-routing is far higher. Drive these from a version-controlled manifest, never from code:

Every threshold change is itself an auditable event: the manifest is committed, the change is logged with an owner, and the model version it pairs with is recorded. Score-distribution drift is monitored with the Population Stability Index (PSI) comparing the live scoring distribution against the training reference; when PSI crosses the configured floor, the pipeline raises a recalibration alert rather than letting a decalibrated model keep routing. This mirrors the staleness-detection discipline in Dynamic Threshold Tuning, where routing boundaries adapt to seasonal surges and catastrophe load.

Every severity output must map to a regulatory requirement. State insurance departments require explicit documentation of how automated decisions affect claim handling, and an engine that cannot reconstruct a decision is an unfair-claims-practice finding waiting to happen. Structured audit records — aligned with the NIST AI Risk Management Framework — capture the input payload hash, the feature transformations, the model version, and every business rule applied.

Copy
import json
import uuid
import hashlib
from datetime import datetime, timezone
from typing import Any, Dict


def generate_audit_record(
    claim_id: str,
    payload: Dict[str, Any],
    features: Dict[str, float],
    score_result: Dict[str, Any],
) -> Dict[str, Any]:
    """Build an immutable, hash-anchored decision record for one scoring event."""
    canonical_payload = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return {
        "audit_id": str(uuid.uuid4()),
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "claim_id": claim_id,
        "input_hash": hashlib.sha256(canonical_payload.encode("utf-8")).hexdigest(),
        "feature_version": score_result["feature_version"],
        "model_version": score_result["model_version"],
        "features_applied": features,
        "score_result": score_result,
        "compliance_flags": {
            "data_minimization": True,
            "state_regulatory_alignment": True,
            "explainability_artifacts_attached": True,
        },
    }

Because the record keys on the immutable input_hash, an examiner can confirm that a disputed score was produced by a specific payload, feature version, and model version — and that the record has not been altered since. Audit records are serialized to append-only storage and indexed by claim ID. Aligning with NAIC Model Audit Rule guidelines and the jurisdiction-specific controls in State Regulation Mapping ensures the scoring pipeline meets statutory documentation requirements across every state it touches.

Severity-scoring failures cluster into a small set of named scenarios. Each has a deterministic diagnostic path and a code-level fix.

The severity tier this component emits is consumed across the triage engine. Upstream, every claim first passes the compliance gate in Coverage Validation Rules so that scoring never runs on uncovered losses, including the patterns for validating deductible thresholds automatically. Downstream, the scored tier feeds Dynamic Threshold Tuning — most visibly when implementing priority queues for catastrophic claims compresses the senior boundary during a surge — before the claim reaches routing high-severity claims to senior adjusters in the assignment layer.

• Claims Triage & Routing Engines — the parent control plane this scoring stage plugs into
• Coverage Validation Rules — the in-force coverage gate every claim clears before scoring
• Dynamic Threshold Tuning — adapts the routing boundaries this engine scores against
• Adjuster Assignment Algorithms — consumes the severity tier to match claims to licensed specialists
• Core Architecture & Compliance Mapping — the audit-log and regulatory backbone for scoring decisions