Tesseract vs Cloud Vision for Scanned Claims

Deciding between local Tesseract and a cloud vision OCR API for scanned and faxed claim documents is a trade of accuracy on degraded scans against cost, latency, and — the factor that most often decides it in insurance — data residency and PII exposure. This comparison under OCR Integration & Sync benchmarks both on realistic claims intake, keeps the cloud client provider-agnostic, and gives a verdict grounded in where confidence outputs and the data boundary actually bite.

Scanned claims packets are the worst input in the pipeline: faxed first-notice-of-loss forms at 150-200 DPI, feeder-skewed declarations, handwritten adjuster notes, and monetary fields where a displaced decimal turns $1,245.00 into $12,450.00. Two OCR strategies can read them, and they fail in opposite ways. Local Tesseract (via pytesseract) runs entirely inside your trust boundary — no bytes leave the worker — at zero marginal cost per page, but its accuracy on degraded, low-contrast, or handwritten scans trails a modern cloud engine, and squeezing quality out of it means owning preprocessing, page-segmentation modes, and language-model tuning yourself. A cloud vision OCR API delivers markedly higher accuracy on exactly those degraded scans and returns rich per-word confidence and bounding boxes out of the box, but it charges per page, adds network latency and a rate-limit failure surface, and — decisively — sends raw claim images containing PII, PHI, and financial account numbers across your data boundary to a third party.

That last point is not a footnote in insurance. A scanned claims packet under HIPAA, GDPR, or a state privacy statute cannot be shipped to an arbitrary region or vendor without an executed data-processing agreement and, often, a residency guarantee. So the decision is rarely “which is more accurate” in isolation — it is “which is accurate enough within the data-residency and cost envelope this document class allows.” This page implements both behind one interface, compares them on accuracy, cost, latency, and confidence, and shows how to route per document class rather than committing the whole pipeline to one engine.

Tesseract needs the system binary plus the Python binding; the cloud path needs only an HTTP client because the vendor SDK stays behind an abstraction. Pin everything so OCR behaviour cannot drift silently under you.

Copy
python -m venv .venv && source .venv/bin/activate
# System Tesseract engine (Debian/Ubuntu shown); bake into the container image.
apt-get install -y tesseract-ocr
pip install "pytesseract==0.3.13" "Pillow==10.4.0" \
            "httpx==0.27.*" "pydantic==2.*" "structlog==24.*"

Establish two environment-driven thresholds before writing extraction code, mirroring the parent stage: a per-field confidence floor (OCR_CONFIDENCE_FLOOR, default 0.85) and a residency policy that names, per document class, whether external transfer is permitted at all. Never hard-code either. The confidence floor is tuned per carrier; the residency policy is set by legal and compliance, not by engineering.

Both engines must return the same shape so the confidence gate, the audit event, and the router never branch on which one ran. OcrResult carries the text plus normalized per-token confidence in [0, 1]; the OcrEngine protocol is the seam the router swaps on.

Copy
from __future__ import annotations

from dataclasses import dataclass
from typing import Protocol

import structlog

log = structlog.get_logger("ocr.compare")


@dataclass(frozen=True, slots=True)
class OcrToken:
    text: str
    confidence: float  # normalized to 0.0-1.0


@dataclass(frozen=True, slots=True)
class OcrResult:
    engine: str
    text: str
    tokens: tuple[OcrToken, ...]
    region: str  # "local" or the cloud region id — an audit field

    @property
    def min_confidence(self) -> float:
        return min((t.confidence for t in self.tokens), default=0.0)


class OcrEngine(Protocol):
    name: str

    def run(self, image_bytes: bytes) -> OcrResult: ...

pytesseract.image_to_data returns per-word text and a conf value on a 0-100 scale (with -1 for non-text boxes), which normalizes cleanly into the shared token model. Tesseract runs in-process, so region is always local and no bytes leave the trust boundary.

Copy
import io

import pytesseract
from PIL import Image
from pytesseract import Output


@dataclass(frozen=True, slots=True)
class TesseractEngine:
    name: str = "tesseract"
    psm: int = 6      # assume a uniform block of text
    lang: str = "eng"

    def run(self, image_bytes: bytes) -> OcrResult:
        image = Image.open(io.BytesIO(image_bytes))
        config = f"--psm {self.psm}"
        data = pytesseract.image_to_data(
            image, lang=self.lang, config=config, output_type=Output.DICT
        )
        tokens: list[OcrToken] = []
        for text, conf in zip(data["text"], data["conf"]):
            word = text.strip()
            conf_val = float(conf)
            if not word or conf_val < 0:      # -1 marks non-text boxes
                continue
            tokens.append(OcrToken(text=word, confidence=conf_val / 100.0))
        full_text = " ".join(t.text for t in tokens)
        log.info("tesseract.done", tokens=len(tokens))
        return OcrResult(
            engine=self.name, text=full_text, tokens=tuple(tokens), region="local"
        )

The cloud engine hides behind a CloudOcrClient dataclass so no real vendor SDK or API key is hard-coded — the endpoint, region, and credential all arrive from configuration, and swapping providers never touches the router. The client posts the image, reads back a provider-neutral JSON of words with confidences, and normalizes them into the same OcrResult. Credentials come from the environment, never from source.

Copy
import os

import httpx
from pydantic import BaseModel


class CloudOcrConfig(BaseModel):
    base_url: str
    region: str
    api_key_env: str = "CLOUD_OCR_API_KEY"   # name of the env var, not the key
    timeout_sec: float = 30.0

    @classmethod
    def from_env(cls) -> "CloudOcrConfig":
        return cls(
            base_url=os.environ["CLOUD_OCR_BASE_URL"],
            region=os.environ["CLOUD_OCR_REGION"],
        )


@dataclass(frozen=True, slots=True)
class CloudOcrClient:
    config: CloudOcrConfig
    name: str = "cloud_vision"

    def run(self, image_bytes: bytes) -> OcrResult:
        api_key = os.environ[self.config.api_key_env]  # resolved at call time
        with httpx.Client(timeout=self.config.timeout_sec) as client:
            resp = client.post(
                f"{self.config.base_url}/v1/ocr",
                headers={"Authorization": f"Bearer {api_key}"},
                json={"region": self.config.region, "return_confidence": True},
                content=image_bytes,
            )
            resp.raise_for_status()
            payload = resp.json()
        # Provider-neutral contract: {"words": [{"text": ..., "confidence": 0..1}]}
        tokens = tuple(
            OcrToken(text=w["text"], confidence=float(w["confidence"]))
            for w in payload["words"]
            if w["text"].strip()
        )
        text = " ".join(t.text for t in tokens)
        log.info("cloud.done", tokens=len(tokens), region=self.config.region)
        return OcrResult(
            engine=self.name, text=text, tokens=tokens, region=self.config.region
        )

The router enforces the ordering that insurance requires: residency and PII first, accuracy second. A document class the residency policy forbids from leaving the trust boundary goes to Tesseract regardless of scan quality; only a transfer-permitted, degraded scan earns the cloud call.

Copy
def choose_engine(
    *,
    external_transfer_allowed: bool,
    scan_quality: float,          # 0.0 (degraded) .. 1.0 (clean)
    quality_floor: float = 0.55,
) -> OcrEngine:
    if not external_transfer_allowed:
        return TesseractEngine()          # bytes must stay local
    if scan_quality < quality_floor:
        return CloudOcrClient(CloudOcrConfig.from_env())   # degraded → cloud accuracy
    return TesseractEngine()              # clean enough for local, and free

Because both engines return OcrResult, a single harness benchmarks them on the same fixture set for accuracy (character error rate against ground truth), latency, and reported confidence. Assert the interface contract first, then measure — never let a benchmark trade correctness for speed.

Copy
import time


def char_error_rate(predicted: str, truth: str) -> float:
    """Levenshtein distance normalized by ground-truth length."""
    import difflib
    matcher = difflib.SequenceMatcher(a=truth, b=predicted)
    return 1.0 - matcher.ratio()


def benchmark(engine: OcrEngine, samples: list[tuple[bytes, str]]) -> dict[str, float]:
    errors, latencies, confidences = [], [], []
    for image_bytes, truth in samples:
        start = time.perf_counter()
        result = engine.run(image_bytes)
        latencies.append(time.perf_counter() - start)
        errors.append(char_error_rate(result.text, truth))
        confidences.append(result.min_confidence)
    n = len(samples)
    return {
        "engine": engine.name,
        "mean_cer": round(sum(errors) / n, 4),
        "p50_latency_s": round(sorted(latencies)[n // 2], 3),
        "mean_min_conf": round(sum(confidences) / n, 3),
    }


def test_engines_share_contract(clean_sample: bytes) -> None:
    for engine in (TesseractEngine(),):   # cloud covered by a mocked transport
        result = engine.run(clean_sample)
        assert isinstance(result, OcrResult)
        assert all(0.0 <= t.confidence <= 1.0 for t in result.tokens)

The characteristic finding across claims fixtures is summarized below. On clean high-DPI scans the two engines are close and Tesseract’s zero cost and zero data-egress win; on faxed, low-contrast, or handwritten pages the cloud engine’s mean character error rate drops well below Tesseract’s, which is precisely the degraded class where paying for accuracy is justified — if residency permits it.

Choose local Tesseract whenever the data class forbids external transfer, and as the default for clean, high-DPI digital-fax scans where its accuracy is already competitive and its zero per-page cost and zero data-egress are decisive. It is the only defensible choice for packets carrying PHI or financial account numbers into a jurisdiction whose residency rules or your executed agreements do not cover the cloud vendor — no accuracy gain outweighs shipping regulated data outside the boundary the Data Boundary Enforcement controls define. To make Tesseract competitive on marginal scans, invest in preprocessing rather than switching engines: deskew and orientation-correct first, exactly as handling rotated pages covers, since a skewed decimal is the most common source of the silent monetary-field corruption the parent stage warns about.

Choose a cloud vision OCR API for the genuinely degraded, low-contrast, or handwritten claim documents where its accuracy advantage is largest — but only for document classes the residency policy clears for external transfer, and always with redaction and confidence gating downstream. The production posture is not either/or: route per document class as Step 4 does, keeping regulated and low-value-clean pages local and spending cloud budget only where a degraded scan both permits transfer and needs the accuracy. Whichever engine runs, its per-token confidence feeds the same confidence gate that promotes a document only when every mapped field clears the carrier floor, and the normalized text flows on to Field Mapping Strategies for canonical ACORD alignment. When a page turns out to carry a usable embedded text layer, neither OCR engine should run at all — divert it to the native-text path and its own pdfplumber-versus-PyMuPDF decision instead.

The engine and its region are compliance facts, not implementation details. Persist, per document, which engine ran, the region the bytes were processed in (local or a named cloud region), the per-field confidence, and whether redaction ran before persistence — all keyed on the SHA-256 document hash in the same immutable audit trail the parent stage maintains. That record lets a regulator confirm a PHI-bearing packet was processed inside the trust boundary and never egressed, and it lets an examiner reconstruct which model and confidence profile produced a disputed monetary field. PII, PHI, and account numbers must be tokenized or redacted before any extracted text is written to durable storage, and for the cloud path the residency determination and the executed data-processing agreement are themselves audit artifacts. Recording the engine also means a later cost-driven shift toward the cloud, or a residency-driven shift back to local, never silently rewrites the provenance of historical extractions.

• Tesseract confidence collapses on faxed pages. Symptom: min_confidence sits far below the floor and the document floods human review. Cause: low DPI, skew, or noise the engine cannot recover. Fix: deskew and binarize before OCR and tune the page-segmentation mode (--psm) to the layout; only escalate to the cloud engine when residency permits and preprocessing is exhausted.
• Cloud call leaks a document class it should not. Symptom: a PHI packet is processed in a disallowed region. Cause: the router checked scan quality before residency. Fix: enforce the residency policy first, as in Step 4, so external_transfer_allowed gates the cloud path unconditionally.
• Retry storm under cloud rate limiting. Symptom: HTTP 429s stall the queue as workers retry in lockstep. Cause: no backoff or circuit breaker on the cloud client. Fix: add jittered exponential backoff, cap the retry budget, and fail over to local Tesseract when the 429 rate crosses a rolling threshold rather than blocking ingestion.
• Confidence scales silently mismatched. Symptom: the gate trusts cloud output it should reject. Cause: Tesseract reports 0-100 while the cloud returns 0-1, and one path was not normalized. Fix: normalize both into the shared OcrToken (0-1) at the engine boundary, as the implementations do, and gate only on the normalized value.
• Silent decimal shift in a monetary field. Symptom: a premium is off by an order of magnitude despite high confidence. Cause: a skewed scan displaced the decimal’s bounding box before either engine read it. Fix: cross-validate extracted totals against line-item summations during validation and confirm deskew ran first; divert mismatches to human review regardless of engine.

• OCR Integration & Sync — the parent stage whose confidence gate both engines feed
• Handling Rotated Pages in Policy Documents — deskew that makes local Tesseract competitive on marginal scans
• Data Boundary Enforcement — the residency and PII controls that can force the local engine
• pdfplumber vs PyMuPDF for Carrier PDFs — the native-text path for pages that never needed OCR
• Field Mapping Strategies — normalizing OCR tokens into canonical ACORD identifiers