CASE STUDY / ENTERPRISE INTENT ROUTING

Routing 77 Banking Intents
in 30 Milliseconds.

How an enterprise financial institution routes high-cardinality customer inquiries with 74.06% Top-3 accuracy on commodity CPU—without cloud LLM API token costs, latency spikes, or customer data egress.

Download PDF Blueprint (4 pages) ↓ Executive TCO Guide (2 pages) ↓

74.06% Top-3 Accuracy Evaluated zero-shot across 3,080 Banking77 test inquiries.

30.6 ms Median Latency Over 250 decisions/sec on a single standard CPU.

100% Air-Gapped (0% Egress) < 64 MB RAM footprint; zero external API calls.

01 / THE PRODUCTION CHALLENGE

The high-cardinality
triage bottleneck.

Financial customer support requires instant, calibrated classification across dozens of fine-grained, overlapping categories. Traditional cloud LLMs and naive keyword filters fail under enterprise SLAs.

[ 01 ] SCALE & SUBTLETY

77 Fine-Grained Intents.

Customer requests share vocabulary across distinct resolution workflows (e.g., distinguishing card_arrival from card_delivery_estimate, or lost_or_stolen_card from compromised_card).

[ 02 ] LATENCY & PRIVACY

Strict SLAs & Data Residency.

Live conversational agents demand sub-50ms routing. Meanwhile, transmitting banking inquiries with account context to public cloud APIs violates GDPR, SOC2, and financial privacy laws.

[ 03 ] THE COST MULTIPLIER

Prohibitive Token Economics.

At 10M routing requests per month, commercial cloud LLMs generate $20,000–$35,000 in monthly recurring inference bills, accompanied by network jitter and queue delays.

02 / ARCHITECTURAL PATTERN

Two stages.
Microsecond scale.

ZEV enforces a mathematically bounded attention budget (K ≤ 10 choices per pass) to preserve order equivariance. For high-cardinality catalogs (77+ choices), a micro-embedding filter feeds candidate subsets into ZEV.

STAGE 1 / STATIC EMBEDDINGS Model2Vec Filter 64-dim vector dot product in 14 µs (86.5% Recall@8)

STAGE 2 / DECISION ENGINE ZEV Scoring One forward pass in 26 ms (ICMB attention, calibrated probabilities)

POLICY GATE Automated Action Auto-dispatch if ≥ 80%; agent triage suggestions if < 80%

Why Stage 1 Works: Model2Vec

Distilled static word embeddings (2–8 MB on disk) require zero GPU or attention layers. By computing a normalized vector matrix product across the 77 intent labels in 14 microseconds, it narrows the field down to 8 candidates with 86.49% recall.

Why Stage 2 Delivers: ZEV

Within the Top-8 candidate window, ZEV evaluates the inquiry against candidate descriptions using Inter-Choice Masked Blindness (ICMB). Its calibrated decision probabilities refine raw semantic similarity, driving Top-1 accuracy to 52.21% and Top-3 accuracy to 74.06%.

03 / INTERACTIVE SIMULATION

See the two-stage
pipeline in action.

Select sample customer inquiries from the Banking77 evaluation set to inspect Stage 1 retrieval candidates and Stage 2 ZEV calibrated reranking scores.

TWO-STAGE ROUTER BANKING77 BENCHMARK REPLAY

CUSTOMER INQUIRY TEXT

“The ATM cash dispenser took my card and didn't give it back. What do I do?”

STAGE 1 CANDIDATES & STAGE 2 ZEV SCORES

Pipeline Latency: 26.8 ms (Stage 1: 0.014 ms · Stage 2: 26.8 ms)

CONFIDENCE: 92.4%

Empirical evaluation replay from test query index #42. Stage 1 retrieves top 8 semantically aligned candidates from 77 classes; Stage 2 applies ZEV contrastive decision scoring.

04 / FULL TEST EVIDENCE

Empirical scorecard across
all 3,080 test queries.

Evaluated against the official PolyAI Banking77 held-out test suite across 77 fine-grained classes. All tests executed on commodity CPU.

Architecture & Pipeline	Stage 1 Filter	Stage 1 Recall@8	Top-1 Accuracy	Top-3 Accuracy	Median Latency	Throughput
Legacy Lexical + ZEV	BM25 Token Overlap	68.80%	14.40%	50.40%	22.7 ms	165 dec/s
Production (2M Embeddings + ZEV)	Micro-Embedding (2M)	82.24%	47.50%	68.41%	30.4 ms	251 dec/s
Production (8M Embeddings + ZEV)	Micro-Embedding (8M)	86.49%	52.21%	74.06%	30.6 ms	251 dec/s
Hierarchical Macro-Tree	10-Domain ZEV Classifier	27.60% (Domain)	4.40%	14.00%	66.1 ms	58 dec/s

The Assistive Support Multiplier: In human-in-the-loop customer support consoles (e.g. Zendesk, Salesforce Service Cloud), presenting 3 pre-ranked action macros allows human representatives to resolve tickets with a single click. ZEV surfaces the correct intent within the top 3 recommendations in 74.06% of all zero-shot customer interactions in ~30 milliseconds.

05 / BUSINESS CASE

The infrastructure &
financial ROI comparison.

Comparing the operational footprint and annual spend of a 10M decision/month enterprise support workload.

Evaluation Parameter	Frontier Cloud LLMs (GPT-4o / Claude 3.5)	ZEV + Micro-Embedding Pipeline	Enterprise Difference
Compute Requirement	Multi-GPU Cloud API clusters	1x Standard 4-Core CPU VPS	Runs on your existing server hardware
Monthly Cost (10M requests)	$20,000 – $35,000 / month	$100 / month flat hosting	> $250,000 annualized savings
Median Decision Latency	800 ms – 2,500 ms (network + queue)	27.0 ms (in-process / LAN)	40× faster response time
Customer PII Egress	100% leaves enterprise perimeter	0% egress (fully air-gapped)	Zero compliance risk under GDPR/SOC2
Order Permutation Variance	12% – 35% positional bias	0.000006% variance (ICMB)	Deterministic, auditable decisions

Download the 2-page Executive TCO & Latency Guide (PDF) ↓

06 / DEPLOYMENT

20 lines to production.

Both the micro-embedding filter and ZEV run on your infrastructure. Deploy as a standalone microservice or embed directly in your Rust/Python backend.

Download PDF Blueprint ↓ Start an evaluation

# 1. Install local dependencies
pip install zev model2vec numpy

# 2. Query the two-stage router
from model2vec import StaticModel
import zev, numpy as np

embedder = StaticModel.from_pretrained("minishlab/potion-base-8M")
# Pre-embed canonical intent labels once at startup
intent_vecs = embedder.encode(ALL_77_INTENTS)

def route_inquiry(inquiry: str):
    # Stage 1: Microsecond candidate filter (14 µs)
    q_vec = embedder.encode([inquiry])
    sims = np.dot(intent_vecs, q_vec[0])
    top8 = [ALL_77_INTENTS[i] for i in np.argsort(-sims)[:8]]
    
    # Stage 2: ZEV decision verification (26 ms)
    result = zev.decide(
        state=inquiry,
        question="Classify banking intent",
        choices=top8
    )
    return result.best_choice, result.top3_suggestions

YOUR WORKLOAD. YOUR ACCEPTANCE CRITERIA.

Ready to evaluate your taxonomy?

Benchmark ZEV against your ticket queues and intent catalogs on your hardware.

Evaluate ZEV

The high-cardinalitytriage bottleneck.