KYC document extraction was long a regex-and-rules problem. Vision-LLMs have since replaced many traditional OCR-plus-template pipelines in production. The accuracy gain on messy inputs is real, the cost is manageable, and the deployment patterns are different enough that it's worth resetting your mental model.
Why the old stack stopped working
Traditional KYC pipelines used Tesseract or AWS Textract for OCR plus a per-document-type template to extract fields. Accuracy was acceptable on clean originals and brittle the moment a customer photographed a slightly tilted PAN card on a phone in poor light. Banks accepted the trade-off and built large human-review queues to absorb the noise.
Current multimodal models from the major providers handle document variance natively. A photo of a folded passport, a slightly water-damaged Aadhaar card or a printed PAN with non-standard spacing can be extracted reasonably well without per-template engineering.
What you give up, and how to get it back
Vision-LLMs are slower and more expensive per document than traditional OCR, and at high volume that matters. Measure both on your own document mix before choosing. Two patterns recover most of the gap:
- Hybrid pipeline. Run cheap OCR first; if confidence is high and validation passes, use the result. If not, fall back to the vision-LLM. Most clean documents take the cheap path.
- Field-level confidence routing. Score confidence per extracted field. High-confidence fields skip review; low-confidence ones queue. Self-reported model confidence is only a rough signal, so calibrate it against your reviewers' corrections and combine it with validation results.
Prefer issuer data over images where you can
Extraction from a photo should not be your only path. The Aadhaar Secure QR code and the offline e-KYC XML carry data digitally signed by UIDAI, and DigiLocker provides documents issued directly by the issuing authority. Where your regulatory permissions allow, verify PAN details against the official verification services. Use vision extraction for what these sources don't cover, and to cross-check what they do.
The validation layer matters more than the extraction
Vision-LLMs sometimes extract wrong values with confidence. Your pipeline must include cross-field validation:
- The Aadhaar number must pass the Verhoeff checksum.
- The PAN must match the structural pattern of five letters, four digits and one letter; for an individual, the fourth character is P.
- Passport MRZ check digits must validate. The machine-readable zone carries check digits for the document number, date of birth and expiry date, as defined in ICAO Doc 9303.
- Dates of birth must be plausible, and expiry dates must be in the future.
- Names must match across documents (PAN name vs Aadhaar name), with fuzzy matching tuned for transliteration, initials and name order.
Catching extraction errors at validation is cheaper than catching them downstream, and it is necessary for regulatory compliance.
Handle identity data carefully
Aadhaar numbers carry specific rules on storage and display. Mask them, showing only the last four digits, in interfaces and logs wherever the full number is not needed, and follow UIDAI's requirements for storing full numbers, including an Aadhaar Data Vault where it applies. Decide deliberately which images a hosted model needs to see, confirm the provider's data-retention terms for your plan, and consider in-country or on-prem deployment for high-volume processing.
Citations as a compliance feature
Many vision-LLM setups can return bounding-box locations along with extracted values, either natively or through a layout-aware OCR step. Persist these in your audit log. When an auditor asks where a date of birth came from, you can show the exact region of the source document. Reviewers also work faster when the interface highlights the source region beside each field.
On-prem deployment
Some banks and regulated entities require on-prem or in-country deployment for KYC processing. Open-weight vision-language models such as Qwen2.5-VL or Llama 3.2 Vision can be deployed on a small GPU cluster and can perform well on clean Indian KYC documents; benchmark them on your own document mix against a commercial API before committing. Throughput is lower, but the compliance posture is stronger.
A review queue that shrinks over time
Route each document by its worst field. If every field validates with high confidence, it goes straight through. Any failed checksum or cross-document mismatch goes to review, with the failing field highlighted. Log every reviewer correction against the field and document type. Those corrections tell you which prompts, image-quality checks or thresholds to change, and they become your regression test set.
Common mistakes
- Trusting extracted values without checksum and format validation.
- Sending full-resolution identity images to a hosted model without checking its retention terms.
- Measuring accuracy on clean scans when customers actually upload phone photos.
- No audit trail linking each extracted value to its source region.
FAQ
Does this approach work outside India?
The pattern does. Passports follow the same ICAO machine-readable zone standard worldwide, and many national ID documents have format rules or check digits you can validate. Only the issuer-data sources and the storage rules are country-specific.
How we approach this at Velura Labs
Our Document Processing service ships KYC pipelines using the hybrid pattern above, with citations, cross-field validation and audit logging built in. For larger archives that need scanning before extraction, see Document Digitisation & Scanning. Read our guardrails guide for the broader compliance posture. Talk to us if your KYC review queue is bigger than it should be.
We ship work like this for clients in the US (California, Texas, Washington, New York), across Europe (France, Italy and the EU), the Gulf (UAE and Saudi Arabia) and India — with an India delivery base that keeps cost down and time-zone overlap high. Talk to us.