The default RAG pipeline most teams ship still assumes English documents and English queries. In India, that assumption shuts out the majority of people, who think, type and speak in Hindi, Bengali, Tamil, Telugu, Marathi and many other languages, often mixed with English. If you are building for Bharat, the pipeline needs to be designed multilingual from day one, not retrofitted after launch.
Where naive pipelines break
Many embedding models are trained predominantly on English. When you embed a Hindi query alongside a Tamil document, semantic similarity can collapse to noise. You get retrievals that share surface tokens but not meaning, or no useful retrievals at all because the query and the corpus sit in different vector neighbourhoods.
The first symptom is a quiet one: answer-faithfulness scores look fine on your English test set, then fall away when real users from Patna or Coimbatore start asking questions in their own languages. Three Indic-specific problems make it worse:
- Transliteration. Users often type Hindi in Latin script ("kya haal hai") while the corpus is in Devanagari.
- Code-mixing. Real queries mix English words into Indic sentences: "mera refund status kya hai".
- Messy sources. Government PDFs set in legacy non-Unicode fonts, scanned documents and inconsistent Unicode normalisation break text extraction before retrieval even starts.
The four design choices that matter
1. Embedding model
Use multilingual embeddings with good coverage of Indic scripts, such as Multilingual-E5 or BGE-M3, or domain-specific Indian models. Quality varies by language, and English-heavy models tend to degrade most in lower-resource languages, so benchmark candidates on a sample of your own queries in each target language before you choose.
2. Chunking
Sentence boundaries and punctuation differ across scripts; Hindi ends sentences with the danda, for example. Token counts differ too, because many tokenisers split Indic text into far more tokens than the equivalent English, so a naive fixed-size chunker will cut Devanagari mid-sentence. Use script-aware sentence splitters, set chunk size per language, and where the domain vocabulary is specialised, consider fine-tuning the embedding or reranking model.
3. Reranking
A multilingual cross-encoder reranker, such as Cohere's multilingual rerank models or BAAI's bge-reranker-v2-m3, closes much of the gap left by first-stage retrieval. For Indic corpora it is often the single most effective retrieval upgrade available.
4. Generation model
Frontier models handle Hindi and English well, and their Indic fluency keeps improving, but quality still drops in lower-resource languages. India-focused models such as Sarvam's and Krutrim, and the government-backed Bhashini platform for translation and speech, are worth testing alongside them, particularly for regional-language output and voice.
Why hybrid retrieval matters more in Indic
BM25 plus dense retrieval tends to beat either alone for Indic corpora, partly because transliteration is messy. Users type "namaste" in Latin script, the corpus has it in Devanagari, and a sparse index over a transliterated copy of the corpus bridges the gap. We rarely ship a Bharat-facing RAG system without hybrid retrieval and a reranker on top. A practical pipeline:
- Normalise Unicode and convert legacy-font documents before indexing.
- Detect the language and script of each chunk and each query.
- Index the original text densely, and keep a transliterated copy for sparse retrieval.
- Retrieve from both indexes, merge the results with reciprocal rank fusion, then rerank.
- Generate in the user's language and script, citing the source passages.
Evaluation across languages
If you are running RAGAS or DeepEval, run them per language. Take a hypothetical system with a 92% faithfulness average: it looks fine until you realise it is 96% in English and 68% in Bengali. Build per-language eval splits, score them separately, and ship only when the worst language meets the bar, not the average.
- Write eval questions with native speakers rather than machine-translating English ones.
- Include transliterated and code-mixed queries in every split.
- Have native speakers spot-check LLM-as-judge scores, because judges are also weaker in low-resource languages.
Building and labelling these sets is careful human work, and data tagging with native-speaker annotators is usually the most reliable way to do it.
Common mistakes
- Translating everything into English for retrieval and back again, which loses nuance and adds two more places for errors to creep in.
- Testing only in Hindi and assuming other Indic languages behave the same way.
- Ignoring voice. Many users prefer speaking to typing, so plan for speech-to-text errors in queries.
- Answering in a formal, textbook register that nobody uses day to day.
FAQ
Should we retrofit an English pipeline or rebuild?
If the corpus is mostly English and only queries are multilingual, a new embedding model, hybrid retrieval and a reranker are often enough. If the corpus itself is multilingual or scanned, plan for a rebuild of ingestion.
Do we need a separate index per language?
Not necessarily. A single multilingual index with language metadata works for many systems; separate indexes help when languages have very different volumes or retrieval quality.
How we approach this at Velura Labs
Our RAG and knowledge systems work handles multilingual corpora natively, with hybrid retrieval, Indic-tuned reranking and per-language evaluation built in. For citizen-facing systems with chatbot front ends, the same patterns extend into AI and data solutions for government engagements. If you are stuck with an English-only pipeline that now needs to serve Bharat, talk to us; a short conversation is usually enough to tell whether to retrofit or rebuild.
Whether you are in California, Texas or Washington in the US, France or Italy in Europe, the UAE or Saudi Arabia in the Gulf, or here in India, Velura Labs delivers this end to end. Talk to us about your context.