Sanskrit, Hindi and Marathi AI Translation: Building an Indian-Language NLP Pipeline

India runs on many languages. A clinic in Pune may receive messages in Marathi, Hindi and English on the same morning; a temple trust may hold centuries of Sanskrit documents that few staff can read; a business in Mumbai may want one report summarised for teams who prefer different languages. An Indian language AI translator that handles this well would be genuinely useful — but building one responsibly is harder than connecting a general model to a "translate" button.

This article describes how we approach translation and summarisation research for Sanskrit, Hindi and Marathi: the pipeline, the importance of dataset quality, how to evaluate with humans, the risk of hallucinations, and what to consider before deploying anything to real users. It reflects ongoing research in our AI Transformation Lab; we describe methods here, not finished products or benchmark claims.

Why Sanskrit, Hindi and Marathi?

The three languages make a useful research set because they share the Devanagari script but differ sharply in available data and usage:

Working on all three helps separate problems caused by the script from problems caused by data scarcity.

The Translation Pipeline

A reliable translation system is a pipeline, not a single model call.

1. Input normalisation. Detect the language and script, convert transliterated text where needed, apply Unicode normalisation and standardise punctuation. Sanskrit text in romanised transliteration (such as IAST) and in Devanagari should end up in one consistent form.
2. Segmentation. Split text into sentences or verses. For Sanskrit, optional sandhi and compound segmentation can make the input easier for the model to interpret.
3. Glossary protection. Mark names, places, deities, technical terms, product names and numbers so they are carried through correctly instead of being "translated" into something else.
4. Translation. Use a model adapted to the language pair and domain. Open multilingual translation models that cover Indian languages — including Sanskrit — are a sensible starting point for fine-tuning, subject to their licences.
5. Post-checks. Compare names and numbers between source and output; flag outputs that are suspiciously short or long; check that every source sentence has a translation.
6. Human review. Route flagged or high-stakes output to a reviewer before it is used.

The Summarisation Pipeline

Summarisation adds its own risk: a summary must be shorter than the source, which means choosing what to leave out.

1. Chunking. Split long documents into sections the model can handle, keeping headings and context together.
2. Extract, then abstract. First identify the key sentences, then write a fluent summary from them. Grounding the summary in extracted sentences reduces invented content.
3. Language choice. Decide whether to summarise in the source language and then translate, or summarise directly into the target language. Each approach has different error patterns; we test both.
4. Attribution. Where possible, link each summary point back to the passage it came from, so reviewers can check it quickly.
5. Length and coverage checks. Ensure the summary respects the length limit and does not drop critical information such as dates, amounts or warnings.

Dataset Quality Matters More Than Size

For low-resource languages, a smaller set of carefully checked examples is usually worth more than a large, noisy one.

• Licensing: record the source and licence of every text and translation pair.
• Alignment: check that each source sentence really matches its translation; misaligned pairs teach the model to omit or invent content.
• Domain coverage: classical verse, prose commentary, modern administrative text, healthcare messages and business documents behave very differently.
• Register: formal Marathi or Hindi differs from everyday WhatsApp language; include both if both will be used.
• Code-mixing: real messages often mix English with Hindi or Marathi. Exclude it and the system fails on real input.
• Test-set hygiene: keep evaluation material out of training data, including near-duplicates.

Human Evaluation

Automatic metrics are useful for comparing versions quickly. Character-level metrics such as chrF tend to suit morphologically rich languages better than word-level BLEU. But metrics cannot tell whether a translation is faithful, appropriate and safe to use.

Our human evaluation approach:

• Reviewers with genuine expertise in each language — Sanskrit scholars for classical texts, native speakers for Hindi and Marathi.
• Adequacy and fluency ratings: does it say the same thing, and does it read naturally?
• An error taxonomy: mistranslation, omission, addition, wrong terminology, grammar, wrong register.
• Blind comparison: reviewers rate outputs without knowing which system produced them.
• Regular re-review whenever the model, prompts or data change.

Hallucination Risks

Hallucination — fluent output that is not supported by the source — is the most serious risk in translation and summarisation, because readers usually cannot check the original.

• Invented content: a summary that adds a "fact" not in the document.
• Silent omission: a negation, condition or warning dropped from a sentence.
• Changed numbers and names: dates, doses, amounts and proper nouns altered.
• Over-confident interpretation: a philosophically or legally ambiguous Sanskrit passage rendered with false certainty.

Mitigations include glossary protection, entity and number checks, retrieval with source citations, confidence flags, conservative output length limits and — most importantly — human review wherever errors could mislead.

Multilingual Deployment Considerations

• Use-case risk: translating a marketing message is low risk; translating medical, legal or financial content needs qualified human review every time.
• Transparency: tell users when text has been machine-translated or summarised.
• Privacy: messages often contain personal data. Minimise what is sent to any model, control where it is processed, and follow applicable Indian data-protection law.
• Fonts and rendering: test Devanagari rendering on the phones and browsers your users actually have.
• Cost: Indic text can use more tokens per sentence than English with some tokenisers; measure it and see our guide to reducing LLM token costs.
• Fallbacks: when confidence is low, show the original text and offer human assistance instead of a doubtful translation.

Where This Fits

Translation and summarisation research builds directly on our work on a Sanskrit small language model. In business settings, the same techniques could later support multilingual customer communication in products such as our AI-Powered CRM — but only after evaluation shows they are reliable enough for that use.