Ramayana LLM

One epic, many models

Each model is named after a figure from the Ramayana whose role matches what the model does.

A name earned through grammar

When Hanuman first meets Rama in the Kishkindha Kanda, Rama tells Lakshmana that this messenger must have mastered grammar, because he spoke at length without a single mistake. Getting grammar right is the first thing a language model learns, and it is what this model does best.

नूनं व्याकरणं कृत्स्नमनेन बहुधा श्रुतम्।
बहु व्याहरतानेन न किञ्चिदपशब्दितम्॥ Valmiki Ramayana, Kishkindha Kanda 3.29: "Surely he has studied the whole of grammar many times; though he spoke much, not one word was wrong."

What Hanuman writes

Unedited samples (temperature 0.8, top-p 0.95). Bold is the prompt and the rest is the model. The grammar is correct; facts are sometimes invented.

Ask it a question

Angad is Hanuman fine-tuned for about an hour on 37k Hindi question–answer pairs. Answer perplexity fell from 58.4 → 16.9, and it now ends every answer cleanly. It has learned how to answer, but not yet what is true.

prompt = "<hi>प्रश्न: होली का त्योहार क्यों मनाया जाता है?\nउत्तर: "
# model: ashishpatel26/angad-hi-42m  ·  generate until </s>

Perplexity on held-out Hindi

Measured on Wikipedia text the models never saw during training (lower is better). Making the model 4× larger cut perplexity by more than half.

Instruction tuning: answer perplexity

Measured on 380 held-out Hindi question–answer pairs, scoring the answer tokens only. Fine-tuning made answers 3.5× more predictable.

From raw Wikipedia to a chat model

Run it in a few lines

Hanuman and Jatayu are base models: they continue the text you give them, so start the prompt with <hi>. Angad answers questions; use the template shown below.

# pip install torch transformers sentencepiece huggingface_hub

import torch, sentencepiece as spm
from huggingface_hub import hf_hub_download
from transformers import LlamaForCausalLM

repo = "ashishpatel26/hanuman-hi-42m"
model = LlamaForCausalLM.from_pretrained(repo)
sp = spm.SentencePieceProcessor(model_file=hf_hub_download(repo, "spm.model"))

x = torch.tensor([sp.encode("<hi>हिमालय पर्वत")])
y = model.generate(x, max_new_tokens=100, do_sample=True,
                   top_p=0.95, temperature=0.8, repetition_penalty=1.15)
print(sp.decode(y[0].tolist()))

# Angad: ask a question
repo = "ashishpatel26/angad-hi-42m"
model = LlamaForCausalLM.from_pretrained(repo)
sp = spm.SentencePieceProcessor(model_file=hf_hub_download(repo, "spm.model"))

x = torch.tensor([sp.encode("<hi>प्रश्न: स्वस्थ रहने के तीन उपाय बताइए।\nउत्तर: ")])
y = model.generate(x, max_new_tokens=150, do_sample=True, top_p=0.9, temperature=0.7,
                   repetition_penalty=1.15, eos_token_id=sp.eos_id(), pad_token_id=sp.pad_id())
print(sp.decode(y[0, x.shape[1]:].tolist()))

Roadmap

• ✅ Jatayu-hi-11M: proof that the pipeline works
• ✅ Hanuman-hi-42M: Hindi base model
• ✅ Angad-hi-42M: instruction-tuned on Hindi Q&A
• ⏳ Jambavan (~110M): pretrained on 1–2B tokens of AI4Bharat Sangraha, for better facts
• ⏳ Angad v2: instruction tuning on top of Jambavan
• ⏳ Setu: one multilingual model for Hindi, Marathi, Bengali, Tamil, Telugu and more

Limitations

• Facts are often made up. At 42M parameters the model has learned how Hindi is written, not what is true.
• Pretrained on Wikipedia only, so the style is encyclopaedic and conversational Hindi is weak.
• Angad's instruction data is machine-translated, so it inherits translation errors and Western-centric answers.
• No RLHF and no safety tuning. For research and learning only.
• Context window is 512 tokens.