The problem
W3Techs, which tracks the language of websites, reported in October 2026 that English is used by 49.5% of the websites whose content language it knows. It lists Bengali at 0.1% and Hindi, Marathi, Tamil, Telugu, Urdu, Gujarati, Kannada and Malayalam each below 0.1%. The Census of India 2011 found that 96.7% of people have one of the 22 Scheduled languages as their mother tongue.
A person who types or speaks to an AI tool in Kannada or Tamil is asking it to work from far less material than a person asking in English. These tools learn from text and recorded speech, and there is much less of both online in Indian languages. The authors of IndicLLMSuite, a research paper at ACL 2024, say progress on models for languages other than English has been hindered by the scarcity of tailored resources. Their own answer was to build datasets covering 22 Indian languages, with 251 billion tokens.
Researchers and companies are working on this, and some pay people in villages to record their own language. Whether these efforts reach the languages and dialects most people speak, and the speech of people who do not read or type, is not shown by anything we found.
- Who has this problem
- People who speak an Indian language and want AI chat or voice tools in it.
- Languages needed
- Hindi and regional languages
Where the demand comes from
What people do today
- The workaround
- We found no study of what people do instead of using these tools in their own language.
- Why that fails
- AI models learn from text and recordings, and there is far less of both in Indian languages than in English.
How others solved it2
Products and services that work on a similar problem, in India and elsewhere.
- KaryaIndia
Pays people in villages to record and check text and speech in their own languages. It says it pays at least $5 an hour, a figure we could not verify independently.
- Sarvam AIIndia
Builds language, speech and document models around India's 22 official languages. It makes some models available as open weights.
Read and watch3
Where the facts on this page come from.
- ArticleUsage statistics and market share of content languages for websites, October 2026
W3Techs
- ArticleCensus of India 2011: languages and mother tongues
Office of the Registrar General and Census Commissioner, India (copy hosted by Down To Earth)
- ArticleIndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages
ACL 2024, via arXiv