Share a website in your language
Help build a map of the web for every language — find yours and add a site you trust.
We're already tracking ··· languages — of them already have a source.
About this project
LingoCrawl is an unofficial contribution form for Common Crawl's web-languages project, a crowd-sourced, public-domain effort to improve web crawling for under-resourced languages.
“We don't have enough of languages like Hindi (500 million speakers!), smaller country languages like Hungarian, and regional languages like Catalan. We are interested in languages from all over the world.”
Volunteers read through new links before adding them to the public-domain (CC0) web-languages list. The project also runs a Discord server for discussion.
How it works
- 1. Search your language — find it above, by name or an alternate spelling.
- 2. Share a link — paste the address of a website written in it.
- 3. Pick a category — news, culture, government, political, or other — then submit.
What kind of websites help?
- Websites written in your language. A site that is all English is not what we need.
- Sites that are important and widely used. If many regions speak your language, try to include different regions.
- If your language is only one part of a site, share that exact address, for example example.com/catalan/.
- If your language has its own Wikipedia, please add it under “Other”.
- Not sure? Do your best. You don't need to fill every category.