Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook

Yudhanjaya Wijeratne, Nisansa de Silva
arXiv preprint arXiv:2007.07884

This paper presents two colloquial Sinhala language corpora from the language efforts of the Data, Analysis and Policy team of LIRNEasia, as well as a list of algorithmically derived stopwords. The larger of the two corpora spans 2010 to 2020 and contains 28,825,820 to 29,549,672 words of multilingual text posted by 533 Sri Lankan Facebook pages, including politics, media, celebrities, and other categories; the smaller corpus amounts to 5,402,76 words of only Sinhala text extracted from the larger. Both corpora have markers for their date of creation, page of origin, and content type.

Keywords: Natural Language Processing | Machine Learning / Deep Learning | Big Data | Sinhala |