This paper presents two colloquial Sinhala language corpora from the language efforts of the Data, Analysis and Policy team of LIRNEasia, as well as a list of algorithmically derived stopwords. The larger of the two corpora spans 2010 to 2020 and contains 28,825,820 to 29,549,672 words of multilingual text posted by 533 Sri Lankan Facebook pages, including politics, media, celebrities, and other categories; the smaller corpus amounts to 5,402,76 words of only Sinhala text extracted from the larger. Both corpora have markers for their date of creation, page of origin, and content type.
Keywords: Natural Language Processing | Machine Learning / Deep Learning | Big Data | Sinhala | Corpus Linguistics | Social Media |
@article{wijeratne2020sinhala,
title={{Sinhala Language Corpora and Stopwords from a Decade of Sri Lankan Facebook}},
author={Wijeratne, Yudhanjaya and de Silva, Nisansa},
journal={arXiv preprint arXiv:2007.07884},
abstract={This paper presents two colloquial Sinhala language corpora from the language efforts of the Data, Analysis and Policy team of LIRNEasia, as well as a list of algorithmically derived stopwords. The larger of the two corpora spans 2010 to 2020 and contains 28,825,820 to 29,549,672 words of multilingual text posted by 533 Sri Lankan Facebook pages, including politics, media, celebrities, and other categories; the smaller corpus amounts to 5,402,76 words of only Sinhala text extracted from the larger. Both corpora have markers for their date of creation, page of origin, and content type.},
year={2020},
doi={10.2139/ssrn.3650976},
misc={https://bit.ly/3jqCUzj,NLP:ML:BIG:SIN,https://i1.rgstatic.net/publication/342975821_Sinhala_Language_Corpora_and_Stopwords_from_a_Decade_of_Sri_Lankan_Facebook/links/5f0fc728a6fdcc3ed70b5725/largepreview.png,https://bit.ly/3jqCUzj}
}
ActivePI: Nisansa de Silva
We propose to advance natural language processing for low-resource settings by leveraging data augmentation, transfer learning, and model adaptation techniques to overcome the scarcity of annotated data and linguistic resources.
ActivePI: Nisansa de Silva
We propose to build and analyze Sinhala language resources, datasets, and models to strengthen NLP research for this low-resource language, while connecting it to English and Tamil through bilingual lexicons, embeddings, and cross-lingual methods.
ActivePI: Nisansa de Silva
We propose to advance social media text analysis by combining linguistic study, preprocessing, and deep learning models to better capture sentiment, structure, and meaning in informal, large-scale user-generated text.