In the process of numerically modeling natural languages, developing language embeddings is a vital step. However, it is challenging to develop functional embeddings for resource-poor languages such as Sinhala, for which sufficiently large corpora, effective language parsers, and any other required resources are difficult to find. In such conditions, the exploitation of existing models to come up with an efficacious embedding methodology to numerically represent text could be quite fruitful. This paper explores the effectivity of several one-tiered and two-tiered embedding architectures in representing Sinhala text in the sentiment analysis domain. With our findings, the two-tiered embedding architecture where the lower-tier consists of a word embedding and the upper-tier consists of a sentence embedding has been proven to perform better than one-tier word embeddings, by achieving a maximum F1 score of 88.04\% in contrast to the 83.76\% achieved by word embedding models. Furthermore, embeddings in the hyperbolic space are also developed and compared with Euclidean embeddings in terms of performance. A sentiment data set consisting of Facebook posts and associated reactions have been used for this research. To effectively compare the performance of different embedding systems, the same deep neural network structure has been trained on sentiment data with each of the embedding systems used to encode the text associated.
Keywords: Natural Language Processing | Machine Learning / Deep Learning | Sinhala | Sentence Embedding | Low-resource Languages | Semantics | Representation |
@inproceedings{weeraprameshwara2022sinhala,
title={{Sinhala Sentence Embedding: A Two-Tiered Structure for Low-Resource Languages}},
author={Gihan Weeraprameshwara and Vihanga Jayawickrama and Nisansa de Silva and Yudhanjaya Wijeratne},
booktitle={Proceedings of the 36th Pacific Asia Conference on Language, Information and Computation},
pages={325--336},
year={2022},
doi = {10.48550/ARXIV.2210.14472},
abstract={In the process of numerically modeling natural languages, developing language embeddings is a vital step. However, it is challenging to develop functional embeddings for resource-poor languages such as Sinhala, for which sufficiently large corpora, effective language parsers, and any other required resources are difficult to find. In such conditions, the exploitation of existing models to come up with an efficacious embedding methodology to numerically represent text could be quite fruitful. This paper explores the effectivity of several one-tiered and two-tiered embedding architectures in representing Sinhala text in the sentiment analysis domain. With our findings, the two-tiered embedding architecture where the lower-tier consists of a word embedding and the upper-tier consists of a sentence embedding has been proven to perform better than one-tier word embeddings, by achieving a maximum F1 score of 88.04\% in contrast to the 83.76\% achieved by word embedding models. Furthermore, embeddings in the hyperbolic space are also developed and compared with Euclidean embeddings in terms of performance. A sentiment data set consisting of Facebook posts and associated reactions have been used for this research. To effectively compare the performance of different embedding systems, the same deep neural network structure has been trained on sentiment data with each of the embedding systems used to encode the text associated.},
misc={https://aclanthology.org/2022.paclic-1.36.pdf,NLP:ML:SIN,https://goo.gl/iY6aTr}
}
ActivePI: Nisansa de Silva
The Knowledge Representation project investigates methods for structuring, aligning, and reasoning over knowledge extracted from language, with particular emphasis on low-resource settings, domain-specific semantics, and intelligent information access.
ActivePI: Nisansa de Silva
We propose to advance natural language processing for low-resource settings by leveraging data augmentation, transfer learning, and model adaptation techniques to overcome the scarcity of annotated data and linguistic resources.
ActivePI: Nisansa de Silva
We propose to build and analyze Sinhala language resources, datasets, and models to strengthen NLP research for this low-resource language, while connecting it to English and Tamil through bilingual lexicons, embeddings, and cross-lingual methods.
ActivePI: Nisansa de Silva
We propose to advance social media text analysis by combining linguistic study, preprocessing, and deep learning models to better capture sentiment, structure, and meaning in informal, large-scale user-generated text.