Automatic headline generation for Sinhala presents a significant challenge due to the language's unique characteristics and the limited resources available for its natural language processing (NLP). This research addresses this critical gap by utilizing advanced models such as mBART, mT5, and Llama 3 to generate accurate, contextually relevant headlines for Sinhala news articles. The research utilizes parameter-efficient fine-tuning (PEFT) methods, including adapters for feed-forward networks and attention layers, to improve performance and computational efficiency. The models are evaluated across four diverse Sinhala news datasets, and the findings indicate that mBART outperforms both mT5 and Llama 3 in generating concise and coherent headlines, especially for structured datasets. This demonstrates mBART's robustness in handling the complexities of Sinhala. However, while mBART excels with formal content, it struggles with informal datasets like Hiru and Gossip Lanka, highlighting the need for further refinement and specialized techniques to handle the diverse linguistic features of these sources. Overall, this work not only advances automatic headline generation for Sinhala but also makes a valuable contribution to the broader field of NLP, addressing the challenges posed by low-resource languages and the varying content styles they encompass.
Keywords: Natural Language Processing | Machine Learning / Deep Learning | Sinhala | Text Summarization | Text Generation | LLM |
@phdthesis{cooray2025headline,
title={Headline Generation for Sinhala Newspaper Articles using Pre-trained Language Models},
author={Cooray, Pubudu},
year={2025},
abstract={Automatic headline generation for Sinhala presents a significant challenge due to the language's unique characteristics and the limited resources available for its natural language processing (NLP). This research addresses this critical gap by utilizing advanced models such as mBART, mT5, and Llama 3 to generate accurate, contextually relevant headlines for Sinhala news articles. The research utilizes parameter-efficient fine-tuning (PEFT) methods, including adapters for feed-forward networks and attention layers, to improve performance and computational efficiency. The models are evaluated across four diverse Sinhala news datasets, and the findings indicate that mBART outperforms both mT5 and Llama 3 in generating concise and coherent headlines, especially for structured datasets. This demonstrates mBART's robustness in handling the complexities of Sinhala. However, while mBART excels with formal content, it struggles with informal datasets like Hiru and Gossip Lanka, highlighting the need for further refinement and specialized techniques to handle the diverse linguistic features of these sources. Overall, this work not only advances automatic headline generation for Sinhala but also makes a valuable contribution to the broader field of NLP, addressing the challenges posed by low-resource languages and the varying content styles they encompass.},
school={University of Moratuwa},
misc={https://nisansads.staff.uom.lk/Group/people.php?name=Pubudu_Cooray,NLP:ML:SIN,https://goo.gl/iY6aTr},
url={https://nisansads.staff.uom.lk/Group/people.php?name=Pubudu_Cooray}
}
ActivePI: Nisansa de Silva
We propose to advance natural language processing for low-resource settings by leveraging data augmentation, transfer learning, and model adaptation techniques to overcome the scarcity of annotated data and linguistic resources.
ActivePI: Nisansa de Silva
We propose to build and analyze Sinhala language resources, datasets, and models to strengthen NLP research for this low-resource language, while connecting it to English and Tamil through bilingual lexicons, embeddings, and cross-lingual methods.
ActivePI: Nisansa de Silva
We propose to advance automatic summarization and generation techniques, with a focus on aspect-aware methods that capture fine-grained information and sentiment within concise textual representations.