Data Quality Estimation and Improvement
Principal Investigator: Surangika Ranathunga
The Data Quality Estimation and Improvement project focuses on analysing, measuring, and enhancing the quality of datasets used in natural language processing, particularly for multilingual and low-resource language applications.
The Data Quality Estimation and Improvement project focuses on analysing, measuring, and enhancing the quality of datasets used in natural language processing, particularly for multilingual and low-resource language applications. As large web-mined datasets increasingly support modern language technologies, concerns regarding noise, incorrect language labels, and inconsistent data quality have become critical challenges. This project investigates these issues and develops techniques to systematically evaluate and improve dataset quality to ensure reliable downstream performance.
A key component of the project involves auditing multilingual corpora and identifying widespread quality problems, including mislabeled language data and corpora containing large proportions of low-quality sentences. These findings highlight the risks associated with relying on automatically collected data without careful quality assessment. Complementing this analysis, the project explores parallel data curation and filtering methods that rank sentence pairs using multilingual language models. By analysing biases present in different models and applying heuristic strategies, the research demonstrates how noisy parallel sentences can be effectively removed, leading to improved machine translation performance.
In addition, the project develops data augmentation and representation enhancement techniques to generate higher-quality parallel data for low-resource language pairs such as English–Sinhala, English–Tamil, and Sinhala–Tamil. Through improved augmentation strategies and refined cross-lingual representations, the research shows that better data quality can significantly enhance translation performance and contribute to more reliable multilingual language technologies.
Objectives:
- Evaluate the quality of multilingual and web-mined datasets by identifying issues such as noise, incorrect language labels, and inconsistencies that affect the reliability of data used in language technologies.
- Develop methods for filtering and curating parallel datasets to remove low-quality sentence pairs and improve the effectiveness of multilingual applications such as machine translation.
- Design data augmentation and representation enhancement techniques that generate high-quality synthetic data and improve cross-lingual sentence representations, particularly for low-resource language pairs.
- Promote reliable and reproducible research practices by analysing dataset openness, artefact availability, and data quality standards within the natural language processing research community.
Keywords: Natural Language Processing | Machine Learning / Deep Learning | Big Data | Sinhala | Parallel Corpora | Low-resource Languages | Parallel Data Curation | Cross-lingual Representations |
Publications
Dissertations
W.A.S.A Fernando, "Data Augmentation to Induce High Quality Parallel Data for Low-Resource Neural Machine Translation", University of Moratuwa, 2025
Journal Papers
Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, and others, "Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets", Transactions of the Association for Computational Linguistics, vol. 10, pp. 50--72, 2022. doi: 10.1162/tacl_a_00447
Conference Papers
Aloka Fernando, Nisansa de Silva, Menan Velayuthan, Charitha Rathnayaka, and Surangika Ranathunga, "Improving the quality of Web-mined Parallel Corpora of Low-Resource Languages using Debiasing Heuristics", in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 28252--28269. doi: 10.18653/v1/2025.emnlp-main.1435
Surangika Ranathunga, Nisansa de Silva, Dilith Jayakody, and Aloka Fernando, "Shoulders of Giants: A Look at the Degree and Utility of Openness in NLP Research", in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
Surangika Ranathunga, Nisansa de Silva, Menan Velayuthan, Aloka Fernando, and Charitha Rathnayake, "Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corpora", in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), St. Julian{'}s, Malta: Association for Computational Linguistics, mar. 2024, pp. 860--880.
Team
External Collaborators: | Julia Kreutzer | Isaac Caswell |
Faculty
MSc Students
Alumni-PhD Students
Alumni-MSc Students
Alumni-Undergraduates
Grants
This project was partially supported by the following grants:
View grant