Multi-document summarisation (MDS) involves generating concise summaries from clusters of related documents. PRIMERA (Pyramid-based Masked Sentence Pre-training for Multi-document Summarisation) is a pre-trained model specifically designed for MDS, utilizing the LED architecture to handle long sequences effectively. Despite its capabilities, fine-tuning PRIMERA for specific tasks remains resource-intensive. To mitigate this, we explore the integration of adapter modules-small, trainable components inserted within transformer layers-that allow models to adapt to new tasks by updating only a fraction of the parameters, thereby reducing computational requirements.
Keywords: Natural Language Processing | Machine Learning / Deep Learning | Multi-document summarization | Pre-trained Models |
@inproceedings{hewapathirana2025adapter,
title={{Adapter-based Fine-tuning for PRIMERA}},
author={Hewapathirana, Kushan and de Silva, Nisansa and Athuraliya, C. D.},
booktitle={Applied Data Science \& Artificial Intelligence Symposium},
month={04},
year={2025},
abstract={Multi-document summarisation (MDS) involves generating concise summaries from clusters of related documents. PRIMERA (Pyramid-based Masked Sentence Pre-training for Multi-document Summarisation) is a pre-trained model specifically designed for MDS, utilizing the LED architecture to handle long sequences effectively. Despite its capabilities, fine-tuning PRIMERA for specific tasks remains resource-intensive. To mitigate this, we explore the integration of adapter modules-small, trainable components inserted within transformer layers-that allow models to adapt to new tasks by updating only a fraction of the parameters, thereby reducing computational requirements.},
doi={10.31705/ADScAI.2025.57},
misc={https://dl.lib.uom.lk/server/api/core/bitstreams/5bd121c1-3a11-4eb3-b5a4-4bacb770c3b5/content,NLP:ML,https://goo.gl/iY6aTr}
}
ActivePI: Nisansa de Silva
We propose to advance natural language processing for low-resource settings by leveraging data augmentation, transfer learning, and model adaptation techniques to overcome the scarcity of annotated data and linguistic resources.
ActivePI: Nisansa de Silva
We propose to advance automatic summarization and generation techniques, with a focus on aspect-aware methods that capture fine-grained information and sentiment within concise textual representations.