JavaBackports: A Dataset for Benchmarking Automated Backporting in Java

K. Kahapola, S. Galappaththi, D. Ranasinghe, R. Shariffdeen, N. de Silva, S. Perera, and S. Wickramanayake 23rd International Conference on Mining Software Repositories (MSR), 2026, Manually backporting critical patches to long-term support versions is both error-prone and often overlooked, resulting in substantial security risks. Progress in this area is constrained by the absence of datasets that capture the semantic complexities across versions, inherent to backporting in large Java ecosystems. To address this gap, we present JavaBackports, a curated dataset of 491 realworld backport instances, systematically selected and manually validated from more than 11,000 candidate patches in fifteen widely used open-source Java projects: Druid, Elasticsearch, Hadoop, Kafka, among others and four major JDK versions (jdk11, jdk17, jdk21, jdk25). To assess the utility of JavaBackports, we conduct preliminary experiments to evaluate the effectiveness of the state of-the-art Large Language Models (LLMs) in zero-shot automatic patch backporting. The results indicate that current LLMs struggle with backporting tasks, particularly when the required changes involve non-trivial logical or structural modifications. These findings demonstrate both the difficulty of the problem and the potential of JavaBackports to stimulate new research directions in automated software maintenance and repair.