Measuring Political Bias and Temporal Opinion Alignment of Large Language Models in Low-Resource Languages (Proposal Defence)

Large language models (LLMs) are increasingly used to read, interpret and generate political content. Yet existing measures of their political bias rely on Western instruments, most prominently the Political Compass Test, and on high-resource languages. Whether these tools work for Sinhala has not been tested. Sinhala is a low-resource language spoken by about 17 million people, and its political context is shaped by ethnicity, religion and devolution rather than a Western left–right divide. This research addresses that gap through three connected studies. First, we construct the first annotated corpus of Sinhala political discourse from YouTube comments. Native speakers label each comment along the economic and social axes of the Political Compass and by time period, which lets us test whether the framework fits Sinhala discourse. Second, we evaluate LLMs across families and scales as political annotators. We compare their continuous two-dimensional ideology scores against human judgements and examine whether their errors come from limited Sinhala ability or from systematic political asymmetry. Third, we use Sri Lanka’s sharp opinion shifts between 2021 and 2026 to measure whether model-expressed opinions reflect present-day public opinion or a snapshot fixed at the training cutoff. We also test whether fine-tuning on time-segmented discourse shifts a model’s political alignment. The work contributes a reusable resource and annotation framework, evidence on LLM annotation reliability, and the first measurement of temporal drift in LLM political opinion.