LLM'ler Nadir Kategoriler İçin Sınıflandırma Doğruluğunu Artırmak Üzere Sentetik Metin Üretiyor

Özet (abstract)

Abstract Annotated text data are central to many empirical applications in political science, yet many annotation tasks involve categories that are sparsely populated. These low- N categories pose problems for both encoder-only and decoder-only (generative) Transformer large language models (LLMs), leading to low performance, biased evaluation and measurement error in downstream analyses. This article introduces the synthetic imputation approach, a targeted method for expanding sparse categories while remaining consistent with the category-defining content of the human-coded examples. The approach uses a generative LLM to produce additional texts for underrepresented categories by conditioning on small, rotating subsets of human-coded examples. Rather than paraphrasing or replacing the original data, synthetic imputation adds informative variation only where data are thin. I evaluate the approach using both encoder-only and decoder-only LLM classifiers across multiple political science applications. The results show that synthetic imputation substantially improves performance and evaluation stability in sparse settings, recovering full-sample performance once a modest number of true observations is available, with limited and predictable overfitting at very low levels of N .

Yazarların özeti; kaynağından alınmıştır. Political Analysis, 2026 · DOI ↗