The Journal of Politics· 2021Q1
Word Embeddings: What Works, What Doesn’t, and How to Tell the Difference for Applied Research
- 245citations
- Q1SCImago
- 2021year
Short summary
Political science research using word embeddings shows results are generally robust to parameter choices (context window, dimensions, pretrained vs. local fit) across corpora and languages.
AI-generated from the title and abstract; the full text is not read.
Key points
- Word embedding results in political science are generally robust to common parameter choices like context window length and vector dimensions.
- Pretrained word embedding models perform comparably to or better than human coders and locally fit models.
- A novel crowdsourced Turing test method is introduced to evaluate the substantive performance of text-based models.
- Best practice advice is provided for situations requiring local fitting of word embedding models.
AI-generated from the title and abstract; the full text is not read.
Abstract
Word embeddings are becoming popular for political science research, yet we know little about their properties and performance. To help scholars seeking to use these techniques, we explore the effects of key parameter choices—including context window length, embedding vector dimensions, and pretrained versus locally fit variants—on the efficiency and quality of inferences possible with these models. Reassuringly we show that results are generally robust to such choices for political corpora of various sizes and in various languages. Beyond reporting extensive technical findings, we provide a novel crowdsourced “Turing test”–style method for examining the relative performance of any two models that produce substantive, text-based outputs. Our results are encouraging: popular, easily available pretrained embeddings perform at a level close to—or surpassing—both human coders and more complicated locally fit models. For completeness, we provide best practice advice for cases where local fitting is required.
The authors' abstract, as published at the source. The Journal of Politics, 2021 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: General Social Sciences
General Social SciencesSocial Sciences