Computational Linguistics· 2026Q1· Review
What is the Role of Small Models in the LLM Era: A Survey
- 8citations
- Q1SCImago
- 2026year
Short summary
This survey systematically examines the relationship between Large Language Models (LLMs) and Small Models (SMs) from the perspectives of collaboration and competition, highlighting the underappreciated significance of SMs in practical settings.
AI-generated from the title and abstract; the full text is not read.
Key points
- LLMs' increasing size leads to high computational costs and energy consumption, making them impractical for resource-limited users.
- Small Models (SMs) are frequently used in practical settings but their significance is underestimated.
- This survey systematically examines the relationship between LLMs and SMs from two key perspectives: Collaboration and Competition.
- The work aims to provide insights for practitioners on the contribution of small models and promote efficient resource use.
AI-generated from the title and abstract; the full text is not read.
Abstract
Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities in various reasoning tasks, which leads to the development of increasingly large models. However, scaling up model sizes results in significantly higher computational costs and energy consumption, which makes these models impractical for academic researchers and businesses with limited resources. At the same time, Small Models (SMs) are frequently used in practical settings, although their significance is currently underestimated. This raises important questions about the role of small models in the era of LLMs, a topic that has received limited attention in prior surveys. In this work, we systematically examine the relationship between LLMs and SMs from two key perspectives: Collaboration and Competition (or Complementarity). We hope this survey provides valuable insights for practitioners, fostering a deeper understanding of the contribution of small models and promoting more efficient use of computational resources. Code and additional resources for this survey are available at: https://github.com/tigerchen52/awesome_role_of_small_models.
The authors' abstract, as published at the source. Computational Linguistics, 2026 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Information Systems
Information SystemsComputer Science