ACM Computing Surveys· 2025Q1· Review
Instruction Tuning for Large Language Models: A Survey
- 179citations
- Q1SCImago
- 2025year
Short summary
Instruction tuning (IT) is a key technique for aligning large language models (LLMs) with human instructions by training them on (instruction, output) pairs, bridging the gap between next-word prediction and user intent.
AI-generated from the title and abstract; the full text is not read.
Key points
- Instruction tuning (IT) trains LLMs on (instruction, output) pairs to align their behavior with human instructions.
- IT bridges the gap between LLMs' next-word prediction objective and users' need for instruction adherence.
- The survey systematically reviews IT methodology, dataset construction, training, and applications across modalities and domains.
- Analysis includes factors influencing IT outcomes and potential pitfalls, with suggestions for future research.
AI-generated from the title and abstract; the full text is not read.
Abstract
This article surveys research works in the quickly advancing field of instruction tuning (IT), a crucial technique to enhance the capabilities and controllability of large language models (LLMs). Instruction tuning refers to the process of further training LLMs on a dataset consisting of (instruction, output) pairs in a supervised fashion, which bridges the gap between the next-word prediction objective of LLMs and the users’ objective of having LLMs adhere to human instructions. In this work, we make a systematic review of the literature, including the general methodology of IT, the construction of IT datasets, the training of IT models, and applications to different modalities, domains and application, along with analysis of aspects that influence the outcome of IT (e.g., generation of instruction outputs, size of the instruction dataset). We also review the potential pitfalls of IT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies and suggest some avenues for fruitful research.
The authors' abstract, as published at the source. ACM Computing Surveys, 2025 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Artificial Intelligence
Artificial IntelligenceComputer Science