npj Precision Oncology· 2026Q1
Human-in-the-loop large language model framework for identification of cutaneous immune-related adverse events
- 0citations
- Q1SCImago
- 2026year
Short summary
A retrieval-augmented, multi-agent LLM framework with human oversight improved the detection of cutaneous immune-related adverse events (cirAEs) from clinical notes, achieving higher accuracy (F1=0.88 vs 0.77) and inter-rater agreement (Cohen’s κ=0.82 vs 0.50) compared to manual review alone.
AI-generated from the title and abstract; the full text is not read.
Key points
- LLM framework with human oversight achieved higher accuracy (F1=0.88) in detecting cirAEs than manual review (F1=0.77).
- Inter-rater agreement was substantially improved with the LLM framework (Cohen’s κ=0.82) compared to manual review (κ=0.50).
- The LLM-assisted review process was approximately one-third faster than manual review.
- The framework utilizes a retrieval-augmented, multi-agent LLM approach.
AI-generated from the title and abstract; the full text is not read.
Abstract
This study evaluated a retrieval-augmented, multi-agent large language model (LLM)–driven, human-in-the-loop framework for detecting cutaneous immune-related adverse events (cirAEs) from clinical notes. Compared with the unassisted manual-review group, the LLM-assisted group demonstrated higher accuracy (F1 = 0.88 vs 0.77) and inter-rater agreement measured by Cohen’s κ ( κ = 0.82 vs 0.50), and reported an approximately one-third shorter average review time. This framework pilots how LLMs can be applied to identify immune-related toxicities and, more broadly, support accurate, scalable, and transparent adverse event data extraction.
The authors' abstract, as published at the source. npj Precision Oncology, 2026 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
ToxicologyPharmacology, Toxicology and Pharmaceutics