Case-knowledge-guided construction-phase safety appraisal report generation for hydraulic and hydropower projects using large language models

Abstract

Safety appraisal during the construction phase of hydraulic and hydropower projects requires professionals to review large volumes of multi-source documents. This manual process is labor-intensive, time-consuming, and vulnerable to evidence omission. Existing automated tools struggle to adapt: rule-checking relies heavily on formalized rules, while current retrieval-augmented generation (RAG) applications are fragmented into single-pass tasks, lacking macroscopic planning and explicit control over evidence use. To address this gap, this study proposes a case-knowledge-guided workflow for generating appraisal reports using large language models (LLMs). The workflow integrates structured knowledge extraction, executable appraisal planning, and iterative evidence-grounded conclusion generation, yielding checkable intermediate products at each stage. We also introduce an LLM-based evaluation suite utilizing Conclusion Support Rate (CSR) and Hallucination Rate (HR), which was validated against expert annotations and showed sufficient agreement for comparative evaluation. A real-world study used 30 historical reports and 9 application cases encompassing 160 evidence documents. Evaluation showed that standard RAG achieved a CSR of only 32% with a 15% HR. In contrast, the proposed workflow reached an 82% CSR after one round of gap-driven evidence supplementation, accompanied by a moderate HR increase to 28%. These results reveal that further evidence supplementation increased hallucination risk in this setting, making a single augmentation round the best observed trade-off. Ultimately, explicit appraisal planning and case-derived knowledge significantly improve conclusion supportability. By making the end-to-end generation process observable step-by-step, this framework provides an auditable pathway for applying LLMs to high-risk engineering appraisals while preserving human expert review.

The authors' abstract, as published at the source. Engineering Applications of Artificial Intelligence, 2026 · DOI ↗