From 3D perception to safety reasoning: an intelligent hybrid framework for real-time underground mine monitoring

Abstract

Underground coal mining requires personnel and heavy equipment to operate within shared, confined, and poorly illuminated spaces where hazards such as equipment proximity violations, structural instabilities, and occluded blind spots are difficult to anticipate. Conventional monitoring systems, including fixed cameras and rule-based proximity alerts, can detect some predefined events, but often lack the three-dimensional scene understanding and contextual memory needed to identify complex or developing hazards. This paper presents an intelligent hybrid monitoring framework that converts colourised 3D point clouds into structured safety reasoning outputs. The framework combines 3D semantic perception, uncertainty-based anomaly detection, temporal graph representation, rule-based hazard checks, on-device large language model (LLM) reasoning, and retrieval-augmented historical memory analysis to detect immediate hazards and interpret longer-term safety patterns. To overcome the scarcity of labelled underground data, real roadway scans, controlled object placement, and high-fidelity longwall simulation were combined to generate diverse hazard scenarios, while contrastive self-supervised pretraining improved segmentation from limited annotations. Using this strategy, the perception model achieved 92.7% overall accuracy and 0.86 mean intersection over union at 30.2 frames per second. Across 115 controlled hazard scenarios, rule-based checks achieved 57% coverage, increasing to 76% with contextual LLM reasoning and 93% with memory-based reasoning using relevant historical records. Qualitative results show that uncertainty-derived anomaly signals can support the interpretation of unlabelled or out-of-distribution structural hazards beyond predefined semantic classes. Overall, the proposed framework demonstrates that combining safety rules, neural perception, and on-device LLM reasoning provides a practical foundation for continuous intelligent safety monitoring in underground mining.

The authors' abstract, as published at the source. Expert Systems with Applications, 2026 · DOI ↗