MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models

Abstract

Smart contract vulnerabilities pose significant security risks to blockchain systems, potentially leading to severe financial losses. Existing methods face several limitations: (1) Program analysis-based approaches heavily rely on predefined patterns, lacking flexibility and adaptability to new vulnerability types; (2) Deep learning-based methods lack explanations for their detection results; (3) Large language model-based approaches suffer from high false positive rates. To address these challenges, we propose MOS, a smart contract vulnerability detection framework based on mixture-of-experts tuning (MOE-Tuning) of large language models. First, we conduct Continual Pre-training on a large-scale smart contract dataset to provide a domain-enhanced initialization for the subsequent MOE-Tuning process. Second, to provide reliable explanations for detected vulnerabilities, we construct a high-quality MOE-Tuning dataset through a multi-stage pipeline combining large language model generation and expert verification. Third, we design a vulnerability-aware routing mechanism that activates the most relevant expert networks by analyzing code features and their matching degree with experts, addressing the limitation of predefined patterns in program analysis approaches. Finally, we extend the feed-forward layers of the LLM after Continual Pre-training into multiple parallel expert networks, each specializing in specific vulnerability patterns. We also employ a dual-objective loss function: one for optimizing vulnerability detection and explanation performance, and another for balancing expert utilization through an auxiliary loss based on the scaled dot product of expert assignment frequencies and average routing probabilities, thereby improving model performance in complex vulnerability scenarios. Extensive experiments show that MOS consistently outperforms existing state-of-the-art methods in four foundational and historically significant vulnerability types (Reentrancy, Timestamp, Integer Overflow/Underflow, Delegatecall) with average improvements of 6.38% in F1 score and 4.84% in accuracy. The vulnerability explanations generated by MOS also demonstrate high quality, achieving positive ratings (scores of 3 and 4 on a 4-point scale) of 82.96%, 85.21% and 94.58% for correctness, completeness, and conciseness, respectively, through a combined approach of human evaluation and LLM evaluation. Extensibility experiments show that MOS effectively handles new vulnerability types (Access Control, Price Oracle Manipulation, Logic Errors, Lack of Input Validation, Slippage, and Risky First Deposit), consistently outperforming all baseline methods across six additional vulnerability categories. MOS achieved an average F1 score of 83.94% and an average accuracy of 87.82%, demonstrating its strong extensibility and flexibility in handling new vulnerabilities.

The authors' abstract, as published at the source. ACM Transactions on Software Engineering and Methodology, 2026 · DOI ↗