AMF-U-Net: an adaptive multimodal fusion residual attention 3D U-Net for boundary-aware brain tumour segmentation

Abstract

Precise 3D segmentation of brain tumors based on multi-modal magnetic resonance images(MRI) is challenging due to intensity heterogeneity within tumor sub-regions, irregular tumor boundaries, extreme unbalanced class distribution at the voxel level and acquisition variability. This study introduces AMF-U-Net, a multi-stream residual 3D U-Net model, which takes T1, contrast-enhanced T1 (T1ce), T2 and FLAIR images as input and goes through four separate encoders for modality-specific feature extraction. For each scale level in encoders, The proposed system used the Modality Fusion Module, which computes softmax-normalised modality importance weights and fuses modality-specific features before forwarding them to attention-guided decoders. Residual connections make training more stable, while attention gates suppress irrelevant skip connection activations and help to reconstruct boundaries. Hybrid class weight-balanced Dice and Categorical Cross Entropy loss handle regional overlap and class imbalance issues. Brain Tumor Segmentation 2023 and UCSF-PDGM datasets have been harmonised using modality mappings, spatial normalisation, source-aware patient-level split and label transformation to mutually exclusive background, necrotic/non-enhancing tumor, oedema and enhancing tumor classes. For the internal validation cohort, AMF-U-Net yielded Dice scores of 0.845, 0.813 and 0.788 for WT, TC and ET, respectively, which equate to a macro-average Dice score of 0.815 on the region level. When compared with same-split results of 3D U-Net, nnU-Net, UNETR, and Swin UNETR, the proposed method provided better performance in terms of overlap and distance from the boundary. Hence, the contribution here should be regarded as the combination of all three innovations, and not just as the introduction of the individual innovations.

The authors' abstract, as published at the source. Scientific Reports, 2026 · DOI ↗