CMAFNet, VR Futbol Yayınlarında Kullanıcı Deneyimi Değerlendirmesi İçin Çok Modlu Veri Füzyonu ile Algoritma Optimizasyonu

Özet (abstract)

Virtual reality technology has opened new avenues for immersive experiences in football match broadcasting. However, how to fuse multi-source heterogeneous perceptual signals to quantitatively assess users’ real-time immersion levels remains a key bottleneck constraining the optimization of VR sports broadcasting experiences. This paper proposes a Cross-Modal Attention Fusion Network (CMAFNet) taking four inputs: match video, synchronized audio, electroencephalogram (EEG) signals, and eye-tracking trajectories. The dataset comprises 720 60-s VR match clips (24 per participant × 30 participants) across four scenario types, yielding 86,400 temporal analysis samples. After temporal alignment within ±5 ms and projection into a common semantic space, modality-specific features enter a hub-and-spoke hierarchical cross-attention module centered on the video modality. A quality-aware adaptive gating mechanism adjusts fusion weights at the per-sample level according to real-time signal quality, while an ordinal regression loss preserves the ordinal structure of immersion levels. CMAFNet achieves an overall accuracy of 81.6% and a mean absolute error of 0.274 on the four-level immersion classification task, outperforming eight baselines including recent 2024 transformer-based approaches, with a 3.4 percentage point accuracy gain and 13.8% MAE reduction over the strongest baseline. Ablation experiments confirm that each of the three core modules provides an irreplaceable contribution. Modality-specific linear probing on the frozen projected features further shows that each channel attains its highest decoding accuracy on the attribute it is claimed to encode, with aligned probing accuracies ranging from 74.2 to 88.7%, and semantically targeted perturbations that remove only the attributed information cause performance degradations 4.1 to 17.4 times larger than control perturbations applied to the same modality, confirming the complementary and non-substitutable nature of the four modalities. Leave-one-subject-out cross-validation across all 30 participants yields a mean accuracy of 79.3 ± 2.1%, and accuracy remains above 68% when any single modality is entirely absent. Computational profiling on consumer-grade GPU hardware achieves 22.4 FPS, satisfying real-time deployment requirements. Under full-modality fusion, presence and emotional involvement increase by 32.6% and 42.4% relative to the no-fusion baseline. This study provides a feasible technical solution for intelligent user experience assessment in immersive VR sports broadcasting.

Yazarların özeti; kaynağından alınmıştır. Discover Artificial Intelligence, 2026 · DOI ↗