Leakage-aware benchmarking and controlled ablation of attention models for student performance prediction across multiple tabular datasets

Abstract

This study presents a leakage- and duplicate-aware benchmark of seven models for student performance prediction across UCI Dropout, Mendeley AI, and Kaggle 2024. The target-proxy ExamScore variable was removed from Kaggle 2024, and identical feature groups were kept disjoint through group-aware validation. UCI Dropout and Kaggle 2024 used 10 repetitions of five outer folds (50 folds each), whereas Mendeley AI used 10 repetitions of two outer folds (20 folds) because its 560 observations are distributed across 10 small grade classes. All preprocessing and hyperparameter tuning were confined to the relevant training partitions, and the self-attention model was compared with a jointly tuned, capacity-matched no-attention ablation. The resulting benchmark comprised 840 model-fold evaluations. Gradient Boosting ranked first on all three datasets, with Macro-F1 ranging from 0.6988 to 0.9185. Self-attention did not significantly outperform its matched ablation on any dataset after corrected paired testing and remained less competitive than the strongest tree ensembles. These findings provide a reproducible negative result: under leakage-aware validation and controlled model capacity, feature self-attention did not establish a consistent predictive advantage for the evaluated educational tabular tasks.

The authors' abstract, as published at the source. Discover Artificial Intelligence, 2026 · DOI ↗