Explainable multimodal learning analytics for model-based counterfactual decision support using literature-calibrated synthetic data

Abstract

Predictive learning analytics can identify students who are at risk of failing, however, there is a lack of information to guide advisors on what they should do. This paper proposes an end-to-end synthetic framework using eight structured indicators of Learning Management System (LMS) and an aggregated VADER sentiment. The XGBoost pipeline model had accuracy of 80.7%, precision of 76.5%, recall of 90.3%, an F1 score of 0.828, and an ROC-AUC score of 0.905 at the fixed cut-point of 0.28; logistic regression had the best ROC-AUC score of 0.930. With sentiment removal, the accuracy of XGBoost dropped to 76.7% and the value of ROC-AUC to 0.875. The constrained greedy counterfactual search screened 149 XGBoost cases separately, dropped 80 cases (53.7\%) below the target of 0.40, and reduced the score by a mean of 33.3% points. Threshold crossing rate for logistic regression and random forest were 100.0% and 41.4% respectively with the same search, highlighting model dependence. A 27-configuration, training-only sensitivity analysis placed the fixed XGBoost configuration 12th by five-fold cross-validated ROC-AUC (0.929), compared with 0.933 for the best configuration. DiCE-random crossed the target for 74.5% of the same XGBoost-screened cases but required larger normalized changes and approximately four times the runtime of greedy search. These results characterize a reconstructed synthetic model and its search procedures; they do not establish educational validity, causal effectiveness, calibrated institutional risk, least-burden recourse, or deployment readiness.

The authors' abstract, as published at the source. Scientific Reports, 2026 · DOI ↗