A Survey-Aware Evaluation Protocol for Feature Selection in Educational Data Mining: Variable-Length Particle Swarm Optimisation Benchmarked on PISA 2018

Abstract

Educational data mining (EDM) often reports classifiers and feature selectors fitted to large-scale assessments without respecting how these surveys are designed. This study specifies an evaluation protocol for such data—a published predictor allowlist enforced in code, school-grouped nested cross-validation, outcome categories derived from each of ten plausible values and combined by Rubin’s rules, and design-based standard errors from replicate weights for the fitted prediction rules—and uses it to benchmark variable-length particle swarm optimisation (VLPSO) against fifteen alternatives on the Spanish sample of PISA 2018 (29,786 students, 1087 schools). Mathematics proficiency bands were discriminated with an area under the curve of 0.8850 (Low vs. High), 0.7520 (Low vs. Medium) and 0.6963 (Medium vs. High); a label-permutation null averaged 0.4989, and measurement error contributed about half of the variance. Across 12,000 matched selector fits, no method improved on the full predictor set. VLPSO retained four to six of 31 items at a cost of 0.0257–0.0368 in AUC, did not differ from binary swarm optimisation, and was less stable than most filter and embedded selectors. SHAP rankings were concordant across 50 models per task, and models transferred to Portugal without loss. VLPSO compresses predictors at a measurable cost; it does not improve prediction.

The authors' abstract, as published at the source. Applied System Innovation, 2026 · DOI ↗