UGSA-Net: Uncertainty-Gated Attention and Self-Supervised Alignment for Calibration-Aware Hyperspectral Image Classification

Abstract

Uniform spatial weighting in patch-based hyperspectral classifiers indiscriminately aggregates spectral information, even when individual pixels are already class-diagnostic, degrading accuracy and interpretability. We propose UGSA-Net, an ambiguity-guided spatial–spectral transformer that infers a per-pixel ambiguity field without direct label supervision. A lightweight ambiguity head maps each token to a scalar in (0,1), trained end-to-end with no target of its own, so that the field is shaped only by the training losses that reach it through the token mixer. This field dually modulates the token mixer: it biases query and key projections with a spectral angle mapper (SAM) prior to emphasise spectrally similar patches, and rescales token outputs to amplify contributions from ambiguous locations. The value pathway remains unmodulated, so the network reweights spatial information without distorting the spectral content entering the mixture. The ambiguity head and the classification backbone are trained end-to-end, following a self-supervised latent-alignment stage that uses no labels; model selection jointly scores overall accuracy (OA), few-shot accuracy and expected calibration error (ECE) on validation data. Across six benchmark scenes from four sensors, under a stratified pixel-level split in which training and test windows overlap, UGSA-Net exceeds 98.9% OA on every scene over three runs, and on Indian Pines it is comparable to competing architectures retrained under the same protocol. These values are upper bounds rather than estimates for new scenes: on two of the six scenes, almost no labelled pixel has a competing class inside the 9×9 input window. A single validation-tuned temperature reduces ECE from 0.24–0.28 to below 0.016 without changing any prediction, revealing that residual miscalibration is governed by the decision head rather than by the data representations. The benefit of ambiguity gating emerges when labels are scarce: with a reduced training set on Indian Pines, gating raised OA over the ungated model on all three partitions, in either direction, whereas, with the full training set, component effects fall within run-to-run variation.

The authors' abstract, as published at the source. Machines, 2026 · DOI ↗