Evaluating Spurious Feature Reliance Beyond Worst-Group Accuracy
Worst-group accuracy (WGA) has become the de facto metric for assessing whether a machine learning model fails on underserved subsets of data. WGA measures the accuracy of the full-trained predictor on the lowest-performing group, and it is widely reported in fairness and robustness studies. However, WGA evaluates the combined effect of a frozen backbone and a learned head. It does not characterize how the backbone behaves when a new head is trained, nor does it isolate the role of spurious features in the model's internal representations. This limitation matters because two models can achieve the same WGA while relying on different feature cues, and the head can be retrained to exploit or suppress those cues. The paper introduces BiasFlow to address this gap.
Geometric Diagnostics: IBMI and W-IBMI
BiasFlow is a hook-based toolkit that monitors three geometric properties of the backbone's feature space. The first, class-attribute centroid alignment (IBMI), computes the cosine similarity between the vector from a class centroid to its associated attribute direction and the reverse direction. IBMI is confounded by correlation between class labels and spurious attributes—for instance, when "watery" texture co-occurs with "grass" label in training data, a high IBMI score may reflect correlation rather than causal reliance on the texture feature. The second diagnostic, within-class centroid separation (W-IBMI), measures the distance between centroids of different attribute values within the same class label. The third, feature-projection sensitivity, quantifies how much a backbone's output changes when the learned head's weights are perturbed, offering a view of head-backbone interactiveness.
The authors argue that IBMI alone is insufficient as a measure of causal feature reliance because class-attribute correlation can produce inflated alignment scores without the model truly encoding the attribute as a predictive feature. W-IBMI provides a scale-dependent check: it verifies that the quantity BiasFlow Regularization (BFR) optimizes, but W-IBMI does not independently establish that spurious attributes have been removed from the representation.
BiasFlow Regularization (BFR)
BiasFlow Regularization is a supervised, composable penalty applied to the training objective. It operates on class-conditional centroids: for each class, the alignment between the class centroid and its associated spurious attribute direction is minimized, while the within-class separation of opposite attribute directions is encouraged. The penalty is class-conditional, meaning it is applied independently per label, and it is composable, meaning it can be added to existing objectives such as empirical risk minimization or GroupDRO without modifying the backbone architecture. BFR is evaluated by monitoring W-IBMI during training, which the authors show tracks the optimization target of the regularizer.
Empirical Results on Small-Scale Benchmarks
Across the small-scale benchmarks reported in the paper, adding BFR improves or preserves mean worst-group accuracy. The largest gain is +26.0 percentage points on the UrbanCars dataset, where the bias arises from background-context correlation spurious to the vehicle label. On other datasets, the effect is positive but smaller, and in some cases BFR preserves WGA without improvement. These results suggest that centroid alignment regularization can shift the representation geometry in ways that improve worst-group performance, at least within the scale of the evaluated experiments.
Independent Stress Test: Freezing Backbones and Retraining Heads
The principal stress test freezes CelebA-Std backbones and trains fresh heads on biased data. Without regularization, the head trained on biased data yields a worst-group accuracy of 40.7%. Adding BFR alongside GroupDRO raises this to 64.1%, a gain of 23.4 percentage points. Critically, the authors also measure Male probe accuracy, which reflects how much attribute information (in this case, gender-predictive features) remains recoverable from the frozen backbone. Male probe accuracy drops from 92.5% to 72.2% when BFR is active, indicating that the regularizer reduces the recoverability of the spurious attribute without eliminating it entirely. Attribute information remains partially recoverable, and cross-task results are mixed: gains on UrbanCars do not always transfer to CelebA or other domains.
Synthetic-Watermark ImageNet Experiment
A controlled experiment using a synthetic watermark embedded in ImageNet evaluates BFR's behavior under matched training conditions. The watermark-shift accuracy, which measures the ability to detect whether a model has been trained on a modified version of ImageNet, improves by +23.0 percentage points with BFR. This result supports the interpretation that BFR shapes the backbone's feature geometry in a way that is detectable and measurable when the training distribution shifts, even when the modification is subtle (a synthetic watermark overlaid on a fraction of images).
What the Results Support
The authors conclude that evaluating centroid geometry and resistance to biased head retraining alongside WGA provides a fuller picture of spurious feature reliance. BFR and the associated diagnostics give researchers tools to monitor and regularize the backbone's feature space, not just the final predictor's worst-group performance. The results show that WGA can improve when the backbone is regularized toward centroid alignment, and that spurious attribute recoverability can be reduced without complete removal—a practical middle ground for deployment scenarios where some attribute information must be preserved for downstream tasks.
Limitations and Open Questions
The authors note that the reported benchmarks are small-scale, and the transferability of BFR effects to larger models and broader domains remains an open question. The method is supervised, requiring class and attribute labels at training time, which limits its applicability in unsupervised or self-supervised settings. Cross-task results are mixed, indicating that centroid alignment regularization may interact with dataset-specific bias structures in ways that are not yet fully characterized. The relationship between W-IBMI optimization and actual reduction in spurious feature reliance is verified only within the reported protocols, and other generalization guarantees are not claimed.
Practical Considerations for Researchers
For practitioners dealing with spurious feature reliance, the paper offers a concrete starting point: monitor IBMI and W-IBMI using the provided hook-based toolkit, then add BFR as a regularizer alongside existing objectives such as GroupDRO. The experimental results suggest that this combination can improve worst-group accuracy on biased benchmarks, particularly when the spurious correlation is strong and structured around class-conditional attributes. The synthetic-watermark experiment indicates that BFR's effects are measurable even under distribution shift, which may be useful for audit scenarios.
Conclusion
Worst-group accuracy alone does not characterize how a frozen backbone behaves when a new head is learned, and it does not isolate the role of spurious features in the model's internal representations. BiasFlow provides a suite of geometric diagnostics—IBMI, W-IBMI, and feature-projection sensitivity—that monitor class-attribute centroid alignment, within-class separation, and head-backbone interactiveness, respectively. Paired with BiasFlow Regularization, a class-conditional centroid-alignment penalty, the toolkit enables backbone regularization that improves or preserves worst-group accuracy across reported benchmarks, with gains up to +26.0 percentage points on UrbanCars. The independent stress test shows that freezing CelebA-Std backbones and training fresh heads on biased data yields worse-group accuracy of 64.1% with BFR+GroupDRO, down from 40.7% without, while Male probe accuracy decreases from 92.5% to 72.2%, indicating reduced but not eliminated recoverability of spurious attribute information. A synthetic-watermark ImageNet experiment improves watermark-shift accuracy by +23.0 percentage points under matched training. These results support evaluating centroid geometry and resistance to biased head retraining alongside WGA, within the tested protocols.
Read the paper on arXiv