Cardiovascular screening models trained on the CDC's Behavioral Risk Factor Surveillance System (BRFSS) routinely report AUROC values near 0.89. That number has been repeated across dozens of studies and several public-facing tools. This paper asks a pointed question: is that accuracy real, or is it an artifact of target leakage, where features recorded after the diagnosis appear among the predictors? The authors benchmark ten classifiers across five feature tiers of decreasing leakage risk and find that two post-diagnostic features account for the entire reported headroom. Once removed, every model, from logistic regression to a 265,240-row tabular foundation model, collapses into a band 0.0045 AUROC wide. The binding constraint on this field is evaluation practice, not model capacity.
The Leakage Problem in Survey-Based Screening
Target leakage is the most common defect in machine-learning-based science. It occurs when predictors are consequences of the outcome rather than antecedents of it, inflating apparent performance without conferring any actual screening ability. In cardiovascular survey data, diagnosis codes recorded in the same encounter as the outcome produce exactly this failure. The BRFSS asks respondents whether they have ever been diagnosed with myocardial infarction, angina, or coronary heart disease, and also asks about chest CT scans. A model that uses prior angina or chest scan history to predict prior myocardial infarction is not screening for heart attacks; it is transcribing a record back to itself.
Prior work has noted this problem in single pipelines, but the field lacked a dose-response measurement: how much of the reported accuracy survives when leakage-prone predictors are removed in controlled steps, and whether the answer depends on the learner. The authors designed five nested feature tiers to answer this directly. T0 (39 features) is the full set used by prior work, retaining two direct post-diagnostic markers. T1 (37 features) removes both. T1-ns removes prior stroke as well, isolating the strongest remaining comorbidity proxy. T1-portable (34 features) restricts T1 to items asked in both 2022 and 2023. T2 (15 features) retains only self-reportable items with no clinician-mediated content.
0.89 Is Leakage, Not Learning
On T0, all ten models reached test AUROC between 0.8905 and 0.8933, reproducing the literature's headline figure regardless of model class. Removing the two post-diagnostic markers (T1) cost every model 0.049 to 0.051 AUROC, and the entire field collapsed into a band 0.0045 wide (0.8395 to 0.8440). Removing prior stroke (T1-ns) cost a further 0.0075 to 0.0086 with re-convergence into a 0.0053-wide band. Restricting to self-report-only predictors (T2) cost another approximately 0.021.
The uniformity of the decline across every model class is the finding. Logistic regression, random forest, XGBoost, LightGBM, CatBoost, a neural network, the explainable boosting machine, and both tabular foundation models (TabPFN v2 and TabICL) all surrendered the same ~0.05 AUROC when the two features were removed. This gives quantitative, dose-response form to the leakage taxonomies of prior work and generalizes the single-pipeline correction to ten models and five tiers.
The Glass-Box Model Matches Everything
The explainable boosting machine (EBM), a glass-box generalized additive model, reached 0.8430 AUROC at T1 against CatBoost 0.8439, XGBoost 0.8430, the MLP 0.8432, and full-context TabICL 0.8440. Under a pre-specified non-inferiority margin of 0.005 AUROC, the EBM was non-inferior to every comparator at every tier (Holm-adjusted p < 0.001), and two-sided equivalence held against all nine comparators at T0 and T1-portable. The sole exception favored the EBM: subsampled-context TabPFN v2 trailed it by 0.0042.
The accuracy parity concealed an extreme compute asymmetry. The EBM needed 0.1 seconds to score the 88,413-respondent test partition. TabICL required 3.5 seconds to ingest its context but 1,153.8 seconds of GPU inference for the identical workload. TabPFN v2 needed 642.5 seconds. Scaled to a national cohort, this separates sub-second scoring from tens of GPU minutes per pass. The foundation models matched the best boosted trees with zero tuning and arrived natively calibrated, but the cost asymmetry and the shrinkage of any foundation-model edge under survey weighting suggest that in-context scale currently buys parity rather than superiority for population screening on tabular surveys.
Calibration Divergence Where Discrimination Cannot
Class-weighted classical models were severely miscalibrated in absolute risk (expected calibration error 0.234 to 0.291) while their calibration slopes remained near ideal (0.91 to 1.20). This is an arithmetic consequence of balanced weighting, not a property of model class, confirmed by refitting without class weights (error 0.0031 for the EBM and 0.0019 for CatBoost). The two foundation models, which apply no class weighting, were natively calibrated (error 0.010). Isotonic recalibration restored every model to error of 0.005 or less without altering discrimination, after which all competitive models showed positive net benefit across the 1 to 30 percent threshold range.
This is a caution for any study reporting class-weighted probabilities as risks. The raw probability quality separated the classes where discrimination could not, and the miscalibration was introduced by the imbalance handling, not by the model itself.
Fairness at the Operating Point
At the pre-specified screening threshold (sensitivity of 0.85), the EBM detected 75.4 percent of women's prior infarctions against 89.0 percent of men's. That is a true-positive-rate gap of 0.128. The gap was essentially identical for XGBoost (0.127), showing it is a property of the data-plus-threshold system, not of any model's opacity. Under a common selection objective, per-group thresholds closed the gap to 0.013, reweighing to 0.009, and shape repair with intercept equalization to 0.010. Deleting the sex terms outright stalled at 0.043 while costing the most specificity, because proxy pathways carry most of the disparity.
The EBM's additive structure makes these proxy pathways visible. Sex enters the learned interaction structure through age, general health, and smoking status rather than in isolation. Shape repair is a reversible, machine-readable, governable edit of the model itself, with per-group effective thresholds disclosed rather than implicit. This converts the interactive editing paradigm into an auditable fairness protocol.
Closing the TPR gap closed the false-positive-rate gap (0.141 to 0.007) but widened the positive-predictive-value gap (0.038 to 0.070), making the calibration-error-rate impossibility concrete. Baseline age-band gaps were far larger (0.597) and closed only at heavy specificity cost.
Conformal Prediction Hides Group-Level Failure
Split conformal prediction met its marginal 90 percent guarantee overall but redistributed it unevenly: coverage was 0.86 for men and 0.82 for adults over 60, the group at highest cardiovascular risk. The pattern replicated across the EBM, XGBoost, and TabICL. Deferral tracked the same gradient, from 0.035 in adults 18 to 39 to 0.586 in adults over 60, whose singleton predictions were wrong 44 percent of the time.
Mondrian calibration restored every stratum to 0.90 coverage at two distinct prices: honest deferral in the oldest stratum (0.586 to 0.712), and empty prediction sets (9.5 percent) in the youngest, which marginal calibration never produces. A hybrid retaining marginal calibration for ages 18 to 39 and Mondrian elsewhere dominated both pure policies under every utility weighting tested.
Post-Hoc Explanations Are Not Glass-Box Explanations
Against the EBM's exact global importances, TreeSHAP on tuned XGBoost plateaued at Kendall tau of 0.884 even with 10,000 explanation samples. KernelSHAP applied to the EBM itself recovered its own knowable truth only gradually (tau of 0.724 at 25 samples, 0.870 at 500). LIME never converged at any budget (tau of approximately 0.626; top-10 Jaccard of 0.45, misidentifying roughly half of the ten strongest drivers). Where stakes justify explanation, they justify models whose explanations are identities rather than estimates.
Temporal Transport Within 0.002 AUROC
Applied without refitting or re-thresholding to all 430,755 respondents of the 2023 BRFSS, every T1-portable model lost between 0.0009 and 0.0016 AUROC relative to its internal estimate. Calibration slopes stayed within 0.91 to 1.19, and the frozen 2022 thresholds delivered 2023 sensitivities of 0.855 to 0.862 against the 0.85 target. This is a strong result for temporal stability: models and thresholds frozen at one year transported to the next without degradation.
Sensitivity analyses added context. SMOTE-NC oversampling, standard practice in this literature, reduced T1 AUROC from 0.843 to 0.809 while inflating training time approximately 150-fold. Complete-case analysis cost 0.0023 AUROC in addition to discarding 42.9 percent of respondents. Re-evaluating every frozen model under survey design weights raised discrimination almost uniformly by 0.020 to 0.022 AUROC for all ten models, leaving the ranking statistically unchanged.
What This Means for practitioners
If you build screening models on survey data, audit your features for post-diagnostic markers before reporting accuracy. The headline AUROC in this literature is a property of the feature set, not the learner. A transparent, editable, sub-second glass-box model meets tuned gradient boosting and full-context tabular foundation models head-on once leakage is removed. Its additive structure supports what deployment actually requires: explicit operating points, auditable fairness repair, group-conditional uncertainty with honest deferral, exact rather than estimated explanations, and thresholds that transport across survey years.
The foundation models evaluated here matched the best classical methods with zero tuning and arrived natively calibrated. But their inference cost is orders of magnitude higher, they cannot ingest per-instance survey weights in their zero-shot interface, and their edge under survey weighting was null or reversed. For population screening on tabular surveys, the practical recommendation is clear: use a glass-box model, audit your features for leakage, and report at an explicit threshold.