Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report

Why T-Staging Laryngopharyngeal Cancer Needs a New Approach

Laryngopharyngeal cancer affects the voice box and throat, and accurate T-staging determines whether a patient receives organ-preserving surgery or more aggressive total laryngectomy. Biopsy remains the reference standard but is invasive and time-consuming. CT-based staging offers a non-invasive alternative, yet manually tracking tumor extension across dozens of axial slices introduces subjectivity, especially when invasion borders are unclear. Computer-aided approaches have largely ignored the structured anatomy of the laryngopharyngeal region and the detailed organ-specific commentary found in routine radiology reports. This paper addresses the gap by proposing an anatomy-aware multimodal framework that integrates organ-level CT context with report text into a unified representation for T-staging.

Where Prior Work Falls Short

Prior work on cancer T-staging has used radiomics, convolutional neural networks, and more recently multimodal fusion of image and text. Many studies focus on single-modality CT analysis and report limited dataset sizes, often from a single center. Others apply global feature fusion that merges entire CT volumes with entire reports, which prevents precise alignment between specific anatomical structures and their corresponding textual descriptions. For laryngopharyngeal cancer, where tumor spread follows anatomical pathways between primary sites and surrounding organs, such coarse fusion loses critical structural context. Existing methods either require labor-intensive tumor segmentation or process whole volumes without explicit anatomical constraints, making it difficult to model inter-organ invasion patterns. This paper identifies these limitations and proposes a framework that explicitly models anatomy-structured organ relationships and performs fine-grained cross-modal alignment at the organ level.

Building the Anatomy-Structured Organ Graph

The first module, Anatomy-Structured Organ Graph (AOG), constructs a graph from pre-trained organ segmentations. Rather than relying on tumor masks or whole-volume processing, AOG uses a publicly available segmentation model to obtain organ-specific feature vectors. These features are projected through a shared layer and connected according to two edge sets: edges between the three primary site nodes (supraglottic, glottic, subglottic) and edges from each primary site to the four potentially invaded organs (esophagus, thyroid gland, trachea, oral cavity). The graph topology reflects clinically known spread patterns, and graph attention networks propagate messages across these edges, allowing the model to dynamically weight relevant invasion relationships. This approach provides an anatomy-aware foundation without requiring manual tumor delineation.

How Organ-Anchored Cross-Modal Alignment Works

The second module, Organ-Anchored Cross-Modal Alignment (OCA), enhances each organ node with textual evidence from the paired radiology report. Organ features from CT serve as queries that attend to tokens in the report through a cross-attention mechanism. Projection matrices map both visual and textual features into a shared space, and attention coefficients weight the contribution of each token to the organ representation. The enhanced organ feature is the sum of the original visual feature and the weighted text aggregation. This design ensures that each organ representation incorporates relevant clinical descriptors, such as mentions of swelling, obstruction, or invasion, while preserving the anatomy-guided graph topology established by AOG.

Injecting Report Evidence into Graph Structure

The third module, Report-Enhanced Graph-Refinement (REG), incorporates explicit invasion statements from the report into the graph structure. A large language model parses each radiology report using a prompt that identifies which of the four target organs are described as invaded. A dedicated text node is created from the report's [CLS] token and connected to all primary site nodes by default, plus to any target organs that the LLM flags as invaded. The original organ-organ edges from AOG remain unchanged, so anatomy-based risk connections are retained while being selectively strengthened along pathways supported by the report. The resulting multimodal organ graph is processed by a graph neural network, and global average pooling over all node features produces the final T-stage prediction. Training uses cross-entropy loss against pathological T-stage labels derived from AJCC 8th edition criteria.

Experimental Setup and Dataset Scale

Experiments are conducted on a multi-center cohort of 656 patients across four hospitals. The primary training cohort comprises 439 patients from Fudan University Zhongshan Hospital; external test sets include 25 patients from Chang Gung Memorial Hospital, 87 from Eye and ENT Hospital of Fudan University, and 105 from TCGA. All cases are pathologically confirmed. Two classification settings are evaluated: binary (T1-T2 vs. T3-T4) and three-class (T1, T2, T3-T4). The framework achieves an F1 macro score of 80.83 and accuracy of 81.53 in the binary setting on the internal fold, and 61.43 F1 and 63.33 accuracy in the three-class setting. On external test data, the model maintains consistent improvements over imaging-only baselines such as SwinViT and VDPF, as well as multimodal baselines including CPAM, TGCAM, and MMI. Ablation results show that each component (AOG, OCA, and REG) contributes to the performance gain, and the full model with all three modules consistently outperforms every compared approach.

Limitations the Authors Acknowledge

The authors acknowledge several limitations. The LLM-based invasion extraction relies on prompt design and may miss subtle or equivocal mentions in reports. The framework depends on organ segmentation quality from a pre-trained model, which may vary across scanners or patient populations. The dataset, while large for this task, still represents a subset of real-world clinical diversity, and performance on completely unseen populations may differ. The graph structure encodes anatomical prior knowledge that may not generalize to cancer types with different spread patterns.

What This Means for Developers

In practice, the framework can be integrated into radiology workflows as a post-processing step. Given a CT volume and its paired report, the system generates an organ-level graph, enriches it with textual evidence, and outputs a T-stage probability. Developers can replace the organ segmentation model, the language model, or the graph neural network architecture without redesigning the entire pipeline, making the approach adaptable to other anatomical regions or cancer types. The code and pretrained models are released to facilitate reproducibility and further research.

Closing Perspective

This work demonstrates that explicit anatomy-structured modeling and fine-grained cross-modal alignment improve T-staging accuracy when CT and radiology reports are combined. By moving away from global feature fusion and toward organ-level graph reasoning, the framework provides a practical pathway toward non-invasive, computationally aided cancer staging.