The Platonic Representation Hypothesis (PRH) suggests that as models of different modalities scale, their representations converge toward a shared statistical model of reality. Images and text, for instance, both probe the same underlying world, and as vision and language models improve, their nearest-neighbor agreements grow stronger. This paper asks whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations align with the geometry of physical motion? The answer is yes, and the authors propose Sensorimotor Alignment (S²) as a training-free offline metric to measure it.
Sensorimotor Alignment (S²): Constructing and Comparing Neighborhoods
To make sensorimotor alignment measurable, the authors define S² by constructing two representations for each episode in a robot dataset. Sensory representations come from a model's image and text features; motor representations come from the recorded end‑effector trajectory—position, orientation, and gripper state over a future horizon. Alignment is measured via mutual nearest‑neighbor overlap: if episodes with similar sensory embeddings also have similar trajectories, the overlap is large, indicating strong sensorimotor alignment.
The metric uses cosine distance in sensory space and Dynamic Time Warping (DTW) in motor space, allowing comparison of trajectories at different speeds. For each episode, the k‑nearest‑neighbor sets in sensory and motor spaces are computed, their intersection size divided by k, and the result averaged over all episodes. A centered, normalized variant of this mutual‑k‑NN estimator is used in the reported results, with k = 10 unless otherwise noted.
The authors construct sensory embeddings by encoding an image frame and its task instruction with the vision and text encoders of a robot foundation model, then average‑pooling the output token features into a single d‑dimensional vector. Motor representations extract end‑effector positions, orientations, and gripper states over the next h steps, with positions encoded as vectors in ℝ³ and orientations as rotations in SO(3) so that distances reflect translation and rotation geometry. Reference datasets are DROID (Franka Panda) and BridgeDataV2 (WidowX), which pair sensory inputs with physical motions.
S² Correlates with Task Success Across Embodied Model Families
The central empirical finding is that S² positively correlates with downstream robot task success across multiple model families. In simulation and on real robots, models with higher sensorimotor alignment tend to achieve higher success rates. The correlation strength varies by family: fine‑tuned vision‑language‑action models (VLAs) show a Pearson r of +0.717 with SimplerEnv task success, raw vision‑language models (VLMs) yield r = +0.681 for the success of the VLAs later fine‑tuned from them, and world action models (WAMs) achieve r = +0.718 on RoboTwin 2.0. For real‑robot policies, S² computed offline correlates with physical‑robot success at r = +0.831 across 5 policies on 40 tasks.
These correlations support S² as a low‑cost offline proxy for prioritizing models before expensive closed‑loop evaluation. The relationship holds across reference datasets, model families, scales, and VLA training setups, as confirmed by robustness checks involving different horizons and embodiment configurations.
S² as an Offline Proxy for VLA Backbone Prioritization
Beyond correlating with task success within a model family, S² can identify promising vision‑language model backbones before any robot‑specific fine‑tuning. The authors compute S² on 8 raw VLM backbones from the VLM4VLA suite and find that these scores correlate with the later VLA fine‑tuning success at r = +0.681 (one‑tailed p = 0.031), whereas aggregate VQA benchmark scores show virtually no correlation (r = −0.154). This means researchers can evaluate a VLM's sensorimotor alignment early in the pipeline and use that signal to select which backbones deserve costly VLA fine‑tuning.
The same principle extends to WAMs: S² correlates more strongly with RoboTwin success than standard video‑quality metrics such as FID, SSIM, and PSNR. While FID even runs backward (lower video quality associates with higher success), S² at r = +0.999 correctly recovers the success‑rate ordering among ALOHA‑trained WAMs. The result suggests that sensorimotor alignment captures a dimension of model quality that pixel‑level or generation‑quality metrics miss entirely.
What Drives VLA Success: Vision Features, Not Text
Several phenomena observed in VLAs through closed‑loop results receive a representation‑level explanation via S². One such phenomenon is modality dominance: prior work notes that VLAs often struggle with instruction grounding, acting on visual cues rather than language instructions. The authors show that after fine‑tuning, VLA task success correlates with the S² of vision features (r = +0.694) but not text features (r = +0.237). Joint vision‑text features still correlate strongly (r = +0.699), indicating that the visual stream carries the primary signal for motion production.
Raw VLMs exhibit the inverse pattern: before any robot fine‑tuning, text features correlate at r = +0.501, more strongly than vision features at r = +0.425, and joint features peak at r = +0.681. The shift after fine‑tuning implies that VLA training repositions the model to rely more on what it sees and less on the language instruction—a trade‑off that S² makes visible at the representation level.
Freezing Vision Encoders Suppresses S² and Task Gains
Prior work reports that freezing the vision encoder during VLA fine‑tuning degrades downstream task performance. S² explains why: freezing prevents the sensorimotor‑alignment gain that end‑to‑end fine‑tuning provides. In experiments with Qwen2.5‑VL‑3B VLAs fine‑tuned on BridgeDataV2, freezing the vision encoder reduces S² by 41% and the SimplerEnv success rate by 40%. The alignment between sensory and motor representations is thus a mediating factor through which vision‑encoder updates improve performance.
Early Denoising Features Already Align with Robot Trajectories
World action models generate future video through iterative denoising and decode actions from the resulting latents. Fast‑WAM shows that WAMs can skip this video generation and decode actions directly, with little loss in task success. S² reveals the representation-level reason: sensorimotor alignment is already high at the earliest denoising step and remains nearly flat across the schedule, varying by at most 6% across three WAMs. Early‑step features are already sufficiently aligned with robot trajectories to generate actions, which explains why iterative denoising can be skipped without significant task‑success penalties.
Mid‑Depth Blocks Peak in Sensorimotor Alignment
Prior analysis of Chain‑of‑Steps identifies mid‑depth layers in video diffusion transformers as especially important for semantic reasoning over objects and interactions. The authors measure S² at different depths of three WAMs and find that Block 14 yields the highest alignment in every model. This peak falls in the mid‑depth range that Chain‑of‑Steps identifies as semantically important, suggesting that the layers handling video semantic reasoning also align most closely with robot motion.
Related Work: Representation Hypotheses and Robotics Evaluation
Representation learning in robotics has long examined which pretrained visual features improve task performance, but findings have been uneven. Some studies find stable correlations between vision encoder properties and downstream success; others report that stability depends on the policy‑learning method. S² differs by comparing nearest‑neighbor sets in both sensory and motor spaces, grounding evaluation in robot behavior rather than passive perception quality. The same S² measure correlates with success across VLAs, raw VLMs, and WAMs, and it also encompasses the frozen‑encoder evaluations that prior work has examined in isolation.
The Platonic Representation Hypothesis has been applied to image generation (REPA, CREPA), multimodal understanding (VIRAL), and cross‑modal alignment (Schnaus et al.), but never to robotics. This paper extends PRH by treating robot trajectories as another projection of the world and finding that more capable embodied models align their sensory representations more closely with it. The extension is empirical: S² makes sensorimotor alignment measurable, and the observed correlations provide support for the PRH‑inspired view of embodiment.
Limitations and Future Directions
The positive relationship between S² and task success is correlational across the tested models; the authors have not yet demonstrated that sensorimotor alignment can be used directly during training to improve task success rates. Models with similar S² values can still achieve different success rates, so S² is best used as a low‑cost offline proxy for identifying promising models and VLM backbones before costly fine‑tuning or closed‑loop testing. Evaluation covers limited model families, benchmarks, and training recipes, with few models per comparison and a single real‑robot setting; broader generalization remains to be tested.
Future work might explore whether optimizing S² directly during model training improves downstream performance, whether S² can guide data collection for robot datasets, and whether the metric generalizes to other embodiments such as aerial or legged platforms. The representation‑level diagnostic power of S² also opens avenues for understanding how design choices—freezing strategies, layer selection, modality weighting—shape embodied model behavior.