Most unified multimodal models split along a fundamental divide. Some treat both language and images as discrete tokens, while others pair discrete language prediction with continuous image generation. The token-based approach introduces a visual quantization loss, and the hybrid setup demands separate objectives and sampling steps for each modality. A fully continuous alternative sidesteps these compromises and permits a shared generative process, yet has seen limited exploration for multimodal pretraining. Multimodal Flow directly addresses this gap.
The core idea organizes text blocks and images as ordered continuous hyperchunks. Textual token order stays intact, and visual spatial structure remains readable by the model. A single chunk-causal flow backbone learns one vector field over these hyperchunks through flow matching. Cross-modal signals flow through a shared attention module, while each modality also passes through its own feed-forward networks. During training the system predicts several target chunks in parallel; at inference time it generates hyperchunks one after another.