Pixel-space diffusion models have re-emerged as an alternative to latent diffusion, eliminating the need for a separately trained autoencoder and its associated bottleneck. A prior result from JiT demonstrated that a vanilla diffusion transformer can generate directly in pixel space by substituting the commonly used velocity prediction, or u-prediction, with clean image prediction, or x0-prediction. This work revisits that design choice and introduces MFlow, an adaptive parameterization that learns to interpolate between x0- and u-prediction across principal directions of the data, with the interpolation weights learned end-to-end during diffusion training. The authors observe that the preferred parameterization shifts systematically with both the data representation and the model's capacity; notably, larger models tilt toward velocity prediction. MFlow is evaluated on ImageNet, UCF-101 video sequences, and large-scale text-to-image generation, where it achieves competitive or improved generative performance relative to fixed parameterizations.
Beyond parameterization, the authors identify a persistent high-frequency deficit in vanilla pixel diffusion transformers, regardless of whether they use x0- or u-prediction. Generated images consistently lack the high-frequency statistics present in natural photographs. To address this without retraining the entire diffusion backbone, they propose ConvRefiner, a lightweight convolutional module trained post-hoc with an adversarial loss. ConvRefiner operates on the frozen diffusion output and, with a modest amount of additional training, restores high-frequency details while preserving the fid score of the original generator.
MFlow’s adaptive interpolation is straightforward in principle: for each principal direction identified in the data, the model learns a gating signal that blends x0- and u-prediction targets. The gating function is parameterized as a small multi-layer perceptron applied to direction-dependent embeddings, allowing the parameterization to adapt as the diffusion process progresses. Training objectives remain the standard diffusion loss, but the intermediate prediction transitions smoothly between the two extremes rather than committing to one. Empirically, the authors find that small models prefer x0-prediction, while large models gradually shift toward u-prediction, suggesting that capacity influences which parameterization is more expressive for a given data distribution.
The experimental setup spans three distinct generative tracks. On ImageNet 256×256, MFlow matches or improves upon prior pixel-base line FID scores across model scales. On UCF-101, which involves temporally coherent video frames, MFlow yields cleaner object boundaries and more consistent motion estimation compared to fixed-parameter counterparts. For text-to-image generation at 512×512 scale, MFlow achieves competitive CLIP-score while reducing the high-frequency gap observed in baseline pixel diffusion models. Ablation studies confirm that the interpolation mechanism itself contributes to the gains, and that simply averaging x0- and u-prediction without learned gating provides diminishing returns.
On the high-frequency deficit, quantitative analysis of radial frequency profiles shows that baseline pixel diffusion transformers fall short of natural image spectra across all scales. ConvRefiner’s adversarial loss is applied to a discriminator that evaluates high-frequency content via a Laplacian pyramid, encouraging the refiner to inject details without altering the coarse structure. Because the diffusion backbone remains frozen, ConvRefiner can be swapped or updated independently, making it practical for deployment pipelines where the base generator is fixed. The authors report that, after ConvRefiner training, the frequency profiles of generated images align more closely with those of real images, while fid remains within the baseline range.
These contributions suggest two directions for practitioners working with pixel-space diffusion. First, when scaling models, it may be beneficial to transition from x0- toward u-prediction, and MFlow provides a smooth path that does not require re-engineering the training loop. Second, for applications where image fidelity at fine scales matters—such as photographic synthesis or detail-critical design work—post-hoc refinement with a convolutional module can recover lost high-frequency statistics with minimal overhead. The code and model checkpoints are released alongside the paper, enabling immediate experimentation.
In summary, this work reexamines a foundational choice in pixel-space diffusion transformers, proposes a data-dependent parameterization that adapts to model scale, and identifies a complementary refinement strategy for high-frequency fidelity. The findings offer both a theoretical refinement of how diffusion objectives can be parameterized and a practical tool for improving generated image quality across diverse generative tasks.