Pixel-space generative Transformers typically use one token per image patch, so capturing finer detail requires more tokens and makes global attention increasingly expensive. We observe that finer detail does not necessarily require more global communication: low-frequency structure benefits from global exchange, while high-frequency detail can be modeled locally. Based on this insight, we introduce SpecSlot, a pixel-space rectified-flow Transformer that decouples output detail from the length of the global attention sequence.
Expanding spatial anchors into DCT-coupled slots
SpecSlot expands each spatial anchor into multiple slots, each assigned a disjoint band of DCT coefficients; this separates anchor density from spectral subdivision. Only the lowest-frequency slot from each anchor participates in global attention, while all slots interact locally. This allows the model to increase spectral detail without enlarging the global sequence. A spatial head further adds a local correction to the spectral prediction, while the generative state, target, and loss remain in pixel space.
Rectified-flow Transformer design operating in pixel space
The architecture uses a rectified-flow Transformer backbone that processes tokens in pixel space rather than an abstract latent space. Tokenization occurs through DCT coefficient band assignment: each spatial location generates several slots, and each slot owns a specific frequency band. The lowest-frequency slot from every anchor is the sole participant in global attention operations. All other slots communicate only through local attention patterns, which reduces the quadratic cost associated with long sequences.
Spatial head provides local correction to spectral prediction
A dedicated spatial head predicts a local residual that refines the spectral output from the main attention pathway. This two-stage flow — global spectral prediction followed by local pixel-space correction — allows the model to capture high-frequency detail without forcing every slot into the expensive global attention matrix. The training objective stays in pixel space, meaning the target images and the loss function operate on raw pixel values, not latent representations.
ImageNet-256: comparable FID with faster convergence
On ImageNet-256, SpecSlot achieves a final FID score comparable to a FLOPs-matched JiT control while converging faster. The faster convergence stems from the reduced attention sequence length, which stabilizes the rectified-flow training dynamics early in training. The model can allocate more parameter budget to spectral refinement rather than expanding the token count.
ImageNet-512: FID improvement and FLOP efficiency
On ImageNet-512, SpecSlot improves FID by 2.84 over the FLOPs-matched control while using 3.9% fewer FLOPs. The same model comes within 0.69 FID of JiT-B/16 at 38.2% lower cost, demonstrating that the decoupling of anchor density from spectral subdivision yields consistent gains across resolutions. The reduction in global attention tokens directly translates to lower compute without sacrificing sample quality.
Text-to-image: DPG-Bench superiority at lower forward cost
On text-to-image generation, SpecSlot surpasses JiT-B/8 on DPG-Bench by a margin of 1.06 while operating at 40.6% lower forward cost. The combination of fewer global attention tokens and a local correction head produces higher-fidelity images from text prompts at reduced compute.
# Key design points
# - Each spatial anchor splits into multiple slots
# - Slots own disjoint DCT coefficient bands
# - Only the lowest-frequency slot attends globally
# - All slots interact locally
# - Spatial head adds pixel-space correction
# - Training targets and loss remain in pixel spaceThe approach shows that not every dimension of generated detail requires global communication. By separating the frequency bands that participate in attention from those that are handled locally, SpecSlot achieves stronger image quality at lower computational cost across multiple generation benchmarks.
Read the paper on arXiv