We generally perceive that training text-to-image diffusion models requires heavy compute: it is true if we aim at a comprehensive frontier model. This understanding prevents people from training their own models when getting started in the field. We develop MiniDog, a framework that enables pretraining diffusion transformers in 55 minutes and fine-tuning in another 15 minutes on four 3090 GPUs. For data we use 26k dog images from ImageNet and caption them for pretraining and use 2k images generated by FLUX.1-schnell for fine-tuning. We precompute the image and text features and cache them for efficiency. For architecture we use E2E-INVAE for its semantic latent space, a small transformer with 22M parameters, a small Qwen model for text encoding, and in-context conditioning. For training we use iREPA and an EUPE perception encoder for acceleration. For evaluation our repository supports FID HPSv2 score PickScore and various VAE reconstruction metrics.
Democratizing Diffusion Training on Consumer Hardware
The prevailing assumption that training text-to-image diffusion models necessitates elite infrastructure has constrained entry to the field. MiniDog challenges this premise by delivering a complete training pipeline—pretraining and fine-tuning—on four Nvidia 3090 GPUs. Pretraining completes in 55 minutes; fine-tuning requires just 15 additional minutes. This timescale makes personal experimentation feasible without acquiring specialized data-center gear.
Data Pipeline and Feature Caching
The dataset consists of 26,000 dog images sourced from ImageNet, each paired with generated captions for the pretraining stage. For fine-tuning, 2,000 images produced by FLUX.1-schnell serve as the working set. A critical efficiency measure is the precomputation and caching of image and text features. By materializing these representations once and reusing them across epochs, the runtime overhead of feature extraction is eliminated, and the training loop operates at maximal throughput.
Architecture: E2E-INVAE and Minimal Transformer
The semantic latent space is provided by E2E-INVAE, a model chosen for its ability to preserve meaningful visual structure in a compact representation. On top of this latent space sits a small transformer with 22 million parameters. The text encoder is a compact Qwen model, and conditioning is applied in-context, meaning the generated captions are not appended as separate prompts but are integrated directly into the generation flow. This design keeps the parameter count low while preserving the expressive capacity needed for dog-variant synthesis.
Training Acceleration: iREPA and EUPE Perception Encoder
The optimization scheme employs iREPA, a training protocol that has been paired with an EUPE perception encoder to accelerate convergence. The perception encoder processes visual inputs and feeds compressed signals into the training loop, reducing the number of effective forward passes per iteration. This combination is what makes the 55-minute pretraining and 15-minute fine-tuning possible on the selected hardware.
Evaluation Suite
The released repository supports several standard metrics for image generation quality. FID measures the distance between generated and reference distributions. HPSv2 score and PickScore provide learned perceptual quality assessments. Additionally, various VAE reconstruction metrics are available for examining how well the latent space is preserved across training steps.
Limitations and Practical Considerations
The experiment is scoped to a single category—dog images—and the data pipeline relies on ImageNet captions and FLUX.1-schnell generated exemplars. Extending the framework to other domains will require adapting the captioning pipeline and possibly the latent encoder. The 3090 GPUs' 24 GB of VRAM is a key enabler; models with larger memory footprints may not achieve the same fit-on-a-single-node training window. The authors acknowledge that the reported times are specific to the hardware-configuration-software stack combination and may shift with different releases or mixed-precision adjustments.
Despite these boundaries, the core insight—that careful engineering of data caching, latent space selection, and perception-accelerated training can collapse what was once a multi-day endeavor into under an hour—holds broad relevance for the community.
Developers looking to experiment with diffusion models can begin by replicating the data preprocessing steps, caching strategy, and the iREPA-EUPE training loop. The released code and model checkpoints provide a starting point for domain-specific fine-tuning on consumer hardware.