Diffusion and flow models generate a sample over iterative steps, each a full pass through a large network, and that pass dominates generation cost. Training-free caching accelerates them by reusing activations from recent evaluations instead of recomputing them. Dynamic caching methods decide when to refresh by accumulating a cheap probe of the network input into a staleness signal and spending a fresh evaluation once it crosses a threshold. They perform hard caching: the signal is a binary switch between refresh and reuse, and every step, fresh or cached, advances by the prescribed step size. We observe that the signal carries much more than this switch. How far it sits below the threshold, its headroom, predicts the error induced by reuse. We propose SoftCache, which performs soft caching: instead of a binary switch, the headroom sets how far each cached step goes. Where the headroom is large, the sampler jumps past several scheduled nodes, and as a refresh nears, its steps shrink back toward the original step. These adaptive jumps may land between the scheduled nodes, taking the sampler off its fixed grid. SoftCache requires no training and adds no per-backbone settings: on FLUX for images and on HunyuanVideo and Wan2.1 for video, at speedups from 3.5× to 6.9×, it is ahead of every baseline on all four fidelity metrics.

Background: from hard to soft caching

Diffusion and flow models such as FLUX, HunyuanVideo, and Wan2.1 set the state of the art in image and video synthesis, but inference is costly. Producing a single output requires dozens of full network evaluations, each a pass through the same large transformer. The latent state changes only slightly between steps, which means successive passes largely recompute what the previous one already produced. This redundancy is the resource that acceleration methods draw on.

Several families of methods have explored how to cut sampling cost. Distillation trains a smaller student model that needs fewer steps. Fewer-step solvers reach a given quality with fewer evaluations. Cheaper attention or token reduction makes each individual step less expensive. A different lever is caching: reusing the model's own output from earlier steps, thereby avoiding redundant computation altogether. Methods in this space store intermediate activations or residuals from the transformer stack and decide, at each scheduler node, whether to reuse the cached quantity or trigger a fresh evaluation.

Many dynamic caching methods accumulate a cheap probe of the network output across consecutive nodes and refresh once the accumulated signal crosses a fixed threshold. Liu et al. and Chung et al. fall in this category. The signal serves one purpose: a binary decision at every node — either the cached residual is still trustworthy and can be reused, or it has gone stale and a fresh evaluation is required. Once the decision is made, every step, whether fresh or cached, advances by exactly the step size the schedule prescribes. The distance between the signal and the threshold, which the authors call headroom, is discarded; a node where the cache has barely aged and one on the verge of refresh are treated identically.

SoftCache identifies that this ignored distance is informative. Below the threshold, the staleness accumulator closely tracks the reuse error, the gap between the cached computation and a fresh one at the current node. The distance from the threshold measures how safely the cache can be trusted. Rather than use it only as a binary switch, SoftCache reads the headroom as a step size: cached steps jump off the scheduler's grid in proportion to it, the tail is re-spaced so that every node passed is dropped from the trajectory, and the cached residual is extrapolated linearly to the landing at no extra network cost.

The method: headroom as step size

Let the staleness accumulator at node i be Ai, built by summing per-node probe changes ρj = ||Ij − Ij−1||1 / ||Ij−1||1 + ε from the anchor node a to i. A refresh threshold τ determines when a cached node is cleared and a fresh evaluation is performed: the node is cached while Ai stays below τ, and refreshed once Ai ≥ τ. Hard caching reads Ai against τ as a single comparison and, regardless of how far below τ the accumulator sits, takes the scheduled step Δi = ti+1 − ti.

SoftCache defines the headroom Hi = 1 − Ai/τ, which lies in [0, 1] at a cached node. When Hi is large, the cache is still reliable after many steps, so the sampler takes a long step that skips several scheduled nodes and can land between them. When Hi is small, a refresh is near, so the step shrinks back toward the scheduled one. The step size from headroom is

Δ̄i = Δi                                 if Ai ≥ τ (fresh)
     min{1 − ti, Δi + (J − 1)Hi Δi}     if Ai < τ (cached)

where Δi is the scheduled spacing at node i and J > 1 caps how many scheduled steps a single step may span. At a cached node the step exceeds the scheduled one in proportion to the headroom: approaching Δi as a refresh nears, at most J Δi, and never past the end of the schedule. After a jump, the probe change is measured from the node the step left, so a long step fills the accumulator more than a short one.

Two guards disable the jump: at the start of the trajectory and at a node where the probe change itself is large. In both cases the cached node takes the scheduled step Δi. These thresholds use quantities the sampler already holds with the same values across every configuration.

When a jump lands at a time the grid does not contain, the sampler is off its fixed schedule. The residual from the anchor cannot be held unchanged — the further a node sits from the anchor, the worse the reuse becomes. Instead, SoftCache extends the residual along the line through the two most recent fresh residuals, the residual motion of the cache:




where β controls how far along the line the extrapolation reaches. The ratio λi is built from time gaps, so it is defined both on-grid and off-grid. At every cached node, whether or not the node was reached by a jump, the residual is extrapolated this way. It costs one additional stored tensor, ra−1, and no extra network evaluation.



The jump also passes several scheduled nodes. The nodes after the landing are replaced. If mi nodes of the old grid remain ahead of the new position ti+1, they are spread evenly over [ti+1, 1], counting ti+1 as the first:



ti+1+k = ti+1 + (1 − ti+1) k / (mi − 1),   k = 1, …, mi − 1



This re-spacing is evenly divided in t since it requires no tuning. Sampling continues from (xi+1, ti+1) along this rebuilt grid, which ends at ti+mi = 1.



Experiments and results



The authors evaluate on three backbones. On FLUX.1-dev, 200 DrawBench prompts yield one 1024×1024 image each, with 50 Euler steps. Compared methods include TeaCache, SeaCache, DPCache, and TaylorSeer. SoftCache achieves speedups from 3.79× to 6.91× and is ahead of every baseline on all four fidelity metrics — PSNR, SSIM, LPIPS, and DreamSim — in every speed band. At the fastest band (6.91×), SoftCache leads SeaCache by 1.12 dB PSNR and 0.082 LPIPS. SoftCache is also faster than SeaCache and DPCache in every band; the only runs that outpace it are TeaCache at the slowest band (0.23× slower) and TaylorSeer at the middle band (0.15× slower).



On HunyuanVideo, 946 VBench prompts generate 65-frame 640×480 videos with 50 Euler steps. Compared against SeaCache and DPCache, the two strongest baselines on FLUX, SoftCache leads both on all four metrics in all three speed bands, by 0.54 to 2.78 dB PSNR over the better baseline. No baseline is more than 0.06× (about 1%) faster in any band. On VBench, which scores each sample independently, SoftCache matches DPCache within 0.003 on the Total across all bands and scores above SeaCache in all three.



On Wan2.1-T2V-1.3B, 100 VBench prompts yield 81-frame 832×480 videos. At the two evaluated points, SoftCache leads SeaCache and DPCache on all four metrics at both speed settings.



Component ablations confirm each SoftCache contribution. Removing the jump (setting J = 1) so the sampler stays on the scheduled grid trails SoftCache at 3×× and by 4.18 dB PSNR and 0.219 LPIPS at 7××. Removing headroom scaling so a cached node jumps the full J Δi regardless of headroom also degrades performance. Holding the tail re-spacing fixed so mi does not adjust after a jump costs 1.21 dB PSNR. Dropping residual motion (setting β = 0) harms LPIPS and DreamSim. The damping parameter β is set to 1 below 4×× and 0.5 above it, the same rule on all three backbones.



Limitations and trade-offs



The authors acknowledge that the headroom signal depends on the quality of the staleness probe; substituting a different probe, such as TeaCache's, yields lower fidelity. The jump guard that disables adaptive steps when the probe change is large limits SoftCache's adaptivity in settings with highly variable per-node outputs. The cap J on how many scheduled steps a single step may span is set per experiment; raising J beyond 2 yields diminishing returns while preserving fewer high-frequency details. The β parameter, which damps from 1 to 0.5 above 4×× speed, reflects a trade-off: higher β gives larger jumps but extrapolation becomes less reliable far from the last fresh evaluation, while lower β preserves texture at the cost of shorter steps. The method also requires two stored residuals, ra and ra−1, per cached node, a modest memory overhead.



What it means in practice



For a working developer, SoftCache offers a way to accelerate diffusion and flow model inference without training a new model, without calibrating per-backbone parameters, and without changing the network architecture. The method plugs into any dynamic caching pipeline that already computes a staleness probe, because SoftCache reads the same headroom signal and adds only a step-size computation, a residual extrapolation, and a tail re‑spacing step. The adaptive jumps mean the sampler can move between scheduled nodes, which reduces the number of evaluated nodes and improves fidelity at the same wall-clock budget. On image models like FLUX and video models like HunyuanVideo and Wan2.1, SoftCache attains the highest fidelity among caching methods at every operating point from 3.5× to 6.9× speedup, often with room to spare. A practitioner interested in integrating SoftCache would need to ensure the caching pipeline accumulates a per‑node staleness accumulator Ai and maintains the most recent two fresh residuals; from there, the headroom‑driven step size and the linear residual extrapolation are the only additions required.


Read the paper on arXiv