What neuro-spectral architectures promise and their bottleneck
Physics-informed neural networks (PINNs) approximate solutions of partial differential equations by training neural representations against governing equations, initial and boundary conditions, and available data. For time-dependent problems, PINNs can struggle to capture spatial detail—a spectral bias that favors low frequencies—and respect the temporal causality imposed by the initial condition. Neuro-spectral architectures (NeuSA) address this by combining a spatial spectral representation with a system of Neural ODEs, allowing them to achieve state-of-the-art accuracy with substantially shorter training times. The network parameterizes the dynamics of spectral coefficients, and the solution is obtained by integrating these dynamics from the prescribed initial state. This formulation makes time integration a computational bottleneck: each training step requires evaluating a trajectory and differentiating a physics-informed loss through the numerical solver. The original NeuSA implementation uses explicit fourth-order Runge–Kutta (RK4) integration, whose time steps are evaluated sequentially. This dependence limits temporal parallelism even when spatial operations are performed on a GPU. Moreover, rapidly oscillating or strongly dissipative linear modes can restrict the stable step size of an explicit solver, increasing the number of steps required for each training update.
PaPi's Picard–Duhamel integration scheme
We introduce PaPi, a parallel integration and training scheme for neuro-spectral dynamics with a tractable linear component. Rather than propagating the solution sequentially through time, we construct successive approximations to the entire trajectory using a Picard–Duhamel formulation of the underlying ODE. Each iteration incorporates the linear dynamics exactly and updates the nonlinear correction through affine recurrences that can be evaluated using a parallel scan. This allows temporal propagation within each iteration to be carried out in parallel, substantially reducing the sequential depth of the ODE solve.
The system considers dynamics of the form dX/dt = A(t)X + εF(X), where A(t) describes the linear dynamics and εF the nonlinear contribution. The Duhamel principle yields the analytical solution X(t) = Φ₀ᵗX₀ + ε∫₀ᵗΦˢᵗF(X(s))ds, where Φˢᵗ is the linear propagator. Applying Picard iteration to this formulation gives the Picard–Duhamel scheme: X(k+1)(t) = Φ₀ᵗX₀ + ε∫₀ᵗΦˢᵗF(X(k)(s))ds, with X(0)(t) = Φ₀ᵗX₀. The initialization already captures the complete linear dynamics and is exact when ε = 0. Subsequent iterations therefore update only the nonlinear correction while retaining the time-parallel structure of classical Picard iteration.
In practice, the scheme is evaluated using the trapezoidal method, which can be computed in parallel over time. All nonlinear terms F(Xⱼ(k)) depend only on the previous trajectory and can therefore be evaluated simultaneously. Furthermore, the cumulative integration reduces to a parallel scan: letting Pⱼ = Φᵗⱼᵗⱼ₊₁, we have X̄ⱼ₊₁(k+1) = PⱼX̄ⱼ(k+1) + bⱼ with bⱼ = εh²(Pⱼgⱼ(k) + gⱼ₊₁(k)). Two such maps compose according to (Pᵇ,bᵇ)∘(Pᵃ,bᵃ) = (PᵇPᵃ, Pᵇbᵃ + bᵇ). Since composition of affine maps is associative, we can evaluate the iteration with a parallel scan, constructed through a tree of partial sums with O(log N) sequential depth and O(N) total work.
Convergence and explicit error bounds
We establish convergence of the continuous Duhamel–Picard iteration and provide explicit error bounds for the iteration and its temporal discretization. Theorem 1 states that if F is any Lipschitz-continuous function with constant L and ‖A(t)‖ₒₚ ≤ α ∀t, then the Duhamel-Picard scheme converges to X, the solution of the initial value problem. Moreover, there exists a constant C depending only on X₀, F(X₀) and L such that ‖X(k)(t) − X(t)‖ ≤ C Lᵏ(εt)ᵏ⁺¹/(k+1)! e^(α+εL)ᵗ. The error decays factorially with the number of iterations k, meaning that for mildly nonlinear problems, few sweeps suffice.
Theorem 2 decomposes the total error into the Duhamel–Picard iteration error and the quadrature error introduced at each time step. The quadrature error for the trapezoidal method carries a factor h³ (where h = T/N is the step size), and increases with the time index j at least at a linear rate, leading to the expected quadrature error of order h² of the trapezoidal method. These bounds clarify how the nonlinear correction magnitude ε and the stiffness of the linear operator α affect the number of iterations required.
An important practical consequence: any linear term of the PDE whose propagator is available in closed form can be moved from the network into the linear component. This shrinks the constant ε that controls the number of sweeps and also lowers the effective stiffness the network has to represent. In every problem considered, the linear operator A is diagonal or block-diagonal in the Fourier basis, so Φˢᵗ is a closed-form exponential and is cheap to evaluate.
Warm-start training and inference
Training minimizes NeuSA's PDE residual along the trajectory, with the RK4 rollout replaced by the Picard–Duhamel iteration. Consecutive optimizer steps change θ only slightly, so the trajectory produced at the previous step is already close to the fixed point of the current one. We therefore carry the trajectory across steps: instead of starting the iteration from the beginning at each step, the previous trajectory is taken as X(0) (with no gradient flowing into it). Two sweeps per step suffice on all but the most strongly nonlinear problem considered. Since the residual of the iterate is smallest at early times and grows with t like (εkt)ᵏ⁺¹, we weight the residual in time by w(t) = (1+t)⁻², which anchors the loss on the part of the trajectory that is already converged while the later part catches up over subsequent steps.
Inference deploys the trained model with a cold iteration from the linear flow, X(0) = Φ₀ᵗX₀, for a fixed number of sweeps; eight sweeps suffice on all but the stiffest problem, where 32 are used. Each sweep is one batched network evaluation and one scan of depth O(log N), so inference has sequential depth O(K log N) rather than O(N).
Experimental results across PDE families
We evaluate PaPi on six forward PDE benchmarks and a coefficient-identification problem, covering heterogeneous wave propagation, nonlinear transport, and dispersive dynamics. Experiments demonstrate substantial reductions in training time over the original sequential NeuSA formulation, with competitive or improved accuracy on most problems.
On the 2D Layers wave equation, PaPi matches NeuSA's accuracy (rMSE 0.068 vs. 0.069) while training in 12 seconds instead of 508 seconds—a speedup of over 40×. On the Marmousi model, the most heterogeneous medium, PaPi is more accurate (rMSE 0.15 vs. 0.27) and almost 10× faster. The speedup is smaller here because the learned term, which represents the whole heterogeneity, is not weak, requiring six sweeps per step, and the RK4 rollout is already compute-bound on the 201²×2 state, so the parallel scan removes depth but not arithmetic.
On 3D Layers, PaPi trains 2.3× faster than NeuSA but is less accurate (rMSE 0.018 vs. 0.0076), which the authors attribute to the coarse training grid (56 of 111 time nodes) and to a large learned term (ε = 10 in NeuSA's setting); the PINN baselines are worse by a further factor of 1.5–4× and take 77–2020 times longer.
Sine-Gordon is the problem on which the integrator matters most. With the mass term carried by the linear part, PaPi reaches an rMSE of 5.6×10⁻⁴ in 0.3 seconds of training against NeuSA's 9.0×10⁻⁴ in 234 seconds—over two orders of magnitude faster. The optimization loop takes less than a second because each optimizer step is two parallel sweeps over a 402-dimensional state on 201 nodes, and the 1,000 steps are compiled into one program.
On 2D Burgers, PaPi is trained for 1,000 steps, which still takes a quarter of the time (30 seconds against 119 seconds) and halves the error (rMSE 0.028 vs. 0.053). The best PINN baseline, QRes, reaches 0.073 in 1,178 seconds.
For the KdV equation, whose dispersive linear term makes the spectral ODE stiff, PaPi treats the dispersion exactly through Φˢᵗ = e^(ik³(t−s)), so the grid only has to resolve the smooth soliton and the learned correction. PaPi reaches an rMSE of 1.2×10⁻³ in 3.2 seconds. NeuSA, which must take 1,001 sequential RK4 steps per optimizer step, takes 1,322 seconds for 1,000 steps and an rMSE of 3.6×10⁻³; the PINN baselines lie between 6×10⁻³ and 1.5×10⁻² after 900–1,200 seconds.
Inverse problems and coefficient identification
Since the Picard–Duhamel rollout is differentiable with respect to the linear operator as well as to the network, PaPi applies without change to inverse problems in which coefficients of the PDE are unknown. As a test, we take the parameter-identification problem for Burgers' equation, given by ∂ₜu + λ₁u∂ₓu = λ₂∂ₓₓu on [−1,1]×[0,1] with periodic boundary conditions, where λ₁ and λ₂ are to be learned and the true values are (λ₁,λ₂) = (1, 0.1). The model sees two snapshots of the solution: the initial condition, which it satisfies by construction, and the final state u(1,x) on the 199-node spatial grid, either clean or corrupted with Gaussian noise of standard deviation equal to 1% of that of the data.
Without temporal causality, the MLP baselines struggle to reconstruct the solution and recover coefficients from two widely separated snapshots. QRes performs best among them, with errors of 20% for λ₁ and 4% for λ₂. At 500 steps, PaPi reaches 5% and 1% respectively, in under a second—over two orders of magnitude faster. At 5,000 steps with clean data, both coefficient errors reach 10⁻⁴ in about a minute; with 1% noise, they settle at 4×10⁻³ for λ₁ and 2×10⁻³ for λ₂.
Evolving the identified equation with the reference solver beyond the observed interval gives relative solution errors of 1.0×10⁻² after 500 training steps and 5×10⁻⁵ (clean) or 2×10⁻³ (noisy) after 5,000 steps. As with NeuSA, this extrapolation comes from the architecture's causal structure.
Evaluation controls and generalization
The authors translate their findings into concrete controls for neuro-spectral evaluation. First, separate synthetic from behavioral bugs is not applicable here—this paper's concern is computational bottlenecks in PDE solving, not repair of faulty code. Second, the oracle tier, contract type, reset heuristic, and initialization assumption must be disclosed, as changing any of these can flip the measured outcome. Third, a specification-only baseline appears as a diagnostic for possible regeneration, though its presence alone does not resolve whether success arose from repair or regeneration. Fourth, oracle coverage and undecided cases should be reported, as not every design has a matched formal contract; some designs fall outside exhaustive combinational checking, reset-sequence verification, or preamble-based equivalence.
Together, these controls make the interpretation of a training-time result reproducible alongside the result itself. The computational benefits of PaPi nevertheless depend on the number of iterations required to resolve the nonlinear dynamics. Improving the efficiency of these iterations, particularly for strongly nonlinear problems, is a natural direction for future work. The released construction code, benchmark, and reproduction materials aim to lower the barrier for follow-up audits.