Reduced-Space Multi-Fidelity Bayesian Optimization of Process Simulation Models

Cutting Simulator Calls by 80% Without Losing the Optimum

Industrial process optimization lives and dies by simulation. A chemical engineer designing a biopharmaceutical production line or a green fuel synthesis plant needs to explore a design space that might have 14 to 18 variables, each with continuous and discrete components. The simulators that evaluate these designs, Aspen HYSYS for continuous chemical processes and SuperPro Designer for batch biopharmaceutical operations, are rigorous but slow. Each evaluation takes 5 to 45 seconds, and the design space is too large for exhaustive search. Worse, the governing equations are black boxes: no derivatives are available, so gradient-based methods are out. This leaves practitioners with derivative-free optimization that must be sample-efficient, and the curse of dimensionality makes even sample-efficient methods struggle.

Bayesian optimization is the natural tool for this setting, but it faces three problems in practice. High dimensionality makes Gaussian process models poorly conditioned and acquisition function optimization expensive. Simulator evaluations are costly and sometimes non-convergent, requiring random restarts. And the design space has operational constraints that restrict feasible regions. A new paper from Imperial College London addresses all three problems simultaneously through reduced-space multi-fidelity Bayesian optimization, and validates it on two completely different industrial simulators across twelve diverse objectives.

The Three-Part Solution

The framework has three components that work together. First, variance-based global sensitivity analysis identifies which input variables actually matter, reducing the design space from 14 or 18 dimensions down to the 6 to 8 variables that dominate the objective variance. Second, an input-augmented multi-fidelity Gaussian process models both a cheap low-fidelity surrogate and the expensive high-fidelity simulator as correlated functions in a single GP, using a continuous fidelity parameter. Third, a cost-aware acquisition function with cooldown and promotion mechanisms adaptively decides whether to query the cheap surrogate or the expensive simulator at each iteration.

The key insight is that these components reinforce each other. Dimensionality reduction improves GP conditioning, which improves acquisition function optimization, which reduces the number of expensive evaluations needed. The multi-fidelity GP leverages cheap surrogate evaluations to reduce uncertainty about the high-fidelity response surface. And the cost-aware acquisition ensures the budget is spent where it provides the most information per dollar.

Dimensionality Reduction Through Sensitivity Analysis

The variance-based GSA computes total-order Sobol indices for each input variable. The total-order index quantifies a variable's overall contribution to output variance, including its main effect and all interactions with other variables. Variables with negligible total-order indices are fixed at nominal values, and the optimization proceeds in the reduced space of influential variables only.

This is not the same as random embedding or latent-space projection. Those methods map the high-dimensional problem into an abstract subspace that lacks physical meaning. Process engineers need to understand which specific physical variables, temperatures, flow rates, equipment sizes, drive performance. GSA preserves this interpretability because it operates in the native coordinate system. The reduced space is a subset of the original variables, not a transformation.

Operating in the reduced space also improves Gaussian process model fitting. In the full 18-dimensional pDNA problem, kernel hyperparameter optimization is often ill-conditioned, leading to over-smoothing and large posterior variance. With only the most influential variables retained, the reduced-space formulation produces more stable marginal likelihood fits and better-calibrated uncertainty estimates. The acquisition function optimization is also faster, cutting its wall-time by approximately 40 to 60 percent compared to full-space operation.

Multi-Fidelity Gaussian Process with Cross-Fidelity Kernel

The GP models both fidelities jointly over an augmented input space where a continuous fidelity parameter s ranges from 0 (low fidelity) to 1 (high fidelity). The covariance kernel has two components: an unbiased kernel shared between fidelities and a bias kernel that applies only at low fidelity. For the same input x, the low-fidelity self-covariance is the sum of both kernels, the high-fidelity self-covariance is just the unbiased kernel, and the cross-fidelity covariance is the unbiased kernel.

This structure captures the reality that low-fidelity surrogates are biased but correlated approximations of the high-fidelity simulator. The shared covariance component couples the two fidelities so that observations at either fidelity contribute to the posterior. Low-fidelity evaluations reduce uncertainty in high-fidelity predictions when the two sources are sufficiently correlated, while retaining a distinct high-fidelity response surface.

The input is a neural network surrogate trained on a small set of high-fidelity evaluations, so its cost is negligible compared to the simulator. The GP models the relationship between this surrogate and the simulator, learning how the surrogate's predictions relate to the true simulator output across the design space.

Cost-Aware Acquisition with Cooldown and Promotion

The acquisition function is a standard upper confidence bound applied separately at each fidelity, with the scores adjusted by cost. The fidelity with the larger cost-adjusted score is selected. A cooldown mechanism forces a high-fidelity evaluation every H steps to prevent over-reliance on the surrogate. A promotion mechanism identifies the top-K low-fidelity points with the highest predicted improvement and promotes them for high-fidelity reevaluation every P steps.

Promotion candidates are checked for redundancy against existing high-fidelity evaluations. Any point within a small distance of an existing high-fidelity point is removed to avoid redundant simulator calls and numerical instability. The surviving points are evaluated at high fidelity and their results are incorporated into the dataset for subsequent model updates.

The cost ratio between fidelities is substantial. High-fidelity evaluations are assigned a normalized cost of 10, while low-fidelity surrogate evaluations cost approximately 0.1. This 100:1 ratio means the framework can afford hundreds of surrogate evaluations for every simulator call. The cooldown period is set to 10, the promotion interval to 10, and one candidate is promoted per interval.

Two Industrial Case Studies, Twelve Objectives

The framework is validated on two completely different process simulators with different physics, different objective landscapes, and different optimization challenges.

The first case study optimizes a plasmid DNA production process in SuperPro Designer. The system involves strictly sequential unit operations: fermentation, lysis, and chromatography, with scheduling constraints including non-decreasing fermentation times. The design space has 18 mixed continuous and discrete variables covering equipment sizing, batch scheduling parameters, and flow rates. Six objectives are optimized: production cost, operating expenditure, capital expenditure, batch size, batch time, and cycle time.

The second case study optimizes a green fuel synthesis plant producing dimethyl ether from captured CO2 and green hydrogen in Aspen HYSYS. This is a steady-state continuous process with complex thermodynamic interactions and rigorous recycle loops that frequently cause simulator non-convergence. The design space has 14 mixed continuous and discrete variables. Six objectives are optimized: DME production rate, energy efficiency, carbon efficiency, operating expenditure, capital expenditure, and production cost.

Results: Competitive Performance, Fraction of the Cost

Across all twelve objectives, RS-MFBO achieves competitive or improved final objective values while using approximately 65 to 80 percent fewer high-fidelity evaluations than pure Bayesian optimization under the same total cost budget.

In the pDNA case study, RS-MFBO converges to the optimum for production cost within the first 10 to 15 evaluations, matching the final median performance of high-fidelity baselines. For capital expenditure, it achieves a visibly lower final median value than both vanilla BO and reduced-space BO, suggesting that multi-fidelity exploration helped escape local optima that trapped single-fidelity methods. The only case where reduced-space methods slightly underperform full-space BO is operating expenditure, which is expected: fixing non-dominant variables to nominal values restricts the optimizer to a manifold that may exclude the true global optimum if those variables have small but non-zero effects.

In the DME case study, the advantages are more pronounced. For energy efficiency maximization, RS-MFBO exploits the high-quality surrogate to initialize the search near 80 percent efficiency, avoiding the initial exploration phase that delays standard BO and RS-BO, which start between 70 and 76 percent. RS-MFBO maintains a performance lead throughout the budget, reaching peak efficiencies of approximately 84 percent. For DME production rate, RS-MFBO identifies the optimal operating region almost immediately, while standard BO requires over 40 function evaluations to match the performance RS-MFBO achieves in fewer than 10.

The acquisition dynamics show a characteristic staircase profile. Flat regions correspond to extended phases of low-fidelity exploration where the acquisition function exploits the cheap surrogate. Vertical jumps occur when the cooldown mechanism forces a correction or the promotion heuristic identifies a promising candidate. This behavior arises naturally from the cost-aware policy, which prioritizes the surrogate unless the cost-adjusted score or cooldown interval indicates high-fidelity refinement is needed.

Surrogate-only methods, RS-ANN-BO and RS-ANN-MILP, are the fastest in absolute terms because they avoid simulator calls except for final validation. But their performance is bounded by surrogate accuracy. RS-ANN-MILP often identifies designs that appear optimal in the surrogate but degrade significantly when validated against the true simulator, highlighting the necessity of the feedback loop RS-MFBO provides.

Limitations and Practical Considerations

The framework currently handles only two fidelity levels. Extending to a richer fidelity hierarchy, for example medium-fidelity physics-based models between the neural network surrogate and the full simulator, is identified as future work. The fixed cooldown and promotion schedules are simple heuristics; learned fidelity selection policies could provide better allocation, though they add complexity.

The GSA screening is performed once before optimization begins, using a threshold on the total-order Sobol index. If the objective's sensitivity structure changes significantly across the design space, which can happen with highly nonlinear interactions, the initial screening might miss variables that matter in certain regions. Adaptive sensitivity analysis could address this, but at additional computational cost.

The framework assumes the low-fidelity surrogate is a neural network trained on a small set of high-fidelity evaluations. If the surrogate is poor, the multi-fidelity GP will learn to ignore it, effectively falling back to single-fidelity behavior. The framework doesn't degrade catastrophically, but it also doesn't gain much from the second fidelity. The quality of the surrogate matters.

The implementation uses BoTorch for the multi-fidelity GP and SALib for the GSA, both well-established libraries. A VBA-Python COM interface enables automated communication with both SuperPro Designer and Aspen HYSYS. The code is available on GitHub, making it straightforward to adapt to other process simulators that expose a scripting interface.

What This Means for Process Engineers

The practical value of RS-MFBO is straightforward: it reduces the number of expensive simulator evaluations by a factor of three to five while finding comparable or better optima. For a process engineer running Aspen HYSYS on a 14-variable design space, that means the difference between hundreds of simulator calls and dozens. For a biopharmaceutical team using SuperPro Designer on an 18-variable space, it means the difference between a week of optimization runs and a day.

The simulator-agnostic nature of the framework matters. It doesn't assume anything about the simulator's internals, only that it accepts design variables and returns an objective value. This makes it applicable to any black-box process simulator, from Aspen to SuperPro to custom in-house tools, as long as a low-fidelity surrogate can be trained on a small set of evaluations.

The interpretability of the reduced space is also important in practice. When the optimizer suggests a design, the engineer needs to understand which variables it adjusted and why. GSA-based reduction preserves this understanding, unlike latent-space methods that operate in abstract dimensions. The engineer sees that the optimizer changed reactor pressure, column tray count, and recycle ratio, and can evaluate whether those changes make physical sense.

For teams considering this approach, the main upfront cost is running the GSA, which requires evaluating the simulator at Sobol sample points in the full design space. This is a one-time cost that pays for itself across multiple optimization runs. The neural network surrogate training is also lightweight. The real bottleneck is the high-fidelity simulator calls during optimization, and that's exactly what RS-MFBO minimizes.

Read the paper on arXiv