Converge Then Diversify: A Two-Stage Approach to Multi-Objective Bayesian Optimization

Summary

Multi-objective Bayesian optimization (MOBO) seeks a high-quality approximation of the Pareto front for expensive black-box functions. It must balance convergence, or getting close to the front, with diversity, or spreading solutions across it. The paper argues that this joint objective is especially difficult when the evaluation budget is very tight, because too few solutions may be available to cover the whole front. It proposes a converge-then-diversify (CTD) strategy that separates the search into two stages: first move quickly toward a single point on the Pareto front, then distribute solutions across the front. The authors give two simple CTD implementations based on widely used acquisition functions. Across 446 pairwise comparisons with state-of-the-art methods, CTD was statistically better in 72.9% of cases, equivalent in 21.1%, and worse in 6.1%. The reported advantage was particularly clear under very tight evaluation budgets and in high-dimensional problems.