Research

Bayesian optimization, optimal design of experiments, and the algorithms that construct them

KARGO: trust-region Bayesian optimization

KARGO (Kriging-based Adaptive-Region Global Optimization) is a Bayesian optimization method for expensive, moderate-dimensional black-box minimization. At every iteration it maintains a local region of interest defined as the intersection of two complementary pieces: a data-driven quantile bounding box, built from the per-coordinate range of the best-performing subset of evaluated points, which adapts the shape of the region to the response distribution; and a scalar trust-region cap whose radius is updated under a sufficient-decrease criterion, which controls its size. Expected improvement is maximized inside this region, but the first inner step of every iteration is reserved for a global EI maximization over the full domain, so the algorithm cannot lock onto a sub-optimal basin.

I benchmark KARGO against EGO, TREGO, and TRIKE on a 14-cell synthetic suite spanning dimensions 2 to 10, with replicated runs under a fixed evaluation budget and a shared initial design that removes surrogate-fit variance from the comparison. Across the suite KARGO is the single strongest method, and a linear-model sensitivity analysis of its trust-region controls places the default settings inside the 95% confidence interval of the predicted optimum. Manuscript under review.

Related talks: Kriging-based Adaptive Region Global Optimization Algorithm (Loma Linda Workshop, 2025); A Trust Region Novel Optimization Algorithm (ICODOE, University of Memphis, Tennessee, 2023).

Optimal designs: UniPro, MaxPro, and maximin

Space-filling designs are the workhorse of computer experiments, and finding good ones is a hard combinatorial optimization problem. My dissertation work (with Hongquan Xu, UCLA) developed a tuned differential evolution algorithm for constructing uniform projection designs (UniPro), with the tuning itself treated as a design-of-experiments problem. The same engine extends to maximum projection (MaxPro) and maximin distance criteria.

The result is published in the Journal of Statistical Planning and Inference (2025) and implemented in UniPro, an R package whose core is native C parallelized with OpenMP, which makes routine the construction of designs at sizes where general-purpose optimizers stall.

Related talk: Generation of Uniform Projection Designs via Differential Evolution (EcoSta 2025, Waseda University, Tokyo).

Hyperparameter optimization

Running through all of the above is a practical interest in hyperparameter optimization for machine-learning models: the same surrogate-based and design-based tools (Bayesian optimization, space-filling initialization, adaptive sampling) applied to tuning deep networks and gradient-boosted models under tight evaluation budgets.