PyBayesOpt.jl Documentation

Julia wrapper for Bayesian optimization tools available in Python.

Overview

PyBayesOpt.jl provides Julia interfaces to popular Python Bayesian optimization libraries:

BayesianOptimization - A Python library for Bayesian optimization
BoTorch - A Bayesian optimization library built on PyTorch for q-batch optimization

The package implements the Optim.jl interface for seamless integration with Julia's optimization ecosystem.

Quick Start

Basic BoTorch Q-Batch Optimization

using PyBayesOpt
using Optim

# Define a test function to minimize
objective(x) = (x[1] - 2)^2 + (x[2] - 1)^2

# Set up optimization parameters
params = BoTorchQBatch(
    bounds = [-5.0 5.0; -5.0 5.0],  # [min max] for each dimension  
    nbatch = 4,     # batch size
    ninit = 10,     # initialization iterations
    nopt = 20       # optimization iterations
)

# Run optimization using Optim.jl interface
result = optimize(objective, params)

# Access results
println("Best point: ", result.minimizer)
println("Best value: ", result.minimum)
println("Function evaluations: ", result.f_calls)

BayesianOptimization Wrapper

# Using the BayesianOptimization wrapper
params = BayesianOptimization(
    bounds = [-5.0 5.0; -5.0 5.0],
    ninit = 10,
    nopt = 50
)

result = optimize(objective, params)

Interactive Optimization Loop

For more control over the optimization process:

# Interactive optimization with BoTorch
state = BoTorchQBatchState(params=BoTorchQBatch(bounds=[-5.0 5.0; -5.0 5.0]))

while !finished(state)
    # Get next batch of points to evaluate
    pts = ask!(state)
    
    # Evaluate function (can be parallelized)
    values = [objective(pts[:, i]) for i in 1:size(pts, 2)]
    
    # Provide results back to optimizer
    tell!(state, pts, values)
    
    # Check current best
    if state.optimization_complete
        best_pt, best_val = bestpoint(state)
        println("Current best: $best_pt -> $best_val")
    end
end

Benchmark Functions

The package includes several standard optimization benchmarks:

# Use benchmark functions
branin = BraninFunction()
result = optimize(branin, BoTorchQBatch(bounds=branin.bounds))

ackley = AckleyFunction(dim=3)  # 3D Ackley function
result = optimize(ackley, BoTorchQBatch(bounds=ackley.bounds))

rosenbrock = RosenbrockFunction(dim=2)
result = optimize(rosenbrock, BoTorchQBatch(bounds=rosenbrock.bounds))

Using Optim.jl Interface

PyBayesOpt implements the Optim.jl optimize function interface, making it a drop-in replacement for other Optim.jl optimizers:

using Optim
using PyBayesOpt

# Your objective function
function objective(x)
    return sum((x .- [2.0, 3.0]).^2)  # minimize at [2.0, 3.0]
end

# Create method instance
method = BoTorchQBatch(
    bounds = [0.0 5.0; 0.0 5.0],
    nbatch = 2,
    ninit = 8,
    nopt = 15,
    acqmethod = :qEI
)

# Use with Optim.optimize
result = Optim.optimize(objective, method)

# Standard Optim.jl result interface
println("Minimizer: ", Optim.minimizer(result))
println("Minimum: ", Optim.minimum(result))
println("F calls: ", Optim.f_calls(result))

# Additional methods from BoTorch
if isa(result, BoTorchQBatchState)
    # Evaluate posterior at any point
    mean, var = evalposterior(result, [2.0, 3.0])
    println("Posterior at [2,3]: mean=$mean, variance=$var")
    
    # Sample from posterior maximum distribution
    max_point, std_dev = sampleposteriormin(result, nsamples=1000)
    println("Estimated maximum: $max_point ± $std_dev")
end

Advanced Usage

Acquisition Functions

BoTorch supports various acquisition functions:

method = BoTorchQBatch(
    acqmethod = :qLogEI,  # Options: :qEI, :qNEI, :qLogEI, :qLogNEI, :qUCB, :qPI
    qUCB_beta = 2.5       # Only used for :qUCB
)

Parallel Evaluation

The q-batch approach naturally supports parallel function evaluation:

using Distributed
addprocs(4)  # Add 4 worker processes

@everywhere using PyBayesOpt

function parallel_eval(objective, state)
    while !finished(state)
        pts = ask!(state)
        
        # Parallel evaluation using pmap
        values = pmap(i -> objective(pts[:, i]), 1:size(pts, 2))
        
        tell!(state, pts, values)
    end
    return state
end

result = parallel_eval(objective, BoTorchQBatchState(params=method))

API Reference

PyBayesOpt — Module

PyBayesOpt

PyBayesOpt.jl

Julia wrapper to some Bayesian optimization tools available in Python:

BayesianOptimization aka bayes_opt or bayesian-optimization
A q-batch Bayesian optimizer implemented using BoTorch

It uses a very partial implementation of the optimizer interface provided by Optim.jl.

Installation

Python prerequisites

The package uses PyCall.jl to access the python code, and therefore requires a python installation which is working well with this package. See requirements.txt for the python packages to be installed.

pip install -r requirements.txt

If you use this under linux, you may have to set the environment variable TORCH_USE_RTLD_GLOBAL=1 in order to avoid some loading problems with the mkl library:

export TORCH_USE_RTLD_GLOBAL=1

Installation via PackageNursery registry

The package can be installed with the Julia package manager in a standard way For the time being, it is registered in the julia package registry https://github.com/j-fu/PackageNursery To add the registry (needed only once), and to install the package, from the Julia REPL, type ] to enter the Pkg REPL mode and run:

pkg> registry add https://github.com/j-fu/PackageNursery

Please be aware that adding a registry to your Julia installation requires to trust the registry maintainer for handling things in a correct way. In particular, the registry should not register higher versions of packages which are already registered in the Julia General Registry. One can check this by visiting the above mentionend github repository URL and inspecting the contents.

Installation via repository URL

using Pkg
Pkg.add(url="https://github.com/j-fu/PyBayesOpt.jl")

Quick Start

Basic Usage with Optim.jl Interface

using PyBayesOpt
using Optim

# Define a function to minimize
f(x) = (x[1] - 2.0)^2 + (x[2] - 1.0)^2

# Create optimizer - BoTorch q-batch is recommended
method = BoTorchQBatch(
    bounds = [0.0 4.0; -1.0 3.0],  # [min max] for each dimension
    nbatch = 4,      # evaluate 4 points per iteration
    ninit = 10,      # 10 initialization iterations  
    nopt = 15        # 15 optimization iterations
)

# Optimize using standard Optim.jl interface
result = optimize(f, method)

# Access results
println("Best point: ", result.minimizer)    # [2.0, 1.0]
println("Best value: ", result.minimum)      # ≈ 0.0
println("Evaluations: ", result.f_calls)     # 100 total evaluations

Interactive Optimization Loop

For maximum control over the optimization process:

state = BoTorchQBatchState(params=method)

while !finished(state)
    # Get next batch of points to evaluate
    candidates = ask!(state)
    
    # Evaluate function (can be parallelized)  
    values = [f(candidates[:, i]) for i in 1:size(candidates, 2)]
    
    # Provide results back to optimizer
    tell!(state, candidates, values)
end

# Final result
best_point, best_value = bestpoint(state)

Using Benchmark Functions

# Built-in benchmark functions
branin = BraninFunction()
result = optimize(branin, BoTorchQBatch(bounds=branin.bounds))

ackley = AckleyFunction(dim=5)  # 5-dimensional
result = optimize(ackley, BoTorchQBatch(bounds=ackley.bounds))

Features

Acquisition Functions

BoTorch supports multiple acquisition functions:

:qEI / :qExpectedImprovement - Expected Improvement
:qLogEI / :qLogExpectedImprovement - Log Expected Improvement (recommended)
:qNEI / :qNoisyExpectedImprovement - Noisy Expected Improvement
:qLogNEI / :qLogNoisyExpectedImprovement - Log Noisy Expected Improvement
:qUCB / :qUpperConfidenceBound - Upper Confidence Bound
:qPI / :qProbabilityOfImprovement - Probability of Improvement

Parallel Evaluation

The q-batch approach naturally supports parallel function evaluation:

using Base.Threads

# In the optimization loop:
values = zeros(nbatch)
@threads for i in 1:nbatch
    values[i] = expensive_function(candidates[:, i])
end

Posterior Analysis

After optimization, you can analyze the Gaussian process posterior:

# Evaluate posterior at any point
mean, variance = evalposterior(result, [1.0, 2.0])

# Sample from posterior maximum distribution
max_point, std_dev = samplemaxpost(result, nsamples=1000)
println("Estimated global optimum: $max_point ± $std_dev")

Optimizers

BoTorchQBatch (Recommended)

Advanced q-batch Bayesian optimization using BoTorch:

BoTorchQBatch(
    bounds = [-1.0 1.0; -1.0 1.0],    # Optimization bounds
    nbatch = 4,                        # Batch size
    ninit = 10,                        # Initialization iterations
    nopt = 20,                         # Optimization iterations  
    acqmethod = :qLogEI,               # Acquisition function
    seed = 1234,                       # Random seed
    verbose = true,                    # Print progress
    acq_nrestarts = 20,                # Acquisition optimization restarts
    acq_nsamples = 512,                # Raw samples for acquisition optimization
    qUCB_beta = 2.0                    # Beta parameter for qUCB
)

BayesianOptimization

Classical Bayesian optimization wrapper:

BayesianOptimization(
    bounds = [-1.0 1.0; -1.0 1.0],    # Optimization bounds
    ninit = 10,                        # Initial random samples
    nopt = 50,                         # Optimization iterations
    verbose = 0,                       # Verbosity level
    seed = 1                           # Random seed
)

Examples

See the examples/ directory for:

quick_start.jl - Basic usage examples
optim_interface_example.jl - Comprehensive Optim.jl interface demo

AI usage statement

Github copilot with Claude Sonnet 4 and GPT-5 was used to design an initial python version of the BoTorch based algorithm, and to brush up documentation and testing infrastructure.