API Reference#

This code is a python port of the famous implementation of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS), algorithm 778 written in Fortran [2,3] (last update in 2011). Note that this is not a wrapper such as minimize in scipy but a complete reimplementation (pure python). The original code can be found here: https://dl.acm.org/doi/10.1145/279232.279236

The aim of this reimplementation was threefold. First, familiarize ourselves with the code, its logic and inner optimizations. Second, gain access to certain parameters that are hard-coded in the Fortran code and cannot be modified (typically wolfe conditions parameters for the line search). Third, implement additional functionalities that require significant modification of the code core.

Main interface#

Interface for scalar function minimization with L-BFGS-B.

minimize_lbfgsb(*, x0, fun, jac[, ...])

Minimize a scalar function of one or more variables using the L-BFGS-B algorithm.

Utilitairies#

Additional utilitairy functions to work with inputs or outputs.

`extract_hess_inv_diag`(hess_inv)	Extract efficiently the diagonal of the L-BFGS approximate inverse Hessian.
`get_grad_projection_inf_norm`(x, grad, ...)	Get the infinite norm of the projected gradient.

Benchmark functions#

Provide the following function and their gradients to benchmark our implementation.

`ackley`(x)	The Ackley function.
`ackley_grad`(x)	The gradient of the Ackley function.
`beale`(x)	The Beale function.
`beale_grad`(x)	The gradient of the Quartic function.
`griewank`(x)	The Griewank function.
`griewank_grad`(x)	The gradient of the Griewank function.
`quartic`(x)	The Quartic function.
`quartic_grad`(x)	The gradient of the Quartic function.
`rastrigin`(x)	The Rastrigin function.
`rastrigin_grad`(x)	The gradient of the Rastrigin function.
`rosenbrock`(x)	The Rosenbrock function.
`rosenbrock_grad`(x)	The gradient of the Rosenbrock function.
`sphere`(x)	The Sphere function.
`sphere_grad`(x)	The gradient of the Sphere function.
`styblinski_tang`(x)	The Styblinski-Tang function.
`styblinski_tang_grad`(x)	The gradient of the Styblinski-Tang function.

Inner functions#

Inner functions of L-BFGS-B.

`base`	Base functions used by the L-BFGS-B routine.
`bfgsmats`	An implicit representation of the BFGS approximation to the Hessian matrix B
`cauchy`	Implement a function to compute the generalized Cauchy point (GCP) for the L-BFGS-B algorithm, mainly for internal use.
`linesearch`	Implement the line search algorithm by Moré and Thuente (1994), currently used for the L-BFGS-B algorithm.
`scalar_function`
`subspacemin`	Subspace minimization procedure of the L-BFGS-B algorithm, mainly for internal use.

References

[1] R. H. Byrd, P. Lu and J. Nocedal. A Limited Memory Algorithm for Bound: Constrained Optimization, (1995), SIAM Journal on Scientific and Statistical Computing, 16, 5, pp. 1190-1208.
[2] C. Zhu, R. H. Byrd and J. Nocedal. L-BFGS-B: Algorithm 778: L-BFGS-B,: FORTRAN routines for large scale bound constrained optimization (1997), ACM Transactions on Mathematical Software, 23, 4, pp. 550 - 560.
[3] J.L. Morales and J. Nocedal. L-BFGS-B: Remark on Algorithm 778: L-BFGS-B,: FORTRAN routines for large scale bound constrained optimization (2011), ACM Transactions on Mathematical Software, 38, 1.