Optimization Methods for Machine Learning

Machine learning (ML) problems are often posed as highly nonlinear and nonconvex unconstrained optimization problems. Methods for solving ML problems based on stochastic gradient descent generally require fine-tuning many hyper-parameters. In this talk we discuss alternative approaches for solving ML problems based on a quasi-Newton trust-region framework that does not require extensive parameter tuning. We will present numerical results that demonstrate the potential of the proposed approaches.