Second Order Momentum

In this project, we explored the effects of applying momentum and Nesterov’s accelerated gradient, in their usual formulations, to second order methods. We applied these acceleration techniques Newton’s method and quasi-Newton methods such as BFGS and cubic regularization. We then compared the empirical rate of convergence of first and second order methods, with and without momentum, on a set of benchmark optimization problems.