Subsampled first-order optimization methods with applications in imaging

This work presents and discusses optimization methods for solving finite-sum minimization problems which are pervasive in applications, including image processing. The procedures analyzed employ first-order models for the objective function and stochastic gradient approximations based on subsampling. Among the variety of methods in the literature, the focus is on selected algorithms which can be cast into two groups: algorithms using gradient estimates evaluated on samples of very small size; algorithms relying on gradient estimates and machinery from standard globally convergent optimization procedures. Neural Networks and Convolutional Neural Networks widely used for image processing tasks are considered and a classification problem of images is solved with some of the methods presented.