This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [F.E. Curtis, K. Scheinberg, and R. Shi, A stochastic trust region algorithm based on careful step normalization, INFORMS. J. Optim. 1(3) (2019), pp. 200–220; F.E. Curtis and R. Shi, A fully stochastic second-order trust region method, Optim. Methods Softw. 37(3) (2022), pp. 844–877]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [R. Bollapragada, R. Byrd, and J. Nocedal, Adaptive sampling strategies for stochastic optimization, SIAM. J. Optim. 28(4) (2018), pp. 3312–3343]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters.

An investigation of stochastic trust-region based algorithms for finite-sum minimization / Stefania Bellavia, Benedetta Morini, Simone Rebegoldi. - In: OPTIMIZATION METHODS & SOFTWARE. - ISSN 1055-6788. - STAMPA. - ...:(In corso di stampa), pp. 0-0. [10.1080/10556788.2024.2346834]

An investigation of stochastic trust-region based algorithms for finite-sum minimization

Stefania Bellavia;Benedetta Morini;
In corso di stampa

Abstract

This work elaborates on the TRust-region-ish (TRish) algorithm, a stochastic optimization method for finite-sum minimization problems proposed by Curtis et al. in [F.E. Curtis, K. Scheinberg, and R. Shi, A stochastic trust region algorithm based on careful step normalization, INFORMS. J. Optim. 1(3) (2019), pp. 200–220; F.E. Curtis and R. Shi, A fully stochastic second-order trust region method, Optim. Methods Softw. 37(3) (2022), pp. 844–877]. A theoretical analysis that complements the results in the literature is presented, and the issue of tuning the involved hyper-parameters is investigated. Our study also focuses on a practical version of the method, which computes the stochastic gradient by means of the inner product test and the orthogonality test proposed by Bollapragada et al. in [R. Bollapragada, R. Byrd, and J. Nocedal, Adaptive sampling strategies for stochastic optimization, SIAM. J. Optim. 28(4) (2018), pp. 3312–3343]. It is shown experimentally that this implementation improves the performance of TRish and reduces its sensitivity to the choice of the hyper-parameters.
In corso di stampa
...
0
0
Stefania Bellavia, Benedetta Morini, Simone Rebegoldi
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in FLORE sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificatore per citare o creare un link a questa risorsa: https://hdl.handle.net/2158/1359393
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact