Adaptive stochastic approximation algorithm
Journal
Numerical Algorithms
Date Issued
2017-02-27
Author(s)
Kresoja, Milena
Lužanin, Zorana
DOI
10.1007/s11075-017-0290-4
Abstract
In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.