# Derivative free algorithms for large scale non-smooth optimization and their applications

Download
2013
Tor, Ali Hakan
In this thesis, various numerical methods are developed to solve nonsmooth and in particular, nonconvex optimization problems. More speciﬁcally, three numerical algorithms are developed for solving nonsmooth convex optimization problems and one algorithm is proposed to solve nonsmooth nonconvex optimization problems. In general, main diﬀerences between algorithms of smooth optimization are in the calculation of search directions, line searches for ﬁnding step-sizes and stopping criteria. However, in nonsmooth optimiza- tion there is one additional diﬀerence between algorithms. These al gorithms may use diﬀerent gener- alizations of the gradient. In order to develop algorithms for solving nonsmooth convex optimization problems we use the concept of codiﬀerential. Although there exists the odiﬀerential calculus, the calculation of the whole codiﬀerential is not an easy task. Therefore, in the ﬁrst numerical method, only a few elements of the codiﬀerential are used to calculate search directions. In order to reduce the number of codiﬀerential evaluations, in the second method elements of the codiﬀerential calculated in previous iterations are used to calculate search directions. In both the ﬁrst and second methods the problem of calculation of search directions is reduced to the solution of a certain quadratic programming problem. The size of this problem can increase signiﬁcantly as the number of variables increases. In order to avoid this problem in the third method, called the aggregate codiﬀerential method, the number of elements of the codiﬀerential used to ﬁnd search directions is ﬁxed. Such an approach allows one to signiﬁcantly reduce the complexity of codiﬀerential methods and to make them applicable for solving large scale problems of nonsmooth optimization. These methods are applied to some well-known nonsmooth optimization test problems, such as, min-max and general type nonsmooth optimization problems. The obtained numerical results are visualized using performance profiles. In addition, the validation of these methods is made by comparing them with the subgradient and bundle methods using results of numerical experiments. The convergence of methods is analyzed. Finally, the first method is extended to minimize nonsmooth convex functions subject to linear inequalities using slack variables. The notion of quasisecant is used to design an algorithm for solving nonsmooth nonconvex unconstrained optimization problems. In this method, to find descent direction the subgradient algorithm is applied for the solution of a set of linear inequalities. The convergence of the proposed method is analyzed, and the numerical experiments are carried out using general type nonsmooth optimization test problems. To validate this method, the results are compared with those by the subgradient method.
Citation Formats
A. H. Tor, “Derivative free algorithms for large scale non-smooth optimization and their applications,” Ph.D. - Doctoral Program, 2013. 