Deep Learning Optimization Tasks and Metaheuristic Methods

In this paper we identify and formulate two optimization tasks solved in connection with training DL models and constructing adversarial examples. This guides our review of optimization methods commonly used within the DL community. Simultaneously, we present findings from the literature concerning metaheuristics and black-box optimization. We focus on well-known optimizers suitable for solving \(ℝ^N\) tasks, which achieve good results on benchmarks and in competitions. Finally, we look into the research connected with utilizing metaheuristic optimization methods in combination with deep learning models.

Recommended citation: Biedrzycki, R., Zawistowski, P., & Twardowski, B. (2019). "Deep Learning Optimization Tasks and Metaheuristic Methods." Fundamenta Informaticae, 168, Article 2–4.
Download Paper