Stochastic Optimization and Reinforcement Learning, Fall 2022

Course Outline

Time and location

Time: Tuesday, Friday 2:00 - 3:50pm

Location: Carnegie 208


Yangyang Xu

Office: Amos Eaton 310

Office hour: in person, 3:50pm - 4:50pm on Tuesday and Friday or by appointment



Reading materials

  • On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems. Lin, Jin, Jordan. ICML, 2020.

  • Adaptive Primal-Dual Stochastic Gradient Method for Expectation-constrained Convex Stochastic Programs. Yan and Xu. Math. Programming Computation, 2022.

  • Primal-dual stochastic gradient method for convex programs with many functional constraints, Xu, SIOPT, 2020.

  • Algorithms for stochastic optimization with function or expectation constraints, Lan and Zhou, COAP, 2020.

  • Escaping Saddles with Stochastic Gradients, Hadi Daneshmand, Jonas Kohler, Aurelien Lucchi, Thomas Hofmann, ICML, 2018.

  • First-order methods almost always avoid strict saddle points, Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael I. Jordan, Benjamin Recht. Math. Programming, 2019.

  • Distributed Learning Systems with First-Order Methods, Liu and Zhang. Foundations and Trends in Databases, 2020.

  • A hybrid stochastic optimization framework for composite nonconvex optimization. Tran-Dinh, Pham, Phan, and Nguyen. Mathematical Programming, 2022.

  • Spiderboost and momentum: Faster variance reduction algorithms. Wang, Ji, Zhou, Liang, and Tarokh. NeurIPS, 2019.

  • Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization. Xu and Xu, 2022.

  • Stochastic model-based minimization of weakly convex functions, David and Drusvyatskiy, SIAM J. On Optimization, 2019.

  • iPiano: Inertial proximal algorithm for nonconvex optimization, Ochs, Chen, Brox, and Pock, SIAM J. On Imaging Sciences, 2014.

  • Gradient methods for minimizing composite functions, Nesterov, Math Programming, 2013.