Optimization Methods in Data Science

Course Description

This course provides a rigorous, accessible introduction to optimization theory and algorithms, with direct application to data science and machine learning workflows. Topics include linear programming and the simplex method, duality and sensitivity analysis, interior-point methods, network flows, and integer programming via branch and bound. The second half of the course is focused on continuous optimization, which underlies modern machine learning. Topics covered include convex analysis and optimality conditions, gradient descent and line search, Newton and quasi-Newton methods, the stochastic first-order methods central to model training, and constrained optimization through the Karush-Kuhn-Tucker conditions. The course closes with derivative-free methods and heuristics for problems where gradients are unavailable. Throughout, students formulate real-world problems as optimization models, work key algorithms by hand on small instances to build intuition, and implement and analyze them in Python using Jupyter notebooks and the scientific-computing stack, including NumPy, SciPy, and CVXPY. No prior experience with these libraries is assumed or needed.

Code

EN.685.664

Credits

3

Subject

Data Science