Single-Objective Optimization Techniques
摘要
This chapter provides a detailed exploration of algorithms for Single-Objective Optimization (SOO), where the goal is to find the best solution for a single, well-defined objective function. It first addresses unconstrained optimization, where variables are not explicitly restricted, and constrained optimization, where the algorithm must respect specific equality or inequality conditions that define a feasible region. For unconstrained techniques, this chapter details gradient-based methods, including Gradient Descent (GD), Stochastic Gradient Descent (SGD), Newton’s method, and quasi-Newton methods, alongside derivative-free approaches like Nelder-Mead, pattern search, and evolutionary algorithms. Regarding constrained optimization, it covers foundational concepts such as Lagrange multipliers and KKT conditions, as well as techniques including penalty and barrier approaches (interior-point methods), Sequential Quadratic Programming (SQP), the simplex method for Linear Programming (LP), and Branch and Bound/Cut techniques for Integer Programming (IP/MIP). We also highlight the real-world relevance of selecting suitable single-objective optimization algorithms based on problem features like differentiability, convexity, scale, and variable types.