Lagrange Multipliers and KKT Conditions (Concept) TODO
Concept
For constrained optimization, the method of Lagrange multipliers builds a Lagrangian by adding the equality constraints to the objective function, each scaled by a multiplier, and then looks for its stationary points. Geometrically, this expresses the condition that at the optimum, the gradient of the objective function must lie within the space spanned by the gradients of the constraints — meaning there's no direction along the constraint that still improves the objective. The KKT conditions extend this to include inequality constraints, and consist of stationarity of the Lagrangian, primal feasibility, non-negativity of the inequality multipliers, and complementary slackness. Complementary slackness requires that a multiplier be zero whenever its constraint isn't active — formalizing the intuition that a slack constraint doesn't affect the optimum. KKT conditions are generally necessary conditions, but for convex problems satisfying an appropriate constraint qualification, they are also sufficient for optimality.
Real-world optimization — allocation under risk limits, position optimization under collateral constraints, scheduling under resource constraints — is almost always constrained, and the multipliers give you a bonus: they're the value of relaxing a constraint by one unit.
Code & Formula
# Day 30 — 라그랑주/KKT(개념)
# 등식 제약 x + y = 1 아래에서 f(x,y) = x^2 + y^2 최소화를 라그랑주 승수법으로 푼다.
# L(x,y,lam) = x^2 + y^2 - lam*(x + y - 1); 정상점 조건: 2x=lam, 2y=lam, x+y=1
import numpy as np
# 정상점 조건을 선형연립방정식으로 세운다: [2, 0, -1; 0, 2, -1; 1, 1, 0] [x,y,lam]^T = [0,0,1]^T
A = np.array([
[2.0, 0.0, -1.0],
[0.0, 2.0, -1.0],
[1.0, 1.0, 0.0],
])
b = np.array([0.0, 0.0, 1.0])
x, y, lam = np.linalg.solve(A, b)
print(f"라그랑주 해: x = {x:.4f}, y = {y:.4f}, lambda = {lam:.4f}")
print(f"제약 확인 x + y = {x + y:.4f} (목표: 1)")
print(f"목적함수 f(x,y) = {x**2 + y**2:.6f}")
def f(x, y):
return x ** 2 + y ** 2
# 대칭성으로 예상되는 답 (0.5, 0.5)과 비교, 제약을 만족하는 다른 점들과 비교해 최소임을 확인
print("\n제약선 위 다른 점들과 비교 (모두 x+y=1을 만족):")
for t in [0.0, 0.3, 0.5, 0.7, 1.0]:
xt, yt = t, 1 - t
print(f" (x,y)=({xt:.2f},{yt:.2f}) -> f = {f(xt, yt):.6f}")
Exercise
Set up and solve by hand the KKT conditions for maximizing a simple convex objective under a budget constraint, then solve the same problem with a numerical optimization library and check that the multiplier values match.
Practical Connection
Setting a market maker's liquidity parameter under a maximum-loss cap, or adjusting a fee structure under constraints, is exactly a constrained optimization problem, and the multiplier reads as the marginal benefit of relaxing that cap.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.