Differentiation, Gradients, and the Chain Rule TODO
Concept
The derivative is the coefficient of the linear function that best approximates a function near a point, so its value tells you how many times the output changes for a small change in the input. For a multivariable function, the gradient collects the partial derivatives with respect to each variable into a vector; it points in the direction of steepest increase at that point, and its magnitude is that rate of increase. The chain rule states that the derivative of a composite function is the product of the derivatives at each stage, and in the multivariable case this generalizes to a product of Jacobian matrices. Backpropagation computes this Jacobian product from the output side toward the input side, reusing intermediate results so that gradients can be obtained far more cheaply when there are many parameters. Gradient descent repeatedly moves a small step in the direction opposite the gradient to find a local minimum; the practical crux is that too large a step size (learning rate) causes divergence, while too small a one makes convergence painfully slow.
Optimization, curve fitting, and parameter tuning all run on gradients, and derivatives are exactly the language you need whenever you have to reason about a function's sensitivity — as with market-maker pricing curves. Without the chain rule you cannot compute the sensitivity of a composed system.
Code & Formula
# Day 27 — 미분·기울기·연쇄법칙
# 수치미분으로 도함수를 근사하고, 연쇄법칙 f(g(x))의 도함수를 직접 계산과 비교한다.
def numerical_diff(f, x, h=1e-6):
return (f(x + h) - f(x - h)) / (2 * h)
def f(x):
return x ** 3 + 2 * x
def f_prime_exact(x):
return 3 * x ** 2 + 2
x0 = 2.0
print(f"f'({x0}) 수치미분 근사 = {numerical_diff(f, x0):.6f}")
print(f"f'({x0}) 해석적 값 = {f_prime_exact(x0):.6f}")
# 연쇄법칙: h(x) = g(f(x)), g(u) = sin(u) 라 하면 h'(x) = g'(f(x)) * f'(x)
import math
def g(u):
return math.sin(u)
def h(x):
return g(f(x))
def h_prime_chain_rule(x):
g_prime = math.cos(f(x)) # g'(u) = cos(u), u = f(x)
return g_prime * f_prime_exact(x)
print(f"\nh'({x0}) 수치미분 근사 = {numerical_diff(h, x0):.6f}")
print(f"h'({x0}) 연쇄법칙 계산 = {h_prime_chain_rule(x0):.6f}")
Exercise
Pick a simple multivariable function, compute its gradient by hand, compare it against a numerical derivative, then run a few steps of gradient descent yourself and observe the learning-rate threshold where convergence turns into divergence.
Practical Connection
An LMSR market maker's price is defined as the partial derivative of the cost function, so derivatives and gradients feed directly into understanding Verex's price calculation and its sensitivity to slippage.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.