Dot Product, Norm, Cosine Similarity TODO
Concept
The dot product of two vectors is defined as the sum of the products of corresponding components, and geometrically it equals the product of the two vectors' magnitudes times the cosine of the angle between them. A norm is a function measuring vector magnitude: the L2 norm is the square root of the dot product of a vector with itself, L1 is the sum of the absolute values of the components, and L∞ is the maximum absolute value. The Cauchy-Schwarz inequality says the absolute value of the dot product never exceeds the product of the two norms, which is exactly why dividing the dot product by the two norms always gives a value between -1 and 1, making cosine similarity well-defined. Cosine similarity ignores vector magnitude and compares direction only, so it's used instead of Euclidean distance when you want to exclude differences in document length or scale. A dot product of zero means orthogonality, which is the starting point for concepts like projection and least squares.
Practical problems like embedding search, similarity ranking, and how results change depending on whether you normalize, ultimately come down to which norm you're measuring with and whether you normalized magnitude.
Code & Formula
# 내적·노름·코사인 유사도 — L1/L2/L∞ 노름, 코사인 유사도, 코시-슈바르츠 부등식을 확인한다.
import numpy as np
u = np.array([3.0, 4.0, 0.0])
v = np.array([1.0, 2.0, 2.0])
dot = np.dot(u, v)
print(f"u={u}, v={v}")
print(f"내적 u·v = {dot}")
l1 = np.linalg.norm(u, 1)
l2 = np.linalg.norm(u, 2)
linf = np.linalg.norm(u, np.inf)
print(f"\nu의 노름: L1={l1}, L2={l2}, L∞={linf}")
cos_sim = dot / (np.linalg.norm(u) * np.linalg.norm(v))
print(f"\n코사인 유사도 cos(u,v) = {cos_sim:.4f}")
# 코시-슈바르츠: |u·v| <= ||u|| * ||v|| → 이 부등식 덕분에 cos_sim이 항상 [-1, 1] 안에 있다
cauchy_schwarz_bound = np.linalg.norm(u) * np.linalg.norm(v)
print(f"|u·v| = {abs(dot):.4f} <= ||u||*||v|| = {cauchy_schwarz_bound:.4f} ? {abs(dot) <= cauchy_schwarz_bound}")
# 크기가 다른 두 벡터라도 방향이 같으면 코사인 유사도는 1
w = u * 10 # u와 방향은 같고 크기만 10배
print(f"\nw = 10*u 의 코사인 유사도(u,w) = {np.dot(u, w) / (np.linalg.norm(u) * np.linalg.norm(w)):.4f} → 스케일 무시, 방향만 비교")
Exercise
For the same dataset, implement Euclidean-distance nearest neighbors and cosine-similarity nearest neighbors separately, and check how the results change before and after normalizing vectors to unit norm.
Practical Connection
This is used directly when clustering similar wallets by on-chain address behavior vectors or wiring up log/document search, and more generally it underlies predictive models like least squares and regression.
If you study this on a given day, add a note link and a ✅ to this line in the source curriculum (docs/knowledge/math-50-curriculum.md) and this spot will lead straight to the note body. You can also write directly on this page — but regenerating overwrites it, so it's safer to keep anything you want to save as markdown under docs/algorithms/.