## Finding Extrema of Multivariable Functions

1. Remember for a curve ${y=f(x)}$, we have maxima and minima occur whenever
(1)$\displaystyle f'(x_{0}) = 0$
What’s the multivariable analog to this notion?

2. Definition. If ${\vec{\nabla}f(\vec{x}_{0})=\vec{0}}$, we say ${\vec{x}_{0}}$ is a “Critical Point” of ${f}$.

Example 1. Consider ${f(x,y) = x^{2}+y^{2} - 2x - 8y}$. What are its critical points?

Solution: We find its gradient first
(2)$\displaystyle \vec{\nabla}f = \langle 2x - 2, 2y - 8\rangle$
Next we need to set each component to vanish. This implies
(3)$\displaystyle \vec{x}_{0} = \langle 1, 4\rangle$
is the only critical point.

3. Problem: How do we determine if a critical point describes a maxima or minima?

Lets consider the critical point ${\vec{x}_{0}}$ for ${f}$. We Taylor expand ${f}$ to second order about ${\vec{x}_{0}}$ writing
(4)$\displaystyle f(\vec{x}_{0}+\vec{h}) \approx f(\vec{x}_{0}) + \vec{h}\cdot\underbrace{\vec{\nabla}f(\vec{x}_{0})}_{=0} + \frac{1}{2} \vec{h}\cdot\mathrm{Hess}(f)\vec{h}$
where we use the matrix
(5)$\displaystyle \mathrm{Hess}(f) = \begin{bmatrix} \partial_{1}^{2} f & \dots & \partial_{1}\partial_{n}f \\ \vdots & \ddots & \vdots \\ \partial_{n}\partial_{1} f & \dots & \partial_{n}^{2}f \end{bmatrix}$
Each row is precisely ${\vec{\nabla}\partial_{j}f}$, and each column likewise is ${\partial_{i}\vec{\nabla}f}$; the intuition is ${\mathrm{Hess}(f) \approx \vec{\nabla}^{2}f}$.

Now, since we are Taylor expanding about a critical point, our approximation becomes
(6)$\displaystyle f(\vec{x}_{0}+\vec{h}) \approx f(\vec{x}_{0}) + \frac{1}{2} \vec{h}\cdot\mathrm{Hess}(f)\vec{h}.$
We can consider the behaviour of ${f}$ near ${\vec{x}_{0}}$ by studying the properties of ${\mathrm{Hess}(f)}$. Specifically, the signs of the eigenvalues tells us whether the critical point is a local maxima (all eigenvalues are positive) or a minima (all are negative) or some saddle point (mixture having both positive and negative eigenvalues). If there exists at least one eigenvalue that vanishes, this test is inconclusive.

4. Parting Thoughts: What if we want to optimize a function constrained to live on a surface?