1. Remember for a curve , we have maxima and minima occur whenever
What’s the multivariable analog to this notion?
2. Definition. If , we say is a “Critical Point” of .
Example 1. Consider . What are its critical points?
Solution: We find its gradient first
Next we need to set each component to vanish. This implies
is the only critical point.
3. Problem: How do we determine if a critical point describes a maxima or minima?
Lets consider the critical point for . We Taylor expand to second order about writing
where we use the matrix
Each row is precisely , and each column likewise is ; the intuition is .
Now, since we are Taylor expanding about a critical point, our approximation becomes
We can consider the behaviour of near by studying the properties of . Specifically, the signs of the eigenvalues tells us whether the critical point is a local maxima (all eigenvalues are positive) or a minima (all are negative) or some saddle point (mixture having both positive and negative eigenvalues). If there exists at least one eigenvalue that vanishes, this test is inconclusive.
4. Parting Thoughts: What if we want to optimize a function constrained to live on a surface?