Lagrange Multipliers

1. So, last time we considered finding extrema for some function {f\colon{\mathbb R}^{n}\rightarrow{\mathbb R}}, but what if we constrain our focus to some surface {g\colon{\mathbb R}^{n}\rightarrow{\mathbb R}}? For example, the unit circle would have
(1)\displaystyle  g(x,y) = x^{2}+y^{2} - 1=0
How do we find extrema for {f(x,y)=xy} on the unit circle?

2. What can we do? First we can consider the level curves {f(x,y)=c}. These are precisely the curves {y=c/x}. The gradient vector for {f} is precisely
(2)\displaystyle  \nabla f=\langle y,x\rangle.
This points in the direction of increasing values of {f}.

3. We want to consider the situation when {\nabla f=\lambda\nabla g}, i.e., when the gradient of {f} is precisely a scaled tangent of {g}. Why? Because we want the gradient of {f} to point in the direction of a tangent to our surface. We draw the circle, the gradient vector {\nabla f} in red, and {\nabla g} in blue. Remember, the red vectors point in the direction of increasing values of {f}, and we restrict our movement along the circle:

Observe when the red and blue vectors are perpendicular, {f=0}. But when they overlap as a purple vector or point in completely opposite direction, what happens?

This happens when
(3)\displaystyle  \langle y,x\rangle = \lambda\langle 2x,2y\rangle
or equivalently
(4)\displaystyle  y=2\lambda x,\quad\mbox{and}\quad x=2\lambda y.
Solving for {2\lambda}, we find
(5)\displaystyle  2\lambda = \frac{y}{x} = \frac{x}{y}
which implies
(6)\displaystyle  x^{2}=y^{2}.
But we’re not quite done!

4. We must remain on the circle, so we also must demand that {x^{2}+y^{2}=1}. This equivalently implies
(7)\displaystyle  x^{2}=\frac{1}{2}\implies x=\pm\frac{\sqrt{2}}{2}.

5. Is this optimal? Lets try approaching the problem differently. We are working on the circle, which is the parametric curve
(8)\displaystyle  x(t) = \cos(t),\quad\mbox{and}\quad y(t) = \sin(t).
Thus the function we are optimizing becomes
(9)\displaystyle  \begin{aligned} f(t) &= f\bigl(x(t),y(t)\bigr)\\ &=\cos(t)\sin(t) \end{aligned}
We see
(10)\displaystyle  f'(t) = -\sin^{2}(t)+\cos^{2}(t) = 0.
We need to solve for {t}, to do so we rearrange terms
(11)\displaystyle  \sin^{2}(t)=\cos^{2}(t)
and divide through by {\cos^{2}(t)}, getting
(12)\displaystyle  \tan^{2}(t) = 1.
But this implies {t=(2n+1)\pi/4} where {n=0}, {1}, {2}, or {3}. Look: that’s precisely describing {(x,y)=(\pm1/\sqrt{2},\pm1/\sqrt{2})}.

6. Lagrange Multipliers, Constraints. One way to consider this situation “Optimize {f} subject to the constraint {g=0}” is to say Okay, so suppose {g=0}, then wouldn’t we have
(13)\displaystyle  f + \lambda g\approx f?
The {g} term vanishes anyways, so intuitively it seems “equal-ish”.

Example 1 (Minimizing Surface Area). Find the dimensions of the cylinder with smallest surface area whose volume is fixed at {16\pi}.

Solution: The outline takes several steps, namely, (1) construct the function, (2) take the derivatives, (3) solve.

Step One: Construct the Functions. We first write
(14)\displaystyle  A(r,h) = \pi r^{2} + \pi r^{2} + 2\pi rh
for the surface area, and
(15)\displaystyle  V(r,h) = \pi r^{2}h
describes the area. The constraint is
(16)\displaystyle  C(r,h) = V(r,h) - 16\pi.
Thus we construct the function
(17)\displaystyle  F(r,h) = A(r,h) - \lambda C(r,h).
This concludes the first step.

Step Two: Take the Derivatives. We find
(18)\displaystyle  \nabla F(r,h) = \langle \partial_{r}A(r,h) - \lambda\partial_{r}C(r,h), \partial_{h}A(r,h)-\lambda\partial_{h}C(r,h)\rangle = 0.
(19)\displaystyle  \partial_{r}A(r,h)=4\pi r+2\pi h,\quad\mbox{and}\quad \partial_{r}C(r,h)=2\pi rh.
We also have
(20)\displaystyle  \partial_{h}A(r,h)=2r\pi,\quad\mbox{and}\quad \partial_{h}C(r,h)=\pi r^{2}.
The derivative with respect to the Lagrange multiplier gives us
(21)\displaystyle  \partial_{\lambda}F(r,h) = C(r,h) = 0.
So we can set up our equations as
(22)\displaystyle  \begin{aligned} (\partial_{r}F&=0)&\quad &\quad &2\pi r+2\pi h&=\lambda 2\pi rh\\ (\partial_{h}F&=0)&\quad &\quad &2\pi r &=\lambda\pi r^{2}\\ (\partial_{\lambda}F&=0)&\quad &\quad &\pi r^{2}h-16\pi&=0. \end{aligned}
That concludes our second step.

Step Three: Solve. We see immediately from the {\partial_{h}F} equation that
(23)\displaystyle  2\pi r = \lambda \pi r^{2}\implies \lambda=\frac{2}{r}.
We plug this into the {\partial_{r}F} equation, we get
(24)\displaystyle  \begin{aligned} 2\pi r+2\pi h &= \lambda2\pi rh\\ &=\left(\frac{2}{r}\right)2\pi rh\\ &=4\pi h \end{aligned}
and subtracting {2\pi h} from both sides yields
(25)\displaystyle  2\pi r=2\pi h\implies r=h.
Now we use the constraint
(26)\displaystyle  \pi r^{2}h = 16\pi
substituting {r=h} we get
(27)\displaystyle  \pi h^{3} = 16\pi\implies h = \sqrt[3]{16}.
Thus when we take {r=h=\sqrt[3]{16}}, we minimize the surface area.

Exercise 1. Find the extrema of {f(x,y,z)=xy+yz+zx} subject to the constraint {g(x,y,z)=x^{2}+y^{2}+z^{2}-1}.

Exercise 2. Find the extrema of {f(x,y)=\exp(-xy)} subject to the constraint {g(x,y)=x^{2}+y^{2}-1}.


About Alex Nelson

I like math. I like programming. Most of all, I love puzzles.
This entry was posted in Derivative, Gradient, Lagrange Multipliers. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s