In our last lecture, we solved optimization problems by substituting the constraint into the objective function to reduce the number of variables. This method works well when the algebra is straightforward. But what if it isn't?
Suppose we want to build a rectangular box with no lid from 12 m$^2$ of cardboard. What is the maximum possible volume?
Objective: Maximize Volume $V = xyz$.
Constraint: Surface Area $g(x,y,z) = xy + 2xz + 2yz = 12$.
If we solve the constraint for $z$, we get $z = \frac{12-xy}{2(x+y)}$. Substituting this into the volume equation gives:
$$V(x,y) = xy \left( \frac{12-xy}{2(x+y)} \right) = \frac{12xy - x^2y^2}{2(x+y)}$$Finding the partial derivatives of this function, setting them to zero, and solving would be extremely tedious and prone to error. This highlights the need for a more elegant and powerful method for constrained optimization.
Formally, we want to find the max/min of an objective function $f(x, y, z)$ subject to a constraint $g(x, y, z) = k$.
The Setup: We have a mountain $f(x,y)$ with its contour lines (lines of constant elevation). We also have a fixed railway track $g(x,y)=k$ running along the side of the mountain. Our goal is to find the highest point on the track.
The Journey: As your railcar moves along the track $g$, you cross various contour lines: 100ft, 200ft, 300ft... You're going uphill.
The Maximum Point: Your railcar reaches its highest point. Let's say you've just touched the 350ft contour line. At this exact moment:
Therefore, at the precise instant you are at the highest point, your direction of travel must be momentarily horizontal. You can't be going up and you can't be going down.
The Tangency: What does it mean to be moving horizontally? It means you are moving parallel to the contour line. If your track were to cross the contour line at an angle, it would mean part of the track lies on the "uphill" side of the line, so you wouldn't be at the maximum yet. The only way for the track to hit the 350ft line as its absolute maximum is to just barely touch it and turn away.
Therefore, at the highest point, the track ($g=k$) must be tangent to the contour line ($f=c$).
Because the gradients are perpendicular to their respective curves, if the curves are tangent, the gradients must be parallel. This gives us the fundamental relationship:
$$\nabla f = \lambda \nabla g$$
The new variable $\lambda$, called the Lagrange multiplier, isn't just a random scalar. It has a very useful meaning: $\lambda$ represents the rate of change of the maximum value of $f$ with respect to the constraint constant $k$.
Let's use a simple economic analogy:
In this scenario, $\lambda$ tells you approximately how much additional happiness you would get for one additional dollar in your budget. If you solve a problem and find $\lambda = 5$, it means that if your budget were increased from $k$ to $k+1$ dollars, your maximum happiness would increase by about 5 units. It's the "bang for your buck" you get from relaxing the constraint.
To find the max/min of $f(x,y,z)$ subject to $g(x,y,z)=k$, we find all values of $x, y, z,$ and $\lambda$ that simultaneously solve the following system of equations:
$$ \nabla f(x,y,z) = \lambda \nabla g(x,y,z) \quad \text{and} \quad g(x,y,z) = k $$This vector equation breaks down into a system:
The solutions $(x,y,z)$ to this system are the candidate points for maxima and minima.
Find the extreme values of $f(x,y) = x+2y$ on the circle $x^2+y^2=5$.
Step 1: Identify functions and set up the system.
Objective: $f(x,y) = x+2y$. Constraint: $g(x,y) = x^2+y^2 = 5$.
Gradients: $\nabla f = \langle 1, 2 \rangle$ and $\nabla g = \langle 2x, 2y \rangle$.
The system $\nabla f = \lambda \nabla g$ becomes:
Step 2: Solve the system.
From (1), $\lambda = \frac{1}{2x}$. From (2), $\lambda = \frac{2}{2y} = \frac{1}{y}$. (Note: $x,y \neq 0$)
Set them equal: $\frac{1}{2x} = \frac{1}{y} \implies y = 2x$.
Now substitute this into the constraint (3): $x^2 + (2x)^2 = 5 \implies 5x^2 = 5 \implies x^2=1$. So, $x=\pm 1$.
If $x=1$, then $y=2(1)=2$. Point is $(1,2)$.
If $x=-1$, then $y=2(-1)=-2$. Point is $(-1,-2)$.
Step 3: Evaluate f at the candidate points.
$f(1,2) = 1 + 2(2) = 5$.
$f(-1,-2) = -1 + 2(-2) = -5$.
The maximum value is 5, and the minimum value is -5.
The graph below shows the surface $z=f(x,y)=x+2y$ (the slanted plane), the constraint $x^2+y^2=5$ (the purple circle on the xy-plane), and level curves of $f$ (the parallel lines). The slider for 'b' controls the height of a level curve $x+2y=b$.
Your task: Move the slider for 'b' until the level curve (the grey line) is just tangent to the purple circle. You'll find two points of tangency. Notice that these tangency points correspond to the highest and lowest values of 'b' (the function's value) that still touch the constraint circle. This is the geometric heart of the Lagrange multiplier method.
Let's solve the box problem from our motivation: find the dimensions $(x,y,z)$ of the rectangular box with no lid and a surface area of 12 m$^2$ that has the largest volume.
Step 1: Set up the system.
Objective: $V(x,y,z)=xyz$. Constraint: $g(x,y,z)=xy+2xz+2yz=12$.
Gradients: $\nabla V = \langle yz, xz, xy \rangle$ and $\nabla g = \langle y+2z, x+2z, 2x+2y \rangle$.
The system is:
Step 2: Solve the system.
This looks intimidating, but a common trick is to multiply the equations to create a common term. Multiply (1) by $x$, (2) by $y$, and (3) by $z$ (assuming $x,y,z \neq 0$):
$xyz = \lambda(xy+2xz)$
$xyz = \lambda(xy+2yz)$
$xyz = \lambda(2xz+2yz)$
Setting the first two equal gives $\lambda(xy+2xz) = \lambda(xy+2yz) \implies 2xz = 2yz$. Since $z \neq 0$, this means $\boldsymbol{x=y}$.
Now substitute $x=y$ into equation (3): $x^2 = \lambda(2x+2x) = \lambda(4x) \implies x=4\lambda \implies \lambda = x/4$.
Substitute $x=y$ and $\lambda=x/4$ into equation (1): $xz = (x/4)(x+2z) \implies 4z = x+2z \implies 2z=x \implies \boldsymbol{z=x/2}$.
We now have all dimensions in terms of $x$: the base is square ($y=x$) and the height is half the base length ($z=x/2$).
Step 3: Use the constraint to find the final values.
Substitute $y=x$ and $z=x/2$ into the constraint (4):
$x(x) + 2x(x/2) + 2x(x/2) = 12 \implies x^2+x^2+x^2=12 \implies 3x^2=12 \implies x^2=4$.
So $x=2$ (since length must be positive). This means $y=2$ and $z=1$. The dimensions for the maximum volume are 2m x 2m x 1m.
Find the maximum and minimum values of $f(x,y) = -(x^2 + 2y^2) + 4.2$ subject to the constraint $g(x,y) = x^2+y^2=1$.
Step 1: Set up the system.
The function is an upside-down elliptic paraboloid (an oval hill steeper in the y-direction). The constraint is the unit circle. We are finding the highest and lowest points on a circular "roller coaster" track on this hill.
Gradients: $\nabla f = \langle -2x, -4y \rangle$ and $\nabla g = \langle 2x, 2y \rangle$.
The system $\nabla f = \lambda \nabla g$ is:
Step 2: Solve the system.
From equation (1), we can write $2x + 2x\lambda = 0 \implies 2x(1+\lambda)=0$. This gives two possibilities: $x=0$ or $\lambda=-1$.
Case 1: $x=0$.
Substitute $x=0$ into the constraint (3): $0^2+y^2=1 \implies y=\pm 1$. This gives us two candidate points: $(0,1)$ and $(0,-1)$.
Case 2: $\lambda=-1$.
Substitute $\lambda=-1$ into equation (2): $-4y = (-1)(2y) \implies -4y=-2y \implies -2y=0 \implies y=0$. Now substitute $y=0$ into the constraint (3): $x^2+0^2=1 \implies x=\pm 1$. This gives two more candidate points: $(1,0)$ and $(-1,0)$.
Step 3: Evaluate f at the candidate points.
We have four points to check:
The maximum value is 3.2, and the minimum value is 2.2.
The graph below shows the "roller coaster" path (in blue) on the surface. The level curves of the surface are ellipses. The constraint is the red circle in the xy-plane. Tangency occurs where a level curve just touches the constraint circle.
Your task:
Today we introduced a powerful new technique for solving constrained optimization problems. The method of Lagrange multipliers allows us to tackle problems where the substitution method is difficult or impossible. The core of this method is a beautiful geometric insight: at an extreme point, the gradient of the objective function must be parallel to the gradient of the constraint function. This translates to the elegant vector equation $\nabla f = \lambda \nabla g$, which is the key to finding the candidates for our constrained maximum and minimum values.
Set up the system of Lagrange multiplier equations for finding the extreme values of $f(x,y)=x^2y$ subject to the constraint $x^2+2y^2=6$. Do not solve.
Find the absolute maximum and minimum values of $f(x,y) = x^2+y^2-x-y+1$ on the disk $x^2+y^2 \le 1$. (This problem requires methods from both 14.7 and 14.8!)