In example 2, why do we put a hat on u? Is it because it is a unit vector, or because it is the vector that we are looking for?

It is because it is a unit vector. Unit vectors will typically have a hat on them.

Hi everyone, I hope you all are well. When Grant writes that "therefore u-hat is proportional to vector v!" in example two, is the exclamation point representing a factorial symbol or just something for "wow" exclamation? Thanks for your help.

Just an exclamation. I do not know how factorial would work for vectors.

Main content

Multivariable calculus

Course: Multivariable calculus > Unit 3

Lesson 6: Constrained optimization (articles)

Lagrange multipliers, examples

Google Classroom

Examples of the Lagrangian and Lagrange multiplier technique in action.

Background

Lagrange multiplier technique, quick recap

When you want to maximize (or minimize) a multivariable function start color #0c7f99, f, left parenthesis, x, comma, y, comma, dots, right parenthesis, end color #0c7f99 subject to the constraint that another multivariable function equals a constant, start color #bc2612, g, left parenthesis, x, comma, y, comma, dots, right parenthesis, equals, c, end color #bc2612, follow these steps:

Step 1: Introduce a new variable start color #0d923f, lambda, end color #0d923f, and define a new function L as follows:
L, left parenthesis, x, comma, y, comma, dots, comma, start color #0d923f, lambda, end color #0d923f, right parenthesis, equals, start color #0c7f99, f, left parenthesis, x, comma, y, comma, dots, right parenthesis, end color #0c7f99, minus, start color #0d923f, lambda, end color #0d923f, left parenthesis, start color #bc2612, g, left parenthesis, x, comma, y, comma, dots, right parenthesis, minus, c, end color #bc2612, right parenthesis
This function L is called the "Lagrangian", and the new variable start color #0d923f, lambda, end color #0d923f is referred to as a "Lagrange multiplier"
Step 2: Set the gradient of L equal to the zero vector.
del, L, left parenthesis, x, comma, y, comma, dots, comma, start color #0d923f, lambda, end color #0d923f, right parenthesis, equals, start bold text, 0, end bold text, left arrow, start color gray, start text, Z, e, r, o, space, v, e, c, t, o, r, end text, end color gray
In other words, find the critical points of L.
Step 3: Consider each solution, which will look something like left parenthesis, x, start subscript, 0, end subscript, comma, y, start subscript, 0, end subscript, comma, dots, comma, start color #0d923f, lambda, end color #0d923f, start subscript, 0, end subscript, right parenthesis. Plug each one into f. Or rather, first remove the start color #0d923f, lambda, end color #0d923f, start subscript, 0, end subscript component, then plug it into f, since f does not have start color #0d923f, lambda, end color #0d923f as an input. Whichever one gives the greatest (or smallest) value is the maximum (or minimum) point your are seeking.

Example 1: Budgetary constraints

Problem

Suppose you are running a factory, producing some sort of widget that requires steel as a raw material. Your costs are predominantly human labor, which is dollar sign, 20 per hour for your workers, and the steel itself, which runs for dollar sign, 170 per ton. Suppose your revenue R is loosely modeled by the following equation:

R, left parenthesis, h, comma, s, right parenthesis, equals, 200, h, start superscript, 2, slash, 3, end superscript, s, start superscript, 1, slash, 3, end superscript

h represents hours of labor
s represents tons of steel

If your budget is dollar sign, 20, comma, 000, what is the maximum possible revenue?

Solution

The dollar sign, 20 per hour labor costs and dollar sign, 170 per ton steel costs tell us that the total cost of production, in terms of h and s, is

\begin{aligned} \quad 20h + 170s \end{aligned}

Therefore the budget of dollar sign, 20, comma, 000 can be translated to the constraint

\begin{aligned} \quad \redE{20h + 170s = 20{,}000} \end{aligned}

Before we dive into the computation, you can get a feel for this problem using the following interactive diagram. You can see which values of left parenthesis, h, comma, s, right parenthesis yield a given revenue (blue curve) and which values satisfy the constraint (red line).

Since we need to maximize a function start color #0c7f99, R, left parenthesis, h, comma, s, right parenthesis, end color #0c7f99, subject to a constraint, start color #bc2612, 20, h, plus, 170, s, equals, 20, comma, 000, end color #bc2612, we begin by writing the Lagrangian function for this setup:

L, left parenthesis, h, comma, s, comma, lambda, right parenthesis, equals, start color #0c7f99, 200, h, start superscript, 2, slash, 3, end superscript, s, start superscript, 1, slash, 3, end superscript, end color #0c7f99, minus, lambda, left parenthesis, start color #bc2612, 20, h, plus, 170, s, minus, 20, comma, 000, end color #bc2612, right parenthesis

Next, set the gradient del, L equal to the start bold text, 0, end bold text vector. This is the same as setting each partial derivative equal to 0. First, we handle the partial derivative with respect to start color #0c7f99, h, end color #0c7f99.

\begin{aligned} \quad 0 &= \dfrac{\partial \mathcal{L}}{\partial \blueE{h}} \\ \\ 0 &= \dfrac{\partial}{\partial \blueE{h}}(200\blueE{h}^{2/3}s^{1/3}-\lambda(20\blueE{h}+170s-20{,}000)) \\ \\ 0 &= 200 \cdot \dfrac{2}{3}\blueE{h}^{-1/3}s^{1/3} - 20\lambda \\ \end{aligned}

Next, we handle the partial derivative with respect to start color #0d923f, s, end color #0d923f.

\begin{aligned} \quad 0 &= \dfrac{\partial \mathcal{L}}{\partial \greenE{s}} \\ \\ 0 &= \dfrac{\partial}{\partial \greenE{s}}(200h^{2/3}\greenE{s}^{1/3}-\lambda(20h+170\greenE{s}-20{,}000)) \\ \\ 0 &= 200 \cdot \dfrac{1}{3}h^{2/3}\greenE{s}^{-2/3} - 170\lambda \end{aligned}

Finally we set the partial derivative with respect to start color #a75a05, lambda, end color #a75a05 equal to 0, which as always is just the same thing as the constraint. In practice, you can of course just write the constraint itself, but I'll write out the partial derivative here just to make things clear.

\begin{aligned} \quad 0 &= \dfrac{\partial \mathcal{L}}{\partial \goldE{\lambda}} \\ \\ 0 &= \dfrac{\partial}{\partial \goldE{\lambda}}(200h^{2/3}s^{1/3}-\goldE{\lambda}(20h+170s-20{,}000)) \\ \\ 0 &= -20h-170s+20{,}000 \\ \\ 20h &+ 170s = 20{,}000 \end{aligned}

Putting it together, the system of equations we need to solve is

\begin{aligned} \quad 0 &= 200 \cdot \dfrac{2}{3}{h}^{-1/3}s^{1/3} - 20\lambda \\ \\ 0 &= 200 \cdot \dfrac{1}{3}h^{2/3}{s}^{-2/3} - 170\lambda \\ \\ 20h &+ 170s = 20{,}000 \end{aligned}

In practice, you should almost always use a computer once you get to a system of equations like this. Especially because the equation will likely be more complicated than these in real applications. Once you do, you'll find that the answer is

\begin{aligned} \quad h &= \dfrac{2{,}000}{3} \approx 666.667 \\ \\ s &= \dfrac{2{,}000}{51} \approx 39.2157 \\ \\ \lambda &= \sqrt[3]{\dfrac{8{,}000}{459}} \approx 2.593 \\ \\ \end{aligned}

This means you should employ about 667 hours of labor, and purchase 39 tons of steel, which will give a maximum revenue of

\begin{aligned} \quad R(667, 39) = 200(667)^{2/3}(39)^{1/3} \approx \boxed{\$51{,}777} \end{aligned}

The interpretation of this constant lambda, equals, 2, point, 593 is left to the next article

Example 2: Maximizing dot product

Problem: Let the three-dimensional vector start bold text, v, end bold text, with, vector, on top be defined as follows.

\begin{aligned} \quad \vec{\textbf{v}} = \left[ \begin{array}{c} 2 \\ 3 \\ 1 \end{array} \right] \end{aligned}

Consider every possible unit vector start bold text, u, end bold text, with, hat, on top in three-dimensional space. For which one is the dot product start bold text, u, end bold text, with, hat, on top, dot, start bold text, v, end bold text, with, vector, on top the greatest?

The diagram below is two-dimensional, but not much changes in the intuition as we move to three dimensions.

If you are fluent with dot products, you may already know the answer. It's one of those mathematical facts worth remembering. If you don't know the answer, all the better! Because we will now find and prove the result using the Lagrange multiplier method.

Solution:

First, we need to spell out how exactly this is a constrained optimization problem. Write the coordinates of our unit vectors as x, y and z:

\begin{aligned} \quad \hat{\textbf{u}} = \left[ \begin{array}{c} x \\ y \\ z \end{array} \right] \end{aligned}

The fact that start bold text, u, end bold text, with, hat, on top is a unit vector means its magnitude is 1:

\begin{aligned} \quad ||\hat{\textbf{u}}|| = \sqrt{x^2 + y^2 + z^2} &= 1 \\ &\Downarrow \\ \redE{x^2 + y^2 + z^2} &\redE{= 1} \end{aligned}

This is our constraint.

Maximizing start bold text, u, end bold text, with, hat, on top, dot, start bold text, v, end bold text, with, vector, on top means maximizing the following quantity:

$\left[ \begin{array}{c} x \\ y \\ z \end{array} \right] \cdot \left[ \begin{array}{c} 2 \\ 3 \\ 1 \end{array} \right] = \blueE{2x + 3y + z}$

The Lagrangian, with respect to this function and the constraint above, is

L, left parenthesis, x, comma, y, comma, z, comma, lambda, right parenthesis, equals, 2, x, plus, 3, y, plus, z, minus, lambda, left parenthesis, x, squared, plus, y, squared, plus, z, squared, minus, 1, right parenthesis, point

We now solve for del, L, equals, start bold text, 0, end bold text by setting each partial derivative of this expression equal to 0.

$\begin{aligned} \dfrac{\partial}{\partial \greenE{x}}(2\greenE{x} + 3y + z - \lambda(\greenE{x}^2 + y^2 + z^2 - 1)) &= 2 - \lambda 2\greenE{x} = 0 \\ \dfrac{\partial}{\partial \goldE{y}}(2x + 3\goldE{y} + z - \lambda(x^2 + \goldE{y}^2 + z^2 - 1)) &= 3 - \lambda 2\goldE{y} = 0 \\ \dfrac{\partial}{\partial \maroonE{z}}(2x + 3y + \maroonE{z} - \lambda(x^2 + y^2 + \maroonE{z}^2 - 1)) &= 1 - \lambda 2\maroonE{z} = 0 \\ \end{aligned}$

Remember, setting the partial derivative with respect to lambda equal to 0 just restates the constraint.

\begin{aligned} \dfrac{\partial}{\partial \redE{\lambda}}(2x + 3y + z - \redE{\lambda}(x^2 + y^2 + z^2 - 1)) &= -x^2-y^2-z^2 + 1 = 0 \\ \end{aligned}

Solving for start color #0d923f, x, end color #0d923f, start color #a75a05, y, end color #a75a05 and start color #9e034e, z, end color #9e034e in the first three equations above, we get

\begin{aligned} \quad \greenE{x} &= 2\cdot \dfrac{1}{2\lambda} \\ \goldE{y} &= 3\cdot \dfrac{1}{2\lambda} \\ \maroonE{z} &= 1\cdot \dfrac{1}{2\lambda} \end{aligned}

Ah, what beautiful symmetry. Each of these expressions has the same start fraction, 1, divided by, 2, lambda, end fraction factor, and the coefficients 2, 3 and 1 match up with the coordinates of start bold text, v, end bold text, with, vector, on top. Being good math students as we are, we won't let good symmetry go to waste. In this case, combining the three equations above into a single vector equation, we can relate start bold text, u, end bold text, with, hat, on top and start bold text, v, end bold text, with, vector, on top as follows:

\begin{aligned} \quad \hat{\textbf{u}} = \left[ \begin{array}{c} x \\ y \\ z \end{array} \right] = \dfrac{1}{2\lambda} \left[ \begin{array}{c} 2 \\ 3 \\ 1 \end{array} \right] = \dfrac{1}{2\lambda}\vec{\textbf{v}} \end{aligned}

Therefore start bold text, u, end bold text, with, hat, on top is proportional to start bold text, v, end bold text, with, vector, on top! Geometrically, this means start bold text, u, end bold text, with, hat, on top points in the same direction as start bold text, v, end bold text, with, vector, on top. There are two unit vectors proportional start bold text, v, end bold text, with, vector, on top,

One which points in the same direction, this is the vector that start color #0d923f, start text, m, a, x, i, m, i, z, e, s, end text, end color #0d923f start bold text, u, end bold text, with, hat, on top, dot, start bold text, v, end bold text, with, vector, on top.
One which points in the opposite direction. This one start color #bc2612, start text, m, i, n, i, m, i, z, e, s, end text, end color #bc2612 start bold text, u, end bold text, with, hat, on top, dot, start bold text, v, end bold text, with, vector, on top.

We can write these two unit vectors by normalizing start bold text, v, end bold text, with, vector, on top, which just means dividing start bold text, v, end bold text, with, vector, on top by its magnitude:

\begin{aligned} \quad \greenE{\hat{\textbf{u}}_{\text{max}}} &= \dfrac{\vec{\textbf{v}}}{||\vec{\textbf{v}}||} \\ \\ \redE{\hat{\textbf{u}}_{\text{min}}} &= -\dfrac{\vec{\textbf{v}}}{||\vec{\textbf{v}}||} \end{aligned}

The magnitude vertical bar, vertical bar, start bold text, v, end bold text, with, vector, on top, vertical bar, vertical bar is square root of, 2, squared, plus, 3, squared, plus, 1, squared, end square root, equals, square root of, 14, end square root, so we can write the maximizing unit vector start color #0d923f, start bold text, u, end bold text, with, hat, on top, start subscript, start text, m, a, x, end text, end subscript, end color #0d923f explicitly as like this:

$\greenE{\hat{\textbf{u}}_{\text{max}}} = \left[ \begin{array}{c} 2 / \sqrt{14} \\ 3 / \sqrt{14} \\ 1 / \sqrt{14} \end{array} \right]$

Just skip the Lagrangian

If you read the last article, you'll recall that the whole point of the Lagrangian L is that setting del, L, equals, 0 encodes the two properties a constrained maximum must satisfy:

Gradient alignment between the target function and the constraint function,
$\begin{aligned} \quad \nabla \blueE{f(x, y)} &= \lambda \nabla \redE{g(x, y)} \\ \end{aligned}$
The constraint itself,
$\begin{aligned} \quad \redE{g(x, y) = c} \end{aligned}$

When working through examples, you might wonder why we bother writing out the Lagrangian at all. Wouldn't it be easier to just start with these two equations rather than re-establishing them from del, L, equals, 0 every time? The short answer is yes, it would be easier. If you find yourself solving a constrained optimization problem by hand, and you remember the idea of gradient alignment, feel free to go for it without worrying about the Lagrangian.

In practice, it's often a computer solving these problems, not a human. Given that there are many highly optimized programs for finding when the gradient of a given function is 0, it's both clean and useful to encapsulate our problem into the equation del, L, equals, 0.

Furthermore, the Lagrangian itself, as well as several functions deriving from it, arise frequently in the theoretical study of optimization. In this light, reasoning about the single object L rather than multiple conditions makes it easier to see the connection between high-level ideas. Not to mention, it's quicker to write down on a blackboard.

In either case, whatever your future relationship with constrained optimization might be, it is good to be able to think about the Lagrangian itself and what it does. The examples above illustrate how it works, and hopefully help to drive home the point that del, L, equals, 0 encapsulates both del, f, equals, lambda, del, g and g, left parenthesis, x, comma, y, right parenthesis, equals, c in a single equation.

Want to join the conversation?

Sort by:

clara.vdw
Posted 7 years ago. Direct link to clara.vdw's post “In example 2, why do we p...”
In example 2, why do we put a hat on u? Is it because it is a unit vector, or because it is the vector that we are looking for?
Button navigates to signup pageButton navigates to signup page
(10 votes)
Answer
- u.yu16
  Posted 7 years ago. Direct link to u.yu16's post “It is because it is a uni...”
  It is because it is a unit vector. Unit vectors will typically have a hat on them.
  Button navigates to signup page
  (13 votes)
Kathy M
Posted 3 years ago. Direct link to Kathy M's post “I have seen some question...”
I have seen some questions where the constraint is added in the Lagrangian, unlike here where it is subtracted. e.g. Lagrangian = f(x) + λg(x)
How does one decide whether to add or subtract?
Thanks!
Button navigates to signup pageButton navigates to signup page
(6 votes)
Answer
LazarAndrei260
Posted 3 years ago. Direct link to LazarAndrei260's post “Hello, I have been thinki...”
Hello, I have been thinking about this and can't really understand what is happening. So suppose I want to maximize
the function:

f(x,y,z) = 5xy + 8xz + 3yz, with the constraint 2xyz = 1920.

After doing the multiplier method, I only get one solution. Does that mean that the function does not have a maximum or a minimum? Or how can I tell if the solution I get represents or not a maximum?
Button navigates to signup pageButton navigates to signup page
(5 votes)
Answer
- zjleon2010
  Posted 2 years ago. Direct link to zjleon2010's post “the determinant of hessia...”
  the determinant of hessian evaluated at a point indicates the concavity of f at that point
  Button navigates to signup page
  (1 vote)
bgao20
Posted 3 years ago. Direct link to bgao20's post “Hi everyone, I hope you a...”
Hi everyone, I hope you all are well. When Grant writes that "therefore u-hat is proportional to vector v!" in example two, is the exclamation point representing a factorial symbol or just something for "wow" exclamation? Thanks for your help.
Button navigates to signup pageButton navigates to signup page
(4 votes)
Answer
- loumast17
  Posted 3 years ago. Direct link to loumast17's post “Just an exclamation. I d...”
  Just an exclamation. I do not know how factorial would work for vectors.
  Comment on loumast17's post “Just an exclamation. I d...”
  (3 votes)
hamadmo77
Posted 6 years ago. Direct link to hamadmo77's post “Instead of constraining o...”
Instead of constraining optimization to a curve on x-y plane, is there which a method to constrain the optimization to a region/area on the x-y plane. Like the region
x^2+y^2<=2 which r all the points in the unit circle including the boundary.
Button navigates to signup pageComment on hamadmo77's post “Instead of constraining o...”
(3 votes)
Answer
nikostogas
Posted 6 years ago. Direct link to nikostogas's post “Hello and really thank yo...”
Hello and really thank you for your amazing site. Can you please explain me why we dont use the whole Lagrange but only the first part? Why we dont use the 2nd derivatives
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
Amos Didunyk
Posted 5 years ago. Direct link to Amos Didunyk's post “In the step 3 of the reca...”
In the step 3 of the recap, how can we tell we don't have a saddlepoint? The fact that you don't mention it makes me think that such a possibility doesn't exist. But it does right?
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
luluping06023
Posted 4 years ago. Direct link to luluping06023's post “how to solve ∇L=0 when th...”
how to solve ∇L=0 when they are not linear equations?
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Dinoman44
  Posted 6 months ago. Direct link to Dinoman44's post “When you have non-linear ...”
  When you have non-linear equations for your variables, rather than compute the solutions manually you can use computer to do it. Apps like Mathematica, GeoGebra and Desmos allow you to graph the equations you want and find the solutions. I myself use a Graphic Display Calculator(TI-NSpire CX 2) for this.
  Button navigates to signup page
  (1 vote)
TornikeO
Posted 7 years ago. Direct link to TornikeO's post “what shuld we do if we ha...”
what shuld we do if we have constraints as well as boundaries and we need a local extrima?
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Naman
  Posted a month ago. Direct link to Naman's post “It should pretty much be ...”
  It should pretty much be the same thing, except you only look at the solutions that are within the boundary, it's possible that the highest or lowest value can be at the boundary, but that would be a global extreme, not a local one.
  Button navigates to signup page
  (1 vote)
Elite Dragon
Posted 5 years ago. Direct link to Elite Dragon's post “Is there a similar method...”
Is there a similar method of using Lagrange multipliers to solve constrained optimization problems for integer solutions?
Button navigates to signup pageButton navigates to signup page
(1 vote)
Answer