Why is that the maximum or minimum value for f lies at the point where the contour lines of f and g are tangent? How did you prove it more rigorously?

I won't prove this, but imagine them not being tangent, that would mean the function's contours cross the constraint at some point. If they cross, however, it means we can always shift the contour to a higher (or lower, depending of we're maximising or minimising) level and still be crossing the constraint. The time when you cannot shift it any higher is precisely at the level where they're only touching (tangent), not crossing. This is all assuming we have well-behaved functions/constraints

what is the geometrical interpretation for lagrange multipliers involving two constraints??

With two constraints, you'll end up with two Lagrange multipliers (one associated with each constraint). The magnitude of each Lagrange multiplier tells you how much improvement you could get by relaxing the _corresponding_ constraint.

Use the method of Lagrange Multipliers to determine the maximum and minimum of f(x,y,z) = x + y + z subject to the two conditions g(x,y,z) = x2 + y2 − 2 = 0 and h(x, y, z) = x + z − 1 = 0

We haven't learned about multiple constraints yet. The Lagrangian Function for multiple constraints is ℒ(x1,x2,...,xn,λ1,λ2,...,λM) = f(x1,x2,...,xn) - Σ λkgk(x1,x2,...,xn), where f(*x*) is the function to be maximized and g1, g2, ..., gM are the constraints (the Σ sum is from k = 1 to M). Note that here we use gk(*x*) = 0 instead of gk(*x*) = c. We take the three gradients and get ∇f = *i* + *j* + *k*, ∇g = 2x *i* + 2y *j*, ∇h = *i* + *k*. ∇f = λ1∇g + λ2∇h, and g = h = 0, so we get: 1 = 2xλ1 + λ2, 1 = 2yλ1, 1 = λ2, x^2 + y^2 = 2, and x + z - 1 = 0. We see that λ2 = 1, so 2xλ1 = 0. because λ1 = 1/2y, x/y = 0, so x must be 0. Then y = +-√2, and λ1 = +-√2/4, and z = 1. So we get two solutions: (0,√2,1) and (0,-√2,1). I think you can figure out which is the maximum. If you can graph this, try it, and you'll see how stupid our computations were. After all this work the answers are on the y-z plane!

Hi, thanks for the article. Question ... how does a parallel gradient tell us that the 2 contour lines are tangent to each other? Parallel gradients should be possible even if the 2 functions don't touch right? Thank you.

Actually, parallel gradients tell us that the contour lines are parallel. Contour lines are tangent when they are both touching and parallel at the place they're touching. Of course, we had to answer the question "Where are they parallel?" The direction of gradients and contour lines are different at every point. Therefore, the gradients of the two functions need to be parallel _at the same point_. ∇f(0,1) being parallel to ∇g(1,0) wouldn't be much use. ∇f(0,1) must be parallel to ∇g(0,1). Of course, this says nothing about where the functions f and g are, only what their gradients (slants or tilts) are. In fact, it doesn't matter whether f(0,1) = g(0,1). After all, we are restricting g to g(x,y) = c, and c may be arbitrary. If the restraint is x + y = 4, we can let g(x,y) = x + y and c = 4, or we can let g(x,y) = x + y - 4 and c = 0, or g(x,y) = x + y + 100 and c = 104. So how "high" g is doesn't matter at all, it's just its gradient (slant or tilt) that we need. Of course, sometimes you might be required to let c = 0, but there's no difference. The "height" of g doesn't matter.

And when we have inequality constrains? How we can formulate the Lagrangian equation? For example x² + y² <= 1. I have doubts about how we can solve this.

There are techniques, such as Kuhn-Tucker conditions, which were created specifically to answer these questions. In short, if the optimal point is _interior_ to the constraints you can solve the problem without constraints. The resulting solution will satisfy the constraints regardless of the fact that you did not enforce them. If the optimal point is _exterior_ to the constraints, then you can solve the problem with the constraint written as an equality constraint.

Main content

Course: Multivariable calculus > Unit 3

Lesson 6: Constrained optimization (articles)

Lagrange multipliers, introduction

Q: And when we have inequality constrains? How we can formulate the Lagrangian equation? For example x² + y² <= 1. I have doubts about how we can solve this.

There are techniques, such as Kuhn-Tucker conditions, which were created specifically to answer these questions. In short, if the optimal point is _interior_ to the constraints you can solve the problem without constraints. The resulting solution will satisfy the constraints regardless of the fact that you did not enforce them. If the optimal point is _exterior_ to the constraints, then you can solve the problem with the constraint written as an equality constraint.

Google Classroom

The "Lagrange multipliers" technique is a way to solve constrained optimization problems. Super useful!

Background

What we're building to:

The Lagrange multiplier technique lets you find the maximum or minimum of a multivariable function $f (x, y, \dots)$ ‍ when there is some constraint on the input values you are allowed to use.
This technique only applies to constraints that look something like this:
$g (x, y, \dots) = c$ ‍
Here, $g$ ‍ is another multivariable function with the same input space as $f$ ‍, and $c$ ‍ is some constant.
The core idea is to look for points where the contour lines of $f$ ‍ and $g$ ‍ are tangent to each other.
This is the same as finding points where the gradient vectors of $f$ ‍ and $g$ ‍ are parallel to each other.
The entire process can be boiled down into setting the gradient of a certain function, called the Lagrangian, equal to the zero vector.
Step 1: Introduce a new variable $λ$ ‍, and define a new function $L$ ‍ as follows:
$L (x, y, \dots, λ) = f (x, y, \dots) - λ (g (x, y, \dots) - c)$ ‍
This function $L$ ‍ is called the "Lagrangian", and the new variable $λ$ ‍ is referred to as a "Lagrange multiplier"
Step 2: Set the gradient of $L$ ‍ equal to the zero vector.
$\nabla L (x, y, \dots, λ) = 0 \leftarrow Zero vector$ ‍
In other words, find the critical points of $L$ ‍.
Step 3: Consider each solution, which will look something like $(x_{0}, y_{0}, \dots, λ_{0})$ ‍. Plug each one into $f$ ‍. Or rather, first remove the $λ_{0}$ ‍ component, then plug it into $f$ ‍, since $f$ ‍ does not have $λ$ ‍ as an input. Whichever one gives the greatest (or smallest) value is the maximum (or minimum) point your are seeking.

Motivating example

Suppose you want to maximize this function:

$f (x, y) = 2 x + y$ ‍

But let's also say you limited yourself to inputs

(x, y)

which satisfy the following equation:

$x^{2} + y^{2} = 1$ ‍

In other words, for which point

(x, y)

on the

unit circle

is the value

2 x + y

biggest?

This is what's known as a constrained optimization problem. The restriction to points where

x^{2} + y^{2} = 1

is called a "constraint", and

f (x, y) = 2 x + y

is the function that needs to be optimized.

Here's one way to visualize this: First draw the graph of

f (x, y)

, which looks like a slanted plane since

f

is linear. Next, project the circle

x^{2} + y^{2} = 1

from the

x y

-plane vertically onto the graph of

f

. The maximum we are seeking corresponds with the highest point of this projected circle on the graph.

Khan Academy video wrapper

See video transcript

More general form

In general, constrained optimization problems involve maximizing/minimizing a multivariable function whose input has any number of dimensions:

$f (x, y, z, \dots)$ ‍

Its output will always be one-dimensional, though, since there's not a clear notion of "maximum" with vector-valued outputs.

The type of constraints that the Lagrange multiplier technique applies to must take the form of some other multivariable function

g (x, y, z, \dots)

being set equal to a constant

c

$g (x, y, z, \dots) = c$ ‍

Since this is meant to be a constraint on the input of

f

, the number of dimensions in the input of

g

is the same as that of

f

. For example, the example outlined above fits this general form as follows:

$f (x, y) = 2 x + y$ ‍

$g (x, y) = x^{2} + y^{2}$ ‍

$c = 1$ ‍

Using contour maps

Reasoning about this problem becomes easier if we visualize

f

not with a graph, but with its contour lines.

As a reminder, a contour line of

f (x, y)

is the set of all points where

f (x, y) = k

for some constant

k

. The following interactive tool shows how this line (drawn in blue) changes as the constant

k

changes. The circle

g (x, y) = 1

is also shown (in red). Try to make

k

as big/small as possible while still allowing contour line of

f

to intersect the circle.

Concept check: What does it mean if for a particular value of

k

, the blue line representing

f (x, y) = k

does not intersect the red circle representing

g (x, y) = 1

Notice, the circle where

g (x, y) = 1

can be thought of as a particular contour line of the function

g

. So with that, here's the clever way to think about constrained optimization problems:

Key observation: The maximum and minimum values of

f

, subject to the constraint

g (x, y) = 1

, correspond with contour lines of

f

that are tangent to the contour representing

g (x, y) = 1

f

were a different function, its contours might not always be straight lines. This is unique to our example since

f

is linear. For example, take a look at this function:

$f (x, y) = 2 x^{2} + \sqrt{5 y}$ ‍,

Its contour lines look like this:

That said, the key observation still holds, and is worth repeating: When

k

is a maximum or minimum of

f

subject to the constraint, the contour line for

f (x, y) = k

will be tangent to contour representing

g (x, y) = 1

Where the gradient comes into play

How do you put the idea of two contour lines being tangent into a formula you can solve?

To answer this, we turn to our loyal friend the gradient. There are many ways to interpret

\nabla f

: The direction of steepest ascent, a tool for computing directional derivatives, etc. But for our purposes here, the property we care about is that the gradient of $f$ ‍ evaluated at a point $(x_{0}, y_{0})$ ‍ always gives a vector perpendicular to the contour line passing through that point.

This means when the contour lines of two functions

f

and

g

are tangent, their gradient vectors are parallel. Here's what that might look like for arbitrary functions

f

and

g

The fact that contour lines are tangent tells us nothing about the magnitude of each of these gradient vectors, but that's okay. When two vectors point in the same direction, it means we can multiply one by some constant to get the other. Specifically, let

(x_{0}, y_{0})

represent a particular point where the contour lines of

f

and

g

are tangent (writing

x_{0}

and

y_{0}

with a

0

subscripts just indicates that we are considering constant values, and hence a specific point). Since this tangency means their gradient vectors align, here's what you might write down:

$\begin{array}{r} \nabla f (x_{0}, y_{0}) = λ_{0} \nabla g (x_{0}, y_{0}) \end{array}$ ‍

Here,

λ_{0}

represents some constant. Some authors use a negative constant,

- λ_{0}

, but I personally prefer a positive constant, as it gives a cleaner interpretation of

λ_{0}

down the road.

Let's see what this looks like in our example where

f (x, y) = 2 x + y

and

g (x, y) = x^{2} + y^{2}

. The gradient of

f

$\begin{array}{r} \nabla f (x, y) = [\begin{array}{c} \frac{\partial}{\partial x} (2 x + y) \\ \frac{\partial}{\partial y} (2 x + y) \end{array}] = [\begin{array}{c} 2 \\ 1 \end{array}] \end{array}$ ‍

and the gradient of

g

$\begin{array}{r} \nabla g (x, y) = [\begin{array}{c} \frac{\partial}{\partial x} (x^{2} + y^{2} - 1) \\ \frac{\partial}{\partial y} (x^{2} + y^{2} - 1) \end{array}] = [\begin{array}{c} 2 x \\ 2 y \end{array}] \end{array}$ ‍

Therefore, the tangency condition ends up looking like this:

$\begin{array}{r} [\begin{array}{c} 2 \\ 1 \end{array}] = λ_{0} [\begin{array}{c} 2 x_{0} \\ 2 y_{0} \end{array}] \end{array}$ ‍

Solving the problem in the specific case

To sum up where we are so far, we are looking for input points

(x_{0}, y_{0})

with the following properties:

$g (x_{0}, y_{0}) = 1$ ‍, which for our example means
$x_{0}^{2} + y_{0}^{2} = 1$ ‍
$\nabla f (x_{0}, y_{0}) = λ_{0} \nabla g (x_{0}, y_{0})$ ‍ for some constant $λ_{0}$ ‍, which for our example means
$\begin{aligned} 2 & = 2 λ_{0} x_{0} \\ 1 & = 2 λ_{0} y_{0} \end{aligned}$ ‍

There are

3

equations and

3

unknowns, so this is a perfectly solvable situation.

The approach will be to first solve for

λ_{0}

, then use the solution to find

x_{0}

and

y_{0}

Using the last two equations above, write

x_{0}

and

y_{0}

in terms of

λ_{0}

$\begin{aligned} 2 & = 2 λ_{0} x_{0} \Rightarrow x_{0} = \frac{1}{λ_{0}} \\ 1 & = 2 λ_{0} y_{0} \Rightarrow y_{0} = \frac{1}{2 λ_{0}} \end{aligned}$ ‍

Now to bring in the third equation, plug these results into the equation

x_{0}^{2} + y_{0}^{2} = 1

\begin{aligned} {x_{0}}^{2} + {y_{0}}^{2} & = 1 \\ {(\frac{1}{λ_{0}})}^{2} + {(\frac{1}{2 λ_{0}})}^{2} & = 1 \\ \frac{1}{λ_{0}^{2}} + \frac{1}{4 λ_{0}^{2}} & = 1 \end{aligned}

To get

λ_{0}

out of the denominators, we multiply everything by

4 λ_{0}^{2}

and simplify.

\begin{aligned} 4 + 1 & = 4 λ_{0}^{2} \\ \frac{5}{4} & = λ_{0}^{2} \\ \pm \sqrt{\frac{5}{4}} & = λ_{0} \\ \frac{\pm \sqrt{5}}{2} & = λ_{0} \end{aligned}

Using the expressions for

x_{0}

and

y_{0}

in terms of

λ_{0}

that we found above, these two solutions correspond with the pairs

$\begin{aligned} (x_{0}, y_{0}) & = (\frac{1}{λ_{0}}, \frac{1}{2 λ_{0}}) \\ = (\frac{2}{\sqrt{5}}, \frac{1}{\sqrt{5}}) or (\frac{- 2}{\sqrt{5}}, \frac{- 1}{\sqrt{5}}) \end{aligned}$ ‍

We can see which of these is a maximum point and which is a minimum point by plugging these solutions into

f (x, y)

and seeing which is bigger.

\begin{aligned} f (\frac{2}{\sqrt{5}}, \frac{1}{\sqrt{5}}) & = 2 \frac{2}{\sqrt{5}} + \frac{1}{\sqrt{5}} \\ = \frac{5}{\sqrt{5}} \\ = \sqrt{5} \leftarrow Maximum \\ f (- \frac{2}{\sqrt{5}}, - \frac{1}{\sqrt{5}}) & = 2 \frac{- 2}{\sqrt{5}} + \frac{- 1}{\sqrt{5}} \\ = \frac{- 5}{\sqrt{5}} \\ = - \sqrt{5} \leftarrow Minimum \end{aligned}

The Lagrangian function

In the 1700's, our buddy Joseph Louis Lagrange studied constrained optimization problems of this kind, and he found a clever way to express all of our conditions into a single equation.

You can write these conditions generally by saying we are looking for constants

x_{0}

y_{0}

and

λ_{0}

that satisfy the following conditions:

The constraint:
$g (x_{0}, y_{0}) = c$ ‍
The tangency condition:
$\nabla f (x_{0}, y_{0}) = λ_{0} \nabla g (x_{0}, y_{0})$ ‍.
This can be broken into its components as follows:
$f_{x} (x_{0}, y_{0}) = λ_{0} g_{x} (x_{0}, y_{0})$ ‍
$f_{y} (x_{0}, y_{0}) = λ_{0} g_{y} (x_{0}, y_{0})$ ‍

Lagrange wrote down a special new function which takes in all the same input variables as

f

and

g

, along with the new kid in town

λ

, thought of now as a variable rather than a constant.

$L (x, y, λ) = f (x, y) - λ (g (x, y) - c)$ ‍

For example, consider our example above.

\begin{aligned} f (x, y) & = 2 x + y \\ g (x, y) & = x^{2} + y^{2} \\ c & = 1 \end{aligned}

Here's how this new function would look:

$L (x, y, λ) = 2 x + y - λ (x^{2} + y^{2} - 1) .$ ‍

Notice, the partial derivative of

L

with respect to

λ

- (g (x, y) - c)

\begin{aligned} L_{λ} (x, y, λ) & = \frac{\partial}{\partial λ} (f (x, y) - λ (g (x, y) - c) \\ = 0 - (g (x, y) - c) \end{aligned}

So we can translate the condition

g (x, y) = c

\begin{array}{r} L_{λ} (x, y, λ) = - g (x, y) + c = 0 \end{array}

What's more, look at what we get when we set one of the other partial derivatives equal to

0

\begin{aligned} L_{x} (x, y, λ) & = 0 \\ \frac{\partial}{\partial x} (f (x, y) - λ (g (x, y) - c)) & = 0 \\ f_{x} (x, y) - λ g_{x} (x, y) & = 0 \\ f_{x} (x, y) & = λ g_{x} (x, y) \end{aligned}

That just so happens to be another one of our conditions! Almost identically, the condition

L_{y} (x, y, λ) = 0

unravels to become

\begin{array}{r} f_{y} (x, y) = λ g_{y} (x, y) \end{array}

Together, these conditions are the same as saying.

\begin{array}{r} \nabla f (x, y) = λ \nabla g (x, y) \end{array}

Therefore, the three conditions we need to solve to find

x, y

and

λ

come down to the various partial derivatives of

L

being equal to

0

. This can be written extremely compactly by setting the gradient of

L

equal to the zero vector:

\begin{array}{r} \nabla L = 0 \end{array}

For example, using our specific functions from above, we see how this encodes the system of equations we need to solve:

\begin{array}{r} \nabla L = [\begin{array}{c} \frac{\partial}{\partial x} (2 x + y - λ (x^{2} + y^{2} - 1)) \\ \frac{\partial}{\partial y} (2 x + y - λ (x^{2} + y^{2} - 1)) \\ \frac{\partial}{\partial λ} (2 x + y - λ (x^{2} + y^{2} - 1)) \end{array}] = [\begin{array}{c} 2 - 2 λ x \\ 1 - 2 λ y \\ - x^{2} - y^{2} + 1 \end{array}] = [\begin{array}{c} 0 \\ 0 \\ 0 \end{array}] \end{array}

As a tribute to ol' Joey Lou, we call this function

L

the "Lagrangian", and the new variable

λ

that we introduce is called a "Lagrange multiplier". Imagine if someone added "-ian" the end of your last name and made it the name of a function everybody uses. Pretty sweet!

Warning: Some authors use a convention where the sign of

λ

is reversed:

\begin{array}{r} L (x, y, λ) = f (x, y) + λ (g (x, y) - c) \end{array}

This doesn't make any difference when it comes to solving the problem, but you should keep it in mind in case the course you are taking or the text you are reading follows this convention.

Side note: What if the constraint isn't so constraining?

There's a slight twist to this story, best illustrated with an example.

Suppose we needed to maximize the function

\begin{array}{r} f (x, y) = e^{- (x^{2} + y^{2})} \end{array}

Subject to the constraint

\begin{array}{r} g (x, y) = x - y = 0 \end{array}

The graph of

f (x, y) = e^{- (x^{2} + y^{2})}

is a "bell curve", with a round bump above the point

(x_{0}, y_{0}) = (0, 0)

, as pictured above.

This constraint can be pictured as a diagonal line in the

x y

-plane (shown in red).

What makes this problem a bit silly is that the (unconstrained) maximum point

(0, 0)

f

already satisfies the constraint

g (x, y) = 0

, since

\begin{array}{r} g (0, 0) = 0 + 0 = 0 \end{array}

It might feel like this makes things easier on us. After all, not having to worry about the constraint should be more straightforward than worrying about it, right? However, if you (or more realistically a computer) were solving a given constrained optimization problem, it's not like you would first find the unconstrained maximum, check if it fits the constraint, then turn to the Lagrange multiplier technique. You would just start with the Lagrange multiplier approach, since it's almost never the case that the unconstrained maximum is also the constrained maximum.

It turns out that the Lagrange multiplier technique still works when the constrained maximum is also an unconstrained maximum. The reason is a bit subtle since our contour-tangency argument above doesn't quite apply.

For example, let's play around with the contours representing

e^{- (x^{2} + y^{2})} = k

k

varies between

0.5

and

1

The contour representing

e^{- (x^{2} + y^{2})} = k

always crosses the line

x - y = 0

, and this contour shrinks to a point when

k

is as big as it can be. Previously, we said the contour lines would be tangent when

f

achieves its constrained maximum, but it feels weird to say a point is tangent to a line.

Why isn't this a problem then?

f

achieves an unconstrained local maximum at some point

(x_{0}, y_{0})

, its gradient at that point will be

0

\begin{array}{r} \nabla f (x_{0}, y_{0}) = 0 \end{array}

You can think of this as saying the tangent plane at a local maximum is flat.

In this case, the property

\begin{array}{r} \nabla f (x_{0}, y_{0}) = λ \nabla g (x_{0}, y_{0}) \end{array}

still holds, because we can set

λ = 0

A funny way to think about this is to say a point is "tangent" to all lines passing through it in the same sense that the zero vector is "proportional" to all other vectors.

So not only does the Lagrange multiplier technique cleverly consolidate many conditions into the one equation

\begin{array}{r} \nabla L = 0, \end{array}

it also automatically takes into account this special edge case!

Summary

When you want to maximize (or minimize) a multivariable function

f (x, y, \dots)

subject to the constraint that another multivariable function equals a constant,

g (x, y, \dots) = c

, follow these steps:

Step 1: Introduce a new variable $λ$ ‍, and define a new function $L$ ‍ as follows:
$L (x, y, \dots, λ) = f (x, y, \dots) - λ (g (x, y, \dots) - c)$ ‍
This function $L$ ‍ is called the "Lagrangian", and the new variable $λ$ ‍ is referred to as a "Lagrange multiplier"
Step 2: Set the gradient of $L$ ‍ equal to the zero vector.
$\nabla L (x, y, \dots, λ) = 0 \leftarrow Zero vector$ ‍
In other words, find the critical points of $L$ ‍.
Step 3: Consider each solution, which will look something like $(x_{0}, y_{0}, \dots, λ_{0})$ ‍. Plug each one into $f$ ‍. Or rather, first remove the $λ_{0}$ ‍ component, then plug it into $f$ ‍, since $f$ ‍ does not have $λ$ ‍ as an input. Whichever one gives the greatest (or smallest) value is the maximum (or minimum) point you are seeking.

Want to join the conversation?

Sort by:

alimuldal
Posted 8 years ago. Direct link to alimuldal's post “In the final "side note" ...”
In the final "side note" example graph, the equation for the red diagonal line should be x - y = 0, not x + y = 0
Button navigates to signup pageComment on alimuldal's post “In the final "side note" ...”
(38 votes)
Answer
Sam
Posted 8 years ago. Direct link to Sam's post “Why is that the maximum o...”
Why is that the maximum or minimum value for f lies at the point where the contour lines of f and g are tangent? How did you prove it more rigorously?
Button navigates to signup pageButton navigates to signup page
(8 votes)
Answer
- Dahn Jahn
  Posted 8 years ago. Direct link to Dahn Jahn's post “I won't prove this, but i...”
  I won't prove this, but imagine them not being tangent, that would mean the function's contours cross the constraint at some point. If they cross, however, it means we can always shift the contour to a higher (or lower, depending of we're maximising or minimising) level and still be crossing the constraint. The time when you cannot shift it any higher is precisely at the level where they're only touching (tangent), not crossing. This is all assuming we have well-behaved functions/constraints
  Comment on Dahn Jahn's post “I won't prove this, but i...”
  (17 votes)
akshita sharma
Posted 6 years ago. Direct link to akshita sharma's post “what is the geometrical i...”
what is the geometrical interpretation for lagrange multipliers involving two constraints??
Button navigates to signup pageButton navigates to signup page
(3 votes)
Answer
- Ian Syndergaard
  Posted 3 years ago. Direct link to Ian Syndergaard's post “With two constraints, you...”
  With two constraints, you'll end up with two Lagrange multipliers (one associated with each constraint).
  The magnitude of each Lagrange multiplier tells you how much improvement you could get by relaxing the corresponding constraint.
  Button navigates to signup page
  (5 votes)
Zaz Brown
Posted 8 years ago. Direct link to Zaz Brown's post ““There's a slight twist t...”
“There's a slight twist to this story, best illustrated with an example.” is repeated.
Button navigates to signup pageButton navigates to signup page
(5 votes)
Answer
Molly Swenson
Posted 7 years ago. Direct link to Molly Swenson's post “Use the method of Lagrang...”
Use the method of Lagrange Multipliers to determine the maximum and minimum of f(x,y,z) = x + y + z subject to the two conditions g(x,y,z) = x2 + y2 − 2 = 0 and h(x, y, z) = x + z − 1 = 0
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Alexander Wu
  Posted 7 years ago. Direct link to Alexander Wu's post “We haven't learned about ...”
  We haven't learned about multiple constraints yet. The Lagrangian Function for multiple constraints is ℒ(x1,x2,...,xn,λ1,λ2,...,λM) = f(x1,x2,...,xn) - Σ λkgk(x1,x2,...,xn), where f(x) is the function to be maximized and g1, g2, ..., gM are the constraints (the Σ sum is from k = 1 to M). Note that here we use gk(x) = 0 instead of gk(x) = c.
  
  We take the three gradients and get ∇f = i + j + k, ∇g = 2x i + 2y j, ∇h = i + k. ∇f = λ1∇g + λ2∇h, and g = h = 0, so we get: 1 = 2xλ1 + λ2, 1 = 2yλ1, 1 = λ2, x^2 + y^2 = 2, and x + z - 1 = 0. We see that λ2 = 1, so 2xλ1 = 0. because λ1 = 1/2y, x/y = 0, so x must be 0. Then y = +-√2, and λ1 = +-√2/4, and z = 1.
  
  So we get two solutions: (0,√2,1) and (0,-√2,1). I think you can figure out which is the maximum.
  
  If you can graph this, try it, and you'll see how stupid our computations were. After all this work the answers are on the y-z plane!
  Button navigates to signup page
  (5 votes)
Kevin
Posted 7 years ago. Direct link to Kevin's post “Hi, thanks for the articl...”
Hi, thanks for the article. Question ... how does a parallel gradient tell us that the 2 contour lines are tangent to each other? Parallel gradients should be possible even if the 2 functions don't touch right? Thank you.
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Alexander Wu
  Posted 7 years ago. Direct link to Alexander Wu's post “Actually, parallel gradie...”
  Actually, parallel gradients tell us that the contour lines are parallel. Contour lines are tangent when they are both touching and parallel at the place they're touching. Of course, we had to answer the question "Where are they parallel?" The direction of gradients and contour lines are different at every point.
  
  Therefore, the gradients of the two functions need to be parallel at the same point. ∇f(0,1) being parallel to ∇g(1,0) wouldn't be much use. ∇f(0,1) must be parallel to ∇g(0,1).
  
  Of course, this says nothing about where the functions f and g are, only what their gradients (slants or tilts) are. In fact, it doesn't matter whether f(0,1) = g(0,1). After all, we are restricting g to g(x,y) = c, and c may be arbitrary. If the restraint is x + y = 4, we can let g(x,y) = x + y and c = 4, or we can let g(x,y) = x + y - 4 and c = 0, or g(x,y) = x + y + 100 and c = 104. So how "high" g is doesn't matter at all, it's just its gradient (slant or tilt) that we need.
  
  Of course, sometimes you might be required to let c = 0, but there's no difference. The "height" of g doesn't matter.
  Button navigates to signup page
  (3 votes)
PTNLemay
Posted 4 years ago. Direct link to PTNLemay's post “What does it mean when yo...”
What does it mean when you have more than one constraint (g and h), so you have 5 equations, 5 unknowns, but the answer you get for the Lagrange multipliers don't come out as a nice scalars? For example lambda ends up being something that depends on x and y.
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
Yvan Ou
Posted a year ago. Direct link to Yvan Ou's post “Hello, In the chapter ex...”
Hello,

In the chapter explaining When the gradient comes into play, we insert the constraint in g(x, y) formula (we have x² + y² - 1 instead of x² + y²).

Why do we insert the constraint -1 when computing the gradient ?
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Isaac
  Posted a year ago. Direct link to Isaac's post “At that phase in the less...”
  At that phase in the lesson, I think there is no difference (constants do not affect the gradient there). The problem is that the latter phases of the lesson use that same g-function multiplied with a Lagrange multiplier, and since we want the partial derivative of λg with respect to λ to be g = function − c so that setting that partial derivative to zero returns the constraint function = c, the constant is a bit more useful in the later phases.
  Button navigates to signup page
  (1 vote)
Alexander Wu
Posted 8 years ago. Direct link to Alexander Wu's post “The contour graph for f(x...”
The contour graph for f(x,y) = 2x^2 + √(5y) doesn't seem right. For example, check the point (0,2). It should equal √10, yet in the contour graph, it has two values, √10 and -√10. And f(2,3) should be 8 + √15, about 12, but on the plot it seems it also has a value at about 4.

I think there is a bug in the program that displays this plot. Once you square everything and solve the equation, you get roots that weren't there before. For example, if you square both sides of x = √2, you get x^2 = 2, so x = +-√2. Though I'm not sure if that's what's wrong with the program.

I graphed the function with my calculator and there was only one value for (0,2) and (2,3). But then, f is a function, so the can only be one output for each pair of inputs. The contour plot above doesn't seem right.
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
gustavo.faria.castro
Posted 7 years ago. Direct link to gustavo.faria.castro's post “And when we have inequali...”
And when we have inequality constrains? How we can formulate the Lagrangian equation? For example x² + y² <= 1. I have doubts about how we can solve this.
Button navigates to signup pageButton navigates to signup page
(1 vote)
Answer
- Ian Syndergaard
  Posted 3 years ago. Direct link to Ian Syndergaard's post “There are techniques, suc...”
  There are techniques, such as Kuhn-Tucker conditions, which were created specifically to answer these questions.
  In short, if the optimal point is interior to the constraints you can solve the problem without constraints. The resulting solution will satisfy the constraints regardless of the fact that you did not enforce them.
  If the optimal point is exterior to the constraints, then you can solve the problem with the constraint written as an equality constraint.
  Button navigates to signup page
  (2 votes)