will there be videos and exercises (mostly interested in the exercises) for these topics any time soon?

Me too. Would love to see exercises in multivariable calculus, differential equations and linear algebra.

Is the Hessian in any way related to the Jacobian matrix?

More formally: *H* (f(*x*)) = *J* (∇ f(*x*))^T. It also relates to the Laplacian as an operator: Δf = ∇²f = trace (*H*(f)).

Should the determinant in the final step be: 180xy^4 - 4?

I agree partially with Marcel Brown; as the determinant is calculated in a 2x2 matrix by ad-bc, in this form bc=(-2)^2 = 4, hence -bc = -4. However, ab.coefficient = 6*-30 = -180, not 180 as Marcel stated.

Why is fyx= d/dy(3x^2-2y) and not d/dx(-2x-6y^5)? Wouldn't it be the partial derivative with respect to x of the first partial derivative with respect to y? I ask the same for fxy= d/dx(-2x-6y^5) not being d/dy(3x^2-2y).

It is both! Whether you derivate with respect to x first then y, or with respect to y first then x, you get the same answer. Notice here that fxy = fyx = -2. That is Clairaut's theorem.

What are some of the practical applications of the determinant of a Hessian matrix?

Evaluating it can tell you whether you are at a maximum, minimum, or a saddle point. It has all the same abilities as a second derivative in a uni-variate function.

Main content

Course: Multivariable calculus > Unit 3

Lesson 2: Quadratic approximations

The Hessian

Google Classroom

The Hessian is a matrix that organizes all the second partial derivatives of a function.

Background:

Second partial derivatives

The Hessian matrix

The "Hessian matrix" of a multivariable function

f (x, y, z, \dots)

, which different authors write as

H (f)

H f

, or

H_{f}

, organizes all second partial derivatives into a matrix:

H f = [\begin{array}{ccc} \frac{\partial^{2} f}{\partial x^{2}} & \frac{\partial^{2} f}{\partial x \partial y} & \frac{\partial^{2} f}{\partial x \partial z} & \dots \\ \frac{\partial^{2} f}{\partial y \partial x} & \frac{\partial^{2} f}{\partial y^{2}} & \frac{\partial^{2} f}{\partial y \partial z} & \dots \\ \frac{\partial^{2} f}{\partial z \partial x} & \frac{\partial^{2} f}{\partial z \partial y} & \frac{\partial^{2} f}{\partial z^{2}} & \dots \\ ⋮ & ⋮ & ⋮ & ⋱ \end{array}]

So, two things to notice here:

This only makes sense for scalar-valued function.
This object $H f$ ‍ is no ordinary matrix; it is a matrix with functions as entries. In other words, it is meant to be evaluated at some point $(x_{0}, y_{0}, \dots)$ ‍.
$H f (x_{0}, y_{0}, \dots) = [\begin{array}{ccc} \frac{\partial^{2} f}{\partial x^{2}} (x_{0}, y_{0}, \dots) & \frac{\partial^{2} f}{\partial x \partial y} (x_{0}, y_{0}, \dots) & \dots \\ \frac{\partial^{2} f}{\partial y \partial x} (x_{0}, y_{0}, \dots) & \frac{\partial^{2} f}{\partial y^{2}} (x_{0}, y_{0}, \dots) & \dots \\ ⋮ & ⋮ & ⋱ \end{array}]$ ‍

As such, you might call this object

H f

a "matrix-valued" function. Funky, right?

In the single variable world, things were so simple. The derivative, second derivative, etc. are all still single variable functions.

Now, when a scalar-valued function

f

has multiple inputs, the analog of the first derivative is the gradient,

\nabla f

, which is a vector-valued function, and the analog of the second derivative is this Hessian

H f

, which is a matrix-valued function.

You might wonder if there are analogous ideas for higher order derivatives, third, fourth, etc. There are! And there is even a nice extension of Taylor's formula using them all.

"Really, what are they?", I hear you asking eagerly. Well, expressing them in their full glory involves something called "multilinear algebra". It is both fascinating and beautiful, but unfortunately a bit outside the scope of our current discussion. Patience.

One more important thing, the word "Hessian" also sometimes refers to the determinant of this matrix, instead of to the matrix itself.

Example: Computing a Hessian

Problem: Compute the Hessian of

f (x, y) = x^{3} - 2 x y - y^{6}

at the point

(1, 2)

Solution: Ultimately we need all the second partial derivatives of

f

, so let's first compute both partial derivatives:

\begin{aligned} f_{x} (x, y) & = \frac{\partial}{\partial x} (x^{3} - 2 x y - y^{6}) = 3 x^{2} - 2 y \\ f_{y} (x, y) & = \frac{\partial}{\partial y} (x^{3} - 2 x y - y^{6}) = - 2 x - 6 y^{5} \end{aligned}

With these, we compute all four second partial derivatives:

\begin{aligned} f_{x x} (x, y) & = \frac{\partial}{\partial x} (3 x^{2} - 2 y) = 6 x \\ f_{x y} (x, y) & = \frac{\partial}{\partial y} (3 x^{2} - 2 y) = - 2 \\ f_{y x} (x, y) & = \frac{\partial}{\partial x} (- 2 x - 6 y^{5}) = - 2 \\ f_{y y} (x, y) & = \frac{\partial}{\partial y} (- 2 x - 6 y^{5}) = - 30 y^{4} \end{aligned}

The Hessian matrix in this case is a

2 \times 2

matrix with these functions as entries:

H f (x, y) = [\begin{array}{cc} f_{x x} (x, y) & f_{y x} (x, y) \\ f_{x y} (x, y) & f_{y y} (x, y) \end{array}] = [\begin{array}{cc} 6 x & - 2 \\ - 2 & - 30 y^{4} \end{array}]

We were asked to evaluate this at the point

(x, y) = (1, 2)

, so we plug in these values:

H f (1, 2) = [\begin{array}{cc} 6 (1) & - 2 \\ - 2 & - 30 (2)^{4} \end{array}] = [\begin{array}{cc} 6 & - 2 \\ - 2 & - 480 \end{array}]

Now, the problem is ambiguous, since the "Hessian" can refer either to this matrix or to its determinant. What you want depends on context. For example, in optimizing multivariable functions, there is something called the "second partial derivative test" which uses the Hessian determinant. When the Hessian is used to approximate functions, you just use the matrix itself.

If it's the determinant we want, here's what we get:

det ([\begin{array}{cc} 6 & - 2 \\ - 2 & - 480 \end{array}]) = 6 (- 480) - (- 2) (- 2) = - 2884

Uses

By capturing all the second-derivative information of a multivariable function, the Hessian matrix often plays a role analogous to the ordinary second derivative in single variable calculus. Most notably, it arises in these two cases:

Quadratic approximations of multivariable functions, which is a bit like a second order Taylor expansion, but for multivariable functions.
The second partial derivative test, which helps you find the maximum/minimum of a multivariable function.

Want to join the conversation?

Sort by:

Jorge R. Martinez Perez-Tejada
Posted 8 years ago. Direct link to Jorge R. Martinez Perez-Tejada's post “will there be videos and ...”
will there be videos and exercises (mostly interested in the exercises) for these topics any time soon?
Button navigates to signup pageButton navigates to signup page
(34 votes)
Answer
- arkanmanva
  Posted 8 years ago. Direct link to arkanmanva's post “Me too. Would love to see...”
  Me too. Would love to see exercises in multivariable calculus, differential equations and linear algebra.
  Button navigates to signup page
  (24 votes)
Maria
Posted 8 years ago. Direct link to Maria's post “Is the Hessian in any way...”
Is the Hessian in any way related to the Jacobian matrix?
Button navigates to signup pageComment on Maria's post “Is the Hessian in any way...”
(7 votes)
Answer
- Jo Marino
  Posted 7 years ago. Direct link to Jo Marino's post “More formally: *H* (f(*x*...”
  More formally: H (f(x)) = J (∇ f(x))^T.
  It also relates to the Laplacian as an operator: Δf = ∇²f = trace (H(f)).
  Button navigates to signup page
  (14 votes)
Marcel Brown
Posted 8 years ago. Direct link to Marcel Brown's post “Should the determinant in...”
Should the determinant in the final step be: 180xy^4 - 4?
Button navigates to signup pageButton navigates to signup page
(7 votes)
Answer
- Kenz
  Posted 8 years ago. Direct link to Kenz's post “I agree partially with Ma...”
  I agree partially with Marcel Brown; as the determinant is calculated in a 2x2 matrix by ad-bc, in this form bc=(-2)^2 = 4, hence -bc = -4. However, ab.coefficient = 6*-30 = -180, not 180 as Marcel stated.
  Button navigates to signup page
  (12 votes)
gschex1112
Posted 8 years ago. Direct link to gschex1112's post “Why is the last second pa...”
Why is the last second partial derivative not -30y^4?
Button navigates to signup pageButton navigates to signup page
(5 votes)
Answer
Emily H
Posted 8 years ago. Direct link to Emily H's post “Why is fyx= d/dy(3x^2-2y)...”
Why is fyx= d/dy(3x^2-2y) and not d/dx(-2x-6y^5)? Wouldn't it be the partial derivative with respect to x of the first partial derivative with respect to y? I ask the same for fxy= d/dx(-2x-6y^5) not being d/dy(3x^2-2y).
Button navigates to signup pageButton navigates to signup page
(4 votes)
Answer
- Tejas
  Posted 8 years ago. Direct link to Tejas's post “It is both! Whether you d...”
  It is both! Whether you derivate with respect to x first then y, or with respect to y first then x, you get the same answer. Notice here that fxy = fyx = -2. That is Clairaut's theorem.
  Button navigates to signup page
  (4 votes)
Aaron Hargrove
Posted 8 years ago. Direct link to Aaron Hargrove's post “What are some of the prac...”
What are some of the practical applications of the determinant of a Hessian matrix?
Button navigates to signup pageButton navigates to signup page
(2 votes)
Answer
- Tejas
  Posted 8 years ago. Direct link to Tejas's post “Evaluating it can tell yo...”
  Evaluating it can tell you whether you are at a maximum, minimum, or a saddle point. It has all the same abilities as a second derivative in a uni-variate function.
  Button navigates to signup page
  (5 votes)
Briahnna A
Posted 8 years ago. Direct link to Briahnna A's post “Around the last paragraph...”
Around the last paragraph, the determinant is used to evaluate specific points. But, I was wonder why, in my multivariate calculate classes, we also use the hessian determinant and f_xx to determine local extrema. I was wondering what was the background or the basis for using the determinant of the Hessian Matrix to decided if critical points are local maximum or local minimums? Basically, what is important about the Hessian Determinant? Thank-You!
Button navigates to signup pageButton navigates to signup page
(4 votes)
Answer
- Tejas
  Posted 8 years ago. Direct link to Tejas's post “We actually don't use the...”
  We actually don't use the Hessian to determine whether the critical points are local maxima or local minima. We actually use the Hessian to determine whether they are local extrema or saddle points. As for using fxx, it doesn't have to be fxx. You could just as easily use fyy to determine whether the local extremum is a maximum or minimum.
  
  If it is a local minimum, the gradient is pointing away from this point. If it is a local maximum, the gradient is always pointing toward this point. Of course, at all critical points, the gradient is 0. That should mean that the gradient of nearby points would be tangent to the change in the gradient. In other words, fxx and fyy would be high and fxy and fyx would be low.
  
  On the other hand, if the point is a saddle point, then the gradient vectors will all be pointing around the critical point. Therefore at nearby points, the change in the gradient will be orthogonal to the gradient, not tangent. In other words, fxy and fyx would be high and fxx and fyy would be low, or fxx and fyy would have opposite signs. Either way, the Hessian determinant would be negative.
  Button navigates to signup page
  (1 vote)
yogesh kumar
Posted 8 years ago. Direct link to yogesh kumar's post “why hessian makes sense o...”
why hessian makes sense only for a scalar valued function?
Button navigates to signup pageComment on yogesh kumar's post “why hessian makes sense o...”
(4 votes)
Answer
muhammadahme2019
Posted 5 years ago. Direct link to muhammadahme2019's post “Where it says The Hessian...”
Where it says The Hessian matrix in this case is a 2 x 2 matrix with these functions as entries:
the fxx and fxy are flipped. Cuz under fxx it should be fxy. Then under fyx it should be fyy.
Button navigates to signup pageComment on muhammadahme2019's post “Where it says The Hessian...”
(2 votes)
Answer
disha.bandy
Posted 7 years ago. Direct link to disha.bandy's post “Since Hessian matrices ar...”
Since Hessian matrices are only used for trigonometric functions, what can be used to optimise a 3 variable trigonometric function?
Button navigates to signup pageButton navigates to signup page
(1 vote)
Answer