Class Notes: Preliminaries
Shurojit Chatterji
Singapore Management University
August 2026
• We will follow the following convention: Formally, a vector in RN
is always a column. (It is sometimes written horizontally as x =
(x1 , ..., xN ) to save space.)
The inner product (also called dot product or scalar product) of two
P
vectors x, y ∈ RN is denoted x · y := N
n=1 xn yn . If these vectors are
seen as matrices, then x · y = xT y where T is the matrix transposition
operator. The expression x· can always be read as xT ; for instance
x · A where A is a N × M matrix, is the same as xT A.
• For a real-valued differentiable function f : X → R, X ⊂ RN , its
gradient vector ▽f (x̄) at some point x̄ is the column vector whose nth
entry is the partial derivative of f with respect to the nth variable
evaluated at x̄.
If f : X → RM is a vector-valued differentiable function, X ⊂ RN ,
at any x ∈ RN , we denote by Df (x) (or by f ′ (x)) the M × N matrix
m (x)
whose mnth entry is ∂f∂x
. In particular, if M = 1 (so that f (x) ∈
n
R) then Df (x) is a 1 × N matrix and ▽f (x) = [Df (x)]T .
1
• For a real-valued differentiable function f : X → R, X ⊂ Rn , its
Hessian matrix D2 f (x) is the derivative matrix of the vector-valued
gradient function, i.e., D2 f (x) = D[▽f (x)].
2
∂ f (x)
Thus D2 f (x) is the N × N matrix whose mnth entry is ∂x
.
m ∂xn
 
a
Example 1 (i) f (x1 , x2 ) = ax1 + bx2 . Then ▽f (x) =  
b
2
2
(ii) f (x1 ,
x2 ) = x31 − 3x
1 + x2 . Then 

▽f (x) = 
3x21 − 6x1
2x2
 and D2 f (x) = 
6x1 − 6 0
0
2

• A real-valued differentiable function f : RN
+ → R,
is homogenous of degree r (for r = ..., −1, 0, 1...), if for every t > 0,
we have f (tx) = tr f (x).
Theorem 2 If f : RN
+ → R, is homogenous of degree r and differentiable, then
(i) ∀n, ∂f∂x(x)
is homogenous of degree r − 1.
n
(ii) [Euler’s Formula] At any x̄, we have ▽f (x̄) · x̄ = rf (x̄).
Example 3 (i)f (x1 , x2 ) = xx21 is homogenous of degree 0. Then
▽f (x̄) · x̄ = x̄12 x̄1 − (x̄x̄21)2 x¯2 = 0.
1
(ii) f (x1 , x2 ) = (x1 x2 ) 2 is homogenous of degree 1. Then
1
1
1
▽f (x̄) · x̄ = 12 ( x̄x̄21 ) 2 x̄1 + 12 ( x̄x̄12 ) 2 x̄2 = (x̄1 x̄2 ) 2 = f (x̄1 , x̄2 ).
2
• The N × N matrix M is negative semidefinite (NSD) if z · M z ≤ 0
for all z ∈ RN . If the inequality is strict for all z ̸= 0, then the
matrix is negative definite (ND). Reversing the inequalities, we get
the notions of positive semidefinite (PSD) and positive definite (PD)
matrices.

Example 4 M = 
a b


b c
M is PSD iff a ≥ 0, c ≥ 0 and |A| = ac − b2 ≥ 0.
M is NSD iff a ≤ 0, c ≤ 0 and |A| = ac − b2 ≥ 0.
Suppose that a = 1, c = −1 and b = 0. Verify that M is indefinite.
3