Practical Sheet 2: Introduction to Probability and Linear Algebra
Instructor: Dr. Md Rushdie Ibne Islam
APL 405: Machine Learning for Mechanics
Department of Applied Mechanics, Indian Institute of Technology Delhi, Winter 2025
Question 1. Random Numbers sampling
1) [1 Marks] Consider the univariate and multivariate normal distributions given as:
(x−µ)2
1
f (x; µ, σ) = √ e− 2σ2
σ 2π
(
)
1
1
⊤ −1
f (x; µ, Σ) = √
exp − (x − µ) Σ (x − µ)
2
(2π)k |Σ|
(a) Given that x is a univariate normal variable with µ = 2.5 and σ = 0.5, generate
1000 samples of x. Plot the histogram.
(b) Consider the random variable vector x that is normally distributed with the
following statistics and generate 1000 samples of x.


P (x) = N µ = 



1 0.5 
1 
,Σ = 
0.5 2
8
Scatter plot the samples to visualize the distribution and mark the mean and 99.97%
confidence interval (CI).
2) [1 Marks] Now assume that the generated random samples in the previous step
are given to you by someone. Write a program to estimate the sample mean and
sample variance of the generated samples and verify that it matches the true mean
and variance of the random numbers. The sample mean and sample variance for a
multivariate random number x is given as,
Sample mean, µx =
Sample Variance, Σij =
N
1 ∑
xi
N − 1 i=1
N
1 ∑
(xi − µx )(yi − µy )
N − 1 i=1
OR
Σ=
1
(x − µx )⊤ (y − µy )
N −1
Report results for M = 100, 1000, and 100000. What do you observe?
3) [2 Marks] Consider a uniform random variable x with µ = 2.5 and σ = 0.5. Generate a new random variable y = x2 . Given the distribution function of x, find the
1
2
distribution of the transformed random variable y. Use the following formula,
dy
py (y) = ∥ ∥−1 py (x)
dx
Plot the PDFs py (y) and py (x) to visualize the distributions. Do you observe nonlinearity in the transformation?
Question 2. Linear Algebra
1) [1 Marks] Consider two random matrices A ∈ R10000×10000 , B ∈ R10000×10000 , and the
random vector C ∈ R10000 . Can you perform the multiplication ABC? Would you
use (AB)C or A(BC)? Compare the time.
2) [1 Marks] Consider the linear system of equations.
Ax = b
where A ∈ R64×32 , and b ∈ R64 . Since the inverse of A does not exist, how will you
solve the system of equations? Hint: multiply both sides by A⊤ to get a square matrix
on the left-hand side.
3) [1 Marks] Consider the two vectors x = [3, 4, 8] and w = [4, 3, −3].
(a) Find the dot and outer product between them.
(b) Find the length of the vectors.
(c) Find the angle between two vectors.
(d) State whether they are orthogonal to each other.
4) [1 Marks] Given the following matrix,


2000 −1000 −500



A=
 −1000 2000 −1000 
−500 −1000 2000
find the diagonal matrix D of A. Hint: D = V−1 AV, where V is the eigenvectors.
5) [1 Marks] Given the eigenvectors of the above matrix, can you show whether V is
an orthogonal matrix? If you find it to be orthogonal, how will you find the inverse
without direct inversion?
6) [1 Marks] The inverse of a positive definite matrix is often done using Cholesky
decomposition as,
K−1 = (L−1 )⊤ L−1
where A = LL⊤ is the Cholesky decomposition. Now generate a random matrix
A ∈ R10000×10000 and perform K = A⊤ A to make it positive definite. Find the inverse
of K using the above formula and compare the error with the standard inverse.
3
Take Home
Question 3. Applications and Norm
1) Consider a signal x = sin(2πt) for some time t ∈ [0, T ]. To simulate a real scenario,
it is often required to add noise to the simulated data artificially. Now perform the
following,
y = x2 + ϵ
where ϵ ∼ N (0, σϵ2 ) is a zero mean Gaussian noise with standard deviation σϵ . Consider σϵ to be 15% of the standard deviation of x2 . Now, find the mean and variance
of y. Do observe linearity in the expectation, i.e., E[y] = E[x2 ] + E[ϵ]. Also, do
you observe any relation between variances of y and ϵ? Assume T = 10 seconds and
∆t = 0.01.
2) Considered the vector x2 and its noisy version y from the above problem. In machine
learning, norms are often used as loss functions. Now find the following norms of the
error between x2 and y:
∑
(a) L1 norm: ∥ϵ∥1 = √ N
i=1 |ϵi |.
∑N 2
(b) L1 norm: ∥ϵ∥2 =
i=1 ϵi .
(c) L1 norm: ∥ϵ∥∞ = max |ϵi |.
(d) Matrix norm: ∥ϵ∥Q = ϵ⊤ Qϵ. Assume Q to be an identity matrix.
In all the cases, verify your results with in-built functions from ‘numpy’.
Question 4. Multivariate Gaussian distribution
1) Given the following bivariate distribution,

 

xa
σ2
ρσa σb 
p(x | µ, Σ) = N   ,  a
ρσa σb
σb2
xb
show that the conditional distribution p(xa | xb ) is given as,
(
ρσa σb
(ρσa σb )2
2
p(xa | xb ) = N xa | µa +
(x
−
µ
),
σ
−
b
b
a
σb2
σb2
)
2) Consider two random variables x1 and x2 with the following joint distribution,

 

1.3   0.4 0.2 
,
P (x1 x2 ) = N 
0.2 0.5
1.5
With six standard deviations, the limits of the random variables are given as X1 ∈
[−1.1, 3.7] and X2 ∈ [−1.5, 4.5].
4
(a) Find the conditional distribution of P (X1 |X2 = 2.5) and plot it along with the
joint distribution.
∫
(b) Then obtain the marginal distribution P (X1 ) = P (X1 X2 )dx2 . Plot it along
with the joint distribution. Observe if the conditional and marginal distributions
follow the Gaussian distribution.
3) Generate 10,000 random numbers from an univariate normal distribution. Similar to
the mean and variance, numerically obtain the third and fourth order moment,
∫
(a) The skewness: E[(x − µ)3 ] = x∈X (x − µ)3 fX (x)dx,
∫
(b) The kurtosis: E[(x − µ)4 ] = x∈X (x − µ)4 fX (x)dx.
Use the law of large numbers to estimate the above statistical quantities. Use inbuilt
functions from Python to verify your results.