National University of Computer & Emerging Sciences
Islamabad
FAST School of Computing
Spring 2026
Islamabad Campus
MT1008 – Multivariable Calculus
Homework # 7
Gradient Descent
In this assignment, you will build a foundational binary classification model to predict university admissions
based on a single standardized test score. Consider the following small dataset of three students:
Student
1
2
3
Test Score (x) Admitted (y)
45
0
90
1
75
1
Model Definitions & Objective
We will use a logistic regression model, defined by the linear combination
z = wx + b
and the sigmoid activation function:
ŷ = σ(z) =
1
1 + e−z
where ŷ represents the predicted probability of a student being admitted (y = 1).
The objective is to minimize the binary cross-entropy loss for a single data point, given by:
Ji (w, b) = − [y log(ŷ) + (1 − y) log(1 − ŷ)]
Initial Parameters
For all subsequent parts, please use the following initial parameters and learning rate (α). Show all steps
clearly in your calculations.
w = 0,
b = 0,
α = 0.001
Part 1: Deriving the Gradients
Given the sigmoid function σ(z):
(a) Prove that its derivative with respect to z is:
dσ
= σ(z)(1 − σ(z))
dz
(b) Apply the chain rule to derive the gradients of the loss function Ji with respect to the parameters w
and b. Specifically, show that:
∂Ji
= (ŷ − y)x
∂w
and
1
∂Ji
= (ŷ − y)
∂b
Part 2: Batch Gradient Descent
Execute two epochs of batch gradient descent. Use all three data points simultaneously for each update.
Provide the updated values for w and b at the end of each epoch.
Part 3: Stochastic Gradient Descent
Perform one epoch of stochastic gradient descent. Process the students sequentially in this exact order:
Student 1 → Student 2 → Student 3
Update the model parameters immediately after processing each student, and record the values of w and
b after every individual update.
Part 4: Mini-Batch Gradient Descent
Conduct one epoch of mini-batch gradient descent using the following two batches:
• Batch A: Student 1
• Batch B: Students 2 and 3
Process the batches in the order of Batch A followed by Batch B. Update the parameters after each batch
and provide the new values for w and b.
Part 5: Conceptual Analysis
Reflect on the calculations you performed above to answer the following questions:
(a) Which of the three gradient descent methods produced the largest overall change in the weight w after
one epoch? Briefly explain the mechanics behind why this happened.
(b) Evaluate the model’s prediction for Student 1 (who was not admitted) after one epoch. Which method
yielded the lowest (and therefore most accurate) predicted probability ŷ (1) ? What does this outcome
suggest about the influence of the dataset on the model updates?
(c) Examine the final value of w across the different methods. Based on its sign, does a higher test score
increase or decrease the predicted probability of admission? Does this mathematical relationship align
with the empirical pattern observed in our dataset?
2