Report: Practical Two
Data Engineering 414
Name: Hendrik de Man
Student Number: 25451324
Date: 2026/03/05
Introduction
The hypothetical situation the practical presents is that I am being interviewed for a
job at a new data science company at techno park. This interview requires me to
implement a linear regression model from scratch in python. They have provided
skeleton code, but the rest I have to code. The objectives the practical covers
includes importing the data, implementing their linear regression model. Use the
accuracy function they gave to calculate the accuracy of the trained model (as well
as an untrained model). Create an error function and calculate the error of the
model. Lastly I will have to implement iterative gradient descent for the model.
Question 1:
Code:
Output:
The question very simply asks to load and then print the shape of the data. As can
be seen from the above the first data set (called test) has 60 000 entries. The
second data set (called train) has 10 000 entries. Each data set is broken up into
data and target matrices. Note how both data matrices have 784 columns. This will
be important when we add a column for the bias at the front.
Question 2:
Question 2 a):
Code:
Question 2 a) requires you to implement the closed form solution of linear
regression. This is very easily done by using built in python functions to calculate the
inverse and then multiply the relevant variables together to get the weights.
Formula for closed from linear regression weights:
Question 2 b):
Code:
Question 2 b) is very easily solved by multiplying your weights calculated in the
previous sub question with the X vector (your data) that has an additional column of
ones at the front to accommodate for the bias vector. The returned Yhat is the
output of the model. It represents the predicted values for the input values of X.
Question 2 c):
Code:
Question 2 c) creates a instance of the linear regression function and initialises it
with 784 (the dimensionality of the model – training data) and 10 (the dimensionality
of the targets). It then trains this instance of the model by accepting input data (in
this case it would be the trainDataFeature and the trainDataTarget). This applies the
training algorithm by calculating the weight vector based on the inputted data. While
not asked by this question, I also added an untrained model here.
Question 3:
Question 3 Code:
Question 3 Output:
Question 3 required to calculate the model’s accuracy. Lr.foreward gives you the
predicted output values of your input data.
Accuracy can very simply be calculated by finding the number of IDs are the same
for Yhat and the ground truth Y. This is done for both the model that we trained on
our data in question 2 and the model we left untrained. (note: the untrained model
initializes its weight vector to random values.
Question 4:
Question 4 Code:
Question 4 Output:
Calculating the error of the model effectively sums the squared distance between the
elements of the Y and Yhat vectors. This gives you an indication of how effective the
model is. The higher your error term, the worse your model performed.
For our model we calculated the error for both the trained and untrained model. As
can be seen from the output, the trained model has a much smaller error value than
the untrained model. This is to be expected.
Question 5:
Q5 Implemented iterative gradient descent model:
This function will loop through the for loop for the amount of epochs you have given
it. Every epoch it will determine an error term for your predicted Yhat value. Using
this error term it will calculate a gradient term. This gradient term will be used to
modify your weights (the amount the gradient will effect the weight will be determined
by your learning rate). More Epochs tend to mean that your model is better, however
there are some pitfalls as if your epoch count is too high, your model will stop
learning patterns in the data and instead start learning the values of the data itself. If
the learning rate it too high, it will mean that individual gradients will have too big of
an impact on the weights. Too small learning rate and the model will take too many
epochs to get good results.
Question 5 a):
Question 5 a) code:
The model is trained with the default 200 epochs and a learning rate of 0.01. It is
trained on both the Data and Target data. The model can now be used to make
predictions.
Question 5 b):
Question 5 b) code:
Question 5 c) output:
This question asks to determine the error and accuracy of both the train and test
data sets on our model. This was very simply done by using the functions we have
previously created in the practical and applying them to the relevant datasets. What
is however quite strange is that both had quite similar accuracy even though the
error differed significantly. The error for the training set was much higher than that of
the test set.
Question 5 c) not done (optional)
Conclusion
This practical laid out how to do linear regression as well as iterative gradient
descent to train models. Once these models have been trained, they can then be
tested on the training and test sets to determine both accuracy and error of the
respective models’ predictions. Even though these models are very simple in their
construction, they yield surprisingly good results.