Machine Learning
CSE 326/426
Sihong Xie
CSE Department
Lehigh University
Aug 22, 2022
Today’s goals:
• Basic calculus
• Linear regression
Readings:
PRML 3.1.1-3.1.3
Basic calculus
o Function
Log
1081×+4
×
•k--
a mapping from input x to output y.
In ⁿʰ%Tx*h
✗
• Example:
7=-3
o Composition of functions (
neural
networks )
• Example:
a-
o Derivative
[
gradient
( the
descent )
increase
• Chain rule
(
• Linearity
direction
gradient descent
P
*
b
that
fastest
f
wid
)
)
&
T
8/21/22
read number
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
2
Basic linear algebra
uT=Eh
mn
o A row vector is an array of numbers
✓
• Vector space
transport #
=
C-
o A matrix
IR
-
-
un ]
I
=
: the set of all vectors with n real numbers.
• Inner product:
-
,
www.t-j-f-gc-1-erow
summation
is a two-dimensional array of numbers
• i-th row
• j-th column
ERM
T
• Transposition of A, denoted by A , is the matrix having columns
as the rows from A rotated.
"
f.
"
• Matrix-vector product
a-
m
8/21/22
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
um
*
=
(
I
z
2
i
I
-
i
]
ñ
=
[¥)
T.im
Añ
=
.
[
< Anni
{ A2
:
i
Column
=
-1 :]
3
,
>
Ñ>
1×1+2×0 -11×1-4
] [ -11×0+(-4×4)
=
2×1
A-
_
Basic linear algebra
and
,
• Transposition of matrix product
• Inverse of a full-rank square matrix
¥
Block matrix multiplication :#
-
%)
linearly
independent
'
=
A
,
/
B )
,
:
=
( Az
,
A. Azt Bicz
[£
'
?
,
]
4
Cz
) >
=
]
.B=j >
C)
,
13=1 >
=P
(2×2)
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
( (
:/
AB
☐
8/21/22
KAI
]
is a matrix of size (n x m) with
( Ai
=
•
,
,
• The product of two matrices
the element
are
'
13=1-8 ;]
o Matrix-matrix product
rows
[
I }
?
by
general
In
=
convention
f )
?
'
g,
Problem setting
Y
f-
=
Living Area (x1)
[
✗
"
i.
if
-
✗ i.
×
d
features
describe
specific
1*-1 : :| ,y )
]
Y'
Price (y)
3
1262
2
1786
3
223500
1717
4
140000
I 2198
5
1362
2
]
]
the
training
)
# of bdrooms (x2)
I
208500
181500
250000
143000
""
'
Y
"
f
c-
)
-
-
-
8/21/22
-
ylm
R
• m: number of training examples
y "
Y •
•
:
i'
example
•
•
: input feature vector
x=[x1,x2]T
-
: output (target) variable
training
: a single example
: the i-th training example
I
'm
✗
.
1710
it
.
(
with
)
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
5
set
,
Linear regression
• Linear regression is a sort of supervised learning.
• General supervised learning procedure.
as
How to represent a model in a computer?
Training
Algorithm
Training
Data
x1
x2
y
1710
3
208500
1262
2
181500
...
...
...
Predictions
Model
Gradient descent
Test Data
8/21/22
function f
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
x1
x2
y
2005
3
?
3500
4
?
...
...
...
6
=
< 0
/
×
>
Linear hypothesis
mm
Training examples of housing prices
Input feature with the bias term 1:
o=
Living Area (x1)
# of bdrooms (x2)
Price (y)
1710
3
208500
1262
2
181500
1786
3
223500
1717
4
140000
2198
5
250000
1362
2
143000
-1%:]
Parameters:
=
Hypothesis:
< 0.x >
*
+ 02×2
Goal: choose parameters so that
The best you can have given only the training examples,
though our goal is to predict y on unseen x.
8/21/22
CSE 326/426 Machine Learning. Fall 2022. Prof. Xie
7