Basics of Neural
Network Programming
Binary Classification
deeplearning.ai
Binary Classification
1 (cat) vs 0 (non cat)
Blue
Green
Red
255 134 93 22
255 134
123 202
94 22
83 2
255 231
123 42
94
83
4 30
34 22
44 187
123 94
2 192
34 83
44
34 187
76
232 124
34 44
92
34 187
76
34 142
67 232
83 194
34 76
67 232
83 124
194 94
67 83 194 202
Andrew Ng
Notation
Andrew Ng
Basics of Neural
Network Programming
Logistic Regression
deeplearning.ai
Logistic Regression
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Logistic Regression
cost function
Logistic Regression cost function
!
!" = % & ' + ) , where % *
=
"
"#$ +,
Given (' (.) , ! (.) ),…,(' (1) , ! (1) ) , want !" (2) ≈ ! 2 .
Loss (error) function:
Andrew Ng
Basics of Neural
Network Programming
Gradient Descent
deeplearning.ai
Gradient Descent
'
Recap: !" = % & ( + * , % + =
1 &, * =
6
1
5
4
78,
6
1
− 5
4
ℒ(!" 7 , ! (7) ) =
78,
,
,-. /0
! (7) log !" 7 + (1 − ! (7) ) log(1 − !" 7 )
Want to find &, * that minimize 1 &, *
1 &, *
&
*
Andrew Ng
Gradient Descent
!
Andrew Ng
Basics of Neural
Network Programming
Derivatives
deeplearning.ai
Intuition about derivatives
! " = 3"
"
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
More derivatives
examples
Intuition about derivatives
! " = "$
"
Andrew Ng
More derivative examples
Andrew Ng
Basics of Neural
Network Programming
Computation Graph
deeplearning.ai
Computation Graph
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Derivatives with a
Computation Graph
Computing derivatives
&=5
"=3
#=2
11
6
!="#
$ =&+!
33
) = 3$
Andrew Ng
Computing derivatives
&=5
"=3
#=2
11
6
!="#
$ =&+!
33
) = 3$
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Logistic Regression
Gradient descent
Logistic regression recap
! = $%& + (
)* = + = ,(!)
ℒ +, ) = −() log(+) + (1 − )) log(1 − +))
Andrew Ng
Logistic regression derivatives
&%
$%
&(
$(
b
! = $% &% + $( &( + )
* = +(!)
ℒ(a, 1)
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Gradient descent
on m examples
Logistic regression on m examples
Andrew Ng
Logistic regression on m examples
Andrew Ng
Basics of Neural
Network Programming
Vectorization
deeplearning.ai
What is vectorization?
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
More vectorization
examples
Neural network programming guideline
Whenever possible, avoid explicit for-loops.
Andrew Ng
Vectors and matrix valued functions
Say you need to apply the exponential operation on every element of a
matrix/vector.
!$
!= ⋮
!&
u = np.zeros((n,1))
for i in range(n):
u[i]=math.exp(v[i])
Andrew Ng
Logistic regression derivatives
J = 0, dw1 = 0, dw2 = 0, db = 0
for i = 1 to n:
! (") = " $ # (") + %
&(") = '(! (") )
* += − - (") log -1 " + (1 − - " ) log(1 − -1 " )
d! (") = &(") (1 − &(") )
(")
d"% += #% d! (")
(")
d"' += #' d! (")
db += d! (")
J = J/m, d"% = d"% /m, d"' = d"' /m, db = db/m
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Vectorizing Logistic
Regression
Vectorizing Logistic Regression
! (#) = & ' ( (#) + *
+(#) = ,(! (#) )
! (-) = & ' ( (-) + *
! (.) = & ' ( (.) + *
+(-) = ,(! (-) )
+(.) = ,(! (.) )
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Vectorizing Logistic
Regression’s Gradient
Computation
Vectorizing Logistic Regression
Andrew Ng
Implementing Logistic Regression
J = 0, d!! = 0, d!" = 0, db = 0
for i = 1 to m:
" ($) = ! & # ($) + %
&($) = '(" ($) )
* += − - ($) log & $ + (1 − - $ ) log(1 − & $ )
d" ($) = &($) −- ($)
($)
d!! += #! d" ($)
($)
d!" += #" d" ($)
db += d" ($)
J = J/m, d!! = d!! /m, d!" = d!" /m
db = db/m
Andrew Ng
Basics of Neural
Network Programming
deeplearning.ai
Broadcasting in
Python
Broadcasting example
Calories from Carbs, Proteins, Fats in 100g of different foods:
Carb
Protein
Fat
Apples
Beef
56.0
1.2
1.8
0.0
104.0
135.0
Eggs Potatoes
4.4 68.0
52.0 8.0
99.0 0.9
cal = A.sum(axis = 0)
percentage = 100*A/(cal.reshape(1,4))
Broadcasting example
1
2
3
4
100
+
1
4
2
5
3
6
1
4
2
5
3
6
+
+
100
200
100
200
101
102
103
104
=
300
=
101
104
202
205
303
306
=
101
204
102
205
103
206
General Principle
Basics of Neural
Network Programming
deeplearning.ai
Explanation of logistic
regression cost function
(Optional)
Logistic regression cost function
Andrew Ng
Logistic regression cost function
If
If
$ = 1:
$ = 0:
( $ ) = $*
( $ ) = 1 − $*
Andrew Ng
Cost on m examples
Andrew Ng