Basics of Neural Network Programming Binary Classification deeplearning.ai Binary Classification 1 (cat) vs 0 (non cat) Blue Green Red 255 134 93 22 255 134 123 202 94 22 83 2 255 231 123 42 94 83 4 30 34 22 44 187 123 94 2 192 34 83 44 34 187 76 232 124 34 44 92 34 187 76 34 142 67 232 83 194 34 76 67 232 83 124 194 94 67 83 194 202 Andrew Ng Notation Andrew Ng Basics of Neural Network Programming Logistic Regression deeplearning.ai Logistic Regression Andrew Ng Basics of Neural Network Programming deeplearning.ai Logistic Regression cost function Logistic Regression cost function ! !" = % & ' + ) , where % * = " "#$ +, Given (' (.) , ! (.) ),…,(' (1) , ! (1) ) , want !" (2) ≈ ! 2 . Loss (error) function: Andrew Ng Basics of Neural Network Programming Gradient Descent deeplearning.ai Gradient Descent ' Recap: !" = % & ( + * , % + = 1 &, * = 6 1 5 4 78, 6 1 − 5 4 ℒ(!" 7 , ! (7) ) = 78, , ,-. /0 ! (7) log !" 7 + (1 − ! (7) ) log(1 − !" 7 ) Want to find &, * that minimize 1 &, * 1 &, * & * Andrew Ng Gradient Descent ! Andrew Ng Basics of Neural Network Programming Derivatives deeplearning.ai Intuition about derivatives ! " = 3" " Andrew Ng Basics of Neural Network Programming deeplearning.ai More derivatives examples Intuition about derivatives ! " = "$ " Andrew Ng More derivative examples Andrew Ng Basics of Neural Network Programming Computation Graph deeplearning.ai Computation Graph Andrew Ng Basics of Neural Network Programming deeplearning.ai Derivatives with a Computation Graph Computing derivatives &=5 "=3 #=2 11 6 !="# $ =&+! 33 ) = 3$ Andrew Ng Computing derivatives &=5 "=3 #=2 11 6 !="# $ =&+! 33 ) = 3$ Andrew Ng Basics of Neural Network Programming deeplearning.ai Logistic Regression Gradient descent Logistic regression recap ! = $%& + ( )* = + = ,(!) ℒ +, ) = −() log(+) + (1 − )) log(1 − +)) Andrew Ng Logistic regression derivatives &% $% &( $( b ! = $% &% + $( &( + ) * = +(!) ℒ(a, 1) Andrew Ng Basics of Neural Network Programming deeplearning.ai Gradient descent on m examples Logistic regression on m examples Andrew Ng Logistic regression on m examples Andrew Ng Basics of Neural Network Programming Vectorization deeplearning.ai What is vectorization? Andrew Ng Basics of Neural Network Programming deeplearning.ai More vectorization examples Neural network programming guideline Whenever possible, avoid explicit for-loops. Andrew Ng Vectors and matrix valued functions Say you need to apply the exponential operation on every element of a matrix/vector. !$ != ⋮ !& u = np.zeros((n,1)) for i in range(n): u[i]=math.exp(v[i]) Andrew Ng Logistic regression derivatives J = 0, dw1 = 0, dw2 = 0, db = 0 for i = 1 to n: ! (") = " $ # (") + % &(") = '(! (") ) * += − - (") log -1 " + (1 − - " ) log(1 − -1 " ) d! (") = &(") (1 − &(") ) (") d"% += #% d! (") (") d"' += #' d! (") db += d! (") J = J/m, d"% = d"% /m, d"' = d"' /m, db = db/m Andrew Ng Basics of Neural Network Programming deeplearning.ai Vectorizing Logistic Regression Vectorizing Logistic Regression ! (#) = & ' ( (#) + * +(#) = ,(! (#) ) ! (-) = & ' ( (-) + * ! (.) = & ' ( (.) + * +(-) = ,(! (-) ) +(.) = ,(! (.) ) Andrew Ng Basics of Neural Network Programming deeplearning.ai Vectorizing Logistic Regression’s Gradient Computation Vectorizing Logistic Regression Andrew Ng Implementing Logistic Regression J = 0, d!! = 0, d!" = 0, db = 0 for i = 1 to m: " ($) = ! & # ($) + % &($) = '(" ($) ) * += − - ($) log & $ + (1 − - $ ) log(1 − & $ ) d" ($) = &($) −- ($) ($) d!! += #! d" ($) ($) d!" += #" d" ($) db += d" ($) J = J/m, d!! = d!! /m, d!" = d!" /m db = db/m Andrew Ng Basics of Neural Network Programming deeplearning.ai Broadcasting in Python Broadcasting example Calories from Carbs, Proteins, Fats in 100g of different foods: Carb Protein Fat Apples Beef 56.0 1.2 1.8 0.0 104.0 135.0 Eggs Potatoes 4.4 68.0 52.0 8.0 99.0 0.9 cal = A.sum(axis = 0) percentage = 100*A/(cal.reshape(1,4)) Broadcasting example 1 2 3 4 100 + 1 4 2 5 3 6 1 4 2 5 3 6 + + 100 200 100 200 101 102 103 104 = 300 = 101 104 202 205 303 306 = 101 204 102 205 103 206 General Principle Basics of Neural Network Programming deeplearning.ai Explanation of logistic regression cost function (Optional) Logistic regression cost function Andrew Ng Logistic regression cost function If If $ = 1: $ = 0: ( $ ) = $* ( $ ) = 1 − $* Andrew Ng Cost on m examples Andrew Ng