Formal neurons I Before we start artificial NN, we look at single neuron first Topics: Neuronal Activity Dendritic tree: receive spikes Soma: generate spikes Axon: sent out spikes Purkinje neuron Pyramidal neuron Neurons Hillock input output Single neuron activity • Membrane potential is the voltage difference between neuron and its surrounding (0 mV) Membrane potential Cell Cell Cell Cell 0 Mv Single neuron activity •If you measure the membrane potential of a neuron and print it out on the screen, it looks like (from time 0 to 60 minutes) spike Single neuron activity •A spike is generated when the membrane potential is greater than its threshold Single neuron activity •We can forget all sub-threshold activity and concentrate on spikes, which are signals sent to other neurons Spikes Spike trains • Sequence of spikes, generated due to simulations (inputs to a neuron) Spikes • Only spikes are important since other neurons receive them (signals) • Neurons communicate with spikes • Information is coded by spikes • If we can manage to measure the spiking time, we decipher how the brain works Life is not so simple • spiking time in the cortex is random With identical input for the identical neuron spike patterns are similar, but not identical Recording from a real neuron: membrane potential Single spiking time is meaningless To extract useful information, we have to average for a group of neurons in a local circuit where neuron codes the same information over a time window to obtain the firing rate r Single spiking time is meaningless To extract useful information, we have to average for a group of neurons in a local circuit where neuron codes the same information over a time window to obtain the firing rate r r = = Local circuit Single spiking time is meaningless To extract useful information, we have to average for a group of neurons in a local circuit where neuron codes the same information over a time window to obtain the firing rate r r = = Local circuit = 6 Hz Time window = 1 sec Hence we have firing rate of a group of neuron Next we turn our attention to how to go from a group of local neurons to neural networks Topics: Neuronal Activity From neurons to neural networks From group of neurons to networks R = f ( w j rj ) r1 w1: synaptic strength wn rn ri is the firing rate of input local circuit The neurons at output local circuits receives signal in the form N wr i =1 i i The output firing rate of the output local circuit is then given by N R R = f ( wi ri ) i =1 where f is the activation function, generally taking as the Sigmoidal or other forms wi weight, (synaptic strength) measuring how strong is the interaction between neurons. Therefore the key for neural networks is to understand the local input-output relationship m yi = f ( wij x j bi ) j =1 A single neuron have 5 components 1. Input x 2. Weight w m yi = f ( wij xj bi ) j =1 A single neuron have 5 components 1. Input x 2. Weight w 3. Bias b m yi = f ( wij xj bi ) j =1 A single neuron have 5 components 1. 2. 3. 4. Input x Weight w Bias b Activation function f m yi = f ( wij xj bi ) j =1 A single neuron have 5 components 1. 2. 3. 4. 5. Input x Weight w Bias b Activation function f Output y m yi = f ( wij xj bi ) j =1 A single neuron have 5 components 1. 2. 3. 4. 5. Input x Weight w Bias b Activation function f Output y y = f ( S xi wi + b ) m yi = f ( wij xj bi ) j =1 Artificial Neural networks Local circuits (average to get firing rates) Single neuron (send out spikes)