Artificial Intelligence CSC462 Dr. Muhammad Humayoun Assistant Professor COMSATS Institute of Computer Science, Lahore. [email protected] Course homepage: https://sites.google.com/a/ciitlahore.edu.pk/ai13/ Modified slides of AI course on Udacity, etc. 1 Machine Learning Chapter 18 2 What we saw • • • • • Bayes Network = reasons with known models Machine learning = Learns models from data Unsupervised Learning (Today’s topic) Supervised Learning (Next topic) ML is a very large field with many different methods and many different applications 3 Taxonomy What? • Parameters: like the probabilities of a Bayes Network. • Structure: like the arc structure of a Bayes Network. • Hidden concepts that makes better sense of data: For example you might find that certain training example form a hidden group. 4 Taxonomy What from? Every ML method is driven by some sort of target information that you care about • Supervised learning: we have target labels • Unsupervised learning: target labels are missing and we use replacement principles to find, for example hidden concepts • Reinforcement learning: an agent learns from feedback with the physical environment by interacting and trying actions and receiving some sort of evaluation from the environment 5 Taxonomy What for? • Prediction: what's going to happen with the future in the stockmarket for example. • Diagnostics • Summarization • … 6 Taxonomy How to learn? • Passive: if your learning agent is just an observer and has no impact on the data itself. • Active: Otherwise, its active. • Online: learning occurs when the data is being generated • Offline: learning occurs after the data has been generated 7 Taxonomy Outputs? • Classification: the output is binary or a fixed number of classes. Ex. something is either a chair or not. • Regression is continuous. Ex. Tomorrow’s temperature might be 13 degrees in our prediction. 8 Taxonomy Internal details? • Generative: seeks to model the data as generally as possible • Discriminative seek to distinguish data (this might sound like a superficial distinction but it has enormous ramification on the learning algorithm) It has taken us many years to fully learn all these words. So don't expect to pick them all up in one class 9 Unsupervised Learning • We just have a data matrix of data items of N features each, with M records • Task of unsupervised learning is to find structure in data of this type 10 Warm-up Quiz • • • • Is there any structure in these data items? Yes. Data does not seem random. How many groups? 2 11 Another Quiz • What is the dimensionality of the space? • 2 • How many dimensions are needed intuitively??? • 1 • Important technique • Dimensionality Reduction • Useful in image resolution reduction 12 (Identically distributed and independently drawn) 13 Google street view 14 Google street view • A huge photographic database of many, many streets in the world. • Ground imagery of almost any location in the world • Vast regularities in these images • Homes, trees, cars, signboards, etc • So one of the fascinating, unsolved, unsupervised learning tasks is: – Given hundreds of billions of images as comprised in the Street View data set can we discover concepts such as trees, lane markers, stop signs, cars, and pedestrians? 15 16 17 18 K-means • K-means (MacQueen, 1967) is one of the simplest unsupervised learning algorithms that solve the well known clustering problem. • The procedure follows a simple and easy way to classify a given data set through a certain number of fixed clusters (assume k clusters). 19 20 21 22 23 24 25 26 27 Calculations 28 Quiz 29 Quiz 30 Quiz 31 32 33 34 35 36 37 38 39 The K-means Algorithm for Clustering kmeans(D, k) choose K initial means randomly (e.g., pick K points randomly from D) while means_are_changing % assign each point to a cluster for i = 1: N membership[x(i)] = cluster with mean closest to x(i) end % update the means for k = 1:K mean_k = average of vectors x(i) assigned to cluster k end % check for convergence if (new means are the same as old means) then halt else means_are_changing = 1 end 40 41 k-Means: Step-By-Step Example • Consider the following data set consisting of the scores of two variables on each of seven individuals: 42 k-Means: Step-By-Step Example • For K=2, the data set is to be grouped into two clusters. • As a first step in finding a sensible initial partition, let the A & B values of the two individuals furthest apart (using the Euclidean distance measure), define the initial cluster means, giving: 43 k-Means: Step-By-Step Example • The remaining individuals are now examined in sequence and allocated to the cluster to which they are closest, in terms of Euclidean distance to the cluster mean. • The mean vector is recalculated each time a new member is added. This leads to the following series of steps: 44 k-Means: Step-By-Step Example • Now the initial partition has changed, and the two clusters at this stage having the following characteristics: 45 k-Means: Step-By-Step Example • But we cannot yet be sure that each individual has been assigned to the right cluster. • So, we compare each individual’s distance to its own cluster mean and to that of the opposite cluster. And we find: 46 k-Means: Step-By-Step Example • Only individual 3 is nearer to the mean of the opposite cluster (Cluster 2) than its own (Cluster 1). • In other words, each individual's distance to its own cluster mean should be smaller that the distance to the other cluster's mean (which is not the case with individual 3). • Thus, individual 3 is relocated to Cluster 2 resulting in the new partition: 47 k-Means: Step-By-Step Example • The iterative relocation would now continue from this new partition until no more relocations occur. • However, in this example each individual is now nearer its own cluster mean than that of the other cluster and the iteration stops, choosing the latest partitioning as the final cluster solution. • Also, it is possible that the k-means algorithm won't find a final solution. • In this case it would be a good idea to consider stopping the algorithm after a pre-chosen maximum of iterations. 48 http://home.deib.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html 49 End 50