CS188 Discussion Unsupervised Learning Wed 04/11
By Nuttapong Chentanez
Unsupervised learning
- Aka. Clustering
- Detect patterns in unlabeled data
Eg. group emails or search results
- Useful when don’t know what you’re looking for
- Require data but not label
K-Mean algorithm
- Pick K random pints as cluster centers
- Alternate
- Phase 1: Assign data instances to closest mean
- Phase 2: Assign each mean to the average to the assigned points
- Stop when no points’s assignments change
- How many clusters to use?
- Representation of data matter
Agglomerative Cluster
- Algorithm
-Initially, each instance in its own cluster
-Repeat
- Pick two “closest” clusters
- Merge them into a new cluster
- Stop when 1 cluster left
- Produce not just cluster, but a hierarchy of
cluster called dendrogram
- What do closest clusters mean?
- Closest pair, farthest pair, average of all
pairs, distance between centroid ?
Why work?
Think of it as an optimization process
Each iteration reduces φ
Phase1: Assign point to closest mean
This reduces φ or at worst, φ remains the same. Why
Phase 2: Re-compute mean
This reduces φ or at worst, φ remains the same. Why?
Issues with K-means
- Requires initial means
- Can get stuck in local minima
Differences from K-Mean
1. No need to average (for the non-centroid case)
2. Need not be “distance”
- Can use distance, weighted distance, distance between string, any
imaginable similarity test.
Eg. For digits, rotate two images independently
Pick the rotations that make the two images as similar as possible
(measured by pixel-pixel comparison).
Take this difference as the measure of similarity.
Applications of unsupervised learning
Biology
- Group organisms together
- Group genes
Social network
- Recognize communities within large group
Vision
- Image segmentations
- Group similar images together
Many others