Visualization and Operations
Learning Outcomes
 Appreciate the need for joint probability distributions
 Visualize joint probability distributions in different formats
 Perform different operations on joint probability distributions
Outline
 Introduction to Joint Probability distributions
 Visualization of Joint probability distributions
 Operations on Joint Probability Distributions
 Marginalization
 Conditional probability
 Independence
Joint Probability
We are concerned about the probability of shaded bit
𝑃(𝑋 ∩ 𝑌)
X
Event : Female
Y
Event: Executive
Recall: Probability distribution for a single
random variable
• Probability is the relative frequency of seeing a particular outcome in an
event
Throw a die
• Probability distribution represents the relative frequency of different
outcomes of an event
5
Joint Probability
 Consider two random variables x and y
 If we observe multiple paired instances, then some combinations of
outcomes are more likely than others
 This is captured in the joint probability distribution
 Written as Pr(x,y)
 Can read Pr(x,y) as “probability of x and y”
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
6
2
Throw 2 dice simultaneously
1
Die A
Die B
3 4
5
Die B
6
Probability distribution of throwing 2 dice
1
2
3
4
Die A
5
6
Joint Probability Distributions
Humidity
172 cm
Height
70kg
160 cm
Weight
Temperature
Average Sales per day
Items
Day of the Week
Temperature
Month
80kg
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Month
8
Probability distribution of possible bigrams
Outline
 Introduction to Joint Probability distributions
 Visualization of Joint probability distributions
 Operations on Joint Probability Distributions
 Marginalization
 Conditional probability
 Independence
Exercise: Joint Probability
 Consider the following table of a retailer who has collected the sales statistics across the months.
 Calculate the probability distribution of the total sales across different months.
 Calculate the probability distribution of the products
P\M
Jan Feb
Mar Apr
May Jun
Jul
Aug Sep
Oct
Nov
Dec
P1
32
5
0
47
0
0
0
12
24
45
56
61
P2
23
23
0
0
0
0
0
1
4
12
32
67
P3
12
35
0
1
0
1
0
0
1
4
7
9
P4
3
10
7
12
3
2
1
12
3
12
23
3
P5
1
5
43
23
13
5
12
2
12
23
53
76
P6
0
1
35
45
54
32
25
32
12
3
0
0
P7
0
1
20
10
13
10
7
2
1
5
6
13
Answer: The PD of the total sales across
different months
P\M
Jan Feb
Mar Apr
May Jun
Jul
Aug
Sep
Oct
Nov
Dec
P1
32
5
0
47
0
0
0
12
24
45
56
61
P2
23
23
0
0
0
0
0
1
4
12
32
67
P3
12
35
0
1
0
1
0
0
1
4
7
9
P4
3
10
7
12
3
2
1
12
3
12
23
3
P5
1
5
43
23
13
5
12
2
12
23
53
76
P6
0
1
35
45
54
32
25
32
12
3
0
0
P7
0
1
20
10
13
10
7
2
1
5
6
13
71 80 105 138 83 50 45 61 57 104 177 229
Answer: The PD of the total sales across
different months
 The attribute of probability:
 The probabilities of all
possibilities should add up to 1

probability
0.059 0.067 0.088 0.115 0.069 0.042 0.038 0.051 0.048 0.087 0.148 0.191
or
 Therefore, we divide by the total
number of events
71 80 105 138 83 50 45 61 57 104 177 229
1200
Alternative method: convert to a joint
probability distribution:
P\M
Jan Feb
Mar Apr
May Jun
Jul
Aug Sep
Oct
Nov Dec
32
5
0
47
0
0
0
12
24
45
56
61
23
23
0
0
0
0
0
1
4
12
32
67
12
35
0
1
0
1
0
0
1
4
7
9
3
10
7
12
3
2
1
12
3
12
23
3
1
5
43
23
13
5
12
2
12
23
53
76
0
1
35
45
54
32
25
32
12
3
0
0
0
1
20
10
13
10
7
2
1
5
6
13
P1
0.027 0.004 0.000 0.039 0.000 0.000 0.000 0.010 0.020 0.038 0.047 0.051
P2
0.019 0.019 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.010 0.027 0.056
P3
0.010 0.029 0.000 0.001 0.000 0.001 0.000 0.000 0.001 0.003 0.006 0.008
P4
0.003 0.008 0.006 0.010 0.003 0.002 0.001 0.010 0.003 0.010 0.019 0.003
P5
0.001 0.004 0.036 0.019 0.011 0.004 0.010 0.002 0.010 0.019 0.044 0.063
P6
0.000 0.001 0.029 0.038 0.045 0.027 0.021 0.027 0.010 0.003 0.000 0.000
P7
0.000 0.001 0.017 0.008 0.011 0.008 0.006 0.002 0.001 0.004 0.005 0.011
0.059 0.067 0.088 0.115 0.069 0.042 0.038 0.051 0.048 0.087 0.148 0.191
67/1200 = 0.056
53/1200 = 0.044
Visualizing the joint distribution of
7
6
Product ID
5
4
3
2
1
1
2
3
4
5
6
7
Month
8
9
10
11
12
Answer: The PD of the total sales across
different months
P\M
Jan Feb
Mar Apr
May Jun
Jul
Aug
Sep
Oct
Nov
Dec
P1
0.027 0.004 0.000 0.039 0.000 0.000 0.000 0.010 0.020 0.038 0.047 0.051
P2
0.019 0.019 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.010 0.027 0.056
P3
0.010 0.029 0.000 0.001 0.000 0.001 0.000 0.000 0.001 0.003 0.006 0.008
P4
0.003 0.008 0.006 0.010 0.003 0.002 0.001 0.010 0.003 0.010 0.019 0.003
P5
0.001 0.004 0.036 0.019 0.011 0.004 0.010 0.002 0.010 0.019 0.044 0.063
P6
0.000 0.001 0.029 0.038 0.045 0.027 0.021 0.027 0.010 0.003 0.000 0.000
P7
0.000 0.001 0.017 0.008 0.011 0.008 0.006 0.002 0.001 0.004 0.005 0.011
0.059 0.067 0.088 0.115 0.069 0.042 0.038 0.051 0.048 0.087 0.148 0.191
Answer: Exercise
P\M
Jan Feb
Mar Apr
May Jun
Jul
Aug
Sep
Oct
Nov Dec
P1
0.027 0.004 0.000 0.039 0.000 0.000 0.000 0.010 0.020 0.038 0.047 0.051
0.235
P2
0.019 0.019 0.000 0.000 0.000 0.000 0.000 0.001 0.003 0.010 0.027 0.056
0.135
P3
0.010 0.029 0.000 0.001 0.000 0.001 0.000 0.000 0.001 0.003 0.006 0.008
0.058
P4
0.003 0.008 0.006 0.010 0.003 0.002 0.001 0.010 0.003 0.010 0.019 0.003
0.076
P5
0.001 0.004 0.036 0.019 0.011 0.004 0.010 0.002 0.010 0.019 0.044 0.063
0.223
P6
0.000 0.001 0.029 0.038 0.045 0.027 0.021 0.027 0.010 0.003 0.000 0.000
0.199
P7
0.000 0.001 0.017 0.008 0.011 0.008 0.006 0.002 0.001 0.004 0.005 0.011
0.073
Note we are trying to add along the margins along the table!
We call this operation on joint PDFs: Marginalization
Answer: The PD of different products


probability
 Check if all the
probabilities add up to one
• Reflect how we got
)
Add over the months
Marginalization
We can recover probability distribution of any variable in a joint
distribution by integrating (or summing) over the other variables
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
19
Marginalization
We can recover probability distribution of any variable in a joint
distribution by integrating (or summing) over the other variables
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
20
Sum rule: defined on joint probability
distributions
 Follows from the definition of marginal probability
Outline
 Introduction to Joint Probability distributions
 Visualization of Joint probability distributions
 Operations on Joint Probability Distributions
 Marginalization
 Conditional probability
 Independence
Exercise: Conditional Probability
 Calculate the probability distribution of product sales for the month of
April.
Answer: Exercise
P\M
P1
P2
P3
P4
P5
P6
P7
Jan Feb
Mar Apr
May Jun
Jul
Aug
Sep
Oct
Nov
Dec
0.027
0.004
0.000
0.039
0.000
0.000
0.000
0.010
0.020
0.038
0.047
0.051
0.019
0.019
0.000
0.000
0.000
0.000
0.000
0.001
0.003
0.010
0.027
0.056
0.010
0.029
0.000
0.001
0.000
0.001
0.000
0.000
0.001
0.003
0.006
0.008
0.003
0.008
0.006
0.010
0.003
0.002
0.001
0.010
0.003
0.010
0.019
0.003
0.001
0.004
0.036
0.019
0.011
0.004
0.010
0.002
0.010
0.019
0.044
0.063
0.000
0.001
0.029
0.038
0.045
0.027
0.021
0.027
0.010
0.003
0.000
0.000
0.000
0.001
0.017
0.008
0.011
0.008
0.006
0.002
0.001
0.004
0.005
0.011
0.115
Answer: The PD of product sales for the month
of April
 The question is asking for a conditional probability: Pr(product|Month = 4);
Apr
0.000
0.001
0.010
0.019
0.038
0.008
0.115
=
probability
0.039
Conditional Probability
 Conditional probability of x given that y=y1 is relative
propensity of variable x to take different outcomes given
that y is fixed to be equal to y1.
 Written as Pr(x|y=y1)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
26
Conditional Probability
 Conditional probability can be extracted from joint
probability
 Extract appropriate slice and normalize
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
27
Exercise: Joint Probability
 Determine if product type and month are independent.
 What is the condition for independence?
 Remember from product rule:

Pr(X,Y) = Pr(X|Y) Pr(Y)
 If Independent:

Pr(X|Y) = Pr(X) -> given Y has occurred doesn’t give any information about X
 This follows: Pr(X,Y) = Pr(X) Pr(Y)
Answer I: Check Pr(X|Y) and Pr(X)
 If they are the same
 Y doesn’t provide any information related to X.
 If not : X depends on Y
Pr(product|Month = 4);
probability
probability
Pr(product);
Answer II: Based on P(X,Y)
 If P(X,Y) is equal to P(X)*P(Y)
 X and Y can be deemed independent
Outline
 Introduction to Joint Probability distributions
 Visualization of Joint probability distributions
 Operations on Joint Probability Distributions
 Marginalization
 Conditional probability
 Independence
Independence
 If two variables x and y are independent then variable x
tells us nothing about variable y (and vice-versa)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
32
Independence
 If two variables x and y are independent then variable x
tells us nothing about variable y (and vice-versa)
Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
33
Learning Outcomes …
Did you learn anything?
 Appreciate the need for joint probability distributions
 Visualize joint probability distributions in different formats
 Perform different operations on joint probability distributions