Covariance
Covariance
Definition
The covariance between two rv’s X and Y is
Cov (X , Y ) = E [(X − µX )(Y − µY )]
(P P
y (x − µX )(y − µY )p(x, y )
= R ∞x R ∞
−∞ −∞ (x − µX )(y − µY )f (x, y )dxdy
X , Y discrete
X , Y continuou
Covariance
Definition
The covariance between two rv’s X and Y is
Cov (X , Y ) = E [(X − µX )(Y − µY )]
(P P
y (x − µX )(y − µY )p(x, y )
= R ∞x R ∞
−∞ −∞ (x − µX )(y − µY )f (x, y )dxdy
X , Y discrete
X , Y continuou
Remark: The covariance depends on both the set of possible pairs
and the probabilities.
Covariance
Definition
The covariance between two rv’s X and Y is
Cov (X , Y ) = E [(X − µX )(Y − µY )]
(P P
y (x − µX )(y − µY )p(x, y )
= R ∞x R ∞
−∞ −∞ (x − µX )(y − µY )f (x, y )dxdy
X , Y discrete
X , Y continuou
Remark: The covariance depends on both the set of possible pairs
and the probabilities.
Proposition
Cov (X , Y ) = E (XY ) − µX · µY
Covariance
Covariance
Example (Problem 75 revisit)
A restaurant serves three fixed-price dinners costing $12, $15, and
$20. For a randomly selected couple dinning at this restaurant, let
X = the cost of the man’s dinner and Y = the cost of
the woman’s dinner. If the joint pmf of X and Y is assumed to
be
y
p(x, y )
12 15 20
12 .05 .05 .10
x
15 .05 .10 .35
20 0 .20 .10
Covariance
Example (Problem 75 revisit)
A restaurant serves three fixed-price dinners costing $12, $15, and
$20. For a randomly selected couple dinning at this restaurant, let
X = the cost of the man’s dinner and Y = the cost of
the woman’s dinner. If the joint pmf of X and Y is assumed to
be
y
p(x, y )
12 15 20
12 .05 .05 .10
x
15 .05 .10 .35
20 0 .20 .10
Cov (X , Y ) = E (XY ) − µX · µY = 276.7 − 15.9 · 17.45 = −0.755
Covariance
Covariance
If we change the unit for the previous example from dollar to cent,
then the joint pmf would be
y
1200 1500 2000
p(x, y )
1200 .05
.05
.10
.10
.35
x
1500 .05
0
.20
.10
2000
Covariance
If we change the unit for the previous example from dollar to cent,
then the joint pmf would be
y
1200 1500 2000
p(x, y )
1200 .05
.05
.10
.10
.35
x
1500 .05
0
.20
.10
2000
And correspondingly,
Cov (X , Y ) = E (XY ) − µX · µY = 7550
Correlation
Correlation
Definition
The correlation coefficient of X and Y , denoted by Corr (X , Y ),
ρX ,Y or just ρ is defined by
ρX ,Y =
Cov (X , Y )
σX · σY
Correlation
Definition
The correlation coefficient of X and Y , denoted by Corr (X , Y ),
ρX ,Y or just ρ is defined by
ρX ,Y =
Cov (X , Y )
σX · σY
e.g. for the previous example, the correlation coefficient of X and
Y is
−0.755
ρ=
= −0.09
2.91 · 2.94
Correlation
Correlation
Proposition
1. Corr (aX + b, cY + d) = Corr (X , Y ) if a · c > 0.
2. −1 ≤ Corr (X , Y ) ≤ 1.
3. ρ = 1 or −1 iff Y = aX + b for some a and b with a 6= 0.
4. If X and Y are independent, then ρ = 0. However, ρ = 0
does not imply that X and Y are independent
Statistics and Their Distributions
Statistics and Their Distributions
Assume we are running an online retail store. Some factors may be
of great interest to us. Like the distributions of buyers nation-wide,
the time a customer spend on the website per each visit,
costumers’ satisfactories and etc.
Statistics and Their Distributions
Assume we are running an online retail store. Some factors may be
of great interest to us. Like the distributions of buyers nation-wide,
the time a customer spend on the website per each visit,
costumers’ satisfactories and etc.
For some factors, we may get the data for the “whole population”,
like the time spent per each visit. While for others, we can only
obtain information of a sample, like costumers’ satisfactories.
Statistics and Their Distributions
Assume we are running an online retail store. Some factors may be
of great interest to us. Like the distributions of buyers nation-wide,
the time a customer spend on the website per each visit,
costumers’ satisfactories and etc.
For some factors, we may get the data for the “whole population”,
like the time spent per each visit. While for others, we can only
obtain information of a sample, like costumers’ satisfactories.
If we take into account of the future customers, we are unable to
get the information about the population theoretically. All the
information we are dealing with now is just from a sample.
Statistics and Their Distributions
Statistics and Their Distributions
For example, we have the following data about the time spent per
each visit (in min.) from a sample of size 10:
1
2
3
4
5
time 24 51 12 95 26
6
7
8
9 10
time 5 33 62 31 27
Statistics and Their Distributions
For example, we have the following data about the time spent per
each visit (in min.) from a sample of size 10:
1
2
3
4
5
time 24 51 12 95 26
6
7
8
9 10
time 5 33 62 31 27
Each observation is “random”, i.e. we can not predict the exact
value before we obtain the observation.
Statistics and Their Distributions
For example, we have the following data about the time spent per
each visit (in min.) from a sample of size 10:
1
2
3
4
5
time 24 51 12 95 26
6
7
8
9 10
time 5 33 62 31 27
Each observation is “random”, i.e. we can not predict the exact
value before we obtain the observation.
Therefore we can associate a random variable Xi to the ith
observation.
Statistics and Their Distributions
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
the mean is 36.6;
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
the mean is 36.6;
the medain is 29;
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
the mean is 36.6;
the medain is 29;
and the standard deviation is 26.4.
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
the mean is 36.6;
the medain is 29;
and the standard deviation is 26.4.
Sometimes, these characteristics are more interesting to us than
the sample data itself.
Statistics and Their Distributions
Often, we are interested in some overall properties of the sample.
For example, the maximum of the previous sample is 95;
the minimum is 5;
the mean is 36.6;
the medain is 29;
and the standard deviation is 26.4.
Sometimes, these characteristics are more interesting to us than
the sample data itself.
We call these characteristics statistics.
Statistics and Their Distributions
Statistics and Their Distributions
Definition
A statistic is any quantity whose value can be calculated from
sample data.
Statistics and Their Distributions
Definition
A statistic is any quantity whose value can be calculated from
sample data.
Remark:
1. A statistic is a random variable.
Statistics and Their Distributions
Definition
A statistic is any quantity whose value can be calculated from
sample data.
Remark:
1. A statistic is a random variable. The reason is prior to
obtaining data, we are not sure what value of any particular
statistic will result.
Statistics and Their Distributions
Definition
A statistic is any quantity whose value can be calculated from
sample data.
Remark:
1. A statistic is a random variable. The reason is prior to
obtaining data, we are not sure what value of any particular
statistic will result.
We use uppercase letters to denote statistics and lowercase
letter to denote the calculated or observed values of statistics.
Statistics and Their Distributions
Statistics and Their Distributions
Remark:
2. A statistic must be calculated from sample data.
Statistics and Their Distributions
Remark:
2. A statistic must be calculated from sample data.
For example, if in addition to the size 10 sample for the
previous example, we also know that the time spent per each
visit is normally distributed with mean µ and variance σ 2 ,
then neither the population mean µ nor the population
variance σ 2 is a statistic.
Statistics and Their Distributions
Remark:
2. A statistic must be calculated from sample data.
For example, if in addition to the size 10 sample for the
previous example, we also know that the time spent per each
visit is normally distributed with mean µ and variance σ 2 ,
then neither the population mean µ nor the population
variance σ 2 is a statistic.
While the sample mean and the sample variance are two valid
statistics, which will be denoted by X and S 2 , respectively.
Statistics and Their Distributions
Remark:
2. A statistic must be calculated from sample data.
For example, if in addition to the size 10 sample for the
previous example, we also know that the time spent per each
visit is normally distributed with mean µ and variance σ 2 ,
then neither the population mean µ nor the population
variance σ 2 is a statistic.
While the sample mean and the sample variance are two valid
statistics, which will be denoted by X and S 2 , respectively.
3. Any statistic, being a random variable, has a probability
distribution.
The probability distribution of a statistic is referred to as its
sampling distribution.
Statistics and Their Distributions
Remark:
2. A statistic must be calculated from sample data.
For example, if in addition to the size 10 sample for the
previous example, we also know that the time spent per each
visit is normally distributed with mean µ and variance σ 2 ,
then neither the population mean µ nor the population
variance σ 2 is a statistic.
While the sample mean and the sample variance are two valid
statistics, which will be denoted by X and S 2 , respectively.
3. Any statistic, being a random variable, has a probability
distribution.
The probability distribution of a statistic is referred to as its
sampling distribution.
The sampling distribution of a statistic DEPENDS on the
sample size n.
Statistics and Their Distributions
Statistics and Their Distributions
Definition
The random variables X1 , X2 , . . . , Xn are said to form a (simple)
random sample of size n if
1. The Xi s are independent random variables.
2. Every Xi has the same probability distribution.
Statistics and Their Distributions
Definition
The random variables X1 , X2 , . . . , Xn are said to form a (simple)
random sample of size n if
1. The Xi s are independent random variables.
2. Every Xi has the same probability distribution.
In words, X1 , X2 , . . . , Xn forms a random sample if the Xi ’s are
independent and identically distributed (iid).
Statistics and Their Distributions
Statistics and Their Distributions
Remark:
When sampling with replacement or from an infinite (conceptual)
population, the two conditions are satisfied and the result can be
regarded as a random sample.
Statistics and Their Distributions
Remark:
When sampling with replacement or from an infinite (conceptual)
population, the two conditions are satisfied and the result can be
regarded as a random sample.
For sampling WITHOUT replacement from a finite population,
although consecutive observations are not independent and
identically distributed, we can still regard the result as a random
sample if the sample size n is much smaller than the population
size N.
Statistics and Their Distributions
Remark:
When sampling with replacement or from an infinite (conceptual)
population, the two conditions are satisfied and the result can be
regarded as a random sample.
For sampling WITHOUT replacement from a finite population,
although consecutive observations are not independent and
identically distributed, we can still regard the result as a random
sample if the sample size n is much smaller than the population
size N.
In practice, if n/N ≤ .05 (at most .05% of the population is
sampled), we can regard the sample as a random sample.