advertisement

Data analysis Ben Graham MA930, University of Warwick October 13, 2015 Ch4: Joint and Marginal Distributions Def 4.1.1nAn n-dimensional random vector is a function n X = (Xi )i from sample space S to R . n = 2; roll two dice =1 I I I rnorm(10) rbinom(10,5,0.5) Discrete case: Joint p.m.f. fX (x , . . . , xn): 1 P[X ∈ A] = X x =(x1 ,...,xn )∈A fX (x ) Continuous case: Joint p.d.f. fX (x , . . . , xn): 1 P[X ∈ A] = ˆ x =(x1 ,...,xn )∈A fX (x )dx 1 . . . dxn Marginal distributions Discrete case: pmf fX Y : R , fX (x ) = 2 →R . X y :fX ,Y (x ,y )>0 Continuous case: pdf fX Y : R , fX (x ) = 2 ˆ fX ,Y (x , y ) . →R fX ,Y (x , y )dy Example Discrete I I I I X , Y ∈ {1, . . . , 6} independent dice rolls Z =X +Y p.m.f. fX ,Y p.m.f. fX ,Z Example Continuous I I I I X , Y ∼ N (0p, 1) i.i.d.r.v Z = ρX + 1 − ρ Y ∼ N (0, 1) fX ,Y (x , y ) = fX (x )fY (y ) fX ,Z (x , z ) = fX (x )fZ |X (z | x ) [(Z | X ) ∼ N (ρX , 1 − ρ 2 bivariate normal distribution 2 )] 4.2 Conditional Distributions and Independence Random variables X and Y are independent if for all x , y , {X < x } and {Y < y } are independent fX Y (x , y ) = fX (x )fY (y ) [continuous p.d.f.s or discrete p.m.f.s] φX Y (t ) = φX (t )φY (t ) [characteristic functions] Examples X , Y ∼ Bernoulli (1/2) X , Y ∼ N (0, 1) Independence →Covariance=0 I I I , + I I I E[XY ] = E[X ]E[Y ] Covariance=0 6→independence X ∼ N (0, 1) Y ∈ {−1, +1} independent of X Cov(X,XY)=0 I I I Sums of Normal distributions Example 4.3.4. I I I I X ∼ N (µX , σX ) Y ∼ N (µY , σY ) X , Y independent Then X + Y ∼ N (µX + µY , σX + σY ). 2 2 2 Random sample I I X 2 , . . . , Xn ∼ N (µ, σ 2 ) P 2 i Xi ∼ N (nµ, nσ ) Then 1 I X̄ = Z = X Xi i I n ∼ N (µ, σ 2 /n) X Xi − µ √ ∼ N( , ) σ n i 01 Sums of Poissons Theorem 4.3.2 X ∼Poisson(θ) Y ∼Poisson(λ) X , Y independent. Then X + Y ∼Poisson(θ + λ) Ex 4.4.1 Conversely Y ∼Poisson(λ) X | Y ∼Bin(Y , p) Then X ∼Poisson(λp) Random Samples X , . . . , Xn ∼Poisson(λ) i.i.d.r.v. P i Xi ∼Poisson(nλ) ≈ N (nλ, nλ) I I I I I I I I 1 I I X̄ ∼≈ N (λ, λ/n) Covariance and correlation Def 4.5.1 Covariance: Cov (X , Y ) = E [(X − EX )(Y − EY )] = E[XY ] − EX EY Def 4.5.2 Correlation Cov (X , Y ) Var (X )Var (Y ) Thm 4.5.6 If X and Y are r.v. and a, b ∈ R, then ρXY = p Var (aX + bY ) = a Var (X ) + b Var (Y ) + 2abCov (X , Y ) 2 Special case X , Y independent. 2 Def 4.5.10 Bivariate distribution I µX , µY ∈ R I σx , σY > I pdf 0 fX ,Y (x , y ) I Or pdf 1 2 1 exp 2 1 2 p − ρ2 )−1 = ( πσX σY x − µ 2 x × ( − ρ2 ) σX x − µ Y − µ Y − µ 2 x Y + Y − ρ σX σY σY f (x ) = p k = 2, x 1 exp (2π)k |Σ| ∈ Rk , µ = µx µY − (x − µ)Σ−1 (x − µ) , Σ= σX2 ρσX σY ρσX σY σY2 Def 4.6.2 Multinomial distribution I I I I m trials / repeat events n possible outcomes, probabilities p , . . . , pn sum to one. Let xi ∈ {0, 1, . . . , n} count the number of type-i outcomes 1 joint pmf f (x 1 I I , . . . , xn ) = x 1 m! ! . . . xn ! px1 . . . pnxn 1 where X xi = m i Negative correlations Cov (Xi , Xj ) = −mpi pj (i 6= j ) Modeling tables of categorical variables. Ch5 Random samples I I I I X , . . . , Xn is called a random sample of size n from population f (x ) if they are independent, identically distributed random variables (i.i.d.r.v.) with marginal distribution function f (x ). 1 Think of them as being a random sample from a population that is much larger than n. The probability distribution represents the true distribution of values in the larger population. Condorcet's Jury Principle: plot(sapply(seq(0, 1, 0.01), function(p) pbinom(6, 12, p))) Getting representative samples can be hard, i.e. telephone surveys. I I I I Does everyone have a landline? Does everyone choose to talk to strangers phoning you up during dinner? Are people shy about expressing some preferences? http://www.bbc.co.uk/news/uk-politics-33228669 Parameters I I Parameter θ controlling the distribution f (x | θ) Frequentist statistics: I I I Bayesian: I I I θ is xed but unknown Choose θ̂ = θ̂(data) to estimate θ. Joint distribution f (θ)f (x | θ) f (θ) is the prior distribution Example. I I I Exponential f (x | θ) = θ exp(−θ Q x ). P Joint distribution f (x | θ) = =1 f (x | θ) = θ exp(−θ N.B. f (x | θ)only depends on the x via their sum. n n i i i i x) i Statistics Def 5.2.1 Let X , . . . , Xn ∼ f (x | θ) Let T (x , . . . , xn) denote some function of the data, i.e. T : Rn → R. T is a statistic I 1 I 1 I I I I I I T (x1 , . . . , x T (x1 , . . . , x n n ) = x1 ) = max mean, median, etc. i x i , Var(X ), etc, are not statistics. The probability distribution of T is called the sampling distribution of T . θ,EX Common statistics I I i.i.d.r.v. Sample mean X 1 , . . . , Xn X̄ I = 1 + · · · + Xn n = n X i =1 Xi Sample variance " n # n X X 1 1 S = (Xi − X̄ ) = Xi − nX̄ n−1 i n−1 i If the mean EXi and variance Var(Xi ) exist, then these are unbiased estimates. 2 2 =1 I X 2 =1 2 *Def 5.4.1 Order Statistics I The order statistics of a random sample X , . . . , Xn are the values placed into increasing order: X , . . . , X n . 1 I I I I I I (1) X(1) = min X X(2) =second smallest i ( ) i ... X( ) = max X The sample range is R = X n − X The sample median is n i i ( ) M= (1) X((n+ )/ ) n odd X(n/ ) + X(n/ + ) n even ( 1 2 2 1 1 2 2 2 1 The median may give a better sense of what is typical than the mean. Quantiles I For p ∈ [0, 1], the p quantile is (R, method 7 of 9) (1−γ)x j +γ x j , (n−1)p < j ≤ (n−1)p +1, γ = np +1−p −j Trimmed/truncated mean () I I I I I I p ∈ [0, 1] ( +1) Remove the smallest and biggest np items from the sample Take the mean of what is left i.e. LIBOR, 18 banks, top and bottom 4 removed Cauchy location parameter Theorem 5.4.4 Distribution of the order statistics I I Sample of size n with c.d.f. FX and pdf fX Binomial distribution: n X n FX j (x ) = [F (x )]k [1 − FX (x )]n k X ( ) I Dierentiate: fX(j ) (x ) = −k k =j n! fX (x ) [FX (x )]j − (j − 1)!(n − j )! 1 1 [ − FX (x )]n−j . More sample mean I Characteristic function φX̄ (t ) = [φX (t /n)]n ≈ I + o (t /n) Thm 5.3.1: If Xi ∼ N (µ, σ ), then 2 I I I I 1 + it EnX X̄ and S 2 are independent X̄ ∼ N (µ, σ2 /n). (n − 1)S 2 /σ 2 ∼ χ2−1 Proof: For simplicity, assume µ = 0 and σ = 1. n 2 n Ingredients for the proof I If e , . . . , en form an orthonormal basis for Rn. n Then (ei · X )i are i.i.d.r.v N (0, 1). Let e = ( n , . . . , n ) The Γ(α, β) distribution is dened f (x ) = C (α, β)x exp(−β x ). The χk distribution is the Γ(k /2, 1/2) distribution. If X ∼ N (0, 1), then X ∼ χ The sum of k independent χ r.v. is χk . 1 =1 I I √1 1 √1 α−1 I I I 2 2 2 1 2 1 2 Derived distributions I If Z ∼ N (0, 1) and V ∼ χk then Z /pV /k ∼ tk (Student's t) 2 I I I X̄ −√µ ∼t S / n n− If U ∼ χk and V ∼ χ are independent, then UV k ∼ Fk Suppose X , . . . , Xm ∼ N (µX , σX ) and Y , . . . , Yn ∼ N (µY , σY ). Then 2 / /` 2 ` ,` 2 1 1 2 SX /σX SY /σY I 1 2 2 2 2 ∼ Fm−1,n−1 These distributions are also used for linear regression/ANOVA.