Slides for Introduction to Stochastic Search and Optimization (ISSO) by J. C. Spall APPENDIX B SOME BASIC TESTS IN STATISTICS •Organization of appendix in ISSO –Standard one-sample test •P-values •Confidence intervals –Basic two-sample tests •Matched pairs t-test •Unmatched pairs t-test with identical variances •Unmatched pairs t-test with nonidentical variances –Other approaches to testing •One- and two-sample tests important in stochastic search, optimization, and Monte Carlo simulation The Standard One-Sample Test • One set of data {Xi } for testing on E(Xi) • Famous test statistics z X μ or t X μ σ n s n • z and t have a N(0, 1) and t-distribution, respectively • t-statistic useful in small samples; both z and t often used with non-normal samples • Large values of |z| or |t | indicate rejection of null hypothesis that is some chosen value (commonly = 0) B-2 P-Values • P-value: Probability that future experiment would have value of test statistic at least as extreme as that observed in the current experiment • Provides info. beyond binary accept/reject null hypothesis – Useful as indicator of strength of rejection • Example: If z = 2.15, P-value is 0.016 based on null hypothesis that 0 – Fairly strong evidence that > 0 B-3 Two-Sample Tests • Two sets of data {Xi } and {Yi } for testing X = Y – E.g., Xi and Yi represent simulation outputs under two scenarios • Generic test statistic form t X Y () where () denotes appropriate variance estimate • Three basic categories of tests affecting () – matched pairs 2) – unmatched pairs; identical variances ( 2X Y 2 – unmatched pairs; non-identical variances ( 2X Y ) • Large values of |t | indicate rejection of null hypothesis that X = Y B-4