advertisement

Verified computation with probabilities Scott Ferson, scott@ramas.com Applied Biomathematics Scientific Computing, Computer Arithmetic and Verified Numerical Computations El Paso, Texas, 3 October 2008 Perspective • Very elementary methods of interval analysis – Low-dimensional, static – Verified computing (but not roundoff error) – Huge uncertainties • Intervals combined with probability theory – Total probabilities (events) – Probability distributions (random variables) Bounding probability is an old idea • • • • • • Boole and de Morgan Chebyshev and Markov Borel and Fréchet Kolmogorov and Keynes Berger and Walley Williamson and Downs Deterministic calculation Probabilistic convolution Second-order probability Interval analysis Probability bounds analysis Terminology • Dependence = stochastic dependence » More general than repeated variables • Independence = stochastic independence • Best possible = tight (almost) » some elements in the set may not be possible Incertitude • Arises from incomplete knowledge • Incertitude arises from – limited sample size – measurement uncertainty or surrogate data – doubt about the model • Reducible with empirical effort Variability • Arises from natural stochasticity • Variability arises from – scatter and variation – spatial or temporal fluctuations – manufacturing or genetic differences • Not reducible by empirical effort They must be treated differently • Variability is randomness Needs probability theory • Incertitude is ignorance Needs interval analysis • Imprecise probabilities can do both at once Risk assessment applications • Environmental pollution heavy metals, pesticides, PMx, ozone, PCBs, EMF, RF, etc. • Engineered systems traffic safety, bridge design, airplanes, spacecraft, nuclear plants $• Financial investments portfolio planning, consultation, instrument evaluation • Occupational hazards manufacturing and factory workers, farm workers, hospital staff • Food safety and consumer product safety benzene in Perrier, E. coli in beef, salmonella in tomato, children’s toys • Ecosystems and biological resources endangered species, fisheries and reserve management Probabilistic logic Total probabilities (events) • Fault or event trees • Logical expressions (Hailperin 1986) • Reliability analyses – – – – Nuclear power plants Aircraft safety system design Gene technology release assessments etc. Interval arithmetic for probabilities x + y = [x1 + y1, x2 + y2] x y = [x1 y2, x2 y1] x y = [x1 y1, x2 y2] x y = [x1 y2, x2 y1] min(x, y) = [min(x1, y1), min(x2, y2)] max(x, y) = [max(x1, y1), max(x2, y2)] Rules are simpler because intervals confined to [0,1] Probabilistic logic • • • • • • Conjunctions (and) Disjunctions (or) Negations (not) Exclusive disjunctions (xor) Modus ponens (if-then) etc. Conjunction (and) P(A |&| B) = P(A) P(B) Example: P(A) = [0.3, 0.5] P(B) = [0.1, 0.2] A and B are independent P(A |&| B) = [0.03, 0.1] Stochastic dependence Independence Probabilities are areas in the Venn diagrams Fréchet case P(A & B)=[max(0, P(A) + P(B)–1), min(P(A), P(B))] • Makes no assumption about the dependence • Rigorous (guaranteed to enclose true value) • Best possible (cannot be any tighter) Fréchet examples Examples: P(A) = [0.3, 0.5] P(B) = [0.1, 0.2] P(A & B) = [0, 0.2] P(C) = 0.29 certain P(D) = 0.22 P(C & D) = [0, 0.22] uncertain } Example: pump system Switch S1 Relay K2 What’s the risk the tank ruptures under pumping? Relay K1 Timer relay R Motor Pressure switch S From reservoir Pump Pressure tank T Outlet valve Fault tree R E5 K2 E2 E1 E4 or E3 or T or K1 or S1 and S Boolean expression of “tank rupturing under pressure” E1 Unsure about the probabilities and their dependencies Results Points, independence Mixed dependencies Fréchet Intervals and Fréchet 105 104 Probability of tank rupturing under pumping 103 Interval probabilities • Allow verified computing of reliabilities • Distinguish two forms of uncertainty • Rigorous bounds always easy to get • Best possible bounds may need mathematical programming because of repeated variables Probabilistic arithmetic Typical problems • Sometimes little or even no actual data – Updating is rarely used • Very simple arithmetic calculations – Occasionally, finite element meshes or differential equations • Usually small in number of inputs – Nuclear power plants are the exception • Results are important and often high profile – But the approach is being used ever more widely Example: pesticides & farmworkers • Total dose is decomposed by pathway – Dermal exposure on hands – Exposures to rest of body – Inhalation (concentration in air, exposure duration, breathing rate, penetration factor, absorption efficiency) • Takes account of related factors – Acres, gallons per acre, mixing time – Body mass, frequency of hand washing, etc. Worst-case analysis • Traditional method • Mix of deterministic and extreme values • Actually a kind of interval analysis • Says how bad it could it be, but not how unlikely that outcome is Probabilistic analysis • State-of-the-art method • Usually via Monte Carlo simulation • Requires the full joint distribution – All the distributions for every input variable – All their intervariable dependencies • Often we need to guess about a lot of it What’s needed • Reliable, conservative assessments of tail risks • Using available information but without forcing analysts to make unjustified assumptions • Neither computationally expensive nor intellectually taxing Probability bounds analysis • Marries intervals with probability theory • Distinguishes variability and incertitude • Solves many problems in uncertainty analysis – Input distributions unknown – Imperfectly known correlation and dependency – Large measurement error, censoring, small sample sizes – Model uncertainty Calculations • All standard mathematical operations – – – – – – Arithmetic operations (+, , ×, ÷, ^, min, max) Logical operations (and, or, not, if, etc.) Transformations (exp, ln, sin, tan, abs, sqrt, etc.) Backcalculation (tolerance solutions) (deconvolutions, updati Magnitude comparisons (<, ≤, >, ≥, ) Other operations (envelope, mixture, etc.) • Faster than Monte Carlo • Guaranteed to bounds answer • Good solutions often easy to compute Probability box (p-box) Cumulative probability Interval bounds on an cumulative distribution function (CDF) 1 0 0.0 1.0 X 2.0 3.0 Duality • Bounds on the probability at a value Chance the value will be 15 or less is between 0 and 25% • Bounds on the value at a probability 95th percentile is between 40 and 70 Cumulative Probability 1 0 0 20 40 X 60 80 Uncertain numbers Cumulative probability Probability distribution Probability box Interval 1 1 1 0 0 0 0 10 20 30 40 10 20 30 40 10 20 30 1 Cumulative Probability Cumulative Probability Probability bounds arithmetic A 0 0 1 2 3 4 5 6 1 B 0 0 2 4 6 What’s the sum of A+B? 8 10 12 14 Cartesian product A+B A[1,3] p1 = 1/3 A[2,4] p2 = 1/3 A[3,5] p3 = 1/3 B[2,8] q1 = 1/3 A+B[3,11] prob=1/9 A+B[4,12] prob=1/9 A+B[5,13] prob=1/9 B[6,10] q2 = 1/3 A+B[7,13] prob=1/9 A+B[8,14] prob=1/9 A+B[9,15] prob=1/9 B[8,12] q3 = 1/3 A+B[9,15] prob=1/9 A+B[10,16] prob=1/9 A+B[11,17] prob=1/9 independence Cumulative probability A+B under independence 1.00 0.75 0.50 0.25 0.00 0 3 6 9 A+B 12 15 18 Generalization of methods • When inputs are distributions, the answers conform with probability theory • When inputs are intervals, it agrees with interval analysis Where do we get p-boxes? • • • • • Assumption Modeling Robust Bayes analysis Constraint propagation Data with incertitude – Measurement error – Sampling error – Censoring A tale of two data sets Skinny (n = 6) 0 2 4 x 6 Puffy (n = 9) 8 10 0 2 4 x 6 8 10 Cumulative probability Empirical distributions 1 1 Skinny Puffy 0 0 0 2 4 6 x 8 10 0 2 4 6 x 8 10 Cumulative probability Fitted distributions 1 1 Skinny Puffy 0 0 0 x 10 20 0 x 10 20 Statisticians often ignore the incertitude, or treat it as though it were (uniform) variability. Constraint propagation 1 1 .5 0 min 0 max 1 0 1 00 min median max min m ode max 1 min m ean 0 max (lognormal ( lo gnor malwith with interval parameters) inter val par ame ters) .2 .4 1 .6 .8 1 0 mean, sd Maximum entropy erases uncertainty (lognormal with interval parameters) Example: PCBs and duck hunters Location: Massachusetts and Connecticut Receptor: Adult human hunters of waterfowl Contaminant: PCBs (polychorinated biphenyls) Exposure route: dietary consumption of contaminated waterfowl Based on the assessment for non-cancer risks from PCB to adult hunters who consume contaminated waterfowl described in Human Health Risk Assessment: GE/Housatonic River Site: Rest of River, Volume IV, DCN: GE-031203-ABMP, April 2003, Weston Solutions (West Chester, Pennsylvania), Avatar Environmental (Exton, Pennsylvania), and Applied Biomathematics (Setauket, New York). Hazard quotient EF IR C 1 LOSS HQ AT BW RfD EF = mmms(1, 52, 5.4, 10) meals per year // exposure frequency, censored data, n = 23 IR = mmms(1.5, 675, 188, 113) grams per meal // poultry ingestion rate from EPA’s EFH C = [7.1, 9.73] mg per kg // exposure point (mean) concentration LOSS = 0 // loss due to cooking AT = 365.25 days per year // averaging time (not just units conversion) BW = mixture(BWfemale, BWmale) // Brainard and Burmaster (1992) BWmale = lognormal(171, 30) pounds // adult male n = 9,983 BWfemale = lognormal(145, 30) pounds // adult female n = 10,339 RfD = 0.00002 mg per kg per day // reference dose considered tolerable Exceedance risk = 1 - CDF Inputs 1 1 EF 0 0 1 IR 10 20 30 40 50 60 meals per year 0 0 1 200 400 600 grams per meal males females 0 0 C BW 100 200 pounds 300 0 0 10 mg per kg 20 Automatically verified results 1 Exceedance risk mean standard deviation median 95th percentile range [3.8, 31] [0, 186] [0.6, 55] [3.5 , 384] [0.01, 1230] 0 0 500 HQ 1000 Uncertainty about distribution shape • PBA propagates uncertainty about distribution shape comprehensively • Neither sensitivity studies nor secondorder Monte Carlo can really do this • Maximum entropy hides uncertainty Stochastic dependence Dependence • Not all variables are independent – Body size and skin surface area – Common-cause variables – Default risks for mortgages • Known dependencies should be modeled • What can we do when we don’t know them? Uncertainty about dependence • Sensitivity analyses usually used – Vary correlation coefficient between 1 and +1 • But this underestimates the true uncertainty – Example: suppose X, Y ~ uniform(0,25) but we don’t know the dependence between X and Y Unknown dependence Cumulative probability 1 Fréchet X,Y ~ uniform(0,25) 0 0 10 20 30 X+Y 40 50 Varying correlation between 1 and +1 1 Cumulative probability Pearson X,Y ~ uniform(0,25) 0 0 10 20 30 X+Y 40 50 Unknown but positive dependence Cumulative probability 1 Positive X,Y ~ uniform(0,25) 0 0 10 20 30 X+Y 40 50 Uncertainty about dependence • Can’t be studied with sensitivity analysis since it’s an infinite-dimensional problem • Fréchet bounding lets you be sure • Intermediate knowledge can be exploited • Dependence can have large or small effect Backcalculation Backcalculation • Generalization of tolerance solutions • E.g., from p-boxes for A and C, finds B such that A+B C • Needed for planning environmental cleanups, designing structures, etc. Backcalculation with p-boxes A = normal(5, 1) C = {0 C, median 1.5, 90th%ile 35, max 50} 1 1 02 A 3 4 5 C 6 7 8 0 0 10 20 30 40 50 60 Getting the answer • Basically reverses the forward convolution • Any distribution totally inside B is sure to satisfy the constraint • Many possible B’s 1 0-10 0 B 10 20 30 40 50 Check it by plugging it back in A + B = C* C 1 C* 0 -10 0 10 20 C 30 40 50 60 Backcalculation • Backcalculation wider under independence (narrower under Fréchet) • Monte Carlo methods don’t generally work except in a trial-and-error approach • Precise distributions can’t express the target Conclusions Probability vs. intervals • Probability theory – Handles likelihoods and dependence well – Has an inadequate model of ignorance – Lying: saying more than you really know • Interval analysis – Handles epistemic uncertainty (ignorance) well – Inadequately models frequency and dependence – Cowardice: saying less than you know Probability bounds analysis • Generalizes them to escape the limits of each • Makes verified calculations about probabilities – Using whatever knowledge is available – Without requiring unjustified assumptions • Well developed methodology – But plenty of interesting and important questions remain for study Diverse applications • • • • • • • Human health risk analyses Conservation biology extinction/reintroduction Wildlife contaminant exposure analyses Chemostat dynamics Global warming forecasts Design of spacecraft Safety of engineered systems (e.g., bridges) What p-boxes can’t do • Give best-possible bounds on non-tail risks • Conveniently get best-possible bounds when dependencies are subtle • Show what’s most likely within the box Acknowledgments Lev Ginzburg Rüdiger Kuhn Vladik Kreinovich David Myers Bill Oberkampf Janos Hajagos Dan Berleant Chris Paredis Mark Stadtherr NIH Sandia National Labs NASA Applied Biomathematics Electric Power Research Institute End Research topics • • • • • • • • • Incorporation of ancillary information Propagation through black boxes Handling subtle dependencies Computing non-tail risks Combination with fuzzy numbers Decision theory Info-gap models Finite-element models ODEs and PDEs