Code verification and mesh uncertainty • The goal is to verify that a computer code produces the right solution to the mathematical model underlying it. • Tests detailed by Oberkampf and Roy (Section 5.1) include 1. 2. 3. 4. 5. Simple tests (symmetry, conservation, Gallilean invariance) Code to code comparisons. Discretization error quantification. Convergence tests. Order of accuracy tests. • We will focus on 3,4 and 5 using a 2002 paper on comparing CFD simulations to published test results. Truncation and discretization errors u ( x) u ( x0 ) du 2 ( x0 )( x x0 ) O x x0 dx • In Taylor series the truncation error is estimated by the last term. • We have used this in the integration of the paper helicopter fall velocity to get the distance travelled h(t t ) h(t ) V (t )t • For a given time increment, the truncation error is O(t 2 ) • The truncation error decreases as the square of the time interval. • The discretization error is the error in the final distance h(tf) • However, it decreases approximately linearly with the time interval. Why? Order of accuracy • Order of accuracy p is the power in the relationship between discretization error and interval size h p p 1 h g p h O(h ) • For simple problems like integration of the paper helicopter distance we can estimate the order of accuracy analytically. • For most complex problems we have to estimate it from different meshes, say hcoarse and hfine p p h coarse g p hcoarse h fine g p h fine • If we know the errors (from an exact solution) we can solve for p. Richardson extrapolation • When we do not know what the error is, we can use Richardson extrapolation to estimate what our solution will be with an infinitely p p 1 fine discretization f h f 0 g p h O (h ) • Now we need f values from three values of h to estimate p and f0 • Unfortunately, the result is exact as h goes to zero, but you often cannot get near enough. • In addition there is noise when fine meshes change boundaries compared to coarse mesh. So it is popular to reduce h by a factor of 2. Top hat question • For 4 meshes with h=1,0.5,0.25, and 0.125, the results were, 20, 14.5, 12.5, 11.7. Estimate the converged value and the order • • • • f0=11, p=1.5 f0=10.5, p=1 f0=11, p=1 f0=11.2, p=1.5 AIAA 2002-5531 AIAA 2002-5531 OBSERVATIONS ON CFD SIMULATION UNCERTAINTIES Serhat Hosder, Bernard Grossman, William H. Mason, and Layne T. Watson Virginia Polytechnic Institute and State University Blacksburg, VA Raphael T. Haftka University of Florida Gainesville, FL 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization 4-6 September 2002 Atlanta, GA Hosder, S. Grossman, B., Haftka, R. T., Mason, W. H., and Watson, L. T. (2006), “Quantitative Relative Comparison of CFD Simulation Uncertainties for a Transonic Diffuser,” Computers and Fluids, 35 (10), 1444-1458, December. 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 6 AIAA 2002-5531 Introduction • • • • • Computational fluid dynamics (CFD) as an aero/hydrodynamic analysis and design tool CFD being used increasingly in multidisciplinary design and optimization (MDO) problems Different levels of fidelity • from linear potential solvers to RANS codes CFD results have an associated uncertainty, originating from different sources Sources and magnitudes of the uncertainty important to assess the accuracy of the results 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 7 AIAA 2002-5531 Drag polar results from 1st AIAA Drag Prediction Workshop (Hemsch, 2001) 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 8 AIAA 2002-5531 Objective of the Paper • Finding the magnitude of CFD simulation uncertainties that a well informed user may encounter and analyzing their sources • We study 2-D, turbulent, transonic flow in a converging-diverging channel • complex fluid dynamics problem • affordable for making multiple runs 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 9 AIAA 2002-5531 Transonic Diffuser Problem 0.9 Weak shock case (Pe/P0i=0.82) 0.7 0.6 experiment 2.5 CFD Pe/P0i 0.5 0.4 2.0 y/ht y/ht 0.8 0.3 1.5 1.0 0.5 0.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 x/ht 0.9 Strong shock case (Pe/P0i=0.72) 0.7 0.6 2.5 streamlines y/ht 2.0 y/ht 0.8 Pe/P0i 0.5 Separation bubble 0.4 0.3 1.5 1.0 0.5 0.0 -4.0 -3.0 -2.0 -1.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 x/ht Contour variable: velocity magnitude 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 10 AIAA 2002-5531 Uncertainty Sources (following Oberkampf and Blottner) • Physical Modeling Uncertainty • PDEs describing the flow • Euler, Thin-Layer N-S, Full N-S, etc. • boundary conditions and initial conditions • geometry representation • auxiliary physical models • turbulence models, thermodynamic models, etc. • Discretization Error • Iterative Convergence Error • Programming Errors We show that uncertainties from different sources interact 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 11 AIAA 2002-5531 Computational Modeling • • • General Aerodynamic Simulation Program (GASP) • A commercial, Reynolds-averaged, 3-D, finite volume Navier-Stokes (N-S) code • Has different solution and modeling options. An informed CFD user still “uncertain” about which one to choose For inviscid fluxes (most commonly used options in CFD) • Upwind-biased 3rd order accurate Roe-Flux scheme • Flux-limiters: Min-Mod and Van Albada Turbulence models (typical for turbulent flows) • Spalart-Allmaras (Sp-Al) • k- (Wilcox, 1998 version) with Sarkar’s compressibility correction 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 12 AIAA 2002-5531 Grids Used in the Computations Grid 2 y/ht Grid level Mesh Size (number of cells) 1 40 x 25 2 80 x 50 3 160 x 100 4 320 x 200 5 640 x 400 A single solution on grid 5 requires approximately 1170 hours of total node CPU time on a SGI Origin2000 with six processors (10000 cycles) Grid 2 is the typical grid level used in CFD applications 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 13 AIAA 2002-5531 Nozzle efficiency Nozzle efficiency (neff ), a global indicator of CFD results: neff H 0i H e H 0 i H es H0i : Total enthalpy at the inlet He : Enthalpy at the exit Hes : Exit enthalpy at the state that would be reached by isentropic expansion to the actual pressure at the exit 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 14 AIAA 2002-5531 Uncertainty in Nozzle Efficiency 0.900 0.875 0.850 Sp-Al grid 1 + grid 2 grid 3 x grid 4 o grid 5 + + + o + + + + + + + + + + + + + + + + + + + + x + o + + + + + + + + x x + + + + + + + + x o + + x x + + + + x + x 0.825 neff k- 0.800 x x o x x x x x x x x x x x x x x x 0.775 x x x x x x x x x x x 0.750 0.725 0.700 x x x x x x x x x x x o 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 P e/P 0i 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 15 AIAA 2002-5531 Uncertainty in Nozzle Efficiency Strong Shock Weak Shock the total variation in nozzle efficiency 10% 4% the discretization error 6% (Sp-Al) 3.5% (Sp-Al) the relative uncertainty due to the selection of turbulence model 9% (grid 4) 2% (grid 2) Maximum value of 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 16 AIAA 2002-5531 Discretization Error by Richardson’s Extrapolation error coefficient order of the method f k f exact h p O(h p 1 ) a measure of grid spacing grid level Turbulence model Sp-Al Sp-Al k- Pe/P0i 0.72 (strong shock) 0.82 (weak shock) 0.82 (weak shock) estimate of p (observed order of accuracy) 1.322 1.578 1.656 estimate of (neff)exact Grid level Discretization error (%) 1 14.298 2 6.790 3 2.716 4 1.086 1 8.005 2 3.539 3 1.185 4 0.397 1 4.432 2 1.452 3 0.461 4 0.146 0.71950 0.81086 0.82889 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 17 AIAA 2002-5531 Major Observations on the Discretization Errors • Grid convergence is not achieved with grid levels that have moderate mesh sizes. For the strong shock case, even with the finest mesh level we can afford, asymptotic convergence is not certain • As a consequence of above result, it is difficult to separate physical modeling uncertainties from numerical errors • Shock-induced flow separation, thus the flow structure, has a significant effect on grid convergence • Discretization error magnitudes are different for different turbulence models. The magnitudes of numerical errors are affected by the physical models chosen. 9th AIAA/ISSMO Symposium on MAO, 09/05/2002, Atlanta, GA 18 Discussion • What are the key ingredients of the Richardson extrapolation? • Why do we get non-integer orders? • Why do we reduce mesh sizes by factors of 2?