1 PARALLEL COMPUTATIONS OF 3D UNSTEADY COMPRESSIBLE EULER EQUATIONS WITH STRUCTURAL COUPLING Master’s Candidate Zhenyin Li Advisor: Dr. H. U. Akay Department of Mechanical Engineering Computational Fluid Dynamics Laboratory Indiana University Purdue University Indianapolis July 19, 2002 3D Unsteady Compressible Euler Equations with Structural Coupling Outline Introduction to Fluid-Structure Coupling Fluid-Structure Coupling Procedure Computational Fluid Dynamics Solver – USER3D Computational Structural Dynamics Solver– SAP4 Test Cases Conclusions and Recommendations Acknowledgements Zhenyin Li, Master’s Thesis Defense, July 19, 2002 2/57 3D Unsteady Compressible Euler Equations with Structural Coupling Introduction to Aeroelasticiy “Aeroelasticity is the phenomenon which exhibits appreciable reciprocal interactions (static or dynamic) between aerodynamic forces and the deformations induced in the structure of a flying vehicle, its control mechanisms, or its propulsion system.” Bisplinghoff (1975) Two major concerns in aeroelasticity are stability and response problem. Experiments and computer simulations are two basic ways to reveal the characteristic of various phenomena in aeroelasticity study. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 3/57 3D Unsteady Compressible Euler Equations with Structural Coupling Studies done in this research Develop a procedure based coupling of on independent CFD (Computational Fluid Dynamics and CSD (Computational Structural Dynamics) solvers to resolve static and dynamic aeroelasticity problems. The developed procedure was demonstrated by AGARD wing 445.6. A dual zone mesh movement method developed for large mesh movements when solving unsteady aerodynamic problems. Parallel computation performance was studied. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 4/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM A basic procedure to obtain an aeroelastic solution includes following steps: 1. Get pressure on CFD mesh nodes from flow calculation 2. Pass the load information to CSD domain 3. Calculate nodal displacements with CSD code 4. Feedback the structure deformation to CFD domain 5. Deform the CFD mesh 6. Repeat steps 1 through 5 Zhenyin Li, Master’s Thesis Defense, July 19, 2002 5/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) Mesh-based Parallel Code Coupling Interface (MPCCI), is used to exchange information between CFD and CSD codes and administer both in-code and out of code communications Process I Process II CFD CSD fluid solver structure solver Application Interface Application Interface MPCCI MPCCI Configuration Zhenyin Li, Master’s Thesis Defense, July 19, 2002 6/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) The current version of MPCCI works well with Message Passing Interface (MPI)-based parallel as well as serial computing programs. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 7/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) A global communication ID (GID) is assigned to each of the processes involved in the coupled computation, and a local communication ID (LID) is assigned to the processes of the current code. MPCCI Control Process GID=0 LID=N/A CODE II Process 1 GID=i+1 LID=0 CODE I: Process 1 GID=1 LID=0 CODE I: Process 2 GID=2 LID=1 MPCCI CODE II Process 2 GID=i+2 LID=1 CODE II Process j GID=i+j+1 LID=j-1 CODE I: Process i GID=i LID=i-1 MPCCI communications ID settings Zhenyin Li, Master’s Thesis Defense, July 19, 2002 8/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) Any CSD/CFD code must define its coupling region at the initial stage. The coupling regions do not need to be identical in either size of the region or the density of the elements. Fluid Model Solid Model MPCCI Non-matching meshes Zhenyin Li, Master’s Thesis Defense, July 19, 2002 9/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) Information Exchange : Pressure and displacements need to be exchanged during the coupling process. Qt (1 u )(1 v)Q1 Qt Q1u Q2 v Q3 w u (1 v)Q2 uvQ3 (1 u )vQ Q3 Q4 Q2 Q3 Q1 u Q1 w Q2 v w u v Qt Qt Triangular element interpolations u v Quadrilateral element interpolations Zhenyin Li, Master’s Thesis Defense, July 19, 2002 10/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) Exchanging Quantities Virtual CSD Surface Mesh Mid-surface Structural Mesh Fu Mc CFD surface Mesh Match Virtual CSD Surface Mesh Real Surface Central Surface Fc Fb Central surface transformations Zhenyin Li, Master’s Thesis Defense, July 19, 2002 11/57 3D Unsteady Compressible Euler Equations with Structural Coupling AEROELASTIC COUPLING ALGORITHM (Cont.) Time Integrations of Coupled System Here, the same ∆t is used for fluid and structure Fluid Solid Pn-1 Δt Step n-1 Un Step n Pn Δt Un+1 Step n+1 Time integration Zhenyin Li, Master’s Thesis Defense, July 19, 2002 12/57 3D Unsteady Compressible Euler Equations with Structural Coupling Construct CFD Mesh Steady State Solution for rigid body Construct CSD virtual surface mesh Calculate new CFD flow field Put pressure on virtual surface Extract fluid surface mesh MPCCI Calculate node pressure on surface mesh Put the displacements on surface mesh Calculate dynamic forces on CSD virtual surface mesh Transform the dynamic forces to structure mesh and solve equilibrium equation MPCCI Deform the CFD mesh Map the displacements to CSD virtual surface mesh Finish Zhenyin Li, Master’s Thesis Defense, July 19, 2002 13/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D Background of USER3D • A parallel finite-volume based unstructured Euler solver; • Serial version of User3D was developed by Oktay (1994) ; • Parallel version of User3D was developed at CFD Laboratory at IUPUI (2000); • This solver was validated in previous studies. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 14/57 {Q}dV3D FUnsteady nˆdS 0 t Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D (Cont.) Governing Equations for USER3D The Arbitrary Lagrangian-Eulerian formulation of the threedimensional time-dependent inviscid fluid-flow equations is expressed in the following form: Where Q is the vector of conserved flow variables F is the normal component of the convective flux vector N is the unit normal vector to the boundary Zhenyin Li, Master’s Thesis Defense, July 19, 2002 15/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D (Cont.) The time integration employed in the flow solver is the cellcentered finite volume formulation. The volume-averaged values are adopted to represent the flow variables. n n V V {Q}n F (Q) n dS {Q}n t t An implicit time integration scheme is used to solve flow field at each time step. n V [ A]n {Q}n {R}n {Q}n t n n V { R } [ A]n [I ] t {Q}n n {R} F (Q ) n ndS Zhenyin Li, Master’s Thesis Defense, July 19, 2002 16/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D (Cont.) Mesh-Movement Algorithm The mechanism of this method is that any two neighboring nodes in the mesh are connected by a spring and the spring stiffness is inversely proportional to the distance of the two nodes. Stiffness K km [( x j xi )2 ( y j yi )2 ( z j zi )2 ]1/ 2 Displacement xi n 1 k x , k m y i n 1 m Zhenyin Li, Master’s Thesis Defense, July 19, 2002 k y , k m m z i n 1 k z k m m 17/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D (Cont.) Limitation of the current scheme • The spring technology needs a large amount of CPU time and memory; • The small size cells near the inner boundary can not afford large amplitude motion; A simple dual-zone smoothing approach is proposed to improve the performance of the current spring system II Region I: The inner zone is moving rigidly with the body ; I Region II: The outer zone is deformed by general mesh deformation method . Zhenyin Li, Master’s Thesis Defense, July 19, 2002 18/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Fluid Dynamics Solver - USER3D (Cont.) Geometric Conservation Law : The geometry conservation equation is required to solve simultaneously with other conservation equations. t dV Ws ndS where Ws denotes the local velocity on the boundary surface S The cell volume can be calculated by Vi n1 Vi n t (xtnm1 Ax ytnm1 Ay ztnm1 Az ) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 20/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Structural Dynamics Solver – SAP4 The finite element discrete aeroelasticity element equation for a structural system can be written as: M (e) {q}(e) C (e) q(e) K (e) q(e) R(e) [M], [C] and [K] are system mass, damping and stiffness matrix For static analysis, equation can be rewritten as: K q R For dynamic analysis, equation can be rewritten as: M {q} Cq K q R(t ) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 21/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Structural Dynamics Solver – SAP4 (Cont.) Mode superposition method 1. Get the generalized eigenvalue solution [ K ]{} 2 [ M ]{} 2. Use first n modes to simulate structural response {q(t )} [{1},{2 },......,{n }]{ X (t )} [ A]{ X (t )} 3. Get the generalized displacement solution X 2 X 2 X F * i i i i i i i Zhenyin Li, Master’s Thesis Defense, July 19, 2002 22/57 3D Unsteady Compressible Euler Equations with Structural Coupling Computational Structural Dynamics Solver – SAP4 (Cont.) A Newmark-family of time integration scheme is used to obtain the solution at the (n+1) time step: 2 (1 2 ) 1 * * [ M ] [ C ] ( 2 )[ K ]* ]{ X }n 2 t t 2 1 (1 2 ) 1 * * * n 1 [ [ M ] [ C ] ( )[ K ] ]{ X } {F } * t 2 t 2 [ M ]*{ X }n 1 [ α β Galerkin method 3/2 4/5 Always The backward difference method 3/2 1 Always The constant acceleration method 1/2 1/4 The linear acceleration method 1/2 1/6 Stable Condition Initial Condition: { X }t 0 { X 0 } { dX }t 0 { X 0 } dt For Flutter Analysis Always t Either 2 3 { X 0 } 0 or { X 0 } 0 { X 0 } 0 { X 0 } 0 i Zhenyin Li, Master’s Thesis Defense, July 19, 2002 23/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES Aeroelastic Research Wing (AGARD Wing 445.6) 1.208 ft 5.2 ft 45O 1.833 ft AGARD wing 445.6 panel dimensions The CFD grid consists of 147,547 cells and 26,228 nodes. The CFD wing surface has 2020 elements and 1077 nodes Zhenyin Li, Master’s Thesis Defense, July 19, 2002 24/57 3D Unsteady Compressible Euler Equations with Structural Coupling In the present application: n processors are used for CFD solution One processor for CSD solution One processor for communication management with MPCCI Zhenyin Li, Master’s Thesis Defense, July 19, 2002 25/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Modal Analysis of Wing 445.6 Table 5.2 Modal frequencies of AGARD wing 445.6 SAP4 400 eles. SAP4 200 eles. SAP4 100 eles. ANSYS 100 eles Experim ent f1 9.60 9.60 9.60 10.85 9.59 f2 39.77 39.81 39.86 44.57 38.16 f3 50.88 50.20 48.30 56.88 48.35 f4 95.37 95.40 95.01 109.10 91.55 Comparison of AGARD wing 445.6 modal frequencies Zhenyin Li, Master’s Thesis Defense, July 19, 2002 26/57 3D Unsteady Compressible Euler Equations with Structural Coupling SAP4 Modal Shape TEST CASES (Cont.) MODE 1 MODE 3 Zhenyin Li, Master’s Thesis Defense, July 19, 2002 MODE 2 MODE 4 27/57 3D Unsteady Compressible Euler Equations with Structural Coupling ANSYS Modal Shape TEST CASES (Cont.) Mode 1 Mode 3 Zhenyin Li, Master’s Thesis Defense, July 19, 2002 Mode 2 Mode 4 28/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Steady Solution of the Rigid Body Steady State Transonic Flow at M∞ = 0.96 and M∞ = 1.141 Zhenyin Li, Master’s Thesis Defense, July 19, 2002 29/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 30/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 31/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Rigid Body Result Static Aeroelastic Analysis at Mach = 0.8 1. The coupling iteration starts from the steady-state solution of the rigid body. 2. In practice, a load factor is used to control the force loaded on the structural system. 3. An alternate approach also performed here is using dynamic analysis to simulate steady case. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 32/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) The tip deflection at the trailing edge was computed to be 0.40 inch which is very close to 0.39 inch from MDICE Zhenyin Li, Master’s Thesis Defense, July 19, 2002 33/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Deformed Mesh Undeformed Mesh Zhenyin Li, Master’s Thesis Defense, July 19, 2002 34/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 35/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Dynamic Aeroelastic Analysis Mach = 0.8, AOA =1.0 degree In this section, the previous steady-state solution is used as a sudden load at time zero. The wing motion is entirely determined by the structural response. The time increment is 1.0e -4 Zhenyin Li, Master’s Thesis Defense, July 19, 2002 36/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 37/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Deformed Mesh Undeformed Mesh Zhenyin Li, Master’s Thesis Defense, July 19, 2002 38/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Flutter Analysis Dynamic instability where-by the system extracts energy from the free stream flow producing a divergent response. The computed flutter characteristics are presented in terms of velocity index Vf which is defined as Stable Neutral V f U / b Unstable Zhenyin Li, Master’s Thesis Defense, July 19, 2002 39/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Mach=0.957, Vf = 0.349 , U∞=14400 inch/s Zhenyin Li, Master’s Thesis Defense, July 19, 2002 40/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Mach=0.957, Vf = 0.250 , U∞=10200 inch/s Zhenyin Li, Master’s Thesis Defense, July 19, 2002 41/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Mach=0.957, Vf = 0.262 , U∞=10800 inch/s Zhenyin Li, Master’s Thesis Defense, July 19, 2002 42/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Comparison of Results Zhenyin Li, Master’s Thesis Defense, July 19, 2002 43/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Initial Velocity Effect Zhenyin Li, Master’s Thesis Defense, July 19, 2002 44/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Parallel Aerodynamic Studies A standard research configuration for missile geometries, is studied under forced pitching conditions. The computational mesh used consists of 144,216 nodes and 706,105 cells, 24 Blocks The steady case was performed with M∞ = 1.58, angle of attack (AOA) = 0.0. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 45/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 46/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) This case is the basic finner performing a sinusoidally pitching motion about its center of gravity. The angle of attack varies as: (t ) m p sin( t ) For this test case, the reduced frequency k = 2.53165, freestream Mach number M∞ = 1.58, the mean angle of pitching αm = 0.0 degree and the amplitude of pitching is 10 degrees. The results were obtained using 2000 steps per cycle of the motion. The time increment of 2e-4 was used Zhenyin Li, Master’s Thesis Defense, July 19, 2002 47/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 48/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Zhenyin Li, Master’s Thesis Defense, July 19, 2002 49/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) Parallel Efficiency Study The parallel efficiency study performed here is based on Indiana University’s IBM SP clusters and Compaq Linux clusters. The speedup is defined as S p T1 / T p Efficiency E is defined as UNIX LINUX Model IBM RISC System / SP6000 POWER3+ Thin Node Compaq ProLiant 1850R rackmounted compute nodes CPU (Each Node) 4 CPU, 375 MHz clock cycle Memory 2GB 256 MB Cache 8MB 512 KB Network 10/100Mb "Fast" Ethernet (100 TX) E 100 S p / p Zhenyin Li, Master’s Thesis Defense, July 19, 2002 Dual Intel 400 MHz Pentium II processors 50/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) 144,216 nodes and 706,105 cells Zhenyin Li, Master’s Thesis Defense, July 19, 2002 51/57 3D Unsteady Compressible Euler Equations with Structural Coupling TEST CASES (Cont.) 144,216 nodes and 706,105 cells Zhenyin Li, Master’s Thesis Defense, July 19, 2002 52/57 3D Unsteady Compressible Euler Equations with Structural Coupling Conclusions A loosely coupled procedure is developed by using parallel Euler equations solver USER3D and finite element structural solver SAP4. The advantage of current method is to provide a flexible and easy implementation for coupling CFD and CSD codes without a large amount of works in existing codes. In steady aeroelastic problems, due to the limitation of mesh deformation scheme, a load factor was used to increase the load gradually. The results are quite consistent with other researcher’s work. Using dynamic aeroelastic solutions with damping the results of static problem is also validated. Dynamic aeroelastic problems were solved using the coupled CFD-CSD procedure. Significant aeroelastic effects were observed in this study. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 53/57 3D Unsteady Compressible Euler Equations with Structural Coupling Conclusions (Cont.) Flutter analysis was implemented by choosing initial perturbation of the structural system and examining whether the initial perturbation will decay, grow or maintain neutral conditions to determine the flutter conditions. The results compared well with previous works and experimental results. A dual-zone dynamic mesh system was successfully employed to solve unsteady aerodynamic problems. High computational efficiency was obtained. Both steady-state solution scheme and unsteady solution showed good speedup and efficiency for multi-block cases. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 54/57 3D Unsteady Compressible Euler Equations with Structural Coupling Future Works The present dynamic grid scheme can prevent two nodes colliding with each other. And the dual-zone scheme can only deal with known motion. This scheme works well with small motion or large simple motion such as sinusoidal motion. Problems will occur when solving aeroelastic problems with large motion. Time increment in the present scheme is same on both CFD and CSD solvers. But, CSD solver usually requires larger time increments than the CFD solver. In the future work, the effect of time sub-cycle should be studied. Another problem in current scheme is only that only the CFD code is a parallel code. In the future study, a parallel CSD code may be required to improve the computational efficiency, especially for large structures such as a complete aircrafts or missiles. The information exchange between CFD and CSD solvers is based on bi-linear interpolations. Although its accuracy is enough for the current problem, a more complex interpolation scheme maybe required for future applications. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 55/57 3D Unsteady Compressible Euler Equations with Structural Coupling Future Works (Cont.) One remaining problem in this procedure is that MPCCI requires that each sub process must define its own coupling region, but some CFD blocks which are partitioned by GD do not include such coupling regions. As the result, the current procedure may be limited to a few blocks which depend on how GD divides a grid. Although reasonable results are obtained for flutter analysis, there are still some differences between the present results and experiments. One possible way to improve the accuracy is to refine the mesh to get more accurate fluid solutions. Another way to improve the accuracy is by improving the present bilinear interpolation scheme to get more accurate quantities exchanging. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 56/57 3D Unsteady Compressible Euler Equations with Structural Coupling Acknowledgement First, I would like to thank my advisor and committee chairman, Dr. Hasan U. Akay. His invaluable guidance helped me in realizing this research throughout the course of my studies. I also would like to extend my thanks to Dr. Hasan U. Akay and Dr. Erdal Oktay for giving me the opportunity to work on this research project; to Dr. Akin Ecer for providing me the opportunity to use the facilities of the CFD Laboratory and serving in my thesis committee; and to Dr. Andrew T. Hsu for serving in my thesis committee. Valuable assistance from Mr. Resat U. Payli contributed a lot to the computational work in this research to which I am grateful. Finally, I would thank to my lovely wife, Jing, without her, none of this would have been possible. Zhenyin Li, Master’s Thesis Defense, July 19, 2002 57/57 3D Unsteady Compressible Euler Equations with Structural Coupling Question? Zhenyin Li, Master’s Thesis Defense, July 19, 2002 58/57