FEM with Parallel SQL
Server: A Case Study
Gerd Heber
Cornell Theory Center
Cornell Fracture Group
Thank You




Dan Fay (MSR)
Jim Gray (MSR)
Todd Needham (MSR)
Alexander Szalay (JHU)
Outline


Parallel SQL Server
Context



Examples



Infrastructure
Application
Complexity
Issues
Conclusions
Parallel SQL Server

Hardware



SMP
Distributed memory
Software

Query level parallelism


Partitioned views (LDPV)
Linked servers

Distributed partitioned views (DPV)
When to Use DPV



Scale-out (DPV) vs. scale-up (SMP)
Good performance on commodity
hardware
Data (and queries) must be suitable for
partitioning

Increased application complexity



Go to the server with most, or all, of the data
For reliability consider failover
clustering
No support for parallel (bulk) inserts
CTC’s Infrastructure Today
Clusters
Tape Robot
Domain
Database Servers
Active Directory
Scheduler
Certificate Server
Web Servers
File Servers
Login Nodes
Cave
SCTS
Basic FEM Analysis

Preprocessing





Solution



Topology/Geometry generation
Mesh generation
Apply boundary conditions
Material properties
Equation solving
Error analysis
Post processing

Data analysis

Visualization
How We Used To Do Things






100% file-based
Monolithic (brittle) code
Disconnected
No data-sharing, except copy
Hard to debug
Plenty of non problem oriented code
Where We Use SQL Server

Data storage

Analysis






Debugging
Visualization
Processing
Checkpoint / restart
Web service state management
Data virtualization

XML repository
Input

100 MB - 1 GB (today)





Files ASCII (incl. XML), binary
Topology, geometry, mesh
Initial / boundary conditions
Material properties
Input may or may not be partitioned
Output

Physical fields



Temperature (1x double per node)
Displacement (3x double per node)
Stress, strain (6x double per Gauss point)



State variables




Tetrahedron: 5 / 11 Gauss points
Hexahedron: 27 / 64 Gauss points
Mises plasticity (13x double per Gauss point)
Polycrystal plasticity (>= 30x double per GP)
…
Produce 10 - 1000 times the input size
Datasets
10-3
10-6
10-9
m|s
Examples
l
Pictures provided by Paul Wawrzynek, Cornell Fracture Group
Visual SQL
DDSim
ES7000
Pictures provided by John Emery, Cornell Fracture Group
Spatial Search

Jim Gray et al., There Goes the Neighborhood:
Relational Algebra for Spatial Data Search, MSR-TR2004-32
Web Services
Adaptive Software Project


NSF-ITR #0085969:
Adaptive Software for
Field-driven
Simulations (09/01/00)
Implement a system
for multi-physics
multi-scale adaptive
CSE simulations



Computational fracture
mechanics
Chemically-reacting
flow simulation
Understand principles
of implementing
adaptive software
systems
Adaptivity in CSE Simulations

Application-level adaptivity



Algorithm-level adaptivity



Change in modeling / governing equations
Example: Elasticity PDE’s vs. molecular-scale
interactions, symmetry
Change in solution method for governing
equations
Example: Finite-element vs. wavelet bases
System-level adaptivity


Response to changing resource availability
Example: Processor / link failure
Test Problems
(Substantial) Infrastructure






Metadata management
State management
Event logging
Data virtualization
Accounting
Transactions
Yukon

NET CLR integration






Stored procedures, user-defined
functions, and triggers in .NET languages
(and T-SQL)
Call unmanaged (unsafe) code
User defined aggregates and types
Native XML data type (schema support)
XQuery support
Database logic can be invoked as Web
service
XML Repository
Polycrystal Generation
Comments / Issues / Wishes






SQL Libraries
Better management tools for linked servers
Embedded SQL renaissance
WSE 2.0 and Yukon
WS interface for WMI
Template(s) for WS state and event
management




O’SOAP
Visual SQL
Virtualization not there (yet)
Data grids
Conclusion
“Language shapes the way we
think, and determines what we
can think about.” (B.L. Whorf)

It’s a slow process


Most engineers are conservatives
Legacy
Sponsors








DARPA
Intel
Microsoft
Microsoft Research
NASA
NSF
Northrop Grumman
Unisys