Technical white paper
HP reference architecture for
small and medium enterprises
Expand simulation capabilities for ANSYS running on HP
Table of contents
Changing landscape of computer-aided engineering ........................................................................................................... 2
Innovate to compete, and the key role of CAE ................................................................................................................... 2
Unique ANSYS positioning in CAE .......................................................................................................................................... 2
Challenges in product design ................................................................................................................................................. 2
HP and Intel enable your innovation ......................................................................................................................................... 2
Managing your workstation compute performance .............................................................................................................. 2
HP Z820 compute capabilities ............................................................................................................................................... 3
Scaling out your workstations with a clustered environment............................................................................................. 3
Why scale out? ........................................................................................................................................................................... 3
Easy cluster operations ........................................................................................................................................................... 3
Scaling out with HP BladeSystem c-Class infrastructure ..................................................................................................... 3
Cluster baseline—recommended configuration for SMEs .............................................................................................. 4
Compute node types ................................................................................................................................................................ 4
Scaling out with HP SL2500 infrastructure ............................................................................................................................. 4
Cluster baseline—recommended configuration for SMEs .............................................................................................. 5
Compute node types ................................................................................................................................................................ 5
Sizing the cluster for ANSYS 15 ................................................................................................................................................. 5
ANSYS Fluent and CFX ............................................................................................................................................................. 5
ANSYS Mechanical ..................................................................................................................................................................... 6
Value of ANSYS Workbench ........................................................................................................................................................ 6
Benefits of ANSYS HPC Packs ..................................................................................................................................................... 6
Moving from symmetric multiprocessing to data management processor solver for ANSYS Mechanical ............... 6
ANSYS HPC Parametric Pack ....................................................................................................................................................... 7
Boosting efficiency with ANSYS RSM ........................................................................................................................................ 7
Click here to verify the latest version of this document
Technical white paper | HP reference architecture for small and medium enterprises
Changing landscape of computer-aided engineering
Innovate to compete, and the key role of CAE
Over the last 10 years, computer-aided engineering (CAE), and more generally virtual prototyping, has been increasingly
recognized in product design. The so-called “simulation-driven prototyping” is now pervasively fueling innovation in the
industry. CAE helps enterprises innovate to:
• Compete
• Cope with increasingly demanding regulations
• Improve product capabilities and quality
• Reduce costs
Unique ANSYS positioning in CAE
Continual ANSYS research has led to advanced, more robust solutions for even the most complex problems. Using ANSYS
software, you can have greater confidence in analytic results so you can rely less on costly physical testing.
ANSYS software contains sophisticated numerics and robust solvers to ensure fast and accurate results for a nearly limitless
range of engineering applications. ANSYS solvers are also highly optimized to deliver outstanding performance and parallel
scaling on today’s multi-core processors.
Challenges in product design
As suppliers to major corporations, small and medium enterprises (SMEs) drive up to 80 percent of the industry’s product
design. Conversely, SMEs face significant challenges as they attempt to expand their simulation capabilities. SMEs can
overcome these challenges by boosting their compute resources in one of two ways:
• Scale up the existing workstation infrastructure—HP and ANSYS provide a wide range of comprehensive
system settings, hardware upgrades, and licensing schemes that can drive tangible productivity improvement for
simulation engineers.
• Scale out the workstation infrastructure in the data center—In some cases, SMEs may find it useful to add clusters to
increase productivity and improve model fidelity. Because simulation engineers should not be required to have to master
clustering technology, clusters must be easy to deploy, configure, and operate.
HP and Intel enable your innovation
To foster expansion of SME capabilities, the HP-Intel® Innovation Initiative seeks to simplify the user experience, and enable
the deployment and maintenance of high performance computing (HPC) infrastructure. The initiative works with business
and technology partners, including ANSYS, to build an ecosystem that fosters the use of advanced simulation technologies
by small- to medium-sized companies. This ecosystem delivers community outreach and provides access to relevant
technology and expertise, such as starter kits and innovation hubs. The ANSYS-optimized recommendations described in
this data sheet were developed with support from the Innovation Initiative.
Managing your workstation compute performance
Engineers use workstations to start, orchestrate, and complete their simulation processes. Best practices dictate that each
workstation must run—either concurrently or in sequence—numerous graphical and compute-intensive tasks. Because of
this, engineers must pay close attention to the execution conditions of ANSYS jobs on each workstation, ensuring each job
gets its fair share of resources.
2
Technical white paper | HP reference architecture for small and medium enterprises
HP Z820 compute capabilities
HP Z820 Workstations ship with up to 24 cores (two 12-core Intel E5-2600 v2 processors) and up to 512 GB of memory,
making the HP Z820 an ideal platform for all ANSYS applications:
• ANSYS Fluent and CFX demonstrate excellent scale-up capabilities beyond 24 cores.
• ANSYS Mechanical can easily run two jobs, 12 cores each, with superior scale-up capabilities; for a given amount of
RAM, solid-state drive (SSD) disks provide a performance boost over SAS and SATA disks for ANSYS Mechanical jobs.
• HP and ANSYS provide regular system setting recommendations for memory, disks, and the operating system to help
you get the best compute performance from your workstations.
Scaling out your workstations with a clustered environment
Why scale out?
As stated above, today’s multi-core capabilities create opportunities for workstation users to benefit from scaling. You can
further optimize by dedicating workstations to graphic-intensive tasks and solver jobs that require less than 16 cores, and
running larger-scale simulation in the data center. Scaling out to clusters can significantly reduce runtimes and improve
engineering productivity, while also allowing your engineers to increase the size and robustness of the models. Clusters are
also particularly suitable for Design of Experiment and Multi-Disciplinary Design Optimization.
Easy cluster operations
To make cluster operations easier and more efficient, HP designed HP Integrated Lights Out (iLO4) and HP Insight Cluster
Management Utility (CMU).
As the supervision and provisioning interface sitting on a chip embedded in each HP ProLiant Gen8 Server, HP iLO4 simplifies
initial server setup, server health monitoring, power and thermal optimization, and remote server administration. One of the
key benefits of HP iLO4 is its ability to collect a wide range of performance information related to the server subsystems,
i.e., processor, memory, networks, local disks, and so on. The Operations Manager can combine this information to build
predictive supervision policies, such as triggering alerts when the number of RAM-related error-correcting code errors grows
beyond a certain threshold, or when disk performance decreases below acceptable levels for certain simulations.
With HP Insight CMU, you can monitor performance of each simulation job using an intuitive interface. HP Insight CMU is a
widely used cluster management solution, which combines provisioning, administration, and supervision features into a
single simplified interface.
Scaling out with HP BladeSystem c-Class infrastructure
Offered in a 10U form factor, the HP BladeSystem c7000 chassis features up to 16 compute nodes, federated by a
non-blocking InfiniBand FDR network. To run the cluster, the compute nodes share all the necessary components—power
supplies, fans, InfiniBand and GbE networking, onboard administrator, federated administration, and the supervision
interface to the cluster.
Most components are hot-pluggable, so the HP BladeSystem c-Class infrastructure is easy to deploy; it is also easy to
operate using the onboard administrator interface. Initial investment for HP BladeSystem c7000 is minimal; the shared
infrastructure needs to be deployed only once, and can grow by populating the compute nodes slots as needed.
3
Technical white paper | HP reference architecture for small and medium enterprises
Cluster baseline—recommended configuration for SMEs
A typical cluster configuration allowing expansion up to 32 compute nodes includes the following baseline:
• Head node to manage access to the cluster
• GbE management network
• Two HP BladeSystem c7000 chassis
• One top-of-rack InfiniBand switch to federate the two chassis and the head node
Compute node types
Supporting up to 24 cores (two 12-core processors), 512 GB of memory, up to two hot plug disks, and InfiniBand
connectivity, the HP ProLiant BL460c Gen8 Server is well suited for ANSYS Fluent and CFX workloads.
Equipped with the D2200sb storage extender and support for up to 12 additional hot plug disks, the
HP ProLiant BL460c Gen8 Server is well suited for ANSYS Mechanical workloads.
Table 1. Recommended cluster configuration
Cluster baseline
Designed for up to 24 compute nodes and two HP BladeSystem c7000 chassis, 10 KW per 42 rack
Head node
One HP ProLiant DL380p Gen8 Server with 2–16X SAS disks
Administration network
(out-of-band and
console)
One HP Networking 2920-24G with 10GbE uplink
High-speed network
One InfiniBand FDR 36-port switch
42U rack, KVM, TFT
-
HP BladeSystem
c7000 chassis
Two HP BladeSystem c7000 chassis, each with 8+2 redundant hot plug fans, one FDR InfiniBand
Managed Switch Module, and 6X 2400 W platinum hot plug power supply
Operating system
64-bit Linux
Cluster management
HP Insight CMU
MPI
MPI software is included in ANSYS installation package
Job management
ANSYS Remote Solve Manager (RSM) and supported job schedulers
Scaling out with HP SL2500 infrastructure
The HP ProLiant SL2500 Scalable System offers a very dense solution and power optimized with up to 4 independent 1U or
2 independent 2U HP ProLiant SL210t Gen8 hot-pluggable server nodes in a standard sharing the same chassis, to provide
increased server uptime.
Like standard rack servers, the SL2500 chassis is rear cabled and fits into 19-inch standard racks; it comes with to disks
options in the front of the chassis: 12 large-form-factor (LFF) or 24 small-form-factor (SFF) drive.
4
Technical white paper | HP reference architecture for small and medium enterprises
Cluster baseline—recommended configuration for SMEs
A typical cluster configuration allowing expansion up to 24 compute nodes includes the following base line:
• Head node to manage access to the cluster
• GbE management network
• 8 HP SL2500 chassis
• 1 top-of-rack InfiniBand switch to federate the two chassis and the head node
Compute node types
The HP ProLiant SL210t Gen8 configuration well suited for ANSYS Fluent and CFX workloads features 1U per server, up to
24 cores (2 Intel® Xeon® E5-2600 v2), up to 512 GB of memory, up to two hot plug disks and InfiniBand connectivity.
The HP ProLiant SL210t Gen8 configuration well suited for ANSYS Mechanical workloads features 2U per server and up to
twelve hot plug disks.
Table 2. Recommended cluster configuration
Cluster baseline
Designed for up to 24 compute nodes and eight SL2500 chassis, 10 KW per 22 rack
Head node
One HP ProLiant DL380p Gen8 Server with 2–16 x SAS disks
Administration network
(out-of-band and
console)
One HP Networking 2920al with 10GbE uplink
High-speed network
One InfiniBand FDR 36-port switch
22U rack, KVM, TFT
-
HP SL2500 chassis
Eight SL2500 chassis, each with 5 fans, and 2X 1200 W platinum hot-plug power supply
Operating system
64-bit Linux
Cluster management
HP Insight CMU
MPI
MPI software is included in ANSYS installation package
Job management
ANSYS Remote Solve Manager (RSM) and supported job schedulers
Sizing the cluster for ANSYS 15
ANSYS Fluent and CFX
• Compute nodes—16 HP compute nodes, each with two Intel Xeon E5-2600 v2 processors:
– up to 24 cores per compute node
– Two 300 GB 15k rpm SAS drives per compute node
– 4 GBs/core RAM
– HP InfiniBand FDR/Ethernet 10 Gb/40 Gb two-port 544M Adapter
• Head node-specific configuration—Specific memory and storage required for very large jobs such as the partitioning
step in CFX:
– Up to 24 cores per node
– 8 GBs/core (DDR3 1,600 MHz) RAM
– Six SFF 15k rpm SAS disk drives
– HP InfiniBand FDR/Ethernet 10 Gb/40 Gb two-port 544M Adapter
5
Technical white paper | HP reference architecture for small and medium enterprises
Suitable for:
• One ANSYS Fluent model greater than 500M cells
• One ANSYS CFX model greater than 100 or 500M nodes (CFX), depending on mesh
• Eight simultaneous ANSYS Fluent models on the scale of 50M cells
• Eight simultaneous ANSYS CFX models with 10 to 50M nodes (CFX), depending on mesh
ANSYS Mechanical
• Compute nodes—Eight compute nodes, each with two Intel Xeon E5-2600 v2 processors and equipped with the
D2200sb storage extender:
– Up to 16 cores per compute node
– Two to 16 internal 15k rpm SAS drives striped RAID 0
– 8 GBs/core (DDR3 1,600 MHz) RAM
– HP InfiniBand FDR/Ethernet 10 Gb/40 Gb two-port 544M Adapter
• Head node-specific configuration—Specific memory and storage required for very large jobs such as the partitioning
step in ANSYS Mechanical:
– Up to 24 cores per node
– 8–16 GBs/core (DDR3 1,600 MHz) RAM
– 6 x 15k rpm SAS RAID 0 disk array
– HP InfiniBand FDR/Ethernet 10 Gb/40 Gb two-port 544M Adapter
Suitable for:
• 256–1024 GB RAM configurations can handle up to six simultaneous models running ANSYS— “mega-models”
of 45–180M degrees of freedom
Value of ANSYS Workbench
ANSYS Workbench is the framework upon which the industry’s broadest and deepest suite of advanced engineering
simulation technology is built. An innovative project schematic view ties together the entire simulation process—guiding
you through complex multi-physics analyses with drag-and-drop simplicity. With bi-directional CAD connectivity, powerful
highly-automated meshing, a project-level update mechanism, pervasive parameter management, and integrated
optimization tools, the ANSYS Workbench platform delivers unprecedented productivity, enabling true simulation-driven
product development.
Benefits of ANSYS HPC Packs
Providing a flexible way to license parallel processing capability, ANSYS HPC Packs are designed for single users who want
to run simulations on a workstation. A single ANSYS HPC Pack provides outstanding value by increasing throughput by up
to eight times.
If you have access to larger HPC resources, you can combine multiple HPC Packs to enable parallel processing on hundreds
or even thousands of processing cores. ANSYS HPC Packs offer virtually unlimited parallel processing for the largest jobs,
combined with the ability to run multiple jobs using everyday parallel capacity.
Moving from symmetric multiprocessing to data management processor
solver for ANSYS Mechanical
The data management processor mode improves the speed of small-scale jobs, and can also be used for large jobs running
on servers with up to 1024 cores.
6
Technical white paper | HP reference architecture for small and medium enterprises
ANSYS HPC Parametric Pack
To amplify the available licenses for individual applications—pre-processing, meshing, solving, HPC, post-processing—the
ANSYS HPC Parametric Pack license enables simultaneous execution of multiple design points, while consuming a single set
of application licenses.
Boosting efficiency with ANSYS RSM
The ANSYS RSM is a job queuing system you can use to efficiently manage Mechanical, CFX, and Fluent analyses on
Microsoft® Windows® and Linux operating systems. The RSM enables you to execute solutions in the background on
local workstations or on remote servers. RSM comes standard with ANSYS Workbench products, so separate licensing is
not required.
RSM enables you to queue several independent analyses to be solved sequentially. For example, you can queue different
projects to solve overnight, and then review the results the next day. And if you have access to a more powerful compute
server, you can use RSM to offload the computationally expensive solving tasks to the server, thereby freeing up resources
on the local workstation. While the simulation runs on the remote server, RSM allows you to disconnect the workstation and
reconnect at a later time to download the results.
In addition, RSM can also submit multiple design-point jobs, with each job executing on multiple processing cores with
third-party job schedulers.
With ANSYS 14.5, a new installation and configuration wizard simplifies deployment of RSM in the HPC environment.
Resources
More on the key role of simulation: compete.org
Learn more about HP workstations for ANSYS:
Z820 system tuning for ANSYS
ANSYS 14.5 study on SSDs performance
Learn more at
hp.com/go/hpc
Sign up for updates
hp.com/go/getupdated
Share with colleagues
Rate this document
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for
HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as
constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
Microsoft and Windows are U.S. registered trademarks of the Microsoft group of companies. Intel and Intel Xeon are trademarks of Intel Corporation in the
U.S. and other countries.
4AA4-8082ENW, December 2013, Rev. 2