HPC and Cybersecurity
How to Efficiently Protect High Performance Computation?
Laurent Bobelin
LIFO
INSA CVL
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
1 / 46
Seminar Topic
HPC systems are mature since decades:
Very good utility of resource
Well known paradigms, tools and algorithms
Threats on the rise for such systems:
Spy
Falsify
Stop
Destroy
How to secure High Performance Computing wile preserving good
performances?
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
2 / 46
Who Am I?
Dr. Laurent Bobelin
PhD on discovering network performances in Grid Computing (2008)
8 years in industry sector on HPC
15 years in academics on HPC and cybersecurity
Current Position
Assistant Professor @ INSA CVL/Laboratoire d’Informatique
d’Orleans (LIFO) lab
Head of System and Data Security Team
Research topics: cybersecurity and privacy, Zero Trust Architectures,
HPC
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
3 / 46
INSA CVL and Bourges
INSA and INSA CVL
INSA Group: 7 engineering schools delivering more than 10% of all
French engineering diplomas.
INSA Centre Val de Loire (CVL): INSA’s group Antenna in Centre
Val de Loire region
Two campuses: Bourges and Blois
INSA CVL @ Bourges
Bourges: major military related activities
INSA CVL @ Bourges: engineering school specialized in safety and
security
My courses: engineering diploma in cybersecurity
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
4 / 46
Some Introductory Words about HPC
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
5 / 46
Some Introductory Words about HPC
Serial Computation
Serial Computation
Traditional serial computation:
Problem → discrete series of instructions
Instructions executed sequentially on a single processor
Only one instruction may execute at any moment in time
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
6 / 46
Some Introductory Words about HPC
Serial Computation
Serial Computation - 2
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
7 / 46
Some Introductory Words about HPC
Parallel Computation
Parallel Programming
Parallel computing: simultaneous use of multiple compute resources to
solve a computational problem:
A problem → discrete parts that can be solved concurrently
Each part → series of instructions
Instructions from each part execute simultaneously on different
processors
An overall control/coordination mechanism is employed
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
8 / 46
Some Introductory Words about HPC
Parallel Computation
High Performance Computing
High Performance Computing relies on:
Parallel Programming
Distributed Computing (using multiple machines to run)
Computing Resources:
A single computer with multiple processors/cores
An arbitrary number of such computers connected by a network
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
9 / 46
Some Introductory Words about HPC
Parallel Computation
Parallel Programming - 2
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
10 / 46
Some Introductory Words about HPC
Parallel Computation
Parallel Programming- 2
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
11 / 46
Some Introductory Words about HPC
Parallel Computation
Parallel Programming Paradigms
Some popular techniques when writing HPC applications:
Message Passing: send and receive messages to communicate
Shared Memory/shared storage: multi-core applications mainly
Master/slave: a unique master sends work to nodes that acts like
slaves
Stateless services: REST-like services providing features to other
nodes.
Pipelining
Divide and Conquer
Are all those techniques impacted the same way when securing
applications?
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
12 / 46
An Introduction to Cybersecurity Concepts
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
13 / 46
An Introduction to Cybersecurity Concepts
Main Concepts
Cybersecurity Overview
Cybersecurity properties
Cybersecurity applied
Basic properties to guarantee:
Where to ensure these properties:
Confidentiality
Securing exchanges
Authentication
Securing storage
Integrity
Securing computation
Tools
Approaches
Usual tools:
Securing by ...
Cryptography
Design
Formal models
Continuous enforcement
System design
Continuous audit
End-to-end security ensure properties in the whole system
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
14 / 46
An Introduction to Cybersecurity Concepts
Main Concepts
Main attack types
Destroying or Stopping the System
DDoS (Distributed Denial of Service): attacker try to make the
service stops to respond service
Ransomware: attacker encrypt data and asks money for the key
...
Spying and Falsifying
Man-In-The-Middle: attacker replaces one of the equipment in
between two hosts communicating
Inference from observation: attacker infer knowledge from the data he
can observe on the network
...
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
15 / 46
An Introduction to Cybersecurity Concepts
Threats and HPC
Threats and Attackers for HPC
Why attack HPC computation?
Steal scientific knowledge
Identify what is simulated
Steal industrial knowledge
Actual Threats for HPC
Computing power is often remotely accessed
Most of the time, shared
Sometimes externally hosted
Zero Trust thread model fits perfectly with threats
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
16 / 46
An Introduction to Cybersecurity Concepts
Threats and HPC
Threat Model: Zero trust
Untrusted Communications
Attacker has almost full control of the channel between end-systems.
Attacker can read anything on the network and undetectably remove,
change, or inject forged packets onto the wire.
Attacker can generate packets that appear to be from a trusted
machine.
Thus, even if the end-system with which you wish to communicate is itself
secure, the environment provides no assurance that packets which claim to
be from that system in fact are!
Untrusted Execution Environment
An attacker may be curious or want to falsify your data:
Attacker can read any memory part which is not encrypted
Attacker can modify any memory part which is not encrypted
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
17 / 46
An Introduction to Cybersecurity Concepts
An Example of an HPC Application
Application Example
METAS
Computational Fluid Dynamics based on Finite Element Method
(mesh-based)
Simulating explosions into buildings/structures
Methods:
Mesh modeling targeted systems
Gradient calculation
Conservation Element/Solution Element based methods that has to be
protected (industrial)
What to be secured
Mesh (obfuscation)
Computation integrity
Library implementing CE/SE methods
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
18 / 46
Securing HPC Networking
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
19 / 46
Securing HPC Networking
Problem overview
Perimeter-based security
Traditional network security architecture:
Different networks (or pieces of a single network) into zones,
Contained by one or more firewalls
Each zone is granted some level of trust
Which determines the network resources it is permitted to reach
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
20 / 46
Securing HPC Networking
Problem overview
The (HPC) Fortress
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
21 / 46
Securing HPC Networking
Problem overview
Possible attacks
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
22 / 46
Securing HPC Networking
Problem overview
Example Attack Progression
Attacks:
1. User/Employee targeted via phishing email
2. Corporate machine compromised, shell shoveled
3. Lateral movement through institution network
4. Privileged workstation located
5. Local privilege escalation on workstation—keylogger installed
6. Cluster administrator password stolen
7. Compromised update uploaded on privileged workstation
8. Access to DMZ using admin account
9. Administrator password used to elevate privileges on front-end or
cluster
10. Install compromised update to initiate data leak
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
23 / 46
Securing HPC Networking
Problem overview
Attack and mitigation
Attacks:
Attack may be initiated by the victim!
Firewall may help mitigate ...
... but most of the time disguised as regular traffic
... and firewall slows everything
Usually:
Attacker drops a Remote Access Tool (RAT) into a network
Moves laterally
firewall can’t stop lateral movements
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
24 / 46
Securing HPC Networking
Problem overview
Problem overview
How to:
1. Stop lateral movements?
2. Deal with unsecure untrusted networks?
3. Do better than perimeter-based security?
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
25 / 46
Securing HPC Networking
Security Models and Tools
Tools That Can Be Used for Security: SDN and NFV
SDN
Allows to do fine-grain network segmentation, based on user, device,
targeted application...
Supported by any (modern) device (physical or virtual)
Allow central management and orchestration
NFV
Allows to deploy services such as firewall or NAT
Supported by any device (physical or virtual)
Allow central management and orchestration
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
26 / 46
Securing HPC Networking
Security Models and Tools
Usual Route Computation
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
27 / 46
Securing HPC Networking
Security Models and Tools
Software Defined Networking Route Computation
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
28 / 46
Securing HPC Networking
Security Models and Tools
Network Function Virtualization (NFV)
Network Function Virtualization (NFV) : virtualizing network functions
(NAT, load balancing, firewalls) to deploy it dynamically on networks
devices/virtual network devices
Network functions are software with standard API and concepts
Network devices act as hypervisor hosts, deploying and hosting
network functions
Network Functions == (composition of) services, with dependencies
Complementary to SDN: shares the data plane/control plane concept
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
29 / 46
Securing HPC Networking
Security Models and Tools
NFV: overview
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
30 / 46
Securing HPC Networking
Zero Trust Networks
Zero Trust Overview
Zero Trust network is an answet to the Zero Trust threat: Do not give
trust to anyone without evidences!
Allow connections only if we have enough confidence on how secure is
the network flow
Each time a flow is started, evaluate if we trust enough the flow
based on:
User identity
Device security
Usual workload
Application trust
...
Use SDN to centralize decision and forbid/allow access
Use NFV to deploy packet analysis
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
31 / 46
Securing HPC Networking
Zero Trust Networks
Zero Trust Big Picture
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
32 / 46
Securing HPC Networking
Zero Trust Networks
HPC and Zero Trust
Zero trust is an efficient answer to threats on HPC
Nowaday devices allow to apply zero trust principles to HPC
Anyhow if not done wisely, can be at an excessive cost:
Ciphering each message may slow the application
Long-term communication may suffer from authorization renewal
Need to adapt your HPC application to reduce security overhead
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
33 / 46
Securing HP Computation
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
34 / 46
Securing HP Computation
Problem Overview
Securing Computation
Threats
If the host is compromised:
Attacker can read your data
Attacker may falsify your data
Attacker may corrupt your program to stop it
In case of a shared infrastructure, how to stop side attacks from malicious
computation on the same host?
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
35 / 46
Securing HP Computation
A Few Words On Hardware-based Security
Hardware-based Security
Trusted Execution Environment
Some portions of your host resource guarantees code and data loaded
inside to be protected with respect to confidentiality and integrity
Small portions of your resources
Very slow!
Example: Intel’s Software Guard Extension (SGX)
Not suitable for HPC!
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
36 / 46
Securing HP Computation
Insights on VMs and Container security
VM Hardware-based Security
x86 privileged rings:
Separate user, drivers
and kernel
Privileged instruction set
Virt support adds an
additional ”ring”
(AMD-V, VT-X)
But ... What if we don’t trust our host?
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
37 / 46
Securing HP Computation
Insights on VMs and Container security
VM Hardware-based Security: MKTME and SEV
Security solutions oriented towards VMs:
Intel (MKTME) and AMD (SEV) solutions
Adding protocols for key exchange at VMs deployment
Adding new instruction set at ring -1
On-the-fly hardware memory encryption, securing both code and data
Very low overhead, but init time increased
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
38 / 46
Securing HP Computation
Insights on VMs and Container security
Container Security
Basic Container Security
Container: isolated process within system:
Full vision of container from host!
Secure container like gVisor based on OCI isolate more the container
from the host, but does not consider the problem of untrusted hosts
Kata-container
Lightweight VM + OCI-based isolated containers
Lightweight VM hosts secure containers
kata-container + MKTME/SEV allows to isolate both container from
host and host from containers
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
39 / 46
Securing HPC Load in Practice
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
40 / 46
Securing HPC Load in Practice
Consequencies of Securing Network and Computation
Computation
MKTME/SEV kata-container allows to run securely code
Slower init + isolation of secure processes
If not all the application has to be secured, unsecure but faster
container and secure ones can coexist
Networking
Zero-trust: slower flow init
Auth renewal: need to configure authorization before communications
happens: pre-init communications is a solution
If unsecure communications can be done, does not encrypt all data.
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
41 / 46
Securing HPC Load in Practice
Adapting an Application
Application rewriting:
Added App Master
Execution Engine
Infrastructure:
Added normal & secure
worker support
Zero Trust configuration
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
42 / 46
Securing HPC Load in Practice
Overview
Code Structure changes
Use master/slave paradigm with some secure slaves
Secure slaves based on kata-containers
Maintain or pre-init connections
Infrastructure changes
Deploy SDN-based zero trust
Adapt enforcement policies to HPC load:
Extend/adapt time of auth renewal between connections
Adapt enforcement policies to the master/slave application structure
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
43 / 46
Conclusion
Plan
1 Some Introductory Words about HPC
2 An Introduction to Cybersecurity Concepts
3
4
5
6
Main Concepts
Threats and HPC
An Example of an HPC Application
Securing HPC Networking
Problem overview
Security Models and Tools
Zero Trust Networks
Securing HP Computation
Problem Overview
A Few Words On Hardware-based Security
Insights on VMs and Container security
Securing HPC Load in Practice
Conclusion
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
44 / 46
Conclusion
Conclusion
Lesson Learned
Securing applications needs some in-depth modifications of the
application structure
Need to adapt Zero Trust to HPC load
Ongoing Work in METAS Example
Secure Computation is done
Ongoing work:
Adapt Zero Trust policies
Mesh obfuscation
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
45 / 46
Conclusion
Thanks for you attention!
Laurent Bobelin (LIFO/INSA CVL)
HPC and Cybersecurity
46 / 46