April 8, 2013
Breaking through HPC
Virtualization Barriers with KVM
HPC Linux for Wall Street
Keynote Panel Discussion
© 2009 IBM Corporation
KVM Panel Discussion
Twitter:
@OpenKVM
#openkvm
Moderator
Jean Staten Healy
Director, Worldwide Linux and Open Virtualization
IBM
Panelists
Moiz Kohari
Vice President, Advance
Platforms Engineering
London Stock Exchange Group
Mike Day
Distinguished Engineer and
Chief Virtualization Architect
Gil Tene
IBM
CTO and Co-Founder
Azul Systems
2
© 2013 IBM Corporation
Cloud Ready
Data Ready
Smarter Computing is the
IT Infrastructure that
powers a Smarter Planet.
3
Security Ready
© 2013 IBM Corporation
Technology has never been more important to business
For the first time, CEOs identify technology as
the most important external force impacting their organizations
2004
2006
2008
2010
2012
71% Technology factors
69% People skills
68% Market factors
Macro-economic factors
Regulatory concerns
Globalization
Source: IBM CEO Study 2012
4
© 2013 IBM Corporation
4
CIOs turn to innovative technologies to deliver outcomes
Technologies for innovation and solving specific business challenges
83%
74%
Analytics
Mobility
68%
Virtualization
61%
Cloud
58%
Security
Source: IBM CIO Study 2011
5
© 2013 IBM Corporation
5
IBM and Virtualization
A brief history of virtualization
Pure
OVA/oVirt
PureSystems
KVM goes upstream
Red Hat & IBM start KVM investment
Intel adds x86 hardware virtualization
System x &
Blade Center
IBM supports Xen
Virtualization on POWER
Virtualization on IBM mainframes
1964
6
1997
2004-2006
2007
Power Systems
2008
2009
2010 2011
2012
© 2013 IBM Corporation
6
IBM commitment to Open Source & Open Standards
Contribution
Enhancement
Strategic player in open source
communities and open standards
bodies – as contributor and consumer
Capture, focus, translate open
source innovation and open
standards interoperability into client
value
Growth
Pervasive Support
By leveraging open source and open
standards gain new users, enter new
markets and expand business
opportunities
7
Cross-platform and end-to-end
support for all HW, SW and services
through open source and open
standards
© 2013 IBM Corporation
New: KVM Center of Excellence for Wall Street
Grand Opening Event: May 2, 2013
Break through HPC virtualization barriers with KVM
High performance scalability for HPC workloads
Enterprise proven security
Cost effective virtualization
Wall Street COE KVM Demos
•
•
•
•
•
•
•
•
•
IBM SmartCloud Entry
IBM SmartCloud Provisioning
IBM Systems Director
IBM Tivoli Solutions
IBM PureSystems
IBM System x
IBM General Parallel File System
Red Hat Enterprise Virtualization
OpenStack
Center of Excellence Priorities
• Drive Awareness of KVM to clients
• Enable client briefings, trainings/skill development
and POCs
• Showcase IBM and Partner solutions with KVM
• Demonstrate IBM's commitment to KVM
8
© 2013 IBM Corporation
Why Open Virtualization with KVM?
Performance
Lower Cost
Cross Platform
KVM holds the Top 7
virtual machine
consolidation scores on
SPECvirt (1) and IBM x3650
M4 delivers best ever x86
virtualized TPC-C
benchmark results (2)
Customers report up
to 67% savings by
using KVM (3)
Support and certification
for all leading x86
operating systems
including Linux and
Microsoft Windows (4)
Security
EAL4+ Certification (4) plus
SE Linux enabling
Mandatory Access Control
between virtual machines
9
Virtualization
Management
Comprehensive range
of tools including Tivoli,
Systems Director and
an ecosystem of
ISVs/tools
(1) Source: SpecVirt_sc2010 results: http://www.spec.org/virt_sc2010/results/specvirt_sc2010_perf.html
(2) Source: http://public.dhe.ibm.com/eserver/benchmarks/news/newsblurb_x3650M4_tpcc_20130225.pdf
(3) Source: Case study on LetterGen, an IT services provider in Belgium: http://www.redhat.com/resourcelibrary/case-studies/lettergen-relies-on-red-hat
(4) Source: Red Hat Government Standards website: http://www.redhat.com/solutions/industry/government/certifications.html
© 2013 IBM Corporation
Stellar Performance Results for KVM
KVM Storage I/O – 1.5 Million IOPS in a Single Guest
Highest virtualized storage I/O rates ever reported for a single virtual
machine (guest) - 50% better than competing hypervisors
This benchmark result translates into (1) fast data transfers into guest
memory and (2) fast transaction processing in a KVM guest
10
© 2013 IBM Corporation
KVM Intra-host MPI Performance over Shared Memory Achieves
Bare-Metal Equivalence
Clearly shows an
opportunity to
improve distributed
HPC applications by
running components
in VMs on the same
host, and replacing
the remote
Messaging
Transport with Intraguest Shared
Memory
 http://www.linux-kvm.org/wiki/images/c/cc/2011-forum-nahanni.v5.for.public.pdf
11
© 2013 IBM Corporation
HPC-Friendly Developments in KVM and Linux
 Partitioning
– Partition the platform on natural boundaries: sockets or cores, memory controllers
– No resource sharing - every guest has dedicated hardware resources
– Local resources - every guest uses memory adjacent to the cpu that is running it
 CPU Isolation - Remove one or more CPUs from the general Linux scheduler using isolcpus
boot parameter, run processes on them using cpusets.
– Isolate CPUs to run KVM guests.
– Within the guest isolate vCPUs to run HPC workloads
– Make sure that soft IRQs are routed to guest CPUs
 IRQ Balancing - control which CPUs process interrupts via the /proc/smp_affinity file
 Shared Memory and Zero-copy Transports
 Scalability - 160 Cores, 4 TB per host supported
 Virtualization Hardware Performance 10^2 increase in VMEXIT performance since 2005
 SR-IOV High-Performance Adapters and Linux VFIO
12
© 2013 IBM Corporation
London Stock Exchange
 The London Stock Exchange is at the heart of the
global financial market and is home to some of the
largest, most successful and dynamic companies in
the world.
 The Exchange has been a pioneer in applying Linux
to front and back -office workloads and is now
applying KVM to new and experimental financial
applications
Moiz Kohari
Vice President, Advance Platforms
Engineering
London Stock Exchange Group
Responsible for low-latency specific developments for
the exchange platform and advance research for posttrade components for clearing and settlement purposes.
13
 The Exchange is the most
international of all the world’s
stock exchanges, with around
3,000 companies from over
70 countries admitted to
trading on its markets.
 From conducting its business
in the coffee houses of 17th
century London, the
Exchange is one of the
world’s oldest stock
exchanges and can trace its
history back more than 300
years.
 Live financial broadcasts are
transmitted throughout the
day from the Exchange's own
TV studios.
 Over 400 firms, mainly
investment banks and
stockbrokers, are members of
the London Stock Exchange.
© 2013 IBM Corporation
Azul Systems
 Azul Systems delivers Java Virtual Machines (JVMs) with
unsurpassed consistency, scalability, manageability and
production-time visibility.
 Zing, Azul’s high-performance JVM supports highly consistent
execution of latency sensitive Java workloads
 Eliminates garbage collection as a concern for enterprise
applications at all scales and data set sizes
 Customers using Zing for pause-free execution ranging
anywhere between 1GB and 100s of GB per process
 Deep low-latency Java expertise. Many cutting-edge financial
accounts. Partners and customers in Algo, Risk, and HFT.
Gil Tene
Azul is a venturebacked company
founded in 2002.
Based in Sunnyvale,
California, with offices
around the globe
Azul’s Zing JVM is
optimized for Linux,
x86 servers
Numerous Industry
firsts, including the
low latency C4
garbage collector
CTO and Co-Founder
Azul Systems
14
© 2013 IBM Corporation
The Latency Consistency Problem
99%’ile is ~60usec
15
Peaks are 30,000%
Higher than “typical”
© 2013 IBM Corporation
Example: a low latency Java based trading system
“Other” JVM
16
Zing
© 2013 IBM Corporation
Azul, Zing &
 Azul has experience supporting both large-memory and
latency-sensitive workloads running on KVM
 We encounter plenty of KVM use, in production, for both large
footprint and human-response-time applications
 We see much interest in expanding production use to more
latency sensitive applications
 Already there for +/- 100msec ranges
 Not quite there for +/- 1msec ranges
 But showing lots of potential…
Gil Tene
Azul is a venturebacked company
founded in 2002.
Based in Sunnyvale,
California, with offices
around the globe
Azul’s Zing JVM is
optimized for Linux,
x86 servers
Numerous Industry
firsts, including the
low latency C4
garbage collector
CTO and Co-Founder
Azul Systems
17
© 2013 IBM Corporation