Securing OpenStack
Chris C. Kemp
About Chris C. Kemp
• OpenStack Co-Founder
• Former CTO for IT, NASA
• Founder and CEO of Nebula, Inc.
Chris C. Kemp / Twitter @Kemp
Open source software for building
private and public clouds.
OpenStack is a wonderful choice for the security-minded
enterprise’s private cloud *if* best practices are followed during
all stages of implementation and operation.
I’m not *only* making outrageous claims, I’m going to make
some points to back this up with the rest of my talk.
OpenStack is a *true* cloud platform
1.
2.
3.
4.
5.
On-demand … through self-service interfaces
Elastic … dynamically scale up and down
Shared … pooled resources
Metered by use … at high level of granularity
Accessible … broadly over the network
OpenStack Details
• Multi-tenant, massively scalable, open source
cloud operating system.
• Supports various hypervisors, including:
Xen/XenServer , KVM, Hyper-V, VMWare/ESX,
Linux Containers (LXC), QEMU, UML
• Flexible network and storage options
• Apache 2.0 open source license
Why Build Private Cloud?
1. Maturity. Ability to overcome barriers to entry
related to culture, process, technology, experience,
and tools.
2. Performance. Need to deploy an application near the
data and services that are already deployed on
premises - lower latency and increased bandwidth.
3. Security. Must keep data inside Company’s security
perimeter, where we trust security team, tools, and
processes.
4. Cost. TCO much higher for predictable IaaS workloads.
5. Architectural Constraints. Application is not
architected to run well in public cloud, or has unique
technical requirements.
By the Numbers
• Community includes 2300 people from 153
Companies
• Over 100 active committers with 250K lines of
code
Who is Involved?
And many more…
OpenStack Conceptual Architecture
So, now we know a little something about OpenStack..
…and we’re forming some initial opinions….
CIO
“Sounds exciting!”
CSO
“Sounds target-rich..”
Security and the OpenStack project
OpenStack Security Community Highlights
OpenStack project groups
–
–
–
–
Vulnerability response
Formalizing security
Audit projects
Multiple security-centric
blueprints
– ongoing code improvements
Commercial efforts
– Professional penetration
Testing / API fuzzing
– Sponsored bugfests with
growing participation
– Active and ongoing source code
review process
Bringing defense in depth to OpenStack clouds
Assuming you’ve laid the foundation….
Before we begin we’ll need the governance, guidance, and groundwork that
will define the requirements..
•
Compliance and Audit
•
ERM
•
Legal and Electronic Discovery
•
Information Lifecycle
•
Corporate policy
•
Portability
•
Interoperability
•
Architecture
•
Operations “touch points”
….let’s take a closer look at the technologies in play.
OpenStack Conceptual Architecture
OpenStack Logical Architecture
OpenStack - Under the hood…
• Mostly implemented in Python
• REST and WSGI communication
between services
• Multiple application choices to
implement backend
• databases
• queue
• networking
Everything needs to be hardened and continuously monitored.
…Luckily, we have a few best practices for doing this stuff with
open source software.
OpenStack compute service (nova)
• Equivalent to Amazon
EC2
• Runs virtual machines on
hypervisor of your choice
• Includes support for block
volumes external to
hypervisor
• The architecture of nova
allows for massive parallel
scaling, but to get there
requires some complexity.
Underlying technologies
• nova-compute
– speaks to libvirt, XenAPI,
etc.
• nova DB
– SQLalchemy & a SQL DB
• queue
– Python Kombu+AMQPlib
– rabbitmq
Underlying technologies
• Nova-network
– Provides connectivity
• Nova-volume
– provides volume API
– Backended with iscsi,
sheepdog, ceph, etc
• nova-scheduler
– Determines available resources
– Assigns workloads
OpenStack Compute Security Considerations
• Secure your hypervisor
– This is a topic for another talk… but certainly not trivial.
• Choose a database
– Consider high availability
– Enhanced security configuration
• Message Queue
– Harden the queue software’s configuration
– Monitor and correlate queue messages
• Choose filesystem(s)
– Enable filesystem’s security features
– Deploy hardened daemons
– Monitor activity
• Monitor API access
OpenStack object store (swift)
• Similar to Amazon S3
• Configurable number of
duplicate object replicas
• Supports geo-replication of
objects
• Internally:
– memcache provides caching for
scale and speed
– SQLite
– rsync
– python greenlets/eventlet
OpenStack Object Store Security considerations
• Properly secure underlying technologies
– memcached, rsync
• Implement and test RBAC
• Restrict admin read access to objects
– Least privilege, is admin read access required?
• Integrated information lifecycle
– Automate / integrate IL processes when possible
• Monitor & correlate API access
– Record all access to the object store
OpenStack Dashboard (Horizon)
• Standard webapp stuff
• django-based
• Uses keystone for authN/Z
OpenStack Dashboard Security considerations
• Use enterprise authentication behind
keystone
• Standard webapp hardening process
• Protect credentials
• Monitor access and correlate
OpenStack image service (glance)
Provides a repository for VM images and
snapshots
• SQL for metadata
• Supports for multiple backend
filesystems
• Ceph
• S3 / Swift
• Local FS
OpenStack Image Service Security Considerations
• Choose distributed filesystem(s)
– Enable filesystem’s security features and
configure hardened endpoints
– Monitor activity
• Choose a database
– Consider high availability
– Enhanced security configuration
• Audit
– Automate audit of images for OS controls
• Patch management
– Automate patching and configuration updates to
OS images
OpenStack Identity Service (keystone)
• authN and authZ provider for OpenStack
• Rewrite introduced a new architecture
– Straightforward integration with commercial / external auth
products and solutions
OpenStack Identity Service Security considerations
• Use backend enterprise authentication provider –
– OpenStack is not an identity project
– Keystone’s backend API provides easy integration for
authN, and acceptable authZ
• Monitor API access
– Attempts
– Failures
• Logging - Monitor and correlate
– Monitor identities across OpenStack
– Debug loglevel is informative but sensitive
Other parts of the OpenStack ecosystem
• OpenStack incubated projects
– Two exciting networking projects
• Quantum
• Mélange
• Other interesting OpenStack projects
– Database-as-a-Service
– Dashboard enhancements and plugins
– Hybrid cloud functionality
(cloudgateway, etc)
…Zooming back out
Enough of the trenches. This is a keynote, after all.
OpenStack seems to be made up of defensible technologies
–
–
–
–
–
–
–
–
–
Lots of readable python
Databases: sqlite, mysql, postgres
Message queue: rabbitMQ
Distributed Filesystems: gluster, ceph
Hypervisors: Xen, KVM, ESXi, Hyper-V*
memcached
django
authN / authZ API interface
Linux security features
OpenStack Logical Architecture
…But It’s the responsibility of the implementer
to turn the “security switches” to “on.”
So - OpenStack isn’t a production-ready cloud?
• Most technical security controls required for compliance are
NOT built in to OpenStack.
•
• That shouldn’t dissuade you.
• The building blocks are all in place.
Monitor
Harden
Integrate
Hardening OpenStack system environments
• Restrict network and data access to least privilege
• Enable security features of underlying software
• Configure security features of the underlying OS
• Harden the hypervisor
• Use PKI for SSL
• Implement database security
Integration - benefits to even the playing field
Some integration points:
•
•
•
•
•
•
SIM/ SIEM
IR automation / Live Forensics
CMDB / Service desk
Asset mgmt / Patch mgmt
Auditing process automation
IPAM
Integrating the underlying cloud
framework into these elements yields
huge benefits
Monitoring – Benefits in the open cloud
OpenStack is powerful foundation to
build advanced security controls
Building complex solutions becomes
relatively simple
•
•
•
•
SIEM sees significant benefits
Automated Incident Response
Cloud-wide flow monitoring
Security appliances: IPS-aaS, FW-aaS, …
Defense in depth of workloads in cloud
• An integrated defense in depth strategy can benefit
from open source software and from private cloud
• OpenStack is a great example
Back out to “The big picture”
- CSA Security Guidance - Critical Areas of Focus in Cloud
Mapping the Cloud Model to the Security Control & Compliance Model
Looking ahead - 2012 and OpenStack security
The coming year looks to be very exciting for the OpenStack project, and
specifically for OpenStack security.
OpenStack-based products could offer powerful security options.
A few ideas we’re kicking around at
:
• Interesting security-as-a-service potential
– Quantum provides some of the missing building blocks needed for metered
and scalable security controls on demand in OpenStack
• IR process integration offers excellent coverage
– Potential for huge efficiency improvements in remediation of incidents and
live response activities
• SIM / SIEM benefits
– Coverage over large infrastructure increases value of SIM integration
– The visibility and control that IaaS offers eases SIM complexity
Conclusion
• OpenStack is a flexible foundation
– It’s a viable option, but not necessarily right out of the box
– It’s not right for every workload or enterprise
– Its open-ness is a big plus for security
• Still some significant unanswered security questions
– Expect to see commercial OpenStack-based products
bridging this gap
• Exciting new developments improving the security of
OpenStack are happening every day
Thanks for listening!
Chris C. Kemp
Twitter: @Kemp