The Virtual Computing Lab (VCL)
Andy Rindos, Head RTP CAS &
Co-leader IBM Cloud Academy (IBM)
© 2010 IBM Corporation
VCL: One of the first state education clouds built on IBM technologies
One of the first true clouds ever developed,
specifically designed for education and research
A partnership with IBM & others to develop a
cloud computing solution specifically targeted
for education & research
• VCL project planning started in 2002-2003
• NCSU production use started in 2004
• Support for both HPC & non-HPC workloads
• Awards incl. Computerworld Honors Prog. Laureate
• Currently NCSU VCL
- Open to 40,000+ NCSU student & faculty, as well
as 250,000+ in NC
- 2,000+ Blades in production
- Delivers over 500,000 CPU hours to general
reservations (desktops, sub-clouds, classroom, etc.)
annually & over 13,000,000 HPC CPU hours
Substantial support
from IBM for VCL
development
through SURs, CAS
Faculty Awards, &
BladeCenter / SWG
development
• VCL sites in US include NC Community College
System, many UNC System universities, VA VCL
(GMU, ODU, VA Tech, NSU, etc.), GSU, UMBC,
MSU, GSU, TSU, Southern, Cal State System, etc.
• User sites outside US include Canada, Japan, China,
India, Middle East, Eastern Europe, etc.
Dr. Mladen Vouk, “the father of VCL”
Professor and Head of Computer Science Department, and Associate ViceProvost for Information Technology, North Carolina State University, Raleigh, NC
WebSphere
Technology
Institute
assisted in the
creation of the
VCL Apache
project
IBM Smart
Cloud for
Education:
VCL solutions
include GBS
VCL Quickstart
Services, etc.
Smarter Classroom through Cloud Computing:
The Virtual Computing Lab (VCL) in North Carolina
+ NC Community College System
+ NC K-12 school districts
WFU
NCA&T
OC12 (622 Mbps Circuit)
OC48 (2.4 Gbps Circuit)
vcl.ncsu.edu
DWDM (10 Gbps Enet)
VCL is a true education cloud computing solution developed by
NC State & in production for nearly a decade.
© 2011 IBM Corporation
North Carolina Cloud
Representing the opportunity for a seamless integration of
resources from the state of NC and the IBM Leadership Data
Center facilities through cloud computing
IBM
The University
of North Carolina
System
Leadership Data Center
Powered by VCL
NC STATE
UNIVERSITY
Public Schools of
North Carolina
K-12
MCNC
State of North Carolina
Government
North Carolina Community
College System
Infrastructure that enables
“Race to the Top”
Cost-effective, secure and “green” statewide on-demand delivery and sharing of
educational content and state-of-the-art
information technology resources and
support. Note that since VCL is a general
purpose production cloud computing solution, it
can also support all state government.
ancouver Education
Cloud:
Alberta Education
Cloud:
Led by Cybera, for U
Alberta, etc.
Nova Scotia Education
Cloud:
Various Ontario-based
Education Clouds:
Led by Dalhousie U,
NSCC, etc.
Led by U Ottawa, UWO,
Carleton, etc.
VCL-based Education & Research Clouds
in North America
Cal State System
TitanCloud:
Cal State Fullerton,
East Bay, etc.
HBCU* Cloud: NCA&T, NCCU,
WSSU, Howard, Southern,
TSU, MSU, AA&MU, NSU
Drexel U, PSB, etc.
Education Clouds
(Pennsylvania)
Chicago IIT Education
Cloud
Various TN-based
Education Clouds:
Other clouds starting in Eastern Europe,
France, Taiwan, Jordan, etc.
* HBCU = Historically
Black Colleges & Universities
NC State Education
Cloud: NC State, &
UNC System, NC
CCs, K-12
Clemson, U of SC, , etc.
Various GA-based
Education Clouds:
Southern U Education
Cloud (Louisiana)
Tec Monterrey, etc.
Morgan State, UMBC,
Hood, etc.
Various SC-based
Education Clouds:
Alabama A&M
Education Cloud
UTEP, Paririe View, etc.
Various Mexico-based
Education Clouds:
Various MD-based
Education Clouds:
Virginia VCL: GMU, VT,
ODU, NSU, etc.
TSU & TBR, Columbia
State CC, etc.
Various Texas –based
Education Clouds:
SUNY, Marist, etc.
Howard U Education
Cloud (Wash DC)
MST Education Cloud
(Missouri)
UNLV Education Cloud
(Nevada)
Various NY-based
Education Cloud:s:
GSU, AASU, etc.
FIU Education Cloud
(Florida)
UPRM
Education
FIU
Education
Cloud Cloud
(Puerto
Rico)
(Florida)
The Internet2 VCL Research Cloud (sponsored by Internet2 Net+ Services,
NC State, George Mason U and IBM)
Internet2 Net+ Services recently approached NC State with the desire to create a VCL-based
research cloud and services proof-of-concept for member institutions (see membership at
http://members.internet2.edu/university/universities.cfm)
NC State (research cloud/services host) invited IBM to participate, along with George Mason
U as the initiative sponsor.
- NC State has turned on 80 servers within its (multi-thousand server) VCL cloud in NC, and will be
making the cloud available to I2 members with accepted projects.
- GMU has agreed to be the initiative sponsor, with the following inaugural partners: NC State,
GMU, VA Tech, GSU, FIU, Clemson, U of SC and UTEP.
- IBM has provided $170K in 2012 Faculty Awards (combined from RTP CAS, GUP, STG UA).
- Internet2 hopes to announce the project at an upcoming member meeting.
- ICA will seek to expand participation to non-I2 ICA and ICT Vision 2020 (HBCU Cloud) members
(and seek international hybrid cloud research).
© 2012 IBM Corporation
2012 Faculty Awards for Internet2 Research Cloud
University
PI
Title
NC State U
Mladen Vouk
vCentennial Campus for a Smarter
Planet
George Mason U
Sharon Pitt
VCL NetPlus Services
VA Tech
Mark Gardner
Extending the Apache VCL for greater
utility and reach
Georgia State U
Yi Pan
Data replication strategies in cloud
computing systems
Clemson
Jim Miller
Integration of Openflow capabilities
within the VCL
U of South Carolina
Michael Huhns
Support for OpenStack in VCL
Florida Intl. U
Masoud Sadjadi
Supporting network provisioning in
VCL
UT El Paso
Pat Teller
Integrating & showcasing IBM
Platform HPC solns. under VCL
© 2012 IBM Corporation
Apache VCL cloud management stack
(code: GUI, db, management daemon, image library)
NC State Education Cloud:
Built with IBM technologies
Smart Cloud
Enterprise
Public-private hybrid
cloud enabled by VCL
plug-in
VCL API-driven plug-ins
Self-service portal
HPC subcloud
Using LSF manager
at NC State, hardware
deployment via xCAT
(Exploration around
Condor at Clemson)
8
Provides clusters (incl.
Hadoop), bare metal or
virtual machines
(VMware, KVM, etc.)
with any SW image
suite; other clouds; HaaS
etc.
IaaS subcloud
PaaS subcloud
Using Smart Cloud
Provisioning (SCP;
KVM-based, currently
using HSLT, moving
to OpenStack)
In futrure, could be
provided by IWD or
Pure Systems. Can
also deploy Smart
Education Platform.
Administration
subcloud
With latest release
VCL will support ERP,
LMS and other
business admin apps
VDI subcloud
Vmware View, Citrix
XENDesktop, RHEV,
Verde, etc.
Amazon
EC2
Apache VCL (cloud management stack, with a self-service portal)
Apache VCL (cloud management stack, with a self-service portal)
xCAT plug-in
(bare metal for
all IBM server
platforms, most
hypervisors)
IBM Workload Deployer
IBM Smart Cloud
IBM Workload
Provisioning (IaaS)
Deployer (PaaS)
with
OpenStack (IaaS)
and IWD (PaaS)
plug-ins complete
Native KVM & VMware
plug-ins: using
Libvirt, can support
all hypervisors
IBMSCE, EC2,
popular VDIs, other
plug-ins
Deploys to all xSeries, soon
enabled for Power and Z
All major hypervisors
supported
Work with IBM BlueGene
Integrated 1Q2012
Server technology: Hardware agnostic, but NC cloud, originally built on IBM HS2x servers,
HV Shims
is moving to IBM PureSystems (next generation) servers; IBM iDataPlex at GSU
VCL code can be installed on BFC (BladeCenter Foundation for Cloud)
Storage technology: NC cloud uses IBM SONAS - and now IBM StorWize V700
virtual storage
Underway: Tighter integration with open sources Quantum/OpenFlow and OpenStack;
IBM Platform, IBM PureApplications and IBM FSM
z/VM
Cloud architecture for NC VCL: 250K users, ~3K servers
Apache has been successfully
replaced by IBM HTTP Server (IHS)
Integrated ITUAM (IBM Tivoli
Usage & Accounting Manager)
SPSS SaasS on VCL
Create VCL images of all AI
SW, starting with Rational.
& analytics (incl. healtcare IT &
Analytics subclouds)
MySQL has been successfully
replaced by IBM DB2 &/or Derby
IBM TCR (Tivoli Common
Reporting) has been running in
VCL for several years
CastIron has been deployed as a VCL image,
IBM Mirage technology
has been running in VCL
for several years
Integrate WebSphere.PaaS solns.
ITM agents are already installed
on many images in the VCL Image Library,
Enable VCL to deploy to
Power 7 blades, System z & BlueGene.
IBM StorWize V7000 Unified
virtual storage is running n VCL.
IBM SONAS is running in VCL.
IBM Director (with VSPs) has been
successfully integrated with VCL (POC )
Integrated Tivoli Smart Cloud Provisioning
as a provisioning node
Integrated IBM Smart Cloud plug-in
as a provisioning node
VCL-IBM
Integration Efforts
VCL has been installed on the IBM
BladeCenter Foundation for Cloud
An image of the VCL code has been
been successfully installed in IWD
Already Integrated
10
Integration underway
© 2009 IBM Corporation
IBM SmartCloud for Education
 Apache VCL-based SmartCloud for Education
solutions:
– GBS VCL Quickstart Services in US; Silver Lining (business
partner) outside the US
– Services-integrated blade servers and storage for VCL (including
IBM PureFlex servers, StorWize V7000 virtual storage at NC
State)
– IBM Smart Cloud Provisioning (SCP) & other Tivoli products
– API interface to the hosted IBM SmartCloud Enterprise
– Special IBM SPSS pricing/licensing for VCL users
© 2012 IBM Corporation
Opportunities for Partnership
 Apache VCL open source development community:
– http://vcl.apache.org
– Mailing list vcl-dev@ivcl.apache.org ; also vcl-user, vcl-commits
– Web Site http://cwiki.apache.org/VCL
 “VCL bootcamp” at NC State and at select campuses
 VCL “Virtual Office Hours” bi-weekly
 IBM Cloud Academy and ICA CON
 IBM University Relations (SURs, Faculty Awards, CAS, Academic Initiative)
© 2012 IBM Corporation
Collaborating on transformative models & approaches
IBM Cloud Academy
The mission of the IBM Cloud Academy is to provide an
organization for K-12 schools and higher education
institutions who are actively integrating cloud technologies
into their infrastructures to share best practices in the use
of clouds and to collaborate with partners to create
innovative cloud technologies and models.
From IBM Cloud Academy Charter as developed
by initial member institutions, 1Q 2010
© 2010 IBM Corporation
IBM SmartCloud for Education
 What Next?
– IOC 1.5 is installed on VMs in the NC Cloud to support energy management
research with the NSF-sponsored FREEDM Center
– BlueMix is being developed to provide PaaS and an environment to build higher
value services from other smaller service building blocks
• Built on open source technologies – CloudFoundry, Cook, etc.
• Plans to establish a BlueMixPOC at NC State to integrate various services (VCL
computing/storage.SW + automated lab measurement tools + collaboratory
environment etc.) in support of virtual classroom, virtual curricula (e.g.,
mainframe program distributed across NCAT, NCCU, NC State & ECU), virtual
university (vCentennial Campus initiative - see below)
• Student classroom projects (in the NC State cloud computing class) augmenting
Cloud Foundry, IWD, etc.
• Potential platform for MOOC/OOC efforts by NC State ECE
• Integration with the vCentennial Campus Initiative, whereby NC State is
virtualizing all campus resources and services (datacenter through the VCL,
libraries, research labs, etc.) as an alternative to traditional brick-and-mortar
outposts abroad (e.g., NYU at Abu Dhabi, CMU at Qatar, etc.)
© 2012 IBM Corporation
Back-up
15
2012 IBM Corporation
© 2010
Global Business Services
IBM Approach to VCL Quickstart Services
IBM
Deliver Installed & Configured VCL Environment
Via Remote Collaboration and Access to Servers
Technical Team
Lead and PM
Installation & Configuration:
 Linux, Apache, Database, VCL Code, etc.
 Creation of Prototype Virtual Images
 Verification of Installation, Configuration, Virtual Images
Instruction & Training:
 VCL Administration
 Creation & Management of Virtual Images
 Level 1 & Level 2 Trouble Shooting Techniques
Additional Support Available:
 Creation & Management of Virtual Images
 Upgrades to base VCL code
Client Project Team
 Works with Technical Lead
and PM to define Scope &
Schedule
 Provides for remote access
to hosting servers
 Assures required SW
Licenses are provided as
needed
 Accepts VCL Environment
 Assists in transition to
Support
IBM Technical Staff
Problem Reporting,
Tracking, and Resolution
Level II
© 2010 IBM Corporation
 1 BladeCenter or PureFlex chassis
 2 Ethernet Switch Modules (BNT
Layer 2/3 copper)
 Power supplies 3&4 (for 7 or more
blades)
 Chassis network module to
connect management node to
storage
– Fiber Channel - Optical pass
through
– iSCSI - Copper pass through
 2-14 HSxy or PureFlex Blades
 At least one blade configured to
attach to external storage for Image
Library (FC, iSCSI, …)
 Server for scheduler, database, and
management node
 Server(s) to deliver VCL services
Small VCL Configuration
Three Networks
Public, Private, Management
Intelligent Images, Security
ESM
MM
ESM
OPM
 Storage for Images
 FC or iSCSI storage array (few TB)
 SONAS or StorWize V7000
 Virtual or Bare-metal
 Mostly virtual (ESX/ESXi), KVM
 xCAT or another loader
Can support up to
300 concurrent users
under VM
17
2012 IBM Corporation
© 2010
VCL Home Page
Home Page
Open
Modular
Flexible
Scalable
Upgradable
Secure
BM & VM
Distributed
Cost-effective
VCL/CIO/SF/mav/Apr-11/V2
18
General Reservation
(VCL-Desktop, VCL-Server)
Short-term reservations (2-3 hours)
Long-term reservations
Need to explicitly manage
- state persistence
- timeout
- backups
VCL/CIO/SF/mav/Apr-11/V2
Frequently used image
19
loads very quickly
Group Reservations
Group Reservation
(VCL-Class)
This type of reservation
does not pay attention
to topology, just to coordinated
delivery of individual
Images.
VCL/CIO/SF/mav/Apr-11/V2
20
Aggregate Environments – Sub-Clouds (VCL-Cloud)
Analytics Cloudlet
Parent and Children know about each other
Lin
Parent
Lin
Win
Lin
Win
This functions allows construction of
Custom sub-clouds: Controller + any
number of (hybrid) non-recursive
children.
WHAT DO WE DO WITH
THIS FUNCTIONALITY?
VCL/CIO/SF/mav/Apr-11/V2
21
Security as a Service
• Variety of authentication options (LDAP, Shibboleth… other)
• High security and isolation (IP-lock, local firewalls, point-topoint VLANs and VPNs, one-time passwords, feedback
confirmation, timeout, traffic monitoring …)
• Sophisticated resource access and mapping privilege tree.
• Real-time monitoring of reliability and security
Policy based Privilege Tree Maps
Auth
Timeout
Auth
One Time
Passwd
IPLock
VPN
Traffic
Monitoring
Activity
VLANs within VCL
VCL/CIO/SF/mav/Apr-11/V2
22
VCL Dashboard
(real time)
VCL/CIO/SF/mav/Apr-11/V2
23
Provenance and Meta-Data
Provenance and performance
statistics for any time period is
available to a general user, including
reliability information.
VCL/CIO/SF/mav/Apr-11/V2
24
Green & Cost-Effective
140
November 2008
100
80
60
40
High-Performance Computing
(over 12 months Mar08-Mar09)
20
800,000
0
0
1
2
3
4
5
6
7
8
700,000
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Time of Day (24 hr clock)
Average daily active reservations
Opportunity to save power or
Increase utilization
600,000
500,000
CPU Hours
Average Number of Reservations
120
400,000
300,000
200,000
100,000
HPC “filler” batch processing,
or other “fillers”.
VCL/CIO/SF/mav/Apr-11/V2
0
Mar08
Apr08
May08
Jun- Jul-08 Aug08
08
Sep08
Month
Oct08
Nov08
Dec08
Jan09
25
Feb09