CS780-3 Lecture Notes
In Courtesy of Mengjun Xie
About
NEESgrid will link earthquake researchers across the U.S. with leadingedge computing resources and research equipment, allowing
collaborative teams (including remote participants) to plan, perform,
and publish their experiments.
NEESgrid, the systems integration component of the NEES project,
uses the newest and fastest communications technologies to tie the
NEES network together.
Remote Users
(Faculty,
Students,
Practitioners)
Instrumented
Structures
and Sites
NEES Resources
Simulation
Tools
Repository
Laboratory
Equipment
Field Equipment
Curated Data
Repository
Leading Edge
Computation
Global
Connections
(FY 2005 – FY 2014)
Remote Users:
Laboratory Equipment
(K-12 Faculty and
Students)
NEES
Network for Earthquake Engineering Simulation

NEES is a distributed array of experimental sites, grid-based
data repositories, tool archives, and computational resources,
all seamlessly linked (hopefully!)
NEES has four components:




The
The
The
The
consortium, which will run NEES after 2004
consortium development (CD) builds the consortium
experimental sites, which provide data and content
systems integration (SI) effort, termed NEESgrid
Network drivers include telepresence, curated
repositories, scalable HPC, experimental-numerical
coupling, short- and long-term QoS issues.
NEES Network Stakeholders
Experimental Facilities

Shake tables, centrifuges, wave tanks, field sites
Resource providers

Computers, software, storage, networks
End users

Researchers, practicing engineers, students, …
Operational facilities

NCSA/NEESgrid  NEES Consortium in 2004
NEESgrid?
A coordinated and secure architecture/environment
A modular and extensible environment with a customizable user
interface
Provides common tools that allow leveraging resources and
experiences
Goal: the Cyber Infrastructure that will facilitate this next
generation of experimentation/simulation in earthquake
engineering
“A Distributed Virtual Laboratory for Advanced Earthquake
Experimentation and Simulation.”
-- B. F. Spencer, Jr. (PI & Project Director)
NEESgrid?
Through the NEESgrid, researchers can:




perform tele-observation and tele-operation of
experiments;
publish to and make use of a curated data
repository using standardized markup;
access computational resources and open-source
analytical tools;
access collaborative tools for experiment planning,
execution, analysis, and publication.
Telepresence
Telepresence means the capability to participate
remotely in experimental trials. There are two main
categories:


Tele-observation: the ability to observe the experiment and
capture trial data from a remote site
Tele-operation: the ability to interact with the experiment
equipment from a remote site.
Collaboration and Visualization
The tasks of Collaboration and Visualization project
include:




prototype a Grid-based collaborative environment;
integrate support for visualization tools into the collaborative
environment;
adapt the Comprehensive Collaborative Framework (CHEF)
for collaborative visualization applications and services; and
produce appropriate final documentation of the collaboration
and visualization components of NEESgrid.
CHEF is a flexible web-based environment for remote
collaboration.
Main Components









Tele-Control Services and APIs
Tele-Observation and Data Visualization
E-Notebook
Streaming data services
DAQ and related services
Data and Metadata Services
Remote Collaboration and Visualization tools and
services
Core Grid Services, deployment efforts, packaging
Simulation Component
Progress
The task of designing and creating this infrastructure
has been awarded to the NCSA at UIUC.
The components of the NEESgrid system will be
completed by September, 2004, when management
and operation of the NEES system will be turned over
to a consortium of earthquake engineer researchers
and practitioners.
Partner
National Center for
Supercomputing
Applications
Randy Butler —Deployment, Operations,
and Support
Mark Marikos —Management
Joe Futrelle —Data and Metadata
University of Illinois at Bill Spencer —Management
Urbana-Champaign
Dan Abrams —Community Building
Argonne National
Laboratory
Nestor J. Zaluzec —Telepresence
Ian Foster —System Configuration
Information Sciences
Institute
Carl Kesselman —System Configuration
University of
Southern California
Jean-Pierre Bardet —Integrated
Demonstrations
School of Information,
University of Michigan
Joseph Hardin —Collaboration
Tom Finholt —User Requirements
System Architecture
System Architecture
System Architecture
System Architecture
System Architecture
System Architecture
System Architecture
Pre-Experiment Collaboration
The remote user (RU) logs into
the equipment site's NEESpop.
After authentication, RU can
access all granted NEESgrid
resources through SSN.
RU can, via CHEF, read and
participate in discussions related
to the equipment site, see a
calendar, share documents, make
announcements, and correspond
with other researchers to plan an
experiment.
Through NEESpop, RU can also
access and edit electronic lab
notebooks hosted on the TPM.
Monitoring Network Status
RU contacts CHEF and requests a
summary of the system's health
and status.
The CHEF server contacts a central
NCSA server that monitors
NEESgrid.
Standard grid services (GIIS at
NCSA, GRIS for other services) are
continually collecting information
on system connectivity and the
availability of grid services such as
GridFTP.
The grid information is summarized
and sent to RU's browser.
Setting Up the Experiment
Using a CHEF teamlet, the
equipment specialist at the
equipment site enters the metadata
describing the configuration of the
experimental setup.
The metadata repository on the
NEESpop holds this configuration
information, making it available for
browsing by RUs.
The NEESpop metadata repository
also sends configuration
information to the DAQ system(s).
Subscribing to Trial Data Streams
RU contacts CHEF to browse
available NEESgrid resources and
discover what data will be available
for streaming.
A CHEF teamlet uses GIIS and
GRIS to query and report features
and availability of NEESgrid
resources, as well as details of
NEES equipment and available data
streams.
RU contacts the NEESpop and
requests a subscription to several
data streams.
The NEESpop negotiates with the
NSDS to broadcast the selected
data streams.
Streaming Near-Real-Time Data
Instrument information is sent in
real time from the DAQ system
through a NSDS daemon to the
NSDS.
Data is streamed by NSDS to the
RU. An applet in the RU's browser
displays the results.
Video services are managed
through the TPM server and
streamed to the RU via a separate
video streaming server.
Ending the Trial
The DAQ equipment moves the
trial data to a data repository
hosted on the NEESpop.
The data is associated with
appropriate metadata information,
such as experiment ID, channel ID,
and sensor type.
Analyzing the Trial Data
The trial data can now be browsed
via CHEF.
The trial data and metadata can be
downloaded to the RU via GridFTP
service.
The RU can then use either
standard software, such as MatLAB,
or their own customized software
to analyze the trial data.
Login Authentication
Chat
CHEF Environment
E-Notebook
CHEF Environment
Data Viewer
CHEF Environment
NTCP Data
CHEF Environment
NEESgrid Central Repository
CHEF Environment
Summary
Characterizations of NEES project:


A very specific application
A wide range of resource sharing:
 Raw data, experimental results, equipment, computation/software tools

Centralized controls, monitoring, and management.
Does this follow the three principles of Grid?

Not really: it is subject to central control, not open source software
based.
But it is a successful model of grid applications.
A Global and general grid is an ideal model, and may become
realistic after many successful NEES projects.