Rule-Based Distributed Data
Management
iRODS 1.0 - Jan 23, 2008
http://irods.sdsc.edu
Reagan W. Moore
Mike Wan
Arcot Rajasekar
Wayne Schroeder
San Diego Supercomputer Center
{moore, mwan, sekar, schroede}@sdsc.edu
http://irods.sdsc.edu
http://www.sdsc.edu/srb/
Data Management Goals
• Support for data life cycle
• Shared collections -> data publication -> reference
collections
• Support for socialization of collections
• Process that governs life cycle transitions
• Consensus building for collection properties
• Generic infrastructure
• Common underlying distributed data management
technology
• iRODS - integrated Rule-Oriented Data System
NSF Software Development for
CyberInfrastructure Data
Improvement Project
• #0721400: Data Grids for Community Driven
Applications
• Three major components:
• Maintain highly successful Storage Resource Broker
(SRB) data grid technology for use by the NSF
research community
• Create an open source production version of the
integrated Rule-Oriented Data System (iRODS)
• Support migration of collections from the current SRB
data grid to the iRODS data grid
Why Data Grids (SRB)?
• Organize distributed data into shared collections
• Improve the ability for researchers to collaborate on
national and international scales
• Provide generic distributed data management mechanisms
•
•
•
•
•
•
•
•
•
Logical name spaces (files, users, storage systems)
Collection metadata
Replicas, versions, backups
Optimized data transport
Authentication and Authorization across domains
Support for community specific clients
Support for vendor specific storage protocols
Support for remote processing on data, aggregation in containers
Management of all phases of the data life cycle
Using a SRB Data Grid - Details
DB
SRB Server
Metadata Catalog
SRB Server
•User asks for data
•Data request goes to SRB Server
•Server looks up information in catalog
•Catalog tells which SRB server has data
•1st server asks 2nd for data
•The 2nd SRB server supplies the data
Extremely Successful
• Storage Resource Broker (SRB) manages 2 PBs of data
in internationally shared collections
• Data collections for NSF, NARA, NASA, DOE, DOD, NIH,
LC, NHPRC, IMLS: APAC, UK e-Science, IN2P3, WUNgrid
•
•
•
•
•
•
•
•
•
•
•
•
Astronomy
Bio-informatics
Earth Sciences
Ecology
Education
Engineering
Environmental science
High energy physics
Humanities
Medical community
Oceanography
Seismology
Data grid
Digital library
Data grid
Collection
Persistent archive
Digital library
Data grid
Data grid
Data Grid
Digital library
Real time sensor data, persistent archive
Digital library, real-time sensor data
• Goal has been generic infrastructure for distributed data
Date
Project
Data Grid
NSF / NVO
NSF / NPACI
Hayden
Pzone
NSF / LDAS-SALK
NSF / SLAC-JCSG
NSF / TeraGrid
NIH / BIRN
NCAR
LCA
Digital Library
NSF / LTER
NSF / Portal
NIH / AfCS
NSF / SIO Explorer
NSF / SCEC
LLNL
CHRON
Persistent Archive
NARA
NSF / NSDL
UCSD Libraries
NHPRC / PAT
RoadNet
UCTV
LOC
Earth Sci
TOTAL
5/17/02
6/30/04
11/29/07
GBs of
1000Хs of GBs of
1000Хs of Users with GBs of
1000Хs of Users with
data stored
files
data stored
files
ACLs data stored
files
ACLs
17,800
1,972
6,800
438
239
514
5,139
1,083
41
31
1
77
51,380
17,578
7,201
812
4,562
4,317
80,354
5,416
8,690
4,694
113
47
16
563
685
3,366
80
380
178
49
66
47
2,962
148
88,216
39,697
8,013
28,799
207,018
23,854
282,536
20,400
70,334
3,787
14,550
7,590
161
17,640
169
2,493
7,257
40,747
325
77
100
380
227
68
67
55
3,267
445
2
2
158
33
27
19
3
5
4
1
233
1,745
462
1,734
15,246
6
48
49
601
1,737
35
384
21
27
52
260
2,620
733
2,750
168,931
18,934
12,863
42
53
94
1,202
3,545
2,338
6,443
36
460
21
27
73
5
5
7
2
63
2,785
127
81
20,054
202
58
119
29
28 TB
6 mil
194 TB
40 mil
4,635
5,023
7,499
5,205
2,576
3,557
7,140
6,644
6,136
1,023 TB
6,430
84,984
1,328
966
1,569
2
192
652
200 mil
58
136
29
28
30
5
8
5
5,539
Generic Infrastructure
• Data grids manage data distributed
across multiple types of storage systems
• File systems, tape archives, object ring buffers
• Data grids manage collection attributes
• Provenance, descriptive, system metadata
• Data grids manage technology evolution
• At the point in time when new technology is
available, both the old and new systems can be
integrated
SRB Maintenance
• Release of SRB 3.5.0 was made on
December 3, 2007
• Eliminated a security vulnerability through use
of bind variables for interacting with databases
• Implemented additional community
requirements
• Provided bulk replication and backup
• Automated reconnection on network read-write
failures
• Incorporated bug fixes
Why iRODS?
• Need to verify assertions about the purpose
of a collection
• Socialization of data collections, map from creator
assertions to community expectations
• Need to manage exponential growth in
collection size
• Improve support for all phases of data life cycle from
shared data within a project, to published data in a
digital library, to reference collections within an archive
• Data life cycle is a way to prune collections, and
identify what is valuable
• Need to minimize labor by automating
enforcement of management policies
Starting Requirements
• Base capabilities upon features required by
scientific research communities
• Started with features in SRB data grid, but needed to
understand impact of management policies and
procedures
• Incorporate trustworthiness assessment
criteria from the preservation community
• Other criteria include human subject approval, patient
confidentiality, time-dependent access controls
• Promote international support for iRODS
development to ensure international use
Data Management Applications
• Data grids
• Share data - organize distributed data as a collection
• Digital libraries
• Publish data - support browsing and discovery
• Persistent archives
• Preserve data - manage technology evolution
• Real-time sensor systems
• Federate sensor data - integrate across sensor streams
• Workflow systems
• Analyze data - integrate client- & server-side workflows
Approach
• To meet the diverse requirements, the
architecture must:
•
•
•
•
•
•
•
Be highly modular
Be highly extensible
Provide infrastructure independence
Enforce management policies
Provide scalability mechanisms
Manipulate structured information
Enable community standards
Observations of Production
Data Grids
• Each community implements different
management polices
• Community specific preservation objectives
• Community specific assertions about
properties of the shared collection
• Community specific management policies
• Need a mechanism to support the
socialization of shared collections
• Map from assertions made by collection
creators to expectations of the users
Tension between Common and
Unique Components
• Synergism - common infrastructure
• Distributed data
• Sources, users, performance, reliability, analysis
• Technology management
• Incorporate new technology
• Unique components - extensibility
• Information management
• Semantics, formats, services
• Management policies
• Integrity, authenticity, availability, authorization
Data Grid Evolution
• Implement essential components needed for
synergism
• Storage Resource Broker - SRB
• Infrastructure independence
• Data and trust virtualization
• Implement components needed for specific
management policies and processes
•
•
•
•
integrated Rule Oriented Data System - iRODS
Policy management virtualization
Map processes to standard micro-services
Structured information management and transmission
Initial iRODS Design
Next-generation data grid technology
• Open source software - BSD license
• Unique capability - Virtualization of
management policies
• Map management policies to rules
• Enforce rules at each remote storage location
• Highly extensible modular design
• Management procedures are mapped to microservices that encapsulate operations performed at the
remote storage location
• Can add rules, micro-services, and state information
• Layered architecture
• Separation of client protocols from storage protocols
Using an iRODS Data Grid - Details
DB
iRODS Server
Rule Engine
Metadata Catalog
Rule Base
iRODS Server
Rule Engine
•User asks for data
•Data request goes to iRODS Server
•Server looks up information in catalog
•Catalog tells which iRODS server has data
•1st server asks 2nd for data
•The 2nd iRODS server applies rules
Data Virtualization
Access Interface
Storage Protocol
Storage System
Traditional
approach:
Client talks
directly to
storage system
using Unix I/O:
Microsoft Word
Data Virtualization (Digital Library)
Access Interface
Digital Library
Storage Protocol
Storage System
Client talks to
the Digital
Library which
then interacts
with the
storage system
using Unix I/O
Data Virtualization (iRODS)
•Map from the
Access Interface
Standard Micro-services
Data Grid
Standard Operations
Storage Protocol
Storage System
actions requested by
the access method to
a standard set of
micro-services.
•The standard microservices use standard
operations.
•Separate protocol
drivers are written
for each storage
system.
iRODS Release 1.0
• Open source software available at wiki:
• http://irods.sdsc.edu
• Since January 23, 2008, more than 590
downloads by projects in 18 countries:
• Australia, Austria, Belgium, Brazil, China,
France, Germany, Hungary, India, Italy,
Norway, Poland, Portugal, Russia, Spain,
Taiwan, UK, and the US
Core Components
• Framework
• Infrastructure that ties together the layered environment
• Drivers
• Infrastructure that interacts with commercial protocols (database,
storage, information resource)
• Clients
• Community specific access protocols
• Rules
• Management policies specific to a community
• Micro-services
• Management procedures specific to a community
• Quality assurance
• Testing routines for code validation
• Maintenance
• Bug fixes, help desk, chat, bugzilla, wiki
Rule Specification
• Rule - Event : Condition : Action set :
Recovery Procedure
•
•
•
•
Event - atomic, deferred, periodic
Condition - test on any state information attribute
Action set - chained micro-services and rules
Recovery procedure - ensure transaction
semantics in a distributed world
• Rule types
• System level, administrative level, user level
Distributed Management System
Data
Transport
Rule
Engine
Metadata
Catalog
Persistent
State
information
Policy
Management
Virtualization
Server
Side
Workflow
Execution
Engine
Execution
Control
Scheduling
Messaging
System
integrated Rule-Oriented Data System
Client Interface
Admin Interface
Rule Invoker
Rule
Modifier
Module
Current
State
Rule
Rule
Base
Config
Modifier
Module
Consistency
Check
Module
Metadata
Modifier
Module
Service
Manager
Consistency
Check
Module
Consistency
Check
Module
Confs
Resources
Resource-based
Services
Micro
Service
Modules
Metadata-based
Services
Micro
Service
Modules
Metadata
Persistent
Repository
iRODS Data Grid Capabilities
• Remote procedures
• Atomic / deferred / periodic
• Procedure execution / chaining
• Structured information
• Structured information
•
•
•
•
•
Metadata catalog interactions / 205 queries
Information transmission
Template parsing
Memory structures
Report generation / audit trail parsing
iRODS Data Grid Capabilities
• Rules
•
•
•
•
User / administrative / internal
Remote web service invocation
Rule & micro-service creation
Standards / XAM, SNIA
• Installation
• CVS / modules
• System dependencies
• Automation
iRODS Data Grid
• Administration
•
•
•
•
User creation
Resource creation
Token management
Listing
• Collaborations
• Development plans
• International collaborators
• Federations
Three Major Innovations
1. Management virtualization
• Expression of management policies as rules
• Expression of management procedures as
remote micro-services
• Expression of assertions as queries on
persistent state information
•
Required addition of three more logical
name spaces for rules, micro-services, and
state information
Second Major Innovation
• Recognition of the need to support structured
information
• Manage exchange of structured information between
micro-services
• Argument passing
• Memory white board
• Manage transmission of structured information
between servers and clients
• C-based protocol for efficiency
• XML-based protocol to simplify client porting (Java)
• High performance message system
Third Major Innovation
• Development of the Mounted Collection
interface
• Standard set of operations (20) for extracting
information from a remote information resource
• Allows data grid to interact with autonomous resources
which manage information independently of iRODS
• Structured information drivers implement the
information exchange protocol used by a particular
information repository
• Examples
• Mounted Unix directory
• Tar file
SDCI Project at SDSC
• Implement using spiral development. Iterate
across development phases:
• Requirements - driven by application communities
• Prototypes - community specific implementation of a
new feature / capability
• Design - creation of generic mechanism that is
suitable for all communities
• Implementation - robust, reliable, high performing code
• Maintenance - documentation, quality assurance, bug
fixes, help desk, testing environment
• We find communities eager to participate in
all phases of spiral development
Example External Projects
• International Virtual Observatory Alliance
(federation of astronomy researchers)
• Observatoire de Strasbourg ported the IVOA VOSpace
interface on top of iRODS
• This means the astronomy community can use their
web service access interface to retrieve data from
iRODS data grids.
• Parrot grid interface
• Ported by Douglas Thain (University of Notre Dame)
on top of iRODS. Builds user level file system across
GridFTP, iRODS, http, and other protocols
Collaborators - Partial List
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Aerospace Corporation
Academia Sinica, Taiwan
UK security project, King's College London
BaBar High Energy Physics
Biomedical Informatics Research Network
California State Archive
Chinese Academy of Sciences
CASPAR - Cultural, Artistic, and Scientific knowledge for Preservation Access and Retrieval
CineGrid - Media cyberinfrastructure
DARIAH - Infrastructure for arts and humanities in Europe
D-Grid - TextGrid project, Germany
DSpace Foundation digital library
Fedora Commons digital library
Institut national de physique nucleaire et de physique des particules
IVOA - International Virtual Observatory Alliance
James Cook University
KEK - High Energy Accelerator Research Organization, Japan
LOCKSS - Lots of Copies Keep Stuff Safe
Lstore - REDDnet Research and Education Data Depot network
NASA Planetary Data System
National Optical Astronomy Observatory
Ocean Observatory Initiative
SHAMAN - Sustaining Heritage through Multivalent Archiving
SNIA - Storage networking Industry Association
Temporal Dynamics of Learning Center
Project Coordination
• Define international collaborators
• Technology developers for a specific development
phase for a specific component.
• Collaborators span:
•
•
•
•
•
Scientific disciplines
Communities of practice (digital library, archive, grid)
Technology developers
Resource providers
Institutions and user communities
• Federations within each community are
essential for managing scientific data life
cycle
Scientific Data Life Cycle
• Shared collection
• Used by a project to promote collaboration between
distributed researchers
• Project members agree on semantics, data formats,
and manipulation services
• Data publication
• Requires defining context for the data
• Provenance, conformance to community format
standards
• Reference collections
• Community standard against which future research
results are compared
Scientific Data Life Cycle
• Each phase of the life cycle requires
consensus by a broader community
• Need mechanisms for expressing the
new purpose for the data collection
• Need mechanisms that verify
•
•
•
•
Authoritative source
Completeness
Integrity
Authenticity
Why iRODS?
• Collections are assembled for a purpose
•
•
•
•
Map purpose to assessment criteria
Use management policies to meet assertions
Use management procedures to enforce policies
Track persistent state information generated by
every procedure
• Validate criteria by queries on state information
and on audit trails
Data Management
iRODS - integrated Rule-Oriented Data System
Data Management
Environment
Management
Functions
Data Management
Infrastructure
Physical
Infrastructure
Conserved
Control
Remote
Properties
Mechanisms
Operations
Assessment Management Capabilities
Criteria
Policies
Data grid Π Management virtualization
Persistent
Rules
Micro-services
State
Data grid Π Data and trust virtualization
Database
Rule Engine
Storage
System
Why iRODS?
•
Can create a theory of data management
•
•
Prove compliance of data management system with specified
assertions
Three components
1. Define the purpose for the collection, expressed as assessment
criteria, management policies, and management procedures
2. Analyze completeness of the system
•
•
•
•
For each criteria, persistent state is generated that can be audited
Persistent state attributes are generated by specific procedure
versions
For each procedure version there are specific management policy
versions
For each criteria, there are governing policies
3. Audit properties of the system
• Periodic rules validate assessment criteria
Major iRODS Research Question
• Do we federate data grids as was done
in the SRB, by explicitly crossregistering information?
• Or do we take advantage of the
Mounted Collection interface and
access each data grid as an
autonomous information resource?
• Or do we use a rule-based database
access interface for interactions
between iCAT catalogs?
Federation Between IRODS Data Grids
Data Access Methods (Web Browser, DSpace, OAI-PMH)
Data Collection A
Data Grid
Data Collection B
Data Grid
• Logical resource name space
• Logical resource name space
• Logical user name space
• Logical user name space
• Logical file name space
• Logical file name space
• Logical rule name space
• Logical rule name space
• Logical micro-service name
• Logical micro-service name
• Logical persistent state
• Logical persistent state
Mounted Collections
• Minimizes dependencies between the
autonomous systems
• Supports retrieval from the remote information
resource, but not pushing of information
• Pull environment
• Can be controlled by rules that automate
interactions
• Chained data grids
• Central archive (archive pulls from other data grids)
• Master-slave data grids (slaves pull from master)
Rule-based Database Access
Interface
• Support interactions by querying the
remote iCAT catalog’s database
• Expect to support publication of schemata
• Ontology-based reasoning on semantics
• Can be used for both deposition and retrieval
of information
• Simplifies exchange of rules and possibly of
micro-services
iRODS Development
• NSF - SDCI grant “Adaptive Middleware for
Community Shared Collections”
• iRODS development, SRB maintenance
• NARA - Transcontinental Persistent Archive
Prototype
• Trusted repository assessment criteria
• NSF - Ocean Research Interactive
Observatory Network (ORION)
• Real-time sensor data stream management
• NSF - Temporal Dynamics of Learning Center
data grid
• Management of IRB approval
iRODS Development Status
• Production release is version 1.0
• January 24, 2008
• International collaborations
• SHAMAN - University of Liverpool
• Sustaining Heritage Access through Multivalent
ArchiviNg
•
•
•
•
UK e-Science data grid
IN2P3 in Lyon, France
DSpace policy management
Shibboleth - collaboration with ASPIS
Planned Development
•
•
•
•
•
•
•
•
•
•
•
•
•
•
GSI support (1)
Time-limited sessions via a one-way hash authentication
Python Client library
GUI Browser (AJAX in development)
Driver for HPSS (in development)
Driver for SAM-QFS
Porting to additional versions of Unix/Linux
Porting to Windows
Support for MySQL as the metadata catalog
API support packages based on existing mounted collection driver
MCAT to ICAT migration tools (2)
Extensible Metadata including Databases Access Interface (6)
Zones/Federation (4)
Auditing - mechanisms to record and track iRODS metadata changes
For More Information
Reagan W. Moore
San Diego Supercomputer Center
moore@sdsc.edu
http://www.sdsc.edu/srb/
http://irods.sdsc.edu/
Collaborations
Requirements
Capability
Prototypes
Design
Generic Mainversion tenance
Technology
Lead
Framework
Core server
TPAP, OOI, TDLC, BIRN, Й
SDCI
SDCI
SDCI
SDCI
Wan (SDSC)
Rule Engine
TPAP, BIRN, TDLC
SDCI
SDCI
SDCI
SDCI
Rajasekar (SDSC)
Rule Base
TPAP
SDCI
SDCI
SDCI
SDCI
Rajasekar (SDSC)
iCAT catalog
TPAP, TDLC, BIRN, OOI
TPAP, SDCI
SDCI
SDCI
SDCI
Schroeder (SDSC)
Extensible iCAT
TPAP
TPAP, SDCI
SDCI
SDCI
SDCI
Schroeder (SDSC)
SDCI
SDCI
SDCI
SDCI
Wan (SDSC)
TPAP, OOI
SDCI
SDCI
SDCI
Teragrid, TPAP, OOI
ExecServer
(Scheduler
Xmessaging Server OOI, TPAP, LSST, Kepler
Transport Layer
DLI2, OOI, Cinegrid, LSST, TPAP, PDS
SDCI
SDCI
SDCI
SDCI
Wan, Rajasekar
(SDSC)
Wan (SDSC)
Transport Layer
(XML)
Monitoring System
TPAP, DLI2, OOI
TPAP, SDCI
SDCI
SDCI
SDCI
Wan (SDSC)
IN2P3, BIRN
IN2P3
IN2P3
IN2P3
IN2P3
Nief(IN2P3)
TPAP, SDCI
SDCI
SDCI
SDCI
Wan (SDSC)
Wan, Rajasekar,
Schroeder (SDSC)
Schroeder, Wan
(SDSC)
Schroeder (SDSC)
Mounted Collection TPAP, OOI, Teragrid, NSDL
Federation
TPAP, OOI, TDLC, NSDL, BIRN
TPAP, SDCI
SDCI
SDCI
SDCI
Master-slave
catalogs
Authentication
BIRN
BIRN, SDCI
SDCI
SDCI
SDCI
BIRN, TPAP, Teragrid, OSG, NeSC
BIRN, SDCI
SDCI
SDCI
SDCI
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Main-tenance Technology Lead
Drivers
iCAT database
drivers
Database access
drivers
Storage drivers
TPAP, SPAWAR, NSDL
SDCI, TPAP
SDCI
SDCI
SDCI
Schroeder (SDSC)
TDLC, TPAP, ROADnet, OOI
ROADnet, TPAP
SDCI
SDCI
SDCI
Schroeder (SDSC)
TPAP, Teragrid, LSST, SNIA, ZIH
SDCI
SDCI
SDCI
SDCI
Structured
Teragrid, TPAP
Information Drivers
Teragrid, TPAP,
SDCI, NeSC
SDCI
SDCI
SDCI
SRB-iRODS
migration
ZIH, JCU, BIRN, TPAP
TPAP, SDCI
SDCI
SDCI
SDCI
Wan (SDSC),
Pasquinelli (Sun)
Wan (SDSC), Hasan
(SHAMAN), Bady
(RMBC)
Schroeder, Rajasekar,
Wan (SDSC)
GSI authentication
SPAWAR, OSG, TeraGrid, NeSC, AS, BMBF
SPAWAR
SDCI
SDCI
SDCI
Schroeder (SDSC)
Shibboleth
authentication
ASPIS, SHAMAN
NeSC
ASPIS
ASPIS
ASPIS
Hasan (SHAMAN),
Beitz(MU)
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Maintenance
Technology
Lead
Clients
Jargon - Java I/O
library
Web client
BIRN, TPAP, OOI, Teragrid, TDLC
BIRN, SDCI, TPAP
BIRN, SDCI
BIRN, SDCI
BIRN, SDCI
Gilbert (SDSC)
BIRN, Teragrid, OSG, TPAP, LUScid
SDCI
SDCI
SDCI
Lu, Antoine (SDSC)
inQ - Window s
brow ser
iCommands - shell
commands
Parrot
TPAP, DLI2, OOI, Teragrid, NSDL
SDCI, TPAP,
LUScid
SDCI, TPAP, NSDL
SDCI
SDCI
SDCI
Teragrid, OOI, TPAP
SDCI
SDCI
SDCI
SDCI
PetaShare, Condor
ND
ND
ND
ND
Cow art, Zhu
(SDSC)
Schroeder, Wan,
Rajasekar (SDSC)
Thain (ND)
PHP client API
TPAP, TDLC
TPAP
SDCI
SDCI
SDCI
PERL load library
TPAP, JCU
JCU
JCU
JCU
JCU
Python load library
TPAP, NCRIS
TPAP
SDCI
SDCI
SDCI
Fedora
NSDL, KCL FNL, RMBC, D-Grid, DARIAH
NSDL, KCL
NSDL
NSDL
NSDL
DSpace
MIT, TPAP, AS
MIT
TPAP
MIT
MIT
LOCKSS
LOCKSS
LOCKSS
LOCKSS
LOCKSS
LOCKSS
Lstore
REDDnet
REDDnet
REDDnet
REDDnet
REDDnet
Rosenthal
(Stanf
ord)
Sheldon
(ACCRE)
Archivist Toolkit
UCSD Libraries
UCSD Libraries
UCSD Libraries
UCSD Libraries
UCSD Libraries
Westbrook (UCSD)
Internet Archive
NDIIPP, NSDL, TPAP, CDL
NDIIPP
CDL
CDL
CDL
Zhu (SDSC)
SRM
AS, OGF, EGEE, DEISA
AS
AS
AS
AS
(AS)
GridFTP
Archer, DEISA
Archer
Archer
Archer
Archer
Sim (JCU)
VOSpace
IVOA
NVO
NVO
NVO
NVO
Sensor Stream
Interface
OOI
OOI
OOI
OOI
OOI
Video Streaming
Interface
Cinegrid
Cinegrid
Cine-grid
Cinegrid
Cinegrid
Wagner (UCSD), Lu
(SDSC)
Rajasekar, Lu
(SDSC), Arrott
(UCSD)
Wan (SDSC)
Lu (SDSC),
Atkinson (JCU)
Atkinson (JCU)
DeTordy (SDSC),
Atkinson (JCU)
Zhu (SDSC),
Hodges (KCL),
Aschenbrenner
(UG)
Smith (MIT)
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Maintenance
Technology
Lead
Rules
Administrative
Teragrid, BIRN, TPAP, NOAO, BaBar, NeSC Teragrid, TPAP
SDCI
SDCI
SDCI
Trustw orth-iness
TPAP, SHAMAN
TPAP, SHAMAN
TPAP
TPAP
TPAP
IRB
TDLC
TDLC
TDLC
TDLC
TDLC
Rajasekar,
Schroeder, Chen
(SDSC)
DeTorcy, Marciano
(SDSC), Giaretta
(CASPAR),
Bady(RMBC),
Thaller (Planets)
Hou(SDSC)
HIPAA
BIRN
BIRN
BIRN
BIRN
BIRN
Rajasekar (SDSC)
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Maintenance
Technology
Lead
Micro-services
Administrative
Teragrid, BIRN, TPAP, NOAO, BaBar, NeSC Teragrid, TPAP
SDCI
SDCI
SDCI
Audit trails
TPAP, RMBC
TPAP
SDCI
SDCI
SDCI
ERA
TPAP, MU
TPAP
TPAP, SDCI
TPAP, SDCI
TPAP, SDCI
Trustw orth-iness
TPAP, SHAMAN
TPAP, SHAMAN
TPAP
TPAP
TPAP
IRB
TDLC
TDLC
TDLC
TDLC
TDLC
HIPAA
BIRN
BIRN
BIRN
BIRN
BIRN
Rajasekar,
Schroeder (SDSC)
DeTorcy, Schroeder
(SDSC), Bady
(UMBC)
Moore, DeTorcy
(SDSC), Treloar
(MU)
DeTorcy, Marciano,
Kremenek, Moore
(SDSC)
Hou, DeTorcy,
Moore (SDSC)
Rajasekar (SDSC)
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Maintenance
Technology
Lead
Quality Assurance
Unit testing
TPAP, BIRN, Teragrid
SDCI
SDCI
SDCI
SDCI
Function testing
TPAP, OOI
SDCI
SDCI
SDCI
SDCI
Load testing
TPAP, OOI, KEK, IN2P3
SDCI, NARA
SDCI
SDCI
SDCI
Performance testing TPAP, IN2P3
SDCI, NARA
SDCI
SDCI
SDCI
Security testing
SDCI
SDCI
SDCI
SDCI
Rajasekar,
Schroeder, Wan,
DeTorcy (SDSC)
Lindsey, Tooby
(SDSC), Nief
(IN2P3)
Lindsey (SDSC),
Smorul (U Md)
Lindsey (SDSC),
Smorul (U Md), Iida
(KEK)
Lindsey (SDSC)
Recovery procedureTPAP, OOI, SHAMAN
testing
System failure
TPAP, OOI, IN2P3
testing
Recoverability
TPAP, OOI, NARA, SHAMAN, IN2P3
testing
Integration testing TPAP, OOI, SHAMAN
SDCI
SDCI
SDCI
SDCI
Lindsey (SDSC)
SDCI
SDCI
SDCI
SDCI
Lindsey (SDSC)
SDCI
SDCI
SDCI
SDCI
SDCI
SDCI
SDCI
SDCI
Parallel execution
testing
Tuning
OOI
SDCI
SDCI
SDCI
SDCI
TPAP, OOI, BIRN, IN2P3, SHAMAN
SDCI
SDCI
SDCI
SDCI
Lindsey (SDSC),
Smorul (U Md)
Zhu, Lindsey
(SDSC)
Wan, Lindsey
(SDSC)
Rajasekar, Wan,
Schroeder (SDSC)
TPAP, TDLC, BIRN
Collaborations
Capability
Requirements
Prototypes
Design
Generic
version
Maintenance
Technology
Lead
Documentation
Micro-service
creation
Micro-services
SHAMAN, IN2P3, TPAP
SHAMAN, IN2P3,
TPAP
SHAMAN, IN2P3,
TPAP
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
SHAMAN, IN2P3,
TPAP
SHAMAN, IN2P3,
TPAP
SHAMAN, IN2P3,
TPAP
SHAMAN, IN2P3,
TPAP
SAA demonstration TPAP
TPAP
TPAP, SDCI
TPAP, SDCI
TPAP, SDCI
Tutorials
AS, NeSC, Teragrid, TPAP
TPAP, SDCI
TPAP, SDCI
TPAP, SDCI
TPAP, SDCI
Concepts
TPAP, OGF
TPAP, SDCI
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
Rajasekar, Tooby
(SDSC)
Rajasekar, Tooby,
Schroeder (SDSC),
Hasan (SHAMAN),
Nief (IN2P3)
Rajasekar, Tooby,
Schroeder (SDSC),
Hasan (SHAMAN),
Nief (IN2P3)
Marciano, DeTorcy,
Moore (SDSC)
Moore, Schroeder,
Rajasekar,
Wan(SDSC), Berry
(NeSC), Hasan
(SHAMAN), Li
(CAS)
Moore (SDSC)
Clients
SHAMAN, IN2P3, TPAP, AS
Community
Sustainability
Standards
DSpace, Fedora, IA
SDCI
SDCI
SDCI
SDCI
Tooby (SDSC)
OGF, SNIA, Sun, SHAMAN
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
SDCI, TPAP
Moore (SDSC), Lee
(AC)
SHAMAN, IN2P3, TPAP, AS