The Storage Resource Broker and the NGS -support.ac.uk

advertisement

http://www.grid-support.ac.uk

http://www.ngs.ac.uk

The Storage Resource Broker and the NGS

Slides from Peter Berrisford, RAL

For a full SRB tutorial, see: http://www.niees.ac.uk/events/srb http://www.nesc.ac.uk/ http://www.pparc.ac.uk/ http://www.eu-egee.org/

• http://www.sdsc.edu/srb/

2

What is SRB?

• Storage Resource Broker (SRB) is a software product developed by the San Diego Supercomputing Centre

(SDSC).

• Allows users to access files and database objects across a distributed environment.

• Actual physical location and way the data is stored is abstracted from the user

• Allows the user to add user defined metadata describing the scientific content of the information

Peter Berrisford

RAL

3

How SRB Works

MCAT

Database c d

MCAT

Server

SRB A

Server b e f a g

SRB

Client

SRB B

Server

• 4 major components:

– The Metadata Catalogue

(MCAT)

– The MCAT-Enabled

SRB Server

– The SRB Storage Server

– The SRB Client

Peter Berrisford

RAL

4

SRB Client Tools

• Provide a user interface to send requests to the SRB server.

• 4 main interfaces:

– Command line (S-Commands)

– MS Windows (InQ)

– Web based (MySRB).

– Java (JARGON)

• Web Services (MATRIX)

Peter Berrisford

RAL

5

Database Servers @ Manchester

DB1

DB n

Online Replication

Planned Deployment Structure

Disk

Farm

MCAT SRB

Server

MCAT Server @ Manchester

Failover link

MCAT Server @ RAL

MCAT SRB

Server

User

DB1

DB n

Database Servers @ RAL

SRB server @ Leeds

Resource Driver

SRB server @ RAL

Resource Driver

SRB server @ Oxford

Resource Driver

SRB server @ HPCX

Resource Driver

Disk Farm

Disk

Farm

Disk

Farm

Disk

Farm

Peter Berrisford

RAL

6

Summary

• SRB provides NGS users with

– a virtual filesystem

– Accessible from all core nodes and from the “UI” / desktop

– (will provide) redundancy – mirrored catalogue server

– Replica files

– Support for application metadata associated with files

Peter Berrisford

RAL

7

SRB practical

8

Background slides

9

Concepts

• Location: A physical node running an SRB Server

• Physical Resource: A storage area managed by an SRB

Server

• Logical Resource: One or more Physical Resources – can be distributed

• Collection – Data abstraction of resources

Peter Berrisford

RAL

10

The MCAT Database

• The MCAT database is a metadata repository that provides a mechanism for storing information used by the SRB system.

• Includes both

– Internal system data required for running the system

– Application (user) metadata regarding data sets being brokered by SRB.

11

The MCAT Server

• At least one SRB Server must be installed on the node that can access the MCAT database. This is known as the MCAT-Enabled

SRB Server.

• MCAT SRB Server works directly against the MCAT database to provide SRB Services

• All other SRB Servers interact through the MCAT Server

12

The SRB Server

• The SRB Server is a middleware application that accepts requests from clients and obtains/queries/manages the necessary data sets.

• It queries the MCAT SRB Server to gather information on datasets and supplies this back to the

SRB client.

13

Download