Experience with iRODS 4.0
Enabling Multisite Collaboration
Tuesday 9th September
iRODS User Group Meeting (UCL)
Presented by Richard Mansfield (Systems Engineer)
2
Deploying WOS resource on iRODS 4.0
• Download DDN WOS resource RPM
– http://irods.org/download/
– Or, build from source on GitHub
• Install WOS driver on iRODS server & iCAT server
– Failure to install on iCAT server will result in errors (!!!)
• Configure WOS resource on iRODS server
– “wos_host” refers to single WOS endpoint
– Be careful of length parameter length and use of semicolon
$ iadmin mkresc wosResc compound
$ iadmin mkresc cacheResc 'unix file system' eiserver1:/Vault
$ iadmin mkresc archiveResc wos eiserver1:/fakePath \
'wos_host=http://wos1.example.com;wos_policy=Replicate’
$ iadmin addchildtoresc wosResc cacheResc cache
$ iadmin addchildtoresc wosResc archiveResc archive
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
3
Node-local HTTP Proxy
• Example haproxy.conf on iRODS Server
global
log /dev/log local0
log /dev/log local1 notice
user haproxy
group haproxy
daemon
maxconn 40000
defaults
log global
mode http
option dontlognull
option tcplog
retries 5
option redispatch
contimeout 600s
clitimeout 10m
srvtimeout 10m
maxconn 19500
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
listen stats *:9091
mode http
balance
timeout client 30s
timeout connect 30s
timeout server 60s
stats uri /stats
stats realm HAProxy\ Statistics
stats auth admin:admin
stats admin if TRUE
frontend wos
bind *:8200
mode http
default_backend wos
backend wos
mode http
timeout check 60s
option httpchk GET /status/summary
http-check expect ! rstatus ^5
balance roundrobin
server 172.16.0.2 172.16.0.2:80 check
server 172.16.0.3 172.16.0.3:80 check
server 172.16.0.4 172.16.0.4:80 check
server 172.16.0.5 172.16.0.5:80 check
inter
inter
inter
inter
60s
60s
60s
60s
rise
rise
rise
rise
3
3
3
3
fall
fall
fall
fall
3
3
3 backup
3 backup
... Continued
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
4
Public Health England & iRODS
• Introduction
–
–
–
–
Executive agency for England’s Dept. of Health
Provides centralised infectious disease sequencing service
Classify DNA samples of potentially harmful bacteria
Patient samples submitted by hospitals & medical establishments
• Sequencing service
–
–
–
–
–
Six Illumina sequencers
Supported by 60 in-house researches
Bacteria sequence analysed on EXAScaler
Results published to WOS archive
Collaboration with clinical partners via iRODS
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
5
Proposed iRODS Architecture
Object referenced via
ObjectID & iRODS
server name
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
6
Remote Collaboration
Client access remote
iRODS server
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
7
Non-optimal Data Affinity
Client prevented from
accessing local data
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
8
Optimal Data Affinity for Collaboration
Object referenced via
ObjectID & local
hostname aliases
iRODS Server auth.
iCAT via common
SAN cert
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
9
SSL Certification for iRODS Servers
• Use SSL “Subject Alternative Names”
– Allows multiple hostnames/IPs embedded in single SSL certificate
– Commonly used for { www | ftp | mail }.example.com
• Multisite iRODS scenario
– Can’t assume common hostname naming scheme
– Certainly will use different IP addresses
• Use site specific common hostname aliases
– “irods-local” & “wos-local” (either in /etc/hosts or site DNS)
– Use host aliases in iRODS driver configuration
• Re-create WOS resource using aliases
$ iadmin mkresc cacheResc 'unix file system' irods-local:/Vault
$ iadmin mkresc archiveResc wos irods-local:/fakePath \
'wos_host=http://wos-local.example.com;wos_policy=Replicate’
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com
10
WOS Driver roadmap on GitHub
© 2014 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
ddn.com