Detailed Design Document
Hippocampus (Donna Rose Addis)
Version 0.4 Draft
Document Version
Date
8/11/12
6/5/12
12/9/13
Version
0.1
0.2
0.3
13/9/13 0.4
Change
Initial Draft
Hippocampus Draft
Minus technical information still waiting on ITS
to provide concrete solution
Technical setup information added (Still waiting
on ITS to provide details on final solution)
Name / Role
James Gander
Dominic Ritchie
Dominic Ritchie
Dominic Ritchie
About This Document
Department
Date of Issue
Review Date
Purpose
Science IT
Owner
Effective Date
Status
DRAFT
This document describes the solution design of new solutions for the Faculty of Science
Audience
Science IT, Faculty of Science and ITS
Distribution
Science IT and project teams
Updates
Contact the xxxxxxxwith any document updates, corrections or requests
Contents
Background ............................................................................................................................................. 4
Purpose................................................................................................................................................ 4
Scope ................................................................................................................................................... 4
In Scope ............................................................................................................................................... 4
Out of Scope ........................................................................................................................................ 4
Design Overview ..................................................................................................................................... 4
Architecture Objectives and Goals ...................................................................................................... 4
Detailed Diagram ................................................................................................................................. 5
Architecture Requirements ................................................................................................................. 7
Design Assumptions ............................................................................................................................ 7
Security Requirements ........................................................................................................................ 7
Detail Design Solution ............................................................................................................................. 8
Hardware/Software Details ............................................................................................................... 10
Network Details ................................................................................................................................. 10
Configuration Details ............................................................................ Error! Bookmark not defined.
Interface Considerations ...................................................................... Error! Bookmark not defined.
Specific Settings .................................................................................... Error! Bookmark not defined.
Security Details .................................................................................................................................. 10
Requirements Traceability Table .......................................................................................................... 11
Traceability ........................................................................................... Error! Bookmark not defined.
Approval ................................................................................................... Error! Bookmark not defined.
Background
Purpose
The purpose of this document is to provide an accurate and complete description of the service
provided for the “Hippocampus” group headed by Donna Rose Addis, for the Faculty of Science. Its
main purpose is to inform owners, users and supporters of the service on how the service provides
the functions required by the users.
It further expands upon the High Level Design document by providing more finely grained
information regarding the setup and design of the service.
Scope
This document will provide current architecture and setup information regarding the service. It
describes the structure of the design and how it integrates with other services, architectures and
technologies. After reading this document, the reader should be able to build the service from
scratch.
In Scope
In scope are current designs including physical server build, operating system installation and setup,
along with authentication mechanisms, and backup. Possible future designs are included, along with
external services used and the mechanisms required to access those services. Differences between
current and future designs will be expanded upon, along with technical details on each.
Out of Scope
Out of scope components including low level networking information, design and architecture of
services provided by groups external to Science IT.
Design Overview
Architecture Objectives and Goals
The final architecture aims to provide a solution to the following objectives and goals:





High availability
Reliability
Support
Flexibility
Performance
The previous design relied on some care being taken by the end user in setting up access for others.
It consisted of a single workstation with a network attached storage device. Support from end to
end was handled by Science IT. Issues regarding this service included all of the goals listed above,
key issues were:
Service provided by a single workstation
 No tolerance for power failure
 No tolerance for network failure
 Little tolerance for hardware failure
Backup solution
 Backup provided by NAS in same building
 NAS size smaller than full backup
The example of hardware failure became evident at the beginning of 2013, as the machine the
service was currently running on suffered an unrecoverable hardware failure. Fortunately we had
identified an issue with the backup previous to the failure occurring, and moved the backup point to
a file server.
Return to service time was measured in months. This was due to the architecture and planning used
in the original design. The original design called for a full software reinstall of the operating system,
authentication and application stacks whenever there was a major fault.
Because of the way that the service is consumed, flexibility is important. The growth of storage
required is one terabyte per year. Unfortunately in the original design, backup space was not
aligned with total storage space, and so the size of the backup became unmanageable.
Detailed Diagram
Previous Design
Single Dell workstation with two one terabyte drives, backed to a NAS. Users connect to the service
using VNC over SSH.
Current Design
Future Design
Conversations with ITS have brought about possible changes to these implementations. This
document will be updated once confirmation is given.
Architecture Requirements
In this instance, the application server is running Matlab, and must be provided in a visual fashion,
enabling users to interact with the system via a GUI. The software that will be used to provide this
visual context is FreeNX. FreeNX is the open source version of the paid product NoMachine.
Design Assumptions
ITS is to provide the virtualization environment and manage the Operating System (RedHat). This
includes all associated updates, and support. The Matlab layer will be managed by Science IT, with
consultation with ITS. Any updates for Matlab will be done under change control as required by
standard ITS process.
Security Requirements
Users will be required to login to the system using either their EC or UOA credentials. The login
mechanism will be LDAP.
Detail Design Solution
In this section I will detail how to install Matlab and required components after sending a standard
request for a VM to ITS.
To being installation, first request a Redhat Virtual Machine (using ITS standard templates) from ITS
using the standard processes within the AskIT system.






4 Cores (Tier 3)
16GB RAM
6TB Disc (At this time it is unknown how the disc will be provided. The diagrams shown
above give a possible solution, however we are still waiting for ITS to come back with an
answer)
Users authenticate to login (EC or UOA)
Only users that are part of the hippocampus-logon.psy group can logon (apart from admins)
and the nx user (to be created later by installing freenx)
Create a user (such as scienceit) with full sudoers access, or give your user full sudoers
access
Once the machine has been provided, and those modifications made by ITS, start on the procedure
below.
Software and Configuration
1. As the ITS provided machine is Redhat Server, you will need to install the desktop
environment before continuing. Unfortunately there are no grouplists on the ITS
repositories so the easiest way of achieving this is the following:
sudo yum install gnome-session xauth gnome-panel nautilus xorg
2. Install freenx-server following the documentation provided here
http://kb.sit.auckland.ac.nz/?article=installing-freenx-server-on-redhat
3. Install Matlab
http://kb.sit.auckland.ac.nz/?article=installing-matlab
4. Install FSL
http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FslInstallation
5. Install ANTS
http://sourceforge.net/projects/advants/files/ANTS/
Extract the tar.gz to /usr/local/ANTS
6. Install AFNI
Download the xorg7 (64bit) package and follow the instructions here
http://afni.nimh.nih.gov/pub/dist/HOWTO/howto/ht00_inst/html/linux_inst_basic.html
7. Create a file called “environment.sh” within /etc/profile.d, copy the following
#!/bin/bash
# MATLAB Setup
PATH=$PATH:/usr/local/MATLAB/R2012b/bin/
# FSL Setup
FSLDIR=/usr/local/fsl
PATH=${FSLDIR}/bin:${PATH}
export FSLDIR PATH
. ${FSLDIR}/etc/fslconf/fsl.sh
# ANTS Setup
PATH=$PATH:/usr/local/ANTS/bin
# AFNI Setup
PATH=$PATH:/usr/local/afni
8. Download and copy the toolboxes listed below to /usr/local/scripts
a. SPM v8 (http://www.fil.ion.ucl.ac.uk/spm/)
b. ArtRepair v4 (http://cibsr.stanford.edu/tools/human-brain-project/artrepairsoftware.html)
c. CONN v.13 (http://www.nitrc.org/projects/conn/)
d. PLSgui/PLScmd v6.130310 (http://www.rotmanbaycrest.on.ca/index.php?section=84)
e. AFNI v2011_12_21_1014 (http://afni.nimh.nih.gov/afni)
9. Users will need to modify MATLAB preferences to point to the scripts under “Set Path”.
Select “Add with Subfolders” and choose /usr/local/scripts
10. Before attaching datastorage SELinux should be set to permissive
Modify /etc/selinux/config
SELINUX=permissive
Hardware/Software Details
Following lists OS, Matlab versions along with the Matlab toolboxes used and freenx.









Operating System Redhat v6.4 (http://nz.redhat.com/)
Matlab 2012b v8.0.0.783 (http://www.mathworks.com.au/products/matlab/)
SPM v8 (http://www.fil.ion.ucl.ac.uk/spm/)
ArtRepair v4 (http://cibsr.stanford.edu/tools/human-brain-project/artrepair-software.html)
CONN v.13 (http://www.nitrc.org/projects/conn/)
PLSgui/PLScmd v6.130310 (http://www.rotman-baycrest.on.ca/index.php?section=84)
AFNI v2011_12_21_1014 (http://afni.nimh.nih.gov/afni)
FSL v5 (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/)
ANTs 1.9v4 (http://stnava.github.io/ANTs/)
Network Details
To be handled by ITS. The standard connection mechanism is over SSH (port 22), though VPN will be
used to access this from outside the University.
Security Details
The group hippocampus-logon.psy should be used to limit access of the service.
Requirements Traceability
The design was set to solve the following objectives:





High availability
Reliability
Support
Flexibility
Performance
High Availabity
This will be accomplished by moving the system from a single workstation into the datacenter, and
moving the data as well. This will enable the data to be easily replicated if required, and also backed
to tape.
Reliability
As mentioned above, moving the system into the datacenter allows backup of data to tape. In this
system, a rebuild of the operating system and required software takes little time while following this
document. The data is most important, and having secure tape solves this issue.
Support
The design allows for a modular support system. In this case, rather than one person doing the
majority of the support from ground up, passing operating system and hardware support to ITS
allows their experts on those systems to provide better support in those areas.
Flexibility
Virtualising the environment then attaching discs from the SAN enables extension of storage easily,
and can be modeled such that disc is purchased ahead of time when necessary. Virtualising also
enables the addition of CPU and Memory as requirements arise.
Performance
Rather than having to purchase new hardware to increase performance, then requiring a rebuild of
the system, as mentioned above performance can be increased dynamically. Moving from single
discs to a SAN greatly increases performance also. Most importantly, application performance can
be monitored and resources modified as necessary to use them as effectively and efficiently as
possible.