EFS Overview
EFS in a nutshell
EFS stands for “Enterprise File System”
EFS is:
 A special-purpose Unix global file system
 A collaborative mechanism for developers to quickly and easily share and deploy
software
 A system for tracking releases of software
(whether internally-produced, open-source, or third-party vendor)
Goal: Overall Reduction in…
Time
Resources
Budget
EFS Benefits
Less time spent deploying software to internal clients
If the software is in EFS, any client attached to EFS will have
access to all core applications available
Recovery is quick and relatively painless:
Roll-back of software is done in near-seconds instead of hours or days
Less time administering individual boxes
Development/Deployment Teams have full responsibility for
deployment; no coordination between teams which can lead to
roadblocks, misunderstandings and additional time to address
these issues
Because of the centralized repository of EFS, no re-installing
software and libraries on different servers to run software
How can I Use EFS?
What does it mean to “use” EFS?
It could mean several things:
•
•
•
Developers and users making use of the
software already deployed in EFS to help
them do their jobs efficiently, without the
need for software-request tickets
Using EFS to make releases of software
developed internally available to regions
worldwide
Using EFS to manage the release
management and deployment of software
run within the firm’s sites worldwide
We refer to these three uses, respectively, as:
Access
Development
Deployment
An EFS Architecture Overview
A NAS filer containing the EFS data is called an EFS “Cell”
 A cell consists of every part of EFS’s infrastructure in a single
location:
 Storage
 EFS management servers
 Caches
Cell
Cell
Set
The collection of cells within a region is referred to as a “Cell Set” or “Regional Cell”
 When any UNIX server is connected to EFS, they access the
“EFS namespace.”
There are two environments in EFS:
DEV
 The DEV environment supports /efs/dev (read-write)
and /efs/dist (read-only, simulating PROD, used for testing, debugging, QA and UAT)
PROD (read-only)
 The PROD environment contains a “locally frozen” form of /efs/dist only
The /efs/dist data in both DEV and PROD is replicated worldwide to all Cell Sets
This replication is on-demand; commands are executed to cause freezing and replication
EFS Internal structure
EFS commands keep the environment very structured
• Namespace begins with /efs
• DEV environments hosts 2 areas:
– /efs/dev
• Read/write space to input binaries and test configuration/installation. This is the
developer’s “sandbox” environment
– /efs/dist
•
•
•
•
Testing area, from initial testing to final testing before moving to production
Read only, no writes are permitted or possible on this mount
“Simulates” PROD environment
Think of this as a QA space to prepare for PROD
• PROD environment hosts 1 area
– /efs/dist
• This area is always a subset of the information available in /efs/dist in the DEV cells
– Releases in DEV are being tested and may never be disted to PROD
– DEV has development tools deemed inappropriate for distribution into the PROD
environment
– PROD environment is on an entirely different filer, even though mounts are
named the same on both DEV and PROD cells
MPRs
Under both /efs/dist and /efs/dev, the next three levels of the file system hierarchy is
called the (MPR) Metaproject Project Release
Metaproject:
Form the top-level contentoriented hierarchy in EFS
 A development group’s own
area and must be defined
by the EFS group
 Permissions, user lists, and
access rights are specified
at this level
Project:
A specific software
product consisting of files
that are built together
 An application, library or
development project
 Has a “home cell” which is the
geographic location where
most writes originate from
Release:
A specific version of a
project
 The location where the
actual files reside
 Releases are versioned
and ordinarily immutable
(“frozen”) once
replicated (“disted”)
Indirection for stability and ease of use
Addressing multiple chipsets
Using Release Aliases
•
•
•
Developers compile for specific chip
architectures
– For languages like C and C++, binaries
must be compiled for the specific chip
set (i.e., Linux x86_64, Solaris SPARC)
– Teams working from that OS compile
the code and place binaries in specific
EFS location
– For languages like Java which do not
require compilation per chip set,
binaries are placed in a single location
– Same process is done for bitness (e.g.,
32 bit, 64 bit)
To run binaries users go to the same
location on any platform
– EFS indirection determines what chip
set and bitness and accesses the
appropriate binaries from paths
resolved at mount-time.
•
Use “ReleaseAiliases” to access binaries lets
dev team control access
ReleaseAliases are just symbolic links to
specific releases
– /efs/dist/foo/bar/1.2.3 may point to
/efs/dist/foo/bar/1.2.3-build001,
/efs/dist/foo/bar/1.2.3-build097, etc.
– By changing the releasealias, your users
get the release you want them to see
– Patches to releases can be done by
creating a new release (e.g.
1.2.3-build002, 1.2.3-build003, etc.) and
point the releaselink
(/efs/dist/foo/bar/1.2.3) to the specific
release
– Users access new release transparently
next time they start application
– No user reconfiguration or application
“re-tooling” is required
EFS Use Case
Challenges: Distributing software without EFS
#1 Challenge: Many moving parts, little control
•
If global, you may have regional abnormalities;
Standards for deployment in one region may
be totally different in another
•
•
•
•
Are all patches in place? For each operating
system?
Are they the same ones you tested with?
Do you need to do a different build for every
Region? Every Platform? Every machine?
Libraries may or may not be available. If there
is a question, do you include every single
library in every installation?
Isn’t that really inefficient?
EFS Use Case
Challenges: Distributing software without EFS (continued)
Getting Started
Document
installing custom
software, include
any custom
settings, based on
your DEV
environment or
Checking and Re-checking
Create
tarball
or other
compressed
package
Find QA
and UAT
machines
(hardware)
When errors occur:
Is it your application
or the environment?
Go back and test the
environment to be
sure
you are not missing
anything.
Set up the
machine
(all software,
all libraries, all
languagespecific items)
Check to be sure
all the libraries
are installed in
the correct
location with the
correct
parameters
If not, get these
re-installed;
change the
documentation
to reflect the
change
Have SAs install
your software,
based on your
instructions
Is this the right version?
Are there undocumented changes?
When completed, do initial testing to be sure everything is
in place. If not, contact SAs to schedule time for a reinstall
Document the changes, re-create tarball or other
compressed package
Repeat for next machine
EFS Use Case
Challenges: Distributing software with EFS
Getting Started
Dist the software to the /efs/dist area
in the DEV environment
It is now available for QA worldwide
Easy deployment to PROD
Change the stage of the release
Dist to the PROD environment
It is now ready to be deployed,
when you pull the switch
It is now available to whatever
worldwide cells you need it.
11010101011010101000101000011101010
HOW?
•
•
•
Since EFS takes care of the distribution and all the clients are connected to EFS,
running one command distributes your binaries throughout the enterprise
EFS DEV environment has a stage that mirrors PROD (/efs/dist), so testing is
done in an environment that simulates your target environment
All libraries are in exactly the same place in DEV and PROD, so missing libraries
is one less thing to check
When errors occur:
Fewer places to debug,
since the DEV and PROD
environments mirror
each other
Changes in process
Developers
 Redesign application to have read/write space
outside application namespace
 Application must run from its home directory
down
 Do not mix software with configuration
Application Support Teams
Be aware of EFS:
If EFS client is not installed, your application will not
work
System Administrators
Need to install EFS client on servers
Further contact
http://openefs.org
Edward Inderrieden
Manager, EFS Team
einderri@openefs.org