Slides: http://dx.doi.org/10.6084/m9.figshare.878120
www.software.ac.uk
Putting the user
back into software
sustainability
16 December 2013, Scientific Software Days, Austin
Neil Chue Hong (@npch), Software Sustainability Institute
ORCID: 0000-0002-8876-7606 | N.ChueHong@software.ac.uk
Supported by
Project funding
Software
from Sustainability Institute
Where indicated
slides licensed under
If I was a crossword compiler…
www.software.ac.uk
Putting the user
back into software
sustainability
But I’m not, so this talk is going to
draw on my experiences with
scientific software
Software Sustainability Institute
The problem with
mushrooms…
Software Sustainability Institute
www.software.ac.uk
OMII: Sowing the first seeds
www.software.ac.uk
11 initial projects funded
by Managed Programme
Many projects
flourished
Software Sustainability Institute
But some wilted and
decayed
OMII-UK: Cultivating and
Nurturing
www.software.ac.uk
• Emphasis on helping
existing software grow
Cultivate and sustain community
software important to research
 Extra gardeners brought in
(Edinburgh and
Manchester) with their
own plant stock
 Making the garden public
through initiatives like
Google Summer of Code
and ENGAGE
 Inviting specialists through
the PALs scheme
Software Sustainability Institute
The Software Sustainability
Institute
www.software.ac.uk
A national facility for cultivating worldclass research through software
• Better software enables better research
• Software reaches boundaries in its
development cycle that prevent
improvement, growth and adoption
• Providing the expertise and services
needed to negotiate to the next stage
• Developing the policy and tools to
support the community developing and
using research software
Software Sustainability Institute
Supported by EPSRC
Grant EP/H043160/1
Can we learn?
Picture by Heisenberg Media
Software
Sustainability
Institute
Picture
by nengard
www.software.ac.uk
Picture by PNNL
Startup
companies are
all about reaching
sustainability
Software Sustainability Institute
www.software.ac.uk
Four stages of startups
www.software.ac.uk
Discovery
Are you solving a problem that
others are interested in?
Validation
Efficiency
Scale
Have you implemented core
features that users want?
Can you support new users by
refining your processes?
Ready to drive growth.
Back-end scalability refactoring
Software Sustainability Institute
http://blog.startupcompass.co/pages/marmer-stages
Stage 1: Discovery
www.software.ac.uk
• Are you solving a problem that others are
interested in?
 Many pieces of software created by researchers
have small user bases – this is particularly true of
scripts
 Not a problem if you are writing the software for
yourself only – but it affects how large the project
to support the software can be
Software Sustainability Institute
Case Study: Ligand Binding
www.software.ac.uk
• Centre for Computational Chemistry, Bristol
 New methods for rapid MC sampling of
biomolecular systems modelled using QM/MM
 Developed two codes ProtoMS (F77) + Sire (C++)
 Water-Swap Reaction Coordinate method to
calculate absolute protein-ligand binding free
energies
• SSI’s work helped assess users + scale devs
 Ran user observations with 4 different users
 ASPIRE/ACQUIRE framework has multiple devs
• Split architecture between ASPIRE (adaptive
multiresolution hybrid MD simulation) and ACQUIRE
(WorkPacket scheduling system with optimisation
for time to result vs “green-ness”
•
•
http://www.software.ac.uk/resources/case-studies/getting-gripsmolecules
http://www.siremol.org/adaptive_dynamics
Software Sustainability Institute
Stage 2: Validation
www.software.ac.uk
• Have you implemented core features that users
want?
 Do you know who’s using your software? Why are
they using it?
 If you asked them “how would you feel if you can
no longer use this software”, how many would be
disappointed?
 These are your core users: what do they want and
how can you give it to them?
Software Sustainability Institute
Case Study:
Climate Policy Modelling
• CIAS team at Tyndall Centre for Climate Change
Research, University of East Anglia
 Develop linked climate and economic models for
detailed analysis
 Their software was not ready to be used by other
groups
• One researcher/developer at UEA, several users
• SSI’s work means the software is robust enough that
it can be installed and used by others
 Enabled use of the software by the WWFN’s
Climascope project and James Cook University
•
•
•
•
Documented software to allow extensions by contributors
Made it easier to maintain and backup
Added job scheduling to improve modeling throughput
New modelling framework enables new models i.e. new
science
• http://www.tyndall.ac.uk/research/cias
Software Sustainability Institute
www.software.ac.uk
Stage 3: Efficiency
www.software.ac.uk
• Can you support new users by refining your
processes?
 If you had conflicting requirements from users,
how would you deal with them?
 What infrastructure changes do you need to make
to support new/additional users?
 At this point, scientific software projects often start
designating specific community/product managers,
user support staff
Software Sustainability Institute
Case Study: ICAT
www.software.ac.uk
• Science and Technology Facilities Council
 Metadata catalogue, used by RAL UK (ISIS,
DIAMOND, CLF), SNS US, ELLETRA Italy
 ICAT operationally critical at sites
 Huge projects looking to use ICAT
(PaNdataODI, EuDAT)
 Scalability issues and lack of proper processes
• SSI’s work provided 33 recommendations
 15 interviews with different stakeholders
 92 observations set out in report
 “…we must focus on doing the right things, and
this report will help us”
• Alistair Mills, STFC
 Governance and outreach changes to support additional users
•
•
http://www.software.ac.uk/preparing-icat-thousands-new-users
http://www.icatproject.org/
Software Sustainability Institute
Stage 4: Scale
www.software.ac.uk
• Are you ready to drive growth of users, to
reengineer and refactor on an ongoing basis?
 This is when software quality considerations
become very important, as you have increased
reputational risk
 This is also the point where traditionally a PI would
step aside to become Chief Technology Officer /
Chief Scientist and enlist new management
• After this, the next stage is sustain
 Though this might be at a small scale if appropriate
Software Sustainability Institute
www.software.ac.uk
Open source
software projects are
all about reaching
sustainability
Software Sustainability Institute
Open Source Software Projects
www.software.ac.uk
• “Every good work of software starts by
scratching a developer's personal itch.”
 Eric Raymond, The Cathedral and the Bazaar
• Producers start by having a direct interest
in the success of the software
 Just like in science
• OSS projects need to satisfy two aims:
 Acquire users (a.k.a. researchers)
 Acquire contributors (a.k.a. collaborating researchers)
• Producing Open Source Software: How to Run a
Successful Free Software Project by Karl Fogel
Software Sustainability Institute
Using infrastructure to
become user aware
Software Sustainability Institute
www.software.ac.uk
Promotion:
- website
Communication:
- mailing list, wiki
Collaboration:
- code repository
Management:
- issue tracker
Scientific
research is
all about reaching
other researchers
Software Sustainability Institute
www.software.ac.uk
Software is pervasive
in research
Software Sustainability Institute
www.software.ac.uk
No common baseline
www.software.ac.uk
Bhargen Basepair
Fan Fullerene
Helen Helmet
Mehrdad Mapping
Bioinformatics RA
Novice coding in Java
and Perl to test groups
pattern algorithms
Chemistry PhD
Running analyses
on behalf of his
supervisor
Mech Eng intern
Exploratory coding
based on tinkering
with legacy code
Forestry student
Field data collection
currently correlating
using Matlab
Software Sustainability Institute
Slide courtesy of Greg Wilson
The modern researcher…
www.software.ac.uk
• … worries about:
Picture of Otto Stern courtesy of
Emilio Segre Visual Archives
 Data management
and analysis
 Reproducible
research
 Scalable simulations
 Integration of
models and
workflows
 Collaboration
Software Sustainability Institute
Why do you need users?
www.software.ac.uk
• Funding
 Direct: fees, subscriptions, …
 Indirect funding: letters of support,
citations, collaborations
 Advertising: recommendations and
referrals
• Direction (indirection?)
 Requirements, bug reports, change
requests
• Community
 Users supporting other users
 Users becoming contributors
 Sustainability and success
Software Sustainability Institute
4 Golden Rules
www.software.ac.uk
- Act Amiably
- Breakdown Barriers
- Cultivate Contributors
- Diaspora + Dispersion
Software Sustainability Institute
A: Act Amiably
www.software.ac.uk
• Bugzilla project did analysis of their
current and previous contributors
 Once someone started contributing, what
kept them around?
•
•
•
•
•
Don’t freeze the trunk for long periods
Turnover is inevitable
Respond to contributions immediately
Be extremely kind and visibly appreciative
Encourage a total absence of negativity
 Software sustainability relies on many people working
happily together!
http://www.codesimplicity.com/post/open-source-community-simplified/
Software Sustainability Institute
B: Breakdown Barriers
www.software.ac.uk
•
•
Basics: Website, mailing list, code repository, issue resolution
Remove barriers to participation, increase efficiency
•
•
•
•
1993: First public release; 2 devs
1995: Code open sourced; 3 devs
1996: r-testers list set up
1997: lists split: r-announce, r-help,
r-devel; public CVS; 11 devs
2000: CRAN split and mirror
2001: BioConductor
2003: Namespaces
2005: I8n, L8n
2007: R-Forge
Today: BioConductor (33 core devs),
R-Forge (532 projects, 1562 devs),
CRAN (1400+ packages)
•
•
•
•
•
•
http://cran.r-project.org/doc/html/interface98-paper/paper_2.html
Software Sustainability Institute
34
C: Cultivate Contributors
www.software.ac.uk
Building intellectual access ramps to support incremental
engagement – building capacity and capability
•
•
•
•
•
•
Individual
Group
Consortium
W/ industry
Community
Active
Teams change as
project matures
Software Sustainability Institute
http://www.castep.org/
35
D: Diaspora + Dispersion
www.software.ac.uk
• Sometimes, serendipity plays a part
• E.g. GEOS-CHEM
• Users spread across
world means they
have to scale
• But they also have
great support and
contributor
mechanisms
http://acmg.seas.harvard.edu/geos/
Software Sustainability Institute
Sustainability is also about
choosing not to continue
www.software.ac.uk
Software proliferation
Not all software should make
it to the next stage
Consolidation is as important
as innovation in achieving
sustainable scientific software
Innovation
Consolidation
Customisation
Time
Software Sustainability Institute
www.software.ac.uk
Take home message
Understand and embrace your
users and contributors if you
want your scientific software to
be sustainable and successful
Software Sustainability Institute
Slides: http://dx.doi.org/10.6084/m9.figshare.878120
Find out more about the SSI
www.software.ac.uk
• Community Engagement (Lead: Shoaib Sufi)
 Fellowship Programme
 Events and Workshops
• Consultancy (Lead: Steve Crouch)
 Open Call for Projects / Collaborations
 Software Evaluation
• Policy and Publicity (Lead: Simon Hettrick)
 Case Studies / Policy Campaigns
 Software and Research Blog
• Training (Lead: Aleksandra Pawlik)
 Software Carpentry (300+ students/year)
 Guides and Top Tips
• Journal of Open Research Software (Editor: Neil Chue Hong)
•
Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton
Software Sustainability Institute
Slides: http://dx.doi.org/10.6084/m9.figshare.878120
A national facility for cultivating
world-class research through software
Some of our collaborations
Become our next collaborators!
Website:
Email:
Twitter:
www.software.ac.uk
info@software.ac.uk
twitter.com/SoftwareSaved
Software Sustainability Institute
www.software.ac.uk