http://www.ukoln.ac.uk/cultural-heritage/events/cilip-scotland-2009/
How Far Have We Come?
From eLib to NOF-digi and Beyond
Brian Kelly
UKOLN
University of Bath
Bath, UK
Acceptable Use Policy
Email:
b.kelly@ukoln.ac.uk
Twitter:
http://twitter.com/briankelly/
Recording of this talk, taking photos,
discussing the content using email,
instant messaging, blogs, SMS, etc.
is permitted providing distractions to
others is minimised.
Blog:
http://ukwebfocus.wordpress.com/
Resources bookmarked using ‘cilips09' tag
UKOLN is supported by:
A centre of expertise in digital information management
This work is licensed under a AttributionNonCommercial-ShareAlike 2.0 licence
(but note caveat) www.ukoln.ac.uk
Introduction
Contents
Introduction
• About me
• About this talk
The National Programmes (provider’s perspective)
• The technical standards
• The support infrastructure
What We Learnt
• What succeeded, what failed and what we
discovered along the way
What Should We Do In The Future
• What do we do next?
A centre of expertise in digital information management
2
www.ukoln.ac.uk
Introduction
About This Talk
Talk is based on experiences of national programmes:
• eLib
• NOF-digi
• DNER/IE
• Michael
Common characteristics:
• Interoperability through open standards
• Managed view of roadmap
Issues:
• Did we get it right?
• Are there alternative approaches?
A centre of expertise in digital information management
3
www.ukoln.ac.uk
The eLib Programme
A centre of expertise in digital information management
4
eLib Programme:
• Response to the Follet
review, 1993
• Initial budget of £15m
over 3 years
• 60+ projects funded
Areas covered:
• Document Delivery
• Access To Network
Resources
• Training & Awareness
• Electronic Journals
• Digitisation / Images
• Electronic Short Loan
Collections
• On Demand Publishing
• Pre-Prints and Grey
Literature
• Supporting
Studies
www.ukoln.ac.uk
eLib Standards Guidelines
A centre of expertise in digital information management
5
eLib Standards
Guidelines:
• Provides
recommendations
for selection & use
of standards
• Strongly
encouraged where
relevant
Covered:
• Data
communications
• Data interchange
• Metadata
• Search & retrieve
• Security,
authentication &
payment
services
www.ukoln.ac.uk
eLib Standards
Editors: Chris Rusbridge
Lorcan Dempsey,& Ann
Mumford (myself as a
contributor)
Things we don’t care about (?):
• Email (SMTP, not X.400), …
Areas we correctly hedged our bets:
• GIF is OK, keep eye on PNG
Areas we were evasive about:
• PostScript & PDF
Areas we got wrong:
• “It is anticipated that SGML will be a key standard ...
Projects are encouraged to .. agree or, where necessary,
develop document type definitions”
• “projects should … supply a URL for public services, and
be prepared to adopt URNs when they are stabilised”
Standards which seem to have disappeared:
• ofCGM
A centre
expertise in digital information management
www.ukoln.ac.uk
6
eLib Standards version 2
eLib Standards version 2
• Published in 1998 (2 years after v 1)
• Introduced a template for descriptions
Relevant standards:
Comments:
Consensus:
• Templates on recommended standards
complemented by technical summaries – and
speculation e.g.
“HTTP-NG should support more secure
authentication and encryption”
A centre of expertise in digital information management
7
www.ukoln.ac.uk
Beyond The Standards
Cross-Searching: The Vision
• whois++ lightweight distributed cross-searching
protocol
• ROADS: Open source software used by most eLib
subject gateways (e.g. SOSIG)
• Z39.50: More heavy-weight solution used in library
context
• eLib SBIGs/RDN: implementation in a distributed
environment (with departmental providers)
Cross-Searching: Today’s Reality
• Intute: Centralised database, distributed data
collection. Cross-searching interfaces, but how
widely used?
Note dangers of using standards outside of DL programmes –
JISC
Web
site inupgrade
mandated
Z39.50 supportwww.ukoln.ac.uk

A centre
of expertise
digital information
management
8
Later, in NOF-digitise
NOF-Digitise programme:
• Ran from August 1999 to December 2004
• £50 million funding to put information that supports
lifelong learning into digitised form.
• Brought together wide range of partnerships &
organisations
NOF Technical Advisory Service (NOF TAS) provided:
• Informed support and advocacy of Technical
Standards and Guidelines
• Assistance in achieving standards compliance
• Detailed and project-specific advice
• Repository of standard and generic advice
Provide
by UKOLN
and AHDS
A centre of expertise
in digital information
management
www.ukoln.ac.uk
9
Development Culture
Different culture to HE digital library development community:
• Tell us the standards which we must mandate
• Caused problems:
 “NOF-digi project Web sites must have 24x7 availability” –
very expensive! Requirement was availability at weekends!
 Recommended standards weren’t mature (e.g. SMIL)
whereas proprietary solutions (Flash) provided compelling
user services
• Providing pragmatic solutions:
 “NOF-digi project Web sites should seek to maximise
their uptime”
 Quarterly reporting template provided a get-out clause:
“You must (a) describe the areas in which compliance will
not be achieved; (b) explain why compliance will not be
achieved (including research on appropriate open
standards); (c) describe your migration strategies to ensure
compliance in the future and (d) how the migration may be
funded”
A centre of expertise
in digital information management
www.ukoln.ac.uk
10
Minerva Experience
A centre of expertise in digital information management
11
Minerva technical
guidelines:
• EU-funded
• Built on
eLib/NOF/JISC
IE resources
• Initially edited
by UKOLN
Continued to
promote plausible:
• Standards
• Best practices
which failed to take
www.ukoln.ac.uk
off
Compliance Issues
What does must mean?
• You must comply with HTML standards
 What if I don't?
 What if nobody does?
 What if I use PDF?
JISC 5/99 programme
~80% of project home
pages were not HTML
compliant
• You must clear rights
on all resources you digitise
• You must provide properly audited
accounts
 What if I don't?
There is a need to clarify the meaning of must
and for an understandable, realistic and reasonable
regime
Acompliance
centre of expertise in
digital information management
www.ukoln.ac.uk
12
QA Focus
QA Focus:
• JISC-funded project provided by UKOLN and (initially) TASI,
then AHDS from 2002-2004
• What QA regime should JISC provide for its development
programmes? What actions should be taken if standards not
conformed with?
• Recommendations:
 Self-assessment, not external validation (projects
explained complexities of standards-compliance)
 Build on culture of sharing and openness
 Have a pragmatic view of ‘open standards’
 Understand complexities of non-conformance / ‘failure’:
 The standard failed, not the project
 The standard may be too expensive to deploy
 Alternatives may become available
A centre of expertise in digital information management
13
www.ukoln.ac.uk
Why Open Standards?
JISC's development programmes:
• Traditionally based on use of open standards to:
 Support interoperability
 Maximise accessibility
 Avoid vendor lock-in
 Provide architectural integrity
 Help ensure long-term preservation
But (thinking the unthinkable):
• Do open standards deliver?
• What happens if open standards fail?
• What is an open standard?
• Is the only alternative to open standards use of
proprietary solutions?
A centre of expertise in digital information management
14
www.ukoln.ac.uk
… But Don't Always Work 
There's a need for flexibility:
• Learning the lesson from OSI networking protocols
Today:
• Is the Web (for example) becoming over-complex
 "Web service considered harmful"
 The lowercase semantic web / Microformats
• Lighter-weight alternatives being developed
• Responses from the commercial world
Other key issues
• What is an open standard?
• What are the resource implications of using them?
• Sometimes proprietary solutions work (and users
like
them).
Is itinformation
politically
incorrect to mention
this!?
A centre
of expertise
in digital
management
www.ukoln.ac.uk
15
What is An Open Standard?
Which of the following are open standards today
(and were open standards in 2006)?
• XHTML 1
 PDF
 Flash
• Java
 MS Word
 RSS (1.0/2.0)
UKOLN's "What Are Open Standards?" briefing paper
refers to characteristics of open standards:
• Neutral organisation which 'owns' standard &
responsible for roadmap
• Open involvement in standards-making process
• Access to standard freely available
•…
Note these characteristics do not apply equally to all
standards bodies e.g. costs of BSI standards; W3C
membership
…
A
centre of expertise inrequirements;
digital information management
www.ukoln.ac.uk
16
RSS Example
Is RSS An Open Standard?
Is RSS an open standard ("are RSSs open standards")?
RSS 1.0 (RDF Site Summary)
• XML application using RDF model
• Developed by Aaron Schwarz
RSS 2.0 (Really Simple Syndication)
• XML application using simpler model
• Developed by Davey Winer
Note that RSS is a widely used and popular application;
with usage growing through its role in podcasts
Issues:
• Are these open standards?
• Are they reliable and robust enough to build
mission-critical services on?
• Is there a clear roadmap for the future? www.ukoln.ac.uk
A centre of expertise in digital information management
17
RSS Example
RSS – Governance Issues
Governance Issues:
• RSS 1.0 spec maintained by Aaron Schwartz:
"Aaron Swartz is a teenage writer, hacker, and
activist. He was a finalist for the ArsDigita Prize for
excellence in building non-commercial web sites
at the age of 13. At 14 he co-authored the RSS
1.0 specification, now used by thousands of sites
to notify their readers of updates."
• RSS 2.0 specification developed by Dave Winer:
"Winer is known as one of the more polarizing
figures in the blogging community. … However ..
there are many people and organizations who
seem unable to maintain a good working
relationship with Dave."
A centre of expertise in digital information management
18
www.ukoln.ac.uk
RSS Example
RSS – Summary
To summarise :
• We thought RSS was a great lightweight
syndication technology
• It was – but competing alternatives were
developed
• No clear winner (RSS 1.0's extensibility & W3C's
support versus RSS 2.0's simplicity and take-up
in podcasting, iTunes, etc)
Conclusions
• Life can be complex, even with simple standards
• Technical merit is never enough – market acceptance can
change things
• RSS can still be useful, and interoperability can be provided by
RSS libraries supporting multiple formats
• Need for a more sophisticated approach such as model in
“AofContextual
Framework
For Standards”, WWW 2006
A centre
expertise in digital
information management
www.ukoln.ac.uk
19
Contextual Issues
The Context
There will be a context to use of standards:
• The intended use:
 Mainstream
 Innovative / research
 Key middleware component  Small-scale deliverable
• Organisational culture:
 HE vs FE
 Service vs Development
 Teaching vs Research
 …
• Available Funding & Resources:
 Significant funding & training to use new standards
 Minimal funding - current skills should be used
• …
An open standards culture is being developed, which is
supportive of use of open standards, but which recognises
the complexities
and
caninformation
avoidmanagement
mistakes made in the
past
A centre of expertise
in digital
www.ukoln.ac.uk
20
The Layered Standards Model
Owner Quality Assurance
External factors: institutional, cultural, legal, …
JISC
3rd
Parties
Context: Policies
Prog. n Funding
Research Sector …
Annotated Standards Catalogue
Purpose Governance Maturity Risks …
JISC /
project
Context: Compliance
External Self assessment Penalties Learning
JISC's layered standards model, developed by UKOLN.
Note
that
one
sizeinformation
doesn't
always fit all
A centre of
expertise
in digital
management
www.ukoln.ac.uk
21
The Standards Catalogue
The information provided aims to be simple and
succinct (but document will still be large when printed!)
Standard: Dublin Core
Example
About the Standard: Dublin Core is a metadata standard made up …
Version: New terms are regularly added to …
Maturity: Dublin Core has its origins in workshops held …
Risk Assessment: Dublin Core plays a key role …. It is an important
standard within the context of JISC development programmes.
Further Information:
• DCMI, <http://dublincore.org/>
Note that as the standards
•…
Author: Pete Johnston, UKOLN
catalogue is intended for
Contributor:
wide use the contents will
Date Created: 04 Oct 2005
need to be fairly general
Update History: Initial version.
The Standards Catalogue is deemed important – but there’s a
A centre of expertise in digital information management
www.ukoln.ac.uk
22 still lack of understanding of the contextual model
What If Web 2.0 Changes Everything?
“Web 2.0 Changes Everything” – what if this is true?
The “Higher Education in a Web 2.0 World” report
suggests that Senior Managers in HE may feel this to
be the case
• “Network is the platform” / The Cloud
• Web infrastructure becomes the infrastructure
(HE follows, no longer leads)
• Growing importance of informal learning
• Growing importance of informal networking
• Growing reluctance to travel (travelling to CILIP-S
on par with dodgy MPs’ expenses claims?)
• …
A centre of expertise in digital information management
23
www.ukoln.ac.uk
The DNER/IE Diagram
Web 2.0 in the context
of Andy Powell’s
famous IE diagram
(early version shown)
..which later was
developed further
A centre of expertise in digital information management
24
www.ukoln.ac.uk
My Take
My Vision
In 2001 I suggested that
application services could
be provided ‘out there’ (in
The Cloud).
I speculated about the
JISC Spellchecker and
JISC ‘delicious’ services
What I Missed!
What I thought about but failed to articulate (it seemed (a)
Thatcherite out-sourcing & (b) too complex) was commercial
provision
of the services & large-scale apps e.g. Google
Docs
A centre of expertise in digital information management
www.ukoln.ac.uk
25
Services In ‘The Cloud’
Will Web 2.0
services in ‘The
Cloud’ make
national initiatives
irrelevant?
Or will there be a
mix of institutional,
national and global
providers of
solutions?
Or will institutional
& national services
make use of
infrastructure in
‘The Cloud’?
A centre of expertise in digital information management
26
www.ukoln.ac.uk
What About The Developers?
In the old days:
• Development was slow and
required significant levels of
funding
• Funders and budget
holders could manage
development process
Today:
• Web infrastructure more
mature (standards,
services, APIs, …)
• Light-weight is ‘cool’
• Developers don’t want 3
year projects (and
associated bureaucracy) –
but food & drink are good!
A centre of expertise in digital information management
27
www.ukoln.ac.uk
What Can Be In A Weekend?
Tony Hurst’s visualisations of
MPs expenses claims
A centre of expertise in digital information management
28
www.ukoln.ac.uk
Community Matters
http://dev8d.jiscinvolve.org/2009/03/30/
Importance of developer
community now being appreciated:
• In JISC Circles (cf dev8d week;
Mashed Library events; Rapid
Innovation Call)
• In Museums sector (cf. Mashed
Museum events)
• In commercial sector (cf
barcamps)
• In government circles (cf.
Government barcamps)
• …
Dev8D: Developer Happiness Days event sponsored by
JISC. See <http://dev8d.jiscinvolve.org/>
A centre of expertise in digital information management
29
www.ukoln.ac.uk
Risk Assessment Model (c. 2006)
SS=f(SB, S, U, En, ..)
Selection of appropriate standard (SS) is function of:
Standards Body (SB): Maturity, stability, status,
openness, responsivity, …
Standard (S): Functionality, complexity / ease-of-use, …
Users (U): Appropriateness for, benefits to adoption by …
Environment (En): Institutional, community, sectoral, …
Other factors:
• Market acceptance: do vendors support it (beyond
proof-of-concept open source examples)
• Risks (am I betting the company of the standard)
• Exit options (can I easily change my mind)
• Advocacy (is the world campaigning for it) and
threats (is the world criticising for it)
A centre of expertise in digital information management
30
www.ukoln.ac.uk
Deployment Strategies
31
Interested in using Web 2.0 in your organisation?
Worried about corporate inertia, power struggles, etc?
There’s a need for a deployment strategy:
• Addressing business needs
• Low-hanging fruits
• Encouraging the enthusiasts
• Gain experience of the browser tools – and see
what you’re missing!
• Staff training & development
• Impact assessment and measurement
• Risk and opportunity management strategy
• Critical Friends and friendly critics
• Culture of sharing
A centre
www.ukoln.ac.uk
• of…expertise in digital information management
Strategies
32
Risk Management
JISC infoNet Risk Management infoKit:
“In education, as in any other environment, you can’t
decide not to take risks: that simply isn’t an option in
today’s world. All of us take risks and it’s a question of
which risks we take”
Examples of people who are likely to be adverse stakeholders:
• People .. required to commit resources to the project
• People who fear loss of control over a function or
resources
• People who will have to do their job in a different way
• People who will have to carry out new functions
• People who will have to use a new technology
A centre of expertise in digital information management
www.ukoln.ac.uk
• ..
Critical Friends
JISC U&I
programme is
encouraging
establishment of
“Critical Friends”
See <http://dev8d.jiscinvolve.org/2009/
See
<http://critical-friends.org/>
02/10/five-minute-interview-paul-walk/>
A centre of expertise in digital information management
33
Paul Walk
(UKOLN) was
described as a
‘critical friend’ of
JISC
www.ukoln.ac.uk
Towards a Framework
Biases
• Critical friends
• Application to
existing
services
• Application to
in-house
development
•…
Intended
Purpose
Benefits
(various
stakeholders
Risks
(various
stakeholders
Missed Opps.
(various
stakeholders
Costs
(various
stakeholders
• Sharing
experiences
• Learning from
successes
& failures
• Tackling biases
•…
“Time To Stop Doing and Start
Thinking: A Framework For
Exploiting Web 2.0 Services”,
Subjective factors
Museums & the Web 2009
A centre of expertise in digital information management conference
www.ukoln.ac.uk
34
Using The Framework
Twitter for individuals Organisational Fb Page
Community
support
Rapid
feedback
Justify ROI
Org. brand
Communitybuilding
Low?
Intended
Purpose
Benefits
(various
stakeholders
Risks
(various
stakeholders
Missed Opps.
(various
stakeholders
Costs
(various
stakeholders
Marketing
events,…
Critical friends:
• Paul Walk / Brian
Kelly blog posts)
Large
• MCG discussions
audiences
Learning
• UKOLN cultural
Ownership,
heritage guest
privacy, lock-in
blog post
Marketing
• Conferences
opportunity
• Papers
•…
Low?
Note personal biases!
Use of approach in two scenarios: use of Twitter & Facebook
A centre of expertise in digital information management
35
www.ukoln.ac.uk
Using The Framework (2)
Semantic Web
No standard
Intended
Purpose
Benefits
(various
stakeholders
Risks
(various
stakeholders
Missed Opps.
(various
stakeholders
Costs
(various
stakeholders
36
Critical friends:
• JISC Advisers
• Developers
• International
community
Learning
• From developers
• Conferences
• Papers
•…
Use of approach with standards: doing nothing (today) might be an
Aoption!
centre of expertise in digital information management
www.ukoln.ac.uk
The Assumptions
Standards:
• Interoperability through open standards
• Avoidance of proprietary lock-in & other benefits
• All we need is to identify the correct open
standards
• This will save us time, money & deliver rich
functionality and usable & useful services
Development:
• The developers can then simply use the
standards
• This will also provide seamless evolution to new
standards
A centre of expertise in digital information management
37
www.ukoln.ac.uk
Challenging The Assumptions
Maybe we want:
• To challenge the unthinking assumptions in national
development programmes –using evidence rather than
assertions
• The benefits promised (but not necessarily delivered) by
open standards
• An understanding that it’s not a binary open standards vs
proprietary world
• The world may choose good enough, whilst we want to
provide the best
• To develop user-focussed services which the commercial
sector seems to be better at
• To recognise the importance of the developers’ perspective
And we should also challenge these views!
A centre of expertise in digital information management
38
www.ukoln.ac.uk
Questions
Any questions, comments, …?
39
Additional Resources:
Papers published on standards and national programmes:
• What Does Openness Mean To The Museum Community?, MW 2008
• Openness in Higher Education: Open Source, Open Standards, Open
Access, elPub 2008
• Addressing The Limitations Of Open Standards, MW 2007
• A Contextual Framework For Standards, WWW 2006
• A Standards Framework For Digital Library Programmes, ichim05
• Interoperability Across Digital Library Programmes? We Must Have
QA!, ECDL 2004
• Deployment Of Quality Assurance Procedures For Digital Library
Programmes, IADIS 2003
• Developing A Quality Culture For Digital Library Programmes, EUNIS
2003
• Ideology Or Pragmatism? Open Standards And Cultural Heritage Web
Sites, ichim03
A centre of expertise in digital information management
www.ukoln.ac.uk
See <http://www.ukoln.ac.uk/web-focus/papers/#standards>