STRATEGY DOCUMENT Digital Library Service Framework V 0.7 Innovation Branch www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Document history Version Date Author Comments 0.1 31/10/2007 Innovation Branch first cut by the Business Integration Taskforce 0.2 2/1/2008 Innovation Branch early internal exposure draft 0.3 8/2/2008 Innovation Branch exposure draft for inclusion in BIT report to CMG 0.4 14/2/2008 Innovation Branch comments from Collection Management and Delivery Group 0.5 21/2/2008 Innovation Branch comments from walkthrough with Warwick 0.6 26/2/2008 Innovation Branch mappings from DLF API recommendations 0.7 28/2/2008 Innovation Branch comments from walkthrough with IT Apps managers 2│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 Introduction The National Library of Australia is developing its digital library services as a single business. Figure 1 is a diagrammatic representation of an overarching service framework that will be used to enable the development of this business through a service-oriented approach. The purpose of this document is to outline how each of the functions in this diagram are currently supported in the Library and to identify strategies for migrating to a more integrated technical infrastructure based on standard protocols where there is a demonstrable benefit to the Library and its users in doing so. In this document function is defined as a business process supported by a service that needs to be distinguished for access control, interoperability or reporting purposes. Service is defined as a system that provides one or more functions of value to the end user (ISO 2146 Information and Documentation – Registry services for libraries and related organizations). This is a working document and new functions may be added and existing ones renamed, split or merged with other functions in order better to provide a roadmap for ongoing development of the Library’s digital library services. Generally speaking, for a function to be included it needs to be executed through a service request or trigger and a service response. Some of the functions included in this document may, on further analysis, resolve to a sequence of functions executed as a workflow that can be described through a Service Usage Model (SUM). An example of a SUM is given in Annex 1. The document may therefore evolve over time into a quite small set of core functions, a set of SUMs and a set of workflows expressed in the form of BPMN diagrams. The Library’s Business Integration Taskforce has been developing the Metadata Services and Discovery Services functions but has still to complete this work. A separate working group was set up to populate the Collection and Delivery Services functions during January but this is a large task that will need to be carried forward through specific projects. The framework is also being tested by APSR against usage scenarios for the future Australian National Data Service (ANDS). In the National Library of Australia, the framework will be used in conjunction with business process modelling diagrams under development through a range of separate Balanced Scorecard initiatives in order to provide a complete view of the interfaces required to support the Library’s digital library workflows. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 3│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Figure 1 – Digital Library Service Framework 4│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 High level usage scenarios The diagram in Figure 1 was included in an early form in the ISO/CD draft of ISO 2146 that was balloted in 2007. In the ISO/DIS draft to be balloted in 2008, the current version will be included in an informative annex providing guidelines for the development of registry services. A registry is defined in ISO 2146 as a collection of registry objects compiled to support the business of a given community. A registry object may be a party, collection, activity or service. Figure 1 shows how this data is the glue that makes a function work1. Figure 2 was developed for the ISO 2146 annex to provide guidelines for the development of collection service registries. It shows the relationship between the functions defined in the framework and the four different kinds of collections defined in the standard – collections (or collective works), repositories, catalogues or indexes and registries. Figure 2 – Collection Service Registries Items and collections (published or unpublished, physical or digital) are located in repositories owned and managed by collecting agencies (libraries, museums, archives, learning and research institutions and data and documentation centres). 1 For example, to execute an authorisation request the requester passes user details such as affiliation and category to the responder. The responder tests these against access policies. To do this information must be managed (or accessible) by the requester about the authorisation attributes needing to be presented to access a given service and the responder must maintain a set of machine-readable access policies. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 5│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 The contents of repositories and collections within them are described by catalogues or indexes2. Registries may describe collections, repositories or catalogues or indexes. A registry is itself a collection and may be described in a super-registry. The functions and protocols needing to be described in collection service registries will depend on the collection type: ▪ Creators employ Content services to create, submit, publish, value and annotate content. ▪ Managers of repositories employ Collection services to select, acquire, ingest, describe, control and preserve content. They also use Delivery services to enable users to resolve to items and collections in the repository and to reproduce or lend items not yet available online or to put on item on hold for a user if it is not immediately available. ▪ Managers of catalogues or indexes and of registries employ Metadata services to aggregate, ingest, augment and share metadata. Users of catalogues and indexes use Discovery services to search and scan, locate, bookmark or request items. Common services (authenticate, authorise, pay, log, audit, alert) and User services (register, ask, personalise, commission, track, report, comply) may apply to all usage scenarios. 2 The Dublin Core Collection Description Type (CDType) Vocabulary categorizes these as analytic finding aids, hierarchical finding aids and indexing finding aids. Analytic finding aids are a collection of individual records, hierarchic finding aids describe items in a collection using a hierarchical arrangement and may not describe each item individually. Indexing finding aids contain information derived from documents or metadata records, for example full-text indexes of digital or digitized works. 6│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 Information model Figure 1 identifies three categories of data needed to support the functions defined in the service framework. These are listed below with their associated information models. Protocols and data schemas appropriate to each function are identified in the service framework itself. Registry object (party, collection, activity, service) ISO/CD 2146 Information and Documentation – Registry Services for Libraries and Related Organizations Metadata (resource, topic) International Standard Bibliographic Description (ISBD) Functional Requirements for Bibliographic Records (FRBR) Functional Requirements for Authority Data (FRAD) Functional Requirements for Subject Authority Data (FRSAD) Resource Description and Acces (RDA) Business data (user, transaction) ISO/DIS 8459 Bibliographic Data Element Directory Users and Information Resources: An Extension of the Analytical Model of Collections and their Catalogues into Usage and Transactions ebxml: Enabling a Global Electronic Market www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 7│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Arrangement This document is arranged according to the service hierarchy outlined in figure 1. Each function is described as follows: E-framework: Link to the corresponding service genre in the E-framework where one exists or an indication of a missing service genre3. DLF: Link to the corresponding APIs in the DLF ILS and Discovery Systems Draft Recommendations (15 February 2008)4. Definition: Definition of the function based on the E-framework definition where one exists but modified to reflect the Library’s single business directions. Use cases: Identification of the primary behaviours or business processes needing to be supported by the function (e.g. Authorise) and the interface objects needing to be exchanged to execute the function (e.g., Decision request, Decision response).5 Protocols: List of known protocols that are candidates for executing the function and schemas that are candidates for packaging content and data. Data: The data needed to support execution of the function. Systems: Summary of the systems already in place in the Library to support this function with an assessment of limitations and gaps needing to be addressed. Strategies: Statement of the Library’s directions for delivering this function. References: Relevant documents. Note that functions to create, read, update and delete data objects are grouped under a single ‘Manage’ use case. 3 The E-framework is “an international initiative that provides information to institutions on investing in and using information technology infrastructure. It advocates service-oriented approaches to facilitate technical interoperability of core infrastructure as well as effective use of available funding.” The partners are DEST (Australia) and JISC (UK), New Zealand’s Ministry of Education (MoE) and the Netherlands' SURF Foundation. Note that this document is not intended to duplicate the E-framework definitions but to test the Library’s digital library requirements against existing service genres as a way of benefiting from the work already done and contributing to the international initiative. 4 The ILS and Discovery Systems group is a group convened by DLF to ‘analyze the issues involved in integrating "integrated library systems" and discovery systems, and create a technical proposal for how such integration should be accomplished’. 5 In most cases the use case name may be the same as the function but some functions may split out into more than one sub-function 8│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 1 Common Services Common services are services with systems as the primary actor. They work across applications to identify who the user is, what they are able to do and the conditions that apply. They manage any e-commerce obligations, log and audit usage and provide notifications or alerts. 1.1 AUTHENTICATE E-framework: Authenticate DLF: Core function (Patron functionality/AuthenticatePatron) Registry (Patron functionality/LookupPatron; GetPatronInfo; GetPatronStatus) Definition: Process whereby a system verifies the identity of a user. The user may be a person using a computer, a computer itself or a computer program. Use cases: Login (Authentication request, Authentication response) Logout (Logout request, Logout response) Protocols: LDAP, Shibboleth, OpenID, Secure Shell Keys, Certificates Data: Business (User, Password, Session) Systems: All systems requiring authenticated access. Strategies: The Library’s longer term solution will be developed by the Identity Management Project. This Project has not yet been scheduled in the IT Operational Plan. In the short term, these services will be provided by a combination of the Libraries Australia Administration module and the Patron Administration service. Internally, our identity management infrastructure is based on LDAP. There is a future need to support both Shibboleth and OpenID. Kent Fitch has also proposed that we build a gateway between Shibboleth and web-based public library authentication services. References: Licensed electronic content (NLA, Jan 10, 2008) https://wiki.nla.gov.au/display/NSLAIAP/Licensed+electronic+content 1.2 AUTHORISE E-framework: Authorise DLF: Doesn’t separate Authenticate from Authorise Definition: Process whereby a system verifies that a known user has the authority to undertake a particular action with a specified resource, with associated registry functions to manage policies and authorisation attributes. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 9│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Use cases: Authorise (Decision request, Decision response) Data: Registry (Access policy) Business (User category, Authorisation attributes) Protocols: XACML, auEduPerson, MAMS Autograph, Unix file permissions, database permissions Systems: Strategies: The Library has based its Rights Management System on a data model that will support XACML-encoded requests and responses. Libraries Australia has a controlled vocabulary for user category that can be mapped to auEduPerson values. The Australian Access Federation has developed support for decision requests and responses using these Protocols. The Library will need to build up its capability to engage in discussions in this space if it wants to interoperate with the research sector. 1.3 PAY E-framework: No equivalent yet. DLF: No recommendation yet. Definition: Process whereby a system supports the making of a payment for a service or product. Use cases: Pay (Payment request, Payment response) Data: Registry (Customer, Service, Product, Charge) Business (Payment details) Protocols: UN/EDIFACT, ANSI X12, XML EDI, Micro Payment Transfer Protocol (MPTP), W3C XML E-commerce Protocols (e.g. Biztalk, WSDL, ebXML) Systems: NAB Payment system (withdrawn); Secure server for collecting payment details for manual processing; Finance 1 Strategies: Proposed projects relating to E-commerce (Finance Branch) and the Online Shop (Publications and Events Branch) will provide long term solutions. The Library’s strategy will be to use modern mainstream protocols and to investigate adapting an existing service developed by someone like CISTI. References: Metadata for Electronic Commerce (Tom Worthington, 2001-2006) http://www.tomw.net.au/technology/it/ecommerce/Protocols.shtml#Protocols 10│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 1.4 LOG E-framework: Log DLF: No recommendation yet. Definition: Process whereby a system records the usage of a service or product for audit, reporting and/or billing purposes Use cases: Log (log trigger, log response) Data: Registry (Service, Product) Business (User, Usage details) Protocols: NISO Standardized Usage Statistics Harvesting Initiative (SUSHI) Systems: Most of the Library’s services have a logging function. Strategies: Build on our experience with Libraries Australia and the guidance provided by protocols such as SUSHI . 1.5 AUDIT E-framework: Audit DLF: No recommendation yet. Definition: Process whereby a system compares a log, or audit trail, against a set of policies. Auditable events will have already been defined and the log to be audited will have already been created Use cases: Undertake audit (Audit trigger, Audit response) Data: Business (Audit details) Protocols: None yet identified Systems: The Library monitors all services to ensure that they are operating but this is done through a test event. Application-specfic audits are also conducted. Strategies: None yet articulated 1.6 ALERT E-framework: Alert DLF: No recommendation yet. Definition: Process whereby a system provides alerts or notifications to users based on a trigger that may be time-based or generated as the outcome of an audit. Use cases: Generate alert (Alert trigger, Alert response) Data: Business (Alert details) www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 11│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Protocols: RSS, Atom, Email, ftp, sms Systems: Some of our internal systems have alerts that result in emails to staff; some of our discovery services have alerts that result in emails or feeds of resource descriptions to external users. Strategies: Implement alerts wherever needed to ensure that users are informed about changes to the content of collections or the state of business transactions or systems and services. For example, implement alerts to inform collection managers about file formats at risk of obsolescence using information from the AONS system as a trigger. Note that alerts can be used as a form of metadata syndication. The function is included under common services rather than metadata services because the alert mechanism may be used in a number of other business contexts. When used as a syndication mechanism, the strategy will be to move to feeds as the default alert mechanism but to continue to offer email and ftp to users who prefer these delivery methods. 12│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 2 Creation Services Creation services support the creation of content, its submission to a publishing system or repository, its publication, valuation and archiving. 2.1 CREATE E-framework: Add6, author activities? DLF: Not applicable? Definition: Process whereby a creator produces a manifestation of an expression of a work. The process may create a new work or a new expression of an existing work or a new manifestation of an existing expression. Create includes update because a significant act of update creates a new expression. Create may also be the outcome of a derivation or transformation workflow. Use cases: Manage content Data: Metadata (Rights) Registry (Rights holder) Business (Creation event) Protocols: File system, DocBook, TEI Systems: Text (Microsoft Word, Adobe In-Design, Adobe Contribute, Atlassian Confluence); audio and video (TBA) Strategies: The National Library of Australia creates print and online books, websites, wikis, pictures, oral histories, etcetera and also hosts an electronic journal publishing system using the Open Journal System (OJS) software. The Library is exploring multi-channel publication technologies for text works based on a master xml document using standards such as DocBook and TEI that are appropriate to the material. 6 E-framework has a generic Add service genre for CRUD activities within collections of like data that can be applied to the creation of content as well as metadata. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 13│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 2.2 SUBMIT E-framework: Under development, see wiki (requires password) DLF: Not applicable? E-framework: Under development, see wiki (requires password) Definition: Process whereby a creator submits an item for publication; or whereby a creator, publisher or distributor submits an item for deposit in a repository or archive. Use cases: Submit (, Submission response), SWORD Data: Content (Submission Information Package) Registry (Submit service protocol information, Access policy) Protocols: FTP, OAI-PMH, SWORD, METS, ORE, other packaging formats Systems: PKP Open Journals System (OJS), Digital Collections Manager, VTLS Vital Strategies: Development and maintenance of an Australian METS profile to support the creation of submission information packages. Development of content modelspecific sub-profiles in line with project priorities. Redevelopment of DCM to separate the submission process from the ingest process using METS as a Submission Information Package. 2.3 PUBLISH E-framework: No equivalent yet. DLF: Not applicable? Definition: Process whereby a creator, publisher or distributor makes an item publically available. Use cases: Publish (Published manifestation) Data: Metadata (Identifier) Protocols: NLA InfoURI scheme, Handle system Systems: PKP Open Journals System (OJS), NLA Website. Strategies: Ensure that all content published by the NLA is managed for persistence. Ensure that the Library’s InfoURI persistent identifier scheme can operate within a national persistent identifier infrastructure. Encourage publishers to manage their publications for persistence. 14│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 2.4 ANNOTATE E-framework: Annotate. DLF: No equivalent yet. Definition: Process that allows users to make comments, notes or other annotations to content, without altering the base item itself. Use cases: Manage annotation Data: Content (Annotation) Registry (User, Access Policy) Protocols: SRU Update, Annotea Systems: Vu Find. Strategies: Annotation is distinct from augmentation, where a metadata record is enhanced through processes such as tagging but in the user interface both functions may be supported together. Our strategy is to work with the NSLA (National and State Libraries of Australasia) to develop a service model and data schema for annotations that will enable data to be shared. 2.5 VALUE E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a creator, publisher or collection manager places a price or value on an item, either for the purposes of sale or as a selection aid or as part of valuing a collection. Note that this may be a service usage model, not a service: e.g., Commission Track Describe Use cases: Commission valuation / Track valuation / Describe value Data: Metadata (Value) Registry (Valuer) Protocols: The value will need to be recorded using an update protocol Systems: Spreadsheets Strategies: The National Library values its own publications in order to set a sale price and also commissions valuations of items being considered for the collection. Much of the commissioning work is done manually but there is a requirement to track the process and to record the outcome in the form of a purchase order or a decision not to proceed if the price is too high. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 15│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 3 Collection Services Collection services support the development and maintenance of a collection. The National Library of Australia has a requirement for systems to support these services in its role as a collection manager. In its role as an aggregator, it needs the National Library of Australia’s systems to support the contribution of metadata and content to the national data store and the sharing of data contributed by others (including end users) that will assist in the Library’s collection management processes and the discovery of the Library’s collections. Currently these services are provided by a combination of the Voyager ILMS, Digital Collection Manager and PANDAS. The Library should plan for the evolution of these services to support collection management workflows through a service-oriented approach. One possible strategy is to participate in the open source ILMS project which will be funded by the Mellon Foundation. The Library could contribute by building open source software to support the workflows that are particularly needed by national libraries. Some of these workflows relate to manuscripts and other unique materials. In addition, the Archivists Toolkit may support some workflow needs. 3.1 SELECT E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a collection manager selects collection material by registering pending items or contributors of items, tracking negotiation and/or evaluation processes and recording the final selection decision. Note that this is probably a service usage model, not a function: e.g., Describe [ Value] Track Approve Blanket orders are covered under Acquire. Use cases: Describe pending item [Value pending item] Track pending item Approve pending item Data: Metadata (Pending item) Registry (Donor, Vendor) Protocols: SRU Update Systems: Voyager ILMS, PANDAS Strategies: The selection workflow itself is very complex and may take place over a long time for original materials. Our understanding of it in a service-oriented architecture is still immature. Use of BPMN to document current and optimal 16│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 workflows will determine exactly what services are required to support the selection workflow and how they will interoperate with describe, audit, alert and track in a service usage model. 3.2 ACQUIRE E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a collection manager acquires collection material, with associated registry functions to manage providers. Use cases: Order (Acquisition request, Acquisition response) Data: Metadata (Content) Registry (Donor, Vendor) Business (Purchase order) Protocols: Edifact, ebxml, EDItEUR Systems: Voyager Strategies: Use modern mainstream protocols and investigate adapting an existing service developed for e-commerce application. Note that an order can be a blanket order. 3.3 INGEST (CONTENT) E-framework: Archive (Ingest)7 DLF: No equivalent yet. Definition: Process whereby a system processes content in order to bring it into a collection. Use cases: Ingest (Ingest request, Ingest response) Data: Content (Archival information package) Metadata (Identifier, Location) Registry (Repository) Business (Ingest event details) Protocols: METS, WARC Systems: DCM, PANDAS, VTLS Vital, OJS, Voyager Strategies: Migration of existing digital content management systems to a service-oriented architecture that receives content in a standard packaging format from a 7 Archive is an umbrella term. Ingest needs to be given its own service genre in the e-Framework. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 17│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 submission workflow system and processes it for ingest. Note that the ingest process may result in metadata being extracted from a Submission Information Package and ingested into a catalogue or index. Note that full-text indexes may also be generated from an ingest process for importing into a catalogue or index. 3.4 DESCRIBE E-framework: Add8. DLF: No equivalent yet. Definition: Process whereby a collection manager describes items in a collection or that are in the process of being selected and acquired. Use cases: Manage manifestation Manage item Manage authority Manage rights Data: Metadata (Resource, Topic) Registry (Repository, Catalogue or index) Protocols: Data (MARCXML, DCMI, MODS, EAD, EAC, RDF, SKOS) Update (SRU Update) Systems: Voyager ILMS, DCM, PANDAS Strategies: Single descriptive record, many uses. Move to RDA as content description standard. Support for multiple data schemas. Development of expertise in RDF and the Dublin Core Information Model so as to participate internationally in the testing of RDA and the development of a replacement to MARC for encoding RDA data elements. Note that the act of description will result in a request to update a catalogue or index and that this request may be passed to more than one catalogue or index in order to mirror content in local, regional and national aggregations. 8 Add is a generic service genre covering both content and metadata. Note that the e-framework Update use case does not yet support selective update but just replaces the whole record. 18│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 3.5 CONTROL E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby collection managers track where collection items are at a given time. This includes the circulation of items to internal and external users, as well as the movement of items for exhibition, preservation management or repository management purposes. Use cases: Manage item Data: Resource (Item) Registry (Location) Business (Movement event details) Protocols: SRU Update Z39-83-2002 Circulation Interchange Part 1: Protocol (NCIP) Z39-83-2002 Circulation Interchange Part 2: Protocol Implementation Profile 1 Systems: Voyager, Digital Collections Manager, PANDAS Strategies: Need to move to a single movement register for physical material; need to register and profile sub-repository locations so that capacity and occupation can be monitored. Note the distinction between sub-repositories where items are stored (LG2C DOSS) and collection sets (maps, pictures) that promote the discovery and use of items, as represented in figure 3: Figure 3 – Sub-repositories and collection sets www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 19│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Our repository contains resources of various types. These are described in our catalogue and located in a sub-repository. Views of our catalogue (collection sets) may be defined that only describe resources of a given type. It is possible that a sub-repository may only contain resources of a given type or that a collection set may only describe resources in a given sub-repository, but this is unlikely in most cases. The ‘Manage sub-repository’ use case needed to support the control function deals with the sub-repositories where collection items are stored. The ‘Manage collection set’ use case is treated in this framework as a metadata service needed to support the aggregate function. 3.6 PRESERVE E-framework: E-framework: Archive (Archival storage)9 DLF: No equivalent yet. Definition: Process whereby a collection manager manages content in a repository to ensure its preservation for future access. Note that this may be a service usage model, not a service: e.g., Commission Track Approve Use cases: Commision preservation action Track preservation event Approve preservation outcomes Data: Content (Submission information package, Dissemination information package) Registry (Agent) Business (Preservation event details) Protocols: METS, WARC Systems: Digital Collections Manager, PANDAS, Voyager, Audio system Strategies: Treat preservation as a continuum of processes applied to both physical and digital collection items. Record all preservation actions as events. Assume that all events logically have an input copy (submission information package) and an output copy (dissemination information package) that records event and agent details. Use the PREMIS event ontology as a starter list for event types: capture - the process whereby a repository actively obtains an object compression - the process of coding data to save storage space or transmission time 9 Archive is an umbrella term. Preserve needs to be given its own service genre in the e-Framework. 20│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 creation - the act of creating a new object deaccession - the process of removing an object from the inventory of a repository decompression - the process of reversing the effects of compression decryption - the process of converting encrypted data to plaintext deletion - the process of removing an object from repository storage digital signature validation - the process of determining that a decrypted digital signature matches an expected value dissemination - the process of retrieving an object from repository storage and making it available to users fixity check - the process of verifying that an object has not been changed in a given period ingestion - the process of adding objects to a preservation repository message digest calculation - the process by which a message digest ("hash") is created migration - a transformation of an object creating a version in a more contemporary format normalization - a transformation of an object creating a version more conducive to preservation replication - the process of creating a copy of an object that is, bit-wise, identical to the original validation - the process of comparing an object with a standard and noting compliance or exceptions virus check - the process of scanning a file for malicious programs www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 21│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 4 Metadata Services Metadata services support the aggregation of metadata into a repository, the delivery of metadata products to users and the sharing of metadata with other aggregators. 4.1 AGGREGATE E-framework: Add, Harvest10, Authenticated harvest DLF: Data aggregation / HarvestBibliographicRecords; HarvestAuthorityRecords; HarvestExpandedRecords; HarvestHoldingsRecords Definition: Process that supports the collection of metadata into a catalogue or index or authority file through a push or pull process, with associated registry functions to manage contributors and collection views. Note that the push process may be fully represented by Describe and may not need to be included here. Use cases: Push (Update request, Update response) Pull (Harvest trigger, Harvest response) Data: Registry (contributing repository) Business (Harvest details) Protocols: Data (MARCXML, DCMI, MODS, EAD, EAC, RDF, SKOS) Push protocol (SRU update) Pull protocol (OAI-PMH, FTP) Strategies: Our principal strategy for the push process is to use SRU update to accept requests from local collection management systems as a by-product of describing content or from ingest systems as a by-product of unpacking a submission information package. We also envisage using Web 2.0 technologies to develop targeted web-based workflow systems to support collaborative development of the national metadata store, with update of local systems through SRU Update or a syndication process. Our principal strategy for the pull process is the Harvest Task which has been scheduled within the People Australia Project. 10 The E-framework Harvest service genre includes the metadata source function Expose which this framework includes under Metadata Services since different organizations and systems are involved. 22│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 4.2 INGEST (METADATA) E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a system ingests metadata into a catalogue or index or authority file with associated checks for duplicates. Use cases: Convert (input copy, output copy) Match (input copy, match algorithm) Merge (input copy, merge algorithm) Resolve (input copy, potential matches, match decision) Index (input copy, index update details) Data: Metadata (Resource, Topic, Index) Registry (Collection set) Business (Conversion rules, Indexing rules, Matching algorithm, Merging algorithm) Protocols: FRBR, FRAD, FRAR Systems: CBS Record Import Service, ARROW Record Import Service. Strategies: Note that convert includes augmentation of records to support discovery at the conversion stage. Our strategy is to provide support for multiple input and output formats but with priority given to MARCXML, MODS, (MADS?), DCMI, EAD and EAC. We have tested and decided not to map to a single internal storage format but have still not settled on exactly how data for different resource types will be stored internally. One strategy will be to build up expertise in RDF and to be actively involved in processes to map RDA data elements to the DCMI Information Model. Currently we apply match and merge algorithms to incoming data to create a single merged record. Match and merge rules are similar to those we would apply to FRBRise result sets. One strategy under consideration is to keep all input copies and to match and merge at the rendering stage. 4.3 AUGMENT E-framework: Classify. DLF: No equivalent yet. Definition: Process whereby users enhance contributed metadata through the addition of rankings, ratings, classifications or recommendations. Use cases: Update (Update request, Update response) Data: Metadata (Tag) Registry (Access policy) www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 23│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Protocols: SRU Update Strategies: Our strategy is to provide the capability to tag metadata in our own discovery services and also to pull tags from other services such as Library Thing or del.icio.us. Note that the distinction being made here between Augment and Annotate is that Augment changes the metadata whereas Annotate creates a new ‘work’ with its own author and access policies that provides commentary on the annotated work. Augment is a term used by the Repository Federation Service Usage Model (FRED) but they include annotation as an augmentation type. This model assumes that annotation is annotating the work while augmentation is augmenting the metadata. 4.4 EXPOSE E-framework: Harvest11 DLF: No equivalent yet. Definition: Process whereby a system exposes metadata for harvesting by other agencies. Use cases: Expose (Harvest request, Harvest response) Data: Metadata (Resource, Topic) Registry (Collection set) Protocols: OAI-PMH Strategies: An OAI exposure service is due to be developed within the People Australia Project by March 2008. Note that metadata may also be shared through search and retrieval protocols. 4.5 PACKAGE E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a system packages metadata in response to a one-off or standing order, based on a range of specified criteria. Use cases: Package (Package Request, Package Response) Data: Metadata (Resource, Topic) Registry (Product, Charge) Business (User, Order details) 11 The E-framework Harvest service genre includes the metadata source function Expose which this framework includes as Syndicate under Metadata Services since different organizations and systems are involved. 24│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 Protocols: RSS, ATOM, email, ftp Strategies: This service is meant to represent the Libraries Australia Products service and is not yet very fully developed. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 25│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 5 Discovery Services Discovery services support the finding of wanted resources and the transfer of requests for access or use to the resource provider. 5.1 SEARCH E-framework: Search DLF: Core functions (Real Time Search/Search; GetRecords, getAuthorityRecords, OPAC Embed and Escape; SearchCourseReserves) Registry (Explain) Definition: Process whereby users find resources or topics by searching for items that match the user’s input criteria. The system returns results in ways that assist the user to further refine the search if necessary or to navigate to more information. Users of this service include third party systems. This service may also be used by a user to search within a found resource or topic. Use cases: Select target (Target selection list, Selected targets) Search selected targets (Search request; Syntax rules; Search result) Data: Metadata (Resource, Topic, Index) Registry (Collection, Collection service) Protocols: SRU, OpenSearch, Z39.50 Systems: Z39.50-SRU Gateway, Teratext, Voyager. Strategies: Collection views, spell checking, topic-based approach to searching, faceted clustering, FRBRised result sets. 5.2 SCAN E-framework: No equivalent yet. DLF: Real Time Search/Scan Definition: Process whereby a user finds resources or topics by scanning an index, with the capability to specify a start place in the index. Use cases: Select index (Index selection list, Selected index) Scan index (Browse start request; Syntax rules; Browse result) Data: Metadata (Resource, Topic, Index) Registry (Collection, Collection service) 26│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 Protocols: SRU, Z39.50 Systems: Z39.50-SRU Gateway, Teratext, Voyager. Strategies: Scan in this model applies to the function called Scan in Z39.50 and SRU that allows a user to step through an index or to be placed at a point in the sequence where the term they have input would appear (whether or not it does). Note that new generation catalogue products such as Vu Find have focused on relevance ranking and faceted clustering of result sets as ways of enhancing the user experience. One of our challenges will be to identify usage scenarios that confirm the ongoing need for the scan function, as SOLR does not natively support it. 5.3 LOCATE E-framework: No equivalent yet. DLF: Real Time Search/GetRecords; GetAvailability Definition: Process whereby a user can view current availability information, detailed holdings information or usage information that may only be available from a local system. Use cases: Get holdings (Holdings request, Holdings response) Data: Metadata (Item) Registry (Collection service ) Protocols: SRU, Z39.50, XML Holdings schema Systems: Voyager, Vu Find, Libraries Australia. Strategies: The Voyager OPAC uses proprietary calls to get this information from the database. The new Vu Find catalogue also has a proprietary relationship with the Voyager database to get this information. Libraries Australia is limited at the moment to deep linking to the local catalogue. Our strategy is to use the XML Holdings Schema over SRU to get this information from local catalogues of all kinds and to present it in a seamless manner to users through our discovery interfaces. This will require ILMS vendors as well as our own systems to support the XML Holdings Schema. ILMS vendors have been slow to make such enhancements so this may be something that we pursue with Ex Libris in a collegiate way; or that we contribute to the development of an open source ILMS. Having a demonstrator that uses the XML Holdings Schema as an interface object is the best way to take this forward, even if outputs from the Voyager database have to be mapped to the schema in the first instance. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 27│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 5.4 BOOKMARK E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a user is able to save searches and metadata between sessions. Use cases: Bookmark (bookmark request, bookmark response) Manage bookmark collection Data: Metadata (Identifier) Registry (Access policy, Collection) Business (User) Protocols: CoINS, SRUUpdate Strategies: This service goes beyond the current Save search and Save record Use cases in Libraries Australia or the trails in Music Australia and Picture Australia to allow users individually or collaboratively to construct their own collections, including user developed trails and subject guides. Our preferred strategy is to identify an existing toolset to support this workflow [is Heurist relevant here?] 5.5 REQUEST E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process that enables a user to lodge a request for a wanted resource with a resource provider, taking into account availability, access and use policies and any conditions and obligations these impose on the requester. Use cases: Request (Resource request, Resource request response) Note that the request use case includes an alternate course when the item is not currently available that allows the user to reserve the item. Note that, when rendering wanted resource details, the system may use an algorithm to calculate current copyright status and this may then determine the options presented to the user as part of the request workflow. Data: Metadata (Resource details, Copyright status) Registry (Access policy) Business (User, Request) Protocols: OpenURL Request transfer message Strategies: The Voyager OPAC has a proprietary relationship with the circulation system. The new Vu Find catalogue also has a proprietary relationship with the Voyager circulation system. Libraries Australia provides deep links to the local 28│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 catalogue where users can place their requests as well as a web form that uses a non-standard schema to exchange requests with local systems. Our strategy is to use the OpenURL Request transfer message to interface with the circulation systems or interlibrary loan systems of libraries in order to enable seamless requesting through our discovery interfaces. This will require ILMS and ILL system vendors as well as our own systems to support the OpenURL Request Transfer message. As with the XML Holdings Schema, ILMS vendors have been slow to make such enhancements so this may be something that we pursue with Ex Libris in a collegiate way; or that we contribute to the development of an open source ILMS. Having a demonstrator that uses the OpenURL Request Transfer Message as an interface object is the best way to take this forward, even if interfacing with Voyager or Relais is done through a kluge in the first instance. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 29│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 6 Delivery Services Delivery services – Resolve, Disseminate, Reproduce, Lend and Hold - support the delivery of wanted resources, either by resolving directly to the resource once conditions have been met for access, by supplying or lending a copy or by reserving a copy if it is currently in use. Note the primacy of the Resolve and Disseminate functions in delivery workflows. A delivery system will use a URI to resolve to a METS Dissemination Information Package and then unpack it using the descriptive metadata, structural maps and file group use attributes to generate a navigation workflow system with behaviours appropriate to the content model. Derivatives needed to deliver the required behaviours (e.g., zoom for still images) will be invoked through the Resolve service, as will related objects defined in the descriptive metadata or through the structural map. 6.1 RESOLVE E-framework: Resolve Identifiers. DLF: No equivalent yet. Definition: Process whereby a URI resolves to a service that provides access to an appropriate copy of a wanted resource once conditions have been met. Use cases: Resolve (Resolve request, Resolve response) Data: Metadata (Identifier) Registry (Access policy) Business (User) Protocols: Handles, OpenURL, InfoURI Systems: NLA Resolver Service Strategies: Our bid to operate a national persistent identifier resolution service; our need to ensure that our own InfoURI identifiers can interoperate in this context. Enhancement of the NLA Resolver Service to utilise rights management information including current copyright status to simplify the resolution process for users and to support a more automated process for getting permissions when these are required. 30│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 6.2 DISSEMINATE E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a repository supplies a digital copy of a resource to another system in response to a trigger from a resolution service. Use cases: Disseminate (Dissemination request, Dissemination response) Data: Content (Dissemination Information Package) Registry (Delivery Service/Disseminate function) Protocols: METS, OpenURL Systems: DCM, Newspapers, etc. Strategies: Our work on the Australian METS Profile. Evolution of our generic delivery system to operate based on a service-oriented approach using METS as an interface object. 6.3 REPRODUCE E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a resource provider supplies a digital or physical reproduction of a resource to a requester once conditions have been met. Note that the workflow may be interrupted due to the need for the requester to get permission from the rights holder. Use cases: Manage request Data: Metadata (Resource details, Copyright status) Registry (Access policy) Business (User, Request, Permission) Protocols: ISO ILL Protocol, OpenURL Request transfer message Systems: Libraries Australia Document Delivery, Relais. Strategies: Single business approach to the supply of items from our collections; enhancement of Copies Direct to utilise rights management information including current copyright status to simplify the requesting process for users and support a more automated process for getting permissions when these are required. Digitisation on demand. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 31│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 6.4 LEND E-framework: No equivalent yet. DLF: Patron functionality/RenewLoan; RecallItem; CancelRecall Definition: Process that lends an item to a requester once conditions have been met and that manages the loan until the item has been returned. Note that the workflow may be interrupted due to the need for the requester to get permission from the rights holder. Note that Reproduce, Lend and Hold could be seen as alternate courses in a more generic function such as Supply. Use cases: Manage request Data: Metadata (Resource details, Copyright status) Registry (Access policy) Business (User, Request, Permission) Protocols: ISO ILL Protocol, OpenURL Request transfer message Systems: Libraries Australia Document Delivery, Relais. Strategies: As for get/supply, with digital lending and further exploration of the netbooks idea. 6.5 HOLD E-framework: No equivalent yet. DLF: Patron functionality/HoldTitle; HoldItem; CancelHold Definition: Process that reserves an item for a requester if it is not currently available for some reason. Note that the workflow may be interrupted due to the need for the requester to get permission from the rights holder. Note that Reproduce, Lend and Hold could be seen as alternate courses in a more generic function such as Supply. Use cases: Manage request Data: Metadata (Resource details, Copyright status) Registry (Access policy) Business (User, Request, Permission) 32│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 Protocols: ISO ILL Protocol, OpenURL Request transfer message Systems: Libraries Australia Document Delivery, Relais, Voyager Strategies: See above. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 33│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 7 User Services User services deal with the relationship of the user with the service - enabling the user to register for value-added services, to engage in a dialogue with the service provider in order to get help or provide feedback, to set preferences for their interaction with the service and to monitor their own usage. Track and report also allows the service provider to monitor the status of transactions and usage across all users and functions. 7.1 REGISTER E-framework: No equivalent yet. DLF: Registry (Patron functionality/LookupPatron; GetPatronInfo; GetPatronStatus) Definition: Process whereby a user registers for a service. Note that this is probably a workflow, not a function. It will be implemented as a version of the registry Manage User function. Use cases: Register (Registration request, Registration response) Data: Business (User) Protocols: SRU Update, X.500, LDAP Strategies: Register once to access multiple services. 7.2 PERSONALISE E E-framework: No equivalent yet. DLF: No equivalent yet Definition: Process that enables a user to customise their experience with a service and its look and feel. Note that this is probably a workflow, not a function. It will be implemented as a version of the registry Manage User profile function. Use cases: Manage personalisation option Data: Business (User profile) Protocols: SRU Update, X.500, LDAP Strategies: Users able to define there own default collections, etc. 34│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 7.3 ASK E-framework: No equivalent yet. DLF: No equivalent yet Definition: Process that enables a user to ask reference questions of a service provider Use cases: Ask (Reference request, Reference response) Data: Business (User, Reference request) Protocols: SMS, Chat, Email Systems: Reference Enquiry System; Ask Now. Strategies: Be where the users are. Collaborate nationally and internationally. 7.4 COMMISSION E-framework: No equivalent yet. DLF: No equivalent yet. Definition: Process whereby a manager commissions a task such as the creation of a work or the valuation of an item from a third party. Use cases: Commission (Commission request, Commission response) Data: Registry (Provider) Business (Task details) Protocols: SRU Update Systems: People’s heads, manual records. Strategies: BPMN analysis of workflows as an extension to the Select workflow. 7.5 TRACK E E-framework: No equivalent yet. DLF: No equivalent yet Definition: Process that monitors the status of a task and allows a task to be advanced to a new status when approvals are required. Use cases: Track (Status request, Status report) Approve (Approval request, Approval response) Note that Approve may end up being a separate function. Data: Business (Status) www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 35│37 12 February 2016 Strategy Document Digital Library Service Framework V 0.7 Protocols: None yet identified Systems: Strategies: None yet articulated. 7.6 REPORT E E-framework: No equivalent yet. DLF: No equivalent yet Definition: Process that enables users to get reports Use cases: Report (Report request, Report response) Data: Business (Report details) Protocols: Standardized Usage Statistics Harvesting Initiative (Sushi) Strategies: 7.7 COMPLY E-framework: E E-framework: Comply DLF: No equivalent yet Definition: process where a user acknowledges acceptance of a policy in order to complete a workflow. Use cases: Get compliance (Compliance request, Compliance response) Data: Registry (Access policy, Obligation) Business (User) Protocols: XACML Systems: Copies Direct. Strategies: To automate the compliance process as much as possible by documenting policies in machine-readable form. The Rights Management System Project will enable this. 36│37 12 February 2016 www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia Strategy Document Digital Library Service Framework V 0.7 8 Workflows and service usage models This document focuses on functions needing to be implemented in order to build workflow systems to support delivery of the Library’s digital library services. Business process workflows will involve a series of functions and stakeholders. For example, the term ‘Archive’ covers the submission and ingest process and may also cover preservation management actions. The e-framework uses Service Usage Models (SUMs) to provide stakeholders with another way of visualising requirements and testing that needs are being met. Below is a hastily-put together SUM for archiving which is really just a starting point. It would be useful for the Library to get some practice in the development of SUMs, perhaps through a workshop with the e-Framework, and also to understand the relationship between SUMs and Business Process Modelling Notation (BPMN) diagrams and narratives as ways of modelling requirements. www.nla.gov.au Creative Commons Attribution-NonCommercial-ShareAlike 2.1 Australia 37│37 12 February 2016