WebDAV and DeltaV: Collaborative Authoring, Versioning, and Configuration Management for the Web Jim Whitehead University of California, Santa Cruz ejw@soe.ucsc.edu 1 WebDAV and DeltaV WebDAV and DeltaV are: Application layer network protocols Extensions to the Hypertext Transfer Protocol (HTTP 1.1) Developed by the Internet Engineering Task Force (IETF) WebDAV Remote collaborative authoring of Web resources Overwrite prevention, metadata management, namespace operations DeltaV Versioning and Configuration Management Extends base WebDAV capabilities Checkout/checkin, version history, logical change tracking, workspaces, configuration management 2 Major WebDAV Clients Application Software: Microsoft: Office 2000/XP (Word, Excel, PowerPoint, Publisher) Adobe: Photoshop 6, Acrobat 5 Web Site Authoring Adobe: Go Live 5 Macromedia: Dreamweaver 4 Remote File Access: Apple: Mac OS X webdavfs OS X also ships with Apache and mod_dav (have to configure mod_dav to make it work) Microsoft: Windows Web Folders Wind River Software: WebDrive Goliath (Mac, open source) Nautilus (GNOME project, Eazel) WebDAV Explorer (UC Irvine, Feise/Kanomata, open source) XML editors Excosoft: Documentor Altova: XML Spy 3.5 3 Major WebDAV Servers Microsoft: IIS 5/6, Exchange 2000, Sharepoint Apache: mod_dav (over 10,000 sites) Oracle: Internet File System Adobe: InScope Xythos: Web File Server Novell: Netware 5.1, Net Publisher W3C: Jigsaw Endeavors: Magi-DAV IBM: DAV4J (DeveloperWorks) DataChannel: DataChannel Server (DCS 4.1) Intraspect: 4i OpenLink: Virtuoso Hyperwave: Information Server 5.5 4 Using WebDAV You have to produce an XML document Fire up your favorite XML processor & start editing You decide to bring on another author Using the same XML processor, save to the Web Give your collaborator the URL of the XML document Start collaboration on the document by editing in-place on the Web A seamless transition from individual to collaborative work 5 Application Use of WebDAV Client File… Open LOCK PROPFIND GET Edit File… Save Exit PUT UNLOCK Server Lock resource Send properties to client Send resource contents to client Save new contents Unlock resource 6 Collaborative Document Authoring Three collaborators, in different cities, use Word 2000 to collaborate on a report they are producing together. BOSTON LOS ANGELES TOKYO Katrina Johnathan Kenji Word 2000 Word 2000 Word 2000 HTTP/DAV HTTP/DAV WebDAV server shared document HTTP/DAV A 7 Collaborative Web Site Authoring Two homes each develop their family Web site using WebDAV to interact with their ISP. HOME 1 INTERNET SERVICE PROVIDER Joe Homeuser Go Live 5 Go Live 5 Joe’s Daughter WebDAV server Jane McPC HTTP/DAV HTTP/DAV /joe /html /images HOME 2 Dream Weaver 4 /jane /html /images Jane’s web site Web site for Joe and his daughter 8 Remote Authoring, Part of Staged Production CHICAGO SEOUL Cindy Yung Kim Acrobat 5 Go Live 5 FRANKFURT Hartmut Photo shop 6 Frank HTTP/DAV Excel 2000 HTTP/DAV General Internet Readers HTTP server production server HTTP/DAV WebDAV server contributions server web site contents /contrib project Web site /chicago /seoul firewall /frankfrt HTTP server Web site development team incorporates contributions submitted to the contributions server. development server web site contents 9 Visions for WebDAV Participants in WebDAV have many views on what it is: A protocol for collaborative authoring of all document types XML, HTML, word processing, spreadsheets, A Web-based network file system, with nice high-latency behavior Better performance than NFS and Samba over the Internet A data integration technology for accessing a wide range of repositories Document mgmt. systems, configuration mgmt. systems, filesystems, etc. Remote software engineering infrastructure A replacement protocol that can handle email, calendaring, directory lookup and more Could replace: POP, IMAP, CAP, LDAP… All views are correct! 10 Filesystem View Exemplars: Web Folders, WebDrive, WebIFS, TeamDrive, Mac OS X 11 Document Authoring Exemplars: Office 2000/XP: Word, Excel, PowerPoint, as well as Photoshop, Documentor, and XML Spy Office: uses filesystem metaphor for WebDAV location 12 Photoshop Workflow metaphor for WebDAV location 13 Web Site Authoring Exemplars: Go Live 5, Macromedia Director Site metaphor for WebDAV location 14 Remote Collaborative Annotation Acrobat 5 views a WebDAV location as a storage location for document annotations Annotations are stored in resources separate from the PDF document One collection per document One annotation resource per user (in collection) 15 Email Access via WebDAV Outlook Web Access for Exchange 2000 uses WebDAV plus some non-standard “batch” methods, for efficiency Client is DHTML running in the browser 16 Facets of WEBDAV There are many ways to view the DAV work: Collaboration infrastructure Metadata repository infrastructure Namespace management infrastructure Access control infrastructure Searching infrastructure – DASL 17 Collaboration Infrastructure Whole resource locking supports: remote collaborative authoring of any media type Web pages, Word processing, Presentations, XML, … Lock characteristics: Long-duration locks Not associated with network connection Client receives a lock token identifying the lock Can disconnect from network after receiving lock token Locks automatically expire after a client specified timeout period Lock single resources, or hierarchies of resources Infrastructure for asynchronous, widely distributed, hypertext-aware, collaborative editing tools. 18 Metadata Recording Infrastructure Metadata support Properties are (name, value) pairs that can be created, modified, deleted, and read on Web resources. Consistency of properties can be maintained by the server or the client Property values are well-formed Extensible Markup Language (XML) Can store RDF as well Property name is a URI or URL Extensible, global property namespace Infrastructure for recording information about Web data A general purpose metadata repository 19 Namespace Management Infrastructure Remote name space management: Copy and Move individual resources hierarchies of resources to/from a locked hierarchy Create and modify collections of resources Retrieve listings of collection members Useful for creating Save…As dialog boxes Adds hierarchical navigation to the Web Augments hypertext navigation and searching Infrastructure for remotely organizing and viewing collections of Web resources 20 WebDAV Methods Overwrite Prevention: LOCK – prevents non-lock holders from writing to the resource UNLOCK – removes a lock Metadata Management: PROPFIND – read properties from a resource Allprop – all property names and values Propname – only return property names Prop – just return specified properties PROPPATCH – write properties on a resource Namespace Management COPY – duplicate a resource MOVE – move a resource (preserving identity) MKCOL – create a new collection 24 Scope of WebDAV Methods Web Resource LOCK UNLOCK COPY MOVE† DELETE† MKCOL† (PUT†) Properties (name, value) pairs PROPFIND Body (primary state) GET PROPPATCH† PUT† † - affected by LOCK 25 DeltaV: Versioning and Configuration Management 26 Remote Collaborative Document Authoring Scenarios A task force of people from geographically dispersed business units need to develop a report together. Team needs to solicit feedback, so they send out copies of the report. As a result, the team needs a permanent copy of the exact report version they are having other people review. BOSTON LONDON TOKYO Rebecca Johnathan Kenji Word 2000 Word 2000 Word 2000 DAV/DeltaV DAV/DeltaV DAV/DeltaV DAV/DeltaV server auto-versioning enabled shared document A A v1 v2 A v3 27 What is DeltaV? DeltaV is: An application layer network protocol Extends the WebDAV protocol WebDAV itself extends HTTP 1.1 Thus DeltaV is also an extension to HTTP 1.1 HTTP is the core network protocol of the Web An interoperability standard Allows document authoring and software development applications to interoperate with versioning and configuration management systems using a common interface A data model DeltaV embodies a data model that is capable of modeling multiple versioning and configuration management repositories The data model is independent of network protocol A data integration infrastructure A common place for cross-application data sharing 28 Version History URL path of Versioned Controlled Resource Foo.htm initial 1 Beta1 Branch Version Name Label Successor Test1 2 3 4 5 6 Line of Descent Revision History Merge Beta2 7 Predecessor 29 Representation of a Version History Abstract version history of foo.html Representation of foo.html in DeltaV version history resource v1 v2 version-controlled resource DAV:version-set /his/73/ver/1 /his/73/ver/2 /his/73/ver/3 DAV:checked-out /his/73/ver/3 DAV:version-history /his/73/vhr DAV:root-version /his/73/ver/1 body of version v3 /his/73/vhr v3 /foo.html v1 v2 v3 /his/73/ver/1 /his/73/ver/2 /his/73/ver/3 version resources 30 Unique Aspects of Versioning Data Model DeltaV data model is different from RCS/SCCS model RCS: placing foo.html under version control causes: foo.html to be made read-only Creation of RCS/foo.html,v archive file Archive contains version history and version data Differences Both use the original name as the place where commands are directed But, version-controlled resource has more metadata DeltaV separates version history from version storage DeltaV gives each version a unique identifier (URL) Note that delta storage can still be used in the underlying repository 31 Rationale for Versioning Data Model Each version is given its own URL, thus allowing hypertext linking to a specific version. Hence, each version must be a separate resource Workspaces are the motivation for separating the version-controlled resource from the history resource. A workspace allows a collaborator to work in isolation, in their own part of the namespace. For example, Greg and Lisa might have workspaces: /ws/users/greg/ and /ws/users/lisa/ If both want foo.html to appear in their local workspace, each needs a version controlled resource. Thus, one version history can be associated with multiple version controlled resources. 32 Autoversioning There are many WebDAV-aware applications They know nothing about versioning! Would like to provide automatic versioning support Two styles of autoversioning: 1. Every modification creates a new version PUT/PROPPATCH CHECKOUT PUT/PROPPATCH CHECKIN Works best for authoring clients that replicate resources to a local disk, and only write to the server at the end of an editing session. 2. Every LOCK/UNLOCK pair creates a new version LOCK LOCK CHECKOUT UNLOCK CHECKIN UNLOCK Works well for authoring clients that work directly on the WebDAV server and take out locks, like Office 2000. Style is controlled by DAV:autocheckout and DAV:autocheckin properties on version controlled resource 33 Activities Frequently, a logical change spans multiple versions in multiple version histories For example, fixing a specific software bug can involve the creation of multiple versions of several files Activities associate a logical change with the set of versions created while making that change. Versions from a given version history are limited to a single line of descent Activities are not versioned Sometimes want to ensure that all work done by team members is on a single line of descent Avoids merging between team members Activities can be used to enforce working on just a single line of descent 34 Workspaces A workspace is a location… … where a person can work in isolation … from ongoing changes made by other collaborators to the same set of resources. Workspace use scenarios: Two people each modify a common source code file, and want to test their work before integrating changes The directory structure of a large project is being changed Don’t want to require everyone to stop working while this takes place Keep namespace changes invisible to team until complete A developer is trying a change that might not pan out New files are in their workspace, but not under version control until the final viability of the change is known 35 Server Workspace Example Version history of main.c: version controlled resources version history resource /projectX/main.c /users/chris/ projectX/main.c /users/geoff/ projectX/main.c /repo/o522/v1 /repo/o522/v2 /his/o522 /repo/o522/v3 /repo/o522/v4 1 2 3 4 5 /repo/o522/v1 /repo/o522/v2 /repo/o522/v3 /repo/o522/v4 /repo/o522/v5 version resources 36 Server Workspace Example main.c makefile defs.h 1 1 1 2 2 2 3 3 3 4 /users/geoff/projectX/ main.c, 5 makefile, 3 defs.h, 2 /users/chris/projectX/ main.c, 4 makefile, 3 defs.h, 3 5 37 Baselines When a large collection of documents is released outside the developing organization, there is a need to record exactly which document versions were released. Especially true for software project releases A baseline can record the exact version of a large set of resources under version control Baselines can also be used for recording intermediate project states The state of the project as of last Friday Quality assurance can then have a stable snapshot to work from Baselines are used to perform configuration management 38 WebDAV Projects at UC Santa Cruz 39 DAV Calendaring and Scheduling Problems: iCal data format is complex and difficult to understand Few network effects with other data formats Compare with XML-based formats A separate protocol is under development for remote collaborative calendaring and scheduling Requirements are not significantly different from WebDAV Approach: Translate iCal data format to Resource Description Framework (RDF) Develop profile for using WebDAV to remotely author calendar event resources Prototype ideas using Gnome Calendar application Work performed by: Michael Arick, UCSC <marick@soe.ucsc.edu> Libby Miller, ILRT <libby.miller@bristol.ac.uk> (RDF work) http://www.cse.ucsc.edu/~marick/ 40 SOAP over DAV SOAP: Simple Object Access Protocol An XML-based remote procedure call protocol Typically marshalled as an XML body to an HTTP POST request Problem: SOAP assumes the response will come back quickly (1-2 minutes at most) Not suitable for long-duration transactions No versioning of interfaces Approach: Marshall SOAP messages using PROPPATCH/PROPFIND Submit request write to a property value Receive response read a property value Can leverage versioning, access control, metadata, searching facilities of WebDAV Work performed by Elias Sinderson <elias@cse.ucsc.edu> 41 Electronic Records Management US Government agencies are required to support standard DoD 5015.2 Describes metadata items useful for archiving electronic records 5015.2 is not an interoperability standard Approach Develop an XML representation of 5015.2 metadata items Store 5015.2 metadata in WebDAV properties Develop server support for automatic archiving/disposal of records automatic setting of 5015.2 property values Develop client support for entry/viewing/searching 5015.2 items Payoff Wide deployment of interoperable, low-cost infrastructure for archiving electronic records Work performed by Lan Xue <lanxue@soe.ucsc.edu> 42 Study Wish List How are people actually using WebDAV technology? Many stories of people using it for file sharing Less evidence of collaborative authoring use In particular, Go Live and Director are designed for group use How are people using the collaboration facilities? WebDAV has relatively poor awareness support –does this affect use? How is the technology used differently in different organizational settings? We finally have a remote collaboration technology that is starting to be broadly used We can now test many long-held assumptions about this technology 43 Project Wish List DAVCam A digital camera that takes a picture… Automatically writes the picture to a DAV store Allows voice-to-text annotation of photo with metadata Open Hypertext integration Export hypertext links as Xlinks in DAV properties Allow authoring of links via these properties Mapping to OHP Storage interface DAV as storage layer for spatial hypertext Is DAV sufficient by itself? Or are additional capabilities or optimizations needed? DAV-based Weblog (“blog”) system Author content via DAV 44 WebDAV Resources 45 WebDAV WebDAV Resources http://www.webdav.org/ A central collection of pages and links to all things WebDAV. WebDAV Working Group http://www.ics.uci.edu/pub/ietf/webdav/ Contains links to active documents, and a complete list of WebDAVsupporting applications. RFC 2518 – WebDAV Distributed Authoring Protocol http://www.ics.uci.edu/pub/ietf/webdav/protocol/rfc2518.pdf This is the WebDAV Distributed Authoring Protocol specification WebDAV: A network protocol for remote collaborative authoring on the Web Proc. of the Sixth European Conference on Computer-Supported Cooperative Work, Sept. 12-16, 1999, Copenhagen, Denmark, pp. 291-310. http://www.ics.uci.edu/~ejw/papers/dav-ecscw.pdf An academic paper giving an overview of the WebDAV Distributed Authoring Protocol. 46 DeltaV Delta-V Working Group web page http://www.webdav.org/deltav/ The home page for the IETF Delta-V Working Group, with links off to the most recent specifications. G. Clemm, J. Amsden, C. Kaler, J. Whitehead, “Versioning Extensions to WebDAV”, Internet-Draft, work-in-progress, draft-ietf-deltav-versioning-16, July 3, 2001. http://www.webdav.org/deltav/protocol/draft-ietf-deltav-versioning16.htm The most recent revision of the versioning and configuration management protocol specification. The Future of Distributed Software Development on the Internet Web Techniques, Vol. 4, No. 10, October, 1999, pages 57-63 http://www.webtechniques.com/archives/1999/10/whitehead/ An introduction to WebDAV and DeltaV that describes the advantages of DeltaV over CVS for remote collaborative software development 47 DASL: DAV Searching & Locating DAV Searching and Locating page http://www.webdav.org/dasl/ A web site containing links to the most recent WebDAV searching protocol specifications. A. Babich, J. Davis, R. Henderson, D. Lowry, Sa. Reddy, Su. Reddy, “DAV Searching and Locating”, work-in-progress. http://www.webdav.org/dasl/protocol/draft-davis-daslprotocol-00.html The most recently edited DASL protocol specification. This document was never submitted as an Internet-Draft. Will be submitted for consideration as an Experimental RFC in the near future. In 1-2 years, will be submitted as a Proposed Standard 48 Access Control Protocol Access Control page http://www.webdav.org/acl/ A web site containing links to current access control protocol specifications. WebDAV Access Control Protocol draft-ietf-webdav-acl-06, June 21, 2001. http://www.webdav.org/acl/protocol/draft-ietf-webdav-acl06.htm The most recent revision of the access control protocol specification. 49 Advanced Collections WebDAV Bindings draft-ietf-webdav-binding-protocol-02, December 17, 1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietfwebdav-binding-protocol-02.txt The most recent revision of the WebDAV Bindings Protocol. WebDAV Redirect Reference Resources draft-ietf-webdav-redirectref-protocol-02, December 17, 1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietfwebdav-redirectref-protocol-02.txt The most recent revision of the Redirect Resources protocol. WebDAV Ordered Collections Protocol draft-ietf-webdav-ordering-protocol-02, December 20, 1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietfwebdav-ordering-protocol-02.txt The most recent revision of the Ordered Collections protocol. 50 Document Roadmap WebDAV Working Group: Distributed Authoring Locking, Properties, Copy/Move RFC 2518 complete Access Control Protocol for remote access control Finish: Fall 2001 Advanced Collections Requirements and protocol for bindings, redirectors, ordered coll. Finish: December 2001 DASL: Searching Requirements and protocol for searching a WebDAV repository Finish: mid 2002 Delta-V Working Group: Versioning & CM Protocol just sent to IESG Finish: Fall 2001 51 52