From Theory to Practice in Digital Libraries: 5S and Educational Applications (NDLTD, CSTC) Edward A. Fox Department of Computer Science Virginia Tech, Blacksburg, VA 24061 USA fox@vt.edu http://fox.cs.vt.edu Abstract The field of digital libraries is becoming ever more important in computer science, since it presents a unique mixture of theoretical, systems, and application challenges. At the same time, it affords exciting opportunities to enhance education, by extending and improving access to new knowledge, especially in such areas as science and technology. To address these challenges and opportunities, we outline the 5S Framework (Societies, Scenarios, Spaces, Structures, and Streams) as a foundation for digital libraries. We show how it aids in the practice of applying digital libraries to education, through two case studies: the Networked Digital Library of Theses and Dissertations (NDLTD) and the Computer Science Teaching Center (CSTC). We argue that 5S, NDLTD, and CSTC can provide a basis for international collaboration on digital libraries, leading to a Networked University Digital Library (NUDL) and other initiatives, like Pronabid in Mexico. Keywords: 5S, computer science education, digital libraries, dissertations, graduate education, Pronabid, research, scenarios, societies, spaces, streams, structures, theses 1. Importance of the Field of Digital Libraries By 1994, the phrase “digital library” (DL) became the generally approved label for a field previously referred to by terms like “electronic library” [1]. DL deals with automated access to information, as called for in writings dating back to the 1960s [2] and in visionary descriptions from the 1940s [3]. Today there are books [4], international conferences [5], and online courseware [6] on the topic. The field goes far beyond supporting traditional libraries, addressing core requirements of the Information Age and the world of Information Technology, as well as new sub-areas like knowledge management and content management. DL also is an important field because it places demands on almost every other computing discipline. To build worldwide DLs [7], there must be efficient systems, able to handle large numbers of users and high volumes of transactions. Some have turned to supercomputers and/or superstorage systems [8]. Others have focussed on algorithms, data structures, or databases to ensure good performance. More specifically, there has been work on linking, browsing, searching, clustering, handling multimedia information, and managing large distributed collections to support discovery and reuse – key topics in the fields of hypertext, information retrieval, and multimedia. As part of the “digital” aspect of DLs, there has been work on capture, compression, representation, conversion, and storage – topics also studied in the disciplines of artificial intelligence, graphics, and hardware (especially with regard to I/O and storage devices, as well as special purpose processors). Modeling and simulation can help with understanding and tuning performance [9]. Networking is crucial for building large DL systems from collections of computers, as well as for serving a distributed set of users. To support those users, DLs also must be effective. Thus there is a clear connection to HCI, with respect to better user interfaces as well as handling of requirements, design, and evaluation. Formal or programming languages can support description of specifications, implementation of scenarios, and scripts for DL personalization. Concern with quality (e.g., adding value to content objects, or ensuring quality of service) is addressed in connection with multimedia information and networking. Many DL efforts have explored a variety of architectures so as to better support individual user needs, wider ranges of services, and diverse organizations and distributions of information. The human side of these issues is studied under the aegis of library and information science as well as computer professionalism, which helps connect with social, economic, and legal concerns. As part of the “library” side of DLs, there is work on user needs, intelligent information services, data and metadata, as well as support for user tasks and activities. These topics also relate to fields like artificial intelligence and geographic information systems. Whether we build DLs for individuals, groups, organizations, disciplines, nations, regions, or the world, we must solve difficult problems. Large-scale interoperable federated systems, in particular, force us to develop the next generation of “super information systems.” Thus, the greatest challenges, and the brightest teams that can be assembled to meet them, both will result from international collaboration. 2. Importance of International Collaboration on Digital Libraries Knowledge knows no boundaries. Our surveys of users and experts indicated that people want DLs to afford them access to the world’s information from their homes and offices [10]. This goal is being addressed in part as a result of DL projects underway all around the world [7], including Pronabid in Mexico.. However, these diverse efforts will not magically coalesce into a seamless set of services that smoothly interoperate. International collaboration will be needed to achieve this vision as well as other desired goals. In a 1998 “summit” to explore international collaboration regarding DLs, many key needs and opportunities were articulated [11]. Clearly there is need for organization on a global scale, and so a second summit/workshop is scheduled in conjunction with the ACM Digital Libraries ’99 conference (see http://fox.cs.vt.edu/DL99). Four other types of collaboration are also of particular value. First, there is collaboration around projects concerned with special collections of information. For example, computer science researchers around the world have developed systems to share their technical reports [12-14], now largely in connection with the Networked Computer Science Technical Reference Library (NCSTRL, http://www.ncstrl.org). Recently, bilateral funding from NSF (USA) and JISC (UK) will be supporting related research based at Cornell University to extend these efforts. Another example is work on the genre of electronic theses and dissertations [15], discussed in Section 4 below. In both cases, as well as many others, critical mass in terms of information value only results when similar objects are (logically or “virtually”) brought together that are authored and managed in hundreds or thousands of sites. Alternatively, there may be a smaller number of large collections, such as from national libraries, museums, and/or archives (e.g., a White House initiated effort for a Digital Library for Education, with approximately $33M proposed to go to the NSF, the Library of Congress, the Smithsonian, the National Park Service, and IMLS – which funds museums and libraries). By involving several different countries, thesecould provide enormously valuable resources to promote education and global understanding. Because of their scope, such big projects could provide a basis for sharing technology as well as broadening access to works from diverse cultures. Second, there is collaboration around particular technical problems. For example, a number of workshops situated in diverse sites around the world have addressed the problem of metadata standards for digital collections [16]. Agreement has been reached, and there is worldwide application of the Dublin Core to support compatible descriptions of digital objects. Similarly, agreements have been reached in the areas of knowledge representation (e.g., KQML), markup (e.g., XML and SGML, as well as HTML). Further, though there is no single solution to broad problems like handling DL collections of geographic information, music, images, or digital video, progress has certainly been accelerated as a result of research efforts in laboratories scattered around the globe. Third, there are particular DL problems that are especially obvious when one contemplates international collaboration, e.g., How can collections be searched that are in many different languages? How can collections be integrated that each covers a “piece of the puzzle”, but together provide a world view (e.g., aerial maps of individual nations as a resource for geographers, old records of city newspapers for students learning about world history)? How can access be controlled and services personalized in accord with varying legal, economic, cultural, and social regulations and conventions? How can resources be discovered among collections that vary widely in coverage, size, and organizing principles? How can discovery and understanding be expedited as a result of multiple perspectives, approaches, and views – supporting each other in collaborative environments? Finally, there are important problems in the field of DL that have particular urgency, or whose solution will have great benefit, that are likely to be solved more quickly as a result of international collaboration. One such class of problems deals with global challenges, like the environment, where gathering data and acting in an informed fashion is necessary in all nations. Another such problem is how to spread awareness of DLs so that the contributions of the field have broader impact, e.g., helping solve the problems of handling information in business and government, as well as in educational institutions. Also key among the problems is the need for a theory and underlying framework to buttress the field of DL. The next section explores one approach to this challenge. 3. “5S” In the 1960s, J.C.R. Licklider challenged us to develop a unified theory to support the development of global digital libraries [2]. Accordingly, we have developed a descriptive framework – “5S” – referring to Societies, Scenarios, Spaces, Structures, and Streams. We argue that this framework not only helps with the design and development of DLs, but also should help users more easily understand related technologies. 5S is particularly oriented to describe information systems. We view DLs as high-end or “super” information systems that integrate a wide variety of more specialized technologies, and so benefit strongly from such a powerful framework. Indeed, we offer yet another definition of “digital libraries” using 5S, i.e., complex systems that help: satisfy information needs of users (Societies), provide information services (Scenarios), organize information in usable ways (Structures), present useful information (Spaces), and communicate information with users (Streams). Advanced information systems often involve multimedia information and distributed processing, both of which require special support for Streams, that fundamentally are sequences of items (e.g., text, audio, video, or network traffic). Various approaches to information organization involve suitable Structures – whether in collections, using trees or relations (as in databases), supported by indices (as in geographic information systems – GIS), through graphs (e.g., of nodes and arcs, as in hypertext), or as complex objects. Scientific visualization, virtual reality simulations, GIS, vector space or probabilistic or conceptual searching, and 2D or 3D graphics interfaces all make use of Spaces (which formally involve domains and constraints). Since DLs provide a range of services, supporting various types of information needs in tailored fashion, and are often designed based on story-like descriptions of interactions, the use of Scenarios (which we define to also include functions, operations, and methods) is particularly important. Finally, since DLs are built to serve particular target users or user communities (i.e., groups of interacting people and computers), it is essential that all Societies involved be carefully studied (i.e., we extend our earlier 4S model [17], adding Societies). Applying the 5S model calls for considering typical aspects, facets, or components of each “S”. Thus, regarding Societies, we may consider artifacts, relationships, rituals, and roles. In the DL case, Scenarios involve actions, e.g., acquire, administer, consult, index, preserve, retrieve, and store. Spaces may be functional, physical, presentational, or temporal. Structures may involve architectures, grammars, links, schema, and taxonomies. Finally, Streams may involve flows, granularities, paths, and protocols. To illustrate the power of 5S, we can consider combinations of several of the “S” constructs. We list five below, in the form of equations to show that many aspects of computer and information science can be easily described: Societies + Scenarios = user model Societies + Scenarios + Spaces + Streams = user interface Streams + Structures = markup Streams + Structures + Scenarios = object Structures + Scenarios = DBMS To further illustrate the value of 5S, and to explain how DLs can be of value to support education, we briefly describe two case-study DL projects in the following sections, using 5S to highlight key aspects. The reader also is referred to a more detailed discussion of these two efforts from the perspective of 5S [18], as well as a focussed description of the need for collaboration to enhance graduate education [19]. 4. NDLTD As of June 1999, the Networked Digital Library of Theses and Dissertations, NDLTD [20-23], included 62 universities and other interested parties working to enhance education and research. Universities from around the globe, along with a number of supporting institutions (e.g., the Coalition for Networked Information and UNESCO), encourage students to prepare theses (undergraduate or graduate) and dissertations electronically, submit them into a DL, and facilitate access by other students and researchers. It is hoped that this federated initiative will spread to every country and university, ensuring that the next generation of scholars is prepared to create expressive electronic documents, and able to work with worldwide DLs [7]. NDLTD encompasses all of the key aspects of DL efforts. It is a project, with a home page (http://www.ndltd.org), and a governing body (i.e., a Steering Committee) that meets in Washington, D.C. twice annually, near the middle of April and near the middle of September. It includes as its collection all of the bibliographic records, metadata, documents, and related works made available by its members or other supporters. The focus of the collection is on theses and dissertations, but reports and other documents are also welcome; thus NDLTD can gradually broaden to include a richer diversity of content as it evolves into a Networked University Digital Library (NUDL – see Section 6.2). The collection can be accessed through browsing and federated searching mechanisms from http://www.theses.org as well as through a variety of connections to repositories run by individual members or groups of members. Students wishing to learn how to create electronic theses and dissertations (ETDs [15, 24]) can benefit from a training site rich in tutorials and multimedia explanations (see http://etd.vt.edu). They are encouraged to use standards (e.g., SGML, XML, PDF, MPEG, JPEG) in order to facilitate preservation. NDLTD involves many Societies: graduate students, those administering graduate programs and university libraries, faculty, publishers, its own governing body and committees, as well as all those who might use the NDLTD collection. NDLTD scenarios include training to empower students with knowledge about electronic publishing and DLs, submission of ETDs, and various modes of using the collection (e.g., searching, browsing). Scenarios also can be categorized according to the Societies served, e.g., users, authors, author-researcher collaborations, librarians, and graduate school staff. Spaces cover the physical location of members, the name space of their servers, vector spaces used in searching, the conceptual space that is specified in Virginia Tech’s proposal for applying Dublin Core to describe ETDs, and the various interfaces (e.g., 2D, 3D, CAVE [23]) developed to access the collection. Streams cover the various text and multimedia content objects and the several protocols supported through the federated search system [25]. Structures support the DBMS application that handles submission and access, the use of document type definitions developed for SGML and XML formatted ETDs, and the inner workings of DL systems considered for NDLTD such as those provided by IBM and OCLC. While NDLTD focuses on graduate education, CSTC, discussed in the next section, has more of an emphasis on undergraduate studies. 5. CSTC The Computer Science Teaching Center (CSTC, see http://www.cstc.org and [26-28]) in general, and in particular its part for Curriculum Resources in Interactive Multimedia (CRIM, see http://www.cstc.org/~crim/ and [29]), continue the NSF-supported efforts related to DLs and computing that began at Virginia Tech in 1991. In addition to tens of thousands of online WWW pages related to CS education [30], reflecting local development, that have led to millions of accesses from around the world [30-32], an open repository for submission of educational modules now is available at Virginia Tech. By encouraging small submissions, in particular of knowledge modules [33], and by instituting a system of peer review to ensure quality, we hope to turn CSTC into an example of how to develop the NSF-funded Science, Mathematics, Engineering and Technology Education (SMETE) Digital Library [34]. Three types of materials are actively solicited for CSTC. First, we seek laboratory resources that can be used during lab sessions in (primarily lower level) courses on computing, or for selfstudy. Second, we seek visualizations of important concepts in computing as well as tools to help prepare such visualizations. Third, we seek interactive multimedia resources teaching about important aspects of computing. Each type requires knowledgeable editorial control, now handled by Deborah Knox, Scott Grissom, and Rachelle Heller / Edward Fox, respectively. Other volunteer editors, from any part of the world, are encouraged to help collect valuable materials, to cover all pedagogical (including topical and regional) requirements. Contributors log onto the CSTC site, fill in online forms to describe their resources, and upload them – for test, review, and eventual archiving. Thus, they provide metadata as well as the digital object or objects to be shared with other teachers or learners. Special interface screens have been devised to support people in the roles of contributor, user (e.g., a teacher discovering a useful resource), reviewer, and editor. These have been refined for more than a year, and will continue to be enhanced to suit the needs of the project. CSTC involves several Societies, especially teachers, students, editors, and reviewers. Scenarios cover submitting resources, editorial assignments to reviewers, preparing and submitting reviews, as well as locating and adapting useful resources for use in classroom, distance education, or self-study situations. Spaces include the conceptual space laid out in accordance with the Dublin Core, the category system from ACM, and the various interfaces to the collection. Streams cover the interactive multimedia resources and their use, as well as the Web prototols involved in using the repository. Structures include the DBMS that underlies the DL along with its various files and logs. CSTC is but one of many efforts that may support undergraduate education in the scientific, engineering, and technical areas as part a US national effort [34], that we hope will extend to a broader international collaboration. 6. Conclusions: 5S, NUDL, and Pronabid 6.1 Extending 5S Building on the 5S framework, we can propose an approach to DL collaboration involving the USA, Mexico, and other nations. The most open type of initiative would be one where collaboration occurs along any of the 5S aspects. The boldest type of initiative might be one where one or more “S” is extended well beyond current conventions. Taking the broadest Society perspective, projects may support the community of world users. This extended “S” approach suggests the need for systems to economically handle very large numbers of users. For example, Virginia Tech has purchased VT-PetaPlex-1, a system produced by Knowledge Systems Incorporated [8]. This “superstorage” unit has 2.5 terabytes storage capacity. With 100 processors (each a Pentium II running at 233 MHz) running Linux, it has roughly 20 gigaflops computing capability as well. Virginia Tech offers to new members of NDLTD access to the PetaPlex system for archiving their ETDs. Many interesting research problems relate to developing this and similar cost effective “digital library machines” as well as larger “information utilities” to support petabytes of data and millions of users. To achieve more focus, Societies may be worldwide groups at the discipline level, such as those involved in computer science teaching and learning (e.g., CSTC). Other groupings of users may arise based on age or educational status, e.g., graduate students involved in NDLTD. Almost any DL initiative will have a clearly defined Society or set of Societies. Some may be defined in terms of Societies of interacting computers or software modules (e.g., agents). Work on extended Scenarios may lead to broader views of DLs in terms of services. Can DLs record all the information generated as a result of all activities over the life of an individual? Can DLs operate “under the covers” of other automated systems to provide personalized services requiring a wide variety of information? For example, can they help students engaged in distance education to learn more effectively, or assist authors in businesses to create new reports that reflect knowledge of the relevant best practices and the latest discoveries? Can DLs support reuse of knowledge from available sources in addition to discovery and retrieval? Other work on Scenarios relates to usability studies for existing or new user interfaces, as well as standard services like searching or hypertext activities like browsing, following links, adding annotations, or following paths/trails. Ultimately there needs to be coverage of all activities carried out by librarians, in gathering, organizing, administering, and supporting use of library resources, applied to the broad range of digital objects that also may be found in museums, archives, and many other information processing organizations. Work on extended Spaces may lead to innovation in spatially referenced information systems (e.g., the Alexandria Digital Library in Santa Barbara) and user interfaces (especially using 3D or virtual reality environments [23]). Better browsing and visualization may result from improving or integrating conceptual spaces based on human-devised category systems (e.g., ACM’s for computing, UMLS for medicine) and artificial methods (e.g., through cluster or citation analysis). Other research related to Spaces may deal with better retrieval using vector or probability spaces (e.g., through better indexing or weighting, for any content type), distributed processing dealing with physically scattered collections, and mobile access for users engaged in travel. Work on extended Streams pushes the state of the art in multimedia information handling and networked communication. Emerging applications focus on handling streaming large video or audio files with high quality of service, useful for education at all levels. A key challenge is to extend and integrate/standardize protocols for DLs, to support interoperability as well as various modes of access to collections in diverse media forms (e.g., combined search across languages, images, audio, and video). Work on extended Structures may lead to integration between what are now poorly connected systems, e.g., DBMS, IR, and hypertext. This is becoming more urgent and feasible with the spread of use of standards like XML that allow encoding of data usable with all those types of systems. Handling diverse views of objects, at different levels of granularity, also is a crucial requirement, as more and larger digital objects become available. Important new research in each of the 5S areas, as well as dealing with combinations of these aspects, will be crucial in the DL field. Many important projects should address these challenges. Two possible efforts are outlined in the next subsection. 6.2 NUDL and Pronabid The Networked University Digital Library (NUDL) is an outgrowth of the NDLTD and CSTC efforts. It aims to foster collaboration among universities around the world, to advance research, practice, and application of DLs. We argue that every university should have its own DL, that should interoperate with other university DLs. Our approach to develop NUDL is to start small, beginning with an effort that already involves a number of other universities, such as NDLTD. Similarly, contributing to and using the content from efforts like CSTC, closely tied to improving education in popular fields (e.g., computer science), may be appealing. Broadening from this type of base, to promote sharing of knowledge and educational resources, NUDL invites support from all interested universities and others desiring to promote learning. In Mexico, there is growing interest in DLs. “Pronabid” is an initiative to develop infrastructure and support for the field. Goals include connecting information handling with high speed networking, addressing national goals and priorities, expanding the workforce of trained personnel, building valuable and usable digital collections, preserving and increasing access to important cultural and intellectual resources, and supporting future needs with a sound reference model and funding program. Through projects like NUDL, and national initiatives like Pronabid, broader collaboration will arise in the DL field. In time, there will be better support for national and global needs for knowledge. 7. Acknowledgements The NSF Division of Undergraduate Education, through grants 9752190 and 9752408, supports the CSTC and CRIM projects, involving co-PIs Deborah Knox (CSTC director, at The College of New Jersey), Scott Grissom (Univ. Illinois, Springfield), and Rachelle Heller (The George Washington Univ.). ACM supports CSTC and supported earlier NSF-funded research at Virginia Tech on computer science and digital libraries. The U.S. Department of Education, through its Fund for the Improvement of Post Secondary Education, P116B61190, for 1996-1999, has sponsored Virginia Tech co-PIs Edward A. Fox (project director), John Eaton, and Gail McMillan in their work on “Improving Graduate Education with a National Digital Library of Theses and Dissertations”. Adobe, IBM, Microsoft, and OCLC have donated substantial hardware and software to help with NDLTD. Robert Akscyn and his company Knowledge Systems Incorporated have provided VT-PetaPlex-1 and collaborated on related research. Special thanks go to graduate students Neill Kipp, Paul Mather, Constantinos Phanouriou, and David Watkins. Many other faculty and students at Virginia Tech, and around the globe, also have contributed. Their assistance is gratefully acknowledged. References [1] E. A. Fox, “Sourcebook on Digital Libraries: Report for the National Science Foundation”. Technical Report TR-93-35. Blacksburg, VA: Dept. of Computer Science, Virginia Tech, December, 1993. Available by FTP from directory pub/DigitalLibrary on fox.cs.vt.edu [2] J. C. R. Licklider, Libraries of the Future. Cambridge, MA: MIT Press, 1965. [3] V. Bush, “As We May Think,” Atlantic Monthly, 1945. [4] M. Lesk, Practical Digital Libraries: Books, Bytes and Bucks. San Francisco: Morgan Kaufmann Publishers, 1997. [5] E. A. Fox and G. Marchionini, “Proceedings of the First ACM International Conference on Digital Libraries, DL'96”Bethesda, MD; New York: ACM, March 20-23, 1996. [6] E. A. Fox, “Digital Library Courseware”. WWW site. Blacksburg, VA: Virginia Tech Department of Computer Science, 1998. http://ei.cs.vt.edu/~dlib/ [7] E. A. Fox and G. Marchionini, “Toward a Worldwide Digital Library; Guest Editors' Introduction to Special Section on Digital Libraries: Global Scope, Unlimited Access,” Comm. ACM, vol. 41, pp. 28-32, 1998. http://purl.lib.vt.edu/dlib/pubs/CACM199804 [8] R. M. Akscyn, “The PetaPlex Project,” Status Briefing for National Security Agency 9 March 1998, 1998. http://ks.com [9] O. Balci, C. Ulusaraç, P. Shah, and E. A. Fox, “A Library of Reusable Model Components for Visual Simulation of the NCSTRL System,” in Proceedings of the 1998 [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] Winter Simulation Conference. Washington, DC: IEEE, Piscataway, NJ, 1998, pp. 14511459. E. A. Fox, D. Hix, L. Nowell, D. Brueni, W. Wake, L. Heath, and D. Rao, “Users, User Interfaces, and Objects: Envision, a Digital Library,” J. American Society Information Science, vol. 44, pp. 480-491, 1993. R. M. Akscyn and I. H. Witten, “Report of First Summit on International Cooperation on Digital Libraries”Marriott City Center, Pittsburgh, PA: Workshop of ACM Digital Libraries '98, June 27-28, 1998. http://ks.com/icdl, http://www.ks.com/idla-wp-oct98/ K. Maly, J. French, A. Selman, and E. A. Fox, “Wide Area Technical Report Service,” in Second International WWW '94: Mosaic and the Web, WWW'94. Chicago, IL, 1994, pp. 523-533. E. A. Fox, “World-Wide Web and Computer Science Reports,” Communications of the ACM, vol. 38, pp. 43-44, 1995. J. French, E. A. Fox, K. Maly, and A. Selman, “Wide Area Technical Report Service technical reports online,” Communications of the ACM, vol. 38, pp. 45, 1995. E. A. Fox, G. McMillan, and J. Eaton, “The Evolving Genre of Electronic Theses and Dissertations,” presented at Digital Documents Track of HICSS-32, Thirty-second Annual Hawaii International Conference on Systems Sciences (HICSS), Maui, HI, 1999. http://scholar.lib.vt.edu/theses/presentations/Hawaii/ETDgenreALL.pdf Dublin-Core-Community, “Dublin Core Metadata Initiative”. WWW site. Dublin, Ohio: OCLC, 1999. http://purl.org/dc E. A. Fox, N. Kipp, and P. Mather, “How Digital Libraries Will Save Civilization,” Database Programming & Design, vol. 11, pp. 60-65, 1998. http://www.dbpd.com/foxweb.html E. A. Fox, “The 5S Framework for Digital Libraries and Two Case Studies: NDLTD and CSTC,” in Proceedings NIT'99. Taipei, 1999. E. A. Fox, “Networked Digital Library of Theses and Dissertations,” in Proceedings DLW15. Japan: ULIS, 1999. E. A. Fox, J. Eaton, G. McMillan, N. Kipp, L. Weiss, E. Arce, and S. Guyer, “National Digital Library of Theses and Dissertations: A Scalable and Sustainable Approach to Unlock University Resources,” D-Lib Magazine, vol. 2, 1996. http://www.dlib.org/dlib/september96/theses/09fox.html E. A. Fox, J. L. Eaton, G. McMillan, N. Kipp, P. Mather, T. McGonigle, W. Schweiker, and B. DeVane, “Networked Digital Library of Theses and Dissertations: An International Effort Unlocking University Resources,” D-Lib Magazine, vol. 3, 1997. http://www.dlib.org/dlib/september97/theses/09fox.html E. A. Fox, R. Hall, N. A. Kipp, J. L. Eaton, G. McMillan, and P. Mather, “NDLTD: Encouraging International Collaboration in the Academy,” Special Issue on Digital Libraries of DESIDOC Bulletin of Information Technology (DBIT), vol. 17, pp. 45-56, 1997. C. Phanouriou, N. Kipp, O. Sornil, P. Mather, and E. A. Fox, “A Digital Library for Authors: Recent Progress of the Networked Digital Library of Theses and Dissertations,” presented at The Fourth ACM Conference on Digital Libraries, DL '99, Berkeley, CA, 1999. ARL, Electronic Theses and Dissertations, vol. 7, Spec Kit 236 ed. Washington, D.C.: Association of Research Libraries, 1998. http://www.arl.org/transform/ [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] J. Powell and E. Fox, “Multilingual Federated Searching Across Heterogeneous Collections,” D-Lib Magazine, vol. 4, 1998. http://www.dlib.org/dlib/september98/powell/09powell.html E. A. Fox, “Digital Libraries: The Networked Digital Library of Theses and Dissertations (NDLTD) and the Computer Science Teaching Center (CSTC)”. Keynote. Puebla, Mexico: Computer Science Workshop, sponsored by CONACyT and NSF, June 10-12, 1998. http://www.ndltd.org/talks/CONACyTNSF.pdf E. A. Fox, “Improving Education through the Networked Digital Library of Theses and Dissertations (NDLTD) and the Computer Science Teaching Center (CSTC),” in Russian-American Digital Libraries Workshop. Moscow, 1998. http://www.ndltd.org/talks/russia13.htm E. A. Fox, “Helping Learners through Digital Libraries: The Networked Digital Library of Theses and Dissertations (NDLTD) and the Computer Science Teaching Center (CSTC),” presented at Dept. of Computer Science, Univ. of Mass. Seminar, Amherst, MA, 1998. http://www.ndltd.org/talks/UMass98.ppt E. A. Fox, R. S. Heller, A. Long, and D. Watkins, “CRIM: Curricular Resources in Interactive Multimedia,” in Proceedings ACM Multimedia '99, Oct. 30 - Nov. 5, 1999. Orlando: ACM, 1999. E. A. Fox, “Education Infrastructure/Innovation Courseware”. WWW site. Blacksburg, VA: Virginia Tech Department of Computer Science, 1998. http://ei.cs.vt.edu E. A. Fox, “Interactive Learning with a Digital Library in Computer Science,” presented at NSF CISE EI PIs Workshop at Frontiers in Education - FIE'96, Salt Lake City, Utah, 1996. E. A. Fox, “Digital Libraries, WWW, and Educational Technology: Lessons Learned,” presented at ED-MEDIA 96, World Conference on Educational Multimedia and Hypermedia, Boston, MA, 1996. http://ei.cs.vt.edu/~fox/EDMEDIA96 E. A. Fox and L. Kieffer, “Multimedia Curricula, Courses and Knowledge Modules,” ACM Computing Surveys, vol. 27, pp. 549-551, 1995. W. Arms, “Report of the NSF Science, Mathematics, Engineering, and Technology Education Library Workshop, July 21-23, 1998,” National Science Foundation, Division of Undergraduate Education. NSF 99-112, 1999. http://www.dlib.org/smete/public/report.html (Paper prepared for NSF – CONACyT – ISTEC Workshop on Digital Libraries, Albuquerque, New Mexico, USA, July 7-9, 1999)