A framework for distributed digital object services

The following paper was written by the authors over a period of approximately 16 months during the period November 1993 to May 1995 in an attempt to explore a set of open research issues and to integrate them with certain ideas of a small group of researchers who had been briefed on the notions inherent in the Digital Object Architecture that one of the authors (Kahn) had been developing at the Corporation for National Research Initiatives (CNRI). This research group had been organized by CNRI as part of its "Computer Science Technical Reports" (CSTR) project that was funded by the Defense Advanced Research Projects Agency (DARPA). The CSTR project had several objectives, of which one was to digitize existing collections of technical reports from five leading computer science departments at universities in the U.S and make the reports available on the Internet. A second objective was to fund research at these institutions on digital libraries, especially research that made good use of their local collections. A third objective was to link heterogeneous electronic libraries, such as were being developed in the program. The difficulties inherent in this third objective ultimately led to this paper. In particular, the motivation arose from a set of concerns first expressed by these researchers at a meeting at CNRI on October 25, 1993 when the Digital Object Architecture was first presented to the research group by Kahn. In the discussion that resulted, many issues emerged, some of which we ultimately decided must be resolved in any architecture, and some of which-perhaps just as crucially-could be deferred for the time being. For example, questions arose about the role of semantics in identifiers; we decided to address them early on. Another issue that proved controversial was how to deal with situations in which the digital objects were actually mobile programs in the network. Since a digital object could contain other digital objects, this led directly to the notion of a mobile repository. The basic question was how best to describe this kind of situation and explain how to access information that was not at a known specific location or IP address on the net. We decided not to cover this aspect of the architecture in the interest of getting closure on the paper. A critical part of this effort was dealing with various intellectual property issues; and the architecture was designed to take this important aspect into account. We were fortunate to have the help of Ms. Patrice Lyons, an experienced intellectual property lawyer, in the formulation of certain key elements of the architecture dealing with terms and conditions for access to information, and in the drafting of the paper. Although her name does not appear on the paper, her insights and contributions played an important role in the preparation of the paper. One key component of the architecture is a general-purpose resolution system, known as the Handle System (R) , an implementation of which has been operational on the Internet since 1994. At present, close to 50 million identifiers are assigned to digital objects and resolvable by the system, which has been available 7x24 for many years. It is anticipated that the number of resolvable identifiers in the system will grow rapidly and could easily exceed a billion in the not too distant future. Much progress has occurred in the past decade on the remainder of the architectural components, such as repositories, metadata registries, and the associated implementations; and the Handle System continues to evolve to meet new requirements. Uses in both government and the private sector continue to expand. Digital ID World selected the Digital Object Architecture for its 2003 award for balancing innovation with practical reality. Many groups are now using the technology. For example, the International DOI Foundation relies on the Handle System for administering its identifiers, and the Advanced Distributed Learning project, an effort of the U.S. Department of Defense, requires the use of the Handle System for identification of its materials and for resolution of identifiers. Since the mid-1980s, CNRI had been working on various aspects of digital libraries and mobile programs, referred to as "Knowbot programs." Certain key aspects of this work were subsequently written up in a patent application that was filed with the U.S. Patent and Trademark Office prior to its presentation to the research group in late 1993. The purpose of the patent filing was to insure that the technology would not be constrained by others and its integrity could be maintained. It has since been made available by CNRI under an open source license. The CNRI patent was issued in 2000, but during the intervening years significant progress was made by CNRI and by others in implementing key components of the architecture. One reason the Internet was broadly appealing was its openness to applications of all kinds that could use it as an underlying infrastructure. The same basic attribute can be seen in the digital object architecture. It neither constrains the choice of applications, nor mandates how they must be implemented. However, as a powerful reference model, with demonstrated implementations of its components, it enables others to confront difficult system design and implementation projects from a well-formulated starting point that enables, and, more accurately, reinforces the ability for heterogeneous information systems to interoperate with each other. By taking a complex subject and rendering it accessible in simpler terms, it was possible to make a basic set of ideas more understandable to a larger audience. It was not easy to write the paper, and indeed we went through close to a dozen iterations over more than a year in the sometimes elusive search for clarity of expression. We are pleased to see the paper included in this special issue.