DOMS Implementation Project

Digital Collections Manager Functional Specifications

Last updated 30 September 2002

1. Introduction

1.1 Overview

This document provides detailed specifications for a Digital Collections Manager which meets the Library’s needs for recording and processing management and technical information about image digital surrogates. The Library has embarked on a major digitisation program to provide improved access to and long-term preservation of collection materials. The initial focus of this program is capture of material such as pictures, maps, manuscripts, rare books, 19th century serials and sheet music to produce high resolution digital masters for preservation and low resolution display copies for access via the Internet. The Digital Collections Manager will assist the Library’s collection managers to manage the digitisation process, as well as ongoing format migration and preservation processes through recording of appropriate metadata. It will replace a number of existing in-house systems and provide for long-term management of digital material by the Library.

In the future it is planned to enhance the Digital Collections Manager to support:

The estimated disk storage requirements for the digital materials to be managed by the Digital Collections Manager are 2 terabytes in 2001 rising to 20 terabytes in 2006. This corresponds to approximately 200,000 digital assets in 2001 rising to over 2,000,000 by 2006. The Digital Collections Manager must be capable of managing this volume of data and its associated metadata with no effect on functionality or performance.

1.2 Infrastructure Requirements

The Library has purchased a four processor Sun 450 system with 2 TB of RAID disk and a tape library for backup and archiving of data. The Digital Collections Manager will be developed using Oracle and the Library’s Application Development Framework.

1.3 Performance Requirements

The Digital Collections Manager must be able to meet the following performance requirements:

Response times in this section apply to the speed of response from the server whose performance is being measured to client software in the same room or on the same network segment. The response will be in the form of a web page viewed using Microsoft Explorer Version 5 and including no markup beyond that required to delimit the database fields displayed. The response time will be measured from the moment the user submits the transaction until the first screen of the response is completely displayed. The hardware platform for measurement of performance is that specified in section 1.2. Performance will be measured using the data requiring to be loaded on day 1 and a PC based within the Library, connected to the DCM via the Local Area Network. Response time will be measured from the moment the user submits the transaction until the first screen of the response is completely displayed.

As part of a separate project the Library is building a Generic Delivery System for Digital Objects. The Generic Delivery System will use metadata stored in the Digital Collections Manager to construct the user interface and navigation workflows required to access each object stored in the Digital Object Storage System. The performance requirements specified above should not be impacted by the activity of the Generic Delivery System. This may require the data to be duplicated in a second instance that is targeted by the Generic Delivery System.

1.4 Specification Methodology and User interface Requirements

The user interface must be web browser based, so that special software will not need to be installed and upgraded. The Library uses Microsoft Explorer Version 6 as its default browser with all staff able to access web based services via a screen size of 800 x 600 pixels. The user interface must be optimised for use with this browser and screen size. However, only features supported by the browser that are aligned with W3C standards should be used for core functionality. In addition to the W3C markup and style sheet standards, the Library requires support at least to level 1 of the W3C Content Accessibility Guidelines for all of its web browser based user interfaces.

The Digital Collections Manager will deliver a mission-critical workflow system to the Library. The guiding principles for development of the user interface must be simplicity, speed and ease of use. The user interface must allow the user to do as much as possible with as few keystrokes/ mouse clicks as possible. Standard alternatives to mouse clicks must be provided for occupational health and safety reasons. The user interface must present the most likely action as the default and do as much as possible to prevent common mistakes and maintain the integrity of the Digital Collections Manager. It must remember the context in which the user is working and provide appropriate “templates” and default values.

1.5 Definitions and Workflows

In the Specifications the term data is used to refer to objects in digital form, eg master copies of pictures in TIFF format or objects, while the term metadata is used for structured information that describes the data (data about data). Data may also be used generically, to describe both data and metadata, for example, when specifying data migration requirements. Metadata may also be used for structured information that describes a work that has not yet been digitised.

A work is anything that is collected by the Library. It may be a physical object, book, manuscript, map, picture, sheet music, serial or sound recording. The term collection is used to describe all works collected by the Library and also to refer to a specific sub-collection, eg the Pictorial Collection, that is managed by a separate group of collection managers.

The Library may choose to manage a work as a unit or a subunit. A unit may be an “item” which consists of just one primary object, eg a book or painting; or a “set”, which consists of multiple subunits (eg the papers of a famous person or a photograph album or a group of glass negatives with shared provenance). A subunit may itself be an item or a set; or it may be a part. A “part” is a component of a complex item such as a book or three-dimensional object, eg a cover, page or view. Using this terminology, a manuscript collection such as the Barton Papers is a set-level unit. The diary series is a set-level subunit. A diary in this series is an item-level subunit. A page in this diary is a part. A component is a part that is also treated as an item in its own right, for example, a plate in a book.

The table below maps units and subunits against ten likely work hierarchies. This model will support a six level hierarchy, eg a detail (level 6) on a page (level 5) in a piece of manuscript music (Level 4) forming a component part of a bound volume (level 3) in a series (level 2) of a manuscript collection (level 1). Ideally, however, there should be no limit to the number of levels supported. Decisions to minimise the number of levels should be business, not system-driven.

    Likely Work Hierarchies
    item item item item item set set set set set set set set
unit 1   part part comp comp item item item item set set set set
subunit 2     part part part part part comp comp item item set set
subunit 3         part   part part part part part comp comp
subunit 4                 part   part part part
subunit 5                         part
subunit 6                          

A copy is a manifestation or instantiation of a work. A work can have multiple copies. The Library may hold one or more copies of a work, eg the original paper version of the page described above, a number of microform replicas and a set of digital replicas made from either the original or from the microform copy. In this specification a copy may also be an access aid such as a transcript or finding aid. The Library may also hold one or more original copies of a work, each of which may be a target for digitisation for access or preservation reasons.

When a work is selected for acquisition by the Library, a catalogue entry is raised for the unit and copy-specific details are recorded. If the work is a set, subunits are listed and electronic list(s) or finding aid(s) generated that are linked to the catalogue entry through a URL. These processes may occur as part of raising the catalogue entry or at a different time, for example, when physical copies are being batched for digitisation. Subunit items and components may be given their own catalogue entry if the work is considered sufficiently significant to be listed in catalogue result sets.

For units and subunits with catalogue entries, the resource description in the NLA Catalogue is the master copy. For subunits without their own catalogue entry but listed in finding aids, the resource description in the finding aid is the master copy. For all other subunits, the resource description in the Digital Collections Manager is the master copy.

A job is an event that causes a process or set of processes to be performed on a copy or group of copies. A process is an event that uses a physical or digital copy as input and may result in transformation of the existing copy (eg, through repair, restoration or media transfer) or the creation of a new one (eg through digitisation, microfilming, photocopying or format migration). A project is an event that provides the rationale for a job. A project may involve multiple jobs and processes over time.

The Library’s in-house digitisation teams will generally digitise copies on a one-by-one basis. The Digital Collections Manager will first know about the digital copies when a job is initiated to upload them. In this case the digitisation workflow will be tightly coupled with the loading workflow. A group of copies may also be batched and submitted for digitisation as a single job. The batch will be accompanied by a job submission report itemising the input copies. The digitiser may upload the copies via an online process or return the files together with a process report in the standard import format defined below. The latter workflow will generally be used by contractors. In all cases, procedures will be supported to enable various degrees of checking, with appropriate correction action taken if quality control standards are not met.

Metadata will be imported to and exported from the Digital Collections Manager via an import file and export file that conform respectively to a standard import format and standard export format. This format should be defined in the form of an XML DTD or schema.

1.6 Persistent Identifiers

The Library will assign a persistent identifier to each work and copy managed by the Digital Collections Manager. The Library will maintain a separate Persistent Identifier resolver service that will algorithmically determine the appropriate means of web access to the referenced work or copy.

Work persistent identifiers

The persistent identifier for a work will be composed of the following elements:

collectionId-workId

- where:

The work persistent identifier will correspond to the string included after the http://nla.gov.au/ part of the URL in the catalogue entry and will also be used as the work ID in the DIgital Collections Manager. Records will not be able to be added to the Digital Collections Manager from the NLA Catalogue or a finding aid unless the work ID can be derived from the catalogue entry or the entry in the finding aid.

Generally the persistent identifier for a child work will be based on the persistent identifier for the parent work with an added sequence number: e.g.:

Exceptions to this rule include members of sets that have been catalogued separately or that have been physically numbered in a flatter sequence that the Library prefers to reflect in the persistent identifier: e.g.:

Policies for assigning persistent identifiers to a work are outlined in Persistent Identifier Scheme Adopted by the National Library of Australia.

Copy persistent identifiers

The persistent identifier for a copy will be composed of the following elements:

collectionId - copyId - role - version

where:

The copyId component of the persistent identifier will generally be the same as the workId, e.g.:

Where there are multiple original copies of a work, each copy will have its own unique persistent identifier. In this case, the persistent identifier of the "best copy" will usually be the same as the persistent identifier of the work.

1.7 File System Requirements

Digital copies to be managed by the Digital Collections Manager will be created using a range of technologies including flatbed scanner, digital camera, roll film scanner (for microfilm scanning) and overhead scanner. The digital master images created will be stored in 24 bit RGB TIFF version 6 format. Completed master images will be moved from a temporary work area to a network accessible master file system. This file system will be an Digital Collections Manager “hot folder” and images stored in the file system will be automatically imported into the Digital Collections Manager.

1. How files are named or stored must be transparent to the Digital Collections Manager.

2. The Digital Collections Manager must be able to automatically create derivatives of imported master files.

3. It must be possible to vary the number of derivatives and their maximum dimension by Collection; that is, to maintain separate derivative profiles for each Collection.

1.8 Data Standards and Frameworks

The following draft standards and standards development activities have been instrumental in the development of these specifications:

Reference Model for an Open Archival Information System (OAIS).

Making of America II Document Type Definition (MOA II).

Metadata Encoding and Transmission Standard (METS).

Technical Metadata for Digital Still Images.

OCLC/ RLG Preservation Metadata Working Group.

National Library of Australia. Preservation Metadata for Digital Collections.

The OAIS provides a reference framework for an Open Information Archival System. This specification does not use the OAIS terminology because it is a functional requirements specification based on use case methodology, not a system design document. A deliberate effort has been made to use terms that are familiar to collection managers in both the use cases and the user interface templates. However, as part of the system design process, the system should be audited against the OAIS framework as a quality assurance method.

The Library has already developed a Delivery System using metadata based on the MOAII DTD for generating contextual displays (see http://www.nla.gov.au/ms/findaids/ms51/. However, a decision has now been taken for performance and scalability reasons to derive the information needed to support delivery of the Library’s digital objects dynamically from the Digital Collections Manager. However, the data model has been aligned with standards such as METS so that data may be output in METS or an allied format if required at a later stage, for example, to participate in a federated delivery system.

Most preservation metadata initiatives are based on the development of data dictionaries or languages to support the recording of preservation metadata. In the logical data model in this specification, the metadata is described as properties of a range of different objects that need to be brought together for the purposes of import, display or export. In addition, certain preservation metadata elements required by the above draft standards are not defined in the data model, as they can be generated externally. For instance, because the Library manipulates scanned images before uploading them to the Digital Collections Manager, only the display orientation is stored. A default orientation of “normal” can be set if an XML schema or DTD requires this element.

1.9 Current Status

In April 2002, the Library implemented v1.0 of the Digital Collections Manager (DCM). DCM v1.0 supports the following functions:

DCM V1.0 was designed as an early deliverable to support the upload process. This enabled the spreadsheets that were being used to store technical metadata to be thrown away. However, V1.0 did not include a means of keeping work details updated in line with the Catalogue or of generating updates to the Pictures Catalogue. A second early deliverable, DCM V1.1 was implemented at the end of June to deliver the required catalogue interfaces. This enabled the Pictorial Manager system previously used to manage the digitisation of pictures to be deprecated. A small group of fixes to DCM V1.1 was implemented soon afterward as V1.1.1.

Work has now begun on DCM V1.2, which has as its aim to complete the implementation of all functions required for the management of still images.

2. System Interface And Data Migration Requirements

This section provides information on related systems and the sources of metadata to be loaded into the Digital Collections Manager on implementation and managed by the Digital Collections Manager in accordance with the Specifications. Interfaces between the Digital Collections Manager and existing systems used by the Library are also described and requirements identified for sub-tasks to be included in the project plan to cover all system interface and data migration requirements. Future releases of the Digital Collections Manager will have additional data migration requirements.

2.1 Related Systems and Data Sources

PANDAS

PANDAS is the system used to collect and manage web sites selected for long term archiving in the PANDORA archive. No interfaces will be required between PANDAS and the Digital Collections Manager and there is no future requirement for the Digital Collections Manager to replace PANDAS. The workflows to collect and manage web sites are sufficiently different from the workflows to collect and manage digital surrogates for two separate applications to be needed. However, there is a requirement to ensure that the data models for the two systems are aligned. This will enable future enhancements, such as the enhancement to support the recording of intellectual property information and the enhancement to support management of the preservation of the Library’s digital collections, to operate across both data collections.

Current status: The task to align the data models of the two services has not yet been scheduled.

ILMS

The Library already has an Integrated Library Management System (ILMS), which is used for management of the Library’s collections. This system does not meet the Library’s needs for digital object management outlined in Section 1.1. In particular, it does not support the digitisation process. The new ILMS to be implemented during 2002/2003 will also not support the digitisation process. The Digital Collections Manager is being developed to provide this functionality.

A critical success factor for the Digital Collections Manager will be the extent to which collection management workflows currently supported by the ILMS and the workflows needing to be supported to manage the digitisation process are integrated from the point of view of the collection manager.

To achieve this, the ILMS will be the master source for catalogue entries. To keep the two databases aligned, a utility will be developed that exports new and updated catalogue entries from the ILMS for automatic loading to the Digital Collections Manager. The Digital Collections Manager must check for and load a new import file at short time intervals or, ideally, synchronously through definition of the file as a named pipe.

Currently the Library uses the national bibliographic utility, Kinetica as the master source for its catalogue records and downloads the records to the ILMS. The same process will be used for extracting in-scope records for the Digital Collections Manager. As part of implementing the new ILMS a task has been identified to review the ILMS-Digital Collections Manager interface requirements.

A single standard record identifier (the Immutable Number) will be used by the Digital Collections Manager for matching updated and deleted entries. Because it will not be possible to alert the Digital Collections Manager of deleted entries in the ILMS through this process, the Digital Collections Manager will support a use case for deleting entries. A small number of records in the ILMS do not have an Immutable Number. These will be matched through a separately defined standard record identifier.

From time to time, copies will be identified for digitisation from one of the Library’s other collections, eg a map, manuscript, out-of-print book or a 19th century periodical. The Importing a Catalogue Entry use case must enable the ad hoc importing of catalogue entries in this scenario. This will work by enabling the user to initiate an immutable number search of the ILMS using Z39.50 protocols . It will be necessary to develop and maintain a utility that conducts this search and imports the record to the Digital Collections Manager.

As part of the implementation process, it will be necessary to identify and download the in-scope ILMS metadata. The following table identifies the likely number of records that will be involved and the estimated growth rate per year for the sheet music and pictures collections.

  Release 1 Subsequent releases Annual growth rate
music 76,786 - 3,500
picture 32,812 - 30,000
ad hoc -   500
       
TOTAL  109,599    34,000

This means that, after implementation, new and updated records from the ILMS will be added to the Digital Collections Manager at a rate of approximately 150 records per working day depending on digitisation priorities.

Current status: The ILMS-Digital Collections Manager interface is fully implemented. The task to review the ILMS-Digital Collections Manager interface requirements as part of implementation of the new ILMS has been scheduled.

Pictorial Manager

Pictorial manager is the system currently used by the Library to manage the digitisation of its Picture Collection. The Digital Collections Manager will entirely replace the Pictorial Manager System.

Metadata to be migrated from Pictorial Manager to the Digital Collections Manager are as follows:

It will be necessary to provide this metadata in the standard import format for loading to the Digital Collections Manager during implementation.

The master image files for the bulk of items from the Pictorial Collection are held in Kodak PhotoCD (.pcd) format on CD-ROM. All items digitised since July 2001 will have TIFF masters. The Library plans to gradually migrate the PhotoCD masters to TIFF format after implementation of the Digital Collections Manager. Derivatives (thumbnails and medium resolution) for material with PhotoCD masters will need to be migrated to the Digital Collections Manager. It is assumed that derivatives for material with TIFF masters will be regenerated from the master as part of the loading process. A process will need to be defined to ensure a concordance between current file naming conventions for the PhotoCD masters and their derivatives and the new naming conventions based on the persistent identifier.

Current status: The Pictorial Manager has now been deprecated. The task to migrate the PhotoCD masters to TIFF has not yet been scheduled.

Pictorial Lists

Pictorial lists contain item level metadata such as picture captions. Until recently, the Library has used a range of separate tools (eg, Microsoft Word, Access and Excel) for creating pictorial lists. In March this year, a decision was taken to raise a catalogue entry in the ILMS for all pictorial items in a set, using MARC tags to establish the parent-child relationship. This policy has now been implemented for all new material. Catalogue entries for items in existing pictorial lists will be migrated to the ILMS via Kinetica. There is thus no requirement for the Digital Collections Manager to support the creation of subunit records for pictorial material or the loading of pictorial lists; and there is no requirement to migrate any existing pictorial lists to the Digital Collections Manager.

Current status: The new policy is currently under review because of its impact on cataloguing workloads. There may be a requirement to maintain pictorial lists for some pictorial collections.

Manuscript Finding Aids

Manuscript finding aids contain detailed descriptions of the content of significant manuscript collections. The description will generally not go down to item level; but will give a broad idea of what is contained in specific series or folders. Some specific items may be listed. Finding aids are currently produced in Word and then marked up in HTML for publication on the Library’s web site. A URL linking to the finding aid is inserted in the catalogue entry.

The Library plans to migrate the existing HTML manuscript finding aids to the Encoded Archival Description (EAD) format and to develop a simple web-based form to enable manuscript collection managers to create and maintain finding aids directly in EAD, bypassing Word. A tool for doing this has not yet been identified, and it is possible that the Digital Collections Manager could provide this capability. The Digital Collections Manager will in any case need to support the creation of subunit entries for objects that are digitised on demand and some way of updating the finding aid to make the digital copy accessible through the Library’s resource discovery services. In addition, the Digital Collections Manager will need to support a process to generate subunits from an existing finding aid, for example, when a collection is to be digitised. To do this it will be necessary to develop conversion utilities from EAD to the Digital Collections Manager and vice versa.

Because a manuscript set can be very large, digitisation of a set will generally be treated as a project and resources provided to prepare the required metadata. This may be done through four quite separate processes:

· The lister may add additional descriptive details at a later stage to improve the usability of the finding aid and associated structural maps for subunits, either through the application, or by enhancing the finding aid and re-importing it.

The Digital Collections Manager must be flexible enough to support all of these workflows.

Current status: A separate system has now been implemented for creating and maintaining manuscript finding aids outside of the Digital Collections Manager and it is planned to extend this system to manage finding aids for other collection materials. The requirement to support all of the above workflows is still relevant, but the Finding aid will now be the master source for the subunits it describes. There is thus no requirement to export finding aids from the Digital Collections Manager.

Digitisation Systems and Projects

The Library has two digitisation teams equipped to produce TIFF images using a range of devices and manipulation software. The Library may also, from time to time, submit batches of material to a contractor for digitisation. The Digital Collections Manager will not interface directly with the devices and software used by the digitisers. Technical metadata generated by the digitisation process will be extracted directly from TIFF masters when they are uploaded to the Digital Collections Manager. However, metadata not derivable from the file will need to be recorded by the digitiser. Use cases have been defined that will enable a range of different digitisation workflows to be supported. These include support for uploading of images and recording of metadata as they are scanned through to a full batch submission and processing capability.

Pending implementation of the Digital Collections Manager, metadata is being created and stored in the form of spreadsheets. The following metadata will need to be migrated to the Digital Collections Manager as part of the development process:

Quantities will be as follows:

Project

Sets

Items

Parts
  Units Subunits Units Subunits  
Barton 1 13   1843 4,000
Wills diary - - 1 - 120
Sheet music TBA        
Rare maps - - 25 - 25
Pictures TBA        
Maps TBA        

The files produced by these projects will already be on the System in the form of TIFF masters. It is assumed that the thumbnail and medium resolution versions will be generated by the Digital Collections Manager according to the derivatives profile for the collection.

Large maps are dependent on a delivery system that uses files in Mr Sid format. This category of material has additional derivative-generating and storage requirements that will need to be taken into account when the data and metadata is migrated.

Current status: All of this metadata has now been migrated and the spreadsheets deprecated.

Australian Periodicals 1845-1850

This is a collection of 70,000 Tiff master copies, 7,000 PDF files and 1,,000 OCR text files created to support an online delivery system for 70 ninetenth century Australian periodicals. It is not intended to migrate these copies to the Digital Collections Manager as part of the first release, but There will be a need to migrate them once the system is in production. The delivery system is based on multi-page PDF files. It is recommended that this technical solution be reviewed before the files are migrated. A solution based on structural maps and JPEG derivatives would have been adopted if we had this project to do again now.

Current status: The task to bring these files under DOSS control has not yet been scheduled.

NLA Catalogue

The NLA Catalogue is a public web browser based service that interfaces with the ILMS to provide the Library’s end users with access to catalogue entries. If the catalogue entry is for an item-level unit or subunit that has been digitised, the end user can exploit a persistent identifier for the digital copy to view files and associated metadata through different workflows depending on the record type

If the catalogue entry is for a set, the end user can browse a linked list or finding aid. From the finding aid, the end user can exploit persistent identifiers for digital copies of subunits to view files and associated metadata through different workflows depending on the record type.

No direct interface is required between the NLA Catalogue and the Digital Collections Manager. A Generic Delivery System for Digital Objects is being developed through a separate project that will use metadata stored in the Digital Collections Manager to support this process.

Numerous other resource discovery services access the Library’s catalogue entries through Z39.50 search and retrieval protocols or the sharing of metadata. These services will exploit the same Generic Delivery System as the NLA Catalogue and there is no requirement for them to interface with the Digital Collections Manager.

Current status: The Generic Delivery System has been implemented for sheet music and maps. Tasks to develop the Generic Delivery System to support pictures and manuscripts are also scheduled. The relationship between the NLA Catalogue, the Digital Collections Manager and the Generic Delivery System has been documented further in the paper Architectures for Web Collection Delivery.

Special Materials Catalogues

The NLA Catalogue provides a single point of entry to all of the Library’s collections at the catalogue entry level. There are also separate manuscripts, maps, music, oral history and pictures catalogues. These are subsets of the NLA Catalogue that provide enhanced access to the collections through more extended indexes. The Pictures Catalogue (formerly Images1) is based on a separate metadata repository derived from Pictorial Manager.

The Digital Collections Manager must be capable of interfacing with the Pictures Catalogue by placing an export file in a designated directory on the Library’s LAN at regular intervals containing all picture records that have been added, updated or deleted since the last file was generated. Ideally, the export file for the Pictures Catalogue should be updated at the same time that the Digital Collections Manager is updated, but the Library will accept a service level of once a day.

In order to support the generation of the Pictures Catalogue from the Digital Collections Manager, a full work description will be “passed through” from the ILMS to the Pictures Catalogue.

There may be a future requirement to support additional special materials catalogues for manuscripts, maps, music and oral history through the same process. There may also be a future requirement to support a resource discovery service that provides access to all of the Library’s digitised collections at primary object level. Both of these needs must be capable of being met by the same process as for the Pictures Catalogue.

These specifications have not taken into consideration the data migration or resource discovery needs of collections other than the pictures collection. As delivery systems are developed for the other special materials collections, there may be a requirement to extend the number of work description elements and to migrate all the unit-level records for the collection from the ILMS.

Current status: The interface between the Digital Collections Manager and the Pictures Catalogue has been implemented but will not be needed in the longer term. The Library’s new ILMS will support the delivery of catalogue interfaces for the special materials subsets with the same level of functionality as the current Pictures Catalogue. A decision has been taken to derive all of the catalogue subsets from the ILMS. Therefore, there is no longer a requirement to generate additional special materials catalogues from the Digital Collections Manager.

Orders Management Systems

Where a work needs to be replicated in order to fulfill an order, the Library will create a master copy to the standards set by its Digitisation Projects and upload and manage the master copy and its derivatives through the Digital Collections Manager. It is not intended that the Digital Collections Manager will replace existing manual (or future automated) processes for managing orders. However, it will be possible to trace the progress of orders that require a master to be created, via an Order Number in the process record. A use case has also been defined in this specification for downloading copies of master files to a designated directory for copying to an appropriate carrier or delivery online.

Current Status: This level of support for orders has been implemented.

Rights Management Systems

The Library maintains a range of manual rights management systems in paper and card files. The Digital Collections Manager will enable coded rights management information and expiry dates to be stored at any level in the hierarchy. This information will be sufficient to suppress the display of metadata and/or derivatives in delivery systems or to make display conditional on domain address. However, there will be no support in the first release for linking this rights management information to rights holders or varying it based on user category, as specified in the original Digital Services Project Information Paper. PANDAS has some additional support for recording the details of rights holders. Full support for rights management is planned for a future release. At that time it will be particularly important to align the data models of PANDAS and the Digital Collections Manager to ensure an appropriate rights management framework and language can be set in place to meet the Library’s needs across all applications.

Current Status: High level specifications have been developed for the Rights Management System and it has been scheduled for release after DCM V1.2.

Multimedia Publications

The Library publishes a range of online exhibitions and other multimedia publications via the NLA Web site. These are (or will be) catalogued and have their own archiving and preservation management requirements, which will be addressed either through PANDAS or through the Library’s future website archiving strategy. Where digital copies are created of a work for exhibition or publication purposes, the Library will create a master to the standards set by its Digitisation Projects and upload and manage the master copy and its derivatives through the Digital Collections Manager.

It is not intended for the Digital Collections Manager to operate as a pre-press system for the Library’s multimedia managers or that copies of multimedia publications will be uploaded and managed through the Digital Collections Manager. Multimedia managers will use the persistent identifier of the appropriate derivative or structural map to create their own contextual and interpretative displays. Alternatively, they will create derivatives from a downloaded version of the TIFF master to their own standards, as any other publisher.

Online exhibitions currently exist that may contain images of works in the Library’s collections that have not been digitised and assigned persistent identifiers according to the Library’s current digitisation policies and guidelines. These works will be brought under the Digital Collection Manager’s control as priorities dictate. The Raper Collection is a significant example that may need early attention.

Current Status: Support for the management of digital objects derived from non-NLA collections will be included in DCM V1.2.

Oral History Stocktake Database

This is an in-house system used to manage sound recordings. The Library plans to migrate the functionality supported by this system to the Digital Collections Manager as part of a later phase.

Current Status: This task has not yet been scheduled.

Quadriga Sound System

This is a third party system used to migrate audio material from analogue to digital format or to create and manipulate digital audio files. Master audio files will be stored in BWF format and include a full set of sound preservation metadata. The Quadriga system can import and export metadata to standard XML files. In the first instance, descriptive metadata for works migrated from analogue to digital format via Quadriga will be derived from the Oral History Stocktake Database. The first release of the Digital Collections Manager will not deal with sound. However, there will be a requirement in a later phase to migrate BWF and sound files generated by the Quadriga Sound System to the Digital Collections Manager. Once the Digital Collections Manager has been enhanced to support sound, sound process managers will interact with the Digital Collections Manager to upload BWF and sound files through the range of ad-hoc and batch workflows already defined for works in other formats.

Current Status: This task has not yet been scheduled.

Security System

The Library has a Radius Authentication Server in place for authenticating access by staff to the LAN and other networked services, including http authentication for Intranet services. The Digital Collections Manager will support a Log In use case that enables staff to use the same userid and password for accessing the Digital Collections Manager that is used for accessing other services. Radius supply a Java client for querying the authentication server. The Digital Collections Manager will still need to support a user object for each user that records user roles and profiles but passwords need not be stored. User roles and profiles for the Digital Collections Manager (other than passwords) will be managed by members of the Administrator role.

Current Status: This has been implemented.

Directory System

The Library has a Directory System that records staff names, sections, roles, telephone numbers, email addresses etc. These details are currently stored in an Oracle database but the Library plans to migrate them to an X.500 directory server so that third-party software which requires information about library staff can query the directory using the LDAP protocol. The Digital Collections Manager will not be required to support X.500 or LDAP protocols at this stage but it should be a strategic direction for the Application Framework.

Current Status: No change.

2.2 Summary of System Interface and Data Migration Tasks

3. Logical Data Model

This section defines the objects needing to be supported by the Digital Collections Manager and their properties. An entity-relationship model is used to detail the requirements.

Obl = Obligation (M = mandatory, O = optional, C = conditional)
Occ = Occurrence (S = single, R = repeatable)

Requirements for repeatable fields are included but could be met by storing values in a single field separated by semi-colons.

3.1 Entity-Relationship Diagram

The following sections define each entity in the Entity-Relationship diagram and their required attributes (metadata fields).

Entity-relationship diagram

Dublin Core mappings are provided for the Work and Copy entities in case a decision is taken to support the descriptive properties of these entities in the Application Framework through an implementation of Dublin Core.

3.2 Generic record Status attributes

All entities must have the following record status attributes or equivalent:

Name Definition Obl Occ Datatype Length Form Comments
dateTimeCreated Date and time the record was created M S Numeric 19 Date/Time Oracle date
recordCreator Agent responsible for creating the record M S char 38 Identifier User: userid
dateTimeUpdated Date and time the record was last updated M S Numeric 19 Date/Time Oracle date
recordUpdater Agent responsible for last updating the record M S char 38 Identifier User: userid

3.3 Work

Anything that is a target for collection by the Library. It may be a book, manuscript, map, picture, sheet music, serial or sound recording.

1. A work may be the subunit of just one other work.

2. A work may have as subunit one or more other works.

3. A work must be either a set or item or part.

4. An item must be the subunit of just one set.

5. A subunit must be of just one subunit type.

6. A part must be the subunit of just one item.

7. A work must have at least one copy of copyRole “original”.

8. A work may have two or more copy(s).

DC Name Definition Obl Occ Datatype Length Form Comments

identifier

workPID The persistent identifier assigned to the work M S char 38 Text This can be used as the unique key for this work
altworkPID An alternative persistent identifier assigned to the work at some stage and needing to remain persistent O R char 38 Text  
immutable The unique identifier of the catalogue entry, for records sourced from the ILMS. C S char 38 Text Mandatory if recordCreator=”ILMS”
workNo A unique number associated with a work for collection management purposes O S char 38 Text  
subunitNo A distinguishing number for the subunit intrinsic to the arrangement of the parent work , eg page number. O S char 10 Text I, ii, iii; 1, 2, 3, A, B, C, etc.
standardID A standard identifier such as ISSN, ISBN or ISMN assigned to the work. O R char 38 Text  

type

collection A code identifying the collection to which the work belongs. M S char 7 Code Enumeration: nla.bk (book); nla.ms (manuscript); nla.map (map); nla.mus (music); nla.pic (picture); nla.ser (serial); nla.oh (sound)
form A code identifying the form of the work if different from that implied by collection. O S char 7 Code Enumeration: nla.bk (book); nla.ms (manuscript); nla.map (map); nla.mus (music); nla.pic (picture); nla.ser (serial); nla.oh (sound)
bibLevel A code identifying the bibliographic level of the work M S char 4 Code Enumeration: set, item, part
subunitType A code identifying the type of subunit. M S char 4 Code Codes will vary based on collection and bibliographic level. (See separate subunitType object)
digitalStatus A code identifying whether the work or any of its descendents has been digitized M S char 1 Code Digitised; Non-digitised; Partially digitised.
Local digitalStatusDate Date the digital status was assigned M S num 10 Date Oracle date

title

 

title The name given to the work by the creator or publisher. O S char 254 Text  
otherTitle Another name by which the work is known. O R char 254 Text  

creator

 

creator The entity primarily responsible for creating the work. O S char 254 Text  
creator
Statement
A statement of responsibility associated with the work. O S char 254 Text  
background Biographical or administrative history note O S char 4000 Text  

subject

subject A subject heading that describes the topic of the work O R char 254 Text  
addressee For correspondence the party to which the correspondence is addressed O S char 254 Text  
occupation A term that describes the occupation of the subject. O R char 254 Text  
classification A character string that classifies the work O R char 254 Text  

description

edition A means of identifying works issued at different times with substantially the same content O S char 254 Text  
notes A general note about the work. O S char 4000 Text  
summary A summary of the scope of the work O S char 4000 Text  
contents A note listing the contents of the work O S char 4000 Text  
publisher

 

publisher The publisher, place of publication and/or date of publication of the work. O S char 254 Text  
contributor

 

contributor Another entity who has made a significant contribution to the work. O R char 254 Text  

date

startDate The first date associated with the work, expressed in date form for searching purposes. O S Numeric 4 Date layout: YYYY
endDate The last date associated with the work, expressed in date form for searching purposes. O S Numeric 4 Date layout: YYYY
format extent Physical description of the work O S char 254 Text  

relation

parentPID The persistent identifier assigned to the parent work C S char 38 Text Mandatory if work is a subunit
firstPart For a multi-part item, the persistent identifier of the child work to be used as the thumbnail for the parent work and treated as the first part in displays O S char 38 Text If not used defaults to the first child in the sequence.
series A series of which this work is a part. O R char 254 Text  
childRange A range of workPIDs assigned to children of the work O S char 100 Text Takes the form n/n/n…-n/n/n… The full workPID can be constructed by replacing "/" with "-" and adding the workPID of the top-level antecedent to the string
indexes Identifies indexes listing the contents of the work O S char 254 Text  

coverage

scaleEtc Cartographic mathematical data O R char 254 Text  
coverage The spatial coverage of the work. O R char 254 Text  

rights

access Restrictions on access note O S char 4000 Text  
rights Terms governing use and reproduction of the work O S char 4000 Text  
language language Information about the language of a work O S char 254 Text  
local index A flag indicating to the Delivery System that an index should be generated for children of this work O S char 1 Code Enumeration: (name)

3.4 Subunit Type

A means of assigning a subunit type based on collection and bibliographic level.

1. A subunit type may apply to one or more work(s).

Name Definition Obl Occ Datatype Length Form Comments
form The form to which the profile applies M S char 7 Code Uses the work: form code set
subunitType The subunit type, eg page, cover, series, index M S char 38 Text  
bibLevel A code for the bibliographic level M S char 1 Code Uses the biblevel code set.
isSupported A tag indicating whether this type is relevant to the specified combination of collection and bibliographic level. M S char 1 Boolean yes; no

3.5 Copy

A specific instantiation of a work.

1. A copy must be the instantiation of just one work.

2. A copy may be the input for one or more processe(s).

3. A copy may be the output of one and only one process.

4. A copy may be a file.

DC Name Definition Obl Occ Datatype Length Form Comments

identifier

copyPID A persistent identifier (PID) assigned to the copy M S char 254 Text  
altCopyPID An alternative persistent identifier assigned to the copy at some stage and needing to remain persistent O R char 254 Text  
currentVersion A code indicating whether this is the current version or not M S char 1 Boolean Enumeration: Y,N
version The version number of the copy O S Numeric 10 Pos. integer Enumeration: 1,2,3…
accessionNo A unique number assigned by the Library to a copy when it was accessioned. O S char 38 Text  
callNo A number uniquely identifying the physical location of a copy O S char 254 Text Digital copies may have a physical location if stored offline.
otherNo Other numbers associated with the copy O S char 38 Text  

type

 
copyType A code distinguishing physical from digital copies M S char 1 Code Enumeration: p(physical), d (digital) (note: digital includes physical format digital publications)
copyRole A code distinguishing the role played by a copy in use of the work M S char 2 Code Enumeration: o (Original); t (Thumbnail); v (View); e (Examination), m (Master); c (Co- master); a (Archive); s (Summary); t (Transcript); fa (Finding aid); mf (Microform); sd (Special delivery); p (Print)
bestCopy A code indicating this copy is preferred for delivery purposes O S char 1 Code Enumeration: (Y)
date dateCreated For replicas, the date the copy was created C S Numeric 10 Date Process:dateStarted
source sourceCopy For replicas, the copyPID of the source copy C S char 38 Text Process:inputCopy

description

notes A general note about the copy O S char 4000 Text  
condition A description of the condition of the copy O S char 4000 Text  
relation exhibition Information about exhibitions in which the copy has appeared O S char 4000 Text  

rights

access A code indicating any access conditions O S char 2 Code Unrestricted; Copyright; Indigenous; Adult content
expiry date The year in which an access condition expires C S Numeric 4 Date ISO (YYYY)
format carrier A code indicating the carrier when a work consists of one or more files M S char 10 Code Enumeration: online; CD-R; CD-ROM

Plus

If file

DC Name Definition Obl Occ Datatype Length Form Comments
identifier filename The asset’s file name M S char 254 Text Eg 1234001v.jpg

format

MIMEType The MIME type associated with the file M S char 38 Text Eg image/tiff
fileFormat The format of the file M S char 38 Text Should include format name and version number
fileSize The physical size of the file M S Numeric 38 Positive integer  
compression The compression scheme used to store the file M S char 50 Text Default = “none”
checksum A value generated prior to or during load that is used to monitor file integrity O S char 38 Text  
checksumType the type of checksum used, eg, MD5 C S char 38 Text mandatory if checksum is included
encoding The system used to encode the file content M S char 10 Text default=”unknown”; Enumeration: IBMPC; MAC; utf-8

If image file

DC Name Definition Obl Occ Datatype Length Form Comments

format

x-resolution The x-spatial resolution of the image, eg 300 ppi M S Numeric 10 Positive integer  
y-resolution The y-spatial resolution of the image, eg 300 ppi M S Numeric 10 Positive integer  
bitDepth The pixel depth or tonal resolution of the image M S Numeric 10 Positive integer  
colourSpace designates the colour space of the decompressed image M S char 20 Text Photometric Interpretation in the NISO draft
orientation A code designating the orientation of the image M S char 30 Code portrait; landscape
imageWidth The width of the image in pixels M S Numeric 10 Positive integer horizontal or x dimension
imageLength The length of the image in pixels M S Numeric 10 Positive integer vertical or y dimension
colourProfile A code indicating whether the Colour Profile resides within the image or at an external location O S char 1 Code within image; external
CPLocation The URL of the Colour Profile if stored externally C S char 254 URI Mandatory if ColourProfile=”external”
zoomLevel The maximum zoom level C S Numeric 2 Positive integer Mandatory if a Mr Sid file

If text file

Text file properties will be defined in a future specification.

If sound file

Sound file properties will be defined in a future specification.

If video file

Video file properties will be defined in a future specification.

3.6 Project

A business initiative providing the authority for conducting a set of digitisation or other processes over time.

1. A project may be the rationale for one or more job(s).

Name Definition Obl Occ Datatype Length Form Comments
projectId Unique identifier of the project M S char 10 Identifier  
projectName The name given by the Library to the project M S char 254 Text  
parentProject The projectId of a parent project O S char 10 Text Used to group projects for reporting purposes
projectStatus A code defining the status of the project M S char 15 code Enumeration: Registered, Started, Completed, On hold, Restarted, Aborted
DateStatusAssigned Date status assigned M S char 8 Date ISO YYYYMMDD
dateStarted Date the project was started M S Numeric 8 Date ISO YYYYMMDD
scope A description of the scope of the project 0 S char 4000 Text  
hostComputer Computer and/or operating system in use at the time of the project O S char 4000 Text  
notes General notes O S char 4000 Text  
URI Link to a file containing a full description of the project O S char 254 URI  

3.7 Job

An event that causes a process or set of processes to be performed.

1. A job must generate one or more process(es).

2. A job must have its rationale in just one project.

Name Definition Obl Occ Datatype Length Form Comments
jobNo A running number assigned to the Job by the system M S char 10 Identifier agent-x
agent A code for the agent initiating the job M S char 2 Code Digitisation Term; Imaging Team; Contractor1; etc
projectId Identifier of the project providing the rationale for the job M S char 10 FK  
jobType A code identifying the type of job M S char 2 Code Enumeration: (Unassigned; Upload)
orderNo An additional number for tracking commercial orders O S char 10 Text  
customer customer details for a commercial order O S char 254 Text  
priority a code identifying the priority of the job if not routine. O S char 1 Code Value: High
jobStatus A code designating the current status of the job M S char 15 Code Enumeration: Pending; Requested; Submitted; Started; Completed; Acquitted; Aborted
DateStatusAssigned Date status assigned M S char 8 Date ISO YYYYMMDD
Notes Notes O S char 4000 Text  

3.8 Process

An event that uses a physical or digital copy as input and may result in transformation of the existing copy (eg, through repair, restoration or media transfer) or the creation of new copies (eg through digitisation, microfilming, photocopying or format migration).

1. A process must have as input one and only one copy.

2. A process may have as output one or more copies.

3. A process may have one or more sub-process(es).

4. A process must be initiated by just one job.

Name Definition Obl Occ Datatype Length Form Comments
processed Unique identifier for the process M S char 10 Identifier  
jobNo Identifier of the job which initiated this process O S char 38 Identifier Business-assigned
processType A code itentifying the process type O S char 2 code Enumeration: (Unassigned; Replicate; Ingest; Derive)
processRole A code distinguishing primary from secondary processes M S char 1 code Enumeration: (Primary; Secondary)
inputCopy Identifier of the input copy M S char 254 Identifier  
outputCopy Identifier of the output copy O S char 254 Identifier  
replaceCurrentVersion A means of recording whether to replace the current version O S char 1 Boolean Enumeration: y; n
manipulation A code indicating the form of manipulation applied to a Master to generate a Co-master O R char 1 Code Only applicable to Co-masters. Enumeration: (Cropped; Enhanced; Downsized; Rotated)
processStatus A code defining the status of the process O S char 2 code Enumeration: (Pending; Requested; Submitted; Started; Completed; Failed; Acquitted; Aborted)
dateStatusAssigned Date status assigned M S char 8 Date ISO YYYYMMDD
notes notes O S char 4000 Text  

3.9 Tool Role

A sub-process requiring use of a tool such as a scanner or camera or software.

1. A tool role must be used in just one process.

2. A tool role must use just one tool.

Name Definition Obl Occ Datatype Length Form Comments
ProcessId The process in which this tool role was used. M S Foreign Key      
ToolId The tool used in this tool role. M S Foreign Key      

3.10 Tool

A means of storing information about a tool.

1. A tool may be used in one or more tool role(s).

Name Definition Obl Occ Datatype Length Form Comments
toolCategory A code designating whether the tool is a device or software M S char 1 Code d (device); s (software)
toolType A code designating the type of tool. M S char 38 Code Enumeration: transmission scanner; reflection print scanner; digital still camera; still from video; image capture software; OCR software; other
toolName The manufacturer, name or number and version. M S char 254 Text  
toolResolution Maximum tool optical capture resolution. O S char 254 Text Eg 800x1600 ppi.
notes Notes. O S char 4000 Text  

3.11 Derivative Profile

A means of specifying what derivatives should be generated based on collection.

Name Definition Obl Occ Datatype Length Form Comments
form The form to which the profile applies M S char 7 Code Uses the work:form code set
copyRole A code for the role of the derivative M S char 2 Code t (thumbnail); v (view); e (examination)
isGenerated A tag indicating whether a derivative of this role should be generated for copies of works in the nominated collection M S char 1 Boolean yes; no; maybe
maxDimension The maximum dimension for the derivative (eg, 150 pixels) M S char 4 Numeric (in pixels)
collectionPattern The collection pattern to which the profile applies M S Character 254 Text  
writeFooter A tag indicating whether the persistent identifier should be stamped on the footer of the jpeg M S Character 1 Boolean yes; no
convertOptions Parameters used in the program to create derivatives O S Character 254 Text  

3.12 Collection Pattern

A means of specifying valid patterns for persistent identifiers.

Name Definition Obl Occ Datatype Length Form Comments
pattern A pattern for pi M S Character 254 Text In the form of a Perl regular expression
directoryLevel A number indicating the directory level of the pattern. It is used to tell where the object will be stored in DOSS M S Numeric 2 Positive integer  

3.13 User

A means of recording authority levels and default values for users of the system. Collection managers that do not have a User Profile in the Digital Collections Manager will still be able to login and perform collection manager use cases but they will not be able to perform roles requiring a higher authority level or to set default values until a user record has been created for them.

Name Definition Obl Occ Datatype Length Form Comments
userid The user’s LAN userid M S char 8 Text  
userRole A code setting the user’s authority level O S char 2 Code lister; senior lister; finding aid manager; process manager; curator; administrator
collection The user’s default collection. O S char 7 Code Uses the work:collection code set
Defaults To be defined.            
               
agent The user's default agent. O S Character 2 Code Uses the job:agent code set
project The user's default project. O S IDREF     project instance
software The user's default software O S IDREF     tool instance
device The user's default device. O S IDREF     tool:instance
autoAcquit A flag indicating whether the user is authorised to bypass acquittal O S Char 2 Code Values: Y

Associated Documents (not available online)

A. Collection Manager Use Cases

B. Digitisation use Cases

C. Update Use Cases

D. Administration Use Cases

E. MARC-DCM Conversion Rules

About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us