Staff Papers

Migration Trials: Migrating publications on floppy disk to CD-R

Paper by Colin Webb, Manager, Information Preservation and Deborah Woodyard, Electronic Preservation presented at the National Consultative Meeting on Physical Format Electronic Publications, held at the National Library on 27-28 October 1997.


Introduction

The collections of the National Library of Australia contain many publications on computer disks which require innovative preservation responses. Unless they are managed carefully, media instability and technological obsolescence will limit the potential for ongoing accessibility.

CD-ROMs are still a current and popular technology with relatively stable media, so attention has first been focussed on the floppy disks in the collection. Copying and migration of information on computer disks is likely to be the most useful preservation method currently available. To examine how we might implement these strategies a series of trials will be conducted. The results of the first trial are described below.

Aim

The overall aim of this project was to migrate information from published Australian floppy disks to CD-R (recordable compact disc), to investigate the usefulness of this migration pathway.

It became clear that this was more an exercise in copying data from one medium to another, which is only a part of the process of migration, rather than complete migration across technologies. This has a valid purpose, considering the average life of a floppy disk is estimated at less than ten years. However it does not adequately address the problems of changing operating systems or associated software dependencies. Further research is needed to establish reliable preservation pathways that will work longer term.

Method

1. Identifying and locating disks

Searching the NLA ILMS (Integrated Library Management System) was chosen as the most useful way of identifying and locating disks in the general collections. (The search terms and results are presented in Note ).

This produced a total of 1,539 records and appears to have captured a majority of items. Many of these records, however were for overseas publications, including over 400 records for material mounted on the ADONIS system.

The only serials located in this search were those where the primary material was a computer file. Print serials with accompanying disks were not located.

2. Selecting representative sample for trial

To determine the various complications that may be encountered in transferring publications to CD-R, a representative sample had to be chosen for testing. A survey was carried out of all Australian material that may be considered for long-term access, located in the general collection.

From the 405 surveyed titles found to contain computer disks, 66 containing floppy disks were chosen as a representative sample for a migration trial . The publications were chosen to represent a number of variables:

· operating system (eg DOS, Mac OS),

· hardware requirements (eg processor speed),

· software requirements (eg Word 1.1, Word 6 etc),

· disk size (5 1/4", 3 1/2"),

· number of disks per title (some titles contained multiple disks),

· age (up to 12 years old),

· proportion of publication on floppy disk (whole publication, supplementary material, set-up software for CD-ROM).

3. The copying process

The copying process devised in conjunction with the Information Technology Branch and the operators of the CD-R writing equipment in NLA (the Photographics section at the time of the trial), was to transfer the data from the original floppy disks to a network drive, download the data to the hard drive of the CD-R host PC, and then write to CD-R. The entire process (with the exception of file comparison) was carried out with hardware and software currently available in the Library.

Of the 66 items trialed:

· 22 items (all PC) were functional before and after copying to CD-R.

· 5 items (all Mac) were functional before, but not after, copying to CD-R.

· 23 items were found to be untestable (and unusable) at the first step, as the Library was unable to locate the necessary hardware or operating software

· 1 item was found to be untestable (and unusable) at the first step because the sliding disk cover had been fouled by a security tag

· 1 item was found at the 3rd step (file scanning) to have accumulated so many errors that it could not be read. (This also affected another item, but a backup disk had been supplied which was still readable.)

· 12 items were found at the 4th step not have the required software to function properly. These were written to the CD-R, although they could not be fully tested afterwards. However, file comparison results, and tests with similar software to that required, suggested that these items had copied faithfully

· 1 item was found at the 4th step to be blank.

Other problems detected included incorrect path names in supplied disk instructions, and material of such an obscure technical nature that it was difficult to tell whether copying had successfully maintained functionality or not.

A detailed description of the copying process follows.

i. Select appropriate hardware and operating systems, eg PC or Mac, with correct size of floppy disk drive.

As noted above, a significant number of older items failed at this step, as the Library has not been able to maintain the necessary range of support facilities for all material entering the collections - in some cases they had never been available.

ii. Virus check of original floppy disks

Using the latest version of scanning software in the Library, no viruses were found.

iii. Physical disk scan to look for media deterioration of original floppy disks

PC based material was checked with Scandisk, Mac material was checked with Nortons Utilities Disk Doctor. Consistent backup file errors on older Mac disks may have reflected a change in the Mac operating system rather than a high level of disk errors.

iv. Install and run from floppy disks

At this stage of testing further significant failures were discovered: one disk being found to be blank, and 12 disks did not have the full range of required software or hardware for full functionality.

v. Copy contents of floppy disks to a network drive

This was successful for all 40 items tested.

vi. Run comparison software to check for differences between original and copy

This was successful for all PC items. Software for this step for Mac material could not be located so they were not tested.

vii. Set as "Read Only" on network drive

This was mainly to see if the files could continue to function as required in a "read only" state, as they would be on the CD-R.

viii. Install and run from network drive

Four items, hard coded to work only from an A: drive, had to be copied back from the network drive to a floppy disk to run; one of these even required renaming of the floppy disk, before it would run.

ix. Unset "Read Only"

x. Organise titles in appropriate file structure using appropriate naming convention

Files were organised into a directory structure according to the ISO 9660 standard. The naming convention would not accept the original file names from the publications, because of illegal characters or extended names (especially with Mac files).

Following a suggestion on file naming from the National Archives of Canada, it was decided to use a "call number" system for naming and arranging files on the CD-R.

xi. Create and attach documents with appropriate details such as installation instructions

A general file was created in ASCII text to describe what is on the CD-R, how files are arranged, and where detailed instructions can be found for each item.

Individual ASCII text "read.me" documents were created for each item, containing:

· call number

· hardware required

· software required

· notes to assist installation

· file name

· number of files

· storage size (in MB)

· original format and number of disks

· title

· publisher

· existence of accompanying materials

· content description (eg text, database, software)

· date the original was published (or copyrighted)

· whether there has been a print version of electronic contents located

· any copyright statement from the publisher

In future the text files should also document the date and new format of each migration.

xii. Write to CD-R

This part of the procedure (copying from the network drive to a local PC, automated checking of file names, and writing the CD-R) took approximately 45 minutes, during which roughly 1,900 files, containing 164 MB, were handled.

xiii Run comparison software to check for differences between original and CD-R copy

Comparison checking suggested that PC material had been copied without loss or changes from the original. Mac material could not be tested due to lack of comparison software.

xiv. Install and run from CD-R

The PC files did not appear to have any problems when they were installed and run from the CD-R, but the Mac files did not work.

4. Technical problem solving.

Further investigation has shown that Mac files need to be written to a CD-R using the HFS file system, which will require different CD writing software. This suggests that it may be simplest to copy Mac and PC files onto separate CD-Rs.

5. Time requirements

It was very difficult to estimate processing times in the trial, as so many steps required investigation, consultation and trial and error. However, as an indication, steps a-h required an average of 40 minutes per item tested; later steps are estimated to require an average of 35 minutes per item when being processed in a large batch (the set up time for some steps being significant whether a small or large number of items is involved). These are very rough estimates and do not take account of problems that often seem to arise, which can take many hours or days to solve, and in some cases may be insoluble.

6. Further action required

i. Continue tests for Macintosh materials

The migration trial conducted to date was not successful for Macintosh disks. We need to do more work to establish procedures and resources that will work for this important class of material, even if it means adopting parallel procedures.

ii. Assess priorities among "unpreservable" items and explore options

We need to assess whether any of the material we are currently unable to access is a high priority for preservation action, requiring migration or emulation. For any items identified, we will need to investigate and trial the available options.

iii. Outline preservation guidelines for selection and subsequent procedures

Once we have a better understanding of what we can hope to preserve, we will need to develop guidelines for preservation selection or preservation priorities, to complement the acquisition guidelines already in use.

iv. Define metadata and record keeping procedures

It has become apparent that more information than is held in catalogue records is needed for successful preservation management of this material.

The Library is planning to include the extra metadata and information required in the PANDORA database which is being created primarily for online publications. Using the same database for all electronic collection materials will help simplify the management of metadata. Decisions still need to be made on how, where, and when this metadata entry is best done.

v. Upgrade the facilities for preservation management

In the short term the main need is to acquire file comparison software for PC and Mac (when located) that is required to check the effectiveness of copying.

vi. Establish ongoing copying procedures for floppy disks

For the kinds of material successfully handled in the trial, copying to CD-R should be undertaken as a routine operation. It will be most efficient to batch the intake of floppies for copying each year, copying all the Mac files to one disk and the PC files to another, (subject to result of further testing of procedures for Mac files).

vii. Define access and use procedures

Storage and access arrangements for the newly created CD-Rs need to be finalised. Storage is expected to include backup provisions; controlled access arrangements are not expected to be significantly different from current practice, although it will be necessary to take account of the changed access format.

viii. Continue trials on other forms of migration

Many questions about migration will remain unanswered until further testing is undertaken to examine the feasibility of conversion between operating systems, and other, more demanding, software dependencies, to maintain accessibility. Even if some data elements can be successfully transferred, the effect on the look and feel of the publication will need to be assessed.

ix. Report the results of this trial

It is important for this basic applied research to be seen in the context of cooperative action on preservation of physical format electronic publications. We need to maintain and develop dialogue with other key players.

Conclusion

The testing has produced useful results. Not all the material in the sample was successfully migrated, but this was expected. We now have a better idea of the issues involved, and of the hardware, software and copying processes required to use and maintain the collection.

We are very interested in discussing the implications of such trials, both here and in other institutions. Comments can be addressed to Colin Webb, Manager, Information Preservation, email cwebb@nla.gov.au or Deborah Woodyard, Electronic Preservation, email dwoodyar@nla.gov.au .


Last updated 16 October 1997

Top

About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us