A paper by Warwick Cathro, Assistant Director-General, Innovation, National Library of Australia, presented at the international conference Archiving Web Resources, Canberra, 9-11 November 2004.
This paper is concerned with the challenge of preserving the outputs of research. Research is undertaken in many contexts. It is undertaken largely by our universities, but also by industry, government agencies and independent scholars. Research generates primary outputs (such as data sets, images, video files and sound recordings) and secondary outputs (such as books, pre-prints, journal articles, conference papers, theses, technical reports, unpublished papers and web sites) which interpret and summarise the research findings.
The secondary outputs have of course traditionally been preserved, in printed form, in our library collections. However the majority of these outputs are now available in electronic form through electronic journals, personal web sites of academics, and institutional and departmental web sites. This change has raised issues about how these documents should best be managed for long term access.
The primary outputs are typically managed by academics on faculty servers, or (in the case of some research disciplines) by national and international data centres. The long term preservation of this data is threatened by inadequate data management practices, by the steady emergence of new file formats and by the technological obsolescence which accompanies this process.
You may be asking “is this topic in scope for an “Archiving Web Resources” conference? What exactly are “Web resources”? I will assume that a “web resource” is any document or information resource that can be accessed via the World Wide Web, whether or not that resource is protected by an authentication barrier. For those primary and secondary outputs of research that are in digital form, a significant proportion would meet the definition of “web resources”. Many of them form part of the “deep web”.
This paper will examine the key issues and challenges in preserving the outputs of research. It will survey recent responses to these challenges, especially in the United Kingdom and Australia, and will suggest additional steps which are required to meet the challenges.
How well are the outputs of research being preserved, and what factors are hindering the preservation process?
A useful investigation was undertaken early this year by the University Libraries at the University of California Santa Barbara (UCSB). The project involved structured interviews and roundtable discussions with faculty members (Pritchard, 2004).
UCSB is typical of many universities in which there is no centralised academic computing organization. Individual colleges and departments have their own servers and their own support for research and learning data. This investigation found that:
It is worth commenting, I think, on the perception by most researchers that their data has negligible value after five years. I would argue that researchers are sometimes wrong in this belief. There are many categories of research data that are likely to have long term value. To identify just a few:
Another observation about the issues in managing research outputs was made by Clifford Lynch in his paper on institutional repositories, published early last year.
Lynch argued that the researchers in universities have the skills and apparatus to create new knowledge, but they are not necessarily best placed to “maintain the record of this process of creation”:
Most individual faculty lack the time, resources, or expertise to ensure preservation of their own scholarly work even in the short term, and clearly can't do it in the long term that extends beyond their careers ... our university networks are riddled with vulnerable faculty machines intended to serve as points of distribution for scholarly works. And faculty create content at risk because they typically do not back it up appropriately, ensure its integrity… and curate it properly
Another important and recent relevant investigation was the “e-science curation report” undertaken in the United Kingdom (Lord, 2003). The report presents an audit of the current provision for curation of primary research data in the United Kingdom. “Curation” was defined as “the activity of managing and promoting the use of data from its point of creation to ensure that it is fit for contemporary purpose and available [for as long as it is needed] for discovery and re-use”. “Archiving” was seen as an activity within the curation process, and “preservation” as an activity within the archiving process.
Among the findings of the report were:
In 2001 the Social Sciences and Humanities Research Council of Canada, in conjunction with the National Archives of Canada, gathered information from the research community, archivists and librarians concerning the need for a national research data archiving service. Among the findings were that “a significant number of researchers are not convinced or have a limited understanding of the value of a research data archive”.
The key issues that have been highlighted by these investigations are:
It is heartening to observe that a number of significant responses to these challenges have been initiated during the past two years.
One of the key issues was the lack of any infrastructure to support academics in dealing with the problem of preserving their research outputs. In this area, much recent discussion has focussed on the development of “institutional repositories” as an appropriate response.
In his paper on institutional repositories, Lynch asserted that:
Only an institutionally based approach to managing these data resources, which operates in alignment with what the faculty at each individual institution are actually doing, can provide a comprehensive dissemination and preservation mechanism for the data that supports the new scholarship for the digital world.
Lynch noted that tools (such as the DSpace software from Massachusetts Institute of Technology, the Fedora software from Cornell University and the University of Virginia, and the E-prints software from the University of Southampton) were now emerging to assist universities with solutions to the repository challenge. He envisioned the institutional repository as “most essentially an organizational commitment to the stewardship of these digital materials [those created by the institution and its community members] including long-term preservation where appropriate”. He stressed that the repository is “not simply a fixed set of software and hardware”: it includes management processes such as “migration of digital content from one set of technologies to the next as part of the organizational commitment”.
The institutional repository concept owes much to the work undertaken on behalf of ISO by the Consultative Committee for Space Data Systems, in developing the Reference Model for an Open Archival Information System (OAIS). That model specifies a framework of terms and concepts, and a set of functions and management procedures. It was designed “to create a consensus on what is required for an archive to provide a permanent, or indefinite long-term, preservation of digital information”.
In Australia, a number of universities have been using the Southampton E-prints software for the last couple of years. Some others, most notably the Australian National University, are implementing DSpace as their repository solution.
In a significant development, the Arrow Project, funded by the Australian Government’s Systemic Infrastructure Initiative (see below), will be using the VITAL software from VTLS Inc., which is based on the open source Fedora software. The Arrow Project is funding identified enhancements to VITAL, and these new modules will become available within the open source paradigm.
As a partner in the Arrow Project, the National Library of Australia will be trialling the enhanced VITAL software. The Library is in an interesting position because it already has a digital services architecture which supports both the PANDORA Archive and the digital content which the Library creates through its digitisation workflows. This architecture has been described by Cathro and Boston (2003).
The Library currently creates digital content by harvesting web sites using the PANDAS software, and by digitising collection items using its Digital Collection Manager software. However, the Library does not currently have a satisfactory solution for a third ingestion method, in which digital content is submitted or deposited by publishers. Trialling the VITAL software will give the Library an opportunity to evaluate a potential solution for this ingestion method, and an opportunity to use this information in a reassessment of the medium term future of its digital collection architecture.
So far we have discussed a key potential institutional response to the challenges of preserving the outputs of research. Another of the issues which we identified was the lack of national-level structures to support the data retention and curation process.
In the United Kingdom, a number of actions have been either initiated or prominently advocated to start to put these national-level structures in place.
First, JISC has taken the initiative to establish a national Digital Curation Centre. This Centre was formally launched last month. Its funding (£1 million per year) has been provided by JISC and the e-Science Core Programme. It will be run by a consortium centred at the University of Edinburgh. It aims to support UK institutions in the storage, management and preservation of digital data across the life-cycle of scholarly interest.
The Centre will deliver a number of outputs (Burnhill, 2004), including:
Second, the e-Science curation report called for a number of strategic actions to address the issues that it identified. Some of its key recommendations were:
Third, national action was advocated earlier this year in a report of the House of Commons Science and Technology Committee. Though this Committee was dealing specifically with science and technology, its arguments apply equally to all research fields. The Committee:
(It should be observed that while the House of Commons Committee dealt with the data retention issue, its major concern was to examine the issue of current access to research findings by the academic community and wider public, and to examine the impact of the new publishing trends on the scientific process).
In the United States, the major national program is undoubtedly the National Digital Information Infrastructure and Preservation Program, which has been funded through a US$99.8 million appropriation, and is being led by the Library of Congress. Last month the Library of Congress announced grants totalling US$14.9 million to eight projects aimed at preserving at-risk digital materials of significant cultural and historical value. An example is the grant of US$2.6 million to the University of California at Santa Barbara and Stanford University for the development of a National Geospatial Federated Digital Repository. Most of these grants are supporting the preservation of “research resources” rather than the “research outputs” which are the primary focus of this paper.
Early this year the US National Science Board established a task force on “Long-lived Data Collection”. The Task Force will examine questions such as:
In Australia, the Government has initiated a response through its Systemic Infrastructure Initiative, which is aimed at strengthening Australia’s research capability by funding improvements to research information infrastructure. An advisory committee established by the Government recommended in early 2003 that the development of “institutional repositories” become a key strategy for advancing these goals. The Government subsequently announced the allocation of A$12 million to four projects which would support the development of institutional repositories and of middleware services to improve the management of and access to research output.
One of these projects is the Australian Partnership for Sustainable Repositories (APSR). This project, led by the Australian National University (ANU), aims to develop demonstrator repositories and support continuity and sustainability of digital collections, including research data sets. The demonstrator repositories will be developed at ANU, University of Sydney and University of Queensland. The National Library of Australia, which is also partner in APSR, will take a leading role in the development of advisory services to higher education institutions relating to digital continuity. These services will have a similar objective to those planned by the UK Digital Curation Centre.
As we have already observed, another of the projects is ARROW (Australian Research Repositories Online to the World). This project, led by Monash University, aims to identify and test software to support institutional repositories at the member sites, and to develop and test a national resource discovery service using metadata harvested from the institutional repositories.
The implementation of a successful infrastructure for preserving research outputs is a complex matter. It will require more than well managed institutional repositories, and the resourcing of national-level programs. Success will also depend on productive collaboration between all of the stakeholders: academics, university libraries, university IT centres, discipline-based data centres, and appropriate organisations (such as national libraries) with preservation skills and experience.
As Beth Dulabahn said in her abstract for this Conference:
The volume, diversity and complexity of digital content compels a distributed, collaborative approach that enables individual organisations to manage content while also sharing best practices for preservation.
It is clear that significant unanswered questions remain about the roles and responsibilities of the various stakeholders. For example:
We have already noted that academics do not have a high level of awareness of long term data preservation issues. Other studies have reported on the relatively poor success rates so far in persuading academics to deposit their secondary research outputs into institutional repositories.
For these reasons it will be interesting to observe the progress of the APSR testbed at the University of Sydney. That university lacks a strong central IT service. Its approach to the APSR Project involves a strong partnership between the university library, selected faculty, and cross faculty support services such as the Spatial Science Innovation Unit. The University is working to implement a “devolved repository” model, focussed on the deposit of primary research outputs, that will also meet the challenges of sustainability. The strong involvement of academics and the challenge of achieving data longevity within a devolved model make this an interesting case study.
By contrast, the leading role undertaken by Monash University in the Arrow Project involves a strong partnership between the university library and the university IT centre. The Project is aiming to implement a central physical repository (based on the VITAL solution) with centrally managed processes to encourage the deposit of secondary research outputs.
The UK’s E-Science Curation report raised the issue of the role of university libraries in curating primary research data. The investigators reported that they had found “conflicting views on this question, both views stated with emphasis”. One view was that the university library is the natural organisation to take on the role of curating primary research data, while others argued that the libraries lacked the appropriate mindset and skills to undertake this role.
Harboe-Ree and Treloar have also commented on the roles of university libraries. They believe that academic libraries should be well positioned to exercise a leadership role, either as advisors, managers or practitioners, but they recognise that libraries are only one of the groups within a university that may see themselves as partial owners of this territory. They noted particular barriers to the involvement of university libraries, including:
In my view, university libraries are well placed to play a major role in this field. They have relationships with many of the other key stakeholders, including academics, university IT centres and national libraries.
What is the appropriate role for national libraries in responding to the challenge of preserving the outputs of research? We have seen that the UK House of Commons Science and Technology Committee identified a potential role for the British Library, particularly as a “safety net” for data not stored in any other institutional repository. On the other hand, the E-Science Curation report did not mention the possibility of the British Library playing a role.
It seems to me that national libraries have three potential roles in this challenge:
In Australia, the National Library is currently undertaking all three of these roles. It has exercised the third role in two ways. First, it continues to publish and support the PADI (Preserving Access to Digital Information) web site, with its wealth of information on digital preservation policies, strategies, guidelines, topics and projects.
Second, the Library is responsible for the “Digital Continuity Programme” within the APSR Project, and has a full time officer engaged in this project. This project will provide a focus for the National Library in its engagement with the university community. The Library hopes to gain a much better understanding of the challenge faced by the universities, and to find practical mechanisms for sharing its own experience and skills in the field of digital preservation. The Library is committed to working with the universities to meet the challenge of preserving the outputs of research.
Our goal is to ensure that researchers have access to an effective infrastructure to support the sound management of the outputs of their research into the future. To achieve that goal, we will need recognition and support from governments. We will also need to build a collaborative framework linking researchers with expertise in managing and accessing digital information resources. University libraries, university computing centres and national libraries have roles to play in this framework.
Burnhill, Peter. So who’s that new kid on the block? DPC Forum: Digital preservation – the global context, 23 June 2004. http://www.dpconline.org/graphics/events/for040623.html
Cathro, Warwick and Boston, Tony. Development of a digital services architecture at the National Library of Australia. http://www.nla.gov.au/nla/staffpaper/2003/cathro1.html
Consultative Committee for Space Data Systems. Reference Model for an Open Archival Information System (OAIS). Blue Book. Issue 1. January 2002. (This Recommendation has been adopted as ISO 14721:2003). http://www.ccsds.org/CCSDS/recommandreports.html#interchange
Harboe-Ree, Cathrine and Treloar, Andrew. Connecting the dots down under: towards an institutional integrated approach to digital content management. High energy physics libraries webzine, Issue 9 (February 2004). http://library.cern.ch/HEPLW/9/papers/1/
Lord, Philip and MacDonald, Alison. E-Science curation report. Data curation for e-Science in the UK: an audit to establish requirements for future curation and provision / prepared for the JISC Committee for the Support of Research by Philip Lord and Alison Macdonald. http://www.jisc.ac.uk/uploaded_documents/e-ScienceReportFinal.pdf
Lynch, Clifford A. Institutional repositories: essential infrastructure for scholarship in the digital age. ARL Bimonthly report 226, February 2003. http://www.arl.org/newsltr/226/ir.html
Pritchard, Sarah, and Anand, Smiti. UCSB campus informatics: collaboration for knowledge management. http://www.cni.org/tfms/2004a.spring/abstracts/PB-ucsb-pritchard.html
Social Sciences and Humanities Research Council of Canada. National research data archive consultation, Phase One: needs assessment report, May 2001. http://www.sshrc.ca/web/whatsnew/initiatives/data_archives_e.asp
United Kingdom. Parliament. House of Commons. Select Committee on Science and Technology. Tenth report. “Scientific publications: free for all?” 20 July 2004. http://www.publications.parliament.uk/pa/cm/cmsctech.htm
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |