Weibel Lines

Blue Ribbon Task Force on Digital Preservation - 1

The NSF Blue Ribbon Task Force on Digital Preservation has released its Interim report, which sets the conceptual framework for a second report in about a year.  That report will complete the NSF brief, with recommendations and strategies for an overall approach to preserving public digital information.  This initial report is a fine piece of work that will inform the national agenda on digital preservation for the foreseeable future.  It synthesizes a broad spectrum of testimony by practitioners and embeds that testimony within principles of sustainability and business models that are the central core of the problem.  And there is a lot of data, as well... tables summarizing notable preservation activities in many countries that are navigating uncharted waters.

In 2007 we passed a major inflection point.  According to IDC analysts, quoted in the report (page 10),

the quantity of digital information created, captured, or replicated exceeded available storage capacity.

And the discrepancy is projected to grow, leaving half of all digital information without a permanent home by 2011 [seen through the perspective of someone who has jettisoned a third of his life ‘cargo’ in the past year, this is not an unalloyed tragedy]. The question becomes one of selection and election… How do we choose what to save, and how do we pay for it?

The features of the vulnerable datasets give texture to what otherwise might easily be a dull abstraction (however important).   The task force entertained testimony from managers of data with wildly different characteristics and value propositions, including:
•    Scientific data sets, including the Protein Data Bank and the ICSPR data repository
•    Commercial endeavors  (Boeing, RealNetworks, Microsoft)
•    Libraries
•    Think tanks and educational organizations (IDC, the Wharton School of Business)
•    Public Television and the Academy of Motion Picture Arts and Sciences
•    The Internet Archive , Portico,  and LOCKSS

To bring the issue home (literally)… do you know who is charged with the responsibility for looking after all those great public television shows you grew up with?  No one… not stations, not the creators of the work, not the Corporation for Public Broadcasting, not PBS… no one.   In fact, the pervading climate is one of resistance to preservation, or at least to the expenditure of scarce resources on then when the now is hard enough to pay for.  At least for some of these materials, there are resale markets (be part of the solution… go buy some DVDs from your local PBS station!).  The rest is teetering on the edge of digital oblivion… or already gone.  And this is the stuff that has presence in the public mind.

The problem gets harder for more obscure data (that is, almost everything else). The then versus now issue is core.  Though a portion of this report speaks of the tension between 'dividing the pie' to include preservation (and doing less of something else), versus 'growing the pie', this seems a hollow distinction.  Make the pie bigger, and you still have an allocation problem.  The question will always be what part of the budget must we allocate for preservation, and, more to the point, what portion of current production must be foregone in order to do so?  

Beyond the cost issues there lie systemic organizational and social issues which will be difficult to solve, paraphrased here from the executive summary:
•    Funding models which do not match long term needs
•    Unclear preservation responsibilities among various stakeholders
•    Inadequate incentives for collaboration among the stakeholders
•    Complacency – we shouldn’t feel desperate yet
•    Fear – we feel too desperate – there is no solution!

This report is a lucid characterization of the problem, and documents the current state of affairs well.  It deserves careful study (my own efforts in that direction will be noted in additional posts here).  The next part – actionable strategies and recommendations -- is going to be far harder still.

Congratulations to the task force on their achievements to date.
---------------

I've lost the view that afforded images of The Mountain like this for the paltry cost of turning my head as I awoke in the morning.  I wonder if I'll ever have it again?  Seattle is a great city... even (especially?) kneeling in the snow.

Digital Preservation: How will we know we've succeeded?

My esteemed colleague of many years, Priscilla Caplan, offers a perspective on the past decade of digital preservation in a recently published editorial in Library Hi Tech.  Priscilla is among our community's most able leaders, unafraid of the future and eager to help create it.  She has a depth of knowledge on the subject of digital preservation far deeper than my own, and shares the perspective of that experience in this editorial. 
Part of her theme is familiar to me from my own efforts on the international infrastructure and standardization circuit.  She alludes to the relative strength in Europe of centralized support for infrastructure development and broad-based training for information professionals who become the reservoir of expertise on which our standards float (or fail to). 
In the states, by and large, there is greater focus on competitive research grants, a different set of incentives with which our discipline has never been entirely at home.
I can't say which approach is most productive, in spite of struggling with the difference for a long time searching for sustainability models for the Dublin Core Metadata Initiative.  As it happens, I am currently engaged in thinking about sustainability in connection with the NSF DataNet solicitation.  Different domains of digital preservation, but a lot of the same problems, and the National Science Foundation is not lobbing soft pitches here, either.
Reading Priscilla's editorial, I found myself looking for evidence... does the idiom of centralized infrastructure support and aggressive promulgation of training result in more effective curation?  It is clear that Priscilla thinks so, and I have no evidence that she is wrong... or right, for that matter.  Quite possibly this is because of my own lack of knowledge, but intuition tells me it is probably too early in the digital content era to be very sure of either conclusion.   So, in another decade, will we know?  What evidence will we bring to bear on the question?

Thanks to my colleague, Brian Lavoie, for bringing this editorial to my attention.

-----

The view from my room with a view, 3800 Aurora Avenue N., Seattle