Checksum generation and verification recommendations

Revised September 2022 Discussed and Adopted November 2022

The policy was established and reviewed collaboratively by the Digital Preservation Working Group from February through September 2022.

Scope

Built on consensus and discussion in the Digital Preservation Working Group, the policy is intended to apply to archival, library, and museum material digitized in-house or by a vendor according to the specifications stipulated by the Center or a Partner organization. Moreover, the policy should also inform born-digital accessions or accretions accepted by the Center or Partners from the onset of stewardship of materials.

Best practices/literature review

• • [Levels of Digital Preservation (https://osf.io/2mkwx/)
  • [Fixity and Checksums (https://www.dpconline.org/handbook/technical-solutions-and-tools/fixity-and-checksums) -- relies heavily on NDSA guidelines
  • [Importance of Checksums – advice from Archives New Zealand (https://www.archives.govt.nz/manage-information/how-to-manage-your-information/digital/checksums/the-importance-of-checksums)
  • [“What is Fixity, and When Should I Be Checking It?” (2014) (https://www.digitalpreservation.gov/documents/NDSA-Fixity-Guidance-Report-final100214)
  • [“Technical Guidelines for Digitizing Cultural Heritage Materials, Third Edition” (draft 2022) Federal Agencies Digital Guidelines Initiative (FADGI) Digitization Guidelines (https://digitizationguidelines.gov/guidelines/DRAFT%20Technical%20Guidelines%20for%20Digitizing%20Cultural%20Heritage%20Materials%20-%203rd%20Edition.pdf)

Fixity check tools

• • [Karen’s Directory Printer (https://www.karenware.com/powertools/karens-directory-printer)
  • [Application only available on Windows OS
  • [Teracopy (https://www.codesector.com/teracopy)
  • [Checksum by Corz (https://corz.org/windows/software/checksum/)
  • [Fixity Pro (https://www.weareavp.com/products/fixity-pro/)
  • Command line using a programming language (e.g. Python, etc.)

Other helpful tools and resources

• • [DROID (https://www.nationalarchives.gov.uk/information-management/manage-information/preserving-digital-records/droid/)
  • [PRONOM – The Technical Registry (https://www.nationalarchives.gov.uk/PRONOM/)
  • [Digital POWRR (https://digitalpowrr.niu.edu/)
  • [Community Owned digital Preservation Tool Registry (COPTR) (https://coptr.digipres.org/index.php/Main_Page)

Recommendations related to digitized files (vendor or in-house)

• • Identify file format for preservation masters and access copies
  • Verify integrity information when moving or copying preservation master files
  • Verify integrity information in response to specific events or activities related to preservation master files
  • Events can include embedding metadata, derivative creation, file re-organization, etc.
  • Generate fixity after receiving or digitizing preservation master files
  • When possible, request vendor fixity check prior to physical or online delivery of files
  • Depending on duration between receipt and deposit, periodically recheck preservation masters

Recommendations related to born-digital files (upon accretion or collection)

• Verify integrity information if it has been provided with donation or accretion of digital files
• Generate integrity information if not provided with donation or accretion of digital files
• When accepting donations or accretions, in addition to fixity check:
• Virus check all donated material; isolate files/directories/folders for quarantine as needed
• Create manifest/inventory for donation, including the following information: file size, format, file extension, last modified date, file path/hierarchy
• Use write-blockers when working with original media and, when possible, work from a copy of donated files

Recommendations related to files in the digital repository (Rosetta)

• • Verify/generate integrity information when depositing files into the digital repository
  • Run scheduled fixity checks in the repository – annual or more frequent as needed