Benchmarks for digital image quality

Establishing benchmarks

The benchmarks for digital image quality should be determined based on the purpose of the digitisation project and the characteristics of the records being digitised.

Key factors include:

• Fit for purpose: The level of quality required should align with business and stakeholder needs. Sometimes lower quality images may suffice, such as for internet access to historical records that still exist in their original format. Alternatively, for detailed records, like maps with fine notations, optimal quality is necessary to maintain accuracy and legibility.
• Essential characteristics: Certain elements of the original records, such as colour or annotations, may be vital to retain the meaning and evidential value of the document. For example, in legal cases, different coloured pens may signify different information, and reproducing these colours accurately in the digitised image is critical.

It is essential to define benchmarks that ensure the digital images are fit for purpose, preserve the essential characteristics of the original records, and meet all stakeholder needs.

Note: If records are required as State archives, your organisation must contact Museums of History NSW to discuss suitable benchmarks for image quality for masters. 

Some technical aspects of the quality of images can be evaluated using software.

For example:

Noise in digital images is caused by random pixel fluctuations and may make images appear grainy. Software can be used to measure the level of noise in images, to check that it is minimised to an acceptable level.

Checking for digital image quality

The process of quality checking for digital images should be integrated into the workflow to ensure consistency and accuracy. 

Below are the critical areas to examine during the quality check:

• Smallest detail captured
  Ensure that even the smallest details, such as fine text (small type sizes), punctuation marks and decimal points, are clearly legible in the digital image.
• Completeness of detail
  Check for the completeness of the captured details, ensuring there are no missing segments, broken characters, or pixels that could compromise the image quality.
• Dimensional accuracy
  The dimensions of the digital image should be compared with the original record to ensure it accurately represents the original document's size.
• Scanner-generated speckle
  Verify that no scanner-generated speckle is present in the digital image, which would not be found in the original document.
• Completeness of image area
  Ensure that the entire image area is captured correctly, with no information missing from the edges of the image.
• Colours and tones
  Assess the colours and tonal values in the digital image, making sure they match the original document. This includes verifying:
  • density of solid black areas
  • correct colour reproduction (for example, ensuring colours are captured in colour where necessary)
  • correctness of tonal values and colour balances
  • brightness and contrast levels.
• Sharpness of the image
  The sharpness of the digital image should closely match the original. This involves checking for over-sharpening, unnatural appearance, or any halos around dark edges.
• Accuracy of OCR
  If Optical Character Recognition (OCR) is used, ensure the accuracy of the captured text. Any discrepancies in text recognition can compromise the image's value and usefulness.
• File formats and settings
  Confirm that images are saved in the correct file formats, with the appropriate compression ratios (if applicable), bit-depths, and resolutions used to maintain quality.

The goal of the quality check is to ensure that all the essential characteristics of the original paper records – defined during the benchmarking phase – are faithfully represented in the digital images. This ensures that the digitatal image meets the required standards and is fit for the intended purpose. Representational accuracy is especially important if the hard-copy records will be destroyed once digitisation and verification is complete. 

Completeness of digitisation

To ensure that all of the required original paper records are digitised, checks should be conducted on the completeness of the work. 

This may include validating the number of pages in paper records against the number of digital images created. 

For multi-page items, the number of pages within an image should accurately reflect the original paper records, and the pages should be structured and arranged in the correct order.

Sampling for quality checking of digital images

When faced with time and financial constraints, it may be necessary to use sampling as a method for quality checking digital images. Sampling options and their advantages and disadvantages:

• Checking all digital images and metadata
  • Pros: All images will meet the minimum required quality baselines.
  • Cons: Time- and resource-intensive.
• Checking only random samples of digital images and metadata
  • Pros: Requires less time and fewer resources.
  • Cons: There is a lower degree of certainty that all images meet the quality baselines.

If sampling is used, the following should be considered:

• Benchmark frequency: The frequency of sampling should be based on system usage and anticipated periods of deterioration. It’s crucial to include both the images and their metadata in the sampling process. System vendors may provide guidance on the appropriate frequency. ³
• Frequency of sampling: Initially, more frequent sampling may be required, but as benchmarks, equipment, and processes stabilise, random sampling of 5-10% may be sufficient. ⁴
• Representative samples: Ensure that the samples represent the full range of digitised records, including those with poor quality in the original paper records. If new staff, equipment repairs, or service providers are involved, every image may need to be checked until standards are fully met.

Environment for quality checking

• A controlled environment is necessary for consistent quality assurance. Factors like excessive glare, reflections, or improper setup of computer systems could lead to misjudging the quality of a high-quality image.
• Output device: The quality assurance checks should be conducted using the same output device intended for the image (for example, if the image is meant for printing, it should be printed and checked against the quality baselines for printed images).
• Checking may need to be conducted on a variety of printers and monitors to detect any variations. 

User fault reporting

• Users should report any faults to the digitisation team so these can be corrected. Faults should be included in quality assurance reports, as they may help identify common issues that need to be addressed.

Re-digitising

• If images do not meet the required benchmarks, the organisation or service provider must re-digitise them. Approaches include:
  • If more than 1% of images and metadata in a randomly selected sample are defective, the entire batch must be re-inspected and corrected.
  • If less than 1% are defective, only the faulty images and metadata should be redone.

While some organisations may set an ‘acceptable margin of error,’ the goal should be to achieve 100% accuracy. If the essential characteristics of the original records are compromised, the images may no longer be fit for their intended purpose and should not be retained.