Tutorial

The final result ready for submission into the Digital Preservation Service is the Submission Information Package (SIP). The SIP consists of a Metadata Encoding and Transmission Standard (METS) document, a digital signature and file(s).

The following must be provided to construct a valid minimal SIP:

• base METS information (METS profile, contract identifier and creator)

• files (automatically imported from directory or manually by hand)

• descriptive metadata (added anywhere in the SIP)

Rest of the required information is automatically generated by siptools-ng.

1. Create base METS

The METS object contains various metadata about the SIP and its contents. It is mostly populated by siptools-ng, though you need to provide some basic information first:

from mets_builder import METS, MetsProfile

mets = METS(
    mets_profile=MetsProfile.RESEARCH_DATA,
    contract_id="urn:uuid:abcd1234-abcd-1234-5678-abcd1234abcd",
    creator_name="Sigmund Sipenthusiast",
    creator_type="INDIVIDUAL"
)

Once the SIP object is created, the METS object will be available via the SIP.mets property.

Note

The METS specification is explained in more detail in the DPRES metadata requirements.

In many cases, siptools-ng uses dpres-mets-builder to automatically generate metadata without requiring user action.

2.1 Import files from directory

The fastest and easiest way to create a SIP object is to automatically import a directory from the local filesystem using the siptools_ng.sip.SIP.from_directory() factory method:

from siptools_ng.sip import SIP

sip = SIP.from_directory(mets=mets, path="/path/to/local/directory")

2.2 Import files manually

Files can also be imported manually and then added into a SIP object using siptools_ng.sip.SIP.from_files():

from siptools_ng.file import File

files = [
    File(
        path="/home/sigmund/Videos/source-files/video.mkv",
        digital_object_path="media/video.mkv"
    ),
    File(
        path="/home/sigmund/Documents/pdf-a/document.pdf",
        digital_object_path="documents/document.pdf"
    )
]

sip = SIP.from_files(mets=mets, files=files)

The generated technical metadata for individual files can be fine-tuned by running siptools_ng.file.File.generate_technical_metadata() manually with existing known technical metadata.

Warning

Do not add or modify File instances after you have created the SIP instance.

3. Add descriptive metadata

At least one piece of descriptive XML metadata needs to be added into the SIP. This metadata can concern a file or the package as a whole; the only requirement is that at least one piece of descriptive metadata is provided somewhere.

Note

The National Digital Preservation schema catalog contains a variety of different metadata document schemas that are accepted by the Digital Preservation Service.

You can look them up in the DPRES national specifications.

The metadata XML document can be automatically imported using ImportedMetadata, which will automatically detect the XML schema.

from mets_builder.metadata import (
    ImportedMetadata, MetadataType, MetadataFormat
)

# Import metadata automatically from an external file...
dmd_md = ImportedMetadata.from_path("/path/to/descriptive_metadata.xml")

# ...or enter metadata schema information manually
dmd_md = ImportedMetadata(
    metadata_type=MetadataType.DESCRIPTIVE,
    metadata_format=MetadataFormat.DC,
    format_version="2008",
    data_path="/path/to/descriptive_metadata.xml"
)

You can add the descriptive metadata to either a file or the SIP:

# Add metadata to SIP
sip.add_metadata([dmd_md])

# Add metadata to File, and the File to SIP
file.add_metadata([dmd_md])
sip = SIP.from_files(mets=mets, files=[file])

4. Export SIP

Once you have created a SIP using either method, you can export it using the siptools_ng.sip.SIP.finalize() method.

This will generate a tar archive with a digital signature, a METS document and copies of all the files.

sip.finalize(
     output_filepath="sip.tar",
     sign_key_filepath="rsa-keys.crt"
 )

Note

rsa-keys.crt is the signing key used to create a digital signature for the SIP.

See the instructions on the National Digital Preservation Service site (in Finnish) for generating this signing key.

The generated sip.tar file can then be uploaded into the Digital Preservation Service.