datapack Background

adapted from the dataone and datapack vignettes

datapack is written differently than most R packages you may have encountered in the past. This is because it uses the S4 system instead.

library(dataone)
library(datapack)
library(uuid)

Data packages

Data packages are a class that has slots for relations (provenance), objects(the metadata and data file(s)) and systemMetadata.

Navigating data packages

Nodes

Using this example on arcticdata.io

d1c_test <- dataone::D1Client("STAGING", "urn:node:mnTestARCTIC")

To use the member node information, use the mn slot

d1c_test@mn

To access the various slots using objects created by datapack and dataone (e.g. getSystemMetadata) requires the @ which is different from what you might have seen in the past. This is because these use the S4 system.

Get an existing package from the Arctic Data Center. Make sure you know as you go through this training whether you are reading or writing to test or production. We don’t want to upload any of your test datasets to production!

d1c <- dataone::D1Client("PROD", "urn:node:ARCTIC")
resource_map_id = "resource_map_urn:uuid:1f9eee7e-2d03-43c4-ad7f-f300e013ab28"
dp <- dataone::getDataPackage(d1c, identifier = resource_map_id, lazyLoad = TRUE)

Data Objects

You can see what slots are in an S4 object after typing the subsetting operator @, or pressing TAB with the cursor after an existing @. Try viewing the slots of the data package by pressing TAB after writing the following:

dp@

Check out the objects slot

dp@objects

The objects slot contains a list of object PIDs that are accessed using the $ subsetting operator. Both are found within the structure of data packages in R.

Get the number of data and metadata files associated with this data package with datapack’s getSize function.

getSize(dp)

Get the file names and corresponding PIDs using datapack’s getValue function. You can also get other sysmeta slots such as formatId and size by changing the name argument.

getValue(dp, name="sysmeta@fileName")

Get identifiers

You can search by any of the sysmeta slots such as fileName and formatId and get the corresponding identifier(s):

metadataId <- selectMember(dp, name="sysmeta@ADD THE NAME OF THE SLOT", value="PATTERN TO SEARCH BY")

Example:

selectMember(dp, name="sysmeta@formatId", value="image/tiff") 
selectMember(dp, name="sysmeta@fileName", value="filename.csv")

These will give you the PIDs of the files that are of the formatId “image/tiff” and the file name “filename.csv”, respectively.

Provenance

View the provenance as a dataTable. This explains the relationships between different files. We will get into detail in the Building provenance chapter.

dp@relations$relations