Skip to contents

Every present chunk becomes a virtual reference (location, offset, length) to the source bytes, with the source's etag (or mtime) as the checksum, so Icechunk refuses chunks whose source has changed. (Icechunk checks "unmodified since"; for file:// locations the recorded time is mtime + 1, because local filesystems keep sub-second mtimes and byterefs records whole seconds.) Chunks with an inline data blob are written as native chunks, absent chunks are left unwritten (they read as the fill value), and coordinate arrays (affine, inline) are materialised as single native chunks. The root group's attributes hold meta, plus a byterefs entry with the remap rules and target, so ic_to_byterefs() can recover the verbatim source paths.

Usage

ic_write_byterefs(
  session,
  br,
  target = NULL,
  checksums = TRUE,
  validate = TRUE
)

Arguments

session

a writable icechunk_session.

br

a byterefs store (see ic_byterefs_read()).

target

remap target (see ic_byterefs_containers()).

checksums

record etag / mtime checksums on virtual refs.

validate

check every location falls within a declared container.

Value

a data frame, one row per array: array, refs (virtual), native (inline or coordinate chunks), absent.

Details

Arrays become Zarr v3: dtype and byte order from dtype, bytes then numcodecs.shuffle then zstd / numcodecs.zlib / gzip from codec, with imagecodecs_delta (TIFF predictor 2) ahead of bytes.

Examples

ex <- ic_example_repo()
repo <- ic_repo_open(ex$path, authorize = ex$authorize)
br <- ic_to_byterefs(ic_session_readonly(repo, branch = "main"))

# edit the table, then write it into a session of another repository
i <- which(br$arrays$array == "from_bin")
br$arrays$attrs[[i]] <- list(units = "K")
dst <- ic_repo_create(tempfile("dst-"), containers = ic_byterefs_containers(br),
                      authorize = ex$authorize)
w <- ic_session_writable(dst, "main")
ic_write_byterefs(w, br)
#>      array refs native absent
#> 1 from_bin    4      0      0
#> 2  from_nc    1      0      0
#> 3    small    0      4      0
ic_commit(w, "from byterefs")
#> [1] "XVEPB7VE7JM8KE4DCSY0"
jsonlite::fromJSON(rawToChar(ic_get(w, "from_bin/zarr.json")))$attributes
#> $units
#> [1] "K"
#>