Reads a byterefs table, either the SQLite working store (store.sqlite,
or an open DBI connection) or the Parquet "flat" export (a directory
holding arrays.json, sources.parquet and refs_<array>.parquet).
byterefs/0.1 and its predecessor mdim-refs/0.2 are accepted.
Value
a byterefs object: a list with format_version, meta (named
list of strings), sources, arrays and remap data frames, and
refs(array), a function returning that array's global refs rows
(dim_0.., present, path, offset, size, info, data) joined
to the source's etag, mtime and source_size. Read from SQLite,
refs() queries the file lazily, so the connection stays open: call
$close() when done (functions given a file path close it themselves).
Examples
ex <- ic_example_repo()
s <- ic_session_readonly(ic_repo_open(ex$path, authorize = ex$authorize), branch = "main")
# write the store's references as a byterefs SQLite file, read it back
if (requireNamespace("RSQLite", quietly = TRUE)) {
db <- tempfile(fileext = ".sqlite")
ic_to_byterefs(s, dsn = db)
br <- ic_byterefs_read(db)
print(br)
print(br$arrays[, c("array", "kind", "shape", "chunks", "dtype")])
print(head(br$refs("from_bin")))
br$close()
}
#> <byterefs> byterefs/0.1
#> sources: 2
#> from_bin data referenced <f4 64x48
#> from_nc data referenced <f4 64x48
#> small data referenced <f8 4x6
#> array kind shape chunks dtype
#> 1 from_bin data 64, 48 16, 48 <f4
#> 2 from_nc data 64, 48 64, 48 <f4
#> 3 small data 4, 6 2, 3 <f8
#> dim_0 dim_1 present
#> 1 0 0 1
#> 2 1 0 1
#> 3 2 0 1
#> 4 3 0 1
#> path offset size
#> 1 /tmp/RtmpVaRScy/icechunk-example-35c971f6c977/source/grid.bin 0 3072
#> 2 /tmp/RtmpVaRScy/icechunk-example-35c971f6c977/source/grid.bin 3072 3072
#> 3 /tmp/RtmpVaRScy/icechunk-example-35c971f6c977/source/grid.bin 6144 3072
#> 4 /tmp/RtmpVaRScy/icechunk-example-35c971f6c977/source/grid.bin 9216 3072
#> info data etag mtime source_size
#> 1 <NA> <NA> <NA> NA NA
#> 2 <NA> <NA> <NA> NA NA
#> 3 <NA> <NA> <NA> NA NA
#> 4 <NA> <NA> <NA> NA NA