Connects to a Zarr store at the given path or URI. Dispatches to the appropriate backend based on the URI scheme:
Local paths: base R (no arrow or GDAL needed)
s3://,gs://: listed and read through Arrow when arrow is installed, else through GDAL's/vsis3/or/vsigs/(gdalraster). When access is known (anonymous, or S3 keys in the environment) chunks are fetched as HTTPS through the curl pool instead, seezaro_fetch()http://,https://: curl (no GDAL); chunk reads are batched through one connection pool, seezaro_fetch(). An HTTP store can't be listed, so it needs consolidated metadata (or array names, seezaro_dataset())/vsi...: any GDAL virtual file system path, through gdalraster (e.g./vsicurl/https://...to keep GDAL's HTTP handling)reference+json://: Kerchunk JSON reference storereference+parquet://: Kerchunk Parquet references, either the directory kerchunk and VirtualiZarr write (.zmetadataand<array>/refs.N.parq; only the files a selection needs are read) or a single table of key, url, offset and lengthvirtualizarr://: the same directory layout, by its older name herebyterefs://(or the oldermdimrefs://): a byterefs SQLite byte-reference table (https://github.com/byterefs/spec),byterefs/0.1or the legacymdim-refs/0.2. Source paths are remapped for the store'smeta.default_target; passtarget = "public"for another target, ortarget = ""for the verbatim sourcesflatrefs://: a byterefs flat refs store, the Parquet export (a directory, local or remote, witharrays.jsonand onerefs_<array>.parquetper array, as raad.virtualize publishes). Chunk references are read by row group, only those whose dim statistics can hold the selection, so a table of tens of millions of rows is never read whole.targetas forbyterefs://icechunk://: an Icechunk repository through tabularberg, e.g.icechunk:///path/to/repo?tag=v1; the query picksbranch,tagorsnapshot. Anicechunk_repo,icechunk_sessionoricechunk_storageobject works too. Seezaro_icechunk()
Arguments
- source
character. Path, URI, or reference store locator; or an Icechunk repository, session or storage from tabularberg (see
zaro_icechunk()).- verbose
logical. Emit progress and diagnostic messages. Default
getOption("zaro.verbose", TRUE); setoptions(zaro.verbose = FALSE)to silence zaro everywhere.- ...
store options: for
s3://andgs://, arguments for the Arrow filesystem constructors (e.g.anonymous = TRUE,region,endpoint_override), also used for curl-pool access; forbyterefs://andflatrefs://,target(the path remap to apply); for Icechunk sources, the arguments ofzaro_icechunk()(branch,tag,snapshot,authorize,direct).- validate
logical. Read the root metadata at open, warning if that fails. Default
FALSE.
Details
For S3 stores, if Arrow returns a 301 redirect (wrong region), zaro will
fall through to gdalraster's /vsis3/ which handles region detection
automatically.
Known public buckets (e.g. cmip6, aodn-cloud-optimised) are accessed
anonymously by default. For other public stores, pass
anonymous = TRUE.
Opening reads nothing from the store by default (source is only checked
to be a non-empty character string); set validate = TRUE to read the
root metadata at open.
Examples
## a small Zarr V3 store shipped with the package
f <- system.file("extdata", "example.zarr", package = "zaro")
store <- zaro(f)
#> [zaro] opening local store: /usr/local/lib/R/site-library/zaro/extdata/example.zarr
store
#> <ZaroStore> [zaro::LocalStore]
#> root: /usr/local/lib/R/site-library/zaro/extdata/example.zarr
zaro_list(store)
#> [1] "detail" "lat" "lon" "temp" "time" "zarr.json"
zaro_read(store, "temp", start = c(0, 0, 0), count = c(1, 4, 8))
#> [zaro] found temp/zarr.json (Zarr V3)
#> [zaro] dtype: float32 | shape: [3, 8, 16] | chunks: [1, 4, 8]
#> [zaro] codecs: bytes -> gzip
#> [zaro] dimensions: time, lat, lon
#> [zaro] reading 1 chunk(s) for path 'temp' (V3)
#> , , 1
#>
#> [,1] [,2] [,3] [,4]
#> [1,] NaN 14.875 14.625 14.375
#>
#> , , 2
#>
#> [,1] [,2] [,3] [,4]
#> [1,] NaN 14.975 14.725 14.475
#>
#> , , 3
#>
#> [,1] [,2] [,3] [,4]
#> [1,] NaN 15.075 14.825 14.575
#>
#> , , 4
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 15.425 15.175 14.925 14.675
#>
#> , , 5
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 15.525 15.275 15.025 14.775
#>
#> , , 6
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 15.625 15.375 15.125 14.875
#>
#> , , 7
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 15.725 15.475 15.225 14.975
#>
#> , , 8
#>
#> [,1] [,2] [,3] [,4]
#> [1,] 15.825 15.575 15.325 15.075
#>
## other sources (see the "Connecting to stores" article):
## zaro("s3://bucket/store.zarr", anonymous = TRUE)
## zaro("gs://bucket/store.zarr")
## zaro("virtualizarr://https://host/dataset.parq")
## zaro("byterefs:///path/to/store.sqlite")
## zaro("flatrefs://https://host/store", target = "public")
if (FALSE) { # interactive() && requireNamespace("blosc", quietly = TRUE)
## a public store over HTTPS (network access; blosc-compressed chunks)
aodn <- zaro(paste0("https://aodn-cloud-optimised.s3.ap-southeast-2.amazonaws.com/",
"model_sea_level_anomaly_gridded_realtime.zarr"))
zaro_meta(aodn, "GSLA")
zaro_read(aodn, "GSLA", start = c(0, 100, 100), count = c(1, 2, 3))
}