Skip to contents

Returns an ordinary R array (a double, integer or logical vector with a dim attribute and no class) whose values are read from the store chunk by chunk, only when R asks for them. It is an ALTREP object built by altarr::altarr(), with zaro's chunk fetch and codec pipeline behind it. Nothing is read when the array is created beyond the metadata.

Usage

zaro_array(
  store,
  path = "",
  meta = NULL,
  parallel = FALSE,
  verbose = zaro_verbose(),
  cf = FALSE
)

Arguments

store

a store object from zaro(), or a zaro_dataset() (whose metadata is then used, no request made).

path

character. Path to the array within the store.

meta

optional pre-fetched ZaroMeta, or the root group meta with consolidated entries, as for zaro_read().

parallel

logical. If TRUE, each batch of 4 or more chunks is fetched and decoded with future.apply::future_lapply(), as for zaro_read() (the same caveats about store serialization apply).

verbose

logical. Emit diagnostic messages when reading metadata. Default getOption("zaro.verbose", TRUE).

cf

logical. Apply CF decoding (masking and unpacking, see below). Default FALSE.

Value

A lazy array, see altarr::altarr().

Details

The altarr package is not on CRAN; install it from R-universe with install.packages("altarr", repos = "https://hypertidy.r-universe.dev").

Subsetting with x[i], x[cbind(i, j, k)] and altarr::altarr_extract(x, i, j, k) plans the chunks it needs and fetches them in one batch; sum(), mean(), min() and max() stream the chunk grid. See ?altarr::altarr_contract for what base R does with a lazy array, and altarr::altarr_stats() to count fetches.

Dimension order

A C-order Zarr array of shape (s1, ..., sn) (all Zarr V3 arrays, and V2 arrays with "order": "C") is returned with R dim c(sn, ..., s1), and the dimension names reversed to match. This is the order the bytes are stored in, so a chunk is used as decoded. For a typical (time, lat, lon) array, x[lon, lat, time] is the R subscript. V2 arrays with "order": "F" keep their shape. This is the transpose of what zaro_read() returns.

Missing chunks and types

Chunks absent from the store read as the array's fill_value. Integer types up to 32 bits give an integer array (uint32 and 64-bit integers give double, as in zaro_read()), floats give double, bool gives logical. No CF packing (scale_factor, add_offset, _FillValue) is applied unless cf = TRUE. String arrays are refused (a lazy array is numeric or logical); read them with zaro_read().

CF decoding

With cf = TRUE, values equal to the array's fill_value or to the _FillValue or missing_value attributes (and NaN) read as NA, then scale_factor and add_offset are applied. Missing chunks read as NA. A packed array reads as double; an unpacked one keeps its type. This is what xarray's default decoding and GDAL's nodata plus unscaling give.

Saving

saveRDS() on the result writes the recipe (shape, chunk shape and the fetch function), not the values. Stores backed by Arrow or GDAL hold external pointers, so a saved array does not read in a new R session; re-open it with zaro_array().

Examples

f <- system.file("extdata", "example.zarr", package = "zaro")
store <- zaro(f)
#> [zaro] opening local store: /usr/local/lib/R/site-library/zaro/extdata/example.zarr
x <- zaro_array(store, "temp")
#> [zaro] found temp/zarr.json (Zarr V3)
#> [zaro]   dtype: float32 | shape: [3, 8, 16] | chunks: [1, 4, 8]
#> [zaro]   codecs: bytes -> gzip
#> [zaro]   dimensions: time, lat, lon
dim(x)                       # lon, lat, time: the Zarr shape reversed
#> [1] 16  8  3
x[1:4, 1:2, 1]               # reads one chunk
#>        [,1]   [,2]
#> [1,]    NaN 14.875
#> [2,]    NaN 14.975
#> [3,]    NaN 15.075
#> [4,] 15.425 15.175
altarr::altarr_stats(x)[c("fetch_calls", "chunks_fetched")]
#>    fetch_calls chunks_fetched 
#>              1              1 
if (FALSE) { # interactive() && requireNamespace("altarr", quietly = TRUE)
## a public store over HTTPS (network access; blosc-compressed chunks)
if (requireNamespace("blosc", quietly = TRUE)) {
  aodn <- zaro(paste0("https://aodn-cloud-optimised.s3.ap-southeast-2.amazonaws.com/",
                      "model_sea_level_anomaly_gridded_realtime.zarr"))
  x <- zaro_array(aodn, "GSLA", cf = TRUE)
  dim(x)                     # LONGITUDE, LATITUDE, TIME
  x[cbind(101, 101, 1:5)]    # one pixel, five time steps, one fetch
  altarr::altarr_stats(x)[c("fetch_calls", "chunks_fetched")]
}
}