Iterates over chunks and applies a function to each, accumulating results. Chunks are fetched and decoded one at a time (or in parallel) so memory usage is proportional to one chunk, not the full array.
Usage
zaro_chunk_apply(
store,
path,
fun,
ranges = NULL,
meta = NULL,
shaped = TRUE,
parallel = FALSE,
verbose = zaro_verbose()
)Arguments
- store
a store object from
zaro()- path
character. Path to the array.
- fun
function. Called with a single chunk result list (as from
zaro_chunk()). NULL chunks (missing) are skipped.- ranges
optional range specification (as in
zaro_chunks()).- meta
optional pre-fetched ZaroMeta (or root group meta).
- shaped
logical. If TRUE, reshape each chunk's values.
- parallel
logical. Use future.apply for parallel execution.
- verbose
logical. Emit diagnostic messages. Default
getOption("zaro.verbose", TRUE).
Examples
f <- system.file("extdata", "example.zarr", package = "zaro")
store <- zaro(f)
#> [zaro] opening local store: /usr/local/lib/R/site-library/zaro/extdata/example.zarr
## per-chunk mean, one chunk in memory at a time
means <- zaro_chunk_apply(store, "temp", function(chunk) {
mean(chunk$values, na.rm = TRUE)
})
#> [zaro] found temp/zarr.json (Zarr V3)
#> [zaro] dtype: float32 | shape: [3, 8, 16] | chunks: [1, 4, 8]
#> [zaro] codecs: bytes -> gzip
#> [zaro] dimensions: time, lat, lon
#> [zaro] reading 12 chunk(s) for path 'temp'
unlist(means)
#> [1] 15.08707 16.10000 17.10000 14.10000 15.10000 16.10000 15.90000 16.90000
#> [9] 17.90000 14.90000 15.90000 16.90000
## first time step only, all spatial chunks
zaro_chunk_apply(store, "temp", function(chunk) {
c(mean = mean(chunk$values, na.rm = TRUE),
n_valid = sum(is.finite(chunk$values)))
}, ranges = list(0, NULL, NULL))
#> [zaro] found temp/zarr.json (Zarr V3)
#> [zaro] dtype: float32 | shape: [3, 8, 16] | chunks: [1, 4, 8]
#> [zaro] codecs: bytes -> gzip
#> [zaro] dimensions: time, lat, lon
#> [zaro] reading 4 chunk(s) for path 'temp'
#> [[1]]
#> mean n_valid
#> 15.08707 29.00000
#>
#> [[2]]
#> mean n_valid
#> 14.1 32.0
#>
#> [[3]]
#> mean n_valid
#> 15.9 32.0
#>
#> [[4]]
#> mean n_valid
#> 14.9 32.0
#>