Support for In-Memory xarray Datasets and On-the-Fly NDVI Change Detection (dNDVI

[weirdsim14]

I'm interested in serving xarray datasets that reside entirely in memory (e.g., results from computations or streaming data) using TiTiler, without the need to write them to disk as NetCDF or Zarr files.​

Additionally, I'd like to compute NDVI change detection (dNDVI) on-the-fly, by calculating the difference between NDVI values at two time points, and serve the resulting tiles dynamically.​

Currently, the titiler.xarray extension appears to support datasets loaded from disk or cloud storage. However, for workflows involving real-time data processing or temporary datasets, it would be beneficial to serve these in-memory datasets directly.​

Proposed Solution:

Introduce support for in-memory xarray datasets by allowing users to pass a dataset object directly to a custom reader.​

Enable on-the-fly computation of NDVI and dNDVI within the tile server, allowing users to specify time points and compute differences dynamically.​

[vincentsarago]

import attr
from typing import Annotated
from fastapi import FastAPI, Query
from titiler.xarray.factory import TilerFactory
import xarray

from titiler.xarray.io import Reader, get_variable

my_dataset: xarray.Dataset = ... # define you dataset here 


# We create dataset PathParams with default and we don't include it to the openAPI docs
def DatasetPathParams(url: str = Query(":memory:", include_in_schema=False)) -> str:
    """Create dataset path from args"""
    return ":memory:"


@attr.s
class CustomReader(Reader):
    """Reader: Open Zarr file and access DataArray."""

    def __attrs_post_init__(self):
        """Set bounds and CRS."""
        self.ds = my_dataset
        self.input = get_variable(
            self.ds,
            self.variable,
            datetime=self.datetime,
            drop_dim=self.drop_dim,
        )
        super().__attrs_post_init__()


app = FastAPI(
    title="TiTiler with support of Multidimensional dataset",
    openapi_url="/api",
    docs_url="/api.html",
    version="0.1.0",
)


md = TilerFactory(
    router_prefix="/md",
    path_dependency=DatasetPathParams,
    reader=CustomReader,
)
app.include_router(md.router, prefix="/md", tags=["Multi Dimensional"])

You could have something like this! that is said, it seems to me that titiler might be over complex for what you're trying to achieve and maybe something like xpublish might be better.

[vincentsarago]

for the NDVI part, well it's a bit more complex and will depends how you structure the xarray dataset variable. By default rio-tiler's Xarray reader will only work with DataArray so if your Red and NIR data are stored in two variables then titiler won't be able to merge them at request time.

I've started cogeotiff/rio-tiler#779 to explore more dataset support but there is a lack of contribution and I'm not a Xarray user/expert so it's hard for me to create code without knowing what the use case would be 😅

[hrodmn]

Enable on-the-fly computation of NDVI and dNDVI within the tile server, allowing users to specify time points and compute differences dynamically.​

This is a cool idea! It is not immediately clear to me how we would support such a complex use-case in the existing factory methods. Maybe we could incorporate index slicing into the expression parameter kind of like we can do with bands within a geotiff? Even if we could it would only work for cases where a single file (or zarr store) contains data for all time steps.

For now I think you will need to create a custom application with a custom Factory class with endpoints that accept two datetime parameters which would get used to construct the before/after arrays.