Platform Services

Dynamic Data Cube

DDC Dynamic Data Cubes is a service that creates, stores, and updates Data Cubes. According to the data prescription defined via the DDC AOI Manager service, it gathers the required data layers from multiple sources and stores them into a standardized data cube in Zarr format. Besides creating the data cubes, according to the initial time interval and the data layers defined by users, the service also provides periodic data updates. That means that users do not need direct interaction to update their data cubes; it happens automatically according to the set update intervals (daily or weekly, for example).

The Dynamic Data Cube service uses an extensible plugin-based paradigm that utilizes multiple data access patterns to populate data cubes. Dynamic Data Cubes create a conventional truth representing the state of the natural and economic environment for applications that are not connected directly and are developed by separate teams. It also creates a strong foundation for modeling and running what-if scenarios.

Service logic

Dynamic Data Cube is a backend data gathering service that requires defined AOIs (Areas of Interest) and a list of data layers to collect data automatically without user interaction. AOIs are defined by polygon or multi-polygon geometries, and data layers are defined as a simple list.

AOIs serve as Data Cube prescriptions, and to create Data Cubes, AOI must be created first with the layer list definition. The figure bellow shows how the DDC AOI Manager helps to define and manage the Data Cube prescriptions.

(Click to enlarge the image.)

_images/Dynamic_Data_Cube.png

Create and access Data Cubes

The Data Cube creation process involves the AOI creation and also the initial data gathering, which happens automatically after the AOI creation. When the initial data gathering is done, the Data Cube can be accessed.

Data Cubes can be created in two ways:

  1. Via the AOI Manager microservice API Reference, using the create-aoi endpint.

  2. With the ddc-utility python package, using the create_aoi() method of the ddc_client.

Once the AOI and the underlying Data Cube are created, it can be opened with the open_cube() method of the open_cube.

Example notebook on using the DDC Utility Python package can be found here: https://github.com/DanubeDataCube/DanubeDataCubeExamples/blob/main/tutorials/ddc-firststeps-example.ipynb

Algo Hosting

  • Micro-frontends

  • Microservices

  • Complex applications