Data Model and xarray

This project adopts xarray, an open-source class with methods for labelled multi-dimensional array, as the common data model.

The adoption of xarray enables more efficient division of development work. It also facilitates interoperability of data and algorithms from various sources. The workflow is greatly simplified, with a typical example illustrated below:

graph TD A(Ingest radar data with swirlspy.rad.*) -->B(Forecast radar echoes with swirlspy.qpf.*) B --> C(Convert reflectivity to rainfall rate) C --> D(Accumulate rainfall rate to forecast rainfall amount) D --> E(Generate forecast rainfall map with xarray methods) D --> F(Produce guidance products for forecasters)

In the above example, xarray serves as inputs and outputs of modules in each stage. Developers can then work on his/her own module without inventing and communicating the data formats to upstream/downstream application developers.

It would be possible to swap in various algorithms as and when necessary. For example, ROVER, a module under swirlspy.qpf.* based on varitional optical flow, may run in parallle with TrajGRU, another QPF module based on deep learning. Their reuslts can be compared and benchmarked, or even be aggregated to form a super ensemble for operational use.