swirlspy.qpe package

Submodules

swirlspy.qpe.cal_zr module

swirlspy.qpe.cal_zr.cal_zr(reflectivity_rf, duration)

Calibrate the Z=aR^b relationship through deriving (a,b) by linear regression, also compute the correlation of 2 time series of reflectivity and rainfall

Parameters

reflectivity_rf (array of tuples (reflectivity, rainfall_amount)) – reflectivity in dBZ, rainfall amount in mm
duration (integer) – duration of rainfall accumulation in minutes, default to 60

Returns

a (float)
b (float) – a,b value in the Z=aR^b relationship
correlation (float) – correlation of 2 time series of reflectivity and rainfall

swirlspy.qpe.rfmap module

swirlspy.qpe.rfmap.comp_qpe(area_def, rg_object=None, rg_interp=None, rad_rf=None, sat=None, rg_radius=10000, radar_radius=256000, sat_radius=None, r_sigma_ratio=2.5, max_truth={'radar': 0.1, 'rg': 1.0, 'sat': 0.01})

Calculate composite QPE from Rain Gauge, Radar and Satellite. The dimensions and coordinates of the input xarray.DataArrays must be the same. All data must also be in the same coordinate system.

Parameters

area_def (pyresample.geometry.AreaDefinition) – AreaDefinition of the grid.
rg_object (Rain) – Rain object containing raingauge data. The x, y values in rg_object must be in the same coordinate system as the other data.
rg_interp (xarray.DataArray) – Contains rainfall values from rain gauge interpolated to user-defined grid or the same grid as rad_rf and/or sat.
rad_rf (xarray.DataArray) – Contains rainfall derived from radar reflectivity.
sat (xarray.DataArray) – Contains rainfall derived from satellite in user-defined grid or the same grid as rad_rf and/or rg_interpolate.
rg_radius (int) – The radius of influence of raingauge. The “truthness” of a raingauge decreases normally with distance from the raingauge. Distance should be given in the units of the user-defined projection. Defaults to 10000m.
radar_radius (int) – The radius of influence of radar. The “truthness” decreases normally with distance from the station. Defaults to 256000m.
sat_radius (int) – The radius of influence of satellite.
r_sigma_ratio (float) – The ratio between radius of influence and standard deviation. Defaults to 2.5.
max_truth (dict of floats) – The maximum ‘truthness’ of raingauge, radar and satellite respectively. Defaults to {‘rg’: 1., ‘radar’: 0.1, ‘sat’: 0.01}.

Returns

wrf – The composite rainfall calculated from Rain Gauge, Radar and Satellite derived rainfall.

Return type

xarray.DataArray

swirlspy.qpe.rfmap.overlay_pil(filename_rfcontour, filename_overlay, dimension, position)

Overlay rainfall map with frame. then save as figure.

Parameters

filename_rfcontour (string) – Filename of rainfall contour, returned from rfmap.savefig.
filename_overlay (string) – Filename of overlaying frame/ basemap.
dimension (tuples (width, height)) – dimension of the desired rainfall contour. Should be customised to fit the size overlaying frame/ basemap.
position (tuples (x,y)) – position of the overlaying frame/ basemap, where the resized rainfall contour will be pasted.

swirlspy.qpe.rfmap.rg_interpolate(rg_object, area_def, interp_type, coord_label=['x', 'y'], gamma_value=2, minimum_neighbors=3, kappa_star=0.01, search_radius=None, rbf_func='multiquadric', rbf_smooth=0, kernel=None, alpha=1e-10, n_restarts_optimizer=10, normalize_y=False)

Interpolate the rainfall on an area, based on metpy.interpolate.interpolate_to_points.

Parameters

rg_object (Rain) – Rain object.
area_def (pyresample.geometry.AreaDefinition) – AreaDefinition of the target grid.
interp_type (str) –
Type of interpolation to be used. Available options include:
1. ”linear”, “nearest”, “cubic”, or “rbf” from scipy.interpolate.
2. ”natural_neighbor”, “barnes”, or “cressman” from metpy.interpolate.
3. ”ordinary_kriging” from sklearn.gaussian_processes.GaussianProcessRegressor.
coord_label (list of strings) – Coordinate labels. Defaults to [‘x’, ‘y’].
gamma_value (float, optional, Barnes) – Adjustable smoothing parameter for the barnes interpolation. Default 2.0.
minimum_neighbors (int, optional, Barnes) – Minimum number of neighbors needed to perform barnes or cressman interpolation for a point. Default is 3. Adjustable.
kappa_star (float, optional, Barnes) – Response parameter for barnes interpolation, specified non-dimensionally in terms of the Nyquist. Default 0.01.
search_radius (float, optional, Barnes) – A search radius to use for the barnes and cressman interpolation values. If search_radius is not specified, it will default to the average spacing of observations.
rbf_func (str, optional, RBF) – Specifies which function to use for Rbf interpolation. Options include: ‘multiquadric’, ‘inverse’, ‘gaussian’, ‘linear’, ‘cubic’, ‘quintic’, and ‘thin_plate’. Default to ‘multiquadric’.
rbf_smooth (float, optional, RBF) – Smoothing value applied to rbf interpolation. Higher values result in more smoothing. Defaults to 0.
kernel (sklearn.gaussian_process.kernels, optional, OK) – The kernel specifying the covariance function of the GP for ordinary kriging. If None is passed, the kernel “1.0 * RBF(1.0)” is used as default. Note that the kernel’s hyperparameters are optimized during fitting. Defaults to None.
alpha (float or array-like, optional, OK) – For ordinary kriging. Value added to the diagonal of the kernel matrix during fitting. Larger values correspond to increased noise level in the observations. This can also prevent a potential numerical issue during fitting, by ensuring that the calculated values form a positive definite matrix. If an array is passed, it must have the same number of entries as the data used for fitting and is used as datapoint-dependent noise level. Note that this is equivalent to adding a WhiteKernel with c=alpha. Allowing to specify the noise level directly as a parameter is mainly for convenience and for consistency with Ridge. Defaults to 1e-10.
n_restarts_optimizer (int, optional, OK) – For ordinary kriging. The number of restarts of the optimizer for finding the kernel’s parameters which maximize the log-marginal likelihood. The first run of the optimizer is performed from the kernel’s initial parameters, the remaining ones (if any) from thetas sampled log-uniform randomly from the space of allowed theta-values. If greater than 0, all bounds must be finite. Note that n_restarts_optimizer == 0 implies that one run is performed. Defaults to 10.
normalize_y (bool, optional, OK) – For ordinary kriging. Whether the target values y are normalized, i.e., the mean of the observed target values become zero. This parameter should be set to True if the target values’ mean is expected to differ considerable from zero. When enabled, the normalization effectively modifies the GP’s prior based on the data, which contradicts the likelihood principle; normalization is thus disabled per default. Defaults to False.

Returns

interpolated_rf – Interpolated values in 2D.

Return type

xarray.DataArray

Notes

Default values set in ‘minimum_neighbors’, ‘gamma’ and ‘kappa_star’, for Barnes interpolation and ‘kernel’, ‘alpha’, ‘n_restarts_optimizer’ and ‘normalize_y’ for Ordinary Kriging for shows best effect for Hong Kong rainfall map. May not be suitable for other countries / regions. Try other values to get the best effect.

swirlspy.qpe.rfmap.show_raingauge(rg_object, ax, show_value=True, color='k', zorder=10, fontsize=12, markersize=12, adjust_labels=False, force_points=0.2, force_text=0.2, expand_points=(1, 1), expand_text=(1, 1), arrowprops={'arrowstyle': '->', 'color': 'black', 'lw': 2})

Generates a scatter plot showing location and rainfall of raingauges.

Parameters

rg_object (Rain) – Rain object.
ax (matplotlib.axes.Axes or cartopy.mpl.geoaxes.GeoAxes) – The plot axes.
show_value (bool) – Show rainfall values of raingauges. Defaults to True.
color (str) – Color of text and markers. Defaults to ‘k’.
fontsize (int) – Font size. Defaults to 12.
markersize (int) – Marker size. Defaults to 12.
adjust_labels (bool) – Choose whether further label adjustments are required. Defaults to False.
force_points (float) – Repel force from points is multiplied by this value. Defaults to 0.2.
force_text (float) – Repel force from text is multiplied by this value. Defaults to 0.2.
expand_text (array_like) – a tuple/list/other sequence data type with 2 multipliers (x, y) by which to expand the bounding box of texts when repelling them from each other. Defaults to (1, 1).
expand_points (array_like) – a tuple/list/other sequence data type with 2 multipliers (x, y) by which to expand the bounding box of texts when repelling them from points. Defaults to (1, 1).
arrowprops (dict) – Customize labelling arrows. Defaults to arrowprops=dict(arrowstyle=’-’, color=’black’, lw=0.5).

swirlspy.qpe.utils module

swirlspy.qpe.utils.acc(rf, ts, acc_period)

Calculate the past rainfall by accumulating rainfall amount. For example: to calculate the past 60 minute rainfall amount at 15:00:

Create past 6-min rainfall amount at 14:06, 14:12 … by rfn().
Concatenate all xarray’s returned by rfn().
Call acc(rf, pandas.Timestamp(‘15:00:00’), pandas.Timedelta(60, ‘m’)).

Parameters

rf (xarray.DataArray) – Rainfall amount at various times.
ts (pandas.Timestamp) – End time of the rainfall accumulation period.
acc_period (pandas.Timedelta) – Rainfall accumulation period.

Returns

acc_rf – accumulated rainfall amount over the designated period

Return type

xarray.DataArray

swirlspy.qpe.utils.dbz2rr(dbz, a=200.0, b=1.6)

Convert reflectivity to rainfall rate by the Z-R relationship. Default a & b are Marshall-Plalmer. You may wish to refer to the Z-R relationship page for more information on the choice of a & b.

Parameters

dbz (xarray.DataArray) – xarray containing the reflectivity xarray in dBZ to convert
a (float) – a in Z = aR^b relationship
b (float) – b in Z = aR^b relationship

Returns

rainrate – xarray containing the rainfall rate converted by the Z=aR^b relationship

Return type

xarray.DataArray

swirlspy.qpe.utils.locate_file(directory, pattern)

Locate a file under path that contains the timestamp string. Certain radar systems generates radar output files with names other than fixed prefix/suffix and timestamp, e.g. TMS180822171807.RAWULST. This utility helps locate the file. If more than one file is located, a list of strings is returned.

Parameters

directory (str) – Directory in which the file is located
pattern (str) – Pattern in the filename, typically a timestamp

Returns

path – Path to the located file(s).

Return type

str or list of strings

swirlspy.qpe.utils.multiple_acc(rainfalls, start_time, end_time, times=Timedelta('0 days 00:30:00'), acc_period=Timedelta('0 days 01:00:00'))

Obtain accumulated rainfall over a time period from a start_time to an end_time at certain times. For example, to generate past hourly rainfall every 30 minutes from start time 14:00 to end time 17:00:

Create past 6-min rainfall amount at 14:06, 14:12 … 17:00 by rfn().
Concatenate all xarray’s returned by rfn().
Call acc(rf, timestamp(‘14:00:00’), timestamp(‘17:00:00’), times=timedelta(30, ‘m’), acc_period=timedelta(1, ‘h’)).

Parameters

rainfalls (xarray.DataArray) – Contains observed and/or forecasted rainfall every n minutes, n being the interval (in minutes) between two timestamps.
start_time (pandas.Timestamp) – The start time of the forecast.
end_time (pandas.Timestamp) – The last forecasted timestep.
times (pandas.Timedelta) – Times at which accumulated rainfall are to be calculated. Defaults to every 30 minutes.
acc_period (pandas.Timedelta) – Rainfall accumulation period. Defaults to 1 hour.

Returns

macc_rf – Contains accumulated rainfall over acc_period every times.

Return type

xarray,DataArray

swirlspy.qpe.utils.rr2rf(rainrate, scan_duration=6, time_shift=0, drift=0)

Convert from radar rainfall rate to accumulated rainfall.

Parameters

rainrate (xarray.DataArray) – rainfall rate, likely outputs from dbz2rr()
scan_duration (int) – number of minutes a radar scan represents, typically 5, 6 or 10
time_shift (timedelta) – temporal shift to account for time required for rain droplets to fall to the ground, and also in case radar data are labelled at the start of the scan while rainfall are labelled at the end (e.g. past 60-minute rainfall)
drift (dict) – spatial displacement to account for rain droplets advected by wind, unit same as that of the xarray

Returns

rf – rainfall amount accumulated during the past n minutes

Return type

xarray.DataArray

swirlspy.qpe.utils.temporal_interp(da, start_time, end_time, intvl=Timedelta('0 days 00:06:00'), interp_type='linear')

Interpolate xarray data values such as rainrates along the time dimension.

Parameters

da (xarray.DataArray) – Contains data values to be temporally interpolated.
start_time (pandas.Timestamp) – User defined start-time.
end_time (pandas.Timestamp) – The user-defined end time of the forecasted period.
intvl (pandas.Timedelta) – The time interval between interpolated data. Defaults to 6 minutes.
interp_type (str) – The interpolation type for interpolating data in the time axis. Options include: ‘linear’, ‘nearest’, ‘zero’, ‘slinear’, ‘quadratic’, ‘cubic’, ‘previous’, ‘next’. Interpolation based on scipy.interpolate.interp1d. Default is linear.

Returns

interp_da – Contains data interpolated along the time axis.

Return type

xarray.DataArray

swirlspy.qpe.utils.timestamps_ending(end_time, duration=Timedelta('0 days 01:00:00'), interval=Timedelta('0 days 00:06:00'), exclude_end=True, format='%y%m%d%H%M')

Generate a list of timestamp strings for the duration ending at time

Parameters

end_time (datetime64[ns]) – time at the end of the period (excluding end).
duration (timedelta64) – duration of the accumulation.
interval (timedelta64) – intervals between each timestamp.
exclude_end (Boolean) – If true, no timestamp for the end time.
format (str) – format of the datetime string, referencing time.strftime().

Returns

timestamps – list of timestamps

Return type

a tuple of strings

swirlspy.qpe.utils.timestamps_period(start_time, end_time, interval, exclude_end=True, format='%y%m%d%H%M')

Generate a list of timestamp strings during the user-defined period.

Parameters

start_time (pandas.Timestamp) – start time of the accumulated rainfall.
end_time (pandas.Timestamp) – end time of the accumulated rainfall.
interval (pandas.Timedelta) – time intervals between each xarray object.
format (str) – format of the datetime string, referencing time.strftime().

Returns

timestamps – list of filenames

Return type

a tuple of strings