swirlspy.qpf.dl package

Submodules

swirlspy.qpf.dl.config module

swirlspy.qpf.dl.config.cfg_from_file(file_name, target)

Load a config file and merge it into the default options.

Parameters:
swirlspy.qpf.dl.config.load_latest_cfg(dir_path, target)

Load latest configuration

Parameters:
swirlspy.qpf.dl.config.ordered_dump(data, stream=None, Dumper=<class 'yaml.dumper.SafeDumper'>, **kwds)
swirlspy.qpf.dl.config.save_cfg(dir_path, source)

Save config file

Parameters:

swirlspy.qpf.dl.encoder_forecaster module

class swirlspy.qpf.dl.encoder_forecaster.EncoderForecasterBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, ctx_num=1, name='encoder_forecaster')

Bases: swirlspy.qpf.dl.prediction_base_factory.PredictionBaseFactory

encoder_data_desc()
encoder_sym()
forecaster_data_desc()
forecaster_sym()
init_encoder_state_info
init_forecaster_state_info
init_gan_state_info
loss_data_desc()
loss_label_desc()
loss_sym()
stack_rnn_encode(data)
stack_rnn_forecast(block_state_list, last_frame)
class swirlspy.qpf.dl.encoder_forecaster.EncoderForecasterStates(factory, ctx)

Bases: object

get_encoder_states()
get_forecaster_state()
reset_all()
reset_batch(batch_id)
update(states_nd)
swirlspy.qpf.dl.encoder_forecaster.encoder_forecaster_build_networks(factory, context, shared_encoder_net=None, shared_forecaster_net=None, shared_loss_net=None, for_finetune=False)
Parameters:
swirlspy.qpf.dl.encoder_forecaster.get_encoder_forecaster_rnn_blocks(batch_size)
swirlspy.qpf.dl.encoder_forecaster.init_optimizer_using_cfg(net, for_finetune)
swirlspy.qpf.dl.encoder_forecaster.load_encoder_forecaster_params(load_dir, load_iter, encoder_net, forecaster_net)
swirlspy.qpf.dl.encoder_forecaster.train_step(batch_size, encoder_net, forecaster_net, loss_net, init_states, data_nd, gt_nd, mask_nd, u_vector_nd=None, v_vector_nd=None, source_term_nd=None, iter_id=None)

Finetune the encoder, forecaster and GAN for one step

Parameters:
  • batch_size (int) –
  • encoder_net (MyModule) –
  • forecaster_net (MyModule) –
  • loss_net (MyModule) –
  • init_states (EncoderForecasterStates) –
  • data_nd (mx.nd.ndarray) –
  • gt_nd (mx.nd.ndarray) –
  • mask_nd (mx.nd.ndarray) –
  • u_vector_nd (mx.nd.ndarray or None) –
  • v_vector_nd (mx.nd.ndarray or None) –
  • source_term_nd (mx.nd.ndarray or None) –
  • iter_id (int) –
Returns:

  • init_states (EncoderForecasterStates)
  • loss_dict (dict)

swirlspy.qpf.dl.hko_benchmark module

class swirlspy.qpf.dl.hko_benchmark.HKOBenchmarkEnv(pd_path, save_dir='hko7_benchmark', mode='fixed')

Bases: object

The Benchmark environment for the HKO7 Dataset

There are two settings for the Benchmark, the “fixed” setting and the “online” setting. In the “fixed” setting, pre-defined input sequences that have the same length will be

fed into the model for prediction.
This setting tests the model’s ability to use the instant past to predict the future.
In the “online” setting, M frames will be given each time and the forecasting model
is required to predict the next K frames every stride steps.
If the begin_new_episode flag is turned on, a new episode has begun, which means that
the current received images have no relationship with the previous images.

If the need_upload_prediction flag is turned on, the model is required to predict the This setting tests both the model’s ability to adapt in an online fashion and

the ability to capture the long-term dependency.

The input frame will be missing in some timestamps.

To run the benchmark in the fixed setting:

env = HKOBenchmarkEnv(…) while not env.done:

# Get the observation in_frame_dat, in_mask_dat, in_datetime_clips, out_datetime_clips, begin_new_episode =

env.get_observation(batch_size=batch_size)

# Running your algorithm to get the prediction prediction = … # Upload prediction to the environment env.upload_prediction(prediction)

done
get_observation(batch_size=1)
Parameters:batch_size (int) –
Returns:
  • in_frame_dat (np.ndarray) – Will be between 0 and 1
  • in_datetime_clips (list)
  • out_datetime_clips (list)
  • begin_new_episode (bool)
  • need_upload_prediction (bool)
print_stat_readable()
reset()
save_eval(print_out=False)
upload_prediction(prediction)
Parameters:prediction (np.ndarray) –

swirlspy.qpf.dl.hko_evaluation module

class swirlspy.qpf.dl.hko_evaluation.HKOEvaluation(seq_len, use_central, no_ssim=True, threholds=None, central_region=None)

Bases: object

begin()
calculate_stat()

The following measurements will be used to measure the score of the forecaster

See Also [Weather and Forecasting 2010] Equitability Revisited: Why the “Equitable Threat Score” Is Not Equitable http://www.wxonline.info/topics/verif2.html

We will denote (a b (hits false alarms

c d) = misses correct negatives)

We will report the POD = a / (a + c) FAR = b / (a + b) CSI = a / (a + b + c) Heidke Skill Score (HSS) = 2(ad - bc) / ((a+c) (c+d) + (a+b)(b+d)) Gilbert Skill Score (GSS) = HSS / (2 - HSS), also known as the Equitable Threat Score

HSS = 2 * GSS / (GSS + 1)

MSE = mask * (pred - gt) **2 MAE = mask * abs(pred - gt) GDL = valid_mask_h * abs(gd_h(pred) - gd_h(gt)) + valid_mask_w * abs(gd_w(pred) - gd_w(gt))

clear_all()
print_stat_readable(prefix='')
save(prefix, print_out=False)
save_pkl(path)
save_txt_readable(path, print_out)
update(gt, pred, mask, start_datetimes=None)
Parameters:
  • gt (np.ndarray) –
  • pred (np.ndarray) –
  • mask (np.ndarray) – 0 indicates not use and 1 indicates that the location will be taken into account
  • start_datetimes (list) – The starting datetimes of all the testing instances
swirlspy.qpf.dl.hko_evaluation.dBZ_to_pixel(dBZ_img)
Parameters:dBZ_img (np.ndarray) –
swirlspy.qpf.dl.hko_evaluation.get_GDL(prediction, truth, mask, sum_batch=False)

Calculate the masked gradient difference loss

Parameters:
  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use
Returns:

gdl – Shape: (seq_len,) or (seq_len, batch_size)

Return type:

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_PSNR(prediction, truth)

Peak Signal Noise Ratio

Parameters:
  • prediction (np.ndarray) –
  • truth (np.ndarray) –
Returns:

ret

Return type:

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_SSIM(prediction, truth)

Calculate the SSIM score following [TIP2004] Image Quality Assessment: From Error Visibility to Structural Similarity

Same functionality as https://github.com/coupriec/VideoPredictionICLR2016/blob/master/image_error_measures.lua#L50-L75

We use nowcasting.helpers.msssim, which is borrowed from Tensorflow to do the evaluation

Parameters:
  • prediction (np.ndarray) –
  • truth (np.ndarray) –
Returns:

ret

Return type:

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_balancing_weights(data, mask, base_balancing_weights=None, thresholds=None)
swirlspy.qpf.dl.hko_evaluation.get_correlation(prediction, truth)
Parameters:
  • prediction (np.ndarray) –
  • truth (np.ndarray) –
swirlspy.qpf.dl.hko_evaluation.get_hit_miss_counts(prediction, truth, mask=None, thresholds=None, sum_batch=False)

This function calculates the overall hits and misses for the prediction, which could be used to get the skill scores and threat scores:

This function assumes the input, i.e, prediction and truth are 3-dim tensors, (timestep, row, col) and all inputs should be between 0~1

Parameters:
  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use
  • thresholds (list or tuple) –
Returns:

  • hits (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) TP
  • misses (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) FN
  • false_alarms (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) FP
  • correct_negatives (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) TN

swirlspy.qpf.dl.hko_evaluation.get_rainfall_mse(prediction, truth)
swirlspy.qpf.dl.hko_evaluation.pixel_to_dBZ(img)
Parameters:img (np.ndarray or float) –
swirlspy.qpf.dl.hko_evaluation.pixel_to_rainfall(img, a=None, b=None)

Convert the pixel values to real rainfall intensity

Parameters:
  • img (np.ndarray) –
  • a (float32, optional) –
  • b (float32, optional) –
Returns:

rainfall_intensity

Return type:

np.ndarray

swirlspy.qpf.dl.hko_evaluation.rainfall_to_pixel(rainfall_intensity, a=None, b=None)

Convert the rainfall intensity to pixel values

Parameters:
  • rainfall_intensity (np.ndarray) –
  • a (float32, optional) –
  • b (float32, optional) –
Returns:

pixel_vals

Return type:

np.ndarray

swirlspy.qpf.dl.hko_factory module

class swirlspy.qpf.dl.hko_factory.HKONowcastingFactory(batch_size, in_seq_len, out_seq_len, name='hko_nowcasting')

Bases: swirlspy.qpf.dl.encoder_forecaster.EncoderForecasterBaseFactory

loss_sym(pred=<Symbol pred>, mask=<Symbol mask>, target=<Symbol target>, source=<Symbol source>, motion_vector_field=[<Symbol u>, <Symbol v>])

Construct loss symbol.

Optional args:

pred: Shape (out_seq_len, batch_size, C, H, W) mask: Shape (out_seq_len, batch_size, C, H, W) target: Shape (out_seq_len, batch_size, C, H, W) source: Shape (out_seq_len, batch_size, C, H, W) motion_vector_field: Shape [(out_seq_len, batch_size, C, H, W),

(out_seq_len, batch_size, C, H, W)]
swirlspy.qpf.dl.hko_factory.get_loss_weight_symbol(data, mask, seq_len)

swirlspy.qpf.dl.hko_iterator module

class swirlspy.qpf.dl.hko_iterator.HKOIterator(pd_path, sample_mode, seq_len=30, max_consecutive_missing=2, begin_ind=None, end_ind=None, stride=None, width=None, height=None, base_freq='6min')

Bases: object

The iterator for HKO-7 dataset

begin_time
check_new_start()
end_time
get_airflow_terms(frames, batch_size)

Retrieve airflow terms: u and v

Parameters:
  • frames (np.ndarray) – Shape: (out_len+2, valid_batch_size, 1, height, width)
  • batch_size (int) – Batch size
Returns:

  • u (np.ndarray) – Shape: (out_len+1, valid_batch_size, 1, height, width)
  • v (np.ndarray) – Shape: (out_len+1, valid_batch_size, 1, height, width)

random_reset()
reset(begin_ind=None, end_ind=None)
sample(batch_size, only_return_datetime=False)

Sample a minibatch from the hko7 dataset based on the given type and pd_file

Parameters:
  • batch_size (int) – Batch size
  • only_return_datetime (bool) – Whether to only return the datetimes
Returns:

  • frame_dat (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)
  • mask_dat (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)
  • datetime_clips (list) – length should be valid_batch_size
  • new_start (bool)

sample_physical_params(frames, batch_size)

Sample u, v and source for physical loss function

Parameters:
  • frames (np.ndarray) – Shape: (out_len+2, valid_batch_size, 1, height, width)
  • batch_size (int) – Batch size
Returns:

  • u (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)
  • v (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)
  • source (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

set_begin_end(begin_ind=None, end_ind=None)
total_frame_num
use_up
class swirlspy.qpf.dl.hko_iterator.HKOSimpleBuffer(df, max_buffer_length, width, height)

Bases: object

get(timestamps)

timestamps must be sorted

Parameters:timestamps
reset()
swirlspy.qpf.dl.hko_iterator.convert_datetime_to_filepath(date_time)

Convert datetime to the filepath

Parameters:date_time (datetime.datetime) –
Returns:ret
Return type:str
swirlspy.qpf.dl.hko_iterator.convert_datetime_to_maskpath(date_time)

Convert datetime to path of the mask

Parameters:date_time (datetime.datetime) –
Returns:ret
Return type:str
swirlspy.qpf.dl.hko_iterator.decode_month(code)

Decode the month code back to the month value

Parameters:code (np.ndarray) – (…, 2) float
Returns:month – (…,) int
Return type:np.ndarray
swirlspy.qpf.dl.hko_iterator.encode_month(month)

Encode the month into a vector

Parameters:month (np.ndarray) – (…,) int, between 1 and 12
Returns:ret – (…, 2) float
Return type:np.ndarray
swirlspy.qpf.dl.hko_iterator.get_exclude_mask()
swirlspy.qpf.dl.hko_iterator.get_valid_datetime_set()
swirlspy.qpf.dl.hko_iterator.pad_hko_dat(frame_dat, mask_dat, batch_size)
swirlspy.qpf.dl.hko_iterator.precompute_mask(img)

swirlspy.qpf.dl.image module

exception swirlspy.qpf.dl.image.UnknownImageFormat

Bases: Exception

swirlspy.qpf.dl.image.cv2_read_img(path, read_storage, grayscale)
swirlspy.qpf.dl.image.cv2_read_img_resize(path, read_storage, resize_storage, frame_size, grayscale)
swirlspy.qpf.dl.image.quick_imsize(file_path)

Return (width, height) for a given img file content - no external dependencies except the os and struct modules from core

Parameters:file_path
swirlspy.qpf.dl.image.quick_read_frames(path_list, im_w=None, im_h=None, resize=False, frame_size=None, grayscale=True)

Multi-thread Frame Loader

Parameters:
  • path_list (list) –
  • resize (bool, optional) –
  • frame_size (None or tuple) –

swirlspy.qpf.dl.mask module

swirlspy.qpf.dl.mask.quick_read_masks(path_list)
swirlspy.qpf.dl.mask.read_mask_file(filepath, out)

Load mask file to numpy array

Parameters:
  • filepath (str) –
  • out (np.ndarray) –
swirlspy.qpf.dl.mask.save_mask_file(npy_mask, filepath)

swirlspy.qpf.dl.movingmnist_iterator module

class swirlspy.qpf.dl.movingmnist_iterator.MovingMNISTAdvancedIterator(digit_num=None, distractor_num=None, img_size=None, distractor_size=5, max_velocity_scale=3.6, initial_velocity_range=(0.0, 3.6), acceleration_range=(0.0, 0.0), scale_variation_range=(0.9090909090909091, 1.1), rotation_angle_range=(-30, 30), global_rotation_angle_range=(-30, 30), illumination_factor_range=(0.6, 1.0), period=5, global_rotation_prob=0.5, index_range=(0, 40000))

Bases: object

draw_distractors(canvas_img, distractor_seeds)
Parameters:canvas_img
draw_imgs(base_img, affine_transforms, prev_affine_transforms=None)
Parameters:
  • base_img (list) – Inner Shape: (H, W)
  • affine_transforms (np.ndarray) – Shape: (digit_num, 2, 3)
  • prev_affine_transforms (np.ndarray) – Shape: (digit_num, 2, 3)
load(file)

Initialize to draw samples from pre-computed parameters.

Parameters:
  • file (Either the file name (string) or an open file (file-like) –
  • from which the data will be loaded. (object)) –
sample(batch_size, seqlen, random=True)
Parameters:
  • batch_size (int) –
  • seqlen (int) –
  • random (take random samples from loaded parameters. Ignored if no parameters are loaded.) –
Returns:

  • seq (np.ndarray) – Shape: (seqlen, batch_size, 1, H, W)
  • motion_vectors (np.ndarray) – Shape: (seqlen, batch_size, 2, H, W)

save(seqlen, num_samples=10000, file=None)

Draw random numbers for num_samples sequences and save them.

This initializes the state of MovingMNISTAdvancedIterator to generate sequences based on the hereby drawn parameters.

Note that each call to sample(batch_size, seqlen) will use batch_size of the num_samples parameters.

Parameters:
  • num_samples (Number of unique MovingMNISTAdvanced sequences to draw) – parameters for
  • file (Either the file name (string) or an open file (file-like) – object) where the data will be saved. If file is a string or a Path, the .npz extension will be appended to the file name if it is not already there.
class swirlspy.qpf.dl.movingmnist_iterator.MovingMNISTIterator

Bases: object

sample(digitnum=3, width=64, height=64, seqlen=20, lower=3.0, upper=5.0, batch_size=32, index_range=(0, 50000))
Parameters:
  • digitnum
  • width
  • height
  • seqlen
  • batch_size
  • index_range
Returns:

seq – Shape: (seqlen, batch_size, 1, width, height)

Return type:

np.NDArray

swirlspy.qpf.dl.movingmnist_iterator.crop_mnist_digit(digit_img, tol=5)

Return the cropped version of the mnist digit

Parameters:digit_img (np.ndarray) – Shape: ()
swirlspy.qpf.dl.movingmnist_iterator.load_mnist(training_num=50000)

Load the mnist dataset

Parameters:training_num
swirlspy.qpf.dl.movingmnist_iterator.move_step(v0, p0, bounding_box)

swirlspy.qpf.dl.my_module module

class swirlspy.qpf.dl.my_module.MyModule(symbol, data_names=('data', ), label_names=('softmax_label', ), logger=<module 'logging' from '/opt/conda/envs/swirlspy/lib/python3.6/logging/__init__.py'>, context=gpu(0), work_load_list=None, fixed_param_names=None, state_names=None, name=None)

Bases: mxnet.module.module.Module

Some enhancement to the mx.mod.Module

acc_grad_with_tmp()
clear_grad()
clip_by_global_norm(max_norm=1.0)

Clips gradient norm. The norm is computed over all gradients together, as if they were

concatenated into a single vector. Gradients are modified in-place.
The method is first used in
[ICML2013] On the difficulty of training recurrent neural networks
Parameters:max_norm (float or int) – The maximum clipping threshold of the gradient norm.
Returns:norm_val – The computed norm of the gradients.
Return type:float

Examples

An example of using clip_grad_norm to clip the gradient before updating the parameters::
>>> #Get the gradient via back-propagation
>>> net.forward_backward(data_batch=data_batch)
>>> norm_val = net.clip_by_global_norm(max_norm=1.0)
>>> net.update()
debug_norm_all(debug_gnorm=True)
get_output_dict()
global_grad_norm()

Calculate global gradient norm. The L2 norm is computed over all gradients together, as if they were

concatenated into a single vector.
Could be used to debug the optimization process.
See http://videolectures.net/deeplearning2015_goodfellow_network_optimization/
Returns:norm_val – The computed norm of the gradients.
Return type:float

Examples

An example of using global_norm to calculate the gradient norm after back-propgation::
>>> #Get the gradient via back-propagation
>>> net.forward_backward(data_batch=data_batch)
>>> norm_val = net.global_grad_norm()
>>> print(norm_val)
load_params_allow_missing(fname)

Loads model parameters from file.

Parameters:fname (str) – Path to input param file.

Examples

>>> # An example of loading module parameters.
>>> mod.load_params('myfile')
save_tmp_grad()
summary(level=2)

Summarize the network parameters.

Parameters:level (int, optional) –

Level of the summarization logs to print. The log becomes more verbose with higher summary level. - Level = 0

Print the total param number + aux param number
  • Level = 1
    Print the shape of all parameters + The total number of paremter numbers
  • Level = 2
    Print the shape of the data/state and other available information in Level 1
swirlspy.qpf.dl.my_module.nd_global_norm(t_list)

Computes the global norm of multiple tensors.

Given a tuple or list of tensors t_list, this operation returns the global norm of the elements
in all tensors in t_list. The global norm is computed as:

global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))

Any entries in t_list that are of type None are ignored.

Parameters:t_list (list or tuple) – The NDArray list
Returns:ret – The global norm. The shape of the NDArray will be (1,)
Return type:NDArray

Examples

>>> x = mx.nd.ones((2, 3))
>>> y = mx.nd.ones((5, 6))
>>> z = mx.nd.ones((4, 2, 3))
>>> print(nd_global_norm([x, y, z]).asscalar())
7.74597
>>> xnone = None
>>> ret = nd_global_norm([x, y, z, xnone])
>>> print(ret.asscalar())
7.74597

swirlspy.qpf.dl.numba_accelerated module

swirlspy.qpf.dl.numba_accelerated.get_GDL_numba

//numba.pydata.org/)

Parameters:
  • prediction
  • truth
  • mask
Returns:

Return type:

gdl

Type:

Accelerated version of get_GDL using numba(http

swirlspy.qpf.dl.numba_accelerated.get_balancing_weights_numba(data, mask, base_balancing_weights=None, thresholds=None)

Get the balancing weights

Parameters:
  • data
  • mask
  • base_balancing_weights
  • thresholds
swirlspy.qpf.dl.numba_accelerated.get_hit_miss_counts_numba(prediction, truth, mask, thresholds=None)

This function calculates the overall hits and misses for the prediction, which could be used to get the skill scores and threat scores:

This function assumes the input, i.e, prediction and truth are 3-dim tensors, (timestep, row, col) and all inputs should be between 0~1

Parameters:
  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)
  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use
  • thresholds (list or tuple) –
Returns:

  • hits (np.ndarray) – (seq_len, batch_size, len(thresholds)) TP
  • misses (np.ndarray) – (seq_len, batch_size, len(thresholds)) FN
  • false_alarms (np.ndarray) – (seq_len, batch_size, len(thresholds)) FP
  • correct_negatives (np.ndarray) – (seq_len, batch_size, len(thresholds)) TN

swirlspy.qpf.dl.ops module

class swirlspy.qpf.dl.ops.ParamsReg

Bases: object

get(name, **kwargs)
get_inner()
reset()
swirlspy.qpf.dl.ops.activation(data, act_type, name=None)
swirlspy.qpf.dl.ops.conv2d(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, name=None, **kwargs)
swirlspy.qpf.dl.ops.conv2d_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.conv2d_bn_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.conv3d(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, name=None, **kwargs)
swirlspy.qpf.dl.ops.conv3d_act(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.conv3d_bn_act(data, num_filter, height, width, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.deconv2d(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, name='deconv2d', **kwargs)
swirlspy.qpf.dl.ops.deconv2d_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, act_type='relu', name='deconv2d', **kwargs)
swirlspy.qpf.dl.ops.deconv2d_bn_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name='deconv2d', use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.deconv3d(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, name=None, **kwargs)
swirlspy.qpf.dl.ops.deconv3d_act(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.deconv3d_bn_act(data, num_filter, height, width, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.downsample_module(data, num_filter, kernel, stride, pad, b_h_w, name, aggre_type=None)
swirlspy.qpf.dl.ops.fc_layer(data, num_hidden, no_bias=False, name='fc', **kwargs)
swirlspy.qpf.dl.ops.fc_layer_act(data, num_hidden, no_bias=False, act_type='relu', name='fc', **kwargs)
swirlspy.qpf.dl.ops.fc_layer_bn_act(data, num_hidden, no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.reset_regs()
swirlspy.qpf.dl.ops.upsample_module(data, num_filter, kernel, stride, pad, b_h_w, name, aggre_type=None)

swirlspy.qpf.dl.prediction_base_factory module

class swirlspy.qpf.dl.prediction_base_factory.PredictionBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, name='forecaster')

Bases: object

reset_all()
class swirlspy.qpf.dl.prediction_base_factory.RecursiveOneStepBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, use_ss=False, name='forecaster')

Bases: swirlspy.qpf.dl.prediction_base_factory.PredictionBaseFactory

swirlspy.qpf.dl.utils module

swirlspy.qpf.dl.utils.cross_entropy_npy(a, b)
swirlspy.qpf.dl.utils.load_params(prefix, epoch)
Parameters:
  • prefix (str) –
  • epoch (int) –
Returns:

  • arg_params (dict)
  • aux_params (dict)

swirlspy.qpf.dl.utils.logging_config(folder=None, name=None, level=20, console_level=10)
Parameters:
  • folder (str or None) –
  • name (str or None) –
  • level (int) –
  • console_level
swirlspy.qpf.dl.utils.parse_ctx(ctx_args)
swirlspy.qpf.dl.utils.safe_eval(expr)