swirlspy.qpf.dl package

Subpackages

Submodules

swirlspy.qpf.dl.config module

swirlspy.qpf.dl.config.cfg_from_file(file_name, target)

Load a config file and merge it into the default options.

Parameters

  • file_name (str) – config file path

  • target (.helpers.ordered_easydict.OrderedEasyDict) – default config

swirlspy.qpf.dl.config.load_latest_cfg(dir_path, target)

Load latest configuration

Parameters

  • dir_path (str) – path to load config

  • target (.helpers.ordered_easydict.OrderedEasyDict) – default configuration

swirlspy.qpf.dl.config.ordered_dump(data, stream=None, Dumper=<class 'yaml.dumper.SafeDumper'>, **kwds)
swirlspy.qpf.dl.config.save_cfg(dir_path, source)

Save config file

Parameters

  • dir_path (str) – path to save the config file

  • source (.helpers.ordered_easydict.OrderedEasyDict) – model configuration

swirlspy.qpf.dl.encoder_forecaster module

class swirlspy.qpf.dl.encoder_forecaster.EncoderForecasterBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, ctx_num=1, name='encoder_forecaster')

Bases: PredictionBaseFactory

encoder_data_desc()
encoder_sym()
forecaster_data_desc()
forecaster_sym()
property init_encoder_state_info
property init_forecaster_state_info
property init_gan_state_info
loss_data_desc()
loss_label_desc()
loss_sym()
stack_rnn_encode(data)
stack_rnn_forecast(block_state_list, last_frame)
class swirlspy.qpf.dl.encoder_forecaster.EncoderForecasterStates(factory, ctx)

Bases: object

get_encoder_states()
get_forecaster_state()
reset_all()
reset_batch(batch_id)
update(states_nd)
swirlspy.qpf.dl.encoder_forecaster.encoder_forecaster_build_networks(factory, context, shared_encoder_net=None, shared_forecaster_net=None, shared_loss_net=None, for_finetune=False)
Parameters
swirlspy.qpf.dl.encoder_forecaster.get_encoder_forecaster_rnn_blocks(batch_size)
swirlspy.qpf.dl.encoder_forecaster.init_optimizer_using_cfg(net, for_finetune)
swirlspy.qpf.dl.encoder_forecaster.load_encoder_forecaster_params(load_dir, load_iter, encoder_net, forecaster_net)
swirlspy.qpf.dl.encoder_forecaster.train_step(batch_size, encoder_net, forecaster_net, loss_net, init_states, data_nd, gt_nd, mask_nd, u_vector_nd=None, v_vector_nd=None, source_term_nd=None, iter_id=None)

Finetune the encoder, forecaster and GAN for one step

Parameters

  • batch_size (int) –

  • encoder_net (MyModule) –

  • forecaster_net (MyModule) –

  • loss_net (MyModule) –

  • init_states (EncoderForecasterStates) –

  • data_nd (mx.nd.ndarray) –

  • gt_nd (mx.nd.ndarray) –

  • mask_nd (mx.nd.ndarray) –

  • u_vector_nd (mx.nd.ndarray or None) –

  • v_vector_nd (mx.nd.ndarray or None) –

  • source_term_nd (mx.nd.ndarray or None) –

  • iter_id (int) –

Returns

  • init_states (EncoderForecasterStates)

  • loss_dict (dict)

swirlspy.qpf.dl.hko_benchmark module

class swirlspy.qpf.dl.hko_benchmark.HKOBenchmarkEnv(pd_path, save_dir='hko7_benchmark', mode='fixed')

Bases: object

The Benchmark environment for the HKO7 Dataset

There are two settings for the Benchmark, the “fixed” setting and the “online” setting. In the “fixed” setting, pre-defined input sequences that have the same length will be

fed into the model for prediction.

This setting tests the model’s ability to use the instant past to predict the future.

In the “online” setting, M frames will be given each time and the forecasting model
is required to predict the next K frames every stride steps.
If the begin_new_episode flag is turned on, a new episode has begun, which means that

the current received images have no relationship with the previous images.

If the need_upload_prediction flag is turned on, the model is required to predict the This setting tests both the model’s ability to adapt in an online fashion and

the ability to capture the long-term dependency.

The input frame will be missing in some timestamps.

To run the benchmark in the fixed setting:

env = HKOBenchmarkEnv(…) while not env.done:

# Get the observation in_frame_dat, in_mask_dat, in_datetime_clips, out_datetime_clips, begin_new_episode =

env.get_observation(batch_size=batch_size)

# Running your algorithm to get the prediction prediction = … # Upload prediction to the environment env.upload_prediction(prediction)

property done
get_observation(batch_size=1)
Parameters

batch_size (int) –

Returns

  • in_frame_dat (np.ndarray) – Will be between 0 and 1

  • in_datetime_clips (list)

  • out_datetime_clips (list)

  • begin_new_episode (bool)

  • need_upload_prediction (bool)

print_stat_readable()
reset()
save_eval(print_out=False)
upload_prediction(prediction)
Parameters

prediction (np.ndarray) –

swirlspy.qpf.dl.hko_evaluation module

class swirlspy.qpf.dl.hko_evaluation.HKOEvaluation(seq_len, use_central, no_ssim=True, threholds=None, central_region=None)

Bases: object

begin()
calculate_stat()

The following measurements will be used to measure the score of the forecaster

See Also [Weather and Forecasting 2010] Equitability Revisited: Why the “Equitable Threat Score” Is Not Equitable http://www.wxonline.info/topics/verif2.html

We will denote (a b (hits false alarms

c d) = misses correct negatives)

We will report the POD = a / (a + c) FAR = b / (a + b) CSI = a / (a + b + c) Heidke Skill Score (HSS) = 2(ad - bc) / ((a+c) (c+d) + (a+b)(b+d)) Gilbert Skill Score (GSS) = HSS / (2 - HSS), also known as the Equitable Threat Score

HSS = 2 * GSS / (GSS + 1)

MSE = mask * (pred - gt) **2 MAE = mask * abs(pred - gt) GDL = valid_mask_h * abs(gd_h(pred) - gd_h(gt)) + valid_mask_w * abs(gd_w(pred) - gd_w(gt))

clear_all()
print_stat_readable(prefix='')
save(prefix, print_out=False)
save_pkl(path)
save_txt_readable(path, print_out)
update(gt, pred, mask, start_datetimes=None)
Parameters

  • gt (np.ndarray) –

  • pred (np.ndarray) –

  • mask (np.ndarray) – 0 indicates not use and 1 indicates that the location will be taken into account

  • start_datetimes (list) – The starting datetimes of all the testing instances

swirlspy.qpf.dl.hko_evaluation.dBZ_to_pixel(dBZ_img)
Parameters

dBZ_img (np.ndarray) –

swirlspy.qpf.dl.hko_evaluation.get_GDL(prediction, truth, mask, sum_batch=False)

Calculate the masked gradient difference loss

Parameters

  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use

Returns

gdl – Shape: (seq_len,) or (seq_len, batch_size)

Return type

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_PSNR(prediction, truth)

Peak Signal Noise Ratio

Parameters

  • prediction (np.ndarray) –

  • truth (np.ndarray) –

Returns

ret

Return type

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_SSIM(prediction, truth)

Calculate the SSIM score following [TIP2004] Image Quality Assessment: From Error Visibility to Structural Similarity

Same functionality as https://github.com/coupriec/VideoPredictionICLR2016/blob/master/image_error_measures.lua#L50-L75

We use nowcasting.helpers.msssim, which is borrowed from Tensorflow to do the evaluation

Parameters

  • prediction (np.ndarray) –

  • truth (np.ndarray) –

Returns

ret

Return type

np.ndarray

swirlspy.qpf.dl.hko_evaluation.get_balancing_weights(data, mask, base_balancing_weights=None, thresholds=None)
swirlspy.qpf.dl.hko_evaluation.get_correlation(prediction, truth)
Parameters

  • prediction (np.ndarray) –

  • truth (np.ndarray) –

swirlspy.qpf.dl.hko_evaluation.get_hit_miss_counts(prediction, truth, mask=None, thresholds=None, sum_batch=False)

This function calculates the overall hits and misses for the prediction, which could be used to get the skill scores and threat scores:

This function assumes the input, i.e, prediction and truth are 3-dim tensors, (timestep, row, col) and all inputs should be between 0~1

Parameters

  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use

  • thresholds (list or tuple) –

Returns

  • hits (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) TP

  • misses (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) FN

  • false_alarms (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) FP

  • correct_negatives (np.ndarray) – (seq_len, len(thresholds)) or (seq_len, batch_size, len(thresholds)) TN

swirlspy.qpf.dl.hko_evaluation.get_rainfall_mse(prediction, truth)
swirlspy.qpf.dl.hko_evaluation.pixel_to_dBZ(img)
Parameters

img (np.ndarray or float) –

swirlspy.qpf.dl.hko_evaluation.pixel_to_rainfall(img, a=None, b=None)

Convert the pixel values to real rainfall intensity

Parameters

  • img (np.ndarray) –

  • a (float32, optional) –

  • b (float32, optional) –

Returns

rainfall_intensity

Return type

np.ndarray

swirlspy.qpf.dl.hko_evaluation.rainfall_to_pixel(rainfall_intensity, a=None, b=None)

Convert the rainfall intensity to pixel values

Parameters

  • rainfall_intensity (np.ndarray) –

  • a (float32, optional) –

  • b (float32, optional) –

Returns

pixel_vals

Return type

np.ndarray

swirlspy.qpf.dl.hko_factory module

class swirlspy.qpf.dl.hko_factory.HKONowcastingFactory(batch_size, in_seq_len, out_seq_len, name='hko_nowcasting')

Bases: EncoderForecasterBaseFactory

loss_sym(pred=<Symbol pred>, mask=<Symbol mask>, target=<Symbol target>, source=<Symbol source>, motion_vector_field=[<Symbol u>, <Symbol v>])

Construct loss symbol.

Optional args:

pred: Shape (out_seq_len, batch_size, C, H, W) mask: Shape (out_seq_len, batch_size, C, H, W) target: Shape (out_seq_len, batch_size, C, H, W) source: Shape (out_seq_len, batch_size, C, H, W) motion_vector_field: Shape [(out_seq_len, batch_size, C, H, W),

(out_seq_len, batch_size, C, H, W)]

swirlspy.qpf.dl.hko_factory.get_loss_weight_symbol(data, mask, seq_len)

swirlspy.qpf.dl.hko_iterator module

class swirlspy.qpf.dl.hko_iterator.HKOIterator(pd_path, sample_mode, seq_len=30, max_consecutive_missing=2, begin_ind=None, end_ind=None, stride=None, width=None, height=None, base_freq='6min')

Bases: object

The iterator for HKO-7 dataset

property begin_time
check_new_start()
property end_time
get_airflow_terms(frames, batch_size)

Retrieve airflow terms: u and v

Parameters

  • frames (np.ndarray) – Shape: (out_len+2, valid_batch_size, 1, height, width)

  • batch_size (int) – Batch size

Returns

  • u (np.ndarray) – Shape: (out_len+1, valid_batch_size, 1, height, width)

  • v (np.ndarray) – Shape: (out_len+1, valid_batch_size, 1, height, width)

random_reset()
reset(begin_ind=None, end_ind=None)
sample(batch_size, only_return_datetime=False)

Sample a minibatch from the hko7 dataset based on the given type and pd_file

Parameters

  • batch_size (int) – Batch size

  • only_return_datetime (bool) – Whether to only return the datetimes

Returns

  • frame_dat (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

  • mask_dat (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

  • datetime_clips (list) – length should be valid_batch_size

  • new_start (bool)

sample_physical_params(frames, batch_size)

Sample u, v and source for physical loss function

Parameters

  • frames (np.ndarray) – Shape: (out_len+2, valid_batch_size, 1, height, width)

  • batch_size (int) – Batch size

Returns

  • u (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

  • v (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

  • source (np.ndarray) – Shape: (seq_len, valid_batch_size, 1, height, width)

set_begin_end(begin_ind=None, end_ind=None)
property total_frame_num
property use_up
class swirlspy.qpf.dl.hko_iterator.HKOSimpleBuffer(df, max_buffer_length, width, height)

Bases: object

get(timestamps)

timestamps must be sorted

Parameters

timestamps –

reset()
swirlspy.qpf.dl.hko_iterator.convert_datetime_to_filepath(date_time)

Convert datetime to the filepath

Parameters

date_time (datetime.datetime) –

Returns

ret

Return type

str

swirlspy.qpf.dl.hko_iterator.convert_datetime_to_maskpath(date_time)

Convert datetime to path of the mask

Parameters

date_time (datetime.datetime) –

Returns

ret

Return type

str

swirlspy.qpf.dl.hko_iterator.decode_month(code)

Decode the month code back to the month value

Parameters

code (np.ndarray) – (…, 2) float

Returns

month – (…,) int

Return type

np.ndarray

swirlspy.qpf.dl.hko_iterator.encode_month(month)

Encode the month into a vector

Parameters

month (np.ndarray) – (…,) int, between 1 and 12

Returns

ret – (…, 2) float

Return type

np.ndarray

swirlspy.qpf.dl.hko_iterator.get_exclude_mask()
swirlspy.qpf.dl.hko_iterator.get_valid_datetime_set()
swirlspy.qpf.dl.hko_iterator.pad_hko_dat(frame_dat, mask_dat, batch_size)
swirlspy.qpf.dl.hko_iterator.precompute_mask(img)

swirlspy.qpf.dl.image module

exception swirlspy.qpf.dl.image.UnknownImageFormat

Bases: Exception

swirlspy.qpf.dl.image.cv2_read_img(path, read_storage, grayscale)
swirlspy.qpf.dl.image.cv2_read_img_resize(path, read_storage, resize_storage, frame_size, grayscale)
swirlspy.qpf.dl.image.quick_imsize(file_path)

Return (width, height) for a given img file content - no external dependencies except the os and struct modules from core

Parameters

file_path –

swirlspy.qpf.dl.image.quick_read_frames(path_list, im_w=None, im_h=None, resize=False, frame_size=None, grayscale=True)

Multi-thread Frame Loader

Parameters

  • path_list (list) –

  • resize (bool, optional) –

  • frame_size (None or tuple) –

swirlspy.qpf.dl.mask module

swirlspy.qpf.dl.mask.quick_read_masks(path_list)
swirlspy.qpf.dl.mask.read_mask_file(filepath, out)

Load mask file to numpy array

Parameters

  • filepath (str) –

  • out (np.ndarray) –

swirlspy.qpf.dl.mask.save_mask_file(npy_mask, filepath)

swirlspy.qpf.dl.movingmnist_iterator module

class swirlspy.qpf.dl.movingmnist_iterator.MovingMNISTAdvancedIterator(digit_num=None, distractor_num=None, img_size=None, distractor_size=5, max_velocity_scale=3.6, initial_velocity_range=(0.0, 3.6), acceleration_range=(0.0, 0.0), scale_variation_range=(0.9090909090909091, 1.1), rotation_angle_range=(-30, 30), global_rotation_angle_range=(-30, 30), illumination_factor_range=(0.6, 1.0), period=5, global_rotation_prob=0.5, index_range=(0, 40000))

Bases: object

draw_distractors(canvas_img, distractor_seeds)
Parameters

canvas_img –

draw_imgs(base_img, affine_transforms, prev_affine_transforms=None)
Parameters

  • base_img (list) – Inner Shape: (H, W)

  • affine_transforms (np.ndarray) – Shape: (digit_num, 2, 3)

  • prev_affine_transforms (np.ndarray) – Shape: (digit_num, 2, 3)

load(file)

Initialize to draw samples from pre-computed parameters.

Parameters

  • file (Either the file name (string) or an open file (file-like) –

  • loaded. (object) from which the data will be) –

sample(batch_size, seqlen, random=True)
Parameters

  • batch_size (int) –

  • seqlen (int) –

  • random (take random samples from loaded parameters. Ignored if no parameters are loaded.) –

Returns

  • seq (np.ndarray) – Shape: (seqlen, batch_size, 1, H, W)

  • motion_vectors (np.ndarray) – Shape: (seqlen, batch_size, 2, H, W)

save(seqlen, num_samples=10000, file=None)

Draw random numbers for num_samples sequences and save them.

This initializes the state of MovingMNISTAdvancedIterator to generate sequences based on the hereby drawn parameters.

Note that each call to sample(batch_size, seqlen) will use batch_size of the num_samples parameters.

Parameters

  • num_samples (Number of unique MovingMNISTAdvanced sequences to draw) – parameters for

  • file (Either the file name (string) or an open file (file-like) – object) where the data will be saved. If file is a string or a Path, the .npz extension will be appended to the file name if it is not already there.

class swirlspy.qpf.dl.movingmnist_iterator.MovingMNISTIterator

Bases: object

sample(digitnum=3, width=64, height=64, seqlen=20, lower=3.0, upper=5.0, batch_size=32, index_range=(0, 50000))
Parameters

  • digitnum –

  • width –

  • height –

  • seqlen –

  • batch_size –

  • index_range –

Returns

seq – Shape: (seqlen, batch_size, 1, width, height)

Return type

np.NDArray

swirlspy.qpf.dl.movingmnist_iterator.crop_mnist_digit(digit_img, tol=5)

Return the cropped version of the mnist digit

Parameters

digit_img (np.ndarray) – Shape: ()

swirlspy.qpf.dl.movingmnist_iterator.load_mnist(training_num=50000)

Load the mnist dataset

Parameters

training_num –

swirlspy.qpf.dl.movingmnist_iterator.move_step(v0, p0, bounding_box)

swirlspy.qpf.dl.my_module module

class swirlspy.qpf.dl.my_module.MyModule(symbol, data_names=('data', ), label_names=('softmax_label', ), logger=<module 'logging' from '/opt/conda/envs/swirlspy/lib/python3.6/logging/__init__.py'>, context=gpu(0), work_load_list=None, fixed_param_names=None, state_names=None, name=None)

Bases: Module

Some enhancement to the mx.mod.Module

acc_grad_with_tmp()
clear_grad()
clip_by_global_norm(max_norm=1.0)

Clips gradient norm. The norm is computed over all gradients together, as if they were

concatenated into a single vector. Gradients are modified in-place.

The method is first used in

[ICML2013] On the difficulty of training recurrent neural networks

Parameters

max_norm (float or int) – The maximum clipping threshold of the gradient norm.

Returns

norm_val – The computed norm of the gradients.

Return type

float

Examples

An example of using clip_grad_norm to clip the gradient before updating the parameters::
>>> #Get the gradient via back-propagation
>>> net.forward_backward(data_batch=data_batch)
>>> norm_val = net.clip_by_global_norm(max_norm=1.0)
>>> net.update()
debug_norm_all(debug_gnorm=True)
get_output_dict()
global_grad_norm()

Calculate global gradient norm. The L2 norm is computed over all gradients together, as if they were

concatenated into a single vector.

Could be used to debug the optimization process.

See http://videolectures.net/deeplearning2015_goodfellow_network_optimization/

Returns

norm_val – The computed norm of the gradients.

Return type

float

Examples

An example of using global_norm to calculate the gradient norm after back-propgation::
>>> #Get the gradient via back-propagation
>>> net.forward_backward(data_batch=data_batch)
>>> norm_val = net.global_grad_norm()
>>> print(norm_val)
load_params_allow_missing(fname)

Loads model parameters from file.

Parameters

fname (str) – Path to input param file.

Examples

>>> # An example of loading module parameters.
>>> mod.load_params('myfile')
save_tmp_grad()
summary(level=2)

Summarize the network parameters.

Parameters

level (int, optional) – Level of the summarization logs to print. The log becomes more verbose with higher summary level. - Level = 0, print the total param number + aux param number - Level = 1, print the shape of all parameters + The total number of paremter numbers - Level = 2, print the shape of the data/state and other available information in Level 1

swirlspy.qpf.dl.my_module.nd_global_norm(t_list)

Computes the global norm of multiple tensors.

Given a tuple or list of tensors t_list, this operation returns the global norm of the elements

in all tensors in t_list. The global norm is computed as:

global_norm = sqrt(sum([l2norm(t)**2 for t in t_list]))

Any entries in t_list that are of type None are ignored.

Parameters

t_list (list or tuple) – The NDArray list

Returns

ret – The global norm. The shape of the NDArray will be (1,)

Return type

NDArray

Examples

>>> x = mx.nd.ones((2, 3))
>>> y = mx.nd.ones((5, 6))
>>> z = mx.nd.ones((4, 2, 3))
>>> print(nd_global_norm([x, y, z]).asscalar())
7.74597
>>> xnone = None
>>> ret = nd_global_norm([x, y, z, xnone])
>>> print(ret.asscalar())
7.74597

swirlspy.qpf.dl.numba_accelerated module

swirlspy.qpf.dl.numba_accelerated.get_GDL_numba(prediction, truth, mask)

Accelerated version of get_GDL using numba(http://numba.pydata.org/)

Parameters

  • prediction –

  • truth –

  • mask –

Return type

gdl

swirlspy.qpf.dl.numba_accelerated.get_balancing_weights_numba(data, mask, base_balancing_weights=None, thresholds=None)

Get the balancing weights

Parameters

  • data –

  • mask –

  • base_balancing_weights –

  • thresholds –

swirlspy.qpf.dl.numba_accelerated.get_hit_miss_counts_numba(prediction, truth, mask, thresholds=None)

This function calculates the overall hits and misses for the prediction, which could be used to get the skill scores and threat scores:

This function assumes the input, i.e, prediction and truth are 3-dim tensors, (timestep, row, col) and all inputs should be between 0~1

Parameters

  • prediction (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • truth (np.ndarray) – Shape: (seq_len, batch_size, 1, height, width)

  • mask (np.ndarray or None) – Shape: (seq_len, batch_size, 1, height, width) 0 –> not use 1 –> use

  • thresholds (list or tuple) –

Returns

  • hits (np.ndarray) – (seq_len, batch_size, len(thresholds)) TP

  • misses (np.ndarray) – (seq_len, batch_size, len(thresholds)) FN

  • false_alarms (np.ndarray) – (seq_len, batch_size, len(thresholds)) FP

  • correct_negatives (np.ndarray) – (seq_len, batch_size, len(thresholds)) TN

swirlspy.qpf.dl.ops module

class swirlspy.qpf.dl.ops.ParamsReg

Bases: object

get(name, **kwargs)
get_inner()
reset()
swirlspy.qpf.dl.ops.activation(data, act_type, name=None)
swirlspy.qpf.dl.ops.conv2d(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, name=None, **kwargs)
swirlspy.qpf.dl.ops.conv2d_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.conv2d_bn_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), dilate=(1, 1), no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.conv3d(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, name=None, **kwargs)
swirlspy.qpf.dl.ops.conv3d_act(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.conv3d_bn_act(data, num_filter, height, width, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), dilate=(1, 1, 1), no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.deconv2d(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, name='deconv2d', **kwargs)
swirlspy.qpf.dl.ops.deconv2d_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, act_type='relu', name='deconv2d', **kwargs)
swirlspy.qpf.dl.ops.deconv2d_bn_act(data, num_filter, kernel=(1, 1), stride=(1, 1), pad=(0, 0), adj=(0, 0), no_bias=True, target_shape=None, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name='deconv2d', use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.deconv3d(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, name=None, **kwargs)
swirlspy.qpf.dl.ops.deconv3d_act(data, num_filter, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, act_type='relu', name=None, **kwargs)
swirlspy.qpf.dl.ops.deconv3d_bn_act(data, num_filter, height, width, kernel=(1, 1, 1), stride=(1, 1, 1), pad=(0, 0, 0), adj=(0, 0, 0), no_bias=True, target_shape=None, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.downsample_module(data, num_filter, kernel, stride, pad, b_h_w, name, aggre_type=None)
swirlspy.qpf.dl.ops.fc_layer(data, num_hidden, no_bias=False, name='fc', **kwargs)
swirlspy.qpf.dl.ops.fc_layer_act(data, num_hidden, no_bias=False, act_type='relu', name='fc', **kwargs)
swirlspy.qpf.dl.ops.fc_layer_bn_act(data, num_hidden, no_bias=False, act_type='relu', momentum=0.9, eps=1.0000001e-05, fix_gamma=True, name=None, use_global_stats=False, **kwargs)
swirlspy.qpf.dl.ops.reset_regs()
swirlspy.qpf.dl.ops.upsample_module(data, num_filter, kernel, stride, pad, b_h_w, name, aggre_type=None)

swirlspy.qpf.dl.prediction_base_factory module

class swirlspy.qpf.dl.prediction_base_factory.PredictionBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, name='forecaster')

Bases: object

reset_all()
class swirlspy.qpf.dl.prediction_base_factory.RecursiveOneStepBaseFactory(batch_size, in_seq_len, out_seq_len, height, width, use_ss=False, name='forecaster')

Bases: PredictionBaseFactory

swirlspy.qpf.dl.utils module

swirlspy.qpf.dl.utils.cross_entropy_npy(a, b)
swirlspy.qpf.dl.utils.load_params(prefix, epoch)
Parameters

  • prefix (str) –

  • epoch (int) –

Returns

  • arg_params (dict)

  • aux_params (dict)

swirlspy.qpf.dl.utils.logging_config(folder=None, name=None, level=20, console_level=10)
Parameters

  • folder (str or None) –

  • name (str or None) –

  • level (int) –

  • console_level –

swirlspy.qpf.dl.utils.parse_ctx(ctx_args)
swirlspy.qpf.dl.utils.safe_eval(expr)