swirlspy.qpf.dl.operators package

Submodules

swirlspy.qpf.dl.operators.base_rnn module

class swirlspy.qpf.dl.operators.base_rnn.BaseStackRNN(base_rnn_class, stack_num=1, name='BaseStackRNN', residual_connection=True, **kwargs)

Bases: object

check_concat(states)
concat_to_split(concat_states)
flatten_add_layout(states, blocked=False)
Parameters:states (list of list or list) –
Returns:ret
Return type:list
init_state_vars()

Initial state variable for this cell.

Returns:state_vars – starting states for first RNN step
Return type:nested list of Symbol
reset()
split_to_concat(split_states)
state_info
state_postfix
to_concat(states)
to_split(states)
unroll(length, inputs=None, begin_states=None, ret_mid=False)
class swirlspy.qpf.dl.operators.base_rnn.MyBaseRNNCell(prefix='MyBaseRNNCell', params=None)

Bases: mxnet.rnn.rnn_cell.BaseRNNCell

get_current_states()
reset()

Reset before re-using the cell for another graph.

unroll(length, inputs=None, begin_state=None, ret_mid=False, input_prefix='', layout='TC', merge_outputs=False)

Unroll an RNN cell across time steps.

Parameters:
  • length (int) – number of steps to unroll
  • inputs (Symbol, list of Symbol, or None) –

    if inputs is a single Symbol (usually the output of Embedding symbol), it should have shape (batch_size, length, …) if layout == ‘NTC’, or (length, batch_size, …) if layout == ‘TNC’.

    If inputs is a list of symbols (usually output of previous unroll), they should all have shape (batch_size, …).

    If inputs is None, Placeholder variables are automatically created.

  • begin_state (nested list of Symbol) – input states. Created by begin_state() or output state of another cell. Created from begin_state() if None.
  • input_prefix (str) – prefix for automatically created input placehodlers.
  • layout (str) – layout of input symbol. Only used if inputs is a single Symbol.
  • merge_outputs (bool) – if False, return outputs as a list of Symbols. If True, concatenate output across time steps and return a single symbol with shape (batch_size, length, …) if layout == ‘NTC’, or (length, batch_size, …) if layout == ‘TNC’.
Returns:

  • outputs (list of Symbol) – output symbols.
  • states (Symbol or nested list of Symbol) – has the same structure as begin_state()
  • mid_info (list of Symbol)

class swirlspy.qpf.dl.operators.base_rnn.MyGRU(num_hidden, zoneout=0.0, act_type='tanh', prefix='gru_', params=None)

Bases: swirlspy.qpf.dl.operators.base_rnn.MyBaseRNNCell

GRU cell.

Parameters:
  • num_hidden (int) – number of units in output symbol
  • prefix (str, default ‘rnn_’) – prefix for name of layers (and name of weight if params is None)
  • params (RNNParams or None) – container for weight sharing between cells. created if None.
state_info

shape(s) of states

swirlspy.qpf.dl.operators.common module

class swirlspy.qpf.dl.operators.common.ConstantOp(data)

Bases: mxnet.operator.CustomOp

Implementation of mask on minibatch layer.

backward(req, out_grad, in_data, out_data, in_grad, aux)

Backward interface. Can override when creating new operators.

Parameters:
  • req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward
forward(is_train, req, in_data, out_data, aux)

Forward interface. Can override when creating new operators.

Parameters:
  • is_train (bool) – whether this is for training
  • req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward
class swirlspy.qpf.dl.operators.common.ConstantOpProp(pkl_data)

Bases: mxnet.operator.CustomOpProp

create_operator(ctx, shapes, dtypes)

Create an operator that carries out the real computation given the context, input shapes, and input data types.

infer_shape(in_shape)

infer_shape interface. Can override when creating new operators.

Parameters:in_shape (list) – List of argument shapes in the same order as declared in list_arguments.
Returns:
  • in_shape (list) – List of argument shapes. Can be modified from in_shape.
  • out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.
  • aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.
list_arguments()

list_arguments interface. Can override when creating new operators.

Returns:arguments – List of argument blob names.
Return type:list
list_outputs()

list_outputs interface. Can override when creating new operators.

Returns:outputs – List of output blob names.
Return type:list
class swirlspy.qpf.dl.operators.common.EntropyMultinomialDist

Bases: mxnet.operator.CustomOp

backward(req, out_grad, in_data, out_data, in_grad, aux)

Backward interface. Can override when creating new operators.

Parameters:
  • req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward
forward(is_train, req, in_data, out_data, aux)

Forward interface. Can override when creating new operators.

Parameters:
  • is_train (bool) – whether this is for training
  • req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward
class swirlspy.qpf.dl.operators.common.EntropyMultinomialDistProp

Bases: mxnet.operator.CustomOpProp

create_operator(ctx, shapes, dtypes)

Create an operator that carries out the real computation given the context, input shapes, and input data types.

infer_shape(in_shape)

infer_shape interface. Can override when creating new operators.

Parameters:in_shape (list) – List of argument shapes in the same order as declared in list_arguments.
Returns:
  • in_shape (list) – List of argument shapes. Can be modified from in_shape.
  • out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.
  • aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.
list_arguments()

list_arguments interface. Can override when creating new operators.

Returns:arguments – List of argument blob names.
Return type:list
list_outputs()

list_outputs interface. Can override when creating new operators.

Returns:outputs – List of output blob names.
Return type:list
class swirlspy.qpf.dl.operators.common.IdentityOp(logging_prefix='identity', input_debug=False, grad_debug=False)

Bases: mxnet.operator.CustomOp

backward(req, out_grad, in_data, out_data, in_grad, aux)

Backward interface. Can override when creating new operators.

Parameters:
  • req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward
forward(is_train, req, in_data, out_data, aux)

Forward interface. Can override when creating new operators.

Parameters:
  • is_train (bool) – whether this is for training
  • req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward
class swirlspy.qpf.dl.operators.common.IdentityOpProp(logging_prefix='identity', input_debug=False, grad_debug=False)

Bases: mxnet.operator.CustomOpProp

create_operator(ctx, shapes, dtypes)

Create an operator that carries out the real computation given the context, input shapes, and input data types.

infer_shape(in_shape)

infer_shape interface. Can override when creating new operators.

Parameters:in_shape (list) – List of argument shapes in the same order as declared in list_arguments.
Returns:
  • in_shape (list) – List of argument shapes. Can be modified from in_shape.
  • out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.
  • aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.
list_arguments()

list_arguments interface. Can override when creating new operators.

Returns:arguments – List of argument blob names.
Return type:list
list_outputs()

list_outputs interface. Can override when creating new operators.

Returns:outputs – List of output blob names.
Return type:list
class swirlspy.qpf.dl.operators.common.LogisticRegressionMaskOutput(ignore_label)

Bases: mxnet.operator.CustomOp

backward(req, out_grad, in_data, out_data, in_grad, aux)

Backward interface. Can override when creating new operators.

Parameters:
  • req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward
forward(is_train, req, in_data, out_data, aux)

Forward interface. Can override when creating new operators.

Parameters:
  • is_train (bool) – whether this is for training
  • req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward
class swirlspy.qpf.dl.operators.common.LogisticRegressionMaskOutputProp(ignore_label)

Bases: mxnet.operator.CustomOpProp

create_operator(ctx, shapes, dtypes)

Create an operator that carries out the real computation given the context, input shapes, and input data types.

infer_shape(in_shape)

infer_shape interface. Can override when creating new operators.

Parameters:in_shape (list) – List of argument shapes in the same order as declared in list_arguments.
Returns:
  • in_shape (list) – List of argument shapes. Can be modified from in_shape.
  • out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.
  • aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.
list_arguments()

list_arguments interface. Can override when creating new operators.

Returns:arguments – List of argument blob names.
Return type:list
list_outputs()

list_outputs interface. Can override when creating new operators.

Returns:outputs – List of output blob names.
Return type:list
class swirlspy.qpf.dl.operators.common.SaveNpyOp(save_name='op', save_dir=None)

Bases: mxnet.operator.CustomOp

backward(req, out_grad, in_data, out_data, in_grad, aux)

Backward interface. Can override when creating new operators.

Parameters:
  • req (list of str) – how to assign to in_grad. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • in_data, out_data, in_grad, aux (out_grad,) – input and output for backward. See document for corresponding arguments of Operator::Backward
forward(is_train, req, in_data, out_data, aux)

Forward interface. Can override when creating new operators.

Parameters:
  • is_train (bool) – whether this is for training
  • req (list of str) – how to assign to out_data. can be ‘null’, ‘write’, or ‘add’. You can optionally use self.assign(dst, req, src) to handle this.
  • out_data, aux (in_data,) – input, output, and auxiliary states for forward. See document for corresponding arguments of Operator::Forward
class swirlspy.qpf.dl.operators.common.SaveNpyOpProp(save_name='op', save_dir='.')

Bases: mxnet.operator.CustomOpProp

create_operator(ctx, shapes, dtypes)

Create an operator that carries out the real computation given the context, input shapes, and input data types.

infer_shape(in_shape)

infer_shape interface. Can override when creating new operators.

Parameters:in_shape (list) – List of argument shapes in the same order as declared in list_arguments.
Returns:
  • in_shape (list) – List of argument shapes. Can be modified from in_shape.
  • out_shape (list) – List of output shapes calculated from in_shape, in the same order as declared in list_outputs.
  • aux_shape (Optional, list) – List of aux shapes calculated from in_shape, in the same order as declared in list_auxiliary_states.
list_arguments()

list_arguments interface. Can override when creating new operators.

Returns:arguments – List of argument blob names.
Return type:list
list_outputs()

list_outputs interface. Can override when creating new operators.

Returns:outputs – List of output blob names.
Return type:list
swirlspy.qpf.dl.operators.common.constant(data, name='constant')
swirlspy.qpf.dl.operators.common.entropy_multinomial(data, name='entropy')
swirlspy.qpf.dl.operators.common.grid_generator(batch_size, height, width, normalize=True)

Generate the grid based on width and height

Parameters:
  • batch_size (int) –
  • width (int) –
  • height (int) –
  • normalize (bool) – Whether to normalize the grid elements into [-1, 1]
Returns:

ret – Shape : (batch_size, 2, height, width), the channel contains (x, y)

Return type:

mx.sym.Symbol

swirlspy.qpf.dl.operators.common.group_add(lhs, rhs)
Parameters:
  • lhs (list of mx.sym.Symbol) –
  • rhs (list of mx.sym.Symbol) –
Returns:

ret

Return type:

list of mx.sym.Symbol

swirlspy.qpf.dl.operators.common.identity(data, name='identity', logging_prefix=None, input_debug=False, grad_debug=False)
swirlspy.qpf.dl.operators.common.logistic_regression_mask_output(data, label, ignore_label, name=None)
swirlspy.qpf.dl.operators.common.masked_gdl_loss(pred, gt, mask)
Parameters:
  • pred (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • gt (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • mask (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
Returns:

gdl – Shape: (seq_len, batch_size)

Return type:

mx.sym.Symbol

swirlspy.qpf.dl.operators.common.masked_hit_miss_counts(pred, gt, mask, thresholds)
Parameters:
  • pred (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • gt (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • mask (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • thresholds (list) –
Returns:

  • hits (mx.nd.NDArray) – Shape: (seq_len, batch_size, len(thresholds))
  • misses (mx.nd.NDArray) – Shape: (seq_len, batch_size, len(thresholds))
  • false_alarms (mx.nd.NDArray) – Shape: (seq_len, batch_size, len(thresholds))
  • correct_negatives (mx.nd.NDArray) – Shape: (seq_len, batch_size, len(thresholds))

swirlspy.qpf.dl.operators.common.multi_segment_slice_axis(data, axis, segment_lengths)

Split the data to multiple segments

Parameters:
  • data (mx.sym.Symbol) –
  • axis (int) –
  • segment_lengths (list or tuple) – Get the segment_lengths
Returns:

ret

Return type:

list

swirlspy.qpf.dl.operators.common.normalize_grid(un_norm_grid, width, height)

Normalize the grid to [-1, 1]

Parameters:
  • un_norm_grid (mx.sym.Symbol) – Shape : (batch_size, 2, height, width)
  • width (int) –
  • height (int) –
Returns:

ret

Return type:

mx.sym.Symbol

swirlspy.qpf.dl.operators.common.one_step_diff(dat, axis)
Parameters:
  • dat (mx.sym.Symbol) –
  • axes (tuple) –
swirlspy.qpf.dl.operators.common.save_npy(data, save_name='op', save_dir='.')
swirlspy.qpf.dl.operators.common.weighted_l1(pred, gt, weight)
swirlspy.qpf.dl.operators.common.weighted_l2(pred, gt, weight)
Parameters:
  • pred (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • gt (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
  • weight (mx.sym.Symbol) – Shape: (seq_len, batch_size, 1, H, W)
Returns:

l2 – Shape: (seq_len, batch_size)

Return type:

mx.nd.NDArray

swirlspy.qpf.dl.operators.common.weighted_mae(pred, gt, weight)
swirlspy.qpf.dl.operators.common.weighted_mse(pred, gt, weight)

swirlspy.qpf.dl.operators.conv_rnn module

class swirlspy.qpf.dl.operators.conv_rnn.BaseConvRNN(num_filter, b_h_w, h2h_kernel=(3, 3), h2h_dilate=(1, 1), i2h_kernel=(3, 3), i2h_stride=(1, 1), i2h_pad=(1, 1), i2h_dilate=(1, 1), act_type='tanh', prefix='ConvRNN', params=None)

Bases: swirlspy.qpf.dl.operators.base_rnn.MyBaseRNNCell

class swirlspy.qpf.dl.operators.conv_rnn.ConvGRU(num_filter, b_h_w, zoneout=0.0, h2h_kernel=(3, 3), h2h_dilate=(1, 1), i2h_kernel=(3, 3), i2h_stride=(1, 1), i2h_pad=(1, 1), i2h_dilate=(1, 1), i2h_adj=(0, 0), no_i2h_bias=False, use_deconv=False, act_type='leaky', prefix='ConvGRU', lr_mult=1.0)

Bases: swirlspy.qpf.dl.operators.conv_rnn.BaseConvRNN

state_info

shape and layout information of states

state_postfix
class swirlspy.qpf.dl.operators.conv_rnn.ConvRNN(num_filter, b_h_w, h2h_kernel=(3, 3), h2h_dilate=(1, 1), i2h_kernel=(3, 3), i2h_stride=(1, 1), i2h_pad=(1, 1), i2h_dilate=(1, 1), act_type='leaky', layer_norm=False, prefix='ConvRNN', params=None)

Bases: swirlspy.qpf.dl.operators.conv_rnn.BaseConvRNN

state_info

shape and layout information of states

swirlspy.qpf.dl.operators.traj_rnn module

class swirlspy.qpf.dl.operators.traj_rnn.TrajGRU(b_h_w, num_filter, zoneout=0.0, L=5, i2h_kernel=(3, 3), i2h_stride=(1, 1), i2h_pad=(1, 1), h2h_kernel=(5, 5), h2h_dilate=(1, 1), act_type='leaky', prefix='TrajGRU', lr_mult=1.0)

Bases: swirlspy.qpf.dl.operators.conv_rnn.BaseConvRNN

state_info

shape and layout information of states

state_postfix
swirlspy.qpf.dl.operators.traj_rnn.flow_conv(data, num_filter, flows, weight, bias, name)

swirlspy.qpf.dl.operators.transformations module

swirlspy.qpf.dl.operators.transformations.CDNA(data, kernels, mask, batch_size, num_filter, kernel_size)

We assume that the kernels and masks are the output of an identity activation

Parameters:
  • data (mx.sym.symbol) – Shape: (batch_size, C, H, W)
  • kernels (mx.sym.symbol) – Shape: (batch_size, M, K, K)
  • mask (mx.sym.symbol) – Shape: (batch_size, M, H, W)
  • batch_size (int) –
  • num_filter (int) – M
  • kernel_size (int) – K
Returns:

ret – Shape: (batch_size, C, H, W)

Return type:

mx.sym.symbol

swirlspy.qpf.dl.operators.transformations.DFN(data, local_kernels, K, batch_size)

[NIPS2016] Dynamic Filter Network

Parameters:
  • data (mx.sym.symbol) – Shape: (batch_size, C, H, W)
  • local_kernels (mx.sym.symbol) – Shape: (batch_size, K*K, H, W)
  • K (int) – size of the local convolutional kernel
  • batch_size (int) – size of the minibatch
swirlspy.qpf.dl.operators.transformations.STP(data, affine_transform_matrices, mask, num_filter, kernel_size)

Spatial Transformer Predictor

Parameters:
  • data (mx.sym.symbol) –
  • affine_transform_matrices
  • mask