SeparableConv2d

class selfeeg.models.layers.SeparableConv2d(in_channels, out_channels, kernel_size, stride=1, padding='same', dilation=1, bias=False, depth_multiplier=1, depth_max_norm=None, depth_min_norm=None, depth_minmax_rate=1.0, point_max_norm=None, point_min_norm=None, point_minmax_rate=1.0, axis_norm=[1, 2, 3])[source]

Separable Convolutional layer with norm constraints.

Pytorch implementation of the Separable Convolutional layer with the possibility of adding a norm constraint on both the depthwise and pointwise filters (feature) dimension. The layer applies first a depthwise conv2d, then a pointwise conv2d (kernel size = 1) Most of the parameters are the same as described in pytorch conv2D help.

Parameters:

  • in_channels (int) – Number of input channels.

  • out_channels (int) – Number of output channels.

  • kernel_size (int or tuple) – Size of the convolving kernel

  • stride (int or tuple, optional) –

    Stride of the convolution.

    Default = 1

  • padding (int, tuple or str, optional) –

    Padding added to all four sides of the input.

    Default = 0

  • dilation (int or tuple, optional) –

    Spacing between kernel elements.

    Default = 1

  • bias (bool, optional) –

    If True, adds a learnable bias to the output.

    Default = True

  • depth_multiplier (int, optional) –

    The depth multiplier of the depthwise block.

    Default = 1

  • depth_max_norm (float, optional) –

    The maximum norm each hidden unit in the depthwise layer can have. If None no constraint will be added.

    Default = None

  • depth_min_norm (float, optional) –

    The minimum norm each hidden unit in the depthwise layer can have. Must be a float lower than max_norm. If given, MinMaxNorm will be applied in the case max_norm is also given. Otherwise, it will be ignored.

    Default = None

  • depth_minmax_rate (float, optional) –

    A constraint for depthwise’s MinMaxNorm setting how weights will be rescaled at each step. It behaves like Keras rate argument of MinMaxNorm contraint. So, using minmax_rate = 1 will set a strict enforcement of the constraint, while rate<1.0 will slowly rescale layer’s hidden units at each step.

    Default = 1.0

  • axis_norm (Union[int, list, tuple], optional) –

    The axis along weights are constrained. It behaves like Keras. So, considering that a Conv2D layer has shape (output_depth, input_depth), set axis to 1 will constrain the weights of each filter tensor of size (input_depth,).

    Default = 1

  • point_max_norm (float, optional) –

    Same as depth_max_norm, but applied to the pointwise Convolutional layer.

    Default = None

  • point_min_norm (float, optional) –

    Same as depth_min_norm, but applied to the pointwise Convolutional layer.

    Default = None

  • point_minmax_rate (float, optional) –

    Same as depth_minmax_rate, but applied to the pointwise Convolutional layer.

    Default = 1.0

Example

>>> from selfeeg.models import SeparableConv2d
>>> import torch
>>> x = torch.randn(4, 1, 8, 64)
>>> mdl = SeparableConv2d(1,4, (1,15), depth_multiplier=4)
>>> out = mdl(x)
>>> print(out.shape) # shoud return torch.Size([4, 4, 8, 64])
>>> print(torch.isnan(out).sum()) # shoud return 0