DepthwiseConv2d
- class selfeeg.models.layers.DepthwiseConv2d(in_channels, depth_multiplier, kernel_size, stride=1, padding='same', dilation=1, bias=False, max_norm=2.0, min_norm=None, axis_norm=[1, 2, 3], minmax_rate=1.0)[source]
Depthwise 2D layer with norm constraints.
Pytorch implementation of the Depthwise Convolutional layer with the possibility to add a MaxNorm, MinMaxNorm, or UnitNorm constraint along the given axis. Most of the parameters are the same as described in pytorch Conv2D help.
- Parameters:
in_channels (int) – Number of input channels.
depth_multiplier (int) – The depth multiplier. Output channels will be depth_multiplier*in_channels.
kernel_size (int or tuple) – Size of the convolving kernel.
stride (int or tuple, optional) –
Stride of the convolution.
Default = 1
padding (int, tuple or str, optional) –
Padding added to all four sides of the input.
Default = 0
dilation (int or tuple, optional) –
Spacing between kernel elements.
Default = 1
bias (bool, optional) –
If True, adds a learnable bias to the output.
Default = True
max_norm (float, optional) –
The maximum norm each hidden unit can have. If None no constraint will be added.
Default = 2.0
min_norm (float, optional) –
The minimum norm each hidden unit can have. Must be a float lower than max_norm. If given, MinMaxNorm will be applied in the case max_norm is also given. Otherwise, it will be ignored.
Default = None
axis_norm (Union[int, list, tuple], optional) –
The axis along weights are constrained. It behaves like Keras. So, considering that a Conv2D layer has shape (output_depth, input_depth, rows, cols), set axis to [1, 2, 3] will constrain the weights of each filter tensor of size (input_depth, rows, cols).
Default = [1,2,3]
minmax_rate (float, optional) –
A constraint for MinMaxNorm setting how weights will be rescaled at each step. It behaves like Keras rate argument of MinMaxNorm contraint. So, using minmax_rate = 1 will set a strict enforcement of the constraint, while rate<1.0 will slowly rescale layer’s hidden units at each step.
Default = 1.0
Note
To Apply a MaxNorm constraint, set only max_norm. To apply a MinMaxNorm constraint, set both min_norm and max_norm. To apply a UnitNorm constraint, set both min_norm and max_norm to 1.0.
Example
>>> from import selfeeg.models import DepthwiseConv2d >>> import torch >>> x = torch.randn(4,1,8,64) >>> mdl = DepthwiseConv2d(1, 2, (1, 15), max_norm = 0.5) >>> mdl.weight = torch.nn.Parameter(mdl.weight*10) >>> out = mdl(x) >>> norms = torch.sqrt(torch.sum(torch.square(mdl.weight), axis=[1,2,3])) >>> print(out.shape) # shoud return torch.Size([4, 2, 8, 64]) >>> print(torch.isnan(out).sum()) # shoud return 0 >>> print(torch.sum(norms>(0.5+1e-3)).item() == 0) # should return True