ReconstructiveSSL

class selfeeg.ssl.generative.ReconstructiveSSL(encoder: Module, decoder: list[int] | Module)[source]

Implementation of a reconstructive pretraining method.

The task is to reconstruct the original EEG sample from its augmented version.

Parameters:

  • encoder (nn.Module) – The encoder part of the module. It is the one you wish to pretrain and transfer to the new model

  • decoder (nn.Module) – The decoder part of the module.

Warning

This class will not check the compatibility of the encoder’s output and the decoder’s input. Make sure that they have the same size.

Example

>>> import pickle, torch, selfeeg
>>> import selfeeg.dataloading as dl
>>> utils.create_dataset()
>>> def loadEEG(path, return_label=False):
...     with open(path, 'rb') as handle:
...         EEG = pickle.load(handle)
...     x , y= EEG['data'], EEG['label']
...     return (x, y) if return_label else x
>>> EEGlen = dl.get_eeg_partition_number('Simulated_EEG',freq=128, window=1,
...                              overlap=0.3, load_function=loadEEG)
>>> EEGsplit = dl.get_eeg_split_table (EEGlen, seed=1234)
>>> TrainSet = dl.EEGDataset(EEGlen,EEGsplit,[128,1,0.3],'train',False,loadEEG)
>>> Loader = torch.utils.data.DataLoader(TrainSet, batch_size=32)
>>> enc = selfeeg.models.ShallowNetEncoder(8)
>>> dec = ... # custom decoder
>>> generative = selfeeg.ssl.ReconstructSSL(enc, dec)
>>> loss_train = simclr.fit(Loader, 1, return_loss_info=True)
fit(train_dataloader, epochs=1, optimizer=None, augmenter=None, loss_func: Callable = None, loss_args: list = [], lr_scheduler=None, EarlyStopper=None, validation_dataloader=None, verbose=True, device: str = None, return_loss_info: bool = False)[source]

fit is a custom fit function designed to perform pretraining on a given model with the given dataloader.

Parameters:

  • train_dataloader (Dataloader) – The pytorch Dataloader used to get the training batches. It must return a batch as a single tensor X, thus without label tensor Y.

  • epochs (int, optional) –

    The number of training epochs. Must be an integer bigger than 0.

    Default = 1

  • optimizer (torch Optimizer, optional) –

    The optimizer used for weight’s update. It can be any optimizer provided in the torch.optim module. If not given Adam with default parameters will be instantiated.

    Default = torch.optim.Adam

  • augmenter (function, optional) –

    Any function (or callable object) used to perform data augmentation on the batch. It is highly suggested to resort to the augmentation module, which implements different data augmentation functions and classes to combine them. If none is given, a default augmentation with random vertical flip + random noise is applied. Note that in this case, contrary to fully supervised approaches, data augmentation is also performed on the validation set, since it’s part of the SSL algorithm.

    Default = None

  • loss_func (Callable, optional) –

    The custom loss function. It can be any loss function which accepts as input only the model’s predictions as required arguments and loss_args as optional arguments.

    Default = None

  • loss_args (Union[list, dict], optional) –

    The optional arguments to pass to the function. It can be a list or a dict.

    Default = None

  • lr_scheduler (torch Scheduler) –

    A pytorch learning rate scheduler used to update the learning rate during the fine-tuning.

    Default = None

  • EarlyStopper (EarlyStopping, optional) –

    An instance of the provided EarlyStopping class.

    Default = None

    Note

    If an EarlyStopping instance is given with monitoring loss set to validation loss, but no validation dataloader is given, monitoring loss will be automatically set to training loss.

  • validation_dataloader (Dataloader, optional) –

    the pytorch Dataloader used to get the validation batches. It must return a batch as a single tensor X, thus without label tensor Y. If not given, no validation loss will be calculated

    Default = None

  • verbose (bool, optional) –

    Whether to print a progression bar or not.

    Default = None

  • device (torch.device or str, optional) –

    The device to use for fine-tuning. If given as a string it will be converted in a torch.device instance. If not given, ‘cpu’ device will be used.

    Default = None

  • return_loss_info (bool, optional) –

    Whether to return the calculated training validation losses at each epoch.

    Default = False

Returns:

loss_info (dict, optional) – A dictionary with keys being the epoch number (as integer) and values a two element list with the average epoch’s training and validation loss.

forward(x)[source]
test(test_dataloader, augmenter=None, loss_func: Callable = None, loss_args: list = [], verbose: bool = True, device: str = None)[source]

Evaluate the loss on a test dataloader. Parameters are the same as described in the fit method, aside for those related to model training which are removed.

It is rare to evaluate the pretraing loss function on a test set. Nevertheless this function provides a way to do that. An example of usage could be to assess the quality of the learned features on the fine-tuning dataset.