OCDocker.OCScore.DNN.DNNOptimizer module

Module to perform the optimization of the Neural Network.

It is imported as:

from OCDocker.OCScore.DNN.DNNOptimizer import DNNOptimizer

class OCDocker.OCScore.DNN.DNNOptimizer.NeuralNet(*args, **kwargs)[source]

Bases: Module

Neural Network class for the optimization of the neural network.

Parameters:

  • input_size (int) – Size of the input layer

  • output_size (int) – Size of the output layer

  • encoder_params (Union[None, dict, tuple[dict, dict, dict]]) – Parameters for the encoder

  • nn_params (dict) – Parameters for the neural network

  • random_seed (int, optional) – Random seed for the neural network, by default 42

  • use_gpu (bool, optional) – Use GPU for the neural network, by default True

  • verbose (bool, optional) – Verbose mode for the neural network, by default False

  • mask (Union[None, list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default None

__init__(input_size, output_size, encoder_params, nn_params, random_seed=42, use_gpu=True, verbose=False, mask=None)[source]

Initialize the NeuralNet class

Parameters:

  • input_size (int) – Size of the input layer

  • output_size (int) – Size of the output layer

  • encoder_params (Union[None, dict, tuple[dict, dict, dict]]) – Parameters for the encoder

  • nn_params (dict) – Parameters for the neural network

  • random_seed (int, optional) – Random seed for the neural network, by default 42

  • use_gpu (bool, optional) – Use GPU for the neural network, by default True

  • verbose (bool, optional) – Verbose mode for the neural network, by default False

  • mask (Union[None, list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default None

Return type:

None

get_model()[source]

Get the neural network model.

Return type:

torch.nn.Module

set_random_seed()[source]

Set the random seed for the Autoencoder. It is used to set the random seed for the Autoencoder.

Return type:

None

train_model(X_train, y_train, X_test, y_test, X_validation=None, y_validation=None, criterion=torch.nn.MSELoss)[source]

Train the neural network

Parameters:

  • X_train (Union[list, torch.Tensor, np.ndarray]) – Training data

  • y_train (Union[list, torch.Tensor, np.ndarray]) – Training labels

  • X_test (Union[list, torch.Tensor, np.ndarray]) – Testing data

  • y_test (Union[list, torch.Tensor, np.ndarray]) – Testing labels

  • X_validation (Union[list, torch.Tensor, np.ndarray, None], optional) – Validation data, by default None

  • y_validation (Union[list, torch.Tensor, np.ndarray, None], optional) – Validation labels, by default None

  • criterion (nn.Module, optional) – Loss function, by default nn.MSELoss()

Return type:

None

class OCDocker.OCScore.DNN.DNNOptimizer.DynamicNN(*args, **kwargs)[source]

Bases: Module

Dynamic Neural Network class for the optimization of the neural network.

Parameters:

  • input_size (int) – Size of the input layer

  • output_size (int) – Size of the output layer

  • hidden_layers (list) – List of hidden layers

  • activation_data (list, optional) – List of activation functions, by default []

  • encoder (Union[None, list], optional) – Encoder for the neural network, by default None

  • device (torch.device, optional) – Device for the neural network, by default torch.device(‘cpu’)

  • mask (Union[None, list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default None

__init__(input_size, output_size, hidden_layers, activation_data=[], encoder=None, device=torch.device, mask=None)[source]

Initialize the DynamicNN class

Parameters:

  • input_size (int) – Size of the input layer

  • output_size (int) – Size of the output layer

  • hidden_layers (list) – List of hidden layers

  • activation_data (list, optional) – List of activation functions, by default []

  • encoder (Union[None, list], optional) – Encoder for the neural network, by default None

  • device (torch.device, optional) – Device for the neural network, by default torch.device(‘cpu’)

  • mask (Union[None, list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default None

Return type:

None

forward(x)[source]

Forward pass through the network.

Parameters:

x (torch.Tensor) – Input tensor

Returns:

Output tensor

Return type:

torch.Tensor

class OCDocker.OCScore.DNN.DNNOptimizer.MultiBranchDynamicNN(*args, **kwargs)[source]

Bases: Module

Multi Branch Dynamic Neural Network class for the optimization of the neural network.

Parameters:

  • input_size (Union[int, list[int]]) – Size of the input layer

  • output_size (int) – Size of the output layer

  • hidden_layers (list) – List of hidden layers

  • activation_data (list, optional) – List of activation functions, by default []

  • encoders (Union[None, list], optional) – Encoder for the neural network, by default None

  • device (torch.device, optional) – Device for the neural network, by default torch.device(‘cpu’)

__init__(input_size, output_size, hidden_layers, activation_data=[], encoders=None, device=torch.device)[source]

Initialize the MultiBranchDynamicNN class

Parameters:

  • input_size (Union[int, list[int]]) – Size of the input layer

  • output_size (int) – Size of the output layer

  • hidden_layers (list) – List of hidden layers

  • activation_data (list, optional) – List of activation functions, by default []

  • encoders (Union[None, list], optional) – Encoder for the neural network, by default None

  • device (torch.device, optional) – Device for the neural network, by default torch.device(‘cpu’)

Raises:

ValueError – If the encoder is not a list

Return type:

None

forward(xs)[source]

Forward pass through the network.

Parameters:

xs (list[torch.Tensor]) – Input tensor

Returns:

Output tensor

Return type:

torch.Tensor

Raises:

  • ValueError – If the input is not a list of tensors

  • ValueError – If the input is not a list or if the number of inputs is not equal to the number of encoders

class OCDocker.OCScore.DNN.DNNOptimizer.CustomDataset(*args, **kwargs)[source]

Bases: Dataset

Custom dataset class for the neural network

Parameters:

  • features (torch.Tensor) – Features tensor

  • target (torch.Tensor) – Target tensor

__getitem__(idx)[source]

Get the item at the specified index

Parameters:

idx (int) – Index of the item

Returns:

Tuple of features and target at the specified index

Return type:

tuple[torch.Tensor, torch.Tensor]

__init__(features, target)[source]

Initialize the CustomDataset class

Parameters:

  • features (torch.Tensor) – Features tensor

  • target (torch.Tensor) – Target tensor

Return type:

None

__len__()[source]

Get the length of the dataset

Returns:

Length of the dataset

Return type:

int

class OCDocker.OCScore.DNN.DNNOptimizer.MultiBranchCustomDataset(*args, **kwargs)[source]

Bases: Dataset

Custom dataset class for the multi branch neural network.

Parameters:

  • features1 (torch.Tensor) – Features tensor for the first branch

  • features2 (torch.Tensor) – Features tensor for the second branch

  • features3 (torch.Tensor) – Features tensor for the third branch

  • target (torch.Tensor) – Target tensor

__getitem__(idx)[source]

Get the item at the specified index

Parameters:

idx (int) – Index of the item

Returns:

Tuple of features1, features2, features3 and target at the specified index

Return type:

tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]

__init__(features1, features2, features3, target)[source]

Initialize the MultiBranchCustomDataset class

Parameters:

  • features1 (torch.Tensor) – Features tensor for the first branch

  • features2 (torch.Tensor) – Features tensor for the second branch

  • features3 (torch.Tensor) – Features tensor for the third branch

  • target (torch.Tensor) – Target tensor

Return type:

None

__len__()[source]

Get the length of the dataset

Returns:

Length of the dataset

Return type:

int

class OCDocker.OCScore.DNN.DNNOptimizer.DNNOptimizer(X_train, y_train, X_test, y_test, X_validation=None, y_validation=None, mask=None, storage='sqlite:///NNoptimization.db', encoder_params=None, output_size=1, random_seed=42, use_gpu=True, verbose=False)[source]

Bases: object

Dynamic Neural Network Optimizer class for the optimization of the neural network.

Parameters:

  • X_train (Union[np.ndarray, pd.DataFrame, pd.Series, list[Union[np.ndarray, pd.DataFrame, pd.Series]]]) – Training data

  • y_train (Union[np.ndarray, pd.DataFrame, pd.Series]) – Training labels

  • X_test (Union[np.ndarray, pd.DataFrame, pd.Series, list[Union[np.ndarray, pd.DataFrame, pd.Series]]]) – Testing data

  • y_test (Union[np.ndarray, pd.DataFrame, pd.Series]) – Testing labels

  • X_validation (Union[None, Union[np.ndarray, pd.DataFrame, pd.Series], list[Union[None, np.ndarray, pd.DataFrame, pd.Series]]], optional) – Validation data, by default None

  • y_validation (Union[None, Union[np.ndarray, pd.DataFrame, pd.Series]], optional) – Validation labels, by default None

  • mask (Union[list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default []

  • storage (str, optional) – Storage string for the study, by default “sqlite:///NNoptimization.db”

  • encoder_params (Union[None, dict, tuple[dict, dict, dict]], optional) – Encoder parameters for the neural network, by default None

  • output_size (int, optional) – Size of the output layer, by default 1

  • random_seed (int, optional) – Random seed for the neural network, by default 42

  • use_gpu (bool, optional) – Use GPU for the neural network, by default True

  • verbose (bool, optional) – Verbose mode for the neural network, by default False

__init__(X_train, y_train, X_test, y_test, X_validation=None, y_validation=None, mask=None, storage='sqlite:///NNoptimization.db', encoder_params=None, output_size=1, random_seed=42, use_gpu=True, verbose=False)[source]

Constructor for the DNNOptimizer class

Parameters:

  • X_train (Union[np.ndarray, pd.DataFrame, pd.Series, list[Union[np.ndarray, pd.DataFrame, pd.Series]]]) – Training data

  • y_train (Union[np.ndarray, pd.DataFrame, pd.Series]) – Training labels

  • X_test (Union[np.ndarray, pd.DataFrame, pd.Series, list[Union[np.ndarray, pd.DataFrame, pd.Series]]]) – Testing data

  • y_test (Union[np.ndarray, pd.DataFrame, pd.Series]) – Testing labels

  • X_validation (Union[None, Union[np.ndarray, pd.DataFrame, pd.Series], list[Union[None, np.ndarray, pd.DataFrame, pd.Series]]], optional) – Validation data, by default None

  • y_validation (Union[None, Union[np.ndarray, pd.DataFrame, pd.Series]], optional) – Validation labels, by default None

  • mask (Union[list[Union[int, bool]], np.ndarray], optional) – Mask for the neural network, by default []

  • storage (str, optional) – Storage string for the study, by default “sqlite:///NNoptimization.db”

  • encoder_params (Union[None, dict, tuple[dict, dict, dict]], optional) – Encoder parameters for the neural network, by default None

  • output_size (int, optional) – Size of the output layer, by default 1

  • random_seed (int, optional) – Random seed for the neural network, by default 42

  • use_gpu (bool, optional) – Use GPU for the neural network, by default True

  • verbose (bool, optional) – Verbose mode for the neural network, by default False

Return type:

None

ablate(network_params, n_trials=1, study_name='NN_Ablation_Optimization', load_if_exists=True, n_jobs=1)[source]

Perform an ablation study on the model. Here Optuna will not optimize the model, but will just run the trials with the given parameters and log them in the database.

Parameters:

  • network_params (dict[str, Any]) – Network parameters for the model

  • n_trials (int, optional) – Number of trials to run, by default 1

  • study_name (str, optional) – Name of the study, by default “NN_Ablation_Optimization”

  • load_if_exists (bool, optional) – Load the study if it exists, by default True

  • n_jobs (int, optional) – Number of jobs to run in parallel, by default 1

Return type:

None

objective(trial)[source]

Objective function for the Optuna study

Parameters:

trial (optuna.Trial) – Optuna trial object

Returns:

RMSE of the model

Return type:

float

objective_ablation(trial)[source]

Objective function for the Optuna study for the ablation study

Parameters:

trial (optuna.Trial) – Optuna trial object

Returns:

RMSE of the model

Return type:

float

optimize(direction='maximize', n_trials=10, study_name='NN_Optimization', load_if_exists=True, sampler=optuna.samplers.TPESampler, n_jobs=1)[source]

Optimize the model using Optuna

Parameters:

  • direction (str, optional) – Direction of the optimization, by default “maximize”

  • n_trials (int, optional) – Number of trials to run, by default 10

  • study_name (str, optional) – Name of the study, by default “NN_Optimization”

  • load_if_exists (bool, optional) – Load the study if it exists, by default True

  • sampler (optuna.samplers.BaseSampler, optional) – Sampler for the study, by default TPESampler()

  • n_jobs (int, optional) – Number of jobs to run in parallel, by default 1

Return type:

None

set_random_seed()[source]

Set the random seed for the Autoencoder. It is used to set the random seed for the Autoencoder.

Return type:

None

train_test_model(model, train_loader, test_loader, optimizer, criterion, clip_grad, trial, epochs=100)[source]

Train and test the model

Parameters:

  • model (nn.Module) – Model to train and test

  • train_loader (DataLoader) – DataLoader for the training data

  • test_loader (DataLoader) – DataLoader for the testing data

  • optimizer (optim.Optimizer) – Optimizer for the model

  • criterion (nn.Module) – Loss function for the model

  • clip_grad (float) – Gradient clipping value

  • trial (optuna.Trial) – Optuna trial object

  • epochs (int, optional) – Number of epochs to train the model, by default 100

Returns:

RMSE of the model

Return type:

float