poutyne.framework

Model

class poutyne.framework.Model(model, optimizer, loss_function, *, metrics=[])[source]

The Model class encapsulates a PyTorch module/network, a PyTorch optimizer, a loss function and metric functions. It allows the user to train a neural network without hand-coding the epoch/step logic.

Parameters
  • model (torch.nn.Module) – A PyTorch module.

  • optimizer (torch.optim.Optimizer) – Initialized PyTorch optimizer.

  • loss_function – Loss function. It can be any PyTorch loss layer or custom loss function. It can also be a string with the same name as a PyTorch loss function (either the functional or object name). The loss function must have the signature loss_function(input, target) where input is the prediction of the network and target is the ground truth.

  • metrics (list) – List of functions with the same signature as the loss function. Each metric can be any PyTorch loss function. It can also be a string with the same name as a PyTorch loss function (either the functional or object name). ‘accuracy’ (or just ‘acc’) is also a valid metric. Each metric function is called on each batch of the optimization and on the validation batches at the end of the epoch. (Default value = [])

model

The associated PyTorch module.

Type

torch.nn.Module

optimizer

The associated PyTorch optimizer.

Type

torch.optim.Optimizer

loss_function

The associated loss function.

metrics

The associated metric functions.

Type

list

Example

Using Numpy arrays (or tensors) dataset:

from poutyne.framework import Model
import torch
import numpy as np

num_features = 20
num_classes = 5

# Our training dataset with 800 samples.
num_train_samples = 800
train_x = np.random.randn(num_train_samples, num_features).astype('float32')
train_y = np.random.randint(num_classes, size=num_train_samples).astype('int64')

# Our validation dataset with 200 samples.
num_valid_samples = 200
valid_x = np.random.randn(num_valid_samples, num_features).astype('float32')
valid_y = np.random.randint(num_classes, size=num_valid_samples).astype('int64')

pytorch_module = torch.nn.Linear(num_features, num_classes) # Our network

# We create and optimize our model
model = Model(pytorch_module, 'sgd', 'cross_entropy', metrics=['accuracy'])
model.fit(train_x, train_y,
          validation_x=valid_x,
          validation_y=valid_y,
          epochs=5,
          batch_size=32)
Epoch 1/5 0.02s Step 25/25: loss: 1.719885, acc: 19.375000, val_loss: 1.667446, val_acc: 22.000000
Epoch 2/5 0.02s Step 25/25: loss: 1.705489, acc: 19.750000, val_loss: 1.660806, val_acc: 22.000000
Epoch 3/5 0.01s Step 25/25: loss: 1.692345, acc: 19.625000, val_loss: 1.655008, val_acc: 22.500000
...

Using PyTorch DataLoader:

import torch
from torch.utils.data import DataLoader, TensorDataset
from poutyne.framework import Model

num_features = 20
num_classes = 5

# Our training dataset with 800 samples.
num_train_samples = 800
train_x = torch.rand(num_train_samples, num_features)
train_y = torch.randint(num_classes, (num_train_samples,), dtype=torch.long)
train_dataset = TensorDataset(train_x, train_y)
train_generator = DataLoader(train_dataset, batch_size=32)

# Our validation dataset with 200 samples.
num_valid_samples = 200
valid_x = torch.rand(num_valid_samples, num_features)
valid_y = torch.randint(num_classes, (num_valid_samples,), dtype=torch.long)
valid_dataset = TensorDataset(valid_x, valid_y)
valid_generator = DataLoader(valid_dataset, batch_size=32)

pytorch_module = torch.nn.Linear(num_features, num_train_samples)

model = Model(pytorch_module, 'sgd', 'cross_entropy', metrics=['accuracy'])
model.fit_generator(train_generator,
                    valid_generator,
                    epochs=5)
Epoch 1/5 0.05s Step 25/25: loss: 6.752676, acc: 0.000000, val_loss: 6.575071, val_acc: 0.000000
Epoch 2/5 0.03s Step 25/25: loss: 6.454859, acc: 0.125000, val_loss: 6.279577, val_acc: 0.000000
Epoch 3/5 0.03s Step 25/25: loss: 6.158523, acc: 2.125000, val_loss: 5.985811, val_acc: 9.500000
...
cpu(*args, **kwargs)[source]

Tranfers the network on the CPU. The arguments are passed to the torch.nn.Module.cpu() method. Notice that the device is saved so that the batches can send to the right device before passing it to the network.

Returns

self.

cuda(*args, **kwargs)[source]

Tranfers the network on the GPU. The arguments are passed to the torch.nn.Module.cuda() method. Notice that the device is saved so that the batches can send to the right device before passing it to the network.

Returns

self.

evaluate(x, y, *, batch_size=32, return_pred=False)[source]

Computes the loss and the metrics of the network on batches of samples and optionaly returns the predictions.

Parameters
  • x (Union[Tensor, np.ndarray]) – Dataset.

  • y (Union[Tensor, np.ndarray]) – Dataset ground truths.

  • batch_size (int) – Number of samples given to the network at one time. (Default value = 32)

  • return_pred (bool, optional) – Whether to return the predictions for x. (Default value = False)

Returns

Float loss if no metrics were specified and return_pred is false.

Otherwise, tuple (loss, metrics) if return_pred is false. metrics is a Numpy array of size n, where n is the number of metrics if n > 1. If n == 1, then metrics is a float. If n == 0, the metrics is omitted.

Tuple (loss, metrics, pred_y) if return_pred is true where pred_y is a Numpy array of the predictions.

evaluate_generator(generator, *, steps=None, return_pred=False)[source]

Computes the loss and the metrics of the network on batches of samples and optionaly returns the predictions.

Parameters
  • generator

    Generator-like object for the dataset. The generator must yield a tuple (x, y) where x is a batch of the dataset and y is the corresponding ground truths. y should be a Tensor or a Numpy array with the first dimension being the batch size since len(y) is taken as the batch size. The loss and the metrics are averaged using this batch size. If y is not a Tensor or a Numpy array, then a warning is raised and the “batch size” defaults to 1.

    See the fit_generator() method for details on the types of generators supported.

  • steps (int, optional) – Number of iterations done on generator. (Defaults the number of steps needed to see the entire dataset)

  • return_pred (bool, optional) – Whether to return the predictions for x. (Default value = False)

Returns

Float loss if no metrics were specified and return_pred is false.

Otherwise, tuple (loss, metrics) if return_pred is false. metrics is a Numpy array of size n, where n is the number of metrics if n > 1. If n == 1, then metrics is a float. If n == 0, the metrics is omitted.

Tuple (loss, metrics, pred_y) if return_pred is true where pred_y is the list of the predictions of each batch with tensors converted into Numpy arrays.

Example

With no metrics:

model = Model(pytorch_module, optimizer, loss_function,
              metrics=[])
loss = model.evaluate_generator(test_generator)

With only one metric:

model = Model(pytorch_module, optimizer, loss_function,
              metrics=[my_metric_fn])
loss, my_metric = model.evaluate_generator(test_generator)

With only several metrics:

model = Model(pytorch_module, optimizer, loss_function,
              metrics=[my_metric1_fn, my_metric2_fn])
loss, (my_metric1, my_metric2) = model.evaluate_generator(test_generator)

With metrics and return_pred flag:

model = Model(pytorch_module, optimizer, loss_function,
              metrics=[my_metric1_fn, my_metric2_fn])
loss, (my_metric1, my_metric2), pred_y = model.evaluate_generator(
    test_generator, return_pred=True
)
evaluate_on_batch(x, y, *, return_pred=False)[source]

Computes the loss and the metrics of the network on a single batch of samples and optionaly returns the predictions.

Parameters
  • x (Union[Tensor, np.ndarray]) – Batch.

  • y (Union[Tensor, np.ndarray]) – Batch ground truths.

  • return_pred (bool, optional) – Whether to return the predictions for x. (Default value = False)

Returns

Float loss if no metrics were specified and return_pred is false.

Otherwise, tuple (loss, metrics) if return_pred is false. metrics is a Numpy array of size n, where n is the number of metrics if n > 1. If n == 1, then metrics is a float. If n == 0, the metrics is omitted.

Tuple (loss, metrics, pred_y) if return_pred is true where pred_y is the list of the predictions of each batch with tensors converted into Numpy arrays.

fit(x, y, validation_x=None, validation_y=None, *, batch_size=32, epochs=1000, steps_per_epoch=None, validation_steps=None, initial_epoch=1, verbose=True, callbacks=[])[source]

Trains the model on a dataset. This method creates generators and calls the fit_generator method.

Parameters
  • x (Union[Tensor, np.ndarray]) – Training dataset.

  • y (Union[Tensor, np.ndarray]) – Ground truth.

  • validation_x (Union[Tensor, np.ndarray]) – Validation dataset. The validation datset is optional. (Default value = None)

  • validation_y (Union[Tensor, np.ndarray]) – Validation ground truth. (Default value = None)

  • batch_size (int) – Number of samples given to the network at one time. (Default value = 32)

  • epochs (int) – Number of times the entire training dataset is seen. (Default value = 1000)

  • steps_per_epoch (int, optional) – Number of batch used during one epoch. Obviously, using this argument may cause one epoch not to see the entire training dataset or see it multiple times. (Defaults the number of steps needed to see the entire training dataset)

  • validation_steps (int, optional) – Same as for steps_per_epoch but for the validation dataset. (Defaults to steps_per_epoch if provided or the number of steps needed to see the entire validation dataset)

  • initial_epoch (int, optional) – Epoch at which to start training (useful for resuming a previous training run). (Default value = 1)

  • verbose (bool) – Whether to display the progress of the training. (Default value = True)

  • callbacks (list of poutyne.framework.Callback) – List of callbacks that will be called during training. (Default value = [])

Returns

List of dict containing the history of each epoch.

Example

model = Model(pytorch_module, optimizer, loss_function)
history = model.fit(train_x, train_y,
                    validation_x=valid_x,
                    validation_y=valid_y,
                    epochs=num_epochs,
                    batch_size=batch_size,
                    verbose=False)
print(*history, sep="\n")
{'epoch': 1, 'loss': 1.7198852968215943, 'time': 0.019999928001197986, 'acc': 19.375, 'val_loss': 1.6674459838867188, 'val_acc': 22.0}
{'epoch': 2, 'loss': 1.7054892110824584, 'time': 0.015421080999658443, 'acc': 19.75, 'val_loss': 1.660806336402893, 'val_acc': 22.0}
{'epoch': 3, 'loss': 1.6923445892333984, 'time': 0.01363091799794347, 'acc': 19.625, 'val_loss': 1.6550078630447387, 'val_acc': 22.5}
...
fit_generator(train_generator, valid_generator=None, *, epochs=1000, steps_per_epoch=None, validation_steps=None, initial_epoch=1, verbose=True, callbacks=[])[source]

Trains the model on a dataset using a generator.

Parameters
  • train_generator

    Generator-like object for the training dataset. The generator must yield a tuple (x, y) where x is a batch of the training dataset and y is the corresponding ground truths. y should be a Tensor or a Numpy array with the first dimension being the batch size since len(y) is taken as the batch size. The loss and the metrics are averaged using this batch size. If y is not a Tensor or a Numpy array, then a warning is raised and the “batch size” defaults to 1.

    If the generator does not have a method __len__(), either the steps_per_epoch argument must be provided, or the iterator returned raises a StopIteration exception at the end of the training dataset. PyTorch DataLoaders object do provide a __len__() method.

    Before each epoch, the method __iter__() on the generator is called and the method __next__() is called for each step on resulting object returned by __iter__(). Notice that a call to __iter__() on a generator made using the python keyword yield returns the generator itself.

  • valid_generator (optional) – Generator-like object for the validation dataset. This generator is optional. The generator is used the same way as the generator train_generator. If the generator does not have a method __len__(), either the validation_steps or the steps_per_epoch argument must be provided or the iterator returned raises a StopIteration exception at the end of the validation dataset. (Default value = None)

  • epochs (int) – Number of times the entire training dataset is seen. (Default value = 1000)

  • steps_per_epoch (int, optional) – Number of batch used during one epoch. Obviously, using this argument may cause one epoch not to see the entire training dataset or see it multiple times. (Defaults the number of steps needed to see the entire training dataset)

  • validation_steps (int, optional) – Same as for steps_per_epoch but for the validation dataset. (Defaults to steps_per_epoch if provided or the number of steps needed to see the entire validation dataset)

  • initial_epoch (int, optional) – Epoch at which to start training (useful for resuming a previous training run). (Default value = 1)

  • verbose (bool) – Whether to display the progress of the training. (Default value = True)

  • callbacks (list of poutyne.framework.Callback) – List of callbacks that will be called during training. (Default value = [])

Returns

List of dict containing the history of each epoch.

Example

model = Model(pytorch_module, optimizer, loss_function)
history = model.fit_generator(train_generator,
                              valid_generator,
                              epochs=num_epochs,
                              verbose=False)
print(*history, sep="\n")
{'epoch': 1, 'loss': 1.7198852968215943, 'time': 0.019999928001197986, 'acc': 19.375, 'val_loss': 1.6674459838867188, 'val_acc': 22.0}
{'epoch': 2, 'loss': 1.7054892110824584, 'time': 0.015421080999658443, 'acc': 19.75, 'val_loss': 1.660806336402893, 'val_acc': 22.0}
{'epoch': 3, 'loss': 1.6923445892333984, 'time': 0.01363091799794347, 'acc': 19.625, 'val_loss': 1.6550078630447387, 'val_acc': 22.5}
...
get_weight_copies()[source]

Returns a dictionary containing copies of the parameters of the network.

get_weights()[source]

Returns a dictionary containing the parameters of the network. The tensors are just references to the parameters. To get copies of the weights, see the get_weight_copies() method.

load_optimizer_state(f)[source]

Loads the optimizer state saved using the torch.save() method or the save_optimizer_state() method of this class.

Parameters

f – File-like object (has to implement fileno that returns a file descriptor) or string containing a file name.

load_weights(f)[source]

Loads the weights saved using the torch.save() method or the save_weights() method of this class. Contrary to torch.load(), the weights are not transfered to the device from which they were saved from. In other words, the PyTorch module will stay on the same device it already is on.

Parameters

f – File-like object (has to implement fileno that returns a file descriptor) or string containing a file name.

predict(x, *, batch_size=32)[source]

Returns the predictions of the network given a dataset x, where the tensors are converted into Numpy arrays.

Parameters
  • x (Union[Tensor, np.ndarray]) – Dataset for which to predict.

  • batch_size (int) – Number of samples given to the network at one time. (Default value = 32)

Returns

Numpy arrays of the predictions.

predict_generator(generator, *, steps=None)[source]

Returns the predictions of the network given batches of samples x, where the tensors are converted into Numpy arrays.

generator: Generator-like object for the dataset. The generator must

yield a batch of samples. See the fit_generator() method for details on the types of generators supported.

steps (int, optional): Number of iterations done on

generator. (Defaults the number of steps needed to see the entire dataset)

Returns

List of the predictions of each batch with tensors converted into Numpy arrays.

predict_on_batch(x)[source]

Returns the predictions of the network given a batch x, where the tensors are converted into Numpy arrays.

Parameters

x (Union[Tensor, np.ndarray]) – Batch for which to predict.

Returns

The predictions with tensors converted into Numpy arrays.

save_optimizer_state(f)[source]

Saves the state of the current optimizer.

Parameters

f – File-like object (has to implement fileno that returns a file descriptor) or string containing a file name.

save_weights(f)[source]

Saves the weights of the current network.

Parameters

f – File-like object (has to implement fileno that returns a file descriptor) or string containing a file name.

set_weights(weights)[source]

Modifies the weights of the network with the given weights.

Parameters

weights (dict) – Weights returned by either get_weights() or get_weight_copies().

to(device)[source]

Tranfers the network on the specified device. The device is saved so that the batches can send to the right device before passing it to the network.

Parameters

device (torch.device) – The device to which the network is sent.

Returns

self.

train_on_batch(x, y, *, return_pred=False)[source]

Trains the model for the batch (x, y) and computes the loss and the metrics, and optionaly returns the predictions.

Parameters
  • x – Batch.

  • y – Batch ground truths.

  • return_pred (bool, optional) – Whether to return the predictions for x. (Default value = False)

Returns

Float loss if no metrics were specified and return_pred is false.

Otherwise, tuple (loss, metrics) if return_pred is false. metrics is a Numpy array of size n, where n is the number of metrics if n > 1. If n == 1, then metrics is a float. If n == 0, the metrics is omitted.

Tuple (loss, metrics, pred_y) if return_pred is true where pred_y is the predictions with tensors converted into Numpy arrays.