Model¶

class
poutyne.framework.
Model
(model, optimizer, loss_function, *, metrics=None, batch_metrics=None, epoch_metrics=None)[source]¶ The Model class encapsulates a PyTorch module/network, a PyTorch optimizer, a loss function and metric functions. It allows the user to train a neural network without handcoding the epoch/step logic.
 Parameters
model (torch.nn.Module) – A PyTorch module.
optimizer (Union[torch.optim.Optimizer, str]) – If torch.optim.Optimier, an initialized PyTorch. If str, should be the optimizer’s name in Pytorch (i.e. ‘Adam’ for torch.optim.Adam). (Default value = ‘sgd’)
loss_function (Union[Callable, str]) – can also be a string with the same name as a PyTorch loss function (either the functional or object name). The loss function must have the signature
loss_function(input, target)
whereinput
is the prediction of the network andtarget
is the ground truth. (Default value = None)metrics (list) – *metrics is deprecated as of version 0.5.1. Use batch_metrics instead.* List of functions with the same signature as the loss function. Each metric can be any PyTorch loss function. It can also be a string with the same name as a PyTorch loss function (either the functional or object name). ‘accuracy’ (or just ‘acc’) is also a valid metric. Each metric function is called on each batch of the optimization and on the validation batches at the end of the epoch. (Default value = None)
batch_metrics (list) – List of functions with the same signature as the loss function. Each metric can be any PyTorch loss function. It can also be a string with the same name as a PyTorch loss function (either the functional or object name). ‘accuracy’ (or just ‘acc’) is also a valid metric. Each metric function is called on each batch of the optimization and on the validation batches at the end of the epoch. (Default value = None)
epoch_metrics (list) – List of functions with the same signature as
EpochMetric
(Default value = None)

model
¶ The associated PyTorch module.
 Type

optimizer
¶ The associated PyTorch optimizer.

loss_function
¶ The associated loss function.

metrics
¶ *metrics is deprecated as of version 0.5.1. Use batch_metrics instead.* The associated metric functions.
 Type
Example
Using Numpy arrays (or tensors) dataset:
from poutyne.framework import Model import torch import numpy as np num_features = 20 num_classes = 5 # Our training dataset with 800 samples. num_train_samples = 800 train_x = np.random.randn(num_train_samples, num_features).astype('float32') train_y = np.random.randint(num_classes, size=num_train_samples).astype('int64') # Our validation dataset with 200 samples. num_valid_samples = 200 valid_x = np.random.randn(num_valid_samples, num_features).astype('float32') valid_y = np.random.randint(num_classes, size=num_valid_samples).astype('int64') pytorch_module = torch.nn.Linear(num_features, num_classes) # Our network # We create and optimize our model model = Model(pytorch_module, 'sgd', 'cross_entropy', batch_metrics=['accuracy']) model.fit(train_x, train_y, validation_data=(valid_x, valid_y), epochs=5, batch_size=32)
Epoch 1/5 0.02s Step 25/25: loss: 1.719885, acc: 19.375000, val_loss: 1.667446, val_acc: 22.000000 Epoch 2/5 0.02s Step 25/25: loss: 1.705489, acc: 19.750000, val_loss: 1.660806, val_acc: 22.000000 Epoch 3/5 0.01s Step 25/25: loss: 1.692345, acc: 19.625000, val_loss: 1.655008, val_acc: 22.500000 ...
Using PyTorch DataLoader:
import torch from torch.utils.data import DataLoader, TensorDataset from poutyne.framework import Model num_features = 20 num_classes = 5 # Our training dataset with 800 samples. num_train_samples = 800 train_x = torch.rand(num_train_samples, num_features) train_y = torch.randint(num_classes, (num_train_samples,), dtype=torch.long) train_dataset = TensorDataset(train_x, train_y) train_generator = DataLoader(train_dataset, batch_size=32) # Our validation dataset with 200 samples. num_valid_samples = 200 valid_x = torch.rand(num_valid_samples, num_features) valid_y = torch.randint(num_classes, (num_valid_samples,), dtype=torch.long) valid_dataset = TensorDataset(valid_x, valid_y) valid_generator = DataLoader(valid_dataset, batch_size=32) pytorch_module = torch.nn.Linear(num_features, num_train_samples) model = Model(pytorch_module, 'sgd', 'cross_entropy', batch_metrics=['accuracy']) model.fit_generator(train_generator, valid_generator, epochs=5)
Epoch 1/5 0.05s Step 25/25: loss: 6.752676, acc: 0.000000, val_loss: 6.575071, val_acc: 0.000000 Epoch 2/5 0.03s Step 25/25: loss: 6.454859, acc: 0.125000, val_loss: 6.279577, val_acc: 0.000000 Epoch 3/5 0.03s Step 25/25: loss: 6.158523, acc: 2.125000, val_loss: 5.985811, val_acc: 9.500000 ...

cpu
(*args, **kwargs)[source]¶ Tranfers the network on the CPU. The arguments are passed to the
torch.nn.Module.cpu()
method. Notice that the device is saved so that the batches can send to the right device before passing it to the network.Note
PyTorch optimizers assume that the parameters have been transfered to the right device before their creations. Furthermore, future versions of PyTorch will no longer modify the parameters of a PyTorch module inplace when transferring them to another device. See this issue and this pull request for details.
Since Poutyne supposes that the optimizer has been initialized before the Poutyne Model, necessarily the parameters are not guaranteed to be in sync with those contained in the optimizer once the PyTorch module is transferred to another device. Thus, this method takes care of this inconsistency by updating the parameters inside the optimizer.
 Returns
self.

cuda
(*args, **kwargs)[source]¶ Tranfers the network on the GPU. The arguments are passed to the
torch.nn.Module.cuda()
method. Notice that the device is saved so that the batches can send to the right device before passing it to the network.Note
PyTorch optimizers assume that the parameters have been transfered to the right device before their creations. Furthermore, future versions of PyTorch will no longer modify the parameters of a PyTorch module inplace when transferring them to another device. See this issue and this pull request for details.
Since Poutyne supposes that the optimizer has been initialized before the Poutyne Model, necessarily the parameters are not guaranteed to be in sync with those contained in the optimizer once the PyTorch module is transferred to another device. Thus, this method takes care of this inconsistency by updating the parameters inside the optimizer.
 Returns
self.

evaluate
(x, y, batch_size=32, return_pred=False)[source]¶ Computes the loss and the metrics of the network on batches of samples and optionally returns the predictions.
 Parameters
x (Union[Tensor, ndarray] or Union[tuple, list] of Union[Tensor, ndarray]) – Input to the model. Union[Tensor, ndarray] if the model has a single input. Union[tuple, list] of Union[Tensor, ndarray] if the model has multiple inputs.
y (Union[Tensor, ndarray] or Union[tuple, list] of Union[Tensor, ndarray]) – Target, corresponding ground truth. Union[Tensor, ndarray] if the model has a single output. Union[tuple, list] of Union[Tensor, ndarray] if the model has multiple outputs.
batch_size (int) – Number of samples given to the network at one time. (Default value = 32)
return_pred (bool, optional) – Whether to return the predictions. (Default value = False)
 Returns
Tuple
(loss, metrics, pred_y)
where specific elements are omitted if not applicable. If only loss is applicable, then it is returned as a float.metrics
is a Numpy array of sizen
, wheren
is the number of batch metrics plus the number of epoch metrics ifn > 1
. Ifn == 1
, thenmetrics
is a float. Ifn == 0
, themetrics
is omitted. The first elements ofmetrics
are the batch metrics and are followed by the epoch metrics. See thefit_generator()
method for examples with batch metrics and epoch metrics.If
return_pred
is True,pred_y
is the list of the predictions of each batch with tensors converted into Numpy arrays. It is otherwise ommited.

evaluate_generator
(generator, *, steps=None, return_pred=False, return_ground_truth=False)[source]¶ Computes the loss and the metrics of the network on batches of samples and optionaly returns the predictions.
 Parameters
generator – Generatorlike object for the dataset. See the
fit_generator()
method for details on the types of generators supported.steps (int, optional) – Number of iterations done on
generator
. (Defaults the number of steps needed to see the entire dataset)return_pred (bool, optional) – Whether to return the predictions. (Default value = False)
return_ground_truth (bool, optional) – Whether to return the ground truths. (Default value = False)
 Returns
Tuple
(loss, metrics, pred_y, true_y)
where specific elements are omitted if not applicable. If only loss is applicable, then it is returned as a float.metrics
is a Numpy array of sizen
, wheren
is the number of batch metrics plus the number of epoch metrics ifn > 1
. Ifn == 1
, thenmetrics
is a float. Ifn == 0
, themetrics
is omitted. The first elements ofmetrics
are the batch metrics and are followed by the epoch metrics.If
return_pred
is True,pred_y
is the list of the predictions of each batch with tensors converted into Numpy arrays. It is otherwise ommited.If
return_ground_truth
is True,true_y
is the list of the ground truths of each batch with tensors converted into Numpy arrays. It is otherwise ommited.
Example
With no metrics:
model = Model(pytorch_module, optimizer, loss_function, batch_metrics=None) loss = model.evaluate_generator(test_generator)
With only one batch metric:
model = Model(pytorch_module, optimizer, loss_function, batch_metrics=[my_metric_fn]) loss, my_metric = model.evaluate_generator(test_generator)
With several batch metrics:
model = Model(pytorch_module, optimizer, loss_function, batch_metrics=[my_metric1_fn, my_metric2_fn]) loss, (my_metric1, my_metric2) = model.evaluate_generator(test_generator)
With one batch metric and one epoch metric:
model = Model(pytorch_module, optimizer, loss_function, batch_metrics=[my_metric_fn], epoch_metrics=[MyEpochMetricClass()]) loss, (my_batch_metric, my__epoch_metric) = model.evaluate_generator(test_generator)
With batch metrics and
return_pred
flag:model = Model(pytorch_module, optimizer, loss_function, batch_metrics=[my_metric1_fn, my_metric2_fn]) loss, (my_metric1, my_metric2), pred_y = model.evaluate_generator( test_generator, return_pred=True )
With batch metrics,
return_pred
andreturn_ground_truth
flags:model = Model(pytorch_module, optimizer, loss_function, batch_metrics=[my_metric1_fn, my_metric2_fn]) loss, (my_metric1, my_metric2), pred_y, true_y = model.evaluate_generator( test_generator, return_pred=True, return_ground_truth=True )

evaluate_on_batch
(x, y, *, return_pred=False)[source]¶ Computes the loss and the metrics of the network on a single batch of samples and optionally returns the predictions.
 Parameters
x – Input data as a batch.
y – Target data as a batch.
return_pred (bool, optional) – Whether to return the predictions for
batch
. (Default value = False)
 Returns
Tuple
(loss, metrics, pred_y)
where specific elements are omitted if not applicable. If only loss is applicable, then it is returned as a float.metrics` is a Numpy array of size
n
, wheren
is the number of metrics ifn > 1
. Ifn == 1
, thenmetrics
is a float. Ifn == 0
, themetrics
is omitted.If
return_pred
is True,pred_y
is the list of the predictions of each batch with tensors converted into Numpy arrays. It is otherwise ommited.

fit
(x, y, validation_data=None, *, batch_size=32, epochs=1000, steps_per_epoch=None, validation_steps=None, batches_per_step=1, initial_epoch=1, verbose=True, callbacks=None)[source]¶ Trains the model on a dataset. This method creates generators and calls the
fit_generator()
method. Parameters
x (Union[Tensor, ndarray] or Union[tuple, list] of Union[Tensor, ndarray]) – Training dataset. Union[Tensor, ndarray] if the model has a single input. Union[tuple, list] of Union[Tensor, ndarray] if the model has multiple inputs.
y (Union[Tensor, ndarray] or Union[tuple, list] of Union[Tensor, ndarray]) – Target. Union[Tensor, ndarray] if the model has a single output. Union[tuple, list] of Union[Tensor, ndarray] if the model has multiple outputs.
validation_data (Tuple[
x_val
,y_val
]) – Same format asx
andy
previously described. Validation dataset on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. (Default value = None)batch_size (int) – Number of samples given to the network at one time. (Default value = 32)
epochs (int) – Number of times the entire training dataset is seen. (Default value = 1000)
steps_per_epoch (int, optional) – Number of batch used during one epoch. Obviously, using this argument may cause one epoch not to see the entire training dataset or see it multiple times. (Defaults the number of steps needed to see the entire training dataset)
validation_steps (int, optional) – Same as for
steps_per_epoch
but for the validation dataset. (Defaults tosteps_per_epoch
if provided or the number of steps needed to see the entire validation dataset)batches_per_step (int) – Number of batches on which to compute the running loss before backpropagating it through the network. Note that the total loss used for backpropagation is the mean of the batches_per_step batch losses. (Default value = 1)
initial_epoch (int, optional) – Epoch at which to start training (useful for resuming a previous training run). (Default value = 1)
verbose (bool) – Whether to display the progress of the training. (Default value = True)
callbacks (List[Callback]) – List of callbacks that will be called during training. (Default value = None)
 Returns
List of dict containing the history of each epoch.
Example
model = Model(pytorch_module, optimizer, loss_function) history = model.fit(train_x, train_y, validation_data=(valid_x, valid_y) epochs=num_epochs, batch_size=batch_size, verbose=False) print(*history, sep="\n")
{'epoch': 1, 'loss': 1.7198852968215943, 'time': 0.019999928001197986, 'acc': 19.375, 'val_loss': 1.6674459838867188, 'val_acc': 22.0} {'epoch': 2, 'loss': 1.7054892110824584, 'time': 0.015421080999658443, 'acc': 19.75, 'val_loss': 1.660806336402893, 'val_acc': 22.0} {'epoch': 3, 'loss': 1.6923445892333984, 'time': 0.01363091799794347, 'acc': 19.625, 'val_loss': 1.6550078630447387, 'val_acc': 22.5} ...

fit_generator
(train_generator, valid_generator=None, *, epochs=1000, steps_per_epoch=None, validation_steps=None, batches_per_step=1, initial_epoch=1, verbose=True, callbacks=None)[source]¶ Trains the model on a dataset using a generator.
 Parameters
train_generator –
Generatorlike object for the training dataset. The generator must yield a batch in the form of a tuple (x, y) where
x
is the input andy
is the targetlen(x)
is taken as the batch size (or the first element ofx
in case of multi inputs). The loss and the metrics are averaged using this batch size. If the batch size cannot be, inferred then a warning is raised and the “batch size” defaults to 1.If the generator does not have a method
__len__()
, either thesteps_per_epoch
argument must be provided, or the iterator returned raises a StopIteration exception at the end of the training dataset. PyTorch DataLoaders object do provide a__len__()
method.Before each epoch, the method
__iter__()
on the generator is called and the method__next__()
is called for each step on resulting object returned by__iter__()
. Notice that a call to__iter__()
on a generator made using the python keywordyield
returns the generator itself.valid_generator (optional) – Generatorlike object for the validation dataset. This generator is optional. The generator is used the same way as the generator
train_generator
. If the generator does not have a method__len__()
, either thevalidation_steps
or thesteps_per_epoch
argument must be provided or the iterator returned raises a StopIteration exception at the end of the validation dataset. (Default value = None)epochs (int) – Number of times the entire training dataset is seen. (Default value = 1000)
steps_per_epoch (int, optional) – Number of batch used during one epoch. Obviously, using this argument may cause one epoch not to see the entire training dataset or see it multiple times. (Defaults the number of steps needed to see the entire training dataset)
validation_steps (int, optional) – Same as for
steps_per_epoch
but for the validation dataset. (Defaults tosteps_per_epoch
if provided or the number of steps needed to see the entire validation dataset)batches_per_step (int) – Number of batches on which to compute the running loss before backpropagating it through the network. Note that the total loss used for backpropagation is the mean of the batches_per_step batch losses. (Default value = 1)
initial_epoch (int, optional) – Epoch at which to start training (useful for resuming a previous training run). (Default value = 1)
verbose (bool) – Whether to display the progress of the training. (Default value = True)
callbacks (List[Callback]) – List of callbacks that will be called during training. (Default value = None)
 Returns
List of dict containing the history of each epoch.
Example
model = Model(pytorch_module, optimizer, loss_function) history = model.fit_generator(train_generator, valid_generator, epochs=num_epochs, verbose=False) print(*history, sep="\n")
{'epoch': 1, 'loss': 1.7198852968215943, 'time': 0.019999928001197986, 'acc': 19.375, 'val_loss': 1.6674459838867188, 'val_acc': 22.0} {'epoch': 2, 'loss': 1.7054892110824584, 'time': 0.015421080999658443, 'acc': 19.75, 'val_loss': 1.660806336402893, 'val_acc': 22.0} {'epoch': 3, 'loss': 1.6923445892333984, 'time': 0.01363091799794347, 'acc': 19.625, 'val_loss': 1.6550078630447387, 'val_acc': 22.5} ...

get_weight_copies
()[source]¶ Returns a dictionary containing copies of the parameters of the network.

get_weights
()[source]¶ Returns a dictionary containing the parameters of the network. The tensors are just references to the parameters. To get copies of the weights, see the
get_weight_copies()
method.

load_optimizer_state
(f)[source]¶ Loads the optimizer state saved using the
torch.save()
method or thesave_optimizer_state()
method of this class. Parameters
f – Filelike object (has to implement fileno that returns a file descriptor) or string containing a file name.

load_weights
(f)[source]¶ Loads the weights saved using the
torch.save()
method or thesave_weights()
method of this class. Contrary totorch.load()
, the weights are not transfered to the device from which they were saved from. In other words, the PyTorch module will stay on the same device it already is on. Parameters
f – Filelike object (has to implement fileno that returns a file descriptor) or string containing a file name.

predict
(x, batch_size=32)[source]¶ Returns the predictions of the network given a dataset
x
, where the tensors are converted into Numpy arrays. Parameters
x (Union[Tensor, ndarray] or Union[tuple, list] of Union[Tensor, ndarray]) – Input to the model. Union[Tensor, ndarray] if the model has a single input. Union[tuple, list] of Union[Tensor, ndarray] if the model has multiple inputs.
batch_size (int) – Number of samples given to the network at one time. (Default value = 32)
 Returns
Numpy arrays of the predictions.

predict_generator
(generator, *, steps=None)[source]¶ Returns the predictions of the network given batches of samples
x
, where the tensors are converted into Numpy arrays. generator: Generatorlike object for the dataset. The generator must yield a batch of
samples. See the
fit_generator()
method for details on the types of generators supported. This should only yield input datax
and not the targety
. steps (int, optional): Number of iterations done on
generator
. (Defaults the number of steps needed to see the entire dataset)
 Returns
List of the predictions of each batch with tensors converted into Numpy arrays.

predict_on_batch
(x)[source]¶ Returns the predictions of the network given a batch
x
, where the tensors are converted into Numpy arrays. Parameters
x – Input data as a batch.
 Returns
The predictions with tensors converted into Numpy arrays.

save_optimizer_state
(f)[source]¶ Saves the state of the current optimizer.
 Parameters
f – Filelike object (has to implement fileno that returns a file descriptor) or string containing a file name.

save_weights
(f)[source]¶ Saves the weights of the current network.
 Parameters
f – Filelike object (has to implement fileno that returns a file descriptor) or string containing a file name.

set_weights
(weights)[source]¶ Modifies the weights of the network with the given weights.
 Parameters
weights (dict) – Weights returned by either
get_weights()
orget_weight_copies()
.

to
(device)[source]¶ Tranfers the network on the specified device. The device is saved so that the batches can send to the right device before passing it to the network.
Note
PyTorch optimizers assume that the parameters have been transfered to the right device before their creations. Furthermore, future versions of PyTorch will no longer modify the parameters of a PyTorch module inplace when transferring them to another device. See this issue and this pull request for details.
Since Poutyne supposes that the optimizer has been initialized before the Poutyne Model, necessarily the parameters are not guaranteed to be in sync with those contained in the optimizer once the PyTorch module is transferred to another device. Thus, this method takes care of this inconsistency by updating the parameters inside the optimizer.
 Parameters
device (torch.torch.device) – The device to which the network is sent.
 Returns
self.

train_on_batch
(x, y, return_pred=False)[source]¶ Trains the model for the batch
(x, y)
and computes the loss and the metrics, and optionaly returns the predictions. Parameters
x – Input data as a batch.
y – Target data as a batch.
return_pred (bool, optional) – Whether to return the predictions. (Default value = False)
 Returns
Float
loss
if no metrics were specified andreturn_pred
is false.Otherwise, tuple
(loss, metrics)
ifreturn_pred
is false.metrics
is a Numpy array of sizen
, wheren
is the number of metrics ifn > 1
. Ifn == 1
, thenmetrics
is a float. Ifn == 0
, themetrics
is omitted.Tuple
(loss, metrics, pred_y)
ifreturn_pred
is true wherepred_y
is the predictions with tensors converted into Numpy arrays.