Utils

Basic Torch Utils

These utils functions only support the following basic Python types: tuple, list and dict.

poutyne.torch_to_numpy(obj, copy=False)[source]

Convert to Numpy arrays all tensors inside a Python object composed of the supported types.

Parameters:

  • obj – The Python object to convert.

  • copy (bool) – Whether to copy the memory. By default, if a tensor is already on CPU, the Numpy array will be a view of the tensor.

Returns:

A new Python object with the same structure as obj but where the tensors are now Numpy arrays. Not supported type are left as reference in the new object.

Example

>>> from poutyne import torch_to_numpy
>>> torch_to_numpy({
...     'first': torch.tensor([1, 2, 3]),
...     'second':[torch.tensor([4,5,6]), torch.tensor([7,8,9])],
...     'third': 34
... })
{
    'first': array([1, 2, 3]),
    'second': [array([4, 5, 6]), array([7, 8, 9])],
    'third': 34
}
See:

torch_apply() for supported types.

poutyne.numpy_to_torch(obj)[source]

Convert to tensors all Numpy arrays inside a Python object composed of the supported types.

Parameters:

obj – The Python object to convert.

Returns:

A new Python object with the same structure as obj but where the Numpy arrays are now tensors. Not supported type are left as reference in the new object.

Example

>>> from poutyne import numpy_to_torch
>>> numpy_to_torch({
...     'first': np.array([1, 2, 3]),
...     'second':[np.array([4,5,6]), np.array([7,8,9])],
...     'third': 34
... })
{
    'first': tensor([1, 2, 3]),
    'second': [tensor([4, 5, 6]), tensor([7, 8, 9])],
    'third': 34
}
poutyne.torch_apply(obj, func)[source]

Apply a function to all tensors inside a Python object composed of the supported types.

Supported types are: list, tuple and dict.

Parameters:

  • obj – The Python object to convert.

  • func – The function to apply.

Returns:

A new Python object with the same structure as obj but where the tensors have been applied the function func. Not supported type are left as reference in the new object.

poutyne.set_seeds(seed)[source]

Set Python, Numpy and Pytorch’s random seeds in order to make the random number generation procedure deterministic and reproducible.

Parameters:

seed (int) – The random number generation seed to use.

poutyne.save_random_states(f: str | PathLike | BinaryIO | IO[bytes])[source]

Save Python, Numpy and Pytorch’s (both CPU and GPU) random states.

Parameters:

f (Union[str, os.PathLike, BinaryIO, IO[bytes]]) – a file-like object (has to implement write and flush) or a string or os.PathLike object containing a file name.

poutyne.load_random_states(f: Any)[source]

Load Python, Numpy and Pytorch’s (both CPU and GPU) random states as saved by save_random_states().

Parameters:

f – a file-like object (has to implement read(), readline(), tell(), and seek()), or a string or os.PathLike object containing a file name

Plotting

poutyne.plot_history(history: List[Dict[str, float | int]] | pd.DataFrame, *, metrics: List[str] | None = None, labels: List[str] | None = None, titles: str | List[str] | None = None, axes: List[matplotlib.axes.Axes] | None = None, show: bool = True, save: bool = False, save_filename_template: str = '{metric}', save_directory: str | None = None, save_extensions: List[str] | Tuple[str] = ('png',), close: bool | None = None, fig_kwargs: Dict[str, Any] | None = None)[source]

Plot the training history in matplotlib. By default, all metrics are plotted.

Parameters:

  • history (Union[List[Dict[str, Union[float, int]]], pandas.DataFrame]) – The training history to plot. Can be either a list of dictionary as returned by fit() or a Pandas DataFrame as read from a CSV output by the CSVLogger callback.

  • metrics (Optional[List[str]], optional) – The list of metrics for which to output the plot. By default, every metric in the history is used.

  • labels (Optional[List[str]], optional) – A list of labels to use for each metric. Must be of the same length as metrics. By default, the names in the history are used.

  • titles (Optional[Union[List[str], str]], optional) – A title or a list of titles to use for each metric. If a list, must be of the same length as metrics. If a string, the same title will be used for all plots. By default, there is no title.

  • axes (Optional[List[matplotlib.axes.Axes]], optional) – A list of matplotlib Axes to use for each metric. Must be of the same length as metrics. By default, a new figure and an new axe is created for each plot.

  • show (bool, optional) – Whether to show the plots. Defaults to True.

  • save (bool, optional) – Whether to save the plots. Defaults to False.

  • save_filename_template (str, optional) – The filename without extension for saving the plot. Should contain {metric} somewhere in it or all the plots will overwrite each other. Defaults to '{metric}'.

  • save_directory (Optional[str], optional) – The directory to save the plots. Default to the current directory.

  • save_extensions (Union[List[str], Tuple[str]], optional) – A list of extensions under which to save the plots. Defaults to (‘png’, ).

  • close (Optional[bool], optional) – Whether to close the matplotlib figures. By default, the figures are closed except when in Jupyter notebooks.

  • fig_kwargs (Optional[Dict[str, Any]], optional) – Any keyword arguments to pass to subplots().

Returns:

A tuple (figs, axes) where figs is the list of instanciated matplotlib Figure and axes is a list of instanciated matplotlib Axes.

Return type:

Tuple[List[matplotlib.figure.Figure], List[matplotlib.axes.Axes]]

poutyne.plot_metric(history: List[Dict[str, float | int]] | pd.DataFrame, metric: str, *, label: str | None = None, title: str = '', ax: matplotlib.axes.Axes | None = None)[source]

Plot the training history in matplotlib for a given metric.

Parameters:

  • history (Union[List[Dict[str, Union[float, int]]], pd.DataFrame]) – The training history to plot. Can be either a list of dictionary as returned by fit() or a Pandas DataFrame as read from a CSV output by the CSVLogger callback.

  • metric (str) – The metric for which to output the plot.

  • label (str, Optional[str]) – A label for the metric. By default, the label is the same as the name of the metric.

  • title (str, optional) – A title for the plot. By default, no title.

  • ax (Optional[matplotlib.axes.Axes], optional) – A matplotlib Axes to use. By default, the current axe is used.

Miscellaneous

poutyne.get_batch_size(*values)[source]

This method infers the batch size of a batch. Here is the inferring algorithm used to compute the batch size. The values are tested in order at each step of the inferring algorithm. If one step succeed for one of the values, the algorithm stops.

  • Step 1: if a value is a tensor or a Numpy array, then the len() is returned.

  • Step 2: if a value is a list or a tuple, then the len() of the first element is returned if it is a tensor or a Numpy array.

  • Step 3: if a value is a dict, then the value for the key 'batch_size' is returned if it is of integral type.

  • Step 4: if a value is a dict, then the len() of the first element of .values() is returned if it is a tensor or a Numpy array.

If inferring the batch size is not possible, the batch size is set to 1 and a warning is raised. To disable this warning, set

from poutyne import warning_settings

warning_settings['batch_size'] = 'ignore'
Parameters:

values – The values used for inferring the batch size.