transformers4rec.torch.block package

Submodules

transformers4rec.torch.block.base module

class transformers4rec.torch.block.base.BlockBase(*args, **kwargs)

Bases: transformers4rec.torch.utils.torch_utils.OutputSizeMixin, torch.nn.modules.module.Module

A subclass of PyTorch’s torch.nn.Module, providing additional functionality for dealing with automatic setting of input/output dimensions of neural networks layers. Specifically, It implements the ‘OutputSizeMixin’ for managing output sizes.

to_model(prediction_task_or_head, inputs=None, **kwargs)

Converts the BlockBase instance into a T4Rec model by attaching it to attaching a ‘Head’ or ‘PredictionTask’.

Parameters

• prediction_task_or_head (Union[PredictionTask, Head]) – A PredictionTask or Head instance to attach to this block.

• inputs (InputBlock, optional) – The input block representing input features. By default None

Raises

ValueError – If prediction_task_or_head is neither a Head nor a PredictionTask.

as_tabular(name=None)

Converts the output of the block into a dictionary, keyed by the provided name

Parameters

name (str, optional) – The output name, if not provided, uses the name of the block class. by default None

class transformers4rec.torch.block.base.Block(module: torch.nn.modules.module.Module, output_size: Union[List[int], torch.Size])

Bases: transformers4rec.torch.block.base.BlockBase

Wraps a PyTorch module, allowing it to be used as a block in a T4Rec model. It carries the module and its expected output size.

Parameters

• module (torch.nn.Module) – The PyTorch module to be wrapped in this block.

• output_size (Union[List[int], torch.Size]) – The expected output size of the module.

forward(inputs, **kwargs)

forward_output_size(input_size)

Calculates the output size of the tensor(s) returned by the forward pass, given the input size.

Parameters

input_size (Union[List[int], torch.Size]) – The size of the input tensor(s) to the module.

Returns

The size of the output from the module.

Return type

Union[List[int], torch.Size]

class transformers4rec.torch.block.base.SequentialBlock(*args, output_size: Optional[Union[List[int], torch.Size]] = None)

Bases: transformers4rec.torch.block.base.BlockBase, torch.nn.modules.container.Sequential

Extends the module torch.nn.Sequential. It’s used for creating a sequence of layers or blocks in a T4Rec model. The modules will be applied to inputs in the order they are passed in the constructor.

Parameters

• *args – The list of PyTorch modules.

• output_size (Union[List[int], torch.Size], optional) – The expected output size from the last layer in the sequential block By default None

property inputs

add_module(name: str, module: Optional[torch.nn.modules.module.Module]) → None

Adds a PyTorch module to the sequential block. If a list of strings is provided, a FilterFeatures block gets added to the sequential block.

Parameters

• name (str) – The name of the child module. The child module can be accessed from this module using the given name.

• module (Optional[Union[List[str], Module]]) – The child module to be added to the module.

add_module_and_maybe_build(name: str, module, parent, idx) → torch.nn.modules.module.Module

Checks if a module needs to be built and adds it to the sequential block.

Parameters

• name (str) – The name of the child module.

• module (torch.nn.Module) – The child module to be added to the sequential block.

• parent (torch.nn.Module) – The parent module.

• idx (int) – The index of the current module in the sequential block.

forward(input, training=False, testing=False, **kwargs)

Applies the module’s layers sequentially to the input block.

Parameters

• input (tensor) – The input to the block.

• training (bool, optional) – Whether the block is in training mode. The default is False.

• testing (bool, optional) – Whether the block is in testing mode. The default is False.

build(input_size, schema=None, **kwargs)

Builds the layers of the sequential block given the input size.

Parameters

• input_size (Union[List[int], torch.Size]) – The size of the input tensor(s).

• schema (Schema, optional) – The schema of the inputs features, by default None

Returns

The built sequential block.

Return type

SequentialBlock

as_tabular(name=None)

Converts the output of the block into a dictionary, keyed by the provided name

Parameters

name (str, optional) – The output name, if not provided, uses the name of the block class. by default None

forward_output_size(input_size)

Calculates the output size of the tensor(s) returned by the forward pass, given the input size.

Parameters

input_size (Union[List[int], torch.Size]) – The size of the input tensor(s) to the module.

Returns

The size of the output from the module.

Return type

Union[List[int], torch.Size]

static get_children_by_class_name(parent, *class_name)

transformers4rec.torch.block.base.build_blocks(*modules)

Builds a SequentialBlock from a list of PyTorch modules.

Parameters

*modules (List[torch.nn.Module]) – List containing PyTorch modules.

Returns

Return type

A SequentialBlock instance created from the provided modules.

class transformers4rec.torch.block.base.BuildableBlock

Bases: abc.ABC

Abstract base class for buildable blocks. Subclasses of BuildableBlock must implement the build method

abstract build(input_size) → transformers4rec.torch.block.base.BlockBase

to_module(shape_or_module)

transformers4rec.torch.block.base.right_shift_block(self, other)

transformers4rec.torch.block.mlp module

class transformers4rec.torch.block.mlp.MLPBlock(dimensions, activation=<class 'torch.nn.modules.activation.ReLU'>, use_bias: bool = True, dropout: Optional[float] = None, normalization: Optional[str] = None, filter_features=None)

Bases: transformers4rec.torch.block.base.BuildableBlock

Defines Multi-Layer Perceptron (MLP) Block by stacking multiple DenseBlock instances.

Parameters

• dimensions (int or list of int) – The dimensions of the layers in the MLP. If an integer is provided, a single layer MLP is created. If a list is provided, it must contain the size of each layer in order.

• activation (optional) – The activation function to apply after each layer. By default torch.nn.ReLU.

• use_bias (bool, optional) – Whether to add a bias term to the dense layers. by default True

• dropout (float, optional) – The dropout rate to apply after each layer, by default None

• normalization (str, optional) – The normalization to apply after each layer, by default None

• filter_features (List[str], optional) – List of features to select from the input., by default None

build(input_shape) → transformers4rec.torch.block.base.SequentialBlock

class transformers4rec.torch.block.mlp.DenseBlock(input_shape: Union[List[int], torch.Size], in_features: int, out_features: int, activation=<class 'torch.nn.modules.activation.ReLU'>, use_bias: bool = True, dropout: Optional[float] = None, normalization=None)

Bases: transformers4rec.torch.block.base.SequentialBlock

A buildable dense Block to represent a fully connected layer.

Parameters

• input_shape (Union[List[int], torch.Size]) – The shape of the input tensor.

• in_features (int) – Number of input features.

• out_features (int) – Number of output features.

• activation (torch.nn.Module, optional) – The activation function to apply after the linear layer. By default torch.nn.ReLU.

• use_bias (bool, optional) – Whether to use bias in the layer. By default True.

• dropout (float, optional) – The dropout rate to apply after the dense layer, if any. By default is None.

• normalization (str, optional) – The type of normalization to apply after the dense layer. Only ‘batch_norm’ is supported. By default is None.

forward_output_size(input_size)

transformers4rec.torch.block.transformer module

class transformers4rec.torch.block.transformer.TransformerPrepare(transformer: Union[transformers.modeling_utils.PreTrainedModel, transformers.configuration_utils.PretrainedConfig], masking: Optional[transformers4rec.torch.masking.MaskSequence] = None)

Bases: torch.nn.modules.module.Module

Base class to prepare additional inputs to the forward call of the HF transformer layer.

Parameters

• transformer (TransformerBody) – The Transformer module.

• masking (Optional[MaskSequence]) – Masking block used to for masking input sequences. By default None.

forward(inputs_embeds) → Dict[str, Any]

training: bool

class transformers4rec.torch.block.transformer.GPT2Prepare(transformer: Union[transformers.modeling_utils.PreTrainedModel, transformers.configuration_utils.PretrainedConfig], masking: Optional[transformers4rec.torch.masking.MaskSequence] = None)

Bases: transformers4rec.torch.block.transformer.TransformerPrepare

TransformerPrepare module for GPT-2.

This class extends the inputs for GPT-2 with a triangular causal mask to the inputs.

forward(inputs_embeds) → Dict[str, Any]

training: bool

class transformers4rec.torch.block.transformer.TransformerBlock(transformer: Union[transformers.modeling_utils.PreTrainedModel, transformers.configuration_utils.PretrainedConfig], masking: Optional[transformers4rec.torch.masking.MaskSequence] = None, prepare_module: Optional[Type[transformers4rec.torch.block.transformer.TransformerPrepare]] = None, output_fn=<function TransformerBlock.<lambda>>)

Bases: transformers4rec.torch.block.base.BlockBase

Class to support HF Transformers for session-based and sequential-based recommendation models.

Parameters

• transformer (TransformerBody) – The T4RecConfig or a pre-trained HF object related to specific transformer architecture.

• masking – Needed when masking is applied on the inputs.

TRANSFORMER_TO_PREPARE: Dict[Type[transformers.modeling_utils.PreTrainedModel], Type[transformers4rec.torch.block.transformer.TransformerPrepare]] = {<class 'transformers.models.gpt2.modeling_gpt2.GPT2Model'>: <class 'transformers4rec.torch.block.transformer.GPT2Prepare'>}

transformer: PreTrainedModel

prepare_module: Optional[TransformerPrepare]

classmethod from_registry(transformer: str, d_model: int, n_head: int, n_layer: int, total_seq_length: int, masking: Optional[transformers4rec.torch.masking.MaskSequence] = None)

Load the HF transformer architecture based on its name

Parameters

• transformer (str) – Name of the Transformer to use. Possible values are : [“reformer”, “gtp2”, “longformer”, “electra”, “albert”, “xlnet”]

• d_model (int) – size of hidden states for Transformers

• n_head – Number of attention heads for Transformers

• n_layer (int) – Number of layers for RNNs and Transformers”

• total_seq_length (int) – The maximum sequence length

training: bool

forward(inputs_embeds, **kwargs)

Transformer Models

forward_output_size(input_size)

Module contents