# Copyright 2022 NVIDIA Corporation. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================

End-to-end session-based recommendations with PyTorch

In recent years, several deep learning-based algorithms have been proposed for recommendation systems while its adoption in industry deployments have been steeply growing. In particular, NLP inspired approaches have been successfully adapted for sequential and session-based recommendation problems, which are important for many domains like e-commerce, news and streaming media. Session-Based Recommender Systems (SBRS) have been proposed to model the sequence of interactions within the current user session, where a session is a short sequence of user interactions typically bounded by user inactivity. They have recently gained popularity due to their ability to capture short-term or contextual user preferences towards items.

The field of NLP has evolved significantly within the last decade, particularly due to the increased usage of deep learning. As a result, state of the art NLP approaches have inspired RecSys practitioners and researchers to adapt those architectures, especially for sequential and session-based recommendation problems. Here, we leverage one of the state-of-the-art Transformer-based architecture, XLNet with Masked Language Modeling (MLM) training technique (see our tutorial for details) for training a session-based model.

In this end-to-end-session-based recommnender model example, we use Transformers4Rec library, which leverages the popular HuggingFace’s Transformers NLP library and make it possible to experiment with cutting-edge implementation of such architectures for sequential and session-based recommendation problems. For detailed explanations of the building blocks of Transformers4Rec meta-architecture visit getting-started-session-based and tutorial example notebooks.

1. Model definition using Transformers4Rec

In the previous notebook, we have created sequential features and saved our processed data frames as parquet files. Now we use these processed parquet files to train a session-based recommendation model with the XLNet architecture.

1.1 Get the schema

The library uses a schema format to configure the input features and automatically creates the necessary layers. This protobuf text file contains the description of each input feature by defining: the name, the type, the number of elements of a list column, the cardinality of a categorical feature and the min and max values of each feature. In addition, the annotation field contains the tags such as specifying the continuous and categorical features, the target column or the item_id feature, among others.

from merlin_standard_lib import Schema
SCHEMA_PATH = "schema_demo.pb"
schema = Schema().from_proto_text(SCHEMA_PATH)
!cat $SCHEMA_PATH

feature {
  name: "session_id"
  type: INT
  int_domain {
    name: "session_id"
    min: 1
    max: 9249733 
    is_categorical: false
  }
  annotation {
    tag: "groupby_col"
  }
}
feature {
  name: "item_id-list_seq"
  value_count {
    min: 2
    max: 185
  }
  type: INT
  int_domain {
    name: "item_id/list"
    min: 1
    max: 52742
    is_categorical: true
  }
  annotation {
    tag: "item_id"
    tag: "list"
    tag: "categorical"
    tag: "item"
  }
}
feature {
  name: "category-list_seq"
  value_count {
    min: 2
    max: 185
  }
  type: INT
  int_domain {
    name: "category-list_seq"
    min: 1
    max: 337
    is_categorical: true
  }
  annotation {
    tag: "list"
    tag: "categorical"
    tag: "item"
  }
}
feature {
  name: "product_recency_days_log_norm-list_seq"
  value_count {
    min: 2
    max: 185
  }
  type: FLOAT
  float_domain {
    name: "product_recency_days_log_norm-list_seq"
    min: -2.9177291
    max: 1.5231701
  }
  annotation {
    tag: "continuous"
    tag: "list"
  }
}
feature {
  name: "et_dayofweek_sin-list_seq"
  value_count {
    min: 2
    max: 185
  }
  type: FLOAT
  float_domain {
    name: "et_dayofweek_sin-list_seq"
    min: 0.7421683
    max: 0.9995285
  }
  annotation {
    tag: "continuous"
    tag: "time"
    tag: "list"
  }
}

We can select the subset of features we want to use for training the model by their tags or their names.

schema = schema.select_by_name(
   ['item_id-list_seq', 'category-list_seq', 'product_recency_days_log_norm-list_seq', 'et_dayofweek_sin-list_seq']
)

1.2 Define the end-to-end Session-based Transformer-based recommendation model

For defining a session-based recommendation model, the end-to-end model definition requires four steps:

1. Instantiate TabularSequenceFeatures input-module from schema to prepare the embedding tables of categorical variables and project continuous features, if specified. In addition, the module provides different aggregation methods (e.g. ‘concat’, ‘elementwise-sum’) to merge input features and generate the sequence of interactions embeddings. The module also supports language modeling tasks to prepare masked labels for training and evaluation (e.g: ‘mlm’ for masked language modeling)

2. Next, we need to define one or multiple prediction tasks. For this demo, we are going to use NextItemPredictionTask with Masked Language modeling: during training, randomly selected items are masked and predicted using the unmasked sequence items. For inference, it is meant to always predict the next item to be interacted with.

3. Then we construct a transformer_config based on the architectures provided by Hugging Face Transformers framework.

4. Finally we link the transformer-body to the inputs and the prediction tasks to get the final pytorch Model class.

For more details about the features supported by each sub-module, please check out the library documentation page.

from transformers4rec import torch as tr

max_sequence_length, d_model = 20, 320
# Define input module to process tabular input-features and to prepare masked inputs
input_module = tr.TabularSequenceFeatures.from_schema(
    schema,
    max_sequence_length=max_sequence_length,
    continuous_projection=64,
    aggregation="concat",
    d_output=d_model,
    masking="mlm",
)

# Define Next item prediction-task 
prediction_task = tr.NextItemPredictionTask(weight_tying=True)

# Define the config of the XLNet Transformer architecture
transformer_config = tr.XLNetConfig.build(
    d_model=d_model, n_head=8, n_layer=2, total_seq_length=max_sequence_length
)

# Get the end-to-end model 
model = transformer_config.to_torch_model(input_module, prediction_task)

/usr/local/lib/python3.8/dist-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (NDCGAt). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_full_state_property`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (DCGAt). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_full_state_property`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (AvgPrecisionAt). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_full_state_property`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (PrecisionAt). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_full_state_property`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
/usr/local/lib/python3.8/dist-packages/torchmetrics/utilities/prints.py:36: UserWarning: Torchmetrics v0.9 introduced a new argument class property called `full_state_update` that has
                not been set for this class (RecallAt). The property determines if `update` by
                default needs access to the full metric state. If this is not the case, significant speedups can be
                achieved and we recommend setting this to `False`.
                We provide an checking function
                `from torchmetrics.utilities import check_forward_full_state_property`
                that can be used to check if the `full_state_update=True` (old and potential slower behaviour,
                default for now) or if `full_state_update=False` can be used safely.
                
  warnings.warn(*args, **kwargs)
Projecting inputs of NextItemPredictionTask to'64' As weight tying requires the input dimension '320' to be equal to the item-id embedding dimension '64'

You can print out the model structure by uncommenting the line below.

#model

1.3. Daily Fine-Tuning: Training over a time window¶

Now that the model is defined, we are going to launch training. For that, Transfromers4rec extends HF Transformers Trainer class to adapt the evaluation loop for session-based recommendation task and the calculation of ranking metrics. The original train() method is not modified meaning that we leverage the efficient training implementation from that library, which manages, for example, half-precision (FP16) training.

Set the training arguments

An additional argument data_loader_engine is defined to automatically load the features needed for training using the schema. The default value is nvtabular for optimized GPU-based data-loading. Optionally a PyarrowDataLoader (pyarrow) can also be used as a basic option, but it is slower and works only for small datasets, as the full data is loaded to CPU memory.

training_args = tr.trainer.T4RecTrainingArguments(
            output_dir="./tmp",
            max_sequence_length=20,
            data_loader_engine='nvtabular',
            num_train_epochs=10, 
            dataloader_drop_last=False,
            per_device_train_batch_size = 384,
            per_device_eval_batch_size = 512,
            learning_rate=0.0005,
            fp16=True,
            report_to = [],
            logging_steps=200
        )

Instantiate the trainer

recsys_trainer = tr.Trainer(
    model=model,
    args=training_args,
    schema=schema,
    compute_metrics=True)

Using amp fp16 backend

Launch daily training and evaluation

In this demo, we will use the fit_and_evaluate method that allows us to conduct a time-based finetuning by iteratively training and evaluating using a sliding time window: At each iteration, we use the training data of a specific time index \(t\) to train the model; then we evaluate on the validation data of the next index \(t + 1\). Particularly, we set start time to 178 and end time to 180.

from transformers4rec.torch.utils.examples_utils import fit_and_evaluate
OT_results = fit_and_evaluate(recsys_trainer, start_time_index=178, end_time_index=180, input_dir='./preproc_sessions_by_day')

***** Launch training for day 178: *****

***** Running training *****
  Num examples = 28800
  Num Epochs = 10
  Instantaneous batch size per device = 384
  Total train batch size (w. parallel, distributed & accumulation) = 768
  Gradient Accumulation steps = 1
  Total optimization steps = 750
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/scatter_gather.py:9: UserWarning: is_namedtuple is deprecated, please use the python checks instead
  warnings.warn("is_namedtuple is deprecated, please use the python checks instead")
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '

[750/750 00:49, Epoch 10/10]

Step	Training Loss
200	7.700900
400	6.610000
600	6.315500

Saving model checkpoint to ./tmp/checkpoint-500
Trainer.model is not a `PreTrainedModel`, only saving its state dict.
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/scatter_gather.py:9: UserWarning: is_namedtuple is deprecated, please use the python checks instead
  warnings.warn("is_namedtuple is deprecated, please use the python checks instead")
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '


Training completed. Do not forget to share your model on huggingface.co/models =)

[6/6 01:08]

***** Running training *****
  Num examples = 20736
  Num Epochs = 10
  Instantaneous batch size per device = 384
  Total train batch size (w. parallel, distributed & accumulation) = 768
  Gradient Accumulation steps = 1
  Total optimization steps = 540

***** Evaluation results for day 179:*****

 eval_/next-item/avg_precision@10 = 0.07868842035531998
 eval_/next-item/avg_precision@20 = 0.08343780785799026
 eval_/next-item/ndcg@10 = 0.10765216499567032
 eval_/next-item/ndcg@20 = 0.12527914345264435
 eval_/next-item/recall@10 = 0.19691714644432068
 eval_/next-item/recall@20 = 0.2674373686313629

***** Launch training for day 179: *****

[540/540 00:36, Epoch 10/10]

Step	Training Loss
200	6.796700
400	6.363300

Saving model checkpoint to ./tmp/checkpoint-500
Trainer.model is not a `PreTrainedModel`, only saving its state dict.
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/scatter_gather.py:9: UserWarning: is_namedtuple is deprecated, please use the python checks instead
  warnings.warn("is_namedtuple is deprecated, please use the python checks instead")
/usr/local/lib/python3.8/dist-packages/torch/nn/parallel/_functions.py:68: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '


Training completed. Do not forget to share your model on huggingface.co/models =)


***** Running training *****
  Num examples = 16896
  Num Epochs = 10
  Instantaneous batch size per device = 384
  Total train batch size (w. parallel, distributed & accumulation) = 768
  Gradient Accumulation steps = 1
  Total optimization steps = 440

***** Evaluation results for day 180:*****

 eval_/next-item/avg_precision@10 = 0.06882524490356445
 eval_/next-item/avg_precision@20 = 0.0728154256939888
 eval_/next-item/ndcg@10 = 0.09575356543064117
 eval_/next-item/ndcg@20 = 0.11073826253414154
 eval_/next-item/recall@10 = 0.17995338141918182
 eval_/next-item/recall@20 = 0.23916083574295044

***** Launch training for day 180: *****

[440/440 00:30, Epoch 10/10]

Step	Training Loss
200	6.877900
400	6.451500

Training completed. Do not forget to share your model on huggingface.co/models =)

***** Evaluation results for day 181:*****

 eval_/next-item/avg_precision@10 = 0.13450393080711365
 eval_/next-item/avg_precision@20 = 0.14182709157466888
 eval_/next-item/ndcg@10 = 0.1758614182472229
 eval_/next-item/ndcg@20 = 0.2022867053747177
 eval_/next-item/recall@10 = 0.30983301997184753
 eval_/next-item/recall@20 = 0.41372913122177124

Visualize the average of metrics over time

OT_results is a list of scores (accuracy metrics) for evaluation based on given start and end time_index. Since in this example we do evaluation on days 179, 180 and 181, we get three metrics in the list one for each day.

OT_results

{'indexed_by_time_eval_/next-item/avg_precision@10': [0.07868842035531998,
  0.06882524490356445,
  0.13450393080711365],
 'indexed_by_time_eval_/next-item/avg_precision@20': [0.08343780785799026,
  0.0728154256939888,
  0.14182709157466888],
 'indexed_by_time_eval_/next-item/ndcg@10': [0.10765216499567032,
  0.09575356543064117,
  0.1758614182472229],
 'indexed_by_time_eval_/next-item/ndcg@20': [0.12527914345264435,
  0.11073826253414154,
  0.2022867053747177],
 'indexed_by_time_eval_/next-item/recall@10': [0.19691714644432068,
  0.17995338141918182,
  0.30983301997184753],
 'indexed_by_time_eval_/next-item/recall@20': [0.2674373686313629,
  0.23916083574295044,
  0.41372913122177124]}

import numpy as np
# take the average of metric values over time
avg_results = {k: np.mean(v) for k,v in OT_results.items()}
for key in sorted(avg_results.keys()): 
    print(" %s = %s" % (key, str(avg_results[key]))) 

 indexed_by_time_eval_/next-item/avg_precision@10 = 0.09400586535533269
 indexed_by_time_eval_/next-item/avg_precision@20 = 0.09936010837554932
 indexed_by_time_eval_/next-item/ndcg@10 = 0.12642238289117813
 indexed_by_time_eval_/next-item/ndcg@20 = 0.14610137045383453
 indexed_by_time_eval_/next-item/recall@10 = 0.22890118261178335
 indexed_by_time_eval_/next-item/recall@20 = 0.3067757785320282

Save the model

recsys_trainer._save_model_and_checkpoint(save_model_class=True)

Saving model checkpoint to ./tmp/checkpoint-440
Trainer.model is not a `PreTrainedModel`, only saving its state dict.

Export the preprocessing workflow and model in the format required by Triton server:

NVTabular’s export_pytorch_ensemble() function enables us to create model files and config files to be served to Triton Inference Server.

from nvtabular.inference.triton import export_pytorch_ensemble
from nvtabular.workflow import Workflow
workflow = Workflow.load("workflow_etl")

export_pytorch_ensemble(
    model,
    workflow,
    sparse_max=recsys_trainer.get_train_dataloader().dataset.sparse_max,
    name= "t4r_pytorch",
    model_path= "/workspace/TF4Rec/models/",
    label_columns =[],
)

2. Serving Ensemble Model to the Triton Inference Server

NVIDIA Triton Inference Server (TIS) simplifies the deployment of AI models at scale in production. TIS provides a cloud and edge inferencing solution optimized for both CPUs and GPUs. It supports a number of different machine learning frameworks such as TensorFlow and PyTorch.

The last step of a machine learning (ML)/deep learning (DL) pipeline is to deploy the ETL workflow and saved model to production. In the production setting, we want to transform the input data as done during training (ETL). We need to apply the same mean/std for continuous features and use the same categorical mapping to convert the categories to continuous integer before we use the DL model for a prediction. Therefore, we deploy the NVTabular workflow with the PyTorch model as an ensemble model to Triton Inference. The ensemble model guarantees that the same transformation is applied to the raw inputs.

In this section, you will learn how to

• to deploy saved NVTabular and PyTorch models to Triton Inference Server

• send requests for predictions and get responses.

2.1. Pull and Start Inference Container

At this point, we start the Triton Inference Server (TIS).

Start triton server
You can start triton server with the command below. You need to provide correct path of the models folder.

tritonserver --model-repository=<path_to_models> --model-control-mode=explicit

Note: The model-repository path for our example is /workspace/TF4Rec/models/. The models have not been loaded yet. Below, we will request the Triton server to load the saved ensemble model.

2.2. Connect to the Triton Inference Server and check if the server is alive

import tritonhttpclient
try:
    triton_client = tritonhttpclient.InferenceServerClient(url="localhost:8000", verbose=True)
    print("client created.")
except Exception as e:
    print("channel creation failed: " + str(e))
triton_client.is_server_live()

client created.
GET /v2/health/live, headers None
<HTTPSocketPoolResponse status=200 headers={'content-length': '0', 'content-type': 'text/plain'}>

/usr/local/lib/python3.8/dist-packages/tritonhttpclient/__init__.py:31: DeprecationWarning: The package `tritonhttpclient` is deprecated and will be removed in a future version. Please use instead `tritonclient.http`
  warnings.warn(

True

2.3. Load raw data for inference

We select the last 50 interactions and filter out sessions with less than 2 interactions.

import pandas as pd
interactions_merged_df = pd.read_parquet("/workspace/data/interactions_merged_df.parquet")
interactions_merged_df = interactions_merged_df.sort_values('timestamp')
batch = interactions_merged_df[-50:]
sessions_to_use = batch.session_id.value_counts()
filtered_batch = batch[batch.session_id.isin(sessions_to_use[sessions_to_use.values>1].index.values)]

2.4. Load the ensemble model to triton

The models should be loaded successfully before we send a request to TIS. If all models are loaded successfully, you should be seeing successfully loaded status next to each model name on your terminal.

triton_client.load_model(model_name="t4r_pytorch")

POST /v2/repository/models/t4r_pytorch/load, headers None
{}
<HTTPSocketPoolResponse status=200 headers={'content-type': 'application/json', 'content-length': '0'}>
Loaded model 't4r_pytorch'

2.5. Send the request to triton server

triton_client.get_model_repository_index()

POST /v2/repository/index, headers None

<HTTPSocketPoolResponse status=200 headers={'content-type': 'application/json', 'content-length': '167'}>
bytearray(b'[{"name":"t4r_pytorch","version":"1","state":"READY"},{"name":"t4r_pytorch_nvt","version":"1","state":"READY"},{"name":"t4r_pytorch_pt","version":"1","state":"READY"}]')

[{'name': 't4r_pytorch', 'version': '1', 'state': 'READY'},
 {'name': 't4r_pytorch_nvt', 'version': '1', 'state': 'READY'},
 {'name': 't4r_pytorch_pt', 'version': '1', 'state': 'READY'}]

If all models are loaded successfully, you should be seeing READY status next to each model.

import nvtabular.inference.triton as nvt_triton
import tritonclient.grpc as grpcclient

inputs = nvt_triton.convert_df_to_triton_input(filtered_batch.columns, filtered_batch, grpcclient.InferInput)

output_names = ["output"]

outputs = []
for col in output_names:
    outputs.append(grpcclient.InferRequestedOutput(col))
    
MODEL_NAME_NVT = "t4r_pytorch"

with grpcclient.InferenceServerClient("localhost:8001") as client:
    response = client.infer(MODEL_NAME_NVT, inputs)
    print(col, ':\n', response.as_numpy(col))

output :
 [[-16.523573 -17.577486 -10.921707 ... -17.596546 -17.69919  -15.932049]
 [-16.530544 -14.984111  -8.974278 ... -20.520447 -21.623093 -16.919617]
 [-17.881346 -17.46621   -9.281072 ... -20.108696 -22.266499 -18.415016]
 ...
 [-18.485788 -18.127209  -8.013737 ... -22.590933 -25.988901 -20.668829]
 [-18.583862 -16.867874  -9.108355 ... -23.238972 -24.597252 -19.22632 ]
 [-18.973335 -18.069975  -9.379853 ... -23.59224  -25.913097 -20.5112  ]]

• Visualise top-k predictions

from transformers4rec.torch.utils.examples_utils import visualize_response
visualize_response(filtered_batch, response, top_k=5, session_col='session_id')

- Top-5 predictions for session `11457123`: 281 || 1781 || 953 || 284 || 27

- Top-5 predictions for session `11467406`: 6510 || 4541 || 1219 || 4136 || 4204

- Top-5 predictions for session `11528554`: 898 || 1037 || 3299 || 202 || 863

- Top-5 predictions for session `11336059`: 8553 || 4204 || 4541 || 1219 || 4136

- Top-5 predictions for session `11445777`: 1453 || 633 || 4204 || 620 || 1219

- Top-5 predictions for session `11493827`: 6510 || 7055 || 4136 || 4541 || 4204

- Top-5 predictions for session `11425751`: 620 || 1453 || 2980 || 1219 || 2031

- Top-5 predictions for session `11399751`: 1453 || 4204 || 633 || 620 || 1219

- Top-5 predictions for session `11311424`: 8553 || 4136 || 4204 || 4541 || 6488

- Top-5 predictions for session `11257991`: 1219 || 10893 || 1157 || 13054 || 651

- Top-5 predictions for session `11561822`: 9459 || 4204 || 4541 || 6488 || 4136

- Top-5 predictions for session `11421333`: 1697 || 186 || 898 || 2290 || 202

- Top-5 predictions for session `11270119`: 4136 || 6407 || 4541 || 2789 || 4204

- Top-5 predictions for session `11401481`: 7055 || 620 || 4204 || 4541 || 4136

- Top-5 predictions for session `11394056`: 6510 || 7055 || 4136 || 4204 || 4541

As you noticed, we first got prediction results (logits) from the trained model head, and then by using a handy util function visualize_response we extracted top-k encoded item-ids from logits. Basically, we generated recommended items for a given session.

This is the end of the tutorial. You successfully

• performed feature engineering with NVTabular

• trained transformer architecture based session-based recommendation models with Transformers4Rec

• deployed a trained model to Triton Inference Server, sent request and got responses from the server.

Unload models

triton_client.unload_model(model_name="t4r_pytorch")
triton_client.unload_model(model_name="t4r_pytorch_nvt")
triton_client.unload_model(model_name="t4r_pytorch_pt")

References

• Merlin Transformers4rec: https://github.com/NVIDIA-Merlin/Transformers4Rec

• Merlin NVTabular: https://github.com/NVIDIA-Merlin/NVTabular/tree/main/nvtabular

• Triton inference server: https://github.com/triton-inference-server