nvtabular.ops.HashBucket

class nvtabular.ops.HashBucket(num_buckets: Union[int, Dict[str, int]])

Bases: Operator

This op maps categorical columns to a contiguous integer range by first hashing the column, then reducing modulo the number of buckets.

Example usage:

cat_names = ["feature_a", "feature_b"]

# this will hash both features a and b to 100 buckets
hash_features = cat_names >> ops.HashBucket({"feature_a": 100, "feature_b": 50})
processor = nvtabular.Workflow(hash_features)

The output of this op would be:

   feature_a  feature_b
0         90         11
1         70         40
2         52          9

If you would like to do frequency capping or frequency hashing, you should use Categorify op instead. See Categorify op for example usage.

Parameters:

num_buckets (int or dictionary:{column: num_hash_buckets}) – Column-wise modulo to apply after hash function. Note that this means that the corresponding value will be the categorical cardinality of the transformed categorical feature. If given as an int, that value will be used as the number of “hash buckets” for every feature. If a dictionary is passed, it will be used to specify explicit mappings from a column name to a number of buckets. In this case, only the columns specified in the keys of num_buckets will be transformed.

__init__(num_buckets: Union[int, Dict[str, int]])

Methods

__init__(num_buckets)
column_mapping(col_selector)	Compute which output columns depend on which input columns
compute_column_schema(col_name, input_schema)
compute_input_schema(root_schema, ...)	Given the schemas coming from upstream sources and a column selector for the input columns, returns a set of schemas for the input columns this operator will use
compute_output_schema(input_schema, col_selector)	Given a set of schemas and a column selector for the input columns, returns a set of schemas for the transformed columns this operator will produce
compute_selector(input_schema, selector[, ...])	Provides a hook method for sub-classes to override to implement custom column selection logic.
create_node(selector)
export(path, input_schema, output_schema, ...)	Export the class object as a config and all related files to the user defined path.
get_embedding_sizes(columns)
inference_initialize(col_selector, model_config)	Configures this operator for use in inference.
load_artifacts([artifact_path])	Load artifacts from disk required for operator function.
output_column_names(col_selector)	Given a set of columns names returns the names of the transformed columns this operator will produce
save_artifacts([artifact_path])	Save artifacts required to be reload operator state from disk
transform(col_selector, df)	Transform the dataframe by applying this operator to the set of input columns
validate_schemas(parents_schema, ...[, ...])	Provides a hook method that sub-classes can override to implement schema validation logic.

Attributes

dependencies	Defines an optional list of column dependencies for this operator.
dynamic_dtypes
export_name	Provides a clear common english identifier for this operator.
is_subgraph
label
output_dtype
output_properties
output_tags
supported_formats
supports	Returns what kind of data representation this operator supports

transform(col_selector: ColumnSelector, df: DataFrame) → DataFrame

Transform the dataframe by applying this operator to the set of input columns

Parameters:

• columns (list of str or list of list of str) – The columns to apply this operator to

• df (Dataframe) – A pandas or cudf dataframe that this operator will work on

Returns:

Returns a transformed dataframe for this operator

Return type:

DataFrame

get_embedding_sizes(columns)

property output_tags

property output_dtype