merlin.dag.Graph

class merlin.dag.Graph(output_node: Node)

Bases: object

Represents an DAG composed of Nodes, each of which contains an operator that transforms dataframes or dataframe-like data

__init__(output_node: Node)

Methods

__init__(output_node)
clear_stats()	Removes calculated statistics from each node in the graph
construct_schema(root_schema[, preserve_dtypes])	Given the schema of a dataset to transform, determine the output schema of the graph
get_nodes_by_op_type(nodes, op_type)
remove_inputs(to_remove)	Removes columns from a Graph
subgraph(name)

Attributes

column_mapping
input_dtypes
input_schema
leaf_nodes
output_dtypes
output_schema

subgraph(name: str) → Graph

property input_dtypes

property output_dtypes

property column_mapping

construct_schema(root_schema: Schema, preserve_dtypes=False) → Graph

Given the schema of a dataset to transform, determine the output schema of the graph

Parameters:

• root_schema (Schema) – The schema of a dataset to be transformed with this DAG

• preserve_dtypes (bool, optional) – Whether to keep any dtypes that may already be present in the schemas, by default False

Returns:

This DAG after the schemas have been filled in

Return type:

Graph

property input_schema

property leaf_nodes

property output_schema

remove_inputs(to_remove)

Removes columns from a Graph

Starting at the leaf nodes, trickle down looking for columns to remove, when found remove but then must propagate the removal of any other output columns derived from that column.

Parameters:

• graph (Graph) – The graph to remove columns from

• to_remove (array_like) – A list of input column names to remove from the graph

Returns:

The same graph with columns removed

Return type:

Graph

classmethod get_nodes_by_op_type(nodes, op_type)

clear_stats()

Removes calculated statistics from each node in the graph

See also

StatOperator.clear