eolearn.core.eoworkflow

The eoworkflow module, together with eotask and eodata, provides core building blocks for specifying and executing workflows.

A workflow is a directed (acyclic) graph composed of instances of EONode objects. Each node may take as input the results of tasks in other nodes as well as external arguments. The external arguments are passed anew each time the workflow is executed. The workflow builds the computational graph, performs dependency resolution, and executes the tasks inside each node. If the input graph is cyclic, the workflow raises a CyclicDependencyError.

The result of a workflow execution is an immutable mapping from nodes to results. The result also contain data that was marked as output through the use of OutputTask objects.

The workflow can be exported to a DOT description language and visualized.

class eolearn.core.eoworkflow.EOWorkflow(workflow_nodes)[source]

Bases: object

An object for verifying and executing workflows defined by inter-dependent EONodes.

Example:

node1 = EONode(task1, name='first task')  # custom names can be provided for better logging
node2 = EONode(task2, inputs=[node1])  # depends on previous task
node3 = EONode(task3, inputs=[node1])
node4 = EONode(task4, inputs=[node2, node3])  # depends on two tasks

workflow = EOWorkflow([node1, node2, node3, node4])

# One can pass keyword arguments to task execution in the form of a dictionary
results = workflow.execute(
    {node2: {'k': 2, 'ascending': True}}
)
Parameters:

workflow_nodes (Sequence[EONode]) – A sequence of EONodes, specifying the computational graph.

classmethod from_endnodes(*endnodes)[source]

Constructs the EOWorkflow from the end-nodes by recursively extracting nodes in the workflow structure.

Parameters:

endnodes (EONode) –

Return type:

EOWorkflow

execute(input_kwargs=None, raise_errors=True)[source]

Executes the workflow.

Parameters:

  • input_kwargs (dict[eolearn.core.eonode.EONode, dict[str, object]] | None) – External input arguments to the workflow. They have to be in a form of a dictionary where each key is an EONode used in the workflow and each value is a dictionary or a tuple of arguments.

  • raise_errors (bool) – In case a task in the workflow raises an error this parameter determines how the error will be handled. If True it will propagate the error and if False it will catch the error, write its stack trace in logs and in the WorkflowResults. In either case workflow execute will stop if an error is raised. This rule is not followed only in case of KeyboardInterrupt exception where the exception is always raised.

Returns:

An immutable mapping containing results of terminal tasks

Return type:

WorkflowResults

static validate_input_kwargs(input_kwargs)[source]

Validates EOWorkflow input arguments provided by user and raises an error if something is wrong.

Parameters:

input_kwargs (dict[eolearn.core.eonode.EONode, dict[str, object]]) – A dictionary mapping tasks to task execution arguments

Return type:

None

get_nodes()[source]

Returns an ordered list of all nodes within this workflow, ordered in the execution order.

Returns:

List of all nodes withing workflow. The order of nodes is the same as the order of execution.

Return type:

list[eolearn.core.eonode.EONode]

get_node_with_uid(uid: str, fail_if_missing: Literal[True] = False) → EONode[source]
get_node_with_uid(uid: str, fail_if_missing: Literal[False] = False) → EONode | None

Returns node with give uid, if it exists in the workflow.

get_dot()[source]

Generates the DOT description of the underlying computational graph.

Returns:

The DOT representation of the computational graph

dependency_graph(filename=None)[source]

Visualize the computational graph.

Parameters:

filename (str | None) – Filename of the output image together with file extension. Supported formats: png, jpg, pdf, … . Check graphviz Python package for more options

Returns:

The DOT representation of the computational graph, with some more formatting

class eolearn.core.eoworkflow.WorkflowResults(outputs, start_time, end_time, stats)[source]

Bases: object

An object containing results of an EOWorkflow execution.

Parameters:
outputs: dict[str, object]
start_time: datetime
end_time: datetime
stats: dict[str, eolearn.core.eonode.NodeStats]
error_node_uid: str | None = None
workflow_failed()[source]

Informs if the EOWorkflow execution failed.

Return type:

bool

drop_outputs()[source]

Creates a new WorkflowResults object without outputs which can take a lot of memory.

Return type:

WorkflowResults