futures
prefect.futures
Functions
as_completed
wait
Wait for the futures in the given sequence to complete.
Args:
futures
: The sequence of Futures to wait upon.timeout
: The maximum number of seconds to wait. If None, then there is no limit on the wait time.
Returns:
- A named 2-tuple of sets. The first set, named ‘done’, contains the
- futures that completed (is finished or cancelled) before the wait
- completed. The second set, named ‘not_done’, contains uncompleted
- futures. Duplicate futures given to futures are removed and will be
- returned only once.
Examples:
resolve_futures_to_states
Given a Python built-in collection, recursively find PrefectFutures
and build a
new collection with the same structure with futures resolved to their final states.
Resolving futures to their final states may wait for execution to complete.
Unsupported object types will be returned without modification.
resolve_futures_to_results
Given a Python built-in collection, recursively find PrefectFutures
and build a
new collection with the same structure with futures resolved to their final results.
Resolving futures to their final result may wait for execution to complete.
Unsupported object types will be returned without modification.
Classes
PrefectFuture
Abstract base class for Prefect futures. A Prefect future is a handle to the asynchronous execution of a run. It provides methods to wait for the to complete and to retrieve the result of the run.
Methods:
task_run_id
The ID of the task run associated with this future
state
The current state of the task run associated with this future
wait
result
add_done_callback
Add a callback to be run when the future completes or is cancelled.
Args:
fn
: A callable that will be called with this future as its only argument when the future completes or is cancelled.
PrefectTaskRunFuture
A Prefect future that represents the eventual execution of a task run.
Methods:
task_run_id
The ID of the task run associated with this future
state
The current state of the task run associated with this future
PrefectWrappedFuture
A Prefect future that wraps another future object.
Methods:
wrapped_future
The underlying future object wrapped by this Prefect future
add_done_callback
Add a callback to be executed when the future completes.
PrefectConcurrentFuture
A Prefect future that wraps a concurrent.futures.Future. This future is used when the task run is submitted to a ThreadPoolExecutor.
Methods:
wait
result
PrefectDistributedFuture
Represents the result of a computation happening anywhere.
This class is typically used to interact with the result of a task run scheduled to run in a Prefect task worker but can be used to interact with any task run scheduled in Prefect’s API.
Methods:
wait
result
add_done_callback
PrefectFlowRunFuture
A Prefect future that represents the eventual execution of a flow run.
Methods:
flow_run_id
The ID of the flow run associated with this future
state
The current state of the flow run associated with this future
wait
result
add_done_callback
PrefectFutureList
A list of Prefect futures.
This class provides methods to wait for all futures in the list to complete and to retrieve the results of all task runs.
Methods:
wait
Wait for all futures in the list to complete.
Args:
timeout
: The maximum number of seconds to wait for all futures to complete. This method will not raise if the timeout is reached.
result
Get the results of all task runs associated with the futures in the list.
Args:
timeout
: The maximum number of seconds to wait for all futures to complete.raise_on_failure
: IfTrue
, an exception will be raised if any task run fails.
Returns:
- A list of results of the task runs.
Raises:
TimeoutError
: If the timeout is reached before all futures complete.
DoneAndNotDoneFutures
A named 2-tuple of sets.
multiple inheritance supported in 3.11+, use typing_extensions.NamedTuple