Develop overview

These pages explain how to develop Prefect workflows.

The Write and run workflows section introduces and explains Prefect’s core concepts.

• Write and run flows introduces the most central Prefect object, the flow.
• Write and run tasks shows you how to create task decorated functions that represent a discrete unit of work in a Prefect workflow.
• Run tasks in the background explains how to use tasks to execute small, discrete units of work quickly.
• Log run activity shows you how to capture fine-grained information about flows and tasks for monitoring, troubleshooting, and auditing.

The Configure runtime behavior section introduces advanced features for interacting with running workflows and automatically taking actions based on workflow state.

• Configure task caching explores how to cache task results to save money and time.
• Write transactions shows how to group tasks together to ensure they run atomically.
• Execute code on state changes shows how to run code when a task or flow changes state.
• Pause and resume flow runs explains how to halt flow runs for input.
• Send and receive flow run inputs highlights how to run Human-in-the-Loop workflows with Prefect.
• Access runtime context shows how to access information about the current run.

The Manage state and configuration section explores how to manage data and configuration in Prefect.

• Manage results explains how to store and retrieve task results.
• Manage states discusses how to learn about a workflow through task and flow states.
• Create run artifacts demonstrates how to create artifacts for human consumption in the UI.
• Set and get variables shows how to store and retrieve configuration values from API.
• Connect to external systems discusses how Prefect blocks can help you connect to external systems.
• Manage cloud-hosted data shows convieniences for moving data to and from cloud storage providers.

The Manage concurrency section explains how to speed up your workflows and limit activities that can take place concurrently.

• Run tasks concurrently or in parallel explains how to run tasks concurrently or in parallel with Dask or Ray.
• Limit concurrent task runs shows how to prevent too many tasks from running simultaneously.
• Apply concurrency and rate limits demonstrates how to control concurrency and apply rate limits using Prefect’s provided utilities.

Finally, Test workflows discusses tools for testing workflows.