Deployments allow you to run flows on a schedule and trigger runs based on events.

Deployments are server-side representations of flows. They store the crucial metadata for remote orchestration including when, where, and how a workflow should run.

In addition to manually triggering and managing flow runs, deploying a flow exposes an API and UI that allow you to:

• trigger new runs, cancel active runs, pause scheduled runs, customize parameters, and more
• remotely configure schedules and automation rules
• dynamically provision infrastructure with work pools - optionally with templated guardrails for other users

In Prefect Cloud, deployment configuration is versioned, and a new deployment version is created each time a deployment is updated.

​

Work pools

Work pools allow you to switch between different types of infrastructure and to create a template for deployments. Data platform teams find work pools especially useful for managing infrastructure configuration across teams of data professionals.

Common work pool types include Docker, Kubernetes, and serverless options such as AWS ECS, Azure ACI, GCP Vertex AI, or GCP Google Cloud Run.

​

Work pool-based deployment requirements

Deployments created through the Python SDK that use a work pool require a name. This value becomes the deployment name. A work_pool_name is also required.

Your flow code location can be specified in a few ways:

1. Bake it into your Docker image (for work-pools that use Docker images). As shown in the example above,Prefect facilitates this as the default method for deployments created with the Python SDK. This method requires that you specify the image argument in the deploy method.
2. Call from_source on a flow and specify one of the following:
   1. the git-based cloud provider location (for example, GitHub)
   2. the cloud provider storage location (for example, AWS S3)
   3. the local path (an option for Process work pools)

See the Retrieve code from storage docs for more information about flow code storage.

​

Run a deployment

You can set a deployment to run manually, on a schedule, or in response to an event.

The deployment inherits the infrastructure configuration from the work pool, and can be overridden at deployment creation time or at runtime.

​

Work pools that require a worker

To run a deployment with a hybrid work pool type, such as Docker or Kubernetes, you must start a worker.

A Prefect worker is a client-side process that checks for scheduled flow runs in the work pool that it matches. When a scheduled run is found, the worker kicks off a flow run on the specified infrastructure and monitors the flow run until completion.

​

Work pools that don’t require a worker

Prefect Cloud offers push work pools that run flows on Cloud provider serverless infrastructure without a worker and that can be set up quickly.

Prefect Cloud also provides the option to run work flows on Prefect’s infrastructure through a Prefect Managed work pool.

These work pool types do not require a worker to run flows. However, they do require sharing a bit more information with Prefect, which can be a challenge depending upon the security posture of your organization.

​

Static vs. dynamic infrastructure

You can deploy your flows on long-lived static infrastructure or on dynamic infrastructure that is able to scale horizontally. The best choice depends on your use case.

​

Static infrastructure

When you have several flows running regularly, the serve method of the Flow object or the serve utility is a great option for managing multiple flows simultaneously.

Once you have authored your flow and decided on its deployment settings, run this long-running process in a location of your choosing. The process stays in communication with the Prefect API, monitoring for work and submitting each run within an individual subprocess. Because runs are submitted to subprocesses, any external infrastructure configuration must be set up beforehand and kept associated with this process.

Benefits to this approach include:

• Users are in complete control of their infrastructure, and anywhere the “serve” Python process can run is a suitable deployment environment.
• It is simple to reason about.
• Creating deployments requires a minimal set of decisions.
• Iteration speed is fast.

​

Dynamic infrastructure

Consider running flows on dynamically provisioned infrastructure with work pools when you have any of the following:

• Flows that require expensive infrastructure due to the long-running process.
• Flows with heterogeneous infrastructure needs across runs.
• Large volumes of deployments.
• An internal organizational structure in which deployment authors or runners are not members of the team that manages the infrastructure.

Work pools allow Prefect to exercise greater control of the infrastructure on which flows run. Options for serverless work pools allow you to scale to zero when workflows aren’t running. Prefect even provides you with the ability to provision cloud infrastructure via a single CLI command, if you use a Prefect Cloud push work pool option.

With work pools:

• You can configure and monitor infrastructure configuration within the Prefect UI.
• Infrastructure is ephemeral and dynamically provisioned.
• Prefect is more infrastructure-aware and collects more event data from your infrastructure by default.
• Highly decoupled setups are possible.

​

Deployment schema

class Deployment:
    """
    Structure of the schema defining a deployment
    """

    # required defining data
    name: str
    flow_id: UUID
    entrypoint: str
    path: str | None = None

    # workflow scheduling and parametrization
    parameters: dict[str, Any] | None = None
    parameter_openapi_schema: dict[str, Any] | None = None
    schedules: list[Schedule] | None = None
    paused: bool = False
    trigger: Trigger | None = None

    # concurrency limiting
    concurrency_limit: int | None = None
    concurrency_options: ConcurrencyOptions(collision_strategy=Literal['ENQUEUE', 'CANCEL_NEW']) | None = None

    # metadata for bookkeeping
    version: str | None = None
    version_type: VersionType | None = None
    description: str | None = None
    tags: list | None = None

    # worker-specific fields
    work_pool_name: str | None = None
    work_queue_name: str | None = None
    job_variables: dict[str, Any] | None = None
    pull_steps: dict[str, Any] | None = None

All methods for creating Prefect deployments are interfaces for populating this schema.

​

Required defining data

Deployments require a name and a reference to an underlying Flow.

The deployment name is not required to be unique across all deployments, but is required to be unique for a given flow ID. This means you will often see references to the deployment’s unique identifying name {FLOW_NAME}/{DEPLOYMENT_NAME}.

For example, to trigger a deployment run from the Prefect CLI:

prefect deployment run my-first-flow/my-first-deployment

or from the Python SDK with run_deployment:

from prefect.deployments import run_deployment


run_deployment(
    name="my-first-flow/my-first-deployment",
    parameters={"my_param": "42"},
    job_variables={"env": {"MY_ENV_VAR": "staging"}},
    timeout=0, # don't wait for the run to finish
)

The other two fields are:

• path: think of the path as the runtime working directory for the flow. For example, if a deployment references a workflow defined within a Docker image, the path is the absolute path to the parent directory where that workflow will run anytime the deployment is triggered. This interpretation is more subtle in the case of flows defined in remote filesystems.
• entrypoint: the entrypoint of a deployment is a relative reference to a function decorated as a flow that exists on some filesystem. It is always specified relative to the path. Entrypoints use Python’s standard path-to-object syntax (for example, path/to/file.py:function_name or simply path:object).

The entrypoint must reference the same flow as the flow ID.

Prefect requires that deployments reference flows defined within Python files. Flows defined within interactive REPLs or notebooks cannot currently be deployed as such. They are still valid flows that will be monitored by the API and observable in the UI whenever they are run, but Prefect cannot trigger them.

​

Workflow scheduling and parametrization

One of the primary motivations for creating deployments of flows is to remotely schedule and trigger them. Just as you can call flows as functions with different input values, deployments can be triggered or scheduled with different values through parameters.

These are the fields to capture the required metadata for those actions:

• schedules: a list of schedule objects. Most of the convenient interfaces for creating deployments allow users to avoid creating this object themselves. For example, when updating a deployment schedule in the UI basic information such as a cron string or interval is all that’s required.
• parameter_openapi_schema: an OpenAPI compatible schema that defines the types and defaults for the flow’s parameters. This is used by the UI and the backend to expose options for creating manual runs as well as type validation.
• parameters: default values of flow parameters that this deployment will pass on each run. These can be overwritten through a trigger or when manually creating a custom run.
• enforce_parameter_schema: a boolean flag that determines whether the API should validate the parameters passed to a flow run against the schema defined by parameter_openapi_schema.

​

Concurrency limiting

Prefect supports managing concurrency at the deployment level to enable limiting how many runs of a deployment can be active at once. To enable this behavior, deployments have the following fields:

• concurrency_limit: an integer that sets the maximum number of concurrent flow runs for the deployment.
• collision_strategy: configure the behavior for runs once the concurrency limit is reached. Falls back to ENQUEUE if unset.
  • ENQUEUE: new runs transition to AwaitingConcurrencySlot and execute as slots become available.
  • CANCEL_NEW: new runs are canceled until a slot becomes available.

prefect deploy ... --concurrency-limit 3 --collision-strategy ENQUEUE

​

Metadata for bookkeeping

Important information for the versions, descriptions, and tags fields:

• version: versions are always set by the client and can be any arbitrary string. We recommend tightly coupling this field on your deployments to your software development lifecycle and choosing human-readable version strings. If left unset, the version field will be automatically populated in one of two ways:
  • If deploying from a directory inside a Git repository or from a CI environment on a supported version control provider, version will be the first eight characters of your commit hash.
  • In all other circumstances, version will be your flow’s version, which if not assigned in the flow decorator (@flow(version="my-version")) will be a hash of the file the flow is defined in.
• version_type: When a deployment is created or updated, Prefect will attempt to infer version information from your environment. Providing a version_type instructs Prefect to only attempt version information collection from an environment of that type. The following version types are available: vcs:github, vcs:gitlab, vcs:bitbucket, vcs:azuredevops, vcs:git, or prefect:simple. vcs:git offers similar versioning detail to officially supported version control platforms, but does not support direct linking to commits from the Prefect Cloud UI. It is meant as a fallback option in case your version control platform is not supported. prefect:simple is for any deployment version created where no Git context is available. If left unset, Prefect will automatically select the appropriate version_type based on the detected environment.
• description: provide reference material such as intended use and parameter documentation. Markdown is accepted. The docstring of your flow function is the default value.
• tags: group related work together across a diverse set of objects. Tags set on a deployment are inherited by that deployment’s flow runs. Filter, customize views, and searching by tag.

​

Worker-specific fields

Work pools and workers are an advanced deployment pattern that allow you to dynamically provision infrastructure for each flow run. The work pool job template interface allows users to create and govern opinionated interfaces to their workflow infrastructure. To do this, a deployment using workers needs the following fields:

• work_pool_name: the name of the work pool this deployment is associated with. Work pool types mirror infrastructure types, which means this field impacts the options available for the other fields.
• work_queue_name: if you are using work queues to either manage priority or concurrency, you can associate a deployment with a specific queue within a work pool using this field.
• job_variables: this field allows deployment authors to customize whatever infrastructure options have been exposed on this work pool. This field is often used for Docker image names, Kubernetes annotations and limits, and environment variables.
• pull_steps: a JSON description of steps that retrieves flow code or configuration, and prepares the runtime environment for workflow execution.

Pull steps allow users to highly decouple their workflow architecture. For example, a common use of pull steps is to dynamically pull code from remote filesystems such as GitHub with each run of their deployment.