Skip to main content

​
prefect.runner.storage

​
Functions

​
create_storage_from_source 

create_storage_from_source(source: str, pull_interval: Optional[int] = 60) -> RunnerStorage
Creates a storage object from a URL. Args:
  • url: The URL to create a storage object from. Supports git and fsspec URLs.
  • pull_interval: The interval at which to pull contents from remote storage to local storage
Returns:
  • A runner storage compatible object

​
Classes

​
RunnerStorage

A storage interface for a runner to use to retrieve remotely stored flow code. Methods:

​
destination 

destination(self) -> Path
The local file path to pull contents from remote storage to.

​
pull_code 

pull_code(self) -> None
Pulls contents from remote storage to the local filesystem.

​
pull_interval 

pull_interval(self) -> Optional[int]
The interval at which contents from remote storage should be pulled to local storage. If None, remote storage will perform a one-time sync.

​
set_base_path 

set_base_path(self, path: Path) -> None
Sets the base path to use when pulling contents from remote storage to local storage.

​
to_pull_step 

to_pull_step(self) -> dict[str, Any] | list[dict[str, Any]]
Returns a dictionary representation of the storage object that can be used as a deployment pull step.

​
GitCredentials

​
GitRepository

Pulls the contents of a git repository to the local filesystem. Args:
  • url: The URL of the git repository to pull from
  • credentials: A dictionary of credentials to use when pulling from the repository. If a username is provided, an access token must also be provided.
  • name: The name of the repository. If not provided, the name will be inferred from the repository URL.
  • branch: The branch to pull from. Defaults to “main”.
  • pull_interval: The interval in seconds at which to pull contents from remote storage to local storage. If None, remote storage will perform a one-time sync.
  • directories: The directories to pull from the Git repository (uses git sparse-checkout)
Examples: Pull the contents of a private git repository to the local filesystem: 
from prefect.runner.storage import GitRepository

storage = GitRepository(
    url="https://github.com/org/repo.git",
    credentials={"username": "oauth2", "access_token": "my-access-token"},
)

await storage.pull_code()
Methods:

​
destination 

destination(self) -> Path

​
is_current_commit 

is_current_commit(self) -> bool
Check if the current commit is the same as the commit SHA

​
is_shallow_clone 

is_shallow_clone(self) -> bool
Check if the repository is a shallow clone

​
is_sparsely_checked_out 

is_sparsely_checked_out(self) -> bool
Check if existing repo is sparsely checked out

​
pull_code 

pull_code(self) -> None
Pulls the contents of the configured repository to the local filesystem.

​
pull_interval 

pull_interval(self) -> Optional[int]

​
set_base_path 

set_base_path(self, path: Path) -> None

​
to_pull_step 

to_pull_step(self) -> dict[str, Any]

​
RemoteStorage

Pulls the contents of a remote storage location to the local filesystem. Args:
  • url: The URL of the remote storage location to pull from. Supports fsspec URLs. Some protocols may require an additional fsspec dependency to be installed. Refer to the fsspec docs for more details.
  • pull_interval: The interval in seconds at which to pull contents from remote storage to local storage. If None, remote storage will perform a one-time sync.
  • **settings: Any additional settings to pass the fsspec filesystem class.
Examples: Pull the contents of a remote storage location to the local filesystem: 
from prefect.runner.storage import RemoteStorage

storage = RemoteStorage(url="s3://my-bucket/my-folder")

await storage.pull_code()
Pull the contents of a remote storage location to the local filesystem with additional settings: 
from prefect.runner.storage import RemoteStorage
from prefect.blocks.system import Secret

storage = RemoteStorage(
    url="s3://my-bucket/my-folder",
    # Use Secret blocks to keep credentials out of your code
    key=Secret.load("my-aws-access-key"),
    secret=Secret.load("my-aws-secret-key"),
)

await storage.pull_code()
Methods:

​
destination 

destination(self) -> Path
The local file path to pull contents from remote storage to.

​
pull_code 

pull_code(self) -> None
Pulls contents from remote storage to the local filesystem.

​
pull_interval 

pull_interval(self) -> Optional[int]
The interval at which contents from remote storage should be pulled to local storage. If None, remote storage will perform a one-time sync.

​
set_base_path 

set_base_path(self, path: Path) -> None

​
to_pull_step 

to_pull_step(self) -> dict[str, Any]
Returns a dictionary representation of the storage object that can be used as a deployment pull step.

​
BlockStorageAdapter

A storage adapter for a storage block object to allow it to be used as a runner storage object. Methods:

​
destination 

destination(self) -> Path

​
pull_code 

pull_code(self) -> None

​
pull_interval 

pull_interval(self) -> Optional[int]

​
set_base_path 

set_base_path(self, path: Path) -> None

​
to_pull_step 

to_pull_step(self) -> dict[str, Any]

​
LocalStorage

Sets the working directory in the local filesystem. Parameters: Path: Local file path to set the working directory for the flow Examples: Sets the working directory for the local path to the flow: 
from prefect.runner.storage import Localstorage
storage = LocalStorage(
    path="/path/to/local/flow_directory",
)
Methods:

​
destination 

destination(self) -> Path

​
pull_code 

pull_code(self) -> None

​
pull_interval 

pull_interval(self) -> Optional[int]

​
set_base_path 

set_base_path(self, path: Path) -> None

​
to_pull_step 

to_pull_step(self) -> dict[str, Any]
Returns a dictionary representation of the storage object that can be used as a deployment pull step.
⌘I