How to produce workflow artifacts
Create link artifacts
To create a link artifact, use the create_link_artifact()
function.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI,
provide a key
argument to the create_link_artifact()
function to track the artifact’s history over time.
Without a key
, the artifact is only visible in the Artifacts tab of the associated flow run or task run.
Specify multiple artifacts with the same key for artifact lineage
You can specify multiple artifacts with the same key to easily track something very specific, such as irregularities in your data pipeline.
After running flows that create artifacts, view the artifacts in the Artifacts page of the UI. Click into the “irregular-data” artifact to see its versions, along with custom descriptions and links to the relevant data.
You can also view information about the artifact such as:
- its associated flow run or task run id
- previous and future versions of the artifact (multiple artifacts can have the same key to show lineage)
- data (in this case a Markdown-rendered link)
- an optional Markdown description
- when the artifact was created or updated
To make the links more readable for you and your collaborators, you can pass a link_text
argument:
In the above example, the create_link_artifact
method is used within a flow to create a link artifact with a key of my-important-link
.
The link
parameter specifies the external resource to link to, and link_text
specifies the text to display for the link.
Add an optional description
for context.
Create progress artifacts
Progress artifacts render dynamically on the flow run graph in the Prefect UI, indicating the progress of long-running tasks.
To create a progress artifact, use the create_progress_artifact()
function. To update a progress artifact, use the update_progress_artifact()
function.
Progress artifacts are updated with the update_progress_artifact()
function. Prefect updates a progress artifact in place, rather than versioning it.
Create Markdown artifacts
To create a Markdown artifact, you can use the create_markdown_artifact()
function.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI, provide a key
argument to the create_markdown_artifact()
function to track an artifact’s history over time.
Without a key
, the artifact is only visible in the Artifacts tab of the associated flow run or task run.
Don’t indent Markdown
Don’t indent Markdown in multi-line strings. Otherwise it will be interpreted incorrectly.
After running the above flow, you should see your “gtm-report” artifact in the Artifacts page of the UI.
You can view the associated flow run id or task run id, previous versions of the artifact, the rendered Markdown data, and the optional Markdown description.
Create table artifacts
Create a table artifact by calling create_table_artifact()
.
To create multiple versions of the same artifact and/or view them on the Artifacts page of the Prefect UI, provide a key
argument to the create_table_artifact()
function to track an artifact’s history over time.
Without a key
, the artifact is only visible in the artifacts tab of the associated flow run or task run.
The create_table_artifact()
function accepts a table
argument. Pass this as a list of lists, a list of dictionaries, or a dictionary of lists.
Create image artifacts
Image artifacts render publicly available images in the Prefect UI. To create an image artifact, use the create_image_artifact()
function.
To create an artifact that links to a private image, use the create_link_artifact()
function instead.
Reading artifacts
In the Prefect UI, you can view all of the latest versions of your artifacts and click into a specific artifact to see its lineage over time. Additionally, you can inspect all versions of an artifact with a given key from the CLI by running:
or view all artifacts by running:
You can also use the Prefect REST API to programmatically filter your results.
Fetching artifacts
In Python code, you can retrieve an existing artifact with the Artifact.get
class method:
Delete artifacts
Delete an artifact in the CLI by providing a key or id: