Learn how to monitor worker health and automatically restart workers when they become unresponsive.
Worker healthchecks provide a way to monitor whether your Prefect workers are functioning properly and polling for work as expected. This is particularly useful in production environments where you need to ensure workers are available to execute scheduled flow runs.
Worker healthchecks work by:
This allows external monitoring systems, container orchestrators, or process managers to detect and restart unhealthy workers automatically.
Start a worker with healthchecks enabled using the --with-healthcheck
flag:
This starts both the worker and a lightweight HTTP health server that exposes a /health
endpoint.
When enabled, the worker exposes an HTTP endpoint at:
For GET requests the endpoint returns:
{"message": "OK"}
when the worker is healthy{"message": "Worker may be unresponsive at this time"}
when the worker is unhealthyYou can customize the health server’s host and port using environment variables:
A worker is considered unhealthy if it hasn’t polled for flow runs within a specific timeframe defined by its configured polling interval. The health check algorithm works as follows:
If a worker hasn’t made a successful poll within the time window of PREFECT_WORKER_QUERY_SECONDS * 30
seconds, it is considered unhealthy
and its health endpoint will return 503 (Service Unavailable). With default settings, a worker is unhealthy if it hasn’t polled in 450 seconds (7.5 minutes).
This generous threshold accounts for temporary network issues, API unavailability, or brief worker pauses without triggering false alarms.
Use Docker’s built-in health check functionality by including these lines in your Dockerfile:
For more information see Docker’s reference guide.
Configure Kubernetes to automatically restart unhealthy worker pods by including this configuration in your worker deployment:
This is enabled by default when using Prefect’s Helm Chart.
Use Docker Compose’s built-in health check functionality by including these lines in your Docker compose file:
For more information see Docker Compose’s reference guide.
--with-healthcheck
http://localhost:8080/health
)PREFECT_WORKER_WEBSERVER_PORT
prefect config view --show-defaults | grep WORKER
PREFECT_LOGGING_LEVEL=DEBUG
on the worker to see detailed polling activityPREFECT_WORKER_QUERY_SECONDS=5
PREFECT_WORKER_QUERY_SECONDS
if your API has high latencyPREFECT_API_URL
is correctly configured and accessibleRelevant settings for worker health and polling behavior:
PREFECT_WORKER_HEARTBEAT_SECONDS
: How often workers send heartbeats to the API (default: 30)PREFECT_WORKER_QUERY_SECONDS
: How often workers poll for new flow runs (default: 15)PREFECT_WORKER_PREFETCH_SECONDS
: How far in advance to submit flow runs (default: 10)PREFECT_WORKER_WEBSERVER_HOST
: Health server host (default: 127.0.0.1)PREFECT_WORKER_WEBSERVER_PORT
: Health server port (default: 8080)For more information on worker configuration, see the Workers concept guide.