Skip to main content

Deploy a Serverless Worker on GCP Cloud Run

View Markdown

This guide walks through deploying a Temporal Serverless Worker to a GCP Cloud Run Worker Pool.

A Cloud Run Worker Pool runs long-lived instances that poll the Task Queue continuously. Temporal's Worker Controller Instance (WCI) adjusts the pool's instance count through the Cloud Run admin API as work arrives and drains. For how the pool scales and how instances are shut down, see Serverless Workers on GCP Cloud Run.

Prerequisites​

  • A Temporal Cloud account with a GCP-hosted Namespace, or a self-hosted Temporal Service v1.31.0 or later. The Namespace's cloud provider must match the serverless compute provider.
  • For self-hosted deployments, complete the self-hosted setup before following this guide.
  • Every Workflow must declare a versioning behavior, or the Worker must set a default versioning behavior.
  • A GCP project with the Cloud Run and Artifact Registry APIs enabled, and permissions to create Worker Pools, service accounts, and Secret Manager secrets.
  • The gcloud CLI installed and authenticated. You may use other tools to perform the GCP steps, such as the Google Cloud console or Terraform.
  • The Go SDK, Python SDK, or TypeScript SDK, depending on which language you are using. Use the tabs to select your language and the rest of the page will update accordingly.

1. Write Worker code​

A Cloud Run Serverless Worker is a standard long-running Worker. It connects to Temporal, registers its Workflows and Activities, declares its Worker Deployment Version, and polls the Task Queue until the instance is shut down.

import asyncio
import os

from temporalio.client import Client
from temporalio.common import VersioningBehavior, WorkerDeploymentVersion
from temporalio.worker import Worker, WorkerDeploymentConfig

from my_workflows import MyWorkflow
from my_activities import my_activity


async def main() -> None:
    client = await Client.connect(
        os.environ["TEMPORAL_ADDRESS"],
        namespace=os.environ["TEMPORAL_NAMESPACE"],
        api_key=os.environ.get("TEMPORAL_API_KEY"),
        tls=True,
    )
    worker = Worker(
        client,
        task_queue=os.environ["TEMPORAL_TASK_QUEUE"],
        workflows=[MyWorkflow],
        activities=[my_activity],
        deployment_config=WorkerDeploymentConfig(
            version=WorkerDeploymentVersion(
                deployment_name="my-app",
                build_id="build-1",
            ),
            use_worker_versioning=True,
            default_versioning_behavior=VersioningBehavior.PINNED,
        ),
    )
    await worker.run()


if __name__ == "__main__":
    asyncio.run(main())

Each Workflow must have a versioning behavior, either PINNED or AUTO_UPGRADE. Set it per-Workflow in the @workflow.defn decorator, or set a Worker-level default with default_versioning_behavior as shown above.

from temporalio import workflow
from temporalio.common import VersioningBehavior


@workflow.defn(versioning_behavior=VersioningBehavior.PINNED)
class MyWorkflow:
    @workflow.run
    async def run(self, input: str) -> str:
        ...

To export traces and Temporal Core metrics to Google Cloud, add the OpenTelemetry plugin to the Worker. See Serverless Workers on GCP Cloud Run - Python SDK.

2. Deploy to a Cloud Run Worker Pool​

Containerize the Worker, push the image to Artifact Registry, and create the Worker Pool.

i. Containerize the Worker​

Package the Worker and its dependencies into a container image that Cloud Run runs for each Worker Pool instance. The image's entrypoint must start your Worker process, so an instance begins polling the Task Queue as soon as it starts.

FROM python:3.12-slim

RUN pip install --no-cache-dir "temporalio>=1.30.0,<2"

WORKDIR /app
COPY . /app

CMD ["python", "-m", "worker"]

ii. Build and push the image​

Build the image and push it to Artifact Registry:

gcloud builds submit \
  --tag <REGION>-docker.pkg.dev/<YOUR_GCP_PROJECT>/<REPOSITORY>/my-temporal-worker:build-1 \
  --project <YOUR_GCP_PROJECT> \
  --region <REGION>

iii. Create the Worker Pool​

Create an empty Worker Pool. Start it at zero instances: the WCI raises the instance count once the Worker Deployment Version is current and Tasks arrive.

Store the API key (or TLS material) in Secret Manager rather than passing it as a plaintext environment variable, and grant the Worker Pool's runtime service account the roles/secretmanager.secretAccessor role on that secret.

gcloud run worker-pools deploy my-temporal-worker-pool \
  --image <REGION>-docker.pkg.dev/<YOUR_GCP_PROJECT>/<REPOSITORY>/my-temporal-worker:build-1 \
  --region <REGION> \
  --project <YOUR_GCP_PROJECT> \
  --service-account <RUNTIME_SERVICE_ACCOUNT> \
  --instances 0 \
  --set-env-vars TEMPORAL_ADDRESS=<your-temporal-address>:7233,TEMPORAL_NAMESPACE=<your-namespace>,TEMPORAL_TASK_QUEUE=my-task-queue \
  --set-secrets TEMPORAL_API_KEY=<SECRET_NAME>:latest
ParameterDescription
--imageThe Worker image you pushed in Step ii.
--service-accountThe runtime service account the Worker Pool instances run as. It needs read access to the secrets referenced below. This is distinct from the invoker service account in Step 3, which Temporal impersonates to scale the pool.
--instancesInitial instance count. Set to 0; the WCI manages the count after the version is current.
--set-env-varsNon-secret Worker configuration. TEMPORAL_TASK_QUEUE overrides the value set in code.
--set-secretsMaps a Secret Manager secret to an environment variable. Use for TEMPORAL_API_KEY or TLS client cert/key material.

The serverless Worker packages read environment variables and configuration files automatically at startup. For the full list of supported environment variables, config file format, and profiles, see Environment configuration.

3. Grant Temporal permission to manage the Worker Pool (Cloud only)​

This section applies to Temporal Cloud. For self-hosted Temporal Service deployments, see Self-hosted setup.

Temporal Cloud scales the Worker Pool by impersonating a service account you create, called the invoker. The invoker reads and scales the pool through the Cloud Run admin API. It does not run the pool. Your Worker instances run as the separate runtime service account you set in Step 2.

When you create a Worker Deployment in the Temporal Cloud UI (Workers → Create Worker Deployment), it provides a Terraform template. Copy it. The template uses the serverless-workers/gcp/cloud-run module with your account's impersonator_service_account_emails already filled in:

module "serverless-worker-cloud-run" {
  source = "github.com/temporalio/terraform-modules//modules/serverless-workers/gcp/cloud-run"

  project_id         = "<YOUR_GCP_PROJECT>"
  invoker_account_id = "temporal-worker-pool-invoker"

  impersonator_service_account_emails = [
    "<provided by Temporal Cloud>",
  ]
}

Set these variables:

VariableDescription
project_idThe GCP project that hosts the Worker Pool and the invoker service account.
invoker_account_idA name for the invoker service account the module creates. The full email becomes <invoker_account_id>@<project_id>.iam.gserviceaccount.com.
impersonator_service_account_emailsTemporal Cloud's service accounts, granted roles/iam.serviceAccountTokenCreator on the invoker so they can impersonate it. Filled in by the template.

Make sure you are logged in to the GCP project, then apply the configuration:

terraform init
terraform apply

Terraform prints an invoker_email output. Use it as the --gcp-cloud-run-service-account value when you create the Worker Deployment Version in Step 4.

4. Create Worker Deployment Version​

Create a Worker Deployment Version whose compute configuration points at your Worker Pool. The compute configuration tells Temporal where the pool lives and which service account to impersonate to manage it. The deployment name and build ID must match the values in your Worker code.

In the Temporal Cloud UI, go to Workers → Create Worker Deployment and fill out the required fields:

  • Name — the Worker Deployment name. Must match deployment_name in your Worker code.
  • Build ID — the version identifier. Must match build_id in your Worker code.
  • Compute Provider — select Google Cloud Run.
  • Resource — the Project ID, Region, and Worker Pool from Step 2.
  • Access — the Service Account email Temporal Cloud impersonates: the invoker_email output from Step 3.

Scaling and Lifecycle is optional. Leave the defaults unless you need to change them. If your Worker runs long-running Activities, set no_sync_quiet_ms longer than your longest Activity runtime and use Activity Heartbeats so an interrupted Activity resumes from its last recorded progress. See GCP Cloud Run lifecycle.

5. Set version as current​

Set the version as current. Without this step, Tasks on the Task Queue will not route to the version, and the WCI will not start any instances.

temporal worker deployment set-current-version \
  --namespace <YOUR_NAMESPACE> \
  --deployment-name my-app \
  --build-id build-1

This command asks you to confirm, because it changes which version new Tasks route to. Pass --yes to skip the prompt.

6. Verify deployment​

Start a Workflow on the same Task Queue to confirm that the WCI starts a Worker instance and processes the Task.

temporal workflow start \
  --namespace <YOUR_NAMESPACE> \
  --task-queue my-task-queue \
  --type MyWorkflow \
  --input '"Hello, serverless!"'

When Tasks arrive with no active pollers, the WCI raises the Worker Pool's instance count. Cloud Run starts an instance, the Worker connects to Temporal, picks up the Task, and processes it.

You can verify the invocation by checking:

  • Temporal UI: The Workflow execution should show Task completions in the event history.
  • Cloud Run logs: Once the WCI starts an instance, the Worker Pool's logs (gcloud run worker-pools logs read my-temporal-worker-pool --region <REGION> --project <YOUR_GCP_PROJECT>) show the Worker startup and Task processing. The pool produces no logs until an instance is running.

If the Workflow does not progress or no instance starts, see Troubleshoot Serverless Workers.