Skip to main content

Self-hosted Archival setup

View Markdown

Use Archival to back up closed Workflow Execution Event Histories and Visibility records from Temporal Service persistence to blob storage.

When a Workflow Execution closes, Temporal schedules close-processing tasks for both Visibility records and Event History Archival. Archival then runs asynchronously after a randomized delay. By default, that delay is up to 5 minutes set by history.archivalProcessorArchiveDelay, and is capped by the Namespace Retention Period.

The closed execution still stays in Temporal persistence until retention cleanup runs. For some time, the same closed execution can exist in both persistence and archival storage. Archival enables Workflow Execution data to persist beyond retention without overwhelming the Temporal Service persistence store.

Use this to keep closed Workflow data available for compliance, audits, and debugging without keeping all closed data in your primary persistence store.

Experimental feature

Archival is an experimental feature and not subject to normal versioning and support policy.

Archival is not supported when running Temporal through Docker. It's disabled by default when installing the system manually and when deploying through helm charts. It can be enabled in the server configuration.

Set up Archival​

To set up Archival, decide the following:

  • Which provider to use: S3, Google Cloud, local file system, or custom.
  • Which URI to use: URI scheme and path identify the provider and destination.
  • Which Namespace should use Archival: Archival must be enabled at both the Temporal Service level and the Namespace level.

Take the following steps to set up Archival:

  1. Choose an Archival provider.
  2. Configure Archival options.
  3. Create an Archiving Namespace.

Choose an Archival provider​

Temporal directly supports several providers:

  • Local file system: The filestore archiver is used to archive data in the file system of whatever host the Temporal server is running on. In the case of temporal helm-charts, the archive data is stored in the history pod. APIs do not function with the filestore archive. This provider is used mainly for local installations and testing and should not be relied on for production environments.
  • Google Cloud: The gcloud archiver is used to connect and archive data with Google Cloud.
  • S3: The s3store archiver is used to connect and archive data with S3.
  • Custom: If you want to use a provider that is not currently supported, you can create your own archiver to support it.

Save the provider URI so you can pass it when you create a Namespace with Archival enabled.

Configure Archival options​

Configure Archival in config/development.yaml:

# Temporal Service level Archival config
archival:
  # Event History configuration
  history:
    # Archival is enabled at the Temporal Service level
    state: 'enabled'
    enableRead: true
    # Namespaces can use either the local filestore provider or the Google Cloud provider
    provider: