Consuming reserved zonal resources

This page shows you how to consume reserved Compute Engine zonal resources in specific GKE workloads. These capacity reservations give you a high level of assurance that specific hardware is available for your workloads.

Ensure that you're already familiar with the concepts of Compute Engine reservations, like consumption types, share types, and provisioning types. For details, see Reservations of Compute Engine zonal resources.

This page is intended for the following people:

• Application operators who deploy workloads that should run as soon as possible, usually with specialized hardware like GPUs.
• Platform administrators who want to obtain a high level of assurance that workloads run on optimized hardware that meets both application and organizational requirements.

About reservation consumption in GKE

Compute Engine capacity reservations let you provision specific hardware configurations in Google Cloud zones, either immediately or at a specified future time. You can then consume this reserved capacity in GKE.

Depending on your GKE mode of operation, you can consume the following reservation types:

• Autopilot mode: specific reservations, or any matching reservation using custom ComputeClasses.
• Standard mode: specific reservations or any matching reservation.

For any matching reservation, you can choose between two behaviors:

• Any matching reservation with fallback: attempts to consume any matching reservation. If none are available, it falls back to provisioning standard on-demand capacity.
• Any matching reservation without fallback: attempts to consume any matching reservation. If none are available, the provisioning fails without falling back to on-demand capacity.

To enable consuming reservations to create your resources, you must specify a reservation affinity, like any, any-reservation-then-fail, or specific.

Reservation consumption options in GKE

GKE lets you consume reservations directly in individual workloads by using Kubernetes nodeSelectors in your workload manifest or by creating Standard mode node pools that consume the reservation. This page describes the approach of directly selecting reservations in individual resources.

You can also configure GKE to consume reservations during scaling operations that create new nodes by using custom compute classes. Custom compute classes let platform administrators define a hierarchy of node configurations for GKE to prioritize during node scaling so that workloads run on your selected hardware.

You can specify reservations in your custom compute class configuration so that any GKE workload that uses that custom ComputeClass indicates to GKE to consume the specified reservations for that ComputeClass.

To learn more, in the "About custom compute classes" page, see Consume Compute Engine reservations.

Before you begin

Before you start, make sure that you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• To use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.

Consume capacity reservations in Autopilot clusters

Autopilot clusters support consuming resources from Compute Engine capacity reservations in the same project or in a shared project. You can set the consumption type property of the target reservation to specific, and explicitly select that reservation in your manifest. Alternatively, you can consume any matching reservation using custom ComputeClasses with the AnyBestEffort reservation affinity. If you don't explicitly specify a reservation or configure a custom ComputeClasses to consume one, Autopilot clusters won't consume reservations. To learn more about reservation consumption types, see How reservations work.

These reservations qualify for Compute flexible committed use discounts. You must use the Accelerator compute class or the Performance compute class to consume capacity reservations.

• Before you begin, create an Autopilot cluster running the following versions:

  • To consume reserved accelerators, such as GPUs: 1.28.6-gke.1095000 or later
  • To run Pods on a specific machine series and with each Pod on its own node: 1.28.6-gke.1369000 and later or version 1.29.1-gke.1575000 and later.
  • To consume any available reservation without falling back to on-demand capacity: 1.36.0-gke.3204000 or later.

Create capacity reservations for Autopilot

Autopilot Pods can consume reservations that have the specific consumption type property in the same project as the cluster or in a shared reservation from a different project. You can consume the reserved hardware by explicitly referencing that reservation in your manifest, or you can consume any matching reservation using custom ComputeClasses. You can consume reservations in Autopilot for the following types of hardware:

• Any of the following types of GPUs:

  • nvidia-gb200: NVIDIA GB200 (Preview)
  • nvidia-b200: NVIDIA B200 (180GB)
  • nvidia-h200-141gb: NVIDIA H200 (141GB)
  • nvidia-h100-mega-80gb: NVIDIA H100 Mega (80GB)
  • nvidia-h100-80gb: NVIDIA H100 (80GB)
  • nvidia-a100-80gb: NVIDIA A100 (80GB)
  • nvidia-tesla-a100: NVIDIA A100 (40GB)
  • nvidia-rtx-pro-6000: NVIDIA RTX PRO 6000
  • nvidia-l4: NVIDIA L4
  • nvidia-tesla-t4: NVIDIA T4

• Any of the following types of TPUs:

  • tpu7x: Ironwood (TPU7x)
  • tpu-v6e-slice: TPU v6e slice
  • tpu-v5p-slice: TPU v5p slice
  • tpu-v5-lite-podslice: TPU v5 lite podslice
  • tpu-v4-lite-device: TPU v4 lite device
  • tpu-v4-podslice: TPU v4 podslice
  • tpu-v3-device: TPU v3 device
  • tpu-v3-slice: TPU v3 podslice

To create a capacity reservation, see the following resources. The reservation must meet the following requirements:

• The machine types, accelerator types, and accelerator quantities match what your workloads will consume.

• The reservation uses the specific consumption type, unless you are using a custom ComputeClass to consume any matching reservation. For example, in the gcloud CLI, you must specify the --require-specific-reservation flag when you create a specific reservation.

• Create a reservation for a single project

• Create a shared reservation

GKE automatically attaches any Local SSDs from the selected specific reservation to your node. You don't need to select individual Local SSDs in your workload manifest. For example, if the reservation that you select includes two Local SSDs, the nodes that GKE creates from that reservation have two Local SSDs attached.

Consume any matching reservation with fallback in Autopilot

This section shows you how to configure your Autopilot workloads to consume any matching reservation with a fallback to on-demand capacity. You do this by creating a custom ComputeClass that specifies the AnyBestEffort reservation affinity.

1. Save the following custom ComputeClass manifest as any-reservation-cc.yaml:

   apiVersion: cloud.google.com/v1
   kind: ComputeClass
   metadata:
     name: any-reservation
   spec:
     priorities:
     - reservations:
         affinity: AnyBestEffort
   

2. Create the custom ComputeClass:

   kubectl apply -f any-reservation-cc.yaml
   

3. Save the following Pod manifest as any-reservation-pod.yaml:

   apiVersion: v1
   kind: Pod
   metadata:
     name: any-reservation-pod
   spec:
     nodeSelector:
       cloud.google.com/compute-class: any-reservation
     containers:
     - name: my-container
       image: "k8s.gcr.io/pause"
       resources:
         requests:
           cpu: 2
           memory: "4Gi"
   

4. Deploy the Pod:

   kubectl apply -f any-reservation-pod.yaml
   

Consume any matching reservation without fallback in Autopilot

This section shows you how to configure your Autopilot workloads to consume any matching reservation without falling back to on-demand capacity if the reservation is exhausted. You do this by creating a custom ComputeClass that specifies the AnyThenFail reservation affinity and sets the value of the whenUnsatisfiable field to DoNotScaleUp. This reservation affinity requires GKE version 1.36.0-gke.3204000 or later.

1. Save the following custom ComputeClass manifest as any-then-fail-cc.yaml:

   apiVersion: cloud.google.com/v1
   kind: ComputeClass
   metadata:
     name: any-then-fail
   spec:
     priorities:
     - reservations:
         affinity: AnyThenFail
       whenUnsatisfiable: DoNotScaleUp
   

2. Create the custom ComputeClass:

   kubectl apply -f any-then-fail-cc.yaml
   

3. Save the following Pod manifest as any-then-fail-pod.yaml:

   apiVersion: v1
   kind: Pod
   metadata:
     name: any-then-fail-pod
   spec:
     nodeSelector:
       cloud.google.com/compute-class: any-then-fail
     containers:
     - name: my-container
       image: "k8s.gcr.io/pause"
       resources:
         requests:
           cpu: 2
           memory: "4Gi"
   

4. Deploy the Pod:

   kubectl apply -f any-then-fail-pod.yaml
   

Consume a specific reservation in the same project in Autopilot

This section shows you how to consume a specific capacity reservation that's in the same project as your cluster. You can use kubectl or Terraform.