Local NVMe Storage¶

Many instance types ship with NVMe disks physically attached to the host. These are far faster than network-attached volumes (EBS, Persistent Disk, Azure Disk) and are the right backing store for scratch space, caches, spill files, and analytical workloads that re-read large working sets.

You reach them the same way on every cloud — a claim on the omnistrate-local-nvme storage class — but how much you get differs:

Using Local NVMe¶

Ask for it with a PersistentVolumeClaim on the omnistrate-local-nvme storage class:

volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ReadWriteOnce]
      storageClassName: omnistrate-local-nvme
      resources:
        requests:
          storage: 100Gi

That is the whole configuration, and it is the same on every cloud. Omnistrate prepares the disks on the node and decides where the bytes actually live, so the spec does not change when you move between AWS, Azure and GCP.

It works from a Helm chart, a Kustomize overlay, an Operator, or a Compose spec — anywhere you can declare a volume claim. Nothing else is required: no hostPath, no mountPropagation, no init container, no privileged pod, no node selector.

The claim is only scheduled onto nodes that actually have local NVMe, so it cannot quietly end up on a node without it and give you OS-disk performance.

What Omnistrate does for you¶

At node bootstrap it finds the local NVMe devices, ignoring the OS disk and any attached data disks, combines them into a single RAID 0 array when there is more than one, and formats and mounts it. Volumes are then created on that array.

Compose-based resources¶

Compose-based resources get a /cache directory on local NVMe automatically whenever the instance type has it. There is nothing to configure.

Choosing an Instance Type with Local NVMe¶

AWS — families whose name carries a d, plus the storage-optimized families:

• m6id, m7gd, c6id, c7gd, r6id, r7gd and similar d-suffixed variants
• i3, i3en, i4i, im4gn, is4gen

aws ec2 describe-instance-types --instance-types m6id.8xlarge \
  --query 'InstanceTypes[].InstanceStorageInfo'

A non-empty result with "NvmeSupport": "required" means it has local NVMe.

Azure — the v6/v7 d families, the storage-optimized L-series, and several HPC and GPU families:

• Ddsv6, Edsv6, Ddsv7, Edsv7, Eadsv7, Dldsv6, Dpdsv6, Epdsv6, Fadsv7
• Lsv2, Lsv3, Lasv3, Lsv4, Lasv4, Laosv4
• FXmdsv2, HBv4, HX
• NVMe-backed GPU families such as NCadsH100v5 and NCADSA100v4

az vm list-skus --location eastus2 --size Standard_E32ds_v6 -o json \
  | jq -r '.[0].capabilities[] | select(.name | startswith("Nvme"))'

A non-zero NvmeDiskSizeInMiB means it has local NVMe. Divide it by NvmeSizePerDiskInMiB for the number of disks.

GCP: Choosing How Many Local SSDs¶

On GCP, local SSD is an attached resource: for many machine families you choose how many disks to attach. That count is set with configurationOverrides on the instance type. You still consume the result through the same storage class.

There are two fields, for the two ways GCP exposes local SSD.

ephemeralStorageLocalSsdConfig¶

Backs the node's ephemeral storage with local SSD, so emptyDir volumes and container image layers land on it. This is the closest equivalent to the AWS and Azure behavior above.

At least one of the two must be set, and any value given must be greater than zero.

x-omnistrate-compose-spec:
  services:
    warehouse:
      x-omnistrate-compute:
        instanceTypes:
          - name: n2-standard-8
            cloudProvider: gcp
            configurationOverrides:
              ephemeralStorageLocalSsdConfig:
                localSsdCount: 2

localNvmeSsdBlockConfig¶

Attaches local SSDs as raw block devices, left unformatted for the workload to manage itself. Use this when your application wants to own the device directly.

localSsdCount is required and must be greater than zero.

services:
  - name: warehouse
    compute:
      instanceTypes:
        - name: n2-standard-8
          cloudProvider: gcp
          configurationOverrides:
            localNvmeSsdBlockConfig:
              localSsdCount: 1

The number of local SSDs you can attach depends on the machine type and zone. See Google's local SSD documentation for the limits.

These Fields Are GCP-Only¶

Both fields are rejected at spec import for any other cloud provider:

ephemeral storage local SSD configuration is not supported for AWS
local NVMe SSD block configuration is not supported for Azure

This is intentional, not a gap. On AWS and Azure the amount of local NVMe is fixed by the instance type and there is no count to choose — see Using Local NVMe above.

Verifying It Works¶

From inside a pod, check where its volume actually lives:

kubectl exec <pod> -- df -h /path/to/volume

A volume on local NVMe shows a device such as /dev/nvme1n1 or /dev/md/.... If it shows the root device instead (/dev/root on Azure), the instance type most likely has no local NVMe — confirm with the aws or az command in Choosing an Instance Type with Local NVMe.

If a claim stays Pending, check its events:

kubectl describe pvc <name>

waiting for first consumer is normal — the volume is a directory on one specific node, so it is not created until a pod using it has been scheduled.

Related Guides¶

• Compute Management — selecting instance types per cloud provider
• Persistent Volumes — durable storage that survives node replacement
• Storage Classes — pre-provisioned classes for network-attached storage