Hyperdisk Storage Pools can be used with Compute Engine for large-scale storage. A Hyperdisk Storage Pool is a pre-purchased collection of capacity, throughput, and IOPS which you can then provision to your applications as needed. You can use Hyperdisk Storage Pools to create and manage disks in pools and use the disks across multiple workloads. By managing disks in aggregate, you can save costs while achieving expected capacity and performance growth. By using only the storage you need in Hyperdisk Storage Pools, you reduce the complexity of forecasting capacity and reduce management toil by going from managing hundreds of disks to managing a single storage pool.
Storage pools include the following benefits:
- Lower Total Cost of Ownership (TCO)—Hyperdisk Storage Pools use thin provisioning and data reduction to help you store your data efficiently and achieve best in-class TCO.
- Higher efficiency—Hyperdisk Storage Pools can take advantage of thin provisioning and data reduction to help you achieve higher resource utilization and lower TCO.
- Less management overhead via Higher Flexibility—Disks in Hyperdisk Storage Pools can be provisioned to larger sizes and only use what they need, freeing workload owners from tedious capacity and performance forecasting, and from downtime related to rescaling.
- Transparent to workloads—There is no change to how individual workloads use Hyperdisk volumes when using storage pools. There is no need for downtime or any other impact to workloads.
Data on individual disks in a storage pool maintain the same isolation they would have if the disks were not created in a storage pool.
When to use storage pools
You can use storage pools to solve these problems:
Difficulty planning the resource requirements when migrating workloads from on-premise workloads that use a SAN to Google Cloud
Estimating the performance and capacity needs of each application is time consuming and error-prone, and can add weeks to a cloud migration or application rollout.
With Hyperdisk Storage Pools, you can create disks in the storage pool using an overestimation of the capacity or performance needed, and then consume from the storage pool only the amount of disk space that you write data to or the IOPS or throughput your workload uses.
Underutilization of resources
Ensuring that your volumes are well utilized can be complex and painfully difficult. Block storage is frequently underutilized because you provision for peak capacity and performance needs to avoid outages or performance slowdowns. However, many applications rarely achieve those peaks. With thin-provisioned capacity and performance, and data reduction, Hyperdisk Storage Pools make it easier to increase your block storage utilization while simplifying your operations.
Complex management of the block storage used by your workloads
Managing hundreds or even thousands of disk volumes is time consuming and takes resources away from delivering new innovation.
When you create disks for your Compute Engine instances in a storage pool, you can specify a size or performance limit that is much larger than you expect the workload to use. As your workloads write to the disks, space and performance resources are taken from the storage pool provisioned capacity and performance. Only the amount of block storage resources that you use is subtracted from the storage pool provisioned capacity and performance, not the amount that you provisioned when creating the disks. The disk size and performance you specify when creating the disks in the storage pool acts as an upper limit on those resources. As a result, you only need to modify the size or provisioned performance for a disk if that limit is reached.
If you create disks in the storage pool, and your workload exceeds your capacity and performance planning across several disks, you can add more capacity or performance to the storage pool. The additional capacity and performance can be used by all the disks created in the storage pool.
Hyperdisk Storage Pool features
Storage pools have the following features:
- Capacity and performance thin provisioning: Capacity and performance are allocated as needed instead of allocating all the resources in advance. This helps to avoid low utilization rates of storage resources, where large amounts of disk space or performance are allocated, but not used.
- Data reduction: Storage pools use a variety of data reduction technologies to increase storage efficiency. Data reduction depends greatly on the type of data stored. Data that is already compressed or encrypted before it is stored in a disk in a Hyperdisk Storage Pool won't yield additional reduction.
How Hyperdisk Storage Pools work
You create a storage pool with the aggregate capacity and performance that your workloads will need, and then create disks in the storage pool. You can then attach the disks to your VMs. When you create the disks, you can create them with a much larger size or provisioned performance limit than is needed. This simplifies planning and provides room for growth later, without necessitating changing the disk's provisioned size or performance at a later date.
If your workloads grow and your disks need more capacity or performance, you can increase the provisioned capacity and performance of the storage pool. The amount of disk space or performance used by the individual disks in the storage pool can then increase up to the limit you specified when you created the disk. By creating the disks in an Advanced capacity or Advanced performance storage pool with a much larger size or performance limit initially, and allocating additional space and performance through the storage pool, you consolidate disk storage management and reduce costs.
If an Advanced capacity storage pool reaches 100% utilization and there is no free space left in the storage pool, writes to all disks in the storage pool fail until you reduce the used space in the storage pool by deleting data or disks. Most commercial software interprets the errors returned when the pool space is exhausted in a manner similar to hardware failures.
It's important that you actively manage and monitor your storage pool to avoid out of space errors. You should also understand how your workload will respond if it encounters an out of space error for a disk in a storage pool.
When the aggregate performance utilization of all the disks in the storage pool reaches the total amount of performance provisioned for the storage pool, the disks can encounter performance contention. If contention for performance resources is detected in an Advanced performance storage pool for any disks in the pools, then the auto-grow feature attempts to automatically increase the IOPS available to disks in the storage pool to prevent performance issues.
Provisioning types for Hyperdisk Storage Pools
When creating a Hyperdisk Storage Pool, you can configure it with either standard or advanced provisioning for capacity and performance.
Standard capacity storage pools
With Standard capacity provisioning, you create disks in the storage pool until the total provisioned capacity of all disks in the storage pool reaches the storage pool's provisioned capacity. Disks in a Standard capacity storage pool consume capacity similarly to non-storage pool disks.
Advanced capacity storage pools
Disks in an Advanced capacity storage pool consume capacity differently than disks in a Standard capacity storage pool and non-storage pool disks. Advanced capacity storage pools offer thin-provisioning and data reduction for capacity, enabling you to provision disks with more capacity than you have purchased. Disks in an Advanced capacity storage pools consume capacity based only on the number of bytes written to your disks after data reduction, which means you can provision more capacity to your end users and applications than you have purchased in your storage pool.
With Advanced capacity, you create disks in the storage pool where the cumulative size of all the disks can exceed the provisioned capacity of the storage pool by up to 500%. The used capacity of the storage pool is defined by the amount of data written and not by the amount of provisioned disk capacity.
You can fill disks in an Advanced capacity storage pool up to their provisioned size as long as the data written to all disks in the storage pool doesn't exceed the storage pool capacity. If the storage pool utilization reaches 80% of the pool provisioned capacity, the auto-grow feature attempts to automatically add capacity to the storage pool. If the storage pool's capacity is fully used, then writes to all storage pool disks will fail until you delete data or delete disks to lower the used capacity of the storage pool. Most software applications interpret the errors returned when the pool space is exhausted in a manner similar to hardware failures, so it's important to both:
- Monitor your storage pool to avoid running out of disk space
- Understand how your workload responds if it does happen
storage pools don't have visibility into your file system, so deleted data
is considered to still be in use until your operating system (OS)
marks it unused with a DISCARD or TRIM command. All Google-provided OS
images are configured to do this by default, as are most common third-party OS
images, but you should confirm this if you are not using a Google-provided OS
image. More information on verifying or configuring this feature is
available at
Disable lazy initialization and enable DISCARD commands.
Standard performance storage pools
Standard performance provisioning is the best option for the following types of workloads:
- Workloads that can't succeed if performance is limited by storage pool resources
- Workloads where the disks in the storage pool are likely to have correlated performance spikes, for example, data disks for databases that are at peak utilization every morning.
With Standard performance provisioning, thin-provisioning isn't available. Also, disks created in a Standard performance storage pool don't share performance resources with the rest of the storage pool. The aggregate amount of performance of all disks created in the storage pool can't exceed the total provisioned IOPS or throughput of the storage pool.
When creating the storage pool, you provision enough performance in the storage pool to cover the combined peak performance needs of all the disks in the storage pool. When you create disks in the storage pool, you provision an amount of performance that covers the peak performance requirements of that disk. You can continue creating disks in the storage pool until the total provisioned IOPS of all disks reaches the storage pool's provisioned amount.
In a Hyperdisk Balanced Storage Pool with Standard performance, the first 3,000 IOPS and 140 MiBps of throughput of each disk in the storage pool (the baseline) don't consume storage pool resources. When you create disks in the storage pool, any IOPS and throughput in excess of the baseline values consume IOPS and throughput from the storage pool.
Advanced performance storage pools
Advanced performance provisioning is best suited for workloads that don't have highly correlated peak usage times. If your workloads all peak at the same time, the Advanced performance storage pool can reach the performance limits of the storage pool, resulting in contention for performance resources.
Advanced performance IOPS and throughput provisioning lets you share a pool of provisioned performance across all disks in a storage pool. The storage pool dynamically allocates performance resources as the disks in the storage pool read and write data. Only the amount of IOPS and throughput used by a disk in the storage pool consumes storage pool performance.
Because Advanced performance storage pools are thinly provisioned, you can allocate more IOPS or throughput to the disks in the storage pool than you have provisioned for the storage pool—up to 500% of the IOPS or throughput provisioned for the storage pool. For example, if you provision 100,000 IOPS for a storage pool, you could have 10 disks in the storage pool, each provisioned with 50,000 IOPS.
In a Hyperdisk Balanced Storage Pool with Advanced performance provisioning, the disks don't have baseline performance. Every read and write operation of a Hyperdisk Balanced disk in the storage pool consumes provisioned storage pool resources.
At any given moment, when the aggregate performance used by all the disks in the storage pool reaches the total amount of performance provisioned for the storage pool, the disks can contend with each other for performance resources. As a result, the disks are not able to achieve the maximum performance levels that you provisioned for the disks. When contention for resources is detected for a prolonged period of time, Compute Engine attempts to automatically add performance to the storage pool. When using Advanced performance storage pools, you should:
- Monitor your storage pools to avoid running out of provisioned performance
- Understand how your workload responds if it does happen
Example
Assume that you have a Hyperdisk Balanced Storage Pool with 100,000 provisioned IOPS.
With Standard performance provisioning:
- You can provision up to 100,000 of aggregate IOPS when creating Hyperdisk Balanced disks in the storage pool.
- You are charged for the 100,000 IOPS of Hyperdisk Balanced Storage Pool provisioned performance.
Like disks created outside of a storage pool, Hyperdisk Balanced disks in Standard performance storage pools are automatically provisioned with up to 3,000 baseline IOPS and 140 MiB/s of baseline throughput. This baseline performance isn't counted against the provisioned performance for the storage pool. Only when you add disks to the storage pool with provisioned performance that's above the baseline does it count against the provisioned performance for the storage pool, for example:
- A disk provisioned with 3,000 IOPS uses 0 pool IOPS and the pool still has 100,000 provisioned IOPS available for other disks.
- A disk provisioned with 13,000 IOPS uses 10,000 pool IOPS and the pool has 90,000 provisioned IOPS remaining that you can allocate to other disks in the storage pool.
With Advanced performance provisioning:
- You can provision up to 500,000 IOPS of aggregate Hyperdisk performance when creating disks in the storage pool.
- You are charged for 100,000 IOPS provisioned by the storage pool.
- If you create a single disk (
Disk1) in the storage pool that has 5,000 IOPS, you don't consume any IOPS from the storage pool provisioned IOPS. However, the amount of IOPS that you can provision to new disks created in the storage pool is now 495,000. - If
Disk1starts to read and write data, and if it uses its maximum of 5,000 IOPS in a given minute, then 5,000 IOPS is consumed from the storage pool provisioned IOPS. Any other disks that you created in the same storage pool can use an aggregated maximum of 95,000 IOPS in that same minute without running into contention.
Types of Hyperdisk Storage Pools
The type of Hyperdisk Storage Pool that you create determines the type of disks that you can create in the storage pool.
- Hyperdisk Throughput Storage Pool: When creating the storage pool, you specify the capacity and throughput to provision for the storage pool. Each Hyperdisk Throughput disk you create in the storage pool uses some of the provisioned capacity and throughput.
- Hyperdisk Balanced Storage Pool: When creating the storage pool, you specify the capacity, throughput, and IOPS to provision for the storage pool. Each Hyperdisk Balanced disk that you create in the storage pool with provisioned capacity and performance above the baseline values uses some of the storage pool provisioned capacity and performance.
Hyperdisk Throughput Storage Pools
You can use Hyperdisk Throughput Storage Pools to manage your Hyperdisk Throughput disk usage.
Machine type support
Hyperdisk Throughput Storage Pools are supported with the same machine series that support Hyperdisk Throughput. For a list of the supported machine series, see Hyperdisk machine type support.
Hyperdisk Throughput Storage Pool regional availability
Hyperdisk Throughput Storage Pools can be used in any zone that offers Hyperdisk Throughput disks. For a list of the available regions, see Hyperdisk regional availability.
You can also check for the latest updates to the available regions and zones by using the following command:
gcloud compute storage-pool-types list --filter="name=hyperdisk-throughput"
Hyperdisk Balanced Storage Pools
You can use Hyperdisk Balanced Storage Pools to manage your Hyperdisk Balanced disk usage.
In Hyperdisk Balanced Storage Pools with Standard performance provisioning, Hyperdisk Balanced disks only consume IOPS and throughput in excess of the baseline values of 3,000 IOPS and 140 MiBps throughput per disk. For example:
- If you provision a Hyperdisk Balanced disk with 3,000 IOPS and 140 MiBps of throughput in a storage pool with Standard performance provisioning, the disk doesn't consume any IOPS or throughput from the storage pool.
- If you provision a Hyperdisk Balanced disk with 4,000 IOPS and 180 MiBps of throughput in a storage pool with Standard performance provisioning, the disk consumes 1,000 IOPS and 40 MiBps of throughput from the storage pool.
Machine type support
Hyperdisk Balanced Storage Pools are supported with the same machine series that support Hyperdisk Balanced. For a list of the supported machine series, see Hyperdisk machine type support.
Hyperdisk Balanced Storage Pool regional availability
Hyperdisk Balanced Storage Pools can be used in any zone that offers Hyperdisk Balanced disks. For a list of the available regions, see Hyperdisk regional availability.
You can also check for the latest updates to the available regions and zones by using the following command:
gcloud compute storage-pool-types list --filter="name=hyperdisk-balanced"
Limits for storage pools
The following tables list the limits for the different storage pool types.
Hyperdisk Balanced Storage Pool
| Limit | Value |
|---|---|
| Maximum storage pool provisioned capacity | 1 PiB |
| Minimum storage pool provisioned capacity | 10 TiB |
| Storage capacity increments | multiples of 1 TiB |
| Maximum number of Hyperdisk Storage Pools per zone | 10 |
| Maximum number of disks in a Hyperdisk Storage Pool | 1,000 |
| Maximum IOPS per storage pool | 4,194,304 |
| Maximum IOPS per GiB | 4 |
| Minimum IOPS per storage pool |
|
| IOPS increments | multiples of 10,000 |
| Maximum Throughput per storage pool | 1 TiBps |
| Maximum Throughput per GiB of capacity | 1 MiBps |
| Minimum Throughput per storage pool | 1 GiBps |
| Throughput increments | multiples of 1 GiBps |
| Frequency of storage pool capacity changes | Twice in a 24 hour period |
| Frequency of storage pool performance changes | Twice in a 24 hour period |
Hyperdisk Throughput Storage Pool
| Limit | Value |
|---|---|
| Maximum storage pool provisioned capacity | 1,024 TiB (1 PiB) |
| Minimum storage pool provisioned capacity | 10 TiB |
| Storage capacity increments | multiples of 1 TiB |
| Maximum number of Hyperdisk Storage Pools per project | 10 |
| Maximum number of disks in a Hyperdisk Storage Pool | 1,000 |
| Maximum Throughput per storage pool | 18 GiBps |
| Maximum Throughput per TiB of capacity | 18 MiBps |
| Minimum Throughput per storage pool | 100 MiBps |
| Minimum throughput per TiB of capacity | 10 MiBps |
| Throughput increments | multiples of 10 MiBps |
| Frequency of storage pool capacity changes | Twice in a 24 hour period |
| Frequency of storage pool performance changes | Twice in a 24 hour period |
Performance
Performance for disks in a storage pool is the same as for disks that were not created in a storage pool.
Limitations of storage pools
Hyperdisk Storage Pools have the following limitations:
Resource limits:
- You can create a Hyperdisk Storage Pool with up to 1 PiB of provisioned capacity.
- You can create a maximum of 5 storage pools per hour.
- You can create a maximum of 10 storage pools per day.
- You can create at most 10 storage pools per project.
- You can't change the provisioning model for a pool; you can't change a Standard capacity storage pool to an Advanced capacity storage pool or an Advanced performance storage pool to a Standard performance storage pool.
- Storage pools are a zonal resource.
- You can create up to 1,000 disks in a storage pool.
- You can use Hyperdisk Storage Pools with only Compute Engine. Cloud SQL instances cannot use Hyperdisk Storage Pools.
- You can change the provisioned capacity or performance of a storage pool at most two times in a 24 hour period.
Limits for disks in a storage pool:
- Only new disks in the same project and zone can be created in a storage pool.
- Moving disks in or out of a storage pool is not permitted. To move a disk in or out of a storage pool, you have to recreate the disk from a snapshot. For more information, see Change the disk type.
- To create boot disks in a storage pool, you must use a Hyperdisk Balanced Storage Pool.
- Storage pools don't support regional disks.
- You can't clone, create instant snapshots of, or configure Persistent Disk Asynchronous Replication for disks in a storage pool.
- Hyperdisk Balanced disks in a storage pool can't be attached to multiple compute instances.
Reservations
You can use Hyperdisk Storage Pool with reservations. You can create a compute instance that consumes a reservation and also uses a Hyperdisk in a storage pool. Alternatively, you can create the instance that consumes a reservation, and afterwards attach a Hyperdisk that was created in a storage pool to the instance. For more information, see how to consume reservations.
Pricing
Hyperdisk Storage Pools are billed monthly for the provisioned storage pool capacity, throughput and IOPS. You are not billed for the provisioned IOPS, throughput, or capacity for the disks created in the storage pool.
With a Standard capacity and Standard performance storage pools, capacity and provisioned performance is priced at the same rate as the underlying disk. For example, the price for Standard capacity for a Hyperdisk Balanced Storage Pool is the same price as standalone Hyperdisk Balanced disk capacity.
Advanced capacity and Advanced performance storage pools are priced higher because of thin-provisioning and data reduction. Even with this premium, the savings from thin-provisioning and data reduction can still reduce your total block storage costs through increased efficiency and utilization.
For more pricing information, see Disk pricing.
Committed use discounts with Hyperdisk Storage Pools
Hyperdisk Storage Pools are not eligible for:
- Resource-based committed use discounts (CUDs)
- Sustained use discounts (SUDs)
What's next?
- Learn how to create a Hyperdisk Storage Pool.
- Learn how to move existing disks to a storage pool, as described in Change the disk type.
- Review Disk pricing information.