IP Library Granted Patent US 12,639,170
Granted Patent B2
US 12,639,170 · App. 18/788,032 · Granted May 26, 2026

Technique to perform incremental hibernate and resume of bare metal clusters

Inventors: Akshay Pandita (Jammu, IN); Veeral Prabodhchandra Shah (Pune, IN); Vishvendra Singh Rana (Santa Clara, CA)
Assignee: Nutanix, Inc.
G06F11/1461G06F9/5016G06F11/1464
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,639,170
App. No.
18/788,032
Filed
Jul 29, 2024
Granted
May 26, 2026
Kind
B2
Art Unit
2132
USPC
711/161
Abstract

The present disclosure describes techniques used in systems, methods, and in computer program products to perform incremental hibernate and resume of bare metal clusters. A replication process can generate full/incremental snapshots at the cluster level and at a resource level of the processes of the cluster. In the even that a user instructs the cluster to hibernate some or all processes of the cluster, the process can capture an incremental snapshot of the corresponding resources far more quickly than would be required to generate full snapshots for those resources. Likewise, system configuration information can be captured to enable the underlying infrastructure of the cluster to be restored in the future. In some embodiments, the approach provided herein also implements an instant restore feature.

Claims (111)

1 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts, the set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule, wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the configuration of the cluster comprises a configuration of a management process for management of the cluster and configuration of a plurality of nodes that form the cluster;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster comprising:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations, and

generating a new cluster configuration snapshot.

2 . The computer readable medium of claim 1 , wherein the cluster comprises at least a plurality of bare metal nodes that form a storage pool having at least some of the logical volumes.

3 . The computer readable medium of claim 1 , wherein the replication operations are implemented as part of a disaster recovery process.

4 . The computer readable medium of claim 1 , wherein the set of acts further comprise restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot.

5 . The computer readable medium of claim 4 , wherein restoring the plurality of processes to the cluster is implemented using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

6 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts, the set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein only a subset of the processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot;

terminating the plurality of processes in response to the command to hibernate; and

releasing resources that are no longer used by remaining processes on the cluster.

7 . The computer readable medium of claim 6 , wherein executing the asynchronous hibernation operations in response to the command to hibernate further comprises:

rebalancing and consolidating remaining processes on the cluster; and

releasing, after rebalancing and consolidating the remaining processes on the cluster, resources that are no longer used by the remaining processes on the cluster.

8 . A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts, the set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein all processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot; and

releasing all resources of the cluster that were previously used by any of the processes.

9 . The computer readable medium of claim 8 , wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the set of acts further comprise restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

10 . A method comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule, wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the configuration of the cluster comprises a configuration of a management process for management of the cluster and configuration of a plurality of nodes that form the cluster;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster comprising:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations, and

generating a new cluster configuration snapshot.

11 . The method of claim 10 , wherein the cluster comprises at least a plurality of bare metal nodes that form a storage pool having at least some of the logical volumes.

12 . The method of claim 10 , wherein the replication operations are implemented as part of a disaster recovery process.

13 . The method of claim 10 , further comprising: restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot.

14 . The method of claim 13 , wherein restoring the plurality of processes to the cluster is implemented using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

15 . A method comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein only a subset of the processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot;

terminating the plurality of processes in response to the command to hibernate; and

releasing resources that are no longer used by remaining processes on the cluster.

16 . The method of claim 15 , wherein executing the asynchronous hibernation operations in response to the command to hibernate further comprises:

rebalancing and consolidating remaining processes on the cluster; and

releasing, after rebalancing and consolidating the remaining processes on the cluster, resources that are no longer used by the remaining processes on the cluster.

17 . A method comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein all processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises;

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot; and

releasing all resources of the cluster that were previously used by any of the processes.

18 . The method of claim 17 , wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the method further comprises restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

19 . A system comprising:

a storage medium having stored thereon a sequence of instructions; and

a processor that executes the sequence of instructions to cause the processor to perform a set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule, wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the configuration of the cluster comprises a configuration of a management process for management of the cluster and configuration of a plurality of nodes that form the cluster;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster comprising:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations, and

generating a new cluster configuration snapshot.

20 . The system of claim 19 , wherein the cluster comprises at least a plurality of bare metal nodes that form a storage pool having at least some of the logical volumes.

21 . The system of claim 19 , wherein the replication operations are implemented as part of a disaster recovery process.

22 . The system of claim 19 , wherein the set of acts further comprise restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot.

23 . The system of claim 22 , wherein restoring the plurality of processes to the cluster is implemented using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

24 . A system comprising:

a storage medium having stored thereon a sequence of instructions; and

a processor that executes the sequence of instructions to cause the processor to perform a set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein only a subset of the processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises:

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot;

terminating the plurality of processes in response to the command to hibernate; and

releasing resources that are no longer used by remaining processes on the cluster.

25 . The system of claim 24 , wherein executing the asynchronous hibernation operations in response to the command to hibernate further comprises:

rebalancing and consolidating remaining processes on the cluster; and

releasing, after rebalancing and consolidating the remaining processes on the cluster, resources that are no longer used by the remaining processes on the cluster.

26 . A system comprising:

a storage medium having stored thereon a sequence of instructions; and

a processor that executes the sequence of instructions to cause the processor to perform a set of acts comprising:

executing replication operations for a cluster as indicated by a snapshot schedule comprising:

generating snapshots for processes on the cluster based on the snapshot schedule, wherein at least some snapshots comprise incremental snapshots, and the snapshots represent logical volumes of respective processes, and

generating a cluster configuration snapshot based on the snapshot schedule;

receiving a command to hibernate a plurality of processes of the cluster; and

executing asynchronous hibernation operations in response to the command to hibernate a plurality of processes of the cluster, wherein all processes on the cluster are hibernated and executing the asynchronous hibernation operations in response to the command to hibernate comprises;

generating new snapshots for the plurality of processes on the cluster, wherein the new snapshots comprise at least one incremental snapshot that captures changes since a last snapshot was generated by executing scheduled replication operations;

generating a new cluster configuration snapshot; and

releasing all resources of the cluster that were previously used by any of the processes.

27 . The system of claim 26 , wherein the cluster configuration snapshot comprises metadata representing a configuration of the cluster, and the set of acts further comprise restoring the plurality of processes to the cluster using at least the incremental snapshots for the plurality of processes and the new cluster configuration snapshot using an instant restore process comprising a copy-on-read process, copy-on-write process, or background process.

Assignments (2)
SECURITY INTEREST Recorded Feb 13, 2025
From: NUTANIX, INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 070206/0463 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2024
From: PANDITA, AKSHAY; SHAH, VEERAL PRABODHCHANDRA; RANA, VISHVENDRA SINGH
To: NUTANIX, INC.
Reel/Frame 068116/0984 →
Priority Claims (1)
IN 202441006335 · Jan 31, 2024 · national
Continuity (1)
Related Publication 20250245105A1 · Jul 31, 2025
References Cited (43)
US 8549518B1 · Aron et al. · 2013 [cited by applicant]
US 8601473B1 · Aron et al. · 2013 [cited by applicant]
US 8850130B1 · Aron et al. · 2014 [cited by applicant]
US 9772866B1 · Aron et al. · 2017 [cited by applicant]
US 10860363B1 · Gabrielson · 2020 [cited by applicant]
US 20100250499A1 · McAlister · 2010 [cited by examiner]
US 20190391880A1 · Wang · 2019 [cited by examiner]
US 20230070982A1 · McAlister et al. · 2023 [cited by applicant]
CN 110633171A · 2019 [cited by applicant]
WO 2012048037A2 · 2012 [cited by applicant]
Bose, S., et al., “CloudSpider: Combining Replication with Scheduling for Optimizing Live Migration of Virtual Machines across Wide Area Networks,” May 2011, DOI: 10.1109/CCGrid.2011.16. [cited by applicant]
“VPSA Hibernation: Another Powerful Tool to Control Your Enterprise Storage-as-a-Service, and Budget,” zadara, URL: https://www.zadara.com/blog/2014/04/23/vpsa-hibernation-another-powerful-tool-to-control-your-enterpris… [cited by applicant]
Glick, J., et al., “CloudBees CI Disaster Recovery(DR) proof of concept using Velero,” CloudBees, URL: https://www.cloudbees.com/blog/cloudbees-ci-disaster-recovery-dr-proof-of-concept-using-velero, date found via Googl… [cited by applicant]
“Create Amazon EBS snapshots,” AWS, URL: https://docs.aws.amazon.com/ebs/latest/userguide/ebs-creating-snapshot.html, date found via Google as Dec. 15, 2012. [cited by applicant]
“Hibernate your On-Demand Instance or Spot Instance,” AWS, URL: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Hibernate.html, date found via Internet Archive as Dec. 9, 2023. [cited by applicant]
“The Best Disaster Recovery Advice for Failover and Failback Success,” Lightedge, URL: https://lightedge.com/failover-failback-success/, dated Mar. 26, 2019. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Apr. 27, 2023), from https://www.nutanixbible.com/pdf/classic.pdf (Publication date based on updated doc publication date; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (May 21, 2024), from https://www.nutanixbible.com/pdf/classic.pdf (Publication date based on updated doc publication date; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Oct. 15, 2013), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 11, 2014), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 20, 2014), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 7, 2015), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 9, 2015), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Sep. 4, 2015), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 12, 2016), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 9, 2016), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 3, 2017), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 8, 2017), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 3, 2018), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 25, 2018), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 8, 2019), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jul. 9, 2019), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Feb. 3, 2020), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Aug. 1, 2020), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 31, 2021), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Sep. 9, 2022), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Cano, I. et al., “Curator: Self-Managing Storage for Enterprise Clusters”, [cited by applicant]
“Citrix XenDesktop 7.1 on Microsoft Hyper-V Server 2012 R2 on Nutanix Virtual Computing Platform—Solution Design,” Citrix Validated Solutions, Prepared by: Citrix APAC Solutions, dated Jun. 25, 2014. [cited by applicant]
“NeuralMesh by WEKA Architecture Whitepaper,” WEKA, URL: https://www.weka.io/resources/white-paper/wekaio-architectural-whitepaper/#, date found via Google as Jun. 20, 2025. [cited by applicant]
“Veeam Backup & Replication—What's new in v10?,” Veeam, URL: https://www.content.shi.com/SHIcom/ContentAttachmentImages/SharedResources/PDFs/Veeam/veeam-030220-backup-10-0-whats-new.pdf, dated Feb. 14, 2020. [cited by applicant]
“Create an incremental snapshot for managed disks,” Microsoft Learn, URL: https://learn.microsoft.com/en-us/azure/virtual-machines/disks-incremental-snapshots?tabs=azure-cli, date found via Google as Dec. 9, 2024. [cited by applicant]
“Azure Backup and Recovery,” DataFlair, URL: https://data-flair.training/blogs/azure-backup-and-recovery/, date found via Google as May 26, 2022. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Sep. 9, 2025), from https://www.nutanixbible.com/pdf/classic.pdf (Publication date based on updated doc publication date; first publication date unknown). [cited by applicant]