IP Library Granted Patent US 10,789,210
Granted Patent B2
US 10,789,210 · App. 15/712,280 · Granted Sep 29, 2020

Method, apparatus, and system for data deduplication

Inventors: Bob Yan (Chengdu, CN); Bernie Hu (Chengdu, CN); Rui Liao (Chengdu, CN); Vincent Wu (Chengdu, CN); Jia Huang (Chengdu, CN)
Assignee: EMC IP Holding Company LLC
G06F16/1748G06F3/061G06F3/067G06F3/0641G06F3/0652G06F3/0659
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,789,210
App. No.
15/712,280
Granted
Sep 29, 2020
Kind
B2
Abstract

The present disclosure provides a method, apparatus and system for data deduplication comprising setting, for a to-be-deduplicated data block, a pointer pointed to the to-be-deduplicated data block as a pointer pointed to a corresponding to-be-reserved data block, and setting a virtual pointer pointed to the to-be-deduplicated data block, cancelling, in response to an available storage capacity reaching a predetermined value, virtual pointers of respective to-be-deduplicated data blocks to be reclaimed, and reclaiming storage spaces of the respective to-be-deduplicated data blocks to be reclaimed, wherein when the virtual pointer pointed to the to-be-deduplicated data block is available, the to-be-deduplicated data block is directly accessed through the virtual pointer.

Claims (41)

1. A method for data deduplication, comprising:

setting, for a to-be-deduplicated data block, a pointer that previously pointed to the to-be-deduplicated data block to instead point to a corresponding to-be-reserved data block, the to-be-deduplicated data block and the to-be-reserved data block both containing identical data;

setting a virtual pointer to point to the to-be-deduplicated data block while the pointer continues to point to the to-be-reserved data block;

cancelling, in response to an available storage capacity reaching a predetermined value, virtual pointers of respective to-be-deduplicated data blocks to be reclaimed, and reclaiming storage spaces of the respective to-be-deduplicated data blocks to be reclaimed,

wherein when the virtual pointer pointing to the to-be-deduplicated data block is available, the to-be-deduplicated data block is accessed through the virtual pointer.

2. The method according to claim 1 , further comprising:

storing the virtual pointer pointing to the to-be-deduplicated data block by adopting, in a dual-pointer structure, a structure similar to that of the pointer set as the pointer pointing to the corresponding to-be-reserved data block.

3. The method according to claim 1 , further comprising:

setting, for the virtual pointer pointing to the to-be-deduplicated data block, a virtual pointer indication bit that indicates whether the virtual pointer is available.

4. The method according to claim 3 , wherein the cancelling virtual pointers of respective to-be-deduplicated data blocks to be reclaimed comprises: setting the virtual pointer indication bit as a value indicating that the virtual pointer is unavailable.

5. The method according to claim 1 , further comprising:

setting, for a pointer pointing to a respective data block, a sharing count bit indicating the number of other data blocks sharing the pointer,

wherein if a value of the sharing count bit is greater than 0 and the virtual pointer is unavailable, a new storage resource is allocated for a write operation on a data block when the write operation is performed, and the pointer of the data block on which the write operation is performed is set to be pointed to the data block stored with the newly allocated storage resource.

6. The method according to claim 5 , wherein when allocating the new storage resource to the write operation, one of the following operations is performed if there is no available free storage resource:

reclaiming continuous data blocks with a relatively long length among reclaimable resources, so as to be allocated to the write operation; and

reclaiming continuous data blocks with a length better matching the write operation among the reclaimable resources, so as to be allocated to the write operation.

7. The method according to claim 1 , further comprising:

determining all reclaimable to-be-deduplicated data blocks, and

determining a part of all reclaimable to-be-deduplicated data blocks as the to-be-deduplicated data blocks to be reclaimed.

8. The method according to claim 7 , wherein the part of all to-be-deduplicated data blocks includes continuous data blocks in all of the to-be-deduplicated data blocks.

9. The method according to claim 1 wherein:

setting the pointer that previously pointed to the to-be-deduplicated data block to instead point to the corresponding to-be-reserved data block includes setting, within a mapping module for a particular address, a first pointer that previously pointed to the to-be-deduplicated data block to instead point to the to-be-reserved data block; and

setting the virtual pointer to point to the to-be-deduplicated data block includes setting, within the mapping module for the particular address, a second pointer to point to the to-be-deduplicated data block while the first pointer continues to point to the to-be-reserved data block.

10. A computer program product comprising a non-transitory computer-readable medium storing instructions, which, when executed by a computer, cause the computer to perform a data deduplication process comprising:

setting, for a to-be-deduplicated data block, a pointer that previously pointed to the to-be-deduplicated data block to instead point to a corresponding to-be-reserved data block, the to-be-deduplicated data block and the to-be-reserved data block both containing identical data;

setting a virtual pointer to point to the to-be-deduplicated data block while the pointer continues to point to the to-be-reserved data block;

cancelling, in response to an available storage capacity reaching a predetermined value, virtual pointers of respective to-be-deduplicated data blocks to be reclaimed, and reclaiming storage spaces of the respective to-be-deduplicated data blocks to be reclaimed,

wherein when the virtual pointer pointing to the to-be-deduplicated data block is available, the to-be-deduplicated data block is directly accessed through the virtual pointer.

11. The computer program product according to claim 10 , wherein the instructions, when executed by the computer, further cause the computer to: store the virtual pointer pointing to the to-be-deduplicated data block by adopting, in a dual-pointer structure, a structure similar to that of the pointer set as the pointer pointing to the corresponding to-be-reserved data block.

12. The computer program product according to claim 10 , wherein the instructions, when executed by the computer, further cause the computer to set, for the virtual pointer pointing to the to-be-deduplicated data block, a virtual pointer indication bit that indicates whether the virtual pointer is available.

13. The computer program product according to claim 12 , wherein the cancelling virtual pointers of respective to-be-deduplicated data blocks to be reclaimed includes setting the virtual pointer indication bit as a value indicating that the virtual pointer is unavailable.

14. The computer program product according to claim 10 , wherein the instructions, when executed by the computer, further cause the computer to:

set, for a pointer pointing to a respective data block, a sharing count bit indicating the number of other data blocks sharing the pointer,

wherein if a value of the sharing count bit is greater than 0 and the virtual pointer is unavailable, a new storage resource is allocated, when performing a write operation on a data block, for the write operation, and the pointer of the data block for the write operation is set to be pointed to the data block stored with the newly allocated storage resource.

15. The computer program product according to claim 14 , wherein when allocating the new storage resource to the write operation, one of the following operations is performed if there is no available free storage resource:

reclaiming continuous data blocks with a relatively long length among reclaimable resources, so as to be allocated to the write operation; and

reclaiming continuous data blocks with a length better matching the write operation among the reclaimable resources, so as to be allocated to the write operation.

16. The computer program product according to claim 10 , wherein the instructions, when executed by the computer, further cause the computer to:

determine all reclaimable to-be-deduplicated data blocks, and

determine a part of all reclaimable to-be-deduplicated data blocks as the to-be-deduplicated data blocks to be reclaimed.

17. The computer program product according to claim 16 , wherein the part of all to-be-deduplicated data blocks includes continuous data blocks in all of the to-be-deduplicated data blocks.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (044535/0109) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO WYSE TECHNOLOGY L.L.C.)
Reel/Frame 060753/0414 →
RELEASE OF SECURITY INTEREST AT REEL 044535 FRAME 0001 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058298/0475 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 044535/0109 →
PATENT SECURITY AGREEMENT (CREDIT) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 044535/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2017
From: YAN, BOB; HU, BERNIE; LIAO, RUI; WU, VICENT; HUANG, JIA
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 044021/0073 →