IP Library Granted Patent US 11,403,021
Granted Patent B2
US 11,403,021 · App. 16/579,349 · Granted Aug 2, 2022

File merging method and controller

Inventors: Bo Ren (Chengdu, CN); Feiteng Huang (Chengdu, CN)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
G06F3/0643G06F3/067G06F3/0622G06F3/0658G06F3/0679G06F16/2246
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,403,021
App. No.
16/579,349
Granted
Aug 2, 2022
Kind
B2
Abstract

A file merging method performed by a controller in a storage system includes reading a first file and a second file on a solid state disk, determining whether a key of the first data is the same as a key of the second data, creating a third file on a mechanical hard disk when the key of the first data is the same as the key of the second data, merging the first data and the second data, and writing the merged data into the third file.

Claims (73)

1. A file merging method applied to a storage system and implemented by a controller, wherein the storage system comprises the controller, a solid state disk, and a mechanical hard disk, and wherein the file merging method comprises:

reading a first file and a second file on the solid state disk, wherein the first file is configured to store first data and a first key of the first data, and wherein the second file is configured to store second data and a second key of the second data;

determining whether the first key is the same as the second key;

creating a third file on the mechanical hard disk when the first key is the same as the second key;

merging the first data and the second data to obtain merged data; and

writing the merged data into the third file,

wherein a cache of the controller is configured to store at least two immutable memtables and one memtable, wherein the first data is from a first immutable memtable of the at least two immutable memtables, wherein the second data is from a second immutable memtable of the at least two immutable memtables, and wherein the at least two immutable memtables and the one memtable are configured to receive data from a host.

2. The file merging method of claim 1 , further comprising:

monitoring whether a quantity of files on the solid state disk reaches a preset threshold; and

reading the first file and the second file on the solid state disk when the quantity of files on the solid state disk reaches the preset threshold.

3. The file merging method of claim 1 , further comprising storing, by a storage class memory of the storage system, a log file, wherein the file merging method further comprises:

receiving a first data writing instruction from the host;

writing the first data writing instruction into the log file, wherein the first data writing instruction comprises the first data and the first key;

receiving a second data writing instruction from the host;

writing the second data writing instruction into the log file, wherein the second data writing instruction comprises the second data and the second key;

playing back the log file to obtain the first data, the first key, the second data, and the second key;

creating the first file on the solid state disk;

writing the first data and the first key into the first file;

creating the second file on the solid state disk; and

writing the second data and the second key into the second file.

4. The file merging method of claim 3 , wherein writing the first data writing instruction into the log file comprises writing the first data writing instruction into the log file through appending.

5. The file merging method of claim 3 , wherein the log file further comprises an additional data writing instruction, and wherein the file merging method further comprises sorting all data writing instructions in the log file based on a key included in each data writing instruction.

6. A controller in a storage system, the controller comprising:

a communications interface configured to communicate with a solid state disk and a mechanical hard disk of the storage system;

a processor coupled to the communications interface and configured to:

read, using the communications interface, a first file and a second file on the solid state disk, wherein the first file is configured to store first data and a first key of the first data, and wherein the second file is configured to store second data and a second key of the second data;

determine whether the first key is the same as the second key;

create a third file on the mechanical hard disk when the first key is the same as the second key;

merge the first data and the second data to obtain merged data; and

write, using the communications interface, the merged data into the third file; and

a cache configured to store at least two immutable memtables and one memtable, wherein the first data is from a first immutable memtable of the at least two immutable memtables, wherein the second data is from a second immutable memtable of the at least two immutable memtables, and wherein the at least two immutable memtables and the one memtable are configured to receive data from a host.

7. The controller of claim 6 , wherein the processor is further configured to:

monitor whether a quantity of files on the solid state disk reaches a preset threshold; and

read, using the communications interface, the first file and the second file on the solid state disk when the quantity of files on the solid state disk reaches the preset threshold.

8. The controller of claim 6 , wherein the storage system further comprises a storage class memory, wherein the storage class memory is configured to store a log file, and wherein the processor is further configured to:

receive, using the communications interface, a first data writing instruction from the host;

write, using the communications interface, the first data writing instruction into the log file, wherein the first data writing instruction comprises the first data and the first key;

receive, using the communications interface, a second data writing instruction from the host;

write, using the communications interface, the second data writing instruction into the log file, wherein the second data writing instruction comprises the second data and the second key;

play back the log file to obtain the first data, the first key, the second data, and the second key;

create the first file on the solid state disk;

write, using the communications interface, the first data and the first key into the first file;

create the second file on the solid state disk; and

write the second data and the second key into the second file.

9. The controller of claim 8 , wherein when writing the first data writing instruction into the log file, the processor is further configured to write, using the communications interface, the first data writing instruction into the log file through appending.

10. The controller of claim 8 , wherein the log file further comprises an additional data writing instruction, and wherein the processor is further configured to sort all data writing instructions in the log file based on a key included in each data writing instruction.

11. The controller of claim 8 , wherein the host is coupled to the controller using a storage area network (SAN).

12. The controller of claim 8 , wherein the storage system is a key-value (KV) storage system, and wherein data storage in the storage system is implemented using a log-structured merge-tree (LSM-Tree).

13. A computer program product comprising computer-executable instructions for storage on a non-volatile computer readable storage medium that, when executed by a controller in a storage system, cause the controller to:

read a first file and a second file on a solid state disk in the storage system, wherein the first file is configured to store first data and a first key of the first data, and wherein the second file is configured to store second data and a second key of the second data;

determine whether the first key is the same as the second key;

create a third file on a mechanical hard disk when the first key is the same as the second key;

merge the first data and the second data to obtain merged data; and

write the merged data into the third file,

wherein a cache of the controller is configured to store at least two immutable memtables and one memtable, wherein the first data is from a first immutable memtable of the at least two immutable memtables, wherein the second data is from a second immutable memtable of the at least two immutable memtables, and wherein the at least two immutable memtables and the one memtable are configured to receive data from a host.

14. The computer program product of claim 13 , wherein the computer-executable instructions further cause the controller to:

monitor whether a quantity of files on the solid state disk reaches a preset threshold; and

read the first file and the second file on the solid state disk when the quantity of files on the solid state disk reaches the preset threshold.

15. The computer program product of claim 13 , wherein the storage system further comprises a storage class memory, wherein the storage class memory is further configured to store a log file, and wherein the computer-executable instructions further cause the controller to:

receive a first data writing instruction from the host;

write the first data writing instruction into the log file, wherein the first data writing instruction comprises the first data and the first key;

receive a second data writing instruction from the host;

write the second data writing instruction into the log file, wherein the second data writing instruction comprises the second data and the second key;

play back the log file to obtain the first data, the first key, the second data, and the second key;

create the first file on the solid state disk;

write the first data and the first key into the first file;

create the second file on the solid state disk; and

write the second data and the second key into the second file.

16. The computer program product of claim 13 , wherein the first file is a first sorted string table (SST) file, and wherein the second file is a second SST file.

17. The computer program product of claim 15 , wherein when writing the first data writing instruction into the log file, the computer-executable instructions further cause the controller to write the first data writing instruction into the log file through appending.

18. The computer program product of claim 15 wherein the log file further comprises an additional data writing instruction, and wherein the computer-executable instructions further cause the controller to sort all data writing instructions in the log file based on a key included in each data writing instruction.

19. The file merging method of claim 1 , wherein the first file is a first sorted string table (SST) file, and wherein the second file is a second SST file.

20. The file merging method of claim 1 , wherein the host is coupled to the controller using a storage area network (SAN).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2019
From: REN, BO; HUANG, FEITENG
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 050778/0736 →
Priority Claims (1)
CN 201710175180.0 · Mar 22, 2017 · national
Continuity (2)
Continuation PCTCN2018071929 · Jan 9, 2018
Related Publication 20200019331A1 · Jan 16, 2020