IP Library › Granted Patent US 11,308,054

Granted Patent B2

US 11,308,054 · App. 16/742,735 · Granted Apr 19, 2022

Efficient large column values storage in columnar databases

Inventors: Fang Zheng (Sunnyvale, CA); Luo Congnan (Sunnyvale, CA); Cheng Zhu (Sunnyvale, CA); Ruiping Li (Sunnyvale, CA); Xiaowei Zhu (Sunnyvale, CA)

G06F16/221 G06F16/2282 G06F16/2379

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,308,054
App. No.: 16/742,735
Granted: Apr 19, 2022
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Efficient large column values storage in columnar databases is disclosed, including: providing a database comprising a write storage, a read storage, and an auxiliary storage, and wherein the auxiliary storage is used to store a large column value of a row that is associated with either the write storage or the read storage, wherein the large column value comprises a column value that meets a set of large column value criteria; and operating the database using the write storage, the read storage, and the auxiliary storage.

Claims (184)

1. A system, comprising:

one or more processors configured to:

provide a database comprising a write storage, a read storage, and an auxiliary storage, and wherein the auxiliary storage is used to store a large column value of a row that is associated with either the write storage or the read storage, wherein the large column value comprises a column value that meets a set of large column value criteria; and

operate the database using the write storage, the read storage, and the auxiliary storage, wherein to operate the database using the write storage, the read storage, and the auxiliary storage includes to:

receive a read request to read the row;

locate the row in the write storage or the read storage;

determine that the row includes a set of large column value associating data;

use the set of large column value associating data to generate a reconstructed column value; and

generate the row based at least in part on the reconstructed column value; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

2. The system of claim 1 , wherein the one or more processors are further configured to:

receive a create-table request to create a new table;

create the new table based at least in part on the create-table request; and

create the auxiliary storage corresponding to the new table.

3. The system of claim 1 , wherein the one or more processors are further configured to:

receive a delete-table request to delete a table;

delete the table based at least in part on the delete-table request; and

delete the auxiliary storage corresponding to the table.

4. The system of claim 1 , wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further comprises to:

receive a request to insert a new row into a table;

determine that a new column value associated with the new row is large;

store the new column value in the auxiliary storage; and

store a second set of large column value associating data corresponding to the new column value in the write storage, wherein the second set of large column value associating data is usable to locate the new column value in the auxiliary storage.

5. The system of claim 4 , wherein to determine that the new column value associated with the new row is large, includes to:

determine that a size associated with the new column value is greater than a large column value threshold size; and

determine that the new column value is associated with a compressibility ratio that is greater than a large column value compressibility ratio threshold.

6. The system of claim 1 , wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive an indication of a merge event;

select one or more rows to merge from the write storage to the read storage;

determine that the one or more rows include a second set of large column value associating data;

determine that a selected column value associated with the second set of large column value associating data is large with respect to the read storage; and

determine that the second set of large column value associating data is to be added to the read storage.

7. The system of claim 1 , wherein the set of large column value associating data comprises a first set of large column value associating data, wherein the reconstructed column value comprises a first reconstructed column value, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive an indication of a merge event;

select one or more rows to merge from the write storage to the read storage;

determine that the one or more rows include a second set of large column value associating data;

determine that a selected column value associated with the second set of large column value associating data is not large with respect to the read storage;

generate a second reconstructed column value using the second set of large column value associating data; and

determine that the second reconstructed column value is to be added to a corresponding column in the read storage.

8. The system of claim 1 , wherein the row comprises a first row, wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive a delete request to delete a second row;

determine that the second row is located in the write storage;

determine that the second row includes a second set of large column value associating data;

use the second set of large column value associating data to locate a corresponding column value in the auxiliary storage;

delete the corresponding column value from the auxiliary storage; and

delete the second row from the write storage.

9. The system of claim 1 , wherein the row comprises a first row, wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive a delete request to delete a second row;

determine that the second row is located in the read storage;

determine that the second row includes a second set of large column value associating data;

set a corresponding flag in a delete bitmap corresponding to the second row;

use the second set of large column value associating data to locate a corresponding column value in the auxiliary storage; and

delete the corresponding column value from the auxiliary storage.

10. The system of claim 1 , wherein the row comprises a first row, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive an update request to update a second row;

determine that the second row is located in the write storage;

determine that the update request includes a selected column value;

determine that the selected column value is large;

store the selected column value in the auxiliary storage;

generate a new set of large column value associating data corresponding to the selected column value; and

store a new row in the write storage using at least any existing column value(s) of the row, the new set of large column value associating data, and any existing set(s) of large column value associating data of the row.

11. The system of claim 1 , wherein the row comprises a first row, wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage further includes to:

receive an update request to update a second row;

determine that the second row is located in the read storage;

determine that the update request includes a selected column value;

determine that the selected column value is large;

store the selected column value in the auxiliary storage;

generate a new set of large column value associating data corresponding to the selected column value;

set a corresponding flag in a delete bitmap corresponding to the second row.

12. The system of claim 1 , wherein the column value comprises a first column value, wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein to operate the database using the write storage, the read storage, and the auxiliary storage, wherein the auxiliary storage comprises a target auxiliary storage, further includes to:

receive an indication of a bulk loading event;

receive one or more input rows to store in the read storage;

determine that the one or more input rows includes a second set of large column value associating data;

determine that a second column value associated with the second set of large column value associating data is greater than a large column value threshold size or the second column value has a compressibility ratio that is greater than a compressibility ratio threshold;

determine that the second set of large column value associating data is to be copied into the read storage; and

copy the second column value associated with the second set of large column value associating data from a source auxiliary storage to the target auxiliary storage.

13. A method, comprising:

providing a database comprising a write storage, a read storage, and an auxiliary storage, and wherein the auxiliary storage is used to store a large column value of a row that is associated with either the write storage or the read storage, wherein the large column value comprises a column value that meets a set of large column value criteria; and

operating the database using the write storage, the read storage, and the auxiliary storage wherein operating the database using the write storage, the read storage, and the auxiliary storage includes:

receiving a read request to read the row;

locating the row in the write storage or the read storage;

determining that the row includes a set of large column value associating data;

using the set of large column value associating data to generate a reconstructed column value; and

generating the row based at least in part on the reconstructed column value.

14. The method of claim 13 , wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein operating the database using the write storage, the read storage, and the auxiliary storage further comprises:

receiving a request to insert a new row into a table;

determining that a new column value associated with the new row is large;

storing the new column value in the auxiliary storage; and

storing a second set of large column value associating data corresponding to the new column value in the write storage, wherein the second set of large column value associating data is usable to locate the new column value in the auxiliary storage.

15. The method of claim 14 , wherein determining that the column value associated with the new row is large, includes:

determining that a size associated with the new column value is greater than a large column value threshold size; and

determining that the new column value is associated with a compressibility ratio that is greater than a large column value compressibility ratio threshold.

16. The method of claim 13 , wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein operating the database using the write storage, the read storage, and the auxiliary storage further includes:

receiving an indication of a merge event;

selecting one or more rows to merge from the write storage to the read storage;

determining that the one or more rows include a second set of large column value associating data;

determining that a selected column value associated with the second set of large column value associating data is large with respect to the read storage; and

determining that the second set of large column value associating data is to be added to the read storage.

17. The method of claim 13 , wherein the row comprises a first row, wherein the set of large column value associating data comprises a first set of large column value associating data, and wherein operating the database using the write storage, the read storage, and the auxiliary storage further includes:

receiving an update request to update a second row;

determining that the second row is located in the read storage;

determining that the update request includes a selected column value;

determining that the selected column value is large;

storing the selected column value in the auxiliary storage;

generating a new set of large column value associating data corresponding to the selected column value;

storing a new row in the write storage using at least any existing column value(s) of the row, the new set of large column value associating data, and any existing set(s) of large column value associating data of the second row; and

setting a corresponding flag in a delete bitmap corresponding to the second row.

18. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

operating the database using the write storage, the read storage, and the auxiliary storage, wherein operating the database using the write storage, the read storage, and the auxiliary storage includes:

receiving a read request to read the row;

locating the row in the write storage or the read storage;

determining that the row includes a set of large column value associating data;

using the set of large column value associating data to generate a reconstructed column value; and

generating the row based at least in part on the reconstructed column value.

19. A system, comprising:

one or more processors configured to:

receive an indication of a merge event;

select one or more rows to merge from the write storage to the read storage;

determine that the one or more rows include a set of large column value associating data;

determine that a selected column value associated with the set of large column value associating data is large with respect to the read storage; and

determine that the set of large column value associating data is to be added to the read storage; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

20. A system, comprising:

one or more processors configured to:

receive an indication of a merge event;

select one or more rows to merge from the write storage to the read storage;

determine that the one or more rows include a set of large column value associating data;

determine that a selected column value associated with the set of large column value associating data is not large with respect to the read storage;

generate a reconstructed column value using the set of large column value associating data; and

determine that the reconstructed column value is to be added to a corresponding column in the read storage; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

21. A system, comprising:

one or more processors configured to:

receive a delete request to delete the row;

determine that the row is located in the write storage;

determine that the row includes a set of large column value associating data;

use the set of large column value associating data to locate a corresponding column value in the auxiliary storage;

delete the corresponding column value from the auxiliary storage; and

delete the row from the write storage; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

22. A system, comprising:

one or more processors configured to:

receive a delete request to delete the row;

determine that the row is located in the read storage;

determine that the row includes a set of large column value associating data;

set a corresponding flag in a delete bitmap corresponding to the row;

use the set of large column value associating data to locate a corresponding column value in the auxiliary storage; and

delete the corresponding column value from the auxiliary storage; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

23. A system, comprising:

one or more processors configured to:

receive an update request to update the row;

determine that the row is located in the write storage;

determine that the update request includes a selected column value;

determine that the selected column value is large;

store the selected column value in the auxiliary storage;

generate a new set of large column value associating data corresponding to the selected column value; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

24. A system, comprising:

one or more processors configured to:

receive an update request to update the row;

determine that the row is located in the read storage;

determine that the update request includes a selected column value;

determine that the selected column value is large;

store the selected column value in the auxiliary storage;

generate a new set of large column value associating data corresponding to the selected column value;

set a corresponding flag in a delete bitmap corresponding to the row; and

one or more memories coupled to the one or more processors and configured to provide instructions to the one or more processors.

Assignments (2)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2026

From: ALIBABA GROUP HOLDING LIMITED

To: CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PRIVATE LIMITED

Reel/Frame 075499/0384 →

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2020

From: ZHENG, FANG; CONGNAN, LUO; ZHU, CHENG; LI, RUIPING; ZHU, XIAOWEI

To: ALIBABA GROUP HOLDING LIMITED

Reel/Frame 052199/0550 →

Continuity (1)

Related Publication 20210216515A1 · Jul 15, 2021