IP Library Granted Patent US 10,997,157
Granted Patent B2
US 10,997,157 · App. 16/944,153 · Granted May 4, 2021

Providing new table metadata

Inventors: Benoit Dageville (San Mateo, CA); Martin Hentschel (Berlin, DE); William Waddington (San Mateo, CA)
Assignee: Snowflake Inc.
G06F16/2365G06F16/00G06F16/2282G06F16/2358G06F16/24552G06F21/602
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,997,157
App. No.
16/944,153
Granted
May 4, 2021
Kind
B2
Abstract

The subject technology stores table metadata for a current version of a table in a current-version set of one or more table-metadata files. The subject technology detects a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files. The subject technology, responsive to detecting the modification to the current version of the table, generates and storing a new-version set of one or more table-metadata files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table

Claims (47)

1. A method comprising:

storing table metadata for a current version of a table in a current-version set of one or more table-metadata files in immutable storage, the immutable storage comprising data storage in which data is not updated in place;

detecting a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files; and

responsive to detecting he modification to the current version of the table, generating and storing a new-version set of one or more table-metadata files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table.

2. The method of claim 1 , wherein the current-version set of one or more table-data files and the new-version set of one or more table-data files are also stored in the immutable storage.

3. The method of claim 1 , further comprising:

storing the current version of the table in the current-version set of one or more table-data files, the table comprising rows and columns.

4. The method of claim 1 , wherein both the table metadata for the current version of the table and the table metadata for the new version of the table are stored in a column-by-column format such that the table metadata corresponding to a particular column of the table is stored within the table metadata as contiguous blocks.

5. The method of claim 1 , further comprising:

storing a hash of the table metadata for a particular column of the table; and

determining that the table metadata has been altered based at least in part on a comparison of the stored hash to a computed hash.

6. The method of claim 5 , wherein storing the hash occurs before decrypting the table metadata.

7. The method of claim 5 , wherein each column of the table metadata is encrypted individually, and a particular hash is stored for each column of the table metadata.

8. The method of claim 1 , wherein the table metadata for the new version of the table indicates at least one new table-data file resulting from the modification, the at least one new table-data file being in the new-version set of one or more table-data files.

9. The method of claim 8 , wherein the table metadata for the new version of the table indicates at least one deleted table-data file resulting from the modification, the at least one deleted table-data file being absent from the new-version set of one or more table-data files after having been in the current-version set of one or more table-data files.

10. A system comprising:

at least one processor; and

a memory device including instructions, which when executed by the at least one processor, cause the at least one processor to perform operations comprising:

storing table metadata. for a current version of a table in a current-version set of one or more table-metadata files in immutable storage, the immutable storage comprising data storage in which data is not updated in place;

detecting a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files; and

responsive to detecting the modification to the current version of the table, generating and storing a new-version set of one or more table-metadata files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table.

11. The system of claim 10 , wherein the current-version set of one or more table-data files and the new-version set of one or more table-data files are also stored in the immutable storage.

12. The system of claim 10 , wherein the operations further comprise:

storing the current version of the table in the current-version set of one or more table-data files, the table comprising rows and columns.

13. The system of claim 10 , wherein both the table metadata for the current version of the table and the table metadata for the new version of the table are stored in a column-by-column format such that the table metadata corresponding to a particular column of the table is stored within the table metadata as contiguous blocks.

14. The system of claim 10 , wherein the operations further comprise:

storing a hash of the table metadata for a particular column of the table; and

determining that the table metadata has been altered based at least in part on a comparison of the stored hash to a computed hash.

15. The system of claim 14 , wherein storing the hash occurs before decrypting the table metadata.

16. The system of claim 14 , wherein each column of the table metadata is encrypted individually, and a particular hash is stored for each column of the table metadata.

17. The system of claim 10 , wherein the table metadata for the new version of the table indicates at least one new table-data file resulting from the modification, the at least one new table-data file being in the new-version set of one or more table-data files.

18. The system of claim 17 , wherein the table metadata for the new version of the table indicates at least one deleted table-data file resulting from the modification, the at least one deleted table-data file being absent from the new-version set of one or more table-data files after having been in the current-version set of one or more table-data files.

19. A non-transitory computer-readable medium comprising instructions, which when executed by at least one processor, cause the at least one processor to perform operations comprising:

storing table metadata for a current version of a table in a current-version set of one or more table-metadata files in immutable storage, the immutable storage comprising data storage in which data is not updated in place;

detecting a modification to the current version of the table, the modification resulting in a new version of the table, the new version of the table being stored in a new-version set of one or more table-data files, the new-version set of one or more table-data files being different than a current-version set of one or more table-data files; and

responsive to detecting the modification to the current version of the table, generating and storing a new-version set of one or more table-metadata. files, the new-version set of one or more table-metadata files comprising table metadata for the new version of the table.

20. The non-transitory computer-readable medium of claim 19 , wherein the current-version set of one or more table-data files and the new-version set of one or more table-data files are also stored in the immutable storage.

21. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:

storing the current version of the table in the current-version set of one or more table-data files, the table comprising rows and columns.

22. The non-transitory computer-readable medium of claim 19 , wherein both the table metadata for the current version of the table and the table metadata for the new version of the table are stored in a column-by-column format such that the table metadata corresponding to a particular column of the table is stored within the table metadata as contiguous blocks.

23. The non-transitory computer-readable medium of claim 19 , wherein the operations further comprise:

storing a hash of the table metadata for a particular column of the table; and

determining that the table metadata has been altered based at least in part on a comparison of the stored hash to a computed hash.

24. The non-transitory computer-readable medium of claim 23 , wherein storing the hash occurs before decrypting the table metadata.

25. The non-transitory computer-readable medium of claim 23 , wherein each column of the table metadata is encrypted individually, and a particular hash is stored for each column of the table metadata.

26. The non-transitory computer-readable medium of claim 19 , wherein the table metadata for the new version of the table indicates at least one new table-data file resulting from the modification, the at least one new table-data file being in the new-version set of one or more table-data files.

27. The non-transitory computer-readable medium of claim 26 , wherein the table metadata for the new version of the table indicates at least one deleted table-data file resulting from the modification, the at least one deleted table-data file being absent from the new-version set of one or more table-data files after having been in the current-version set of one or more table-data files.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 24, 2022
From: DAGEVILLE, BENOIT
To: SNOWFLAKE INC.
Reel/Frame 060300/0419 →
CORRECTIVE ASSIGNMENT TO CORRECT THE EXECUTION DATE TO APRIL 1, 2019, THAT WAS INCORRECTLYRECORDED AS MARCH 14, 2019 PREVIOUSLY RECORDED ON REEL 053562 FRAME 0069. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Aug 11, 2021
From: SNOWFLAKE COMPUTING INC.
To: SNOWFLAKE INC.
Reel/Frame 057319/0068 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: DAGEVILLE, BENOIT; HENTSCHEL, MARTIN; WADDINGTON, WILLIAM
To: SNOWFLAKE COMPUTING INC.
Reel/Frame 053561/0947 →
CHANGE OF NAME Recorded Aug 21, 2020
From: SNOWFLAKE COMPUTING INC.
To: SNOWFLAKE INC.
Reel/Frame 053562/0069 →
Continuity (3)
Continuation 16710414 · Dec 11, 2019
Continuation 15812892 · Nov 14, 2017
Related Publication 20200364208A1 · Nov 19, 2020
Cited By (1)
US 12,450,126