Granted Patent B2

US 12,468,673 · App. 18/499,762 · Granted Nov 11, 2025

Inventors: Cristian Diaconu (Kirkland, WA); Chen Luo (San Mateo, CA); Corbin McElhanney (San Mateo, CA); Wumengjian Zhu (Cupertino, CA)

Assignee: Snowflake Inc.

G06F16/213G06F16/24552G06F16/24573

Abstract

Claims (86)

1 . A system comprising:

at least one hardware processor; and

a memory storing instructions that cause the at least one hardware processor to perform operations comprising:

receiving a request to perform a table scan operation of a table;

determining that the table is being accessed for an initial time;

populating a columnar cache with data of the table provided by the table scan operation;

determining a set of schema versions of a set of rows from the data of the table;

determining, using at least a schema cache, schema information of each schema from the set of schema versions, the schema cache being provided by an execution node;

generating a result rowset and a second rowset comprising a union of columns that have appeared at least once in each row;

performing deserialization of rows from the result rowset and the second rowset;

providing the rows from the result rowset and the second rowset to write to a file in a particular format for storing in the columnar cache;

generating a particular schema version based on a union of columns that have appeared at least once in a set of rows, the particular schema version being utilized as part of a write operation, the write operation comprising writing data from at least one blob file to a particular file in a different format for storing in the columnar cache; and

storing the particular schema version in the schema cache provided by the execution node.

2 . The system of claim 1 , wherein the operations further comprise:

receiving a query including a statement to perform a read operation on a particular table;

determining a set of column ordinals referenced by the query; and

performing a scan of the columnar cache based on the set of column ordinals to determine at least one requested column.

3 . The system of claim 2 , wherein the operations further comprise:

determining that the columnar cache does not store the at least one requested column;

filling the at least one requested column with a current default value; and

providing the at least one requested column with the current default value in a particular result rowset from executing the query.

4 . The system of claim 2 , wherein the operations further comprise:

determining that the at least one requested column is defined as not storing a null value;

determining that the columnar cache does not store the at least one requested column; and

providing an indication of an error as a result of the query.

5 . The system of claim 2 , wherein the operations further comprise:

determining that the at least one requested column is defined as not storing a null value;

determining that the columnar cache includes a null value stored in the at least one requested column; and

providing an indication of an error as a result of the query.

6 . The system of claim 1 , wherein the operations further comprise:

receiving a particular query, the particular query including a query range for processing the query;

sending a request to a key-value store for blob metadata and a set of recent writes for the query range;

receiving the blob metadata, the blob metadata including information related to at least one blob file; and

determining that the at least one blob file includes a set of rows with multiple schema versions.

7 . The system of claim 1 , wherein the particular format comprises a Parquet file.

8 . The system of claim 1 , wherein the columnar cache is stored locally in an execution node, the execution node executing the table scan operation of the table, the table scan operation being performed in connection with a particular query received by the execution node for execution.

9 . The system of claim 1 , wherein the operations further comprise:

determining, by querying a metadata database, the schema information of each schema from the set of schema versions.

10 . The system of claim 1 , wherein the second rowset includes a set of dropped columns based on previous schema information of at least one previous schema version.

11 . A method comprising:

receiving a request to perform a table scan operation of a table;

determining that the table is being accessed for an initial time;

populating a columnar cache with data of the table provided by the table scan operation;

determining a set of schema versions of a set of rows from the data of the table;

determining, using at least a schema cache, schema information of each schema from the set of schema versions, the schema cache being provided by an execution node;

generating a result rowset and a second rowset comprising a union of columns that have appeared at least once in each row;

performing deserialization of rows from the result rowset and the second rowset;

providing the rows from the result rowset and the second rowset to write to a file in a particular format for storing in the columnar cache;

generating a particular schema version based on a union of columns that have appeared at least once in a set of rows, the particular schema version being utilized as part of a write operation, the write operation comprising writing data from at least one blob file to a particular file in a different format for storing in the columnar cache; and

storing the particular schema version in the schema cache provided by the execution node.

12 . The method of claim 11 , further comprising:

receiving a query including a statement to perform a read operation on a particular table;

determining a set of column ordinals referenced by the query; and

performing a scan of the columnar cache based on the set of column ordinals to determine at least one requested column.

13 . The method of claim 12 , further comprising:

determining that the columnar cache does not store the at least one requested column;

filling the at least one requested column with a current default value; and

providing the at least one requested column with the current default value in a particular result rowset from executing the query.

14 . The method of claim 12 , further comprising:

determining that the at least one requested column is defined as not storing a null value;

determining that the columnar cache does not store the at least one requested column; and

providing an indication of an error as a result of the query.

15 . The method of claim 12 , further comprising:

determining that the at least one requested column is defined as not storing a null value;

determining that the columnar cache includes a null value stored in the at least one requested column; and

providing an indication of an error as a result of the query.

16 . The method of claim 11 , further comprising:

receiving a particular query, the particular query including a query range for processing the query;

sending a request to a key-value store for blob metadata and a set of recent writes for the query range;

receiving the blob metadata, the blob metadata including information related to at least one blob file; and

determining that the at least one blob file includes a set of rows with multiple schema versions.

17 . The method of claim 11 , wherein the particular format comprises a Parquet file.

18 . The method of claim 11 , wherein the columnar cache is stored locally in an execution node, the execution node executing the table scan operation of the table, the table scan operation being performed in connection with a particular query received by the execution node for execution.

19 . The method of claim 11 , further comprising:

determining, by querying a metadata database, the schema information of each schema from the set of schema versions.

20 . A non-transitory computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:

receiving a request to perform a table scan operation of a table;

determining that the table is being accessed for an initial time;

populating a columnar cache with data of the table provided by the table scan operation;

determining a set of schema versions of a set of rows from the data of the table;

determining, using at least a schema cache, schema information of each schema from the set of schema versions, the schema cache being provided by an execution node;

generating a result rowset and a second rowset comprising a union of columns that have appeared at least once in each row;

performing deserialization of rows from the result rowset and the second rowset;

providing the rows from the result rowset and the second rowset to write to a file in a particular format for storing in the columnar cache;

generating a particular schema version based on a union of columns that have appeared at least once in a set of rows, the particular schema version being utilized as part of a write operation, the write operation comprising writing data from at least one blob file to a particular file in a different format for storing in the columnar cache; and

storing the particular schema version in the schema cache provided by the execution node.

Assignments (1)

Continuity (2)

References Cited (19)