IP Library › Granted Patent US 12,248,448
Granted Patent B1
US 12,248,448 · App. 18/451,522 · Granted Mar 11, 2025

Configuring check constraint and row violation logging using error tables

Inventors: Raja Suresh Krishna Balakrishnan (Fremont, CA); Ganeshan Ramachandran Iyer (Redmond, WA); David Schultz (Piedmont, CA); Jian Xu (San Jose, CA)
Assignee: Snowflake Inc.
G06F16/215G06F11/0793G06F16/2365G06F16/24545
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,248,448
App. No.
18/451,522
Granted
Mar 11, 2025
Kind
B1
Abstract

Provided herein are systems and methods for configuring integrity constraints (including a check constraint) and row violation logging using error tables. An example method includes decoding a query received at a network-based database system. The query includes a command to perform an operation on a base table. An integrity constraint associated with the base table is retrieved. The integrity constraint specifies a desired configuration for the base table. A verification of the integrity constraint is performed to detect erroneous data of the base table that violates the desired configuration. The erroneous data is input into an error table that is configured as a nested object of the base table. A notification that the erroneous data is available in the error table is generated and output.

Claims (77)

1. A system comprising:

at least one hardware processor; and

at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:

decoding a query received at a network-based database system, the query including a command to perform an operation on a base table;

retrieving an integrity constraint associated with the base table, the integrity constraint specifying a desired configuration for the base table;

performing a verification of the integrity constraint to detect erroneous data of the base table, the erroneous data violating the desired configuration;

inputting the erroneous data into an error table, the error table being configured as a dynamic table maintained as a nested object of the base table; and

generating a notification that the erroneous data is available in the error table.

2. The system of claim 1 , wherein the integrity constraint comprises a check constraint and the desired configuration comprises at least one condition to be met by each row of a plurality of rows stored within the base table.

3. The system of claim 2 , the operations for performing the verification further comprising:

parsing each row of the plurality of rows to determine whether the row satisfies the at least one condition; and

designating the row as the erroneous data when the row does not satisfy the at least one condition.

4. The system of claim 1 , the operations further comprising:

decoding a configuration message received from an account of a user of the network-based database system, the configuration message including the desired configuration of the integrity constraint.

5. The system of claim 4 , the operations further comprising:

determining a Boolean expression based on the configuration message.

6. The system of claim 5 , the operations further comprising:

configuring the integrity constraint as a check constraint, the check constraint using the Boolean expression.

7. The system of claim 4 , the operations further comprising:

determining one or more remediation actions based on the configuration message, wherein a first remediation action of the one or more remediation actions comprises the inputting of the erroneous data into the error table.

8. The system of claim 7 , the operations further comprising:

determining a second remediation action of the one or more remediation actions comprises canceling the operation on the base table.

9. The system of claim 1 , the operations further comprising:

removing the erroneous data from the base table; and

executing the operation on remaining data in the base table.

10. The system of claim 1 , the operations further comprising:

updating the error table to include an identification of the integrity constraint and an expression of the desired configuration associated with the integrity constraint.

11. A method comprising:

decoding, by at least one hardware processor, a query received at a network-based database system, the query including a command to perform an operation on a base table;

retrieving an integrity constraint associated with the base table, the integrity constraint specifying a desired configuration for the base table;

performing a verification of the integrity constraint to detect erroneous data of the base table, the erroneous data violating the desired configuration;

inputting the erroneous data into an error table, the error table being configured as a dynamic table maintained as a nested object of the base table; and

generating a notification that the erroneous data is available in the error table.

12. The method of claim 11 , wherein the integrity constraint comprises a check constraint and the desired configuration comprises at least one condition to be met by each row of a plurality of rows stored within the base table.

13. The method of claim 12 , wherein the performing of the verification further comprises:

parsing each row of the plurality of rows to determine whether the row satisfies the at least one condition; and

designating the row as the erroneous data when the row does not satisfy the at least one condition.

14. The method of claim 11 , further comprising:

decoding a configuration message received from an account of a user of the network-based database system, the configuration message including the desired configuration of the integrity constraint.

15. The method of claim 14 , further comprising:

determining a Boolean expression based on the configuration message.

16. The method of claim 15 , further comprising:

configuring the integrity constraint as a check constraint, the check constraint using the Boolean expression.

17. The method of claim 14 , further comprising:

determining one or more remediation actions based on the configuration message, wherein a first remediation action of the one or more remediation actions comprises the inputting of the erroneous data into the error table.

18. The method of claim 17 , further comprising:

determining a second remediation action of the one or more remediation actions comprises canceling the operation on the base table.

19. The method of claim 11 , further comprising:

removing the erroneous data from the base table; and

executing the operation on remaining data in the base table.

20. The method of claim 11 , further comprising:

updating the error table to include an identification of the integrity constraint and an expression of the desired configuration associated with the integrity constraint.

21. A computer-storage medium comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:

decoding a query received at a network-based database system, the query including a command to perform an operation on a base table;

retrieving an integrity constraint associated with the base table, the integrity constraint specifying a desired configuration for the base table;

performing a verification of the integrity constraint to detect erroneous data of the base table, the erroneous data violating the desired configuration;

inputting the erroneous data into an error table, the error table being configured as a dynamic table maintained as a nested object of the base table; and

generating a notification that the erroneous data is available in the error table.

22. The computer-storage medium of claim 21 , wherein the integrity constraint comprises a check constraint and the desired configuration comprises at least one condition to be met by each row of a plurality of rows stored within the base table.

23. The computer-storage medium of claim 22 , the operations for performing the verification further comprising:

parsing each row of the plurality of rows to determine whether the row satisfies the at least one condition; and

designating the row as the erroneous data when the row does not satisfy the at least one condition.

24. The computer-storage medium of claim 21 , the operations further comprising:

decoding a configuration message received from an account of a user of the network-based database system, the configuration message including the desired configuration of the integrity constraint.

25. The computer-storage medium of claim 24 , the operations further comprising:

determining a Boolean expression based on the configuration message.

26. The computer-storage medium of claim 25 , the operations further comprising:

configuring the integrity constraint as a check constraint, the check constraint using the Boolean expression.

27. The computer-storage medium of claim 24 , the operations further comprising:

determining one or more remediation actions based on the configuration message, wherein a first remediation action of the one or more remediation actions comprises the inputting of the erroneous data into the error table.

28. The computer-storage medium of claim 27 , the operations further comprising:

determining a second remediation action of the one or more remediation actions comprises canceling the operation on the base table.

29. The computer-storage medium of claim 21 , the operations further comprising:

removing the erroneous data from the base table; and

executing the operation on remaining data in the base table.

30. The computer-storage medium of claim 21 , the operations further comprising:

updating the error table to include an identification of the integrity constraint and an expression of the desired configuration associated with the integrity constraint.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2023
From: BALAKRISHNAN, RAJA SURESH KRISHNA; IYER, GANESHAN RAMACHANDRAN; SCHULTZ, DAVID; XU, JIAN
To: SNOWFLAKE INC.
Reel/Frame 065161/0008 →
References Cited (47)
US 5790779A · Ben-Natan et al. · 1998 [cited by applicant]
US 6366915B1 · Rubert et al. · 2002 [cited by applicant]
US 10628394B1 · Gurspan · 2020 [cited by applicant]
US 11797521B1 · Vig et al. · 2023 [cited by applicant]
US 11921700B1 · Al Mahmood et al. · 2024 [cited by applicant]
US 20050149580A1 · Hattori et al. · 2005 [cited by applicant]
US 20060222160A1 · Bank et al. · 2006 [cited by applicant]
US 20080071825A1 · Guo · 2008 [cited by examiner]
US 20080307262A1 · Carlin, III · 2008 [cited by applicant]
US 20100161555A1 · Nica et al. · 2010 [cited by applicant]
US 20100166008A1 · Hashimoto · 2010 [cited by examiner]
US 20100211539A1 · Ho · 2010 [cited by examiner]
US 20140081907A1 · Tran · 2014 [cited by examiner]
US 20150142775A1 · Kang · 2015 [cited by examiner]
US 20160232200A1 · Sherman · 2016 [cited by applicant]
US 20160266920A1 · Atanasov · 2016 [cited by applicant]
US 20180096044A1 · Koza · 2018 [cited by examiner]
US 20190197112A1 · Kaplan · 2019 [cited by applicant]
US 20200110792A1 · Tsabba · 2020 [cited by examiner]
US 20200334268A1 · Vasireddy · 2020 [cited by examiner]
US 20210294291A1 · Murakami · 2021 [cited by examiner]
US 20210319030A1 · Guiney · 2021 [cited by examiner]
US 20240386010A1 · Al Mahmood et al. · 2024 [cited by applicant]
US 20240403276A1 · Ahmadi et al. · 2024 [cited by applicant]
WO 2024238722 · 2024 [cited by applicant]
“U.S. Appl. No. 18/319,886, Response filed Nov. 1, 2023 to Non Final Office Action mailed Aug. 1, 2023”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 18/319,886, Notice of Allowance mailed Nov. 17, 2023”, 9 pgs. [cited by applicant]
AWS, “STL Load Errors”, [Online]. Retrieved from the Internet: https: docs.aws.amazon.com redshift latest dg r_STL_LOAD_ERRORS.html, (Accessed online Nov. 2, 2023), 6 pages. [cited by applicant]
AWS, “STL Loaderror Detail”, [Online]. Retrieved from the Internet: https: docs.aws.amazon.com redshift latest dg r_STL_LOADERROR_DETAIL.html, (Accessed online Nov. 2, 2023), 4 pages. [cited by applicant]
Confluent, “Kafka Connect Deep Dive—Error Handling and Dead Letter Queues”, Online Retrieved from the Internet https: www.confluent.io blog kafka-connect-deep-dive-error-handling-dead-letter-queues, (Accessed online Nov… [cited by applicant]
Databricks, “Handle bad records and files”, [Online]. Retrieved from the Internet: https: docs.databricks.com spark latest spark-sql handling-bad-records.html, (Accessed online Nov. 2, 2023), 3 pages. [cited by applicant]
Databricks, “Databricks data engineering What is Delta Live Tables”, [Online]. Retrieved from the Internet: https: docs.databricks.com data-engineering delta-live-tables index.html, (Accessed online Nov. 2, 2023), 7 pag… [cited by applicant]
Databricks, “What is Delta Live Tables Manage data quality with Delta Live Tables”, [Online]. Retrieved from the Internet: https: docs.databricks.com en delta-live-tables expectations.html, (Accessed online Nov. 2, 2023… [cited by applicant]
Databricks, “Monitor Delta Live Tables pipelines”, [Online]. Retrieved from the Internet: https: docs.databricks.com data-engineering delta-live-tables delta-live-tables-event-log.html#event-log-schema, (Accessed online… [cited by applicant]
Google Cloud, “Big Query Jobs View”, [Online]. Retrieved from the Internet: https: cloud.google.com bigquery docs information-schema-jobs, (Accessed online Nov. 2, 2023), 17 pages. [cited by applicant]
Google Cloud, “Introduction to the BigQuery Storage Write API”, [Online]. Retrieved from the Internet: https: cloud.google.com bigquery docs write-api, (Accessed online Nov. 2, 2023), 15 pages. [cited by applicant]
Google Cloud, “Package google cloud bigquery storage v1”, [Online]. Retrieved from the Internet: https: cloud.google.com bigquery docs reference storage rpc google.cloud.bigquery.storage.v1#google.cloud.bigquery.storage… [cited by applicant]
Microsoft Ignite, “Copy Into Transact SQL”, [Online]. Retrieved from the Internet: https: docs.microsoft.com en-us sql t-sql statements copy-into-transact-sql?view=azure-sqldw-latestandpreserve-view=true, (Accessed onli… [cited by applicant]
Single Store, “View and Handle Pipeline Errors”, [Online]. Retrieved from the Internet: https: docs.singlestore.com cloud reference troubleshooting-reference pipeline-errors view-and-handle-pipeline-errors , (Accessed o… [cited by applicant]
“U.S. Appl. No. 18/319,886, Non Final Office Action mailed Aug. 1, 2023”, 16 pgs. [cited by applicant]
U.S. Appl. No. 18/319,886 U.S. Pat. No. 11,921,700, filed May 18, 2023, Error Tables to Track Errors Associated With a Base Table. [cited by applicant]
U.S. Appl. No. 18/426,772, filed Jan. 30, 2024, Error Tables to Track Errors Associated With a Base Table. [cited by applicant]
U.S. Appl. No. 18/326,158, filed May 31, 2023, Built-in Data Quality Monitoring. [cited by applicant]
“U.S. Appl. No. 18/326,158, Non Final Office Action mailed Sep. 24, 2024”, 15 pages. [cited by applicant]
“U.S. Appl. No. 18/426,772, Notice of Allowance mailed Aug. 16, 2024”, 13 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/029570, International Search Report mailed Jun. 12, 2024”, 3 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/029570, Written Opinion mailed Jun. 12, 2024”, 7 pgs. [cited by applicant]
Cited By (3)
US 12,481,643 US 12,608,538 US 12,705,218