IP Library Granted Patent US 12,579,110
Granted Patent B2
US 12,579,110 · App. 17/324,874 · Granted Mar 17, 2026

Implementation of data access metrics for automated physical database design

Inventors: Michael Brendle (Leimen, DE); Norman May (Karlsruhe, DE); Robert Schulze (Dresden, DE); Alexander Boehm (Schwetzingen, DE); Guido Moerkotte (Schriesheim, DE); Michael Grossniklaus (Kreuzlingen, CH)
Assignee: SAP SE
G06F16/21G06F11/3414G06F11/3428G06F16/213G06F16/215G06F16/217G06F16/2272G06F16/2282G06F16/24545G06F16/2455G06F16/24552G06F16/24575H03M7/6064
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,579,110
App. No.
17/324,874
Filed
May 19, 2021
Granted
Mar 17, 2026
Kind
B2
Art Unit
2164
USPC
707/693
Abstract

The present disclosure involves systems, software, and computer implemented methods for improved design and implementation of data access metrics for automated physical database design. An example method includes identifying a database workload for which index advisor access counters are to be tracked. Each SQL statement in the database workload is executed. For each SQL statement, attribute sets are determined for which a selection predicate filters a result for an SQL statement. An output cardinality of each selection predicate is determined. A logarithmic counter for an attribute set corresponding to the selection predicate is determined based on the output cardinality of the selection predicate. The determined logarithmic counter is incremented. Respective values for logarithmic counters of the determined attributes are provided to an index advisor. The index advisor determines attribute sets for which to propose an index based on the logarithmic counters of the respective attribute sets.

Claims (73)

1 . A computer-implemented method comprising:

identifying a database workload comprising at least one query for which database column access counters are to be maintained;

determining a set of all database columns referenced in the database workload by identifying all database column references that are included in the at least one query of the database workload;

creating, based on the set of all database columns referenced in the database workload, database column access counters for the database workload that track counts of read accesses of data in respective database columns, including:

creating a sequential access counter for each database column referenced in the database workload; and

creating a random access counter for each database column referenced in the database workload;

tracking, for each database column referenced in the database workload, memory access of data in the database column during execution of the database workload, including:

incrementing the sequential access counter for a respective database column in response to determining that a database row that includes data of the database column is sequentially read; and

incrementing the random access counter for a respective database column in response to determining that a database row that includes data of the database column is randomly read;

providing respective values for the database column access counters for each database column referenced in the database workload to a data compression advisor;

identifying, by the data compression advisor, at least one compression proposal rule for determining whether to propose data compression for a given database column, wherein each compression proposal rule is based on values of both the random access counter for the database column, the sequential access counter for the database column, and a rule threshold;

determining, by the data compression advisor, for each database column referenced in the database workload, whether to propose data compression for the database column based on evaluating each compression proposal rule based on the respective values for the database column access counters for the database column and the rule threshold for the compression proposal rule;

generating, by the data compression advisor, a recommendation to compress a first database column referenced in the database workload based on determining, during evaluation of a first compression proposal rule, that the rule threshold for the first compression proposal rule is satisfied;

providing the recommendation to an automatic database tuner; and

automatically implementing, by the automatic database tuner, data compression for the first database column based on the recommendation.

2 . The computer-implemented method of claim 1 , wherein determining whether to propose data compression for a first database column comprises:

determining an estimated execution time for the database workload if the first database column is compressed;

determining an estimated execution time for the database workload if the first database column is not compressed; and

determining to propose data compression for the first database column in response to determining that the estimated execution time for the database workload if the first database column is compressed is less than the estimated execution time for the database workload if the first database column is not compressed.

3 . The computer-implemented method of claim 1 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column comprises:

comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column; and

determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column.

4 . The computer-implemented method of claim 3 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column comprises determining whether the value of the sequential access counter for the first database column is substantially larger than the value of the random access counter for the first database column.

5 . The computer-implemented method of claim 1 , wherein the first compression proposal rule specifies that a value of a sequential access counter for a database column is substantially larger than a value of a corresponding random access counter for the database column when a ratio of the value of the sequential access counter for the database column to the value of the corresponding random access counter for the database column is more than a predetermined threshold.

6 . A system comprising:

one or more computers; and

a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

identifying a database workload comprising at least one query for which database column access counters are to be maintained;

determining a set of all database columns referenced in the database workload by identifying all database column references that are included in the at least one query of the database workload;

creating, based on the set of all database columns referenced in the database workload, database column access counters for the database workload that track counts of read accesses of data in respective database columns, including:

creating a sequential access counter for each database column referenced in the database workload; and

creating a random access counter for each database column referenced in the database workload;

tracking, for each database column referenced in the database workload, memory access of data in the database column during execution of the database workload, including:

incrementing the sequential access counter for a respective database column in response to determining that a database row that includes data of the database column is sequentially read; and

incrementing the random access counter for a respective database column in response to determining that a database row that includes data of the database column is randomly read;

providing respective values for the database column access counters for each database column referenced in the database workload to a data compression advisor;

identifying, by the data compression advisor, at least one compression proposal rule for determining whether to propose data compression for a given database column, wherein each compression proposal rule is based on values of both the random access counter for the database column, the sequential access counter for the database column, and a rule threshold;

determining, by the data compression advisor, for each database column referenced in the database workload, whether to propose data compression for the database column based on evaluating each compression proposal rule based on the respective values for the database column access counters for the database column and the rule threshold for the compression proposal rule;

generating, by the data compression advisor, a recommendation to compress a first database column referenced in the database workload based on determining, during evaluation of a first compression proposal rule, that the rule threshold for the first compression proposal rule is satisfied;

providing the recommendation to an automatic database tuner; and

automatically implementing, by the automatic database tuner, data compression for the first database column based on the recommendation.

7 . The system of claim 6 , wherein determining whether to propose data compression for a first database column comprises:

determining an estimated execution time for the database workload if the first database column is compressed;

determining an estimated execution time for the database workload if the first database column is not compressed; and

determining to propose data compression for the first database column in response to determining that the estimated execution time for the database workload if the first database column is compressed is less than the estimated execution time for the database workload if the first database column is not compressed.

8 . The system of claim 6 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column comprises:

comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column; and

determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column.

9 . The system of claim 8 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column comprises determining whether the value of the sequential access counter for the first database column is substantially larger than the value of the random access counter for the first database column.

10 . The system of claim 6 , wherein the first compression proposal rule specifies that a value of a sequential access counter for a database column is substantially larger than a value of a corresponding random access counter for the database column when a ratio of the value of the sequential access counter for the database column to the value of the corresponding random access counter for the database column is more than a predetermined threshold.

11 . A computer program product encoded on a non-transitory storage medium, the product comprising non-transitory, computer readable instructions for causing one or more processors to perform operations comprising:

identifying a database workload comprising at least one query for which database column access counters are to be maintained;

determining a set of all database columns referenced in the database workload by identifying all database column references that are included in the at least one query of the database workload;

creating, based on the set of all database columns referenced in the database workload, database column access counters for the database workload that track counts of read accesses of data in respective database columns, including:

creating a sequential access counter for each database column referenced in the database workload; and

creating a random access counter for each database column referenced in the database workload;

tracking, for each database column referenced in the database workload, memory access of data in the database column during execution of the database workload, including:

incrementing the sequential access counter for a respective database column in response to determining that a database row that includes data of the database column is sequentially read; and

incrementing the random access counter for a respective database column in response to determining that a database row that includes data of the database column is randomly read;

providing respective values for the database column access counters for each database column referenced in the database workload to a data compression advisor;

identifying, by the data compression advisor, at least one compression proposal rule for determining whether to propose data compression for a given database column, wherein each compression proposal rule is based on values of both the random access counter for the database column, the sequential access counter for the database column, and a rule threshold;

determining, by the data compression advisor, for each database column referenced in the database workload, whether to propose data compression for the database column based on evaluating each compression proposal rule based on the respective values for the database column access counters for the database column and the rule threshold for the compression proposal rule;

generating, by the data compression advisor, a recommendation to compress a first database column referenced in the database workload based on determining, during evaluation of a first compression proposal rule, that the rule threshold for the first compression proposal rule is satisfied;

providing the recommendation to an automatic database tuner; and

automatically implementing, by the automatic database tuner, data compression for the first database column based on the recommendation.

12 . The computer program product of claim 11 , wherein determining whether to propose data compression for a first database column comprises:

determining an estimated execution time for the database workload if the first database column is compressed;

determining an estimated execution time for the database workload if the first database column is not compressed; and

determining to propose data compression for the first database column in response to determining that the estimated execution time for the database workload if the first database column is compressed is less than the estimated execution time for the database workload if the first database column is not compressed.

13 . The computer program product of claim 12 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column comprises:

comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column; and

determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column.

14 . The computer program product of claim 13 , wherein evaluating the first compression proposal rule when determining whether to propose data compression for the first database column based on comparing the value of the sequential access counter for the first database column to the value of the random access counter for the first database column comprises determining whether the value of the sequential access counter for the first database column is substantially larger than the value of the random access counter for the first database column.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2021
From: BRENDLE, MICHAEL; MAY, NORMAN; SCHULZE, ROBERT; BOEHM, ALEXANDER; MOERKOTTE, GUIDO; GROSSNIKLAUS, MICHAEL
To: SAP SE
Reel/Frame 056304/0481 →
Continuity (3)
Continuation 17316901 · May 11, 2021
Provisional Application 63153214 · Feb 24, 2021
Related Publication 20220269658A1 · Aug 25, 2022
References Cited (113)
US 8935205B2 · Hildenbrand et al. · 2015 [cited by applicant]
US 9152626B2 · Seufert et al. · 2015 [cited by applicant]
US 9189520B2 · May et al. · 2015 [cited by applicant]
US 9329899B2 · Ailamaki et al. · 2016 [cited by applicant]
US 9361273B2 · Dannecker et al. · 2016 [cited by applicant]
US 9378233B2 · Lee et al. · 2016 [cited by applicant]
US 9454571B2 · Grosse et al. · 2016 [cited by applicant]
US 9483513B2 · Heidel et al. · 2016 [cited by applicant]
US 9720942B2 · Kaufmann et al. · 2017 [cited by applicant]
US 9747313B2 · Kaufmann et al. · 2017 [cited by applicant]
US 10061808B2 · Kim et al. · 2018 [cited by applicant]
US 10140326B2 · Sherkat et al. · 2018 [cited by applicant]
US 10185744B2 · Bensberg et al. · 2019 [cited by applicant]
US 10248688B2 · Grosse et al. · 2019 [cited by applicant]
US 10261950B2 · Lee et al. · 2019 [cited by applicant]
US 10275508B2 · Bensberg et al. · 2019 [cited by applicant]
US 10282351B2 · Bensberg et al. · 2019 [cited by applicant]
US 10353895B2 · Park et al. · 2019 [cited by applicant]
US 10387127B2 · Hassan · 2019 [cited by examiner]
US 10482085B2 · Moerkotte et al. · 2019 [cited by applicant]
US 10496656B2 · Gaumnitz et al. · 2019 [cited by applicant]
US 10534775B2 · Moerkotte et al. · 2020 [cited by applicant]
US 10545974B2 · Brunel et al. · 2020 [cited by applicant]
US 10592509B2 · Ziegler et al. · 2020 [cited by applicant]
US 10671625B2 · Schulze et al. · 2020 [cited by applicant]
US 10713284B2 · Bensberg et al. · 2020 [cited by applicant]
US 10733185B2 · Psaropoulos et al. · 2020 [cited by applicant]
US 10762071B2 · Schulze et al. · 2020 [cited by applicant]
US 10776353B2 · Schulze et al. · 2020 [cited by applicant]
US 10824673B2 · Rebholz et al. · 2020 [cited by applicant]
US 10872086B2 · Moerkotte et al. · 2020 [cited by applicant]
US 10877956B2 · Park et al. · 2020 [cited by applicant]
US 10885062B2 · Andrei et al. · 2021 [cited by applicant]
US 10891234B2 · Noll et al. · 2021 [cited by applicant]
US 10990427B2 · Kroll et al. · 2021 [cited by applicant]
US 11550762B2 · Brendle et al. · 2023 [cited by applicant]
US 20050203933A1 · Chaudhuri et al. · 2005 [cited by applicant]
US 20050223026A1 · Chaudhuri et al. · 2005 [cited by applicant]
US 20060074872A1 · Gordon · 2006 [cited by applicant]
US 20080071939A1 · Tanaka et al. · 2008 [cited by applicant]
US 20080307009A1 · Anderson et al. · 2008 [cited by applicant]
US 20090077302A1 · Fukuda · 2009 [cited by examiner]
US 20090193042A1 · Hornibrook · 2009 [cited by examiner]
US 20090235252A1 · Weber et al. · 2009 [cited by applicant]
US 20100257151A1 · Lohman et al. · 2010 [cited by applicant]
US 20130204879A1 · Zeng et al. · 2013 [cited by applicant]
US 20140114728A1 · Kaufmann et al. · 2014 [cited by applicant]
US 20140149357A1 · Gupta · 2014 [cited by applicant]
US 20160012089A1 · Sherkat · 2016 [cited by examiner]
US 20160042039A1 · Kaufmann et al. · 2016 [cited by applicant]
US 20160188710A1 · Dulba Naik · 2016 [cited by examiner]
US 20160224241A1 · Verrilli · 2016 [cited by examiner]
US 20170017674A1 · Scheuer et al. · 2017 [cited by applicant]
US 20170031967A1 · Chavan · 2017 [cited by examiner]
US 20170031975A1 · Mishra · 2017 [cited by examiner]
US 20170039232A1 · Jayanth · 2017 [cited by examiner]
US 20170075832A1 · Bhimani et al. · 2017 [cited by applicant]
US 20180011892A1 · Kimura · 2018 [cited by applicant]
US 20180024821A1 · Hassan · 2018 [cited by examiner]
US 20180088853A1 · Kotra et al. · 2018 [cited by applicant]
US 20180096006A1 · Das · 2018 [cited by examiner]
US 20180329974A1 · Bensberg et al. · 2018 [cited by applicant]
US 20180357291A1 · Choi et al. · 2018 [cited by applicant]
US 20190130001A1 · May et al. · 2019 [cited by applicant]
US 20190243816A1 · Gaumnitz et al. · 2019 [cited by applicant]
US 20190266272A1 · Wolf et al. · 2019 [cited by applicant]
US 20190271784A1 · Teigland et al. · 2019 [cited by applicant]
US 20190278608A1 · Psaropoulos et al. · 2019 [cited by applicant]
US 20190370257A1 · Wolf et al. · 2019 [cited by applicant]
US 20200026560A1 · Singh · 2020 [cited by examiner]
US 20200117648A1 · Gaumnitz et al. · 2020 [cited by applicant]
US 20200192884A1 · Bao et al. · 2020 [cited by applicant]
US 20200233661A1 · Grosse et al. · 2020 [cited by applicant]
US 20200250167A1 · Brunel et al. · 2020 [cited by applicant]
US 20200387495A1 · Pathak et al. · 2020 [cited by applicant]
US 20200401405A1 · Lasch et al. · 2020 [cited by applicant]
US 20200401530A1 · Abulila et al. · 2020 [cited by applicant]
US 20200403633A1 · Lasch et al. · 2020 [cited by applicant]
US 20220269653A1 · Brendle et al. · 2022 [cited by applicant]
US 20220269684A1 · Brendle et al. · 2022 [cited by applicant]
Extended European Search Report issued in European Application No. 22157240.7 on Aug. 9, 2022, 10 pages. [cited by applicant]
U.S. Appl. No. 16/739,352, Noll et al., Jan. 10, 2020. [cited by applicant]
U.S. Appl. No. 17/316,901, Brendle et al., May 11, 2021. [cited by applicant]
U.S. Appl. No. 17/324,896, Brendle et al., May 19, 2021. [cited by applicant]
U.S. Appl. No. 17/324,914, Brendle et al., May 19, 2021. [cited by applicant]
Agrawal et al., “Database tuning advisor for microsoft sql server 2005.” Proceedings of the 2005 ACM SIGMOD international conference on Management of data, Jun. 2005, 12 pages. [cited by applicant]
Athanassoulis et al., “Optimal column layout for hybrid workloads.” Proceedings of the VLDB Endowment 12.13, Sep. 2019, 2393-2407, 15 pages. [cited by applicant]
Boncz et al., “JCC-H: adding join crossing correlations with skew to TPC-H.” Technology Conference on Performance Evaluation and Benchmarking. Springer, Cham, Aug. 2017, 17 pages. [cited by applicant]
Curino et al., “Schism: a workload-driven approach to database replication and partitioning.” Proceedings of the VLDB Endowment vol. 3, No. 1, 2010, 10 pages. [cited by applicant]
Damme et al., “From a comprehensive experimental survey to a cost-based selection strategy for lightweight integer compression algorithms.” ACM Transactions on Database Systems (TODS) 44.3, Jun. 2019, 46 pages. [cited by applicant]
Das et al., “Automated demand-driven resource scaling in relational database-as-a-service.” Proceedings of the 2016 International Conference on Management of Data, Jun. 2016, 12 pages. [cited by applicant]
Funke et al., “Compacting transactional data in hybrid OLTP & OLAP databases.” Proceedings of the VLDB Endowment vol. 5, No. 11, Aug. 2012, 12 pages. [cited by applicant]
Gurajada et al., “Btrim: hybrid in-memory database architecture for extreme transaction processing in vldbs.” Proceedings of the VLDB Endowment 11.12, Aug. 2018, 1889-1901, 13 pages. [cited by applicant]
Huang et al., “X-Engine: An optimized storage engine for large-scale E-commerce transaction processing.” Proceedings of the 2019 International Conference on Management of Data, Jun. 2019, 15 pages. [cited by applicant]
Kester et al., “Access path selection in main-memory optimized data systems: Should I scan or should I probe?” Proceedings of the 2017 ACM International Conference on Management of Data, May 2017, 16 pages. [cited by applicant]
Kossmann et al., “Magic mirror in my hand, which is the best in the land? an experimental evaluation of index selection algorithms.” Proceedings of the VLDB Endowment 13.12, Jul. 2020, 2382-2395, 14 pages. [cited by applicant]
Leis et al., “How good are query optimizers, really ?.” Proceedings of the VLDB Endowment 9.3, Nov. 2015, 204-215, 12 pages. [cited by applicant]
Lemke et al., “Speeding up queries in column stores: a case for compression.” DaWaK, 2010, 117-129, 13 pages. [cited by applicant]
Levandoski et al., “Identifying hot and cold data in main-memory databases.” 2013 IEEE 29th International Conference on Data Engineering (ICDE). IEEE, Apr. 2013, 12 pages. [cited by applicant]
Lu et al., “Speedup your analytics: Automatic parameter tuning for databases and big data systems.” Proceedings of the VLDB Endowment, Aug. 2019, 4 pages. [cited by applicant]
May et al., “SAP HANA—The evolution of an in-memory DBMS from Pure OLAP processing towards mixed workloads.” Datenbanksysteme für Business, Technologie und Web, BTW, 2017, 19 pages. [cited by applicant]
Metwally et al., “Efficient computation of frequent and top-k elements in data streams.” International conference on database theory. Springer, Berlin, Heidelberg, Jan. 2005, 21 pages. [cited by applicant]
Nathan et al., “Learning multi-dimensional indexes.” Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, Jun. 2020, 16 pages. [cited by applicant]
Noll et al., “Analyzing memory accesses with modern processors.” Proceedings of the 16th International Workshop on Data Management on New Hardware, Jun. 2020, 9 pages. [cited by applicant]
Rao et al., “Automating physical database design in a parallel database.” Proceedings of the 2002 ACM SIGMOD international conference on Management of data, Jun. 2002, 12 pages. [cited by applicant]
Serafini et al., “Clay: Fine-grained adaptive partitioning for general database schemas.” Proceedings of the VLDB Endowment 10.4, Nov. 2016, 445-456, 12 pages. [cited by applicant]
Sherkat et al., “Native store extension for SAP HANA.” Proceedings of the VLDB Endowment 12.12, Aug. 2019, 2047-2058, 12 pages. [cited by applicant]
Storm et al., “Adaptive self-tuning memory in DB2.” Proceedings of the 32nd international conference on Very large data bases, Sep. 2006, 12 pages. [cited by applicant]
TPC, Transaction Processing Performance Council. “TPC Benchmark™ E. Revision 2.18.0”, 2018, 138 pages. [cited by applicant]
Final Office Action in U.S. Appl. No. 17/324,914, mailed on Oct. 26, 2023, 27 pages. [cited by applicant]
Final Office Action in U.S. Appl. No. 17/324,896, mailed on Jan. 26, 2023, 26 pages. [cited by applicant]
Non-Final Office Action in U.S. Appl. No. 17/324,896, mailed on Oct. 6, 2022, 30 pages. [cited by applicant]
Non-Final Office Action in U.S. Appl. No. 17/324,914, mailed on Apr. 27, 2023, 30 pages. [cited by applicant]