IP Library Granted Patent US 10,216,792
Granted Patent B2
US 10,216,792 · App. 14/883,500 · Granted Feb 26, 2019

Automated join detection

Inventors: Dave Brewster (Redwood City, CA); Victor Tze-Yeuan Tso (Redwood City, CA); Ashley Jin (Los Gatos, CA); Quan Chuong Ta (South San Francisco, CA); Lakshman Roy Sankar (San Francisco, CA); Nenshad Dinshaw Bardoliwalla (Castro Valley, CA)
Assignee: Paxata, Inc.
G06F17/30454G06F17/30498
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,216,792
App. No.
14/883,500
Granted
Feb 26, 2019
Kind
B2
Abstract

Automated join detection includes: identifying a set of one or more candidate joins of a first table and a second table; evaluating a set of one or more quality measures corresponding to the set of one or more candidate joins; obtaining a set of one or more selected joins among the set of one or more candidate joins, the set of one or more selected joins being selected based at least in part on one or more corresponding quality measures; and generating a joined table, including by joining the first table and the second table according to a selected join.

Claims (40)

1. A method, comprising:

identifying, using one or more processors, a set of one or more candidate joins of a first table and a second table, wherein a candidate join specifies a set of one or more conditions associated with joining a first set of one or more columns in the first table and a second set of one or more columns in the second table;

evaluating, using the one or more processors, a set of one or more quality measures corresponding to the set of one or more candidate joins, a quality measure in the set of one or more quality measures being determined based at least in part on an overlap measure and a selectivity measure; wherein: the overlap measure measures an amount of overlap in matching values between values of records in the first set of one or more columns in the first table and values of records in the second set of one or more columns in the second table, and the selectivity measure measures how selectively a join condition maps a row in the first table to one or more rows in the second table and is determined based at least in part on, for one or more selected rows in the first table, one or more numbers of corresponding rows in the second table that match the one or more selected rows in the first table;

obtaining, using the one or more processors, a set of one or more selected joins among the set of one or more candidate joins, the set of one or more selected joins being selected based at least in part on one or more corresponding quality measures; and

generating, using the one or more processors, a joined table, including by joining the first table and the second table according to a selected join.

2. The method of claim 1 , further comprising presenting the set of one or more candidate joins to a user.

3. The method of claim 2 , wherein the one or more selected joins are selected by the user among the set of one or more candidate joins.

4. The method of claim 1 , further comprising preprocessing the first table, the second table, or both.

5. The method of claim 1 , wherein the identifying of the set of one or more candidate joins includes matching columns in the first table with columns in the second table.

6. The method of claim 5 , wherein the matching of the columns in the first table with the columns in the second table includes matching titles.

7. The method of claim 5 , wherein the matching of the columns in the first table with the columns in the second table includes matching patterns.

8. The method of claim 5 , wherein the matching of the columns in the first table with the columns in the second table includes matching values.

9. The method of claim 8 , wherein the matching of the values includes:

establishing a plurality of tries representing values of columns in the second table; and

for a row in the first table, searching for a matching path using the plurality of tries to identify a feasible join.

10. The method of claim 9 , wherein the establishing the plurality of tries includes tokenizing values of corresponding columns in the second table.

11. The method of claim 5 , wherein columns found to match are further subject to a join quality test.

12. The method of claim 11 , wherein the join quality test includes:

selecting one or more candidate joins having corresponding quality measures that at least meet a first threshold; and

discarding one or more candidate joins having corresponding quality measures that do not at least meet a second threshold.

13. The method of claim 12 , wherein the join quality test further includes:

identifying candidate joins having intermediate quality measures; and

performing a correlated join search based on a combination of at least two of the candidate joins having intermediate quality measures to identify one or more additional candidate joins that at least meet the first threshold.

14. The method of claim 11 , wherein the join quality test includes eliminating one or more columns in the first table, eliminating one or more columns in the second table, or both.

15. The method of claim 14 , wherein the columns to be eliminated are identified based on their properties using a machine learning technique.

16. The method of claim 1 , wherein the overlap measure is determined based at least in part on a usage rate, a proportion of unique values in matching records, or both.

17. The method of claim 1 , further comprising providing one or more joined table previews based on different column pairings.

18. A system, comprising:

one or more processors configured to:

identify a set of one or more candidate joins of a first table and a second table, wherein a candidate join specifies a set of one or more conditions associated with joining a first set of one or more columns in the first table and a second set of one or more columns in the second table;

evaluate a set of one or more quality measures corresponding to the set of one or more candidate joins, a quality measure in the set of one or more quality measures being determined based at least in part on an overlap measure and a selectivity measure; wherein: the overlap measure measures an amount of overlap in matching values between values of records in the first set of one or more columns in the first table and values of records in the second set of one or more columns in the second table, and the selectivity measure measures how selectively a join condition maps a row in the first table to one or more rows in the second table and is determined based at least in part on, for one or more selected rows in the first table, one or more numbers of corresponding rows in the second table that match the one or more selected rows in the first table;

obtain a set of one or more selected joins among the set of one or more candidate joins, the set of one or more selected joins being selected based at least in part on one or more corresponding quality measures; and

generate a joined table, including by joining the first table and the second table according to a selected join; and

one or more memories coupled to the one or more processors and configured to provide the one or more processors with instructions.

19. A computer program product embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:

identifying a set of one or more candidate joins of a first table and a second table, wherein a candidate join specifies a set of one or more conditions associated with joining a first set of one or more columns in the first table and a second set of one or more columns in the second table;

evaluating a set of one or more quality measures corresponding to the set of one or more candidate joins, a quality measure in the set of one or more quality measures being determined based at least in part on an overlap measure and a selectivity measure; wherein: the overlap measure measures an amount of overlap in matching values between values of records in the first set of one or more columns in the first table and values of records in the second set of one or more columns in the second table, and the selectivity measure measures how selectively a join condition maps a row in the first table to one or more rows in the second table and is determined based at least in part on, for one or more selected rows in the first table, one or more numbers of corresponding rows in the second table that match the one or more selected rows in the first table;

obtaining a set of one or more selected joins among the set of one or more candidate joins, the set of one or more selected joins being selected based at least in part on one or more corresponding quality measures; and

generating a joined table, including by joining the first table and the second table according to a selected join; wherein

the computer instructions are executed on one or more processors.

Assignments (7)
RELEASE OF SECURITY INTEREST Recorded Apr 7, 2025
From: CITIBANK, N.A.
To: DATAROBOT, INC.; ALGORITHMIA, INC.; DULLES RESEARCH, LLC
Reel/Frame 070750/0866 →
SECURITY INTEREST Recorded Mar 22, 2023
From: DATAROBOT, INC.; ALGORITHMIA, INC.; DULLES RESEARCH, LLC
To: CITIBANK, N.A.
Reel/Frame 063263/0926 →
RELEASE OF SECURITY INTEREST Recorded Feb 2, 2023
From: SILICON VALLEY BANK
To: PAXATA, INC.
Reel/Frame 063251/0119 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 25, 2022
From: DR HOLDCO 2, INC.
To: DATAROBOT, INC.
Reel/Frame 060019/0207 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2020
From: PAXATA, INC.
To: DR HOLDCO 2, INC.
Reel/Frame 052377/0987 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Nov 5, 2019
From: PAXATA, INC.
To: SILICON VALLEY BANK
Reel/Frame 050930/0534 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 11, 2015
From: BREWSTER, DAVE; TSO, VICTOR TZE-YEUAN; JIN, ASHLEY PING; TA, QUAN CHUONG; SANKAR, LAKSHMAN ROY; BARDOLIWALLA, NENSHAD DINSHAW
To: PAXATA, INC.
Reel/Frame 037271/0930 →
Continuity (1)
Related Publication 20170109402A1 · Apr 20, 2017
Cited By (1)
US 12,381,795