IP Library Granted Patent US 11,157,564
Granted Patent B2
US 11,157,564 · App. 16/262,796 · Granted Oct 26, 2021

Natural language question answering systems

Inventors: Amit Prakash (Saratoga, CA); Ravi Tandon (Sunnyvale, CA); Manikanta Balakavi (Mountain View, CA); Pavan Ram Piratla (Campbell, CA); Ashish Shubham (Mountain View, CA); Alonzo Canada (Menlo Park, CA); Rakesh Kothari (San Jose, CA); Maneesh Apte (Redwood City, CA); Amitabh Singhal (Palo Alto, CA); Aditya Viswanathan (Redwood City, CA); Ajeet Singh (Saratoga, CA)
Assignee: ThoughtSpot, Inc.
G06F16/90335G06F16/243G06F16/24534G06N5/04G06N7/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,157,564
App. No.
16/262,796
Granted
Oct 26, 2021
Kind
B2
Abstract

Systems and methods for natural language question answering are described. For example, methods may include determining a set of candidate database queries, including respective sequences of tokens of a database syntax, based on a string; determining a first score for a first candidate database query from the set of candidate database queries, wherein the first score is based on a match between one or more words of the string and a token of the respective sequence of tokens of the first candidate database query; determining a second score for the first candidate database query, wherein the second score is based on natural language syntax data determined for words of the string; selecting, based on the first score and the second score, the first candidate database query from the set of candidate database queries; and invoking a search of the database using the first candidate database query to obtain search results.

Claims (60)

1. A method comprising:

determining a set of candidate database queries, each including a respective sequence of tokens of a database syntax, based on a string;

determining a first score for a first candidate database query from the set of candidate database queries, wherein the first score is based on a match between one or more words of the string and a token of the respective sequence of tokens of the first candidate database query;

determining a second score for the first candidate database query, wherein the second score is based on natural language syntax data determined for words of the string;

selecting, based on the first score and the second score, the first candidate database query from the set of candidate database queries; and

invoking a search of the database using a query based on the first candidate database query to obtain search results.

2. The method of claim 1 , comprising:

extracting an inference pattern from the string and the first candidate database query;

classifying the inference pattern to determine an inference type;

determining a resolution, wherein the resolution includes one or more tokens of the database syntax;

identifying a set of context features for the inference pattern, wherein the set of context features includes words from the string and tokens from the first candidate database query;

determining a confidence score based on the set of context features; and

storing an inference record in an inference store, wherein the inference record includes the set of context features, the resolution, and the confidence score.

3. The method of claim 2 , wherein the confidence score is determined based in part upon receiving feedback via a user interface that was used modify the first candidate database query.

4. The method of claim 2 , wherein the confidence score is determined based in part upon receiving a feedback representing user interaction with a like icon in a user interface while the user interface includes representations of the string and the first candidate database query.

5. The method of claim 2 , comprising:

maintaining a contains probability score associated with a column of the database, wherein the contains probability score is based on a count of times the column has been used to filter data with a contains query; and

using the contains probability score for the column to determine the resolution.

6. The method of claim 1 , comprising:

identifying a current set of context features for the first candidate database query, wherein the current set of context features includes words from the string and tokens from the first candidate database query;

selecting a first inference record from an inference store based on comparison of the current set of context features to context features of inference records in the inference store; and

modifying the first candidate database query using a resolution of the first inference record to obtain an inferred database query, wherein the resolution includes one or more tokens of the database syntax, and wherein the query is based on the inferred database query.

7. The method of claim 6 , wherein the first inference record includes a confidence score and the first inference record is selected based on the confidence score.

8. The method of claim 6 , wherein the current set of context features include part-of-speech tags for words of the string, and comprising:

comparing the part-of-speech tags of the current set of context features to part-of-speech tags of inference records in the inference store, wherein the first inference record is selected based on a match of part-of-speech tags of the first inference record to the part-of-speech tags of the current set of context features.

9. The method of claim 1 , comprising:

receiving an indication that the string matches the first candidate database query;

responsive to the indication, determining a pattern based on the string and the first candidate database query, wherein the pattern includes a collection of token constraints, a rewrite rule that maps one or more input tokens to a sequence of output tokens, and a ranking score adjustment that can be applied to a ranking score for a candidate database query; and

storing the pattern.

10. The method of claim 9 , wherein the indication is based on a user interaction with a like icon in a user interface while the user interface includes representations of the string and the first candidate database query.

11. The method of claim 9 , wherein the indication includes user feedback data concerning the database query that was used to modify the first candidate database query.

12. The method of claim 9 , wherein the collection of token constraints is a sequence of token constraints.

13. The method of claim 1 , comprising:

checking whether the first candidate database query from the set of candidate database queries satisfies a collection of token constraints of a pattern, wherein the pattern includes the collection of token constraints, a rewrite rule that maps one or more input tokens to a sequence of output tokens, and a ranking score adjustment that can be applied to a ranking score for a candidate database query; and

responsive to satisfaction of the collection of token constraints of the pattern, modifying the first candidate database query using the rewrite rule of the pattern, and adjusting the second score based on the ranking score adjustment of the pattern.

14. The method of claim 1 , wherein the first score is determined based on a first inverse document frequency of an unmatched word of the string and a second inverse document frequency of a matched word of the string.

15. The method of claim 1 , comprising:

presenting, via a user interface, respective text representations for tokens in the sequence of tokens;

receiving feedback data concerning the first candidate database query via the user interface;

modifying the first candidate database query based on the feedback data;

invoking a search of a database using a query based on the modified first candidate database query to obtain modified search results; and

presenting data based on the modified search results in the user interface.

16. The method of claim 1 , wherein the first score is determined based on usage data for the token of the respective sequence of tokens of the first candidate database query.

17. A system for providing a search interface for databases, comprising:

a network interface,

a processor, and

a memory, wherein the memory stores instructions executable by the processor to:

determine a set of candidate database queries, each including a respective sequence of tokens of a database syntax, based on a string;

determine a first score for a first candidate database query from the set of candidate database queries, wherein the first score is based on a match between one or more words of the string and a token of the respective sequence of tokens of the first candidate database query;

determine a second score for the first candidate database query, wherein the second score is based on natural language syntax data determined for words of the string;

select, based on the first score and the second score, the first candidate database query from the set of candidate database queries; and

invoke a search of the database using a query based on the first candidate database query to obtain search results.

18. The system of claim 17 , wherein the first score is determined based on usage data for the token of the respective sequence of tokens of the first candidate database query.

19. A non-transitory computer-readable storage medium that includes instructions that, when executed by a processor, facilitate performance of operations comprising:

determining a set of candidate database queries, each including a respective sequence of tokens of a database syntax, based on a string;

determining a first score for a first candidate database query from the set of candidate database queries, wherein the first score is based on a match between one or more words of the string and a token of the respective sequence of tokens of the first candidate database query;

determining a second score for the first candidate database query, wherein the second score is based on natural language syntax data determined for words of the string;

selecting, based on the first score and the second score, the first candidate database query from the set of candidate database queries; and

invoking a search of the database using a query based on the first candidate database query to obtain search results.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the first score is determined based on usage data for the token of the respective sequence of tokens of the first candidate database query.

Assignments (2)
SECURITY INTEREST Recorded Mar 7, 2025
From: THOUGHTSPOT, INC.; THOUGHTSPOT, LLC
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 070442/0499 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2019
From: PRAKASH, AMIT; TANDON, RAVI; BALAKAVI, MANIKANTA; PIRATLA, PAVAN RAM; SHUBHAM, ASHISH; CANADA, ALONZO; KOTHARI, RAKESH; APTE, MANEESH; SINGHAL, AMITABH; VISWANATHAN, ADITYA; SINGH, AJEET
To: THOUGHTSPOT, INC.
Reel/Frame 048586/0517 →
Continuity (3)
Provisional Application 62760563 · Nov 13, 2018
Provisional Application 62637823 · Mar 2, 2018
Related Publication 20190272296A1 · Sep 5, 2019
Cited By (1)
US 12,229,096