IP Library Granted Patent US 11,675,795
Granted Patent B2
US 11,675,795 · App. 14/959,122 · Granted Jun 13, 2023

Method and system for ranking search content

Inventors: Dawei Yin (San Jose, CA); Pengyuan Wang (Sunnyvale, CA); Hua Ouyang (Sunnyvale, CA); Yi Chang (Sunnyvale, CA); Jean-Marc Langlois (Menlo Park, CA)
Assignee: YAHOO ASSETS LLC
G06F16/24578G06F16/9535
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,675,795
App. No.
14/959,122
Granted
Jun 13, 2023
Kind
B2
Abstract

The present teaching relates to ranking search content. In one example, a plurality of documents is received to be ranked with respect to a query. Features are extracted from the query and the plurality of documents. The plurality of documents is ranked based on a ranking model and the extracted features. The ranking model is derived to remove one or more documents from the plurality of documents that are less relevant to the query and order remaining documents based on their relevance to the query. The ordered remaining documents are provided as a search result with respect to the query.

Claims (67)

1. A method, implemented on a machine having at least one

processor, storage, and a communication platform connected to a network for ranking search content, comprising:

receiving, by a computing device, a plurality of documents to be ranked with respect to a query, wherein each of the plurality of documents is represented by a corresponding uniform resource locator (URL);

extracting, by the computing device, features from the query, the plurality of documents, and a user profile;

obtaining assessment data associated with a plurality of query/URL pairs, wherein the plurality of query/URL pairs is classified into two groups: a first group including query/URL pairs each of which has a first score representing a higher relevance between query and URL in the pair, and a second group including query/URL pairs each of which has a second score representing a lower relevance between query and URL in the pair;

assigning one of the first and second scores for each of the plurality of query/URL pairs based on the assessment data;

retrieving features of each of the plurality of query/URL pairs from a database;

training, by the computing device, via machine learning, based on the first and second scores and the features of each of the plurality of query/URL pairs, a ranking model for ranking the plurality of URLs to map the features of the query/URL pair;

ranking, by the computing device based on the ranking model, the plurality of URLs by:

classifying the plurality of URLs into different classes based on the extracted features, wherein multiple URLs classified in first multiple of the different classes are deemed relevant to the query and multiple URLs classified in second multiple of the different classes are deemed irrelevant to the query,

assigning the first score to each of the multiple URLs classified in the first multiple classes such that all of the multiple URLs classified in the first multiple classes correspond to the same first score and assigning the second score to each of the multiple URLs classified in the second multiple classes such that all of the multiple URLs classified in the second multiple classes correspond to the same second score,

removing, based on the second scores, the multiple URLs classified in the second multiple classes, and

ranking the remaining URLs;

providing, by the computing device via a computer application to a user, content represented by the ranked remaining URLs as search results with respect to the query;

obtaining, by the computing device via the computer application, interactions of the user with respect to the search results; and

updating, by the computing device, the ranking model via training based on the interactions.

2. The method of claim 1 , wherein each of the first and second scores represents a degree of relevance between the query and each of the plurality of documents, wherein each of the first and second scores is calculated based on the ranking model and the extracted features; and

wherein the step of removing comprises filtering out the one or more URLs from the plurality of URLs that have scores less than a predetermined threshold.

3. The method of claim 1 , wherein the features extracted from the query and the plurality of documents represent at least one of the following:

a popularity online for each of the plurality of documents, and

a textual relevance between each document and the query.

4. The method of claim 1 , wherein the extracted features include past user behavior related to the plurality of URLs.

5. A system having at least one processor, storage, and a communication platform connected to a network for ranking search content, comprising:

a query and document analyzer configured for receiving a plurality of documents to be ranked with respect to a query, wherein each of the plurality of documents is represented by a corresponding uniform resource locator (URL);

a feature extractor configured for extracting features from the query, the plurality of documents, and a user profile;

a ranking model training engine configured for:

obtaining assessment data associated with a plurality of query/URL pairs, wherein the plurality of query/URL pairs is classified into two groups: a first group including query/URL pairs each of which has a first score representing a higher relevance between query and URL in the pair, and a second group including query/URL pairs each of which has a second score representing a lower relevance between query and URL in the pair;

assigning one of the first and second scores for each of the plurality of query/URL pairs based on the assessment data;

retrieving features of each of the plurality of query/URL pairs from a database; and

training, via machine learning based on the first and second scores and the features of each of the plurality of query/URL pairs, a ranking model for ranking the plurality of URLs to map the features of the query/URL pair;

a search result ranking unit configured for ranking, based on the ranking model, the plurality of URLs by:

classifying the plurality of URLs into different classes based on the extracted features, wherein multiple URLs classified in first multiple of the different classes are deemed relevant to the query and multiple URLs classified in second multiple of the different classes are deemed irrelevant to the query,

assigning the first score to each of the multiple URLs classified in the first multiple classes such that all of the multiple URLs classified in the first multiple classes correspond to the same first score and assigning the second score to each of the multiple URLs classified in the second multiple classes such that all of the multiple URLs classified in the second multiple classes correspond to the same second score,

removing, based on the second scores, the multiple URLs classified in the second multiple classes, and

ranking the remaining URLs; and

a search result filter configured for

providing, via a computer application to a user, content represented by the ranked remaining URLs as search results with respect to the query, and

obtaining, via the computer application, interactions of the user with respect to the search results,

wherein the ranking model training engine is further configured to update the ranking model via training based on the interactions.

6. The system of claim 5 , wherein:

each of the first and second scores represents a degree of relevance between the query and each of the plurality of documents, wherein each of the first and second scores is calculated based on the ranking model and the extracted features; and

the search result filter is configured for filtering out the one or more URLs from the plurality of URLs that have scores less than a predetermined threshold.

7. The system of claim 5 , wherein the features extracted from the query and the plurality of documents represent at least one of the following:

a popularity online for each of the plurality of documents, and

a textual relevance between each document and the query.

8. The system of claim 5 , wherein the extracted features include past user behavior related to the plurality of URLs.

9. A machine-readable, non-transitory and tangible medium having information recorded thereon for ranking search content, the information, when read by the machine, causes the machine to perform the following:

receiving a plurality of documents to be ranked with respect to a query, wherein each of the plurality of documents is represented by a corresponding uniform resource locator (URL);

extracting features from the query, the plurality of documents, and a user profile;

obtaining assessment data associated with a plurality of query/URL pairs, wherein the plurality of query/URL pairs is classified into two groups: a first group including query/URL pairs each of which has a first score representing a higher relevance between query and URL in the pair, and a second group including query/URL pairs each of which has a second score representing a lower relevance between query and URL in the pair;

assigning one of the first and second scores for each of the plurality of query/URL pairs based on the assessment data;

retrieving features of each of the plurality of query/URL pairs from a database;

training, via machine learning, based on the first and second scores and the features of each of the plurality of query/URL pairs, a ranking model for ranking the plurality of URLs to map the features of the query/URL pair;

ranking, based on the ranking model, the plurality of URLs by:

classifying the plurality of URLs into different classes based on the extracted features, wherein multiple URLs classified in first multiple of the different classes are deemed relevant to the query and multiple URLs classified in second multiple of the different classes are deemed irrelevant to the query,

assigning the first score to each of the multiple URLs classified in the first multiple classes such that all of the multiple URLs classified in the first multiple classes correspond to the same first score and assigning the second score to each of the multiple URLs classified in the second multiple classes such that all of the multiple URLs classified in the second multiple classes correspond to the same second score,

removing, based on the second scores, the multiple URLs classified in the second multiple classes, and

ranking the remaining URLs;

providing, via a computer application to a user, content represented by the ranked remaining URLs as search results with respect to the query;

obtaining, via the computer application, interactions of the user with respect to the search results; and

updating the ranking model via training based on the interactions.

10. The medium of claim 9 , wherein each of the first and second scores represents a degree of relevance between the query and each of the plurality of documents, wherein each of the first and second scores is calculated based on the ranking model and the extracted features; and

wherein the information, when read by the machine, further cause the machine to remove the one or more of the plurality of URLs by filtering out the one or more URLs from the plurality of URLs that have scores less than a predetermined threshold.

11. The medium of claim 9 , wherein the features extracted from the query and the plurality of documents represent at least one of the following:

a popularity online for each of the plurality of documents, and

a textual relevance between each document and the query.

12. The medium of claim 9 , wherein the extracted features include past user behavior related to the plurality of URLs.

Assignments (6)
PATENT SECURITY AGREEMENT (FIRST LIEN) Recorded Sep 29, 2022
From: YAHOO ASSETS LLC
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 061571/0773 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2021
From: YAHOO AD TECH LLC (FORMERLY VERIZON MEDIA INC.)
To: YAHOO ASSETS LLC
Reel/Frame 058982/0282 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: OATH INC.
To: VERIZON MEDIA INC.
Reel/Frame 054258/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2018
From: YAHOO HOLDINGS, INC.
To: OATH INC.
Reel/Frame 045240/0310 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2017
From: YAHOO! INC.
To: YAHOO HOLDINGS, INC.
Reel/Frame 042963/0211 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 4, 2015
From: YIN, DAWEI; WANG, PENGYUAN; OUYANG, HUA; CHANG, YI; LANGLOIS, JEAN-MARC
To: YAHOO! INC.
Reel/Frame 037209/0761 →
Cited By (1)
US 12,694,046