IP Library Granted Patent US 11,308,099
Granted Patent B2
US 11,308,099 · App. 16/898,289 · Granted Apr 19, 2022

Method of and system for ranking digital objects based on objective characteristic associated therewith

Inventors: Aleksey Ivanovich Ustimenko (Ivanovo, RU); Aleksandr Leonidovich Vorobyev (p Otradnoye, RU); Gleb Gennadevich Gusev (Moscow, RU); Pavel Viktorovich Serdyukov (Moscow, RU)
Assignee: YANDEX EUROPE AG
G06F16/24578G06N20/00G06Q30/0601
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,308,099
App. No.
16/898,289
Granted
Apr 19, 2022
Kind
B2
Abstract

A method and system for ranking digital object based on an objective characteristic associated therewith are provided. The method comprising: generating a set of digital objects based on a user request, the set of digital objects being rankable according to an objective characteristic thereof; receiving a filter request from the user, the filter request being based on a secondary characteristic of digital objects in the set of digital objects; determining object parameters for the digital objects in the set of digital objects, a given object parameter being indicative of a likelihood that an inclusion of a respective digital object in a re-ranked set of digital objects will increase a quality metric of the re-ranked set of digital objects; selecting digital objects based on object parameters; ranking digital objects based on respective values of the secondary characteristic, thereby generating the re-ranked set of digital objects.

Claims (37)

1. A method for ranking digital objects based on an objective characteristic associated therewith, the ranking being executed in response to a request received from an electronic device associated with a user, the method executable by a server communicatively accessible by the electronic device via a communication network, the method comprising:

based on the request, selecting a set of digital objects from a pool of digital objects, the set of digital objects being relevant to the request based on the objective characteristic of the digital objects, the set of digital objects being rankable in accordance with the objective characteristic thereof;

receiving, from the electronic device, a filter request for a re-ranked set of digital objects from the set of digital objects, the filter request being based on a secondary characteristic that is different from the objective characteristic;

selecting digital objects to be included in the re-ranked set of digital objects by:

determining, for a given digital object of the set of digital objects, a respective object parameter indicative of a likelihood that an inclusion of the given digital object in the re-ranked set of digital objects would increase a quality metric of the re-ranked set of digital objects considering the objective characteristic of the digital objects,

the quality metric being indicative of how relevant the re-ranked set of digital objects is to a search intent of the request;

selecting digital objects based on the respective object parameters; and

ranking the re-ranked set of digital objects based on the respective values of secondary characteristic,

thereby generating the re-ranked set of digital objects having the selected digital objects ranked in accordance with the secondary characteristic.

2. The method of claim 1 , wherein the objective characteristic is indicative of relevance of the given digital object to the request.

3. The method of claim 2 , wherein the secondary characteristic is not indicative of relevance of the given digital object to the request.

4. The method of claim 3 , wherein the secondary characteristic is indicative of a relative value parameter of the given digital object relative to an other one of the set of digital objects.

5. The method of claim 4 , wherein the secondary characteristic enables ranking digital objects in accordance with the secondary characteristic, which ranking is different from a ranking based on the objective characteristic.

6. The method of claim 4 , wherein the secondary characteristic is indicative of an inherent parameter of the given digital object and the other one of the set of digital objects.

7. The method of claim 6 , wherein the inherent parameter is not directly indicative of the relevance of the given digital object to the request.

8. The method of claim 1 , wherein the object parameter is determined by a Machine Learning Algorithm (MLA).

9. The method of claim 8 , wherein the method further comprises training the MLA.

10. The method of claim 9 , wherein the training the MLA comprises feeding the MLA a set of training digital objects, each of the set of training digital objects including: (i) a training digital object that is part of a training set of digital objects; (ii) a training request that was used to generate the training set of digital objects, (iii) an indication of a secondary characteristic of the training digital object; and (iv) an assessor-generated label indicative of whether the training digital object should be included in a sub-set of training digital objects considering the secondary characteristic and relevance of the training digital object to the training request.

11. The method of claim 8 , wherein the MLA is a gradient boosted decision tree based MLA.

12. The method of claim 1 , wherein the re-ranked set of digital objects has fewer digital objects than the set of digital objects.

13. The method of claim 1 , wherein the object parameter is based on a trade-off between a number of digital objects to be included in the re-ranked set of digital objects and relevance of included digital objects of the re-ranked set of digital objects.

14. The method of claim 13 , wherein the trade-off is for a sub-set of higher ranked positions of the re-ranked set of digital objects.

15. The method of claim 14 , wherein the sub-set of higher ranked positions is a top-N number of positions of the re-ranked set of digital objects.

16. The method of claim 1 , wherein the quality metric is based on a relevancy of the re-ranked set of digital objects to the request based on the objective characteristic.

17. The method of claim 1 , wherein the quality metric is based on a predicted value of the secondary characteristic of top N-digital objects of the re-ranked set of digital objects.

18. A system for ranking digital objects based on an objective characteristic associated therewith, the system comprising a hardware server coupled, via a communication network, to an electronic device associated with a user, the hardware server being configured to:

receive, from the electronic device, a request for ranking digital objects;

select, based on the request, a set of digital objects from a pool of digital objects, the set of digital objects being relevant to the request based on the objective characteristic of the digital objects, the set of digital objects being rankable in accordance with the objective characteristic thereof;

receive, from the electronic device, a filter request for a re-ranked set of digital objects from the set of digital objects, the filter request being based on a secondary characteristic that is different from the objective characteristic;

select digital objects to be included in the re-ranked set of digital objects by:

determine, for a given digital object of the set of digital objects, a respective object parameter indicative of a likelihood that an inclusion of the given digital object in the re-ranked set of digital objects would increase a quality metric of the re-ranked set of digital objects considering the objective characteristic of the digital objects,

the quality metric being indicative of how relevant the re-ranked set of digital objects is to a search intent of the request;

select digital objects based on the respective object parameters; and

rank the re-ranked set of digital objects based on the respective values of secondary characteristic,

thereby generating the re-ranked set of digital objects having the selected digital objects ranked in accordance with the secondary characteristic.

19. The system of claim 18 , wherein the hardware server is further configured to determine the object parameter using an MLA.

20. The system of claim 18 , wherein the hardware server is further configured to train the MLA by feeding the MLA a set of training digital objects, each of the set of training digital objects including: (i) a training digital object that is part of a training set of digital objects; (ii) a training request that was used to generate the training set of digital objects, (iii) an indication of a secondary characteristic of the training digital object; and (iv) an assessor-generated label indicative of whether the training digital object should be included in a sub-set of training digital objects considering the secondary characteristic and relevance of the training digital object to the training request.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2024
From: DIRECT CURSUS TECHNOLOGY L.L.C
To: Y.E. HUB ARMENIA LLC
Reel/Frame 068534/0619 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2023
From: YANDEX EUROPE AG
To: DIRECT CURSUS TECHNOLOGY L.L.C
Reel/Frame 065692/0720 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2022
From: CHINTALA, APOORVA
To: CLOCR, INC.
Reel/Frame 061596/0281 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2020
From: USTIMENKO, ALEKSEY IVANOVICH; VOROBYEV, ALEKSANDR LEONIDOVICH; GUSEV, GLEB GENNADEVICH; SERDYUKOV, PAVEL VIKTOROVICH
To: YANDEX LLC
Reel/Frame 053787/0842 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2020
From: YANDEX LLC
To: YANDEX EUROPE AG
Reel/Frame 053787/0996 →
Priority Claims (1)
RU RU2019128014 · Sep 5, 2019 · national
Continuity (1)
Related Publication 20210073238A1 · Mar 11, 2021