IP Library Granted Patent US 11,835,995
Granted Patent B2
US 11,835,995 · App. 17/668,727 · Granted Dec 5, 2023

Automatic unstructured knowledge cascade visual search

Inventors: Eran Nussinovitch (New York, NY); Michael Gormish (Wilmington, DE)
Assignee: Clarifai, Inc.
G06F16/334G06F16/338
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,835,995
App. No.
17/668,727
Granted
Dec 5, 2023
Kind
B2
Abstract

Systems, methods and computer program code are provided to perform visual searches.

Claims (59)

1. A computer implemented method to perform a visual search, the method comprising:

receiving, by a search platform, information identifying an input image;

processing, by the search platform, the input image using a model to generate at least a first prediction that represents a first predicted cluster from among a plurality of clusters in which the input image may be grouped and a representation of accuracy associated with the at least first prediction, wherein the representation of accuracy indicates the at least first predicted cluster has a high degree of relevance to the input image;

automatically selecting, by the search platform, a visual search application that is relevant to the first predicted cluster, the visual search application selected from among a plurality of visual search applications;

automatically initiating, by the search platform, a search of a datastore using the input image, the search performed using the selected visual search application; and

outputting a set of image results predicted to be visually similar to the input image.

2. The computer implemented method of claim 1 , wherein the model is a classification model.

3. The computer implemented method of claim 2 , wherein the classification model is a multiclass classification model trained on a set of clusters.

4. The computer implemented method of claim 3 , wherein the set of clusters are created by extracting features from a base model and clustering the features in unique groups.

5. The computer implemented method of claim 4 , wherein the model is trained on the set of clusters.

6. The computer implemented method of claim 4 , further comprising:

generating, for each cluster in the set of clusters, a visual search application.

7. The computer implemented method of claim 1 , wherein outputting a set of results further comprises:

outputting a confidence score for each result in the set of results.

8. The computer implemented method of claim 7 , wherein outputting a set of results further comprises:

ranking each result in the set of results by the confidence score and presenting the result with the highest confidence score as the most similar result.

9. The computer implemented method of claim 1 , further comprising:

processing the input using a model to generate at least a second prediction that represents a second predicted cluster in which the input may be grouped, and a representation of accuracy associated with the at least second prediction.

10. The computer implemented method of claim 9 , wherein initiating a search of a datastore using the input, the search performed using an application associated with the first predicted cluster further comprises:

determining that the representation of accuracy associated with the at least first prediction is greater than the representation of accuracy associated with the at least second prediction.

11. The computer implemented method of claim 1 , wherein each of the plurality of a visual search applications is created using a method comprising:

identifying a set of input data including a first plurality of images;

grouping the first plurality of images in one or more clusters; and

creating a search application component for each of the one or more clusters.

12. The computer-implemented method of claim 11 , further comprising:

processing the first plurality of images using a predictive model; and

extracting features from the first plurality of images;

wherein the grouping is based at least in part on the extracted features.

13. The computer-implemented method of claim 12 , wherein the predictive model is a base classifier model.

14. The computer-implemented method of claim 11 , further comprising:

identifying meta-data associated with each of the first plurality of images;

wherein the grouping is based at least in part on the meta-data.

15. The computer-implemented method of claim 11 , wherein the meta-data includes data identifying at least one of a category and a label.

16. The computer-implemented method of claim 11 , wherein the meta-data includes data obtained by performing an optical character recognition process.

17. The computer-implemented method of claim 11 , further comprising:

training a multiclass classifier model using each of the clusters.

18. The computer-implemented method of claim 17 , wherein the multiclass classifier model is trained to classify an input as associated with at least one of the search application components.

19. The computer-implemented method of claim 11 , further comprising:

deploying the visual search application for use in performing visual searches on inputs received from one or more remote user devices.

20. A system comprising:

an input device to receive an input image from a remote user device;

a processing unit; and

a memory storage device including program code that when executed by the processing unit causes to the system to:

process the input image using a model to generate at least a first prediction that represents a first predicted cluster from among a plurality of clusters in which the input image may be grouped and a representation of accuracy associated with the at least first prediction, wherein the representation of accuracy indicates the at least first predicted cluster has a high degree of relevance to the input image;

automatically select a visual search application that is relevant to the first predicted cluster, the visual search application selected from among a plurality of visual search applications;

automatically initiate a search of a datastore using the input image, the search performed using the selected visual search application; and

output a set of image results predicted to be visually similar to the input image.

21. The system of claim 20 wherein the model is a classification model.

22. The system of claim 21 , wherein the classification model is a multiclass classification model trained on a set of clusters.

23. The system of claim 22 , wherein the set of clusters are created by extracting features from a base model and clustering the features in unique groups.

24. The system of claim 23 , wherein the model is trained on the set of clusters.

25. The system of claim 22 , further including program code that when executed by the processing unit causes the system to:

generate, for each cluster in the set of clusters, a visual search application.

26. The system of claim 20 , further including program code that when executed by the processing unit causes the system to:

generate at least a second prediction that represents a second predicted cluster in which the input may be grouped;

cause information to be displayed on the remote user device identifying the first predicted cluster and the second predicted cluster; and

receive, from the remote user device, a selection of one of the first predicted cluster and the second predicted cluster as more accurately satisfying a user's search intent.

27. The system of claim 26 , further including program code that when executed by the processing unit causes the system to:

initiate a search of the datastore using the input, the search performed using a visual search application associated with the selected one of the first predicted cluster and the second predicted cluster.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2026
From: CLARIFAI, INC.
To: NEBIUS BV
Reel/Frame 075712/0109 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 10, 2022
From: NUSSINOVITCH, ERAN; GORMISH, MICHAEL
To: CLARIFAI, INC.
Reel/Frame 058972/0813 →
Continuity (1)
Related Publication 20230252059A1 · Aug 10, 2023
Cited By (1)
US 12,675,388