IP Library › Granted Patent US 11,812,184
Granted Patent B2
US 11,812,184 · App. 17/106,851 · Granted Nov 7, 2023

Systems and methods for presenting image classification results

Inventors: Micah Price (The Colony, TX); Chi-San Ho (Allen, TX); Yue Duan (Plano, TX)
Assignee: Capital One Services, LLC
H04N5/272G06F3/0482G06F3/04817G06F8/65G06F9/451G06F16/51G06F16/535G06F16/538G06F18/24G06N3/04G06N3/08G06V10/764G06V10/82G06V2201/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,812,184
App. No.
17/106,851
Granted
Nov 7, 2023
Kind
B2
Abstract

An apparatus for performing image searches including a camera, storage devices storing a set of instructions, and a processor coupled to the at least one storage device and the camera. The instructions configure the at least one processor to perform operations including identifying attributes of the captured image using a classification model; identifying first results based on the identified attributes; selecting a subset of first results based on corresponding probability scores, generating a first graphical user interface including interactive icons corresponding to first results in the subset, an input icon, and a first button. The operations may also include receiving a selection of the first button, performing a search to identify second results, and generating a second graphical user interface displaying the second results.

Claims (89)

1. A system for generating and implementing patches to improve classification model results based on user feedback through input icons, the system comprising:

a camera;

one or more processors; and

one or more memory devices storing instructions that, when executed by the one or more processors, configure the one or more processors to perform operations comprising:

capturing an image with the camera;

generating a first graphical user interface comprising:

one or more first interactive icons corresponding to first results, the first results comprising object recognition results based on attributes identified in the image using a classification model, wherein the classification model comprises a convolutional neural network and associated model hyperparameters, and wherein the associated model hyperparameters comprise at least one of a number of layers, a number of nodes, and an indication of whether the network is fully connected;

an input icon;

and a first button;

upon receiving a user selection of at least one of the first interactive icons:

performing a search to identify second results, the search being based on the selected at least one of the first interactive icons; and

generating a second graphical user interface displaying the second results, the second graphical user interface being different from the first graphical user interface; and

upon receiving a user selection of the first button:

determining whether the input icon is empty;

in response to determining the input icon is not empty, transmitting, to a server, the image and content in the input icon;

receiving, from the server, a patch for the classification model, the patch comprising updated model hyperparameters and a classification model exception for the identified attributes, and wherein the patch includes a script that modifies a response of the classification model to images with attributes including at least one of make, model, trim and color;

based on the patch, retraining the classification model to include the updated model hyperparameters such that the response of the classification model to images with attributes including at least one of make, model, trim and color is modified, wherein retraining the classification model to include the updated model hyperparameters comprises developing the convolutional neural network using backpropagation with gradient descent based on a training dataset; and

performing a conditional routine to substitute third results based on the classification model exception.

2. The system of claim 1 , wherein the classification model comprises a convolutional neural network.

3. The system of claim 2 , wherein the patch comprises updates for connection layers of the convolutional neural network.

4. The system of claim 1 , wherein the second graphical user interface comprises:

vehicle images associated with the second results;

vehicle conditions associated with the second results; and

distances associated with the second results.

5. The system of claim 4 , wherein the second graphical interface further comprises financing option icons for the second results.

6. The system of claim 1 , wherein the patch comprises model exceptions based on the content in the input icon.

7. The system of claim 1 , wherein the operations further comprise updating the classification model by running the patch.

8. The system of claim 1 , wherein the patch is configured to automatically execute commands and invoke patch management systems in an operating system of the one or more processors.

9. The system of claim 1 , wherein the interactive icons display thumbnails of vehicles identified as preliminary results.

10. The system of claim 9 , wherein the interactive icons are configured to change color and transparency when selected.

11. The system of claim 1 , wherein:

the first graphical user interface further comprises a second button; and

the operations further comprise:

upon receiving a user selection of the second button, transmitting an error message to the server.

12. The system of claim 1 , wherein generating the second graphical user interface comprises displaying the second results in a ranking based on financing availability.

13. The system of claim 1 , wherein:

the first graphical user interface further comprises a second button; and

the operations further comprise:

upon receiving a user selection of the second button:

transmitting a repopulate request to the server;

removing the one or more first interactive icons from the first graphical user interface; and

displaying second interactive icons in the first graphical user interface.

14. The system of claim 13 , wherein:

the first graphical user interface further comprises a third button; and

the operations further comprise:

upon receiving a user selection of the third button:

transmitting, to the server, a query for available vehicles without filtering conditions.

15. The system of claim 1 , wherein:

the first graphical user interface further comprises a second button; and

the operations further comprise:

upon receiving a user selection of the second button, generating a third graphical user interface displaying an augmenter reality application.

16. The system of claim 1 , wherein generating the first graphical user interface comprises:

retrieving visualization preferences from a local memory; and

determining the first results by truncating preliminary search results based on the visualization preferences.

17. The system of claim 1 , wherein generating the first graphical user interface comprises preselecting at least one of the one or more first interactive icons based on confidence levels of the first results.

18. The system of claim 17 , wherein preselected first interactive icons are displayed in a different color in the first graphical user interface.

19. A computer-implemented method for generating and implementing patches to improve classification model results based on user feedback through input icons, the computer-implemented method comprising:

capturing an image with a camera;

generating a first graphical user interface comprising:

one or more interactive icons corresponding to first results, the first results comprising object recognition results based on attributes identified in the image using a classification model, wherein the classification model comprises a convolutional neural network and associated model hyperparameters, and wherein the associated model hyperparameters comprise at least one of a number of layers, a number of nodes, and an indication of whether the network is fully connected;

an input icon; and

a first button;

upon receiving a user selection of at least one of the interactive icons:

performing a search to identify second results, the search being based on the selected at least one of the interactive icons; and

generating a second graphical user interface displaying the second results, the second graphical user interface being different from the first graphical user interface; and

upon receiving a user selection of the first button:

determining whether the input icon is empty;

in response to determining the input icon is not empty, transmitting, to a server, the image and content in the input icon;

receiving, from the server, a patch for the classification model, the patch comprising updated model hyperparameters and a classification model exception for the identified attributes, and wherein the patch includes a script that modifies a response of the classification model to images with attributes including at least one of make, model, trim and color;

based on the patch, retraining the classification model to include the updated model hyperparameters such that the response of the classification model to images with attributes including at least one of make, model, trim and color is modified, wherein retraining the classification model to include the updated model hyperparameters comprises developing the convolutional neural network using backpropagation with gradient descent based on a training dataset; and

performing a conditional routine to substitute third results based on the classification model exception.

20. An apparatus for generating and implementing patches to improve classification model results based on user feedback through input icons, the apparatus comprising;

a camera;

at least one memory storing instructions; and

at least one processor programmed by the instructions to:

capture an image with the camera;

generate a first graphical user interface comprising:

one or more interactive icons corresponding to first results, the first results comprising object recognition results based on attributes identified in the image using a classification model, wherein the classification model comprises a convolutional neural network and associated model hyperparameters, and wherein the associated model hyperparameters comprise at least one of a number of layers, a number of nodes, and an indication of whether the network is fully connected;

an input icon; and

a first button;

upon receiving a user selection of at least one of the interactive icons:

perform a search to identify second results, the search being based on the selected at least one of the interactive icons; and

generate a second graphical user interface displaying the second results, the second graphical user interface being different from the first graphical user interface; and

upon receiving a user selection of the first button:

determine whether the input icon is empty;

in response to determining the input icon is not empty, transmit the image and content in the input icon to a server;

receive, from the server, a patch for the classification model, the patch comprising updated model hyperparameters and a classification model exception for the identified attributes, and wherein the patch includes a script that modifies a response of the classification model to images with attributes including at least one of make, model, trim and color;

based on the patch, retraining the classification model to include the updated model hyperparameters such that the response of the classification model to images with attributes including at least one of make, model, trim and color is modified, wherein retraining the classification model to include the updated model hyperparameters comprises developing the convolutional neural network using backpropagation with gradient descent based on a training dataset; and

performing a conditional routine to substitute third results based on the classification model exception.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2020
From: PRICE, MICAH; HO, CHI-SAN; DUAN, YUE
To: CAPITAL ONE SERVICES, LLC
Reel/Frame 054491/0848 →
Continuity (2)
Continuation 16534375 · Aug 7, 2019
Related Publication 20210081446A1 · Mar 18, 2021