IP Library › Granted Patent US 12682674
Granted Patent B2
US 12682674 · App. 17/802,060 · Granted Jul 14, 2026

Systems and methods for improved computer vision in on-device applications

Inventors: Qifei Wang (Sunnyvale, CA); Alexander Kuznetsov (Mountain View, CA); Alec Michael Go (Milpitas, CA); Grace Chu (Santa Clara, CA); Eunyoung Kim (Mountain View, CA); Feng Yang (Sunnyvale, CA); Andrew Gerald Howard (Culver City, CA); Jeffrey M. Gilbert (Los Altos, CA)
Assignee: GOOGLE LLC
G06V30/413G06V10/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12682674
App. No.
17/802,060
Granted
Jul 14, 2026
Kind
B2
Abstract

The present disclosure is directed to object and/or character recognition for use in applications such as computer vision. Advantages of the present disclosure include lightweight functionality that can be used on devices such as smart phones. Aspects of the present disclosure include a sequential architecture where a lightweight machine-learned model can receive an image, detect whether an object is present in one or more regions of the image, and generate an output based on the detection. This output can be applied as a filter to remove image data that can be neglected for more memory intensive machine-learned models applied downstream.

Claims (54)

1 . A computer-implemented method for improving object detection efficiency, the method comprising:

obtaining, by a computing system comprising one or more computing devices, an image;

processing, by the computing system, the image with a first machine-learned model to generate a prediction as to whether one or more of pre-defined regions of the image comprise a pre-defined region of the image in which at least one object included in a class of objects is present;

generating a dataset that represents the one or more of the pre-defined regions where the at least one object included in the class of objects is predicted to be present; and

processing, by the computing system, the dataset that represents the one or more of the pre-defined regions where the at least one object included in the class of objects is predicted to be present, with a second machine-learned model to generate a label for the at least one object,

wherein a number of the one or more of the pre-defined regions of the image represented in the dataset processed with the second machine-learned model to generate the label is less than a number of the pre-defined regions processed with the first machine-learned model to generate the dataset, and wherein processing a reduced number of regions with the second machine-learned model decreases the computational load of the computing system for performing object detection.

2 . The computer-implemented method of claim 1 , wherein the dataset is representative of only the one or more of the pre-defined regions of the image where the at least one object included in the class of objects is present, further comprising:

generating, by the computing system and based at least in part on an output of the first machine-learned model, the dataset representative of only the one or more of the pre-defined regions of the image where the at least one object included in the class of objects is present.

3 . The computer-implemented method of claim 2 , wherein the dataset comprises a masked version or a cropped version of the image.

4 . The computer-implemented method of claim 1 , wherein the label comprises a bounding box containing all of the pre-defined regions where the at least one object included in the class of objects is present.

5 . The computer-implemented method of claim 1 , wherein processing, by the computing system, the dataset that represents the one or more of the pre-defined regions in which the at least one object included in the class of objects is present comprises:

at least one of masking or cropping data in one or more other pre-defined regions in which the at least one object included in the class of objects is not present; and

in response to the at least one of the masking or the cropping of the data in the one or more other pre-defined regions, processing, by the computing system, the dataset that represents the one or more of the pre-defined regions in which the at least one object included in the class of objects is present with the second machine-learned model that ignores the data in the one or more other pre-defined regions.

6 . The computer-implemented method of claim 1 , wherein the class of objects consists of one or more objects from a group: alphabetic characters, numbers, punctuation, words, machine-readable code, and faces.

7 . The computer-implemented method of claim 1 , further comprising:

partitioning, by the computing system, the image into the pre-defined regions to be processed by the first machine-learned model,

wherein partitioning the image into the pre-defined regions that together encompass the whole of the image comprises:

applying at least one horizontal partition to divide the image into an upper region and a lower region; and

applying at least one vertical partition to divide the image into a left region and a right region.

8 . The computer-implemented method of claim 7 , wherein the at least one horizontal partition and the at least one vertical partition are static.

9 . The computer-implemented method of claim 7 , wherein the at least one horizontal partition and the at least one vertical partition are adjustable.

10 . The computer-implemented method of claim 7 wherein the at least one horizontal partition and the at least one vertical partition comprise a learned parameter,

wherein the learned parameter is determined by a third machine-learned model configured to:

generate a heat map of objects included in at least one object for an example image; and

partition in the image into one or more regions based on a constraint, wherein the constraint comprises: maximizing heat per box and minimizing the number of the one or more regions.

11 . The computer-implemented method of claim 1 , wherein the first machine-learned model is configured to generate an output used to generate the dataset that represents the one or more of the pre-defined regions of the image in which the at least one object included in the class of objects is present, and

wherein the second machine-learned model is configured to generate a characteristic of the output.

12 . The computer-implemented method of claim 11 , wherein the characteristic comprises an orientation.

13 . The computer-implemented method of claim 1 , wherein the second machine-learned model is configured to perform optical character recognition (OCR), face detection, or facial recognition.

14 . A computing system configured to perform object detection, the computing system comprising:

one or more processors;

one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors cause the computing system to perform operations, the operations comprising:

obtaining an image depicting at least one object in a class of objects;

partitioning the image into regions that together encompass the whole of the image;

processing each of the regions with a first machine-learned model to generate a prediction as to whether one or more of the regions comprise a pre-defined region of the image in which one of the at least one object included in the class of objects is present;

generating a dataset that represents the one or more of the pre-defined regions where the at least one object included in the class of objects is predicted to be present; and

processing the dataset that represents the one or more of the regions where the one of the at least one object included in the class of objects is predicted to be present, with a second machine-learned model to generate a label for the one of the at least one object,

wherein a number of the one or more of the regions represented in the dataset processed with the second machine-learned model to generate the label is less than a number of the regions processed with the first machine-learned model to generate the dataset, and wherein processing a reduced number of regions with the second machine-learned model decreases the computational load of the computing system for performing object detection.

15 . The computing system of claim 14 , wherein the dataset is representative of only the one or more of the regions of the image where the one of the at least one object included in the class of objects is present, the operations further comprising:

generating, based at least in part on an output of the first machine-learned model, the dataset representative of only the one or more of the regions of the image where the one of the at least one object included in the class of objects is present.

16 . The computing system of claim 14 , wherein the one or more non-transitory computer-readable media are stored on a local device, and wherein the local device is a smartphone.

17 . One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

obtaining an image depicting at least one object in a class of objects;

partitioning the image into regions that together encompass the whole of the image;

processing each of the regions with a first machine-learned model to generate a prediction as to whether one or more of the regions comprise a pre-defined region of the image in which one of the at least one object included in the class of objects is present;

generating a dataset that represents the one or more of the pre-defined regions where the at least one object included in the class of objects is predicted to be present; and

processing the dataset that represents the one or more of the regions where the one of the at least one object included in the class of objects is predicted to be present, with a second machine-learned model to generate a label for the one of the at least one object,

wherein a number of the one or more of the regions represented in the dataset processed with the second machine-learned model to generate the label is less than a number of the regions processed with the first machine-learned model to generate the dataset, and wherein processing a reduced number of regions with the second machine-learned model decreases the computational load for performing object detection.

18 . The computer-implemented method of claim 1 , wherein the dataset is representative of only the one or more of the pre-defined regions of the image, further comprising:

generating, by the computing system and based at least in part on an output of the first machine-learned model, the dataset representative of only the one or more of the pre-defined regions of the image where the one of the at least one object included in the class of objects is present; and

providing, to the second machine-learned model, the dataset representative of only the one or more of the pre-defined regions of the image where the one of the at least one object included in the class of objects is present.

19 . The computer-implemented method of claim 1 , further comprising:

classifying, by the computing system and with analysis via the first machine-learned model, the one or more of the pre-defined regions of the image based at least in part on whether the at least one object included in the class of objects is present in the one or more of the pre-defined regions of the image.

20 . The computer-implemented method of claim 1 , wherein the second machine-learned model is configured to determine training data comprising the label for the at least one object in the respective pre-defined region of the image in response to classifying, via the first machine-learned model, the one or more of the pre-defined regions.