IP Library › Granted Patent US 11,853,852
Granted Patent B2
US 11,853,852 · App. 16/662,660 · Granted Dec 26, 2023

Systems and methods for preventing machine learning models from negatively affecting mobile devices through intermittent throttling

Inventors: Micah Price (Plano, TX); Sunil Subrahmanyam Vasisht (Flowermound, TX); Stephen Michael Wylie (Carrollton, TX); Geoffrey Dagley (McKinney, TX); Qiaochu Tang (The Colony, TX); Jason Richard Hoover (Grapevine, TX)
Assignee: Capital One Services, LLC
G06N20/00G06F1/206G06F1/3212G06F9/541H04N23/61H04W52/0209H04W52/0251
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,853,852
App. No.
16/662,660
Granted
Dec 26, 2023
Kind
B2
Abstract

Systems and methods for preventing machine learning models from negatively affecting mobile devices are provided. For example, a mobile device including a camera, memory devices, and one or more processors are provided. In some embodiments, the processors may be configured to provide images captured by the camera to a machine learning model at a first rate. The processors may also be configured to determine whether one or more of the images includes an object. If one or more of the images includes the object, the processors may be further configured to adjust the first rate of providing the images to the machine learning model to a second rate, and in some embodiments, determine whether to adjust the second rate of providing the images to the machine learning model to a third rate based on output received from the machine learning model.

Claims (44)

1. A method for preventing machine learning models from negatively affecting mobile devices through intermittent throttling, comprising:

receiving one or more movement characteristic measurements from a sensor included in a mobile device;

determining a stillness of the mobile device based on the received one or more movement characteristic measurements;

capturing at least one image with a camera;

determining, using a first machine learning model, whether the at least one image includes an object by retrieving, from a database, at least one property of the object, and comparing at least one property of the at least one image with the at least one property of the object;

providing the at least one image captured by the camera to the first machine learning model or a second machine learning model at a rate determined based on the determined stillness of the mobile device, wherein more images captured by the camera are provided to the first machine learning model or the second machine learning model if the one or more movement characteristic measurements are below a movement threshold level; and

adjusting the rate at a first time based on a determination that the at least one image includes a first category of object type and not a second category of object type, and adjusting the rate at a second time based on an output from the first machine learning model or the second machine learning model, the output including an identified type of the object within the first category and a confidence score indicating a probability that the object matches the identified type of the object.

2. The method of claim 1 , wherein the one or more movement characteristic measurements include at least one of a measurement of gravitational force on the mobile device, a measurement speed of the mobile device, a speed of the mobile device, an acceleration of the mobile device, a rotational speed of the mobile device, a rotational acceleration of the mobile device, or a displacement of the mobile device.

3. The method of claim 1 , wherein the first or the second machine learning model further uses historical data associated with objects for determining whether the at least one image includes the object.

4. The method of claim 1 , further comprising determining a movement threshold based on averaging historical movement characteristic measurements of the mobile device.

5. The method of claim 4 , wherein determining a stillness of the mobile device further comprises determining that the one or more movement characteristic measurements exceed the movement threshold.

6. The method of claim 4 , wherein determining a stillness of the mobile device further comprises determining that an average movement characteristic measurement exceeds the movement threshold.

7. The method of claim 4 , wherein determining the movement threshold further comprises:

identifying a movement characteristic measurements range based on historical movement characteristic measurements; and

identifying a movement threshold based on the movement characteristic measurements range.

8. The method of claim 4 , further comprising adjusting the movement threshold based on at least one of the sensor or a sensor application running on the mobile device.

9. The method of claim 8 , wherein adjusting the movement threshold further comprises increasing the movement threshold when an available battery life of the mobile device is below a certain level.

10. The method of claim 8 , wherein adjusting the movement threshold further comprises decreasing the movement threshold when an available battery life of the mobile device is above a certain level.

11. The method of claim 1 , further comprising:

increasing the determined rate if the confidence score exceeds a confidence threshold.

12. The method of claim 1 , further comprising:

receiving an additional movement characteristic measurement from the sensor after providing the at least one image to the first machine learning model or the second machine learning model; and

determining a stillness of the mobile device based on the additional movement characteristic measurement.

13. The method of claim 1 , further comprising:

adjusting, based on the determined stillness of the mobile device, the determined rate to a second rate if the at least one image includes the object.

14. The method of claim 13 , further comprising determining whether to adjust the second rate to a third rate based on the confidence score received from the first machine learning model or the second machine learning model.

15. The method of claim 13 , wherein the second rate is greater than the determined rate.

16. The method of claim 1 , wherein determining whether the at least one image includes an object further comprises determining that the at least one image includes an object based on determining whether the object is depicted in a particular section of the at least one image.

17. The method of claim 1 , wherein adjusting the rate further comprises deactivating an input device based on the output.

18. The method of claim 1 , wherein the first machine learning model or the second machine learning model is configured to run on the mobile device using a mobile machine learning model framework.

19. A method for preventing machine learning models from negatively affecting mobile devices through intermittent throttling, comprising:

receiving one or more gravitational force measurements from an accelerometer included in a mobile device;

determining a stillness of the mobile device based on the received gravitational force measurements;

capturing at least one image with a camera;

determining, using a first machine learning model, whether the at least one image includes an object by retrieving, from a database, at least one property of the object, and comparing at least one property of the at least one image with the at least one property of the object;

providing the at least one image captured by the camera to the first machine learning model or a second machine learning model at a rate determined based on the determined stillness of the mobile device, wherein more images captured by the camera are provided to the first machine learning model or the second machine learning model if the gravitational force measurements are below a movement threshold level; and

adjusting the rate at a first time based on a determination that the at least one image includes a first category of object type and not a second category of object type, and adjusting the rate at a second time based on an output from the first machine learning model or the second machine learning model, the output including an identified type of the object within the first category and a confidence score indicating a probability that the object matches the identified type of the object.

20. A non-transitory computer-readable medium storing instructions to execute a method by a processor to prevent machine learning models from negatively affecting mobile devices through intermittent throttling, the method comprising:

receiving one or more movement characteristic measurements from a sensor included in a mobile device;

determining a stillness of the mobile device based on the received one or more movement characteristic measurements;

capturing at least one image with a camera;

determining, using a first machine learning model, whether the at least one image includes an object by retrieving, from a database, at least one property of the object, and comparing at least one property of the at least one image with the at least one property of the object;

providing the at least one image captured by the camera to the first machine learning model or a second machine learning model at a rate determined based on the determined stillness of the mobile device, wherein more images captured by the camera are provided to the first machine learning model or the second machine learning model if the one or more movement characteristic measurements are below a movement threshold level; and

adjusting the rate at a first time based on a determination that the at least one image includes a first category of object type and not a second category of object type, and adjusting the rate at a second time based on an output from the first machine learning model or the second machine learning model, the output including an identified type of the object within the first category and a confidence score indicating a probability that the object matches the identified type of the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 6, 2023
From: PRICE, MICAH; VASISHT, SUNIL SUBRAHMANYAM; WYLIE, STEPHEN MICHAEL; DAGLEY, GEOFFREY; TANG, QIAOCHU; HOOVER, JASON RICHARD
To: CAPITAL ONE SERVICES, LLC
Reel/Frame 065474/0658 →
Continuity (3)
Continuation 15913151 · Mar 6, 2018
Continuation 15912239 · Mar 5, 2018
Related Publication 20200057962A1 · Feb 20, 2020