IP Library › Granted Patent US 11,995,530
Granted Patent B2
US 11,995,530 · App. 17/878,724 · Granted May 28, 2024

Systems and methods for providing feedback for artificial intelligence-based image capture devices

Inventors: Aaron Michael Donsbach (Seattle, WA); Christopher Breithaupt (Berkeley, CA); Li Zhang (Seattle, WA); Arushan Rajasekaram (Seattle, WA); Navid Shiee (Seattle, WA)
Assignee: GOOGLE LLC
G06N3/044G06N3/045G06N3/084H04N23/611H04N23/62H04N23/632H04N23/64
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,995,530
App. No.
17/878,724
Granted
May 28, 2024
Kind
B2
Abstract

The present disclosure provides systems and methods that provide feedback to a user of an image capture device that includes an artificial intelligence system that analyzes incoming image frames to, for example, determine whether to automatically capture and store the incoming frames. An example system can also, in the viewfinder portion of a user interface presented on a display, a graphical intelligence feedback indicator in association with a live video stream. The graphical intelligence feedback indicator can graphically indicate, for each of a plurality of image frames as such image frame is presented within the viewfinder portion of the user interface, a respective measure of one or more attributes of the respective scene depicted by the image frame output by the artificial intelligence system.

Claims (53)

1. A computing system, comprising:

an image capture system configured to capture a plurality of image frames;

an artificial intelligence system comprising one or more machine-learned models, the artificial intelligence system configured to analyze each of the plurality of image frames and to output, for each of the plurality of image frames, a respective measure of one or more attributes of a respective scene depicted by the image frame;

a display;

one or more processors; and

one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:

providing, in a viewfinder portion of a user interface presented on the display, a live video stream that depicts at least a portion of a current field of view of the image capture system, wherein the live video stream comprises the plurality of image frames; and

providing, in the viewfinder portion of the user interface presented on the display, a graphical intelligence feedback indicator in association with the live video stream, the graphical intelligence feedback indicator graphically indicating, for each of the plurality of image frames as such image frame is presented within the viewfinder portion of the user interface, the respective measure of the one or more attributes of the respective scene depicted by the image frame output by the artificial intelligence system;

wherein the graphical intelligence feedback indicator comprises a graphical bar that has a size that is positively correlated to and indicative of the respective measure of the one or more attributes of the respective scene depicted by the image frame currently presented in the viewfinder portion of the user interface.

2. The computing system of claim 1 , wherein the graphical bar comprises a horizontal bar at a bottom edge or a top edge of the viewfinder portion of the user interface.

3. The computing system of claim 1 , wherein the graphical bar has a center point and extends along a first axis, and wherein the graphical bar is fixed at the center point of the graphical bar and increases or decreases in size in both directions from the center point of the graphical bar along the first axis to indicate changes in the respective measure of the one or more attributes of the respective scene depicted by the image frame currently presented in the viewfinder portion of the user interface.

4. The computing system of claim 1 , wherein the graphical intelligence feedback indicator comprises a graphical shape and wherein an amount of the graphical shape that is filled with the graphical bar is positively correlated to and indicative of the respective measure of the one or more attributes of the respective scene depicted by the image frame currently presented in the viewfinder portion of the user interface.

5. The computing system of claim 1 , wherein the graphical intelligence feedback indicator comprises a meter that indicates a proximity of the artificial intelligence system to automatic capture and non-temporary storage of imagery.

6. The computing system of claim 1 , wherein the respective measure of the one or more attributes of the respective scene depicted by each image frame comprises:

a respective measure of one or more attributes of use of the respective scene as a self-portrait photograph; or

a respective measure of one or more attributes of use of the respective scene as a group photograph.

7. The computing system of claim 1 , wherein the respective measure of the one or more attributes of the respective scene depicted by each image frame comprises a respective measure of one or more attributes of use of the respective scene as a group photograph.

8. The computing system of claim 1 , wherein the computing system consists of a mobile computing device that includes the image capture system, the artificial intelligence system, the display, the one or more processors, and the one or more non-transitory computer-readable media.

9. The computing system of claim 8 , wherein the mobile computing device comprises a smartphone and the image capture system comprises a forward-facing camera that faces in a same direction as the display.

10. The computing system of claim 1 , wherein the one or more machine-learned models comprise one or more of:

a machine-learned pose detection model; or

a machine-learned facial expression model.

11. The computing system of claim 1 , wherein the presence of one or more of the following in the respective scene results in an increase in the respective measure of the one or more attributes of the respective scene output by the artificial intelligence system:

front facing faces;

posing faces;

faces with smiling facial expressions;

faces with eyes open;

faces with frontal gaze; or

faces with unusual facial expressions.

12. The computing system of claim 1 , wherein the operations further comprise:

receiving a user input that requests operation of the computing system in a photobooth mode; and

in response to the user input, operating the computing system in the photobooth mode, wherein said providing, in the viewfinder portion of the user interface presented on the display, the graphical intelligence feedback indicator in association with the live video stream is performed as part of the photobooth mode.

13. The computing system of claim 1 , wherein the graphical intelligence feedback indicator graphically indicates, for each of the plurality of image frames as such image frame is presented within the viewfinder portion of the user interface, a relative measure of the one or more attributes of the respective scene depicted by the image frame relative to the previous respective measures of the one or more attributes of respective image frames that have previously been presented within the viewfinder portion of the user interface.

14. The computing system of claim 1 , wherein the operations are performed in real-time as the image capture system captures the plurality of image frames.

15. The computing system of claim 1 , wherein the one or more attributes of the respective scene depicted by each image frame comprises a desirability of the respective scene depicted by each image frame or whether content depicted in the respective scene satisfies a photographic goal.

16. The computing system of claim 1 , wherein the one or more attributes of the respective scene depicted by each image frame comprises whether content depicted in the respective scene satisfies a photographic goal.

17. A computer-implemented method, the method comprising:

obtaining, by one or more computing devices, a real-time image stream comprising a plurality of image frames;

analyzing, by the one or more computing devices using one or more machine-learned models, each of the plurality of image frames to determine a respective image quality indicator that describes whether content depicted in the respective image frame satisfies a photographic goal; and

providing, by the one or more computing devices, a feedback indicator for display in association with the real-time image stream in a user interface, wherein the feedback indicator indicates the respective image quality indicator for each image frame while such image frame is presented in the user interface;

wherein the feedback indicator comprises a graphical bar that has a size that is positively correlated to and indicative of the image quality indicator of the respective scene depicted by the image frame currently presented in the viewfinder portion of the user interface.

18. The computer-implemented method of claim 17 , further comprising:

selecting, by the one or more computing devices, at least one of the plurality of image frames for non-temporary storage based at least in part its respective image quality indicator.

19. The computer-implemented method of claim 17 , wherein the photographic goal comprises a self-portrait or a group portrait.

20. A computing system, comprising:

an image capture system configured to capture a plurality of image frames;

an artificial intelligence system comprising one or more machine-learned models, the artificial intelligence system configured to analyze each of the plurality of image frames and to output, for each of the plurality of image frames, a respective measure of one or more attributes of a respective scene depicted by the image frame;

a display;

one or more processors; and

one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising:

providing, in a viewfinder portion of a user interface presented on the display, a live video stream that depicts at least a portion of a current field of view of the image capture system, wherein the live video stream comprises the plurality of image frames; and

providing an intelligence feedback indicator in association with the live video stream, the intelligence feedback indicator indicating, for each of the plurality of image frames as such image frame is presented within the viewfinder portion of the user interface, the respective measure of the one or more attributes of the respective scene depicted by the image frame output by the artificial intelligence system;

wherein the graphical intelligence feedback indicator comprises a graphical bar that has a size that is positively correlated to and indicative of the respective measure of the one or more attributes of the respective scene depicted by the image frame currently presented in the viewfinder portion of the user interface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2022
From: DONSBACH, AARON MICHAEL; BREITHAUPT, CHRISTOPHER; ZHANG, LI; RAJASEKARAM, ARUSHAN; SHIEE, NAVID
To: GOOGLE LLC
Reel/Frame 060758/0609 →
Continuity (3)
Continuation 17266957
Provisional Application 62742810 · Oct 8, 2018
Related Publication 20220366219A1 · Nov 17, 2022