Granted Patent B2

US 12,711,663 · App. 18/364,579 · Granted Aug 18, 2026

Inventors: Daniel V. Klein (Pittsburgh, PA); Ramprasad Sedouram (Bengaluru, IN)

Assignee: Google LLC

G06T7/90G06T11/00G06V20/698G06F3/16G06T2200/24G08B6/00

Abstract

Claims (56)

1 . A computer-implemented method executed on data processing hardware that causes the data processing hardware to perform operations comprising:

receiving image data for a plurality of objects of interest of a user;

receiving a query from the user referring to the plurality of objects and requesting a digital assistant to discern a color matching insight associated with the plurality of objects referred to by the query by requesting the digital assistant to determine whether corresponding colors of the plurality of objects will coordinate with each other;

processing the query and the image data to:

identify, for each particular object of the plurality of objects referred to by the query, one or more corresponding colors of the particular object; and

determine, based on the corresponding colors, the color matching insight associated with the plurality of objects for the digital assistant to discern, wherein determining the color matching insight comprises comparing the corresponding colors identified for the plurality of objects to determine whether the corresponding colors of the plurality of objects aesthetically coordinate with each other based on color theory principles; and

generating, for output from a user device associated with the user, content indicating the discerned color matching insight associated with the plurality of objects,

wherein the query refers to, but does not explicitly identify, the plurality of objects associated with the color matching insight the digital assistant is requested to discern, and

wherein the digital assistant is trained using machine learning.

2 . The computer-implemented method of claim 1 , wherein the image data comprises image data captured by an image capture device in communication with the data processing hardware.

3 . The computer-implemented method of claim 1 , wherein the image data comprises:

first image data captured by an image capture device in communication with the data processing hardware, the first image data of one or more of the plurality of objects referred to in the query; and

second image data stored on memory hardware in communication with the data processing hardware, the second image data of one or more of the plurality of objects referred to in the query.

4 . The computer-implemented method of claim 1 , wherein the image data comprises image data obtained via a network from a remote datastore.

5 . The computer-implemented method of claim 1 , wherein identifying the one or more corresponding colors of the particular object comprises performing image processing on at least a portion of the image data.

6 . The computer-implemented method of claim 1 , wherein identifying the one or more corresponding colors of the particular object comprises:

determining, based on the image data, an identification of the particular object; and

obtaining, based on the identification of particular object, description information comprising the one or more corresponding colors.

7 . The computer-implemented method of claim 1 , wherein:

generating content indicating the discerned color matching insight comprises generating graphical content that indicates the discerned color matching insight; and

the graphical content is superimposed in a graphical user interface displayed on a screen of the user device.

8 . The computer-implemented method of claim 1 , wherein:

generating content indicating the discerned color matching insight comprises generating audible content indicating the discerned color matching insight; and

the audible content is audibly output from the user device.

9 . The computer-implemented method of claim 1 , wherein:

generating content indicating the discerned color matching insight comprises generating haptic feedback indicating the discerned color matching insight; and

the haptic feedback is output by the user device.

10 . A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:

receiving image data for a plurality of objects of interest of a user;

receiving a query from the user referring to the plurality of objects and requesting a digital assistant to discern a color matching insight associated with the plurality of objects referred to by the query by requesting the digital assistant to determine whether corresponding colors of the plurality of objects will coordinate with each other;

processing the query and the image data to:

identify, for each particular object of the plurality of objects referred to by the query, one or more corresponding colors of the particular object; and

determine, based on the corresponding colors, the color matching insight associated with the plurality of objects for the digital assistant to discern, wherein determining the color matching insight comprises, comparing the corresponding colors identified for the plurality of objects to determine whether the corresponding colors of the plurality of objects aesthetically coordinate with each other based on color theory principles; and

generating, for output from a user device associated with the user, content indicating the discerned color matching insight associated with the plurality of objects,

wherein the query refers to, but does not explicitly identify, the plurality of objects associated with the color matching insight the digital assistant is requested to discern, and

wherein the digital assistant is trained using machine learning.

11 . The system of claim 10 , wherein the image data comprises image data captured by an image capture device in communication with the data processing hardware.

12 . The system of claim 10 , wherein the image data comprises:

first image data captured by an image capture device in communication with the data processing hardware, the first image data of one or more of the plurality of objects referred to in the query; and

second image data stored on memory hardware in communication with the data processing hardware, the second image data of one or more of the plurality of objects referred to in the query.

13 . The system of claim 10 , wherein the image data comprises image data obtained via a network from a remote datastore.

14 . The system of claim 10 , wherein identifying the one or more corresponding colors of the particular object comprises performing image processing on at least a portion of the image data.

15 . The system of claim 10 , wherein identifying the one or more corresponding colors of the particular object comprises:

determining, based on the image data, an identification of the particular object; and

obtaining, based on the identification of particular object, description information comprising the one or more corresponding colors.

16 . The system of claim 10 , wherein:

generating content indicating the discerned color matching insight comprises generating graphical content that indicates the discerned color matching insight; and

the graphical content is superimposed in a graphical user interface displayed on a screen of the user device.

17 . The system of claim 10 , wherein:

generating content indicating the discerned color matching insight comprises generating audible content indicating the discerned color matching insight; and

the audible content is audibly output from the user device.

18 . The system of claim 10 , wherein:

generating content indicating the discerned color matching insight comprises generating haptic feedback indicating the discerned color matching insight; and

the haptic feedback is output by the user device.

Assignments (1)

Continuity (3)

References Cited (19)